LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Reinette Chatre <reinette.chatre@intel.com>
To: tglx@linutronix.de, fenghua.yu@intel.com, tony.luck@intel.com,
	vikas.shivappa@linux.intel.com
Cc: gavin.hindman@intel.com, jithu.joseph@intel.com,
	dave.hansen@intel.com, mingo@redhat.com, hpa@zytor.com,
	x86@kernel.org, linux-kernel@vger.kernel.org,
	Reinette Chatre <reinette.chatre@intel.com>
Subject: [PATCH V4 01/38] x86/intel_rdt: Document new mode, size, and bit_usage
Date: Tue, 22 May 2018 04:28:49 -0700	[thread overview]
Message-ID: <cc1e6234f80e07eef65529bd6c25db0a688bba12.1526987654.git.reinette.chatre@intel.com> (raw)
In-Reply-To: <cover.1526987654.git.reinette.chatre@intel.com>
In-Reply-To: <cover.1526987654.git.reinette.chatre@intel.com>

By default resource groups allow sharing of their cache allocations.
There is nothing that prevents a resource group from configuring
a cache allocation that overlaps with that of an existing resource
group.

To enable resource groups to specify that their cache allocations cannot
be shared a resource group "mode" is introduced to support two possible
modes: "shareable" and "exclusive". A "shareable" resource group allows
sharing of its cache allocations, an "exclusive" resource group does
not. A new resctrl file "mode" associated with each resource group is
used to communicate its (the associated resource group's) mode setting
and allow the mode to be changed.  The new "mode" file as well as two
other resctrl files, "bit_usage" and "size", are introduced in this
series.

Add documentation for the three new resctrl files as well as one example
demonstrating their use.

Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
---
 Documentation/x86/intel_rdt_ui.txt | 99 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 2 deletions(-)

diff --git a/Documentation/x86/intel_rdt_ui.txt b/Documentation/x86/intel_rdt_ui.txt
index a16aa2113840..de913e00e922 100644
--- a/Documentation/x86/intel_rdt_ui.txt
+++ b/Documentation/x86/intel_rdt_ui.txt
@@ -65,6 +65,27 @@ related to allocation:
 			some platforms support devices that have their
 			own settings for cache use which can over-ride
 			these bits.
+"bit_usage":		Annotated capacity bitmasks showing how all
+			instances of the resource are used. The legend is:
+			"0" - Corresponding region is unused. When the system's
+			      resources have been allocated and a "0" is found
+			      in "bit_usage" it is a sign that resources are
+			      wasted.
+			"H" - Corresponding region is used by hardware only
+			      but available for software use. If a resource
+			      has bits set in "shareable_bits" but not all
+			      of these bits appear in the resource groups'
+			      schematas then the bits appearing in
+			      "shareable_bits" but no resource group will
+			      be marked as "H".
+			"X" - Corresponding region is available for sharing and
+			      used by hardware and software. These are the
+			      bits that appear in "shareable_bits" as
+			      well as a resource group's allocation.
+			"S" - Corresponding region is used by software
+			      and available for sharing.
+			"E" - Corresponding region is used exclusively by
+			      one resource group. No sharing allowed.
 
 Memory bandwitdh(MB) subdirectory contains the following files
 with respect to allocation:
@@ -163,6 +184,16 @@ When control is enabled all CTRL_MON groups will also contain:
 	A list of all the resources available to this group.
 	Each resource has its own line and format - see below for details.
 
+"size":
+	Mirrors the display of the "schemata" file to display the size in
+	bytes of each allocation instead of the bits representing the
+	allocation.
+
+"mode":
+	The "mode" of the resource group dictates the sharing of its
+	allocations. A "shareable" resource group allows sharing of its
+	allocations while an "exclusive" resource group does not.
+
 When monitoring is enabled all MON groups will also contain:
 
 "mon_data":
@@ -502,7 +533,71 @@ siblings and only the real time threads are scheduled on the cores 4-7.
 
 # echo F0 > p0/cpus
 
-4) Locking between applications
+Example 4
+---------
+
+The resource groups in previous examples were all in the default "shareable"
+mode allowing sharing of their cache allocations. If one resource group
+configures a cache allocation then nothing prevents another resource group
+to overlap with that allocation.
+
+In this example a new exclusive resource group will be created on a L2 CAT
+system with two L2 cache instances that can be configured with an 8-bit
+capacity bitmask. The new exclusive resource group will be configured to use
+25% of each cache instance.
+
+# mount -t resctrl resctrl /sys/fs/resctrl/
+# cd /sys/fs/resctrl
+
+First, we observe that the default group is configured to allocate to all L2
+cache:
+
+# cat schemata
+L2:0=ff;1=ff
+
+We could attempt to create the new resource group at this point, but it will
+fail because of the overlap with the schemata of the default group:
+# mkdir p0
+# echo 'L2:0=0x3;1=0x3' > p0/schemata
+# cat p0/mode
+shareable
+# echo exclusive > p0/mode
+-sh: echo: write error: Invalid argument
+# cat info/last_cmd_status
+schemata overlaps
+
+To ensure that there is no overlap with another resource group the default
+resource group's schemata has to change, making it possible for the new
+resource group to become exclusive.
+# echo 'L2:0=0xfc;1=0xfc' > schemata
+# echo exclusive > p0/mode
+# grep . p0/*
+p0/cpus:0
+p0/mode:exclusive
+p0/schemata:L2:0=03;1=03
+p0/size:L2:0=262144;1=262144
+
+A new resource group will on creation not overlap with an exclusive resource
+group:
+# mkdir p1
+# grep . p1/*
+p1/cpus:0
+p1/mode:shareable
+p1/schemata:L2:0=fc;1=fc
+p1/size:L2:0=786432;1=786432
+
+The bit_usage will reflect how the cache is used:
+# cat info/L2/bit_usage
+0=SSSSSSEE;1=SSSSSSEE
+
+A resource group cannot be forced to overlap with an exclusive resource group:
+# echo 'L2:0=0x1;1=0x1' > p1/schemata
+-sh: echo: write error: Invalid argument
+# cat info/last_cmd_status
+overlaps with exclusive group
+
+Locking between applications
+----------------------------
 
 Certain operations on the resctrl filesystem, composed of read/writes
 to/from multiple files, must be atomic.
@@ -510,7 +605,7 @@ to/from multiple files, must be atomic.
 As an example, the allocation of an exclusive reservation of L3 cache
 involves:
 
-  1. Read the cbmmasks from each directory
+  1. Read the cbmmasks from each directory or the per-resource "bit_usage"
   2. Find a contiguous set of bits in the global CBM bitmask that is clear
      in any of the directory cbmmasks
   3. Create a new directory
-- 
2.13.6

  reply	other threads:[~2018-05-22 19:42 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-22 11:28 [PATCH V4 00/38] Intel(R) Resource Director Technology Cache Pseudo-Locking enabling Reinette Chatre
2018-05-22 11:28 ` Reinette Chatre [this message]
2018-05-22 11:28 ` [PATCH V4 02/38] x86/intel_rdt: Introduce RDT resource group mode Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 03/38] x86/intel_rdt: Associate mode with each RDT resource group Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 04/38] x86/intel_rdt: Introduce resource group's mode resctrl file Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 05/38] x86/intel_rdt: Introduce test to determine if closid is in use Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 06/38] x86/intel_rdt: Make useful functions available internally Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 07/38] x86/intel_rdt: Initialize new resource group with sane defaults Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 08/38] x86/intel_rdt: Introduce new "exclusive" mode Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 09/38] x86/intel_rdt: Enable setting of exclusive mode Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 10/38] x86/intel_rdt: Making CBM name and type more explicit Reinette Chatre
2018-05-22 11:28 ` [PATCH V4 11/38] x86/intel_rdt: Support flexible data to parsing callbacks Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 12/38] x86/intel_rdt: Ensure requested schemata respects mode Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 13/38] x86/intel_rdt: Introduce "bit_usage" to display cache allocations details Reinette Chatre
2018-05-22 21:03   ` Randy Dunlap
2018-05-22 21:09     ` Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 14/38] x86/intel_rdt: Display resource groups' allocations' size in bytes Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 15/38] x86/intel_rdt: Documentation for Cache Pseudo-Locking Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 16/38] x86/intel_rdt: Introduce the Cache Pseudo-Locking modes Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 17/38] x86/intel_rdt: Respect read and write access Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 18/38] x86/intel_rdt: Add utility to test if tasks assigned to resource group Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 19/38] x86/intel_rdt: Add utility to restrict/restore access to resctrl files Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 20/38] x86/intel_rdt: Protect against resource group changes during locking Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 21/38] x86/intel_rdt: Utilities to restrict/restore access to specific files Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 22/38] x86/intel_rdt: Add check to determine if monitoring in progress Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 23/38] x86/intel_rdt: Introduce pseudo-locked region Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 24/38] x86/intel_rdt: Support enter/exit of locksetup mode Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 25/38] x86/intel_rdt: Enable entering of pseudo-locksetup mode Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 26/38] x86/intel_rdt: Split resource group removal in two Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 27/38] x86/intel_rdt: Add utilities to test pseudo-locked region possibility Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 28/38] x86/intel_rdt: Discover supported platforms via prefetch disable bits Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 29/38] x86/intel_rdt: Pseudo-lock region creation/removal core Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 30/38] x86/intel_rdt: Support creation/removal of pseudo-locked region Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 31/38] x86/intel_rdt: resctrl files reflect pseudo-locked information Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 32/38] x86/intel_rdt: Ensure RDT cleanup on exit Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 33/38] x86/intel_rdt: Create resctrl debug area Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 34/38] x86/intel_rdt: Create debugfs files for pseudo-locking testing Reinette Chatre
2018-05-22 19:43   ` Greg KH
2018-05-22 21:02     ` Reinette Chatre
2018-05-23  8:05       ` Greg KH
2018-05-23 17:19         ` Reinette Chatre
2018-05-23 17:27           ` Greg KH
2018-05-22 11:29 ` [PATCH V4 35/38] x86/intel_rdt: Create character device exposing pseudo-locked region Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 36/38] x86/intel_rdt: More precise L2 hit/miss measurements Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 37/38] x86/intel_rdt: Support L3 cache performance event of Broadwell Reinette Chatre
2018-05-22 11:29 ` [PATCH V4 38/38] x86/intel_rdt: Limit C-states dynamically when pseudo-locking active Reinette Chatre

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cc1e6234f80e07eef65529bd6c25db0a688bba12.1526987654.git.reinette.chatre@intel.com \
    --to=reinette.chatre@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=fenghua.yu@intel.com \
    --cc=gavin.hindman@intel.com \
    --cc=hpa@zytor.com \
    --cc=jithu.joseph@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=tglx@linutronix.de \
    --cc=tony.luck@intel.com \
    --cc=vikas.shivappa@linux.intel.com \
    --cc=x86@kernel.org \
    --subject='Re: [PATCH V4 01/38] x86/intel_rdt: Document new mode, size, and bit_usage' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).