[PATCH v5 00/16] x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[PATCH v5 00/16] x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[Babu Moger]

Hi All,

This series adds support for AMD's Privilege-Level Zero Association
(PLZA) so kernel work can use a separate resource allocation and/or
monitoring association from the user task, and wires it up through a
small generic "kernel mode" (kmode) layer in fs/resctrl so future
architectures can plug in without touching(hopefully) core resctrl.

The features are documented in:

   AMD64 Zen6 Platform Quality of Service (PQOS) Extensions,
   Publication # 69193 Revision 1.00, Issue Date March 2026

available at https://bugzilla.kernel.org/show_bug.cgi?id=206537

The patches are based on top of commit (tip/master v7.2):

  e7bb1f1ccd17 Merge branch into tip/master: 'locking/urgent'

Any feedback is apprecated.

Background
==========

When memory bandwidth associated with a CLOSID is aggressively throttled,
and a task with that CLOSID moves into kernel mode, the kernel operations
are also aggressively throttled. This can stall forward progress and
eventually degrade overall system performance.
    
Privilege-Level Zero Association (PLZA) allows the user to specify a CLOSID
and/or RMID for execution at Privilege Level Zero. When PLZA is enabled on
a CPU, kernel work at PL0 uses the CLOSID and/or RMID from MSR
PQR_PLZA_ASSOC; otherwise, the CPU uses the CLOSID and RMID from PQR_ASSOC.

Design
======

A new sysfs file, info/kernel_mode, holds the global policy for resource
allocation and monitoring of kernel work and the resource group (when
applicable) associated with the policy. Reads list the supported modes
and the currently active association; writes change the policy or
associate a different group. Look at the thread below for design
discussion.
https://lore.kernel.org/lkml/14a8ad0a-e842-4268-871a-0762f1169e03@intel.com/
https://lore.kernel.org/lkml/e4c95002-ae8a-48d0-bedd-772db58b937e@intel.com/

Two kernel modes are exposed:

- inherit_user: kernel work inherits allocation and monitoring from the
  user task.
- assign_global_enable_per_cpu: kernel work may use separate allocation
  and/or monitoring associations. ctrl= and mon= select whether each
  dimension is assigned or inherited, and group= identifies the
  associated resource group using <CTRL_MON>/<MON>/ path syntax.

Per-rdtgroup files kmode_cpus and kmode_cpus_list scope the association
to a subset of online CPUs without deactivating and re-associating.
They are visible only on the group currently associated through
info/kernel_mode.

Groups associated through kernel mode cannot enter pseudo-lock setup,
change mode, or be renamed until the association is cleared.

The arch hook, resctrl_arch_configure_kmode(), keeps the fs/resctrl layer
arch-neutral.

resctrl_set_kmode_support() lets architecture code register supported
kernel-mode policies during resctrl initialization. On AMD, PLZA
registers assign_global_enable_per_cpu when the feature is available.

Only AMD PLZA is wired up here; Intel and ARM can add their own support
later by implementing the hooks.

Examples
========

(See Documentation/filesystems/resctrl.rst, "kernel_mode", "kmode_cpus",
and "Examples on working with kernel_mode", for the full UAPI.)

  # Mount resctrl
  # mount -t resctrl resctrl /sys/fs/resctrl
  # cd /sys/fs/resctrl

  # Read the supported modes.  The active mode is bracketed for display
  # only; do not include brackets when writing.
  # cat info/kernel_mode
  [inherit_user]
  assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=//

  # Create a CTRL_MON group and associate kernel-mode allocation with it.
  # mkdir ctrl1
  # echo "assign_global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//" \
          > info/kernel_mode
  # cat info/kernel_mode
  inherit_user
  [assign_global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//]

  # kmode_cpus and kmode_cpus_list are visible only on the associated group.
  # ls ctrl1/kmode_cpus*
  ctrl1/kmode_cpus  ctrl1/kmode_cpus_list

  # Restrict the association to a CPU subset; the write is incremental.
  # echo 0-3 > ctrl1/kmode_cpus_list
  # cat ctrl1/kmode_cpus
  f
  # cat ctrl1/kmode_cpus_list
  0-3

  # Return to the default inherit policy.
  # echo "inherit_user" > info/kernel_mode
  # cat info/kernel_mode
  [inherit_user]
  assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=//

Tested on AMD with PLZA; builds on x86 without PLZA and does not expose
assign_global_enable_per_cpu unless the feature is available.

Layout
======

  01-03  x86: PLZA CPU feature, command-line option, and MSR/data-structure
         plumbing.
  04-07  fs/resctrl + x86: kernel mode policy enum, arch hooks, kmode state,
         and resctrl_set_kmode_support().
  08     x86/resctrl: register assign_global_enable_per_cpu when PLZA is
         available.
  09     fs/resctrl: info/kernel_mode read-only introspection.
  10     fs/resctrl: hidden rdtgroup files for kmode_cpus visibility control.
  11     fs/resctrl: per-rdtgroup kmode_cpus/kmode_cpus_list read interface.
  12     fs/resctrl: program kernel-mode association when a CPU comes online.
  13     fs/resctrl: deactivate the association when a group is removed.
  14     fs/resctrl: info/kernel_mode write interface.
  15     fs/resctrl: incremental kmode_cpus/kmode_cpus_list writes.
  16     fs/resctrl: documentation and end-to-end examples.


Changelog
=========
v5:
  - Collapse the two v4 global-assign modes into a single
    assign_global_enable_per_cpu policy with ctrl= and mon= options.
  - Use "association" terminology consistently in code, errors, and
    documentation.
  - Register assign_global_enable_per_cpu once during x86 resource
    discovery when PLZA is available.
  - Reject mon=inherit when group= selects a monitor group.
  - Split hidden-file support, deactivation-on-teardown, and PLZA mode
    registration into separate patches for easier review.
  - Fix msr_pqr_plza_assoc truncation on 32-bit systems by using u64.
  - Refresh documentation and add a readable end-to-end example walkthrough.
  - Block pseudo-lock setup on kernel-mode-associated groups.
  - Reject mode changes and rename while a group backs the active
    association.

v4:
  - Reorder and split the series into 15 patches: separate read-only
    info/kernel_mode display from the write path; add hotplug support
    when a CPU comes online; add an end-to-end documentation/examples
    patch.
  - Introduced resctrl_set_kmode_support() so architecture code can
    register supported kernel-mode policies during resctrl initialization.
  - info/kernel_mode write: validate group type; run fail paths before
    tearing down the active association so errors retain the old state.
  - kmode_cpus / kmode_cpus_list: writable with incremental enable/disable
    deltas; empty masks allowed; offline CPUs rejected.
  - Hotplug: newly online CPUs are added to the associated group's
    kmode_cpu_mask and programmed when assign_global_enable_per_cpu is
    active.

v3:
  - Generalise the layer beyond AMD: rename "PLZA mode" to "kernel mode"
    (kmode) in code, sysfs, and Documentation.
  - Reset the association when the associated rdtgroup is removed,
    instead of leaving stale state.

v2:
  - Similar to RFC with a new proposal; interface names were not final.
  - Separated Global Bandwidth Enforcement (GLBE) from PLZA; this series
    only adds PLZA support.
  - Used "kmode" instead of "PLZA" in the generic layer.

Previous versions:
v4: https://lore.kernel.org/lkml/cover.1783461016.git.babu.moger@amd.com/
v3: https://lore.kernel.org/lkml/cover.1777591496.git.babu.moger@amd.com/
v2: https://lore.kernel.org/lkml/cover.1773347820.git.babu.moger@amd.com/
v1: https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/


Babu Moger (16):
  x86/cpufeatures: Support Privilege Level Zero Association (PLZA)
  x86/resctrl: Add PLZA support to command-line options
  x86/resctrl: Add PLZA configuration definitions and data structures
  fs/resctrl: Introduce kernel mode policy enum
  x86,fs/resctrl: Introduce architecture hooks to program kernel mode
  fs/resctrl: Introduce kernel mode states for resctrl
  fs/resctrl: Introduce resctrl_set_kmode_support() to register
    supported modes
  x86/resctrl: Expose assign_global_enable_per_cpu when PLZA is
    available
  fs/resctrl: Add interface to display supported and active kernel modes
  fs/resctrl: Add support for hidden resource group files
  fs/resctrl: Introduce kmode_cpus/kmode_cpus_list per rdtgroup
  fs/resctrl: Program kernel mode assignments on CPU hotplug
  fs/resctrl: Deactivate the kernel mode association when a group is
    removed
  fs/resctrl: Add interface to modify kernel mode via info/kernel_mode
  fs/resctrl: Allow user space to write kmode_cpus/kmode_cpus_list
  fs/resctrl: Add documentation on kernel_mode with example

 .../admin-guide/kernel-parameters.txt         |   2 +-
 Documentation/filesystems/resctrl.rst         | 193 +++++
 arch/x86/include/asm/cpufeatures.h            |   1 +
 arch/x86/include/asm/msr-index.h              |   1 +
 arch/x86/kernel/cpu/resctrl/core.c            |   6 +
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c     |  38 +
 arch/x86/kernel/cpu/resctrl/internal.h        |  47 ++
 arch/x86/kernel/cpu/scattered.c               |   1 +
 drivers/resctrl/mpam_resctrl.c                |   6 +
 fs/resctrl/internal.h                         |  60 ++
 fs/resctrl/pseudo_lock.c                      |   5 +
 fs/resctrl/rdtgroup.c                         | 795 ++++++++++++++++++
 include/linux/resctrl.h                       |  75 ++
 13 files changed, 1229 insertions(+), 1 deletion(-)

-- 
2.43.0

[PATCH v5 01/16] x86/cpufeatures: Support Privilege Level Zero Association (PLZA)

[Babu Moger]

When memory bandwidth associated with a CLOSID is aggressively throttled,
and a task with that CLOSID moves into kernel mode, the kernel operations
are also aggressively throttled. This can stall forward progress and
eventually degrade overall system performance.

AMD hardware supports a feature Privilege Level Zero Association (PLZA),
which allows the CPU's CLOSID association to be changed during the
transition from user mode to kernel mode. This allows the kernel to run
using a different CLOSID than user space, which can improve system
performance.

PLZA also applies to cache allocation and monitoring features and provides
flexibility to manage allocation and monitoring associations independently.

The feature is detected via CPUID_Fn80000020_EBX_x00 [Bit 9]: Privilege
Level Zero Association (PLZA).

The PLZA feature details are documented in [1] available from [2].

[1] AMD64 Zen6 Platform Quality of Service (PQOS) Extensions:
    Publication # 69193 Revision: 1.00, Issue Date: March 2026

Signed-off-by: Babu Moger <babu.moger@amd.com>
Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 # [2]
---
v5: Added Acked-by from Boris.
    Updated the changelog to include cache allocation and monitoring.
    Moved the Link tag last.
    Updated subject line for cpufeatures.

v4: Split the patch into 2. This patch only handles x86 changes.
    Re-wrote the changelog along the ABMC changes.

v3: Code did not change. Patch order changed.
    Added documentation link.

v2: Rebased on top of the latest tip.
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 arch/x86/kernel/cpu/scattered.c    | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 73d5c740202d..cd7baa654d85 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -519,6 +519,7 @@
 						      * and purposes if CLEAR_CPU_BUF_VM is set).
 						      */
 #define X86_FEATURE_X2AVIC_EXT		(21*32+20) /* AMD SVM x2AVIC support for 4k vCPUs */
+#define X86_FEATURE_PLZA		(21*32+21) /* Privilege Level Zero Association */
 
 /*
  * BUG word(s)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 8665a6474806..0cb91f24b6cb 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -61,6 +61,7 @@ static const struct cpuid_bit cpuid_bits[] = {
 	{ X86_FEATURE_BMEC,			CPUID_EBX,  3, 0x80000020, 0 },
 	{ X86_FEATURE_ABMC,			CPUID_EBX,  5, 0x80000020, 0 },
 	{ X86_FEATURE_SDCIAE,			CPUID_EBX,  6, 0x80000020, 0 },
+	{ X86_FEATURE_PLZA,			CPUID_EBX,  9, 0x80000020, 0 },
 	{ X86_FEATURE_AMD_WORKLOAD_CLASS,	CPUID_EAX, 22, 0x80000021, 0 },
 	{ X86_FEATURE_TSA_SQ_NO,		CPUID_ECX,  1, 0x80000021, 0 },
 	{ X86_FEATURE_TSA_L1_NO,		CPUID_ECX,  2, 0x80000021, 0 },
-- 
2.43.0

[PATCH v5 02/16] x86/resctrl: Add PLZA support to command-line options

[Babu Moger]

Add a kernel command-line parameter to enable or disable the exposure of
the PLZA (Privilege Level Zero Association) feature to resctrl.

This allows administrators to control PLZA visibility at boot time via the
rdt= option (e.g. rdt=plza or rdt=!plza).

Resctrl exposes PLZA by default without requiring any explicit rdt= option
if supported. The rdt=!plza parameter can be used to disable PLZA exposure
to resctrl, while rdt=plza explicitly enables it.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Updated the changelog to mention about rdt= options.

v4: Split the patch 1 from v3 into 2 patches.
    This patch contains changes for resctrl subsystem.
---
 Documentation/admin-guide/kernel-parameters.txt | 2 +-
 Documentation/filesystems/resctrl.rst           | 1 +
 arch/x86/kernel/cpu/resctrl/core.c              | 2 ++
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 66c1c879a0b2..9772ce838966 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6455,7 +6455,7 @@ Kernel parameters
 	rdt=		[HW,X86,RDT]
 			Turn on/off individual RDT features. List is:
 			cmt, mbmtotal, mbmlocal, l3cat, l3cdp, l2cat, l2cdp,
-			mba, smba, bmec, abmc, sdciae, energy[:guid],
+			mba, smba, bmec, abmc, sdciae, plza, energy[:guid],
 			perf[:guid].
 			E.g. to turn on cmt and turn off mba use:
 				rdt=cmt,!mba
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index e4b66af55ffb..f3e941404967 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -28,6 +28,7 @@ SMBA (Slow Memory Bandwidth Allocation)				""
 BMEC (Bandwidth Monitoring Event Configuration)			""
 ABMC (Assignable Bandwidth Monitoring Counters)			""
 SDCIAE (Smart Data Cache Injection Allocation Enforcement)	""
+PLZA (Privilege Level Zero Association)				""
 =============================================================== ================================
 
 Historically, new features were made visible by default in /proc/cpuinfo. This
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 55214d6fdc49..4ca8bedeb995 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -803,6 +803,7 @@ enum {
 	RDT_FLAG_BMEC,
 	RDT_FLAG_ABMC,
 	RDT_FLAG_SDCIAE,
+	RDT_FLAG_PLZA,
 };
 
 #define RDT_OPT(idx, n, f)	\
@@ -830,6 +831,7 @@ static struct rdt_options rdt_options[]  __ro_after_init = {
 	RDT_OPT(RDT_FLAG_BMEC,	    "bmec",	X86_FEATURE_BMEC),
 	RDT_OPT(RDT_FLAG_ABMC,	    "abmc",	X86_FEATURE_ABMC),
 	RDT_OPT(RDT_FLAG_SDCIAE,    "sdciae",	X86_FEATURE_SDCIAE),
+	RDT_OPT(RDT_FLAG_PLZA,	    "plza",	X86_FEATURE_PLZA),
 };
 #define NUM_RDT_OPTIONS ARRAY_SIZE(rdt_options)
 
-- 
2.43.0

[PATCH v5 03/16] x86/resctrl: Add PLZA configuration definitions and data structures

[Babu Moger]

Privilege Level Zero Association (PLZA) enables the kernel to switch to
different CLOSID and/or RMID than those used in user mode when entering
kernel mode. CLOSID and RMID selection are independent and can be
configured separately.

PLZA associations are programmed on a per-CPU basis through
MSR_IA32_PQR_PLZA_ASSOC (0xc00003fc), which contains the kernel RMID and
CLOSID values along with their corresponding enable bits.

Add the MSR definition and the msr_pqr_plza_assoc union to describe the
register bitfield layout required for PLZA configuration and subsequent
programming support.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Sashiko reported:
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=3
    Fix the union truncate issue on 32 bit systems by using u64.
    Updated the changelog and code comments.

v4: Re-wrote the changelog and code comment.

v3: No code changes. Patch order changed. Improved changelog.

v2: No changes. Just rebasing on top of the latest tip branch.
---
 arch/x86/include/asm/msr-index.h       |  1 +
 arch/x86/kernel/cpu/resctrl/internal.h | 47 ++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 3a8e51a0c9e8..849e8587631c 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -1294,6 +1294,7 @@
 /* - AMD: */
 #define MSR_IA32_MBA_BW_BASE		0xc0000200
 #define MSR_IA32_SMBA_BW_BASE		0xc0000280
+#define MSR_IA32_PQR_PLZA_ASSOC		0xc00003fc
 #define MSR_IA32_L3_QOS_ABMC_CFG	0xc00003fd
 #define MSR_IA32_L3_QOS_EXT_CFG		0xc00003ff
 #define MSR_IA32_EVT_CFG_BASE		0xc0000400
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..2f4c60487a5d 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -222,6 +222,53 @@ union l3_qos_abmc_cfg {
 	unsigned long full;
 };
 
+/*
+ * Privilege Level Zero Association (PLZA) is configured via
+ * MSR_IA32_PQR_PLZA_ASSOC. This MSR must be programmed on every CPU in
+ * a QoS domain. Each domain supports a single PLZA CLOSID and/or RMID
+ * association, and all fields except PLZA_EN must be programmed
+ * identically across the domain to ensure consistent behavior.
+ *
+ * RMID and CLOSID selection are controlled independently. When PLZA is
+ * enabled (plza_en = 1), CPL0 uses CLOSID and RMID values from
+ * MSR_IA32_PQR_PLZA_ASSOC depending on the corresponding enable bits.
+ * When PLZA is disabled (plza_en = 0), the CLOSID and RMID are always
+ * taken from MSR_IA32_PQR_ASSOC regardless of privilege level.
+ *
+ *   - rmid_en = 1: use the RMID programmed in this MSR.
+ *   - rmid_en = 0: use the RMID from MSR_IA32_PQR_ASSOC.
+ *   - closid_en = 1: use the CLOSID programmed in this MSR.
+ *   - closid_en = 0: use the CLOSID from MSR_IA32_PQR_ASSOC.
+ *
+ * This allows PLZA to override only RMID, only CLOSID, or both. If both
+ * rmid_en and closid_en are clear, the effective behavior is the same as
+ * when PLZA is disabled, with RMID and CLOSID derived from
+ * MSR_IA32_PQR_ASSOC irrespective of privilege level.
+ *
+ * @rmid		: RMID programmed for PLZA.
+ * @reserved1		: Reserved.
+ * @rmid_en		: Enable RMID for PLZA.
+ * @closid		: CLOSID programmed for PLZA.
+ * @reserved2		: Reserved.
+ * @closid_en		: Enable CLOSID for PLZA.
+ * @reserved3		: Reserved.
+ * @plza_en		: Enable PLZA. When enabled, PLZA applies to the
+ *			  given CPU.
+ */
+union msr_pqr_plza_assoc {
+	struct {
+		u64	rmid		:12,
+			reserved1	:19,
+			rmid_en		: 1,
+			closid		: 4,
+			reserved2	:11,
+			closid_en	: 1,
+			reserved3	:15,
+			plza_en		: 1;
+	} split;
+	u64	full;
+};
+
 void rdt_ctrl_update(void *arg);
 
 int rdt_get_l3_mon_config(struct rdt_resource *r);
-- 
2.43.0

[PATCH v5 04/16] fs/resctrl: Introduce kernel mode policy enum

[Babu Moger]

Kernel mode traffic can use different allocation and monitoring
associations than the originating user task. On x86, Privilege Level Zero
Association (PLZA) enables the kernel to switch to a different CLOSID/RMID
when entering kernel mode.

Architectures need a common way to name kernel modes before resctrl
can report what is active or what the platform supports.

Introduce enum resctrl_kernel_mode:
  - RESCTRL_INHERIT_USER: Kernel mode inherits allocation and monitoring
    from the user task.
  - RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU: Kernel mode may use allocation
    and/or monitoring associations that differ from the user task. On x86,
    CLOSID and RMID can be configured independently for kernel mode.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Collapse the two global-assign modes into a single
    RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU value. Update the changelog to
    match the enum names and descriptions in the code.

v4: Updated the changelog to be generic as possible.
    Moved the enum resctrl_kernel_mode to include/linux/resctrl.h.
    Updated the code comments to be generic.
    Removed resctrl_kmode_cfg from the code. This definition can be fs
    specific only and architectures dont need to know.
    Changed enum name to resctrl_kernel_mode from resctrl_kernel_modes.

v3: Removed resctrl_kmode definition.
    Changed the kernel mode definitions to enum resctrl_kernel_modes.
    Used BIT() to set/test the features.
    Added details to changelog.

v2: New patch to handle PLZA interfaces with /sys/fs/resctrl/info/ directory.
    https://lore.kernel.org/lkml/2ab556af-095b-422b-9396-f845c6fd0342@intel.com/
---
 include/linux/resctrl.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index dd09c2ce9a0f..4245d1e65ccc 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -704,6 +704,31 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable);
  */
 bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r);
 
+/**
+ * enum resctrl_kernel_mode - Kernel mode control and monitoring association.
+ *
+ * @RESCTRL_INHERIT_USER:
+ *     Kernel mode inherits both allocation and monitoring associations
+ *     from the user space task. On x86, kernel uses the same CLOSID and
+ *     RMID as the user-space task.
+ *
+ * @RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU:
+ *     Kernel mode may use allocation and monitoring associations that
+ *     differ from the user task. On x86, CLOSID and RMID are configured
+ *     independently; either or both may differ from user mode.
+ *
+ *     All online CPUs are included by default. A subset may be selected
+ *     through the resctrl group interface, and a CTRL_MON or MON group
+ *     may be associated with this mode.
+ */
+enum resctrl_kernel_mode {
+	RESCTRL_INHERIT_USER,
+	RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU,
+	RESCTRL_KMODE_LAST = RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU
+};
+
+#define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-- 
2.43.0

[PATCH v5 05/16] x86,fs/resctrl: Introduce architecture hooks to program kernel mode

[Babu Moger]

Kernel modes defined by enum resctrl_kernel_mode must be applied on
the CPUs when user space activates, deactivates, or updates a
configuration.

Generic resctrl has no architecture hook to apply these modes across
a CPU mask when the active mode changes.

Add resctrl_arch_configure_kmode() to program kernel mode allocation and
monitoring associations on @cpu_mask. Accept separate assign_ctrl and
assign_mon parameters so CLOSID and RMID can be assigned independently.

Implement the x86 hook to program per-CPU PLZA settings. On x86, PLZA
programs these associations per CPU, with CLOSID and RMID configured
independently.

Provide an MPAM stub so the filesystem layer can call the hook on systems
without PLZA.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Fix kernel-doc for resctrl_arch_configure_kmode(), wire assign_ctrl
    through to PLZA closid_en, and update the changelog.
    Add calling context in API doc.

v4: Added assign_mon parameter in resctrl_arch_configure_kmode() to program the RMID
    as discussed in below.
    https://lore.kernel.org/lkml/20260605100642.1103628-1-qinyuntan@linux.alibaba.com/
    Reintroduce independent monitoring assignment via assign_mon after removing
    the task-based PLZA approach in v3.
    Changed cpumask type to "const struct cpumask *cpu_mask".
    Added MPAM stub to avoid any linking issues when resctrl_arch_configure_kmode()
    is called from FS layer. Thanks to Qinyun.
    Re-wrote the changelog to be generic.
    Updated code comments.
v3: Removed task based PLZA implementation so related changes are removed.
    Per-task RMID handling is replaced by group type selection in the fs layer.
    Updated the change log with details.
    Removed resctrl_arch_set_kmode() as arch only provides the modes supported.
    It is FS which decided which mode to apply.
v2: Updated the commit message to include the sequence of steps to enable PLZA.
    Added mode code comments for clarity.
    Added kmode to function names to be generic.
---
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 38 +++++++++++++++++++++++
 drivers/resctrl/mpam_resctrl.c            |  6 ++++
 include/linux/resctrl.h                   | 33 ++++++++++++++++++++
 3 files changed, 77 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index e74f1ed54b86..40fd5e31c94e 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -131,3 +131,41 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable)
 
 	return 0;
 }
+
+static void resctrl_kmode_set_one_amd(void *arg)
+{
+	union msr_pqr_plza_assoc *plza = arg;
+
+	wrmsrq(MSR_IA32_PQR_PLZA_ASSOC, plza->full);
+}
+
+/*
+ * Program Privilege Level Zero Association (PLZA) on @cpu_mask.
+ *
+ * When @enable is true, kernel mode allocation on @cpu_mask uses @closid from
+ * MSR_IA32_PQR_PLZA_ASSOC if @assign_ctrl is true, otherwise the CLOSID from
+ * MSR_IA32_PQR_ASSOC. Kernel mode monitoring uses @rmid from
+ * MSR_IA32_PQR_PLZA_ASSOC if @assign_mon is true, otherwise the RMID of the
+ * current task.
+ *
+ * @cpu_mask:	CPUs whose PLZA MSR should be updated.
+ * @closid:	CLOSID to use for kernel mode allocation when @assign_ctrl is true.
+ * @assign_ctrl: Whether PLZA should provide the kernel mode CLOSID.
+ * @rmid:	RMID to use for kernel mode monitoring when @assign_mon is true.
+ * @assign_mon: Whether PLZA should provide the kernel mode RMID.
+ * @enable:	Whether PLZA should provide the kernel mode association.
+ */
+void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid,
+				  bool assign_ctrl, u32 rmid,
+				  bool assign_mon, bool enable)
+{
+	union msr_pqr_plza_assoc plza = { 0 };
+
+	plza.split.rmid = rmid;
+	plza.split.rmid_en = assign_mon;
+	plza.split.closid = closid;
+	plza.split.closid_en = assign_ctrl;
+	plza.split.plza_en = enable;
+
+	on_each_cpu_mask(cpu_mask, resctrl_kmode_set_one_amd, &plza, 1);
+}
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 9d223057953a..286284ac8423 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -139,6 +139,12 @@ bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r)
 	return false;
 }
 
+void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid,
+				  bool assign_ctrl, u32 rmid, bool assign_mon,
+				  bool enable)
+{
+}
+
 void resctrl_arch_pre_mount(void)
 {
 }
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 4245d1e65ccc..8b30eef835aa 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -729,6 +729,39 @@ enum resctrl_kernel_mode {
 
 #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
 
+/**
+ * resctrl_arch_configure_kmode() - Program kernel mode association
+ * @cpu_mask:	CPUs to assign the kernel mode on.
+ * @closid:	CLOSID that matches the RMID to program kernel mode. Depending
+ *		on the architecture, the counter may match traffic of both
+ *		@closid and @rmid, or @rmid only.
+ * @assign_ctrl: true to assign @closid for kernel mode; false to inherit
+ *		association from the user-space task.
+ * @rmid:	RMID to program the kernel mode. Some architectures may use
+ *		CLOSID/RMID separately, others will consider them together.
+ * @assign_mon:	true to assign @rmid for kernel mode; false to inherit
+ *		monitoring association from the user-space task.
+ * @enable:	true to enable kernel mode association on CPUs in @cpu_mask;
+ *		false to disable kernel mode.
+ *
+ * The function can be called in the following scenarios:
+ * - If a per-cpu kernel mode is active when user space switches to a new
+ *   per-cpu kernel mode then resctrl_arch_configure_kmode() will first be
+ *   called to de-activate the active kernel mode on all CPUs that the
+ *   kernel mode is active on.
+ * - When user space switches to a new per-cpu kernel mode then
+ *   resctrl_arch_configure_kmode() is called with cpu_online_mask.
+ * - When user space adds a CPU to an active per-cpu kernel mode.
+ * - When user space removes a CPU from an active per-cpu kernel mode.
+ * - resctrl fs will always provide the same closid, assign_ctrl, rmid,
+ *   and assign_mon parameters when activating a kernel mode, all
+ *   interactions (adding/removing CPU) while the kernel mode is active,
+ *   as well as when de-activating the kernel mode.
+ */
+void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid,
+				  bool assign_ctrl, u32 rmid, bool assign_mon,
+				  bool enable);
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-- 
2.43.0

[PATCH v5 06/16] fs/resctrl: Introduce kernel mode states for resctrl

[Babu Moger]

Architectures need the generic resctrl filesystem to track supported
kernel modes, the active mode, and per-dimension assignment state.

Generic resctrl has no place to store this policy state today.

Add struct resctrl_kmode_cfg to hold supported modes, active mode,
control/monitoring assignment state, and the bound resource group.
Initialize defaults from resctrl_arch_alloc_capable() and
resctrl_arch_mon_capable() during resctrl_init().

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Replace per-mode registration helpers with a single resctrl_kmode_cfg
    structure. Derive ctrl_en and mon_en from resctrl_arch_alloc_capable()
    and resctrl_arch_mon_capable() during filesystem init. Update code
    comments and changelog.

v4: New patch to initialize supported kernel modes.
    https://lore.kernel.org/lkml/737a4228-52fb-4583-ac64-8efe79c107e6@intel.com/
    Changed the kmode_cur to enum resctrl_kernel_mode in resctrl_kmode_cfg.
    Moved the resctrl_kmode_cfg to filesystem code.
---
 fs/resctrl/internal.h | 51 +++++++++++++++++++++++++++++++++++++++++++
 fs/resctrl/rdtgroup.c | 33 ++++++++++++++++++++++++++++
 2 files changed, 84 insertions(+)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index e62a277dee85..4087bf44a06d 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -314,6 +314,57 @@ struct mbm_state {
 	u32	prev_bw;
 };
 
+/**
+ * enum kmode_state - Control or monitoring state for a kernel mode
+ * @KMODE_INHERIT: Inherit from the user space task.
+ * @KMODE_ASSIGN: Use a global assignment for kernel mode.
+ */
+enum kmode_state {
+	KMODE_INHERIT,
+	KMODE_ASSIGN
+};
+
+/**
+ * struct resctrl_kmode_caps - Static kernel mode capabilities
+ * @kmode_sup:	Bitmap of supported kernel modes. Empty when neither
+ *		@ctrl_en nor @mon_en is set and kernel mode policy is
+ *		unavailable on this system.
+ * @ctrl_en:	Whether kernel mode may use global assignment for control.
+ * @mon_en:	Whether kernel mode may use global assignment for monitoring.
+ */
+struct resctrl_kmode_caps {
+	DECLARE_BITMAP(kmode_sup, RESCTRL_NUM_KERNEL_MODES);
+	bool				ctrl_en;
+	bool				mon_en;
+};
+
+/**
+ * struct resctrl_kmode_active - Runtime kernel mode state
+ * @kmode_cur:	Currently selected kernel mode.
+ * @ctrl_mode:	Control assignment state when kernel mode is active.
+ * @mon_mode:	Monitoring assignment state when kernel mode is active.
+ * @k_rdtgrp:	Resource group backing global assignment mode.
+ *
+ * When @kmode_cur is %RESCTRL_INHERIT_USER, assignment state is ignored and
+ * @k_rdtgrp is %NULL.
+ */
+struct resctrl_kmode_active {
+	enum resctrl_kernel_mode	kmode_cur;
+	enum kmode_state		ctrl_mode;
+	enum kmode_state		mon_mode;
+	struct rdtgroup			*k_rdtgrp;
+};
+
+/**
+ * struct resctrl_kmode_cfg - Global kernel mode state
+ * @caps:	Supported modes and assignment capabilities.
+ * @active:	Active mode, assignment state, and assigned group.
+ */
+struct resctrl_kmode_cfg {
+	struct resctrl_kmode_caps	caps;
+	struct resctrl_kmode_active	active;
+};
+
 extern struct mutex rdtgroup_mutex;
 
 static inline const char *rdt_kn_name(const struct kernfs_node *kn)
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 5dcbb0a964e8..3c53f3f74e5a 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -76,6 +76,13 @@ static void rdtgroup_destroy_root(void);
 
 struct dentry *debugfs_resctrl;
 
+/*
+ * Global kernel mode policy state: supported modes, active mode, assignment
+ * capabilities, assignment state, and the resource group selected for a global
+ * assignment.
+ */
+static struct resctrl_kmode_cfg resctrl_kcfg;
+
 /*
  * Memory bandwidth monitoring event to use for the default CTRL_MON group
  * and each new CTRL_MON group created by the user.  Only relevant when
@@ -2297,6 +2304,30 @@ static void io_alloc_init(void)
 	}
 }
 
+/*
+ * Initialize kernel mode policy defaults from architecture capabilities.
+ *
+ * When ctrl_en or mon_en is set, RESCTRL_INHERIT_USER is supported and
+ * selected as the initial active mode. When neither is set, kmode_sup is
+ * left empty, kernel mode policy is unavailable, and kmode_cur remains at
+ * its zero-initialized default (RESCTRL_INHERIT_USER) but is unused.
+ */
+static void resctrl_kmode_init(void)
+{
+	resctrl_kcfg.caps.ctrl_en = resctrl_arch_alloc_capable();
+	resctrl_kcfg.active.ctrl_mode = KMODE_INHERIT;
+	resctrl_kcfg.caps.mon_en = resctrl_arch_mon_capable();
+	resctrl_kcfg.active.mon_mode = KMODE_INHERIT;
+	resctrl_kcfg.active.k_rdtgrp = NULL;
+
+	if (resctrl_kcfg.caps.ctrl_en || resctrl_kcfg.caps.mon_en) {
+		resctrl_kcfg.active.kmode_cur = RESCTRL_INHERIT_USER;
+		__set_bit(RESCTRL_INHERIT_USER, resctrl_kcfg.caps.kmode_sup);
+	} else {
+		bitmap_zero(resctrl_kcfg.caps.kmode_sup, RESCTRL_NUM_KERNEL_MODES);
+	}
+}
+
 void resctrl_file_fflags_init(const char *config, unsigned long fflags)
 {
 	struct rftype *rft;
@@ -4816,6 +4847,8 @@ int resctrl_init(void)
 	if (ret)
 		return ret;
 
+	resctrl_kmode_init();
+
 	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
 	if (ret) {
 		resctrl_l3_mon_resource_exit();
-- 
2.43.0

[PATCH v5 07/16] fs/resctrl: Introduce resctrl_set_kmode_support() to register supported modes

[Babu Moger]

Architectures need to advertise supported kernel modes before resctrl
can expose them through the filesystem.

Generic resctrl has no registration hook for architecture-specific
kernel mode policies.

Introduce resctrl_set_kmode_support() to set the corresponding bit in
resctrl_kcfg.caps.kmode_sup during initialization. Architectures may
call this to register RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU when they
support independent kernel mode control and/or monitoring assignment.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: New patch split from resctrl_kmode_cfg initialization. Register modes
    in resctrl_kmode_cfg.caps.kmode_sup using RESCTRL_INHERIT_USER and
    RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU. Ignore out-of-range mode values.

v4: New patch to initialize supported kernel modes.
    https://lore.kernel.org/lkml/737a4228-52fb-4583-ac64-8efe79c107e6@intel.com/
---
 fs/resctrl/rdtgroup.c   | 13 +++++++++++++
 include/linux/resctrl.h | 17 +++++++++++++++++
 2 files changed, 30 insertions(+)

diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 3c53f3f74e5a..3bb0e3a203ac 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -892,6 +892,19 @@ static int rdtgroup_rmid_show(struct kernfs_open_file *of,
 	return ret;
 }
 
+/**
+ * resctrl_set_kmode_support() - Register a supported kernel mode
+ * @kmode:	Kernel mode policy supported by the architecture.
+ *
+ * Set the corresponding bit in resctrl_kcfg.caps.kmode_sup so @kmode
+ * is visible to the resctrl file system.
+ */
+void resctrl_set_kmode_support(enum resctrl_kernel_mode kmode)
+{
+	if (kmode < RESCTRL_NUM_KERNEL_MODES)
+		__set_bit(kmode, resctrl_kcfg.caps.kmode_sup);
+}
+
 #ifdef CONFIG_PROC_CPU_RESCTRL
 /*
  * A task can only be part of one resctrl control group and of one monitor
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 8b30eef835aa..051aece77eb9 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -729,6 +729,23 @@ enum resctrl_kernel_mode {
 
 #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
 
+/**
+ * resctrl_set_kmode_support() - Register a supported kernel mode
+ * @kmode: Kernel mode supported by the architecture.
+ *
+ * Set the corresponding bit in resctrl_kcfg.caps.kmode_sup so @kmode
+ * can be exposed through the resctrl filesystem.
+ * RESCTRL_INHERIT_USER is registered by resctrl_kmode_init() and does
+ * not need to be registered again.
+ *
+ * Architectures use this interface during resctrl initialization to
+ * advertise supported kernel modes. Registration of a mode indicates
+ * architectural support only. The final decision to expose and enable
+ * a mode is made by resctrl based on system capabilities and
+ * configuration.
+ */
+void resctrl_set_kmode_support(enum resctrl_kernel_mode kmode);
+
 /**
  * resctrl_arch_configure_kmode() - Program kernel mode association
  * @cpu_mask:	CPUs to assign the kernel mode on.
-- 
2.43.0

[PATCH v5 08/16] x86/resctrl: Expose assign_global_enable_per_cpu when PLZA is available

[Babu Moger]

Generic resctrl exposes kernel modes through sysfs only after architectures
register them with resctrl_set_kmode_support().

PLZA provides per-CPU kernel mode allocation and monitoring assignment, but
assign_global_enable_per_cpu is never registered during x86 initialization.

Register RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU when PLZA is available and
resctrl allocation or monitoring resources are present.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Update the patch to adapt new data structure and resctrl_set_kmode_support()
    changes.

v4: New patch to set the supported features during arch init.
---
 arch/x86/kernel/cpu/resctrl/core.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 4ca8bedeb995..39d6a2a60805 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -1021,6 +1021,10 @@ static __init bool get_rdt_resources(void)
 	rdt_alloc_capable = get_rdt_alloc_resources();
 	rdt_mon_capable = get_rdt_mon_resources();
 
+	if (rdt_cpu_has(X86_FEATURE_PLZA) &&
+	    (rdt_alloc_capable || rdt_mon_capable))
+		resctrl_set_kmode_support(RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU);
+
 	return (rdt_mon_capable || rdt_alloc_capable);
 }
 
-- 
2.43.0

[PATCH v5 09/16] fs/resctrl: Add interface to display supported and active kernel modes

[Babu Moger]

Resctrl kernel modes define how resource allocation and monitoring
associations are applied in kernel mode relative to user mode. Generic
resctrl maintains information about supported kernel modes, the currently
active mode, and the resource group associated with global assignment
mode, but this information is not currently visible to user space.

Add info/kernel_mode to expose kernel mode configuration through the
resctrl filesystem. The file lists all supported kernel modes, one per
line, identifies the active mode by enclosing it in square brackets, and
for assign_global_enable_per_cpu also reports ctrl=, mon=, and group=.
Document the read interface in Documentation/filesystems/resctrl.rst.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Use resctrl_kcfg.caps and resctrl_kcfg.active after the struct split
    from patch 0006. Rename resctrl_kmode_bind_str() to
    resctrl_kmode_state_str() and ctrl_bind/mon_bind to ctrl_state/mon_state.
    Used info_kn_lock()/info_kn_unlock().
    Add documentation update in resctrl.rst.

v4: Fixed the display of inherit_ctrl_and_mon policy. It is not
    associated with any group.
    Added "uninitialized" for the non-active group.
    Rewrote the changelog.

v3: New patch to handle the changed interface file info/kernel_mode.
    Changed the group name to "none" if kmode binding is not done.
    Reinette suggested "uninitialized". "none" seemed more relevent.
---
 Documentation/filesystems/resctrl.rst |  29 +++++++
 fs/resctrl/rdtgroup.c                 | 115 ++++++++++++++++++++++++++
 2 files changed, 144 insertions(+)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index f3e941404967..c6e8cf828e18 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -528,6 +528,35 @@ conveyed in the error returns from file operations. E.g.
 	# cat info/last_cmd_status
 	mask f7 has non-consecutive 1-bits
 
+"kernel_mode":
+	In the top level of the "info" directory, "kernel_mode" reports
+	supported and active kernel modes available on the system.
+
+	Reading the file lists one mode per line. The active mode is wrapped in
+	square brackets. inherit_user is shown without options.
+	assign_global_enable_per_cpu is shown as::
+
+	  assign_global_enable_per_cpu:ctrl=<assign|inherit>;mon=<assign|inherit>;group=<ctrl>/<mon>/
+
+	- inherit_user: inherit allocation and monitoring from the user task.
+	- assign_global_enable_per_cpu: kernel mode may use separate
+	  allocation and/or monitoring associations. ctrl= and mon= show
+	  whether each is assigned or inherited, and group= identifies the
+	  associated resource group using <CTRL_MON>/<MON>/ path syntax.
+
+	Only supported modes are listed. On an inactive
+	assign_global_enable_per_cpu line, ctrl= and mon= show platform
+	capability defaults rather than the last active assignment, and
+	group=// is shown. When the mode is active, ctrl= and mon= report
+	the current assignment state and group= identifies the associated
+	resource group.
+
+	Example::
+
+	  # cat info/kernel_mode
+	  [inherit_user]
+	  assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
 Resource alloc and monitor groups
 =================================
 
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 3bb0e3a203ac..992af586a194 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1010,6 +1010,114 @@ static int rdt_last_cmd_status_show(struct kernfs_open_file *of,
 	return 0;
 }
 
+/* Sysfs lines for info/kernel_mode; indexed by enum resctrl_kernel_mode */
+static const char * const resctrl_mode_str[] = {
+	[RESCTRL_INHERIT_USER]	=			"inherit_user",
+	[RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU] =	"assign_global_enable_per_cpu"
+};
+
+static_assert(ARRAY_SIZE(resctrl_mode_str) == RESCTRL_NUM_KERNEL_MODES);
+
+static const char *resctrl_kmode_state_str(enum kmode_state state)
+{
+	return state == KMODE_ASSIGN ? "assign" : "inherit";
+}
+
+static void resctrl_kmode_group_path(struct rdtgroup *rdtgrp,
+				     const char **ctrl, const char **mon)
+{
+	*ctrl = "";
+	*mon = "";
+
+	if (!rdtgrp)
+		return;
+
+	if (rdtgrp->type == RDTMON_GROUP) {
+		*ctrl = rdt_kn_name(rdtgrp->mon.parent->kn);
+		*mon = rdt_kn_name(rdtgrp->kn);
+	} else {
+		*ctrl = rdt_kn_name(rdtgrp->kn);
+	}
+}
+
+/**
+ * resctrl_kernel_mode_show() - Display supported and active kernel modes
+ * @of: kernfs open file
+ * @seq: output seq_file
+ * @v: unused
+ *
+ * Lists one line per mode set in resctrl_kcfg.caps.kmode_sup. Brackets the
+ * active mode. inherit_user is shown without any options.
+ * assign_global_enable_per_cpu is shown as:
+ *
+ *   assign_global_enable_per_cpu:ctrl=<assign|inherit>;mon=<assign|inherit>;\
+ *					group=<ctrl>/<mon>/
+ *
+ * When assign_global_enable_per_cpu is inactive, ctrl=assign and mon=assign
+ * reflect resctrl_kcfg.caps.ctrl_en and resctrl_kcfg.caps.mon_en, and
+ * group=//. When active, assign state comes from resctrl_kcfg.active.ctrl_mode
+ * and resctrl_kcfg.active.mon_mode.
+ *
+ * Return: 0 on success, or -ENOENT on error.
+ */
+static int resctrl_kernel_mode_show(struct kernfs_open_file *of,
+				    struct seq_file *seq, void *v)
+{
+	const char *ctrl_state, *mon_state;
+	enum resctrl_kernel_mode mode;
+	struct rdtgroup *rdtgrp;
+	const char *ctrl, *mon;
+	bool active;
+	int ret = 0;
+
+	if (!info_kn_lock(of->kn))
+		return -ENOENT;
+
+	for (mode = 0; mode < RESCTRL_NUM_KERNEL_MODES; mode++) {
+		if (!test_bit(mode, resctrl_kcfg.caps.kmode_sup))
+			continue;
+
+		active = (resctrl_kcfg.active.kmode_cur == mode);
+
+		if (mode == RESCTRL_INHERIT_USER) {
+			seq_printf(seq, active ? "[%s]\n" : "%s\n",
+				   resctrl_mode_str[mode]);
+			continue;
+		}
+
+		if (active) {
+			ctrl_state = resctrl_kmode_state_str(resctrl_kcfg.active.ctrl_mode);
+			mon_state = resctrl_kmode_state_str(resctrl_kcfg.active.mon_mode);
+			rdtgrp = resctrl_kcfg.active.k_rdtgrp;
+			if (WARN_ON(!rdtgrp)) {
+				rdt_last_cmd_puts("Invalid kernel mode group\n");
+				ret = -ENOENT;
+				goto out_unlock;
+			}
+			resctrl_kmode_group_path(rdtgrp, &ctrl, &mon);
+		} else {
+			ctrl_state = resctrl_kcfg.caps.ctrl_en ? "assign" : "inherit";
+			mon_state = resctrl_kcfg.caps.mon_en ? "assign" : "inherit";
+			ctrl = "";
+			mon = "";
+		}
+
+		if (active) {
+			seq_printf(seq, "[%s:ctrl=%s;mon=%s;group=%s/%s/]\n",
+				   resctrl_mode_str[mode], ctrl_state, mon_state,
+				   ctrl, mon);
+		} else {
+			seq_printf(seq, "%s:ctrl=%s;mon=%s;group=%s/%s/\n",
+				   resctrl_mode_str[mode], ctrl_state, mon_state,
+				   ctrl, mon);
+		}
+	}
+
+out_unlock:
+	info_kn_unlock(of->kn);
+	return ret;
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*
@@ -1995,6 +2103,13 @@ static struct rftype res_common_files[] = {
 		.seq_show	= rdt_last_cmd_status_show,
 		.fflags		= RFTYPE_TOP_INFO,
 	},
+	{
+		.name		= "kernel_mode",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= resctrl_kernel_mode_show,
+		.fflags		= RFTYPE_TOP_INFO,
+	},
 	{
 		.name		= "mbm_assign_on_mkdir",
 		.mode		= 0644,
-- 
2.43.0

[PATCH v5 10/16] fs/resctrl: Add support for hidden resource group files

[Babu Moger]

Some per-group resctrl files are only meaningful once a resource group
has an active kernel mode association (for example, kmode_cpus on the
associated group).

Creating them only at activation would complicate group setup and
teardown, while exposing them on every group would show empty files
with no clear purpose.

Add a hidden property to struct rftype to hide files at creation, and
resctrl_hidden_files_set_visible() to show or hide them when association
state changes.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: New patch to add support for creating hidden files.
---
 fs/resctrl/internal.h |  4 ++++
 fs/resctrl/rdtgroup.c | 30 ++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index 4087bf44a06d..b56f95625072 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -283,6 +283,7 @@ extern int max_name_width;
  * @kf_ops:	File operations
  * @flags:	File specific RFTYPE_FLAGS_* flags
  * @fflags:	File specific RFTYPE_* flags
+ * @hidden:	Hide file at creation; visibility may be restored later
  * @seq_show:	Show content of the file
  * @write:	Write to the file
  */
@@ -292,6 +293,7 @@ struct rftype {
 	const struct kernfs_ops	*kf_ops;
 	unsigned long		flags;
 	unsigned long		fflags;
+	bool			hidden;
 
 	int (*seq_show)(struct kernfs_open_file *of,
 			struct seq_file *sf, void *v);
@@ -480,6 +482,8 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
 void resctrl_bmec_files_show(struct rdt_resource *r, struct kernfs_node *l3_mon_kn,
 			     bool show);
 
+void resctrl_hidden_files_set_visible(struct kernfs_node *kn, bool show);
+
 int resctrl_num_mbm_cntrs_show(struct kernfs_open_file *of, struct seq_file *s, void *v);
 
 int resctrl_available_mbm_cntrs_show(struct kernfs_open_file *of, struct seq_file *s,
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 992af586a194..d2fff8adf915 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -312,6 +312,9 @@ static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
 		return ret;
 	}
 
+	if (rft->hidden)
+		kernfs_show(kn, false);
+
 	return 0;
 }
 
@@ -2377,6 +2380,33 @@ static int rdtgroup_add_files(struct kernfs_node *kn, unsigned long fflags)
 	return ret;
 }
 
+/*
+ * resctrl_hidden_files_set_visible() - Show or hide files marked hidden
+ * @kn: resource group kernfs_node
+ * @show: whether to show or hide
+ *
+ * Iterate res_common_files entries marked hidden and show or hide the
+ * corresponding file under @kn.
+ */
+void resctrl_hidden_files_set_visible(struct kernfs_node *kn, bool show)
+{
+	struct rftype *rft, *rfts = res_common_files;
+	struct kernfs_node *kn_file;
+	int len;
+
+	len = ARRAY_SIZE(res_common_files);
+	for (rft = rfts; rft < rfts + len; rft++) {
+		if (!rft->hidden)
+			continue;
+
+		kn_file = kernfs_find_and_get(kn, rft->name);
+		if (!kn_file)
+			continue;
+		kernfs_show(kn_file, show);
+		kernfs_put(kn_file);
+	}
+}
+
 static struct rftype *rdtgroup_get_rftype_by_name(const char *name)
 {
 	struct rftype *rfts, *rft;
-- 
2.43.0

[PATCH v5 11/16] fs/resctrl: Introduce kmode_cpus/kmode_cpus_list per rdtgroup

[Babu Moger]

Kernel-mode resctrl policies can associate kernel resource allocation and
monitoring activity with a specific rdtgroup, optionally on a subset of
online CPUs.

While info/kernel_mode exposes the active kernel mode and its associated
rdtgroup, there is currently no way for user space to determine which CPUs
participate in that association.

Add read-only kmode_cpus and kmode_cpus_list files to each rdtgroup and
create them hidden. These files expose rdtgrp->kmode_cpu_mask in bitmap and
range-list formats, respectively, matching the existing cpus and cpus_list
interfaces.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Updated the changelog.
    Removed the RDT_MODE_PSEUDO_LOCKED check and added rdtgroup->kmode check.
    Files are created hidden.

v4: Rewrote the change to be generic.

v3: New patch to add "kmode_cpus" and "kmode_cpus_list" to support
    kernel_modes.
---
 fs/resctrl/internal.h |  5 +++++
 fs/resctrl/rdtgroup.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index b56f95625072..01f9f596f9b1 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -216,6 +216,9 @@ struct mongroup {
  * @mon:			mongroup related data
  * @mode:			mode of resource group
  * @mba_mbps_event:		input monitoring event id when mba_sc is enabled
+ * @kmode:			true if this group has an active kernel-mode
+ *				association
+ * @kmode_cpu_mask:		CPUs in this group's kernel-mode association
  * @plr:			pseudo-locked region
  */
 struct rdtgroup {
@@ -229,6 +232,8 @@ struct rdtgroup {
 	struct mongroup			mon;
 	enum rdtgrp_mode		mode;
 	enum resctrl_event_id		mba_mbps_event;
+	bool				kmode;
+	struct cpumask			kmode_cpu_mask;
 	struct pseudo_lock_region	*plr;
 };
 
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index d2fff8adf915..06e74b027044 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -389,6 +389,32 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
 	return ret;
 }
 
+/*
+ * Display CPU masks for the kernel-mode associated resource group.
+ * Supports both "kmode_cpus" (bitmap format) and "kmode_cpus_list"
+ * (range list format); the output format is selected accordingly.
+ *
+ * Returns -ENOENT on error.
+ */
+static int rdtgroup_kmode_cpus_show(struct kernfs_open_file *of,
+				    struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+
+	if (rdtgrp && rdtgrp->kmode) {
+		seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
+			   cpumask_pr_args(&rdtgrp->kmode_cpu_mask));
+	} else {
+		ret = -ENOENT;
+	}
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
 /*
  * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
  *
@@ -2288,6 +2314,23 @@ static struct rftype res_common_files[] = {
 		.flags		= RFTYPE_FLAGS_CPUS_LIST,
 		.fflags		= RFTYPE_BASE,
 	},
+	{
+		.name		= "kmode_cpus",
+		.mode		= 0444,
+		.hidden		= true,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_kmode_cpus_show,
+		.fflags		= RFTYPE_BASE,
+	},
+	{
+		.name		= "kmode_cpus_list",
+		.mode		= 0444,
+		.hidden		= true,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_kmode_cpus_show,
+		.flags		= RFTYPE_FLAGS_CPUS_LIST,
+		.fflags		= RFTYPE_BASE,
+	},
 	{
 		.name		= "tasks",
 		.mode		= 0644,
-- 
2.43.0

[PATCH v5 12/16] fs/resctrl: Program kernel mode assignments on CPU hotplug

[Babu Moger]

Kernel mode resctrl associations are programmed on a per-CPU basis. When
assign_global_enable_per_cpu is selected, the active kernel mode
association is applied only to CPUs that are online at the time of
configuration. CPUs that come online later are not automatically updated.

As a result, hotplugged CPUs, or CPUs that were offline when the kernel
mode was activated, may run without the intended kernel mode association
even though the mode remains active system-wide.

Add resctrl_kmode_online_cpu() and resctrl_kmode_offline_cpu() and call
them from the existing resctrl_online_cpu() and resctrl_offline_cpu()
paths to maintain kmode_cpu_mask and update kernel mode associations as
CPUs enter or leave the online state.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Added resctrl_kmode_offline_cpu() to disable the kernel mode
    association and update kmode_cpu_mask when an associated CPU goes
    offline.
    This addresses sashiko comments:
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=11
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=10
    Also moved the patch earlier than patch 10.
    [RESEND PATCH v4 10/15] fs/resctrl: Reset the kernel mode binding when an rdtgroup is removed
    That seemed much more clear.

v4: New patch in the series. Patch taken from:
    https://lore.kernel.org/lkml/20260611111706.1981788-5-qinyuntan@linux.alibaba.com/
    Updated the code to enable kernel mode by default whenever a CPU comes
    online when one of global-assign mode is enabled.
---
 fs/resctrl/rdtgroup.c | 59 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 06e74b027044..97e4176669e5 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -4951,11 +4951,67 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr
 	return err;
 }
 
+/*
+ * resctrl_kmode_online_cpu() - Program kernel mode association for @cpu
+ * @cpu: CPU that has just been brought online
+ *
+ * If assign_global_enable_per_cpu is active and the rdtgroup has an active
+ * kernel-mode association, add @cpu to kmode_cpu_mask and program the
+ * corresponding kernel mode association.
+ */
+static void resctrl_kmode_online_cpu(unsigned int cpu)
+{
+	struct rdtgroup *rdtgrp = resctrl_kcfg.active.k_rdtgrp;
+	bool assign_ctrl, assign_mon;
+
+	if (resctrl_kcfg.active.kmode_cur != RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU ||
+	    !rdtgrp || !rdtgrp->kmode)
+		return;
+
+	assign_ctrl = (resctrl_kcfg.active.ctrl_mode == KMODE_ASSIGN);
+	assign_mon = (resctrl_kcfg.active.mon_mode == KMODE_ASSIGN);
+
+	cpumask_set_cpu(cpu, &rdtgrp->kmode_cpu_mask);
+
+	resctrl_arch_configure_kmode(cpumask_of(cpu), rdtgrp->closid, assign_ctrl,
+				     rdtgrp->mon.rmid, assign_mon, true);
+}
+
+/*
+ * resctrl_kmode_offline_cpu() - Clear kernel mode association for @cpu
+ * @cpu: CPU being taken offline.
+ *
+ * If assign_global_enable_per_cpu is active, disable the kernel mode
+ * association for @cpu and remove it from kmode_cpu_mask.
+ */
+static void resctrl_kmode_offline_cpu(unsigned int cpu)
+{
+	struct rdtgroup *rdtgrp = resctrl_kcfg.active.k_rdtgrp;
+	bool assign_ctrl, assign_mon;
+
+	if (resctrl_kcfg.active.kmode_cur != RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU ||
+	    !rdtgrp || !rdtgrp->kmode)
+		return;
+
+	if (!cpumask_test_cpu(cpu, &rdtgrp->kmode_cpu_mask))
+		return;
+
+	assign_ctrl = (resctrl_kcfg.active.ctrl_mode == KMODE_ASSIGN);
+	assign_mon = (resctrl_kcfg.active.mon_mode == KMODE_ASSIGN);
+
+	cpumask_clear_cpu(cpu, &rdtgrp->kmode_cpu_mask);
+
+	resctrl_arch_configure_kmode(cpumask_of(cpu), rdtgrp->closid, assign_ctrl,
+				     rdtgrp->mon.rmid, assign_mon, false);
+}
+
 void resctrl_online_cpu(unsigned int cpu)
 {
 	mutex_lock(&rdtgroup_mutex);
 	/* The CPU is set in default rdtgroup after online. */
 	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
+	/* Program any active kernel mode association on this CPU. */
+	resctrl_kmode_online_cpu(cpu);
 	mutex_unlock(&rdtgroup_mutex);
 }
 
@@ -4999,6 +5055,9 @@ void resctrl_offline_cpu(unsigned int cpu)
 		}
 	}
 
+	/* Clear any active kernel mode association on this CPU. */
+	resctrl_kmode_offline_cpu(cpu);
+
 	if (!l3->mon_capable)
 		goto out_unlock;
 
-- 
2.43.0

[PATCH v5 13/16] fs/resctrl: Deactivate the kernel mode association when a group is removed

[Babu Moger]

Resctrl tracks the rdtgroup associated with the active kernel mode,
including the CPU mask used to program kernel mode associations.

When the associated rdtgroup is removed or the filesystem is unmounted,
the hardware configuration programmed for its kmode_cpu_mask must be
torn down and global kernel-mode state restored to the default.

Call rdtgroup_kmode_detach() from group removal and filesystem teardown
paths to disable the active association, hide kmode_cpus files, clear
per-group state, and restore RESCTRL_INHERIT_USER.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Took care of couple of Sashiko comments on previous patch.
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=10
    Re-wrote the changelog.
    Renamed rdtgroup_config_kmode_reset() to rdtgroup_kmode_deactivate().

v4: Re-wrote the changelog.
    Added the call free_all_child_rdtgrp() and rmdir_all_sub()
    Simplified the code comments.

v3: New patch to handle the kernel_mode clean up.
---
 fs/resctrl/rdtgroup.c | 58 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 97e4176669e5..ed4fedbf2193 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1147,6 +1147,57 @@ static int resctrl_kernel_mode_show(struct kernfs_open_file *of,
 	return ret;
 }
 
+/**
+ * rdtgroup_kmode_deactivate() - Tear down the active kernel mode on @rdtgrp
+ * @rdtgrp:	Resctrl group whose active kernel mode is being released.
+ *
+ * Deactivate the kernel mode association on the CPUs in @rdtgrp's
+ * @kmode_cpu_mask, hide kmode_cpus files, and clear the group's kernel-mode
+ * state.
+ */
+static void rdtgroup_kmode_deactivate(struct rdtgroup *rdtgrp)
+{
+	bool assign_ctrl, assign_mon;
+
+	assign_ctrl = (resctrl_kcfg.active.ctrl_mode == KMODE_ASSIGN);
+	assign_mon = (resctrl_kcfg.active.mon_mode == KMODE_ASSIGN);
+
+	resctrl_arch_configure_kmode(&rdtgrp->kmode_cpu_mask, rdtgrp->closid,
+				     assign_ctrl, rdtgrp->mon.rmid, assign_mon,
+				     false);
+
+	resctrl_hidden_files_set_visible(rdtgrp->kn, false);
+	cpumask_clear(&rdtgrp->kmode_cpu_mask);
+	rdtgrp->kmode = false;
+}
+
+/**
+ * rdtgroup_kmode_detach() - Detach @rdtgrp from a kernel mode association
+ * @rdtgrp: Resctrl group being removed or torn down.
+ *
+ * If @rdtgrp backs the active kernel mode association, disable the
+ * hardware association programmed for the group's kmode_cpu_mask,
+ * clear the associated kernel mode state, and restore
+ * RESCTRL_INHERIT_USER as the active mode.
+ */
+static void rdtgroup_kmode_detach(struct rdtgroup *rdtgrp)
+{
+	if (!rdtgrp || !rdtgrp->kmode)
+		return;
+
+	if (resctrl_kcfg.active.k_rdtgrp != rdtgrp) {
+		pr_warn("resctrl: kernel mode group not valid\n");
+		return;
+	}
+
+	rdtgroup_kmode_deactivate(rdtgrp);
+
+	resctrl_kcfg.active.k_rdtgrp = NULL;
+	resctrl_kcfg.active.kmode_cur = RESCTRL_INHERIT_USER;
+	resctrl_kcfg.active.ctrl_mode = KMODE_INHERIT;
+	resctrl_kcfg.active.mon_mode = KMODE_INHERIT;
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*
@@ -3235,6 +3286,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp)
 
 	head = &rdtgrp->mon.crdtgrp_list;
 	list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) {
+		rdtgroup_kmode_detach(sentry);
 		rdtgroup_unassign_cntrs(sentry);
 		free_rmid(sentry->closid, sentry->mon.rmid);
 		list_del(&sentry->mon.crdtgrp_list);
@@ -3272,6 +3324,7 @@ static void rmdir_all_sub(void)
 		cpumask_or(&rdtgroup_default.cpu_mask,
 			   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
 
+		rdtgroup_kmode_detach(rdtgrp);
 		rdtgroup_unassign_cntrs(rdtgrp);
 
 		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
@@ -3367,6 +3420,7 @@ static void resctrl_fs_teardown(void)
 		return;
 
 	rmdir_all_sub();
+	rdtgroup_kmode_detach(&rdtgroup_default);
 	rdtgroup_unassign_cntrs(&rdtgroup_default);
 	mon_put_kn_priv();
 	rdt_pseudo_lock_release();
@@ -4402,6 +4456,8 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 	u32 closid, rmid;
 	int cpu;
 
+	rdtgroup_kmode_detach(rdtgrp);
+
 	/* Give any tasks back to the parent group */
 	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
 
@@ -4452,6 +4508,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 	u32 closid, rmid;
 	int cpu;
 
+	rdtgroup_kmode_detach(rdtgrp);
+
 	/* Give any tasks back to the default group */
 	rdt_move_group_tasks(rdtgrp, &rdtgroup_default, tmpmask);
 
-- 
2.43.0

[PATCH v5 14/16] fs/resctrl: Add interface to modify kernel mode via info/kernel_mode

[Babu Moger]

info/kernel_mode reports supported and active kernel modes, but it is
read-only. User space cannot select a policy, associate a resource group,
or configure independent control and monitoring assignment for
assign_global_enable_per_cpu.

Add resctrl_kernel_mode_write() to accept the same line format as
resctrl_kernel_mode_show(). Parse optional ctrl=, mon=, and group= options,
validate the request before changing state, and program the association
through rdtgroup_kmode_activate() and rdtgroup_kmode_deactivate().

Document the write interface in Documentation/filesystems/resctrl.rst.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Adapt to the resctrl_kcfg caps and active split. Accept
    assign_global_enable_per_cpu:ctrl=;mon=;group=<ctrl>/<mon>/ writes.
    Treat both ctrl=inherit and mon=inherit as a no-op. Use group=// for
    the default control group. Show associated kmode files through
    resctrl_hidden_files_set_visible().
    Added kmode check in rdtgroup_locksetup_enter(), rdtgroup_mode_write()
    and rdtgroup_rename().
    Simplify kernel-doc for write and activate helpers.

v4: Validate input before tearing down the active association so failures
    retain the previous association. Allow assign_mon on CTRL_MON and MON
    groups; require a control group when control assignment is inherit.

v3: New patch to make info/kernel_mode writable.
---
 Documentation/filesystems/resctrl.rst |  33 +++
 fs/resctrl/pseudo_lock.c              |   5 +
 fs/resctrl/rdtgroup.c                 | 319 +++++++++++++++++++++++++-
 3 files changed, 356 insertions(+), 1 deletion(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index c6e8cf828e18..490e8f534d37 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -557,6 +557,39 @@ conveyed in the error returns from file operations. E.g.
 	  [inherit_user]
 	  assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=//
 
+	Writes use the same line format as read, without square brackets and
+	with a trailing newline. To select inherit_user, write that mode name
+	alone.
+
+	For assign_global_enable_per_cpu:
+
+	- Writing the mode name alone selects ctrl=assign, mon=assign, and the
+	  default CTRL_MON group.
+	- ctrl=, mon=, and group= are optional, use the same syntax as on read,
+	  and may appear in any order. ctrl= and mon= default to assign;
+	  group= defaults to the default CTRL_MON group.
+	- Empty ctrl=, mon=, or group= values are rejected.
+	- A write with both ctrl=inherit and mon=inherit is a no-op.
+
+	Selecting a new mode, assignment state, or group tears down any active
+	assign_global_enable_per_cpu association before programming the new
+	one. Writing inherit_user clears the active kernel mode association.
+	The mode name must match a supported value exactly. Errors are
+	reported in "info/last_cmd_status". Example::
+
+	  # mkdir ctrl1
+	  # echo "assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//" \
+	         > info/kernel_mode
+
+	  # cat info/kernel_mode
+	  inherit_user
+	  [assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//]
+
+	  # echo "inherit_user" > info/kernel_mode
+	  # cat info/kernel_mode
+	  [inherit_user]
+	  assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
 Resource alloc and monitor groups
 =================================
 
diff --git a/fs/resctrl/pseudo_lock.c b/fs/resctrl/pseudo_lock.c
index dea2b4bf966f..a0b22b95fc08 100644
--- a/fs/resctrl/pseudo_lock.c
+++ b/fs/resctrl/pseudo_lock.c
@@ -536,6 +536,11 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
 		return -EINVAL;
 	}
 
+	if (rdtgrp->kmode) {
+		rdt_last_cmd_puts("Group has an active kernel-mode association\n");
+		return -EINVAL;
+	}
+
 	if (rdtgroup_locksetup_user_restrict(rdtgrp)) {
 		rdt_last_cmd_puts("Unable to modify resctrl permissions\n");
 		return -EIO;
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index ed4fedbf2193..10d936710899 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1198,6 +1198,309 @@ static void rdtgroup_kmode_detach(struct rdtgroup *rdtgrp)
 	resctrl_kcfg.active.mon_mode = KMODE_INHERIT;
 }
 
+/**
+ * rdtgroup_kmode_activate() - Program kernel mode for @rdtgrp
+ * @rdtgrp:	Resctrl group whose CLOSID/RMID should be programmed.
+ *
+ * @rdtgrp carries the CLOSID/RMID to program. For monitor groups, the CLOSID
+ * matches the parent control group while the RMID belongs to the monitor group.
+ *
+ * Use resctrl_kcfg.active ctrl_mode and mon_mode, initialize
+ * @rdtgrp->kmode_cpu_mask from online CPUs, program the association, and
+ * show hidden kmode_cpus files.
+ */
+static void rdtgroup_kmode_activate(struct rdtgroup *rdtgrp)
+{
+	bool assign_ctrl = (resctrl_kcfg.active.ctrl_mode == KMODE_ASSIGN);
+	bool assign_mon = (resctrl_kcfg.active.mon_mode == KMODE_ASSIGN);
+
+	cpumask_copy(&rdtgrp->kmode_cpu_mask, cpu_online_mask);
+
+	resctrl_arch_configure_kmode(&rdtgrp->kmode_cpu_mask, rdtgrp->closid,
+				     assign_ctrl, rdtgrp->mon.rmid, assign_mon,
+				     true);
+
+	rdtgrp->kmode = true;
+	resctrl_hidden_files_set_visible(rdtgrp->kn, true);
+}
+
+/**
+ * rdtgroup_by_kmode_path() - Resolve a "<ctrl>/<mon>/" path to an rdtgroup
+ * @ctrl_name:	Control-group name, or "" for the default control group.
+ * @mon_name:	Monitor-group name, or "" to select the control group itself.
+ *
+ * Matches the path syntax emitted by resctrl_kernel_mode_show():
+ *   "//"            - the default control group
+ *   "<ctrl>//"      - control group @ctrl_name
+ *   "/<mon>/"       - monitor group @mon_name under the default control group
+ *   "<ctrl>/<mon>/" - monitor group @mon_name under control group @ctrl_name
+ *
+ * Return: Pointer to the matching rdtgroup, or NULL if no such group exists.
+ */
+static struct rdtgroup *rdtgroup_by_kmode_path(const char *ctrl_name,
+					       const char *mon_name)
+{
+	struct rdtgroup *rdtg, *parent = &rdtgroup_default;
+
+	if (*ctrl_name) {
+		parent = NULL;
+		list_for_each_entry(rdtg, &rdt_all_groups, rdtgroup_list) {
+			if (rdtg->type != RDTCTRL_GROUP)
+				continue;
+			if (!strcmp(rdt_kn_name(rdtg->kn), ctrl_name)) {
+				parent = rdtg;
+				break;
+			}
+		}
+	}
+	if (!parent)
+		return NULL;
+
+	if (!*mon_name)
+		return parent;
+
+	list_for_each_entry(rdtg, &parent->mon.crdtgrp_list, mon.crdtgrp_list)
+		if (!strcmp(rdt_kn_name(rdtg->kn), mon_name))
+			return rdtg;
+	return NULL;
+}
+
+static int resctrl_kmode_parse_option(char *val, enum kmode_state *state)
+{
+	val = strim(val);
+
+	if (!*val)
+		return -EINVAL;
+	if (!strcmp(val, "assign")) {
+		*state = KMODE_ASSIGN;
+		return 0;
+	}
+	if (!strcmp(val, "inherit")) {
+		*state = KMODE_INHERIT;
+		return 0;
+	}
+	return -EINVAL;
+}
+
+static int resctrl_kmode_parse_ctrl_mon(char *options, const char *field,
+				    enum kmode_state *state)
+{
+	char *opt, *val, *end;
+	int ret = 0;
+
+	opt = strstr(options, field);
+	if (!opt)
+		return 0;
+
+	val = opt + strlen(field);
+	end = strchr(val, ';');
+	if (end)
+		*end = '\0';
+	if (resctrl_kmode_parse_option(val, state))
+		ret = -EINVAL;
+
+	if (end)
+		*end = ';';
+	return ret;
+}
+
+static int resctrl_kmode_parse_group(char *options, struct rdtgroup **rdtgrp)
+{
+	const char *ctrl_name, *mon_name;
+	char *group_str, *end, *slash;
+	struct rdtgroup *grp;
+	int ret = 0;
+
+	/* Skip parsing when group= is not present. */
+	group_str = strstr(options, "group=");
+	if (!group_str)
+		return 0;
+
+	/* Isolate the group= value from any following options. */
+	group_str += strlen("group=");
+	end = strchr(group_str, ';');
+	if (end)
+		*end = '\0';
+	group_str = strim(group_str);
+	if (!*group_str) {
+		rdt_last_cmd_puts("group= requires <CTRL_MON>/<MON>/\n");
+		ret = -EINVAL;
+		goto out_parse;
+	}
+	/* Split <CTRL_MON>/<MON>/ at the first slash. */
+	slash = strchr(group_str, '/');
+	if (!slash) {
+		rdt_last_cmd_puts("Group must be <CTRL_MON>/<MON>/\n");
+		ret = -EINVAL;
+		goto out_parse;
+	}
+	*slash = '\0';
+	ctrl_name = group_str;
+	mon_name = slash + 1;
+	/* Require a trailing slash after the monitor group name. */
+	slash = strchr(mon_name, '/');
+	if (!slash || slash[1] != '\0') {
+		rdt_last_cmd_puts("Group must be <CTRL_MON>/<MON>/\n");
+		ret = -EINVAL;
+		goto out_parse;
+	}
+	*slash = '\0';
+	/* Resolve the path to an existing rdtgroup. */
+	grp = rdtgroup_by_kmode_path(ctrl_name, mon_name);
+	if (!grp) {
+		rdt_last_cmd_puts("Group not found\n");
+		ret = -EINVAL;
+		goto out_parse;
+	}
+	*rdtgrp = grp;
+
+out_parse:
+	/* Restore the option string after temporary null termination. */
+	if (end)
+		*end = ';';
+	return ret;
+}
+
+/**
+ * resctrl_kernel_mode_write() - Set the active kernel mode policy
+ * @of: kernfs open file
+ * @buf: Write buffer; use the resctrl_kernel_mode_show() line format without
+ *	 brackets and with a trailing newline
+ * @nbytes: length of @buf
+ * @off: unused
+ *
+ * Parse and validate the request, then update the active kernel mode
+ * association.
+ *
+ * Return: @nbytes on success, negative errno on error.
+ */
+static ssize_t resctrl_kernel_mode_write(struct kernfs_open_file *of,
+					 char *buf, size_t nbytes, loff_t off)
+{
+	enum kmode_state ctrl_mode = KMODE_ASSIGN, mon_mode = KMODE_ASSIGN;
+	char *mode_str, *options;
+	enum resctrl_kernel_mode mode;
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	if (!info_kn_lock(of->kn))
+		return -ENOENT;
+
+	rdt_last_cmd_clear();
+
+	if (nbytes == 0 || buf[nbytes - 1] != '\n') {
+		rdt_last_cmd_puts("kernel_mode_write: Invalid input\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+	buf[nbytes - 1] = '\0';
+
+	buf = strim(buf);
+	options = strchr(buf, ':');
+	if (options) {
+		*options = '\0';
+		options++;
+	}
+	mode_str = strim(buf);
+
+	for (mode = 0; mode < RESCTRL_NUM_KERNEL_MODES; mode++)
+		if (!strcmp(mode_str, resctrl_mode_str[mode]))
+			break;
+
+	if (mode == RESCTRL_NUM_KERNEL_MODES) {
+		rdt_last_cmd_puts("Unknown kernel mode\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (!test_bit(mode, resctrl_kcfg.caps.kmode_sup)) {
+		rdt_last_cmd_puts("Kernel mode not available\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (mode == RESCTRL_INHERIT_USER) {
+		rdtgrp = NULL;
+		goto update_mode;
+	}
+
+	rdtgrp = &rdtgroup_default;
+
+	if (!options)
+		goto validate_kmode;
+
+	ret = resctrl_kmode_parse_ctrl_mon(options, "ctrl=", &ctrl_mode);
+	if (ret) {
+		rdt_last_cmd_puts("Invalid ctrl= option\n");
+		goto out_unlock;
+	}
+	ret = resctrl_kmode_parse_ctrl_mon(options, "mon=", &mon_mode);
+	if (ret) {
+		rdt_last_cmd_puts("Invalid mon= option\n");
+		goto out_unlock;
+	}
+
+	ret = resctrl_kmode_parse_group(options, &rdtgrp);
+	if (ret)
+		goto out_unlock;
+
+	if (ctrl_mode == KMODE_INHERIT && mon_mode == KMODE_INHERIT)
+		goto out_unlock;
+
+validate_kmode:
+	if (ctrl_mode == KMODE_ASSIGN && !resctrl_kcfg.caps.ctrl_en) {
+		rdt_last_cmd_puts("Control assignment not supported\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (mon_mode == KMODE_ASSIGN && !resctrl_kcfg.caps.mon_en) {
+		rdt_last_cmd_puts("Monitoring assignment not supported\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	/* Applying mon=inherit on monitor group is no-op. Reject it */
+	if (mon_mode == KMODE_INHERIT && rdtgrp->type == RDTMON_GROUP) {
+		rdt_last_cmd_puts("Monitoring inherit on mon group is no-op\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (rdtgrp->mode != RDT_MODE_SHAREABLE) {
+		rdt_last_cmd_puts("The group is not in shareable mode\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+update_mode:
+	if (resctrl_kcfg.active.kmode_cur == mode &&
+	    resctrl_kcfg.active.k_rdtgrp == rdtgrp &&
+	    resctrl_kcfg.active.ctrl_mode == ctrl_mode &&
+	    resctrl_kcfg.active.mon_mode == mon_mode)
+		goto out_unlock;
+
+	if (resctrl_kcfg.active.kmode_cur != RESCTRL_INHERIT_USER &&
+	    resctrl_kcfg.active.k_rdtgrp)
+		rdtgroup_kmode_deactivate(resctrl_kcfg.active.k_rdtgrp);
+
+	if (mode == RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU) {
+		resctrl_kcfg.active.ctrl_mode = ctrl_mode;
+		resctrl_kcfg.active.mon_mode = mon_mode;
+		rdtgroup_kmode_activate(rdtgrp);
+		resctrl_kcfg.active.k_rdtgrp = rdtgrp;
+	} else {
+		resctrl_kcfg.active.k_rdtgrp = NULL;
+		resctrl_kcfg.active.ctrl_mode = KMODE_INHERIT;
+		resctrl_kcfg.active.mon_mode = KMODE_INHERIT;
+	}
+	resctrl_kcfg.active.kmode_cur = mode;
+
+out_unlock:
+	info_kn_unlock(of->kn);
+	return ret ?: nbytes;
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*
@@ -1745,6 +2048,12 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
 
 	buf[nbytes - 1] = '\0';
 
+	if (rdtgrp->kmode) {
+		rdt_last_cmd_puts("Cannot change mode of a kernel-mode associated group\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
 	mode = rdtgrp->mode;
 
 	if ((!strcmp(buf, "shareable") && mode == RDT_MODE_SHAREABLE) ||
@@ -2185,9 +2494,10 @@ static struct rftype res_common_files[] = {
 	},
 	{
 		.name		= "kernel_mode",
-		.mode		= 0444,
+		.mode		= 0644,
 		.kf_ops		= &rdtgroup_kf_single_ops,
 		.seq_show	= resctrl_kernel_mode_show,
+		.write		= resctrl_kernel_mode_write,
 		.fflags		= RFTYPE_TOP_INFO,
 	},
 	{
@@ -4699,6 +5009,13 @@ static int rdtgroup_rename(struct kernfs_node *kn,
 		goto out;
 	}
 
+	/* Check for kernel mode association */
+	if (rdtgrp->kmode) {
+		rdt_last_cmd_puts("Cannot move a kernel-mode associated group\n");
+		ret = -EPERM;
+		goto out;
+	}
+
 	/*
 	 * Allocate the cpumask for use in mongrp_reparent() to avoid the
 	 * possibility of failing to allocate it after kernfs_rename() has
-- 
2.43.0

[PATCH v5 15/16] fs/resctrl: Allow user space to write kmode_cpus/kmode_cpus_list

[Babu Moger]

Per-CPU kernel mode associates a resource group with kernel-mode allocation
and monitoring and records its CPU scope in kmode_cpus and
kmode_cpus_list. Administrators may need to add, remove, or clear CPUs
after activation without changing the active mode.

The files are read-only, so the CPU mask cannot be changed without
deactivating kernel mode.

Make kmode_cpus and kmode_cpus_list writable for the associated group.
Validate input, update kmode_cpu_mask, and reprogram kernel-mode
association only on CPUs whose assignment changes. Document the interface
in Documentation/filesystems/resctrl.rst.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Adapt to the resctrl_kcfg caps and active split. Use
    resctrl_kcfg.active.ctrl_mode and mon_mode in kmode_cpus_write().
    Use full ctrl=/mon= syntax in the kmode_cpus example.

v4: Empty masks are now allowed and updated masks are in rdtgroup->kmode_cpu_mask.
    Updated the changelog.

v3: New patch to add "kmode_cpus" and "kmode_cpus_list" to support
    kernel_modes.
---
 Documentation/filesystems/resctrl.rst |  30 ++++++
 fs/resctrl/rdtgroup.c                 | 131 +++++++++++++++++++++++++-
 2 files changed, 159 insertions(+), 2 deletions(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 490e8f534d37..3e886f84315b 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -658,6 +658,36 @@ All groups contain the following files:
 "cpus_list":
 	Just like "cpus", only using ranges of CPUs instead of bitmasks.
 
+"kmode_cpus":
+	Accessible only within the resource group currently associated with
+	the active kernel mode (see "info/kernel_mode").
+
+	Bitmask of the logical CPUs associated with the group's kernel mode.
+	When kernel mode is activated through info/kernel_mode, every currently
+	online CPU is included. CPUs that come online later are automatically
+	added to the association.
+
+	Writing a mask reprograms the association: it enables on the CPUs newly
+	added by the write and disables on the CPUs dropped from the previous
+	mask. An empty mask disables the association on all currently online
+	CPUs, but CPUs that come online later are still automatically
+	associated. The mask must contain only online CPUs; masks naming offline
+	CPUs are rejected. Errors are reported in "info/last_cmd_status".
+	Example::
+
+	  # mkdir ctrl1
+	  # echo "assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//" \
+	        > info/kernel_mode
+	  # echo 0-3 > ctrl1/kmode_cpus_list
+	  # cat ctrl1/kmode_cpus
+	  f
+	  # cat ctrl1/kmode_cpus_list
+	  0-3
+
+"kmode_cpus_list":
+	Just like "kmode_cpus", only using ranges of CPUs instead of bitmasks.
+	Writable with the same semantics and restrictions as "kmode_cpus".
+
 
 When control is enabled all CTRL_MON groups will also contain:
 
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 10d936710899..576f19a9baa6 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -415,6 +415,131 @@ static int rdtgroup_kmode_cpus_show(struct kernfs_open_file *of,
 	return ret;
 }
 
+/**
+ * kmode_cpus_write() - Update @rdtgrp's kmode_cpu_mask from @newmask
+ * @rdtgrp:	Resctrl group whose kmode_cpu_mask is being updated.
+ * @newmask:	New CPUs for @rdtgrp's kernel mode association.
+ * @tmpmask:	Caller-allocated scratch cpumask used to compute the
+ *		incremental enable/disable deltas; contents on entry are
+ *		ignored and on return are unspecified.
+ *
+ * Reprogram kernel-mode association only on CPUs added to or removed from
+ * @rdtgrp->kmode_cpu_mask, then update the mask to @newmask.
+ */
+static void kmode_cpus_write(struct rdtgroup *rdtgrp,
+			     cpumask_var_t newmask, cpumask_var_t tmpmask)
+{
+	bool assign_ctrl = (resctrl_kcfg.active.ctrl_mode == KMODE_ASSIGN);
+	bool assign_mon = (resctrl_kcfg.active.mon_mode == KMODE_ASSIGN);
+	u32 closid, rmid;
+
+	closid = rdtgrp->closid;
+	rmid = rdtgrp->mon.rmid;
+
+	/* CPUs dropped from this group: old & ~newmask. */
+	cpumask_andnot(tmpmask, &rdtgrp->kmode_cpu_mask, newmask);
+	if (!cpumask_empty(tmpmask))
+		resctrl_arch_configure_kmode(tmpmask, closid, assign_ctrl, rmid,
+					     assign_mon, false);
+
+	/* CPUs newly added: newmask & ~old. */
+	cpumask_andnot(tmpmask, newmask, &rdtgrp->kmode_cpu_mask);
+	if (!cpumask_empty(tmpmask))
+		resctrl_arch_configure_kmode(tmpmask, closid, assign_ctrl, rmid,
+					     assign_mon, true);
+
+	cpumask_copy(&rdtgrp->kmode_cpu_mask, newmask);
+}
+
+/**
+ * rdtgroup_kmode_cpus_write() - Write kmode_cpus or kmode_cpus_list
+ * @of: kernfs open file
+ * @buf: CPU bitmask or list from userspace
+ * @nbytes: length of @buf
+ * @off: unused
+ *
+ * Parse and validate @buf for the group associated through info/kernel_mode,
+ * then update kernel-mode association via kmode_cpus_write().
+ *
+ * Return: @nbytes on success, negative errno on error.
+ */
+static ssize_t rdtgroup_kmode_cpus_write(struct kernfs_open_file *of,
+					 char *buf, size_t nbytes, loff_t off)
+{
+	cpumask_var_t tmpmask, newmask;
+	struct rdtgroup *rdtgrp;
+	int ret;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		ret = -ENOENT;
+		goto out_unlock;
+	}
+
+	if (!buf) {
+		rdt_last_cmd_printf("%s: Invalid input\n",
+				    is_cpu_list(of) ? "kmode_cpus_list" :
+						      "kmode_cpus");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL) ||
+	    !zalloc_cpumask_var(&newmask, GFP_KERNEL)) {
+		rdt_last_cmd_printf("%s: Kernel allocation failure\n",
+				    is_cpu_list(of) ? "kmode_cpus_list" :
+						      "kmode_cpus");
+		ret = -ENOMEM;
+		goto out_free;
+	}
+
+	if (resctrl_kcfg.active.kmode_cur == RESCTRL_INHERIT_USER) {
+		rdt_last_cmd_puts("No active kernel-mode association\n");
+		ret = -EBUSY;
+		goto out_free;
+	}
+
+	if (resctrl_kcfg.active.k_rdtgrp != rdtgrp) {
+		rdt_last_cmd_puts("Group is not associated with kernel mode\n");
+		ret = -EBUSY;
+		goto out_free;
+	}
+
+	if (!rdtgrp->kmode) {
+		rdt_last_cmd_puts("Kernel mode is not active on this group\n");
+		ret = -EBUSY;
+		goto out_free;
+	}
+
+	if (is_cpu_list(of))
+		ret = cpulist_parse(buf, newmask);
+	else
+		ret = cpumask_parse(buf, newmask);
+
+	if (ret) {
+		rdt_last_cmd_puts("Bad CPU list/mask\n");
+		goto out_free;
+	}
+
+	/* kernel-mode association is only programmed on online CPUs. */
+	cpumask_andnot(tmpmask, newmask, cpu_online_mask);
+	if (!cpumask_empty(tmpmask)) {
+		rdt_last_cmd_puts("Can only assign online CPUs\n");
+		ret = -EINVAL;
+		goto out_free;
+	}
+
+	kmode_cpus_write(rdtgrp, newmask, tmpmask);
+
+out_free:
+	free_cpumask_var(tmpmask);
+	free_cpumask_var(newmask);
+out_unlock:
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret ?: nbytes;
+}
+
 /*
  * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
  *
@@ -2677,17 +2802,19 @@ static struct rftype res_common_files[] = {
 	},
 	{
 		.name		= "kmode_cpus",
-		.mode		= 0444,
+		.mode		= 0644,
 		.hidden		= true,
 		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_kmode_cpus_write,
 		.seq_show	= rdtgroup_kmode_cpus_show,
 		.fflags		= RFTYPE_BASE,
 	},
 	{
 		.name		= "kmode_cpus_list",
-		.mode		= 0444,
+		.mode		= 0644,
 		.hidden		= true,
 		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_kmode_cpus_write,
 		.seq_show	= rdtgroup_kmode_cpus_show,
 		.flags		= RFTYPE_FLAGS_CPUS_LIST,
 		.fflags		= RFTYPE_BASE,
-- 
2.43.0

[PATCH v5 16/16] fs/resctrl: Add documentation on kernel_mode with example

[Babu Moger]

The kernel-mode interface spans info/kernel_mode, kmode_cpus, and
kmode_cpus_list, but the per-file descriptions do not walk through a
full workflow.

Add an end-to-end example showing how to discover supported policies,
associate kernel-mode allocation and/or monitoring with a group.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v5: Update the example for the collapsed kernel mode model:
    inherit_user and assign_global_enable_per_cpu with ctrl=/mon=
    association options, and <CTRL_MON group>/<MON group>/ paths.

v4: New patch. Added examples in the documentation.
---
 Documentation/filesystems/resctrl.rst | 100 ++++++++++++++++++++++++++
 1 file changed, 100 insertions(+)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 3e886f84315b..81017858231a 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -1873,6 +1873,106 @@ View the llc occupancy snapshot::
   11234000
 
 
+Examples on working with kernel_mode
+====================================
+The kernel-mode interface spans three files:
+
+- ``info/kernel_mode`` - select the global policy and associated group
+- ``kmode_cpus`` / ``kmode_cpus_list`` - view or adjust the CPU scope of the
+  active association (visible only on the associated group)
+
+The walkthrough below covers a typical workflow on a platform with PLZA
+(Privilege Level Zero Association) support:
+
+1. Discover supported policies.
+2. Associate ``assign_global_enable_per_cpu`` with a control group.
+3. Narrow the CPU scope through ``kmode_cpus_list``.
+4. Move the association to a monitor group.
+5. Return to ``inherit_user``.
+
+a. Discover supported kernel mode policies
+::
+
+  # mount -t resctrl resctrl /sys/fs/resctrl/
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  [inherit_user]
+  assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
+``inherit_user`` is the active policy. Kernel work inherits the resource
+allocation and monitoring of the current user-space task. The second line
+shows ``assign_global_enable_per_cpu`` is available; square brackets mark
+the active entry when read back after later steps.
+
+b. Associate kernel mode allocation with a control group
+::
+
+  # mkdir /sys/fs/resctrl/ctrl1
+  # echo "assign_global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//" > \
+        /sys/fs/resctrl/info/kernel_mode
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  inherit_user
+  [assign_global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//]
+
+This selects ``assign_global_enable_per_cpu`` with control assignment
+(``ctrl=assign``) and inherited monitoring (``mon=inherit``). The group
+path uses ``<CTRL_MON group>/<MON group>/`` syntax; a control group alone
+is written as ``ctrl1//``.
+
+When the mode is active:
+
+- ``ctrl1/kmode_cpus`` and ``ctrl1/kmode_cpus_list`` become visible.
+- All currently online CPUs are associated initially.
+- CPUs that come online later are added to the association automatically.
+
+c. Narrow the association to CPUs 0-3
+::
+
+  # echo 0-3 > /sys/fs/resctrl/ctrl1/kmode_cpus_list
+  # cat /sys/fs/resctrl/ctrl1/kmode_cpus
+  f
+  # cat /sys/fs/resctrl/ctrl1/kmode_cpus_list
+  0-3
+
+The write reprograms the association:
+
+- CPUs added to the mask are enabled.
+- CPUs removed from the mask are disabled.
+- Offline CPUs in the mask are rejected.
+- An empty mask disables the association on all currently online CPUs, but
+  later hotplug CPUs are still associated automatically.
+
+d. Move the association to a monitor group
+::
+
+  # mkdir /sys/fs/resctrl/ctrl1/mon_groups/mon1
+  # echo "assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1/mon1/" > \
+        /sys/fs/resctrl/info/kernel_mode
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  inherit_user
+  [assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1/mon1/]
+
+Both control and monitoring are now assigned for kernel work. Changing the
+associated group through ``info/kernel_mode``:
+
+- Resets the CPU scope to all currently online CPUs.
+- Hides ``kmode_cpus`` and ``kmode_cpus_list`` on the previous group.
+- Shows them under ``ctrl1/mon_groups/mon1/``.
+
+e. Return to the default inherit policy
+::
+
+  # echo "inherit_user" > /sys/fs/resctrl/info/kernel_mode
+  # cat /sys/fs/resctrl/info/kernel_mode
+  [inherit_user]
+  assign_global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
+Writing ``inherit_user`` clears the active kernel-mode association and
+hides ``kmode_cpus`` and ``kmode_cpus_list`` because no group is
+associated with kernel mode.
+
 Examples on working with mbm_assign_mode
 ========================================
 
-- 
2.43.0