[RFC PATCH 0/3] arm64: expose CPU prefetch and cache modulation controls

[RFC PATCH 0/3] arm64: expose CPU prefetch and cache modulation controls

[KobaK]

From: Koba Ko <kobak@nvidia.com>

This series adds CONFIG_ARM64_CPUMOD, a default-off arm64 interface for
controlled performance characterization of selected implementation-defined
CPU prefetch and cache-management fields. It exposes a small set of named,
per-CPU attributes under the path below, with writable control fields
range-checked before register state is changed:

  /sys/devices/system/cpu/cpuN/cpumod/

The directory is created for online CPUs whose MIDR matches a recognized
Grace or Vera profile, with common and profile-specific attributes selected
for that CPU. Register accesses execute on the CPU that owns the sysfs
instance, and CPU hotplug creates or removes the per-CPU subtree as the CPU
transitions online or offline. Unsupported CPU profiles are skipped without
blocking module load or CPU hotplug.

The implementation keeps each dynamically allocated kobject in a private
per-CPU pointer, avoiding use of the CPU device's shared driver-data slot.
The documentation defines the ABI, firmware prerequisite, and hotplug
lifecycle. The kselftest validates profile-specific layout, unsupported-CPU
skipping, decimal affected_cpus reporting, value validation, optional
writes, all-online-CPU coverage, and cleanup.

The controls require firmware to permit EL1 access to the relevant
implementation control registers. This is an RFC interface for controlled
evaluation, not a general raw-register or production-default tuning ABI.

Compile and static verification passed. DUT runtime validation was not run.

Koba Ko (3):
  arm64: add CPU prefetch and cache modulation sysfs interface
  Documentation/arch/arm64: document arm_cpumod
  selftests: arm64: add arm_cpumod kselftest

 Documentation/arch/arm64/arm-cpumod.rst       |  89 +++
 Documentation/arch/arm64/index.rst            |   1 +
 arch/arm64/Kconfig                            |  17 +
 arch/arm64/kernel/Makefile                    |   1 +
 arch/arm64/kernel/arm_cpumod.c                | 471 ++++++++++++++++
 arch/arm64/kernel/arm_cpumod_internal.h       | 113 ++++
 tools/testing/selftests/arm64/Makefile        |   2 +-
 .../selftests/arm64/arm_cpumod/Makefile       |   5 +
 .../selftests/arm64/arm_cpumod/arm_cpumod.sh  | 526 ++++++++++++++++++
 9 files changed, 1224 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/arch/arm64/arm-cpumod.rst
 create mode 100644 arch/arm64/kernel/arm_cpumod.c
 create mode 100644 arch/arm64/kernel/arm_cpumod_internal.h
 create mode 100644 tools/testing/selftests/arm64/arm_cpumod/Makefile
 create mode 100755 tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh

-- 
2.43.0

[RFC PATCH 1/3] arm64: add CPU prefetch and cache modulation sysfs interface

[KobaK]

From: Koba Ko <kobak@nvidia.com>

Performance characterization on arm64 systems sometimes needs a way to
inspect and bracket selected implementation-defined prefetch and cache
controls without exposing raw register contents. Add an RFC-only arm64
cpumod interface that presents a small set of named, range-checked
per-CPU sysfs attributes under each CPU device.

Establish the Kconfig and Makefile plumbing, CPU profile detection for
Grace and Vera from the target CPU's MIDR, register-field descriptors,
per-CPU kobject and attribute helpers, locking, and target-CPU read/write
callbacks. Common attributes are exposed for all supported profiles,
while Grace- and Vera-specific fields are exposed only for the matching
profile. Place the option at the end of the top-level Kernel Features
menu because these implementation controls are not tied to an Arm
architecture revision.

Per-CPU cpumod sysfs state follows the CPU lifecycle. Module
initialization populates subtrees for every online CPU while holding
cpus_read_lock(), so initial enumeration and CPU hotplug state
registration are serialized against concurrent topology changes. Offline
CPUs are intentionally skipped during initial setup because target-CPU
MIDR detection requires an IPI-capable online CPU; a later online callback
creates their subtree.

Profile detection reads MIDR through a synchronous callback on the CPU
owning the sysfs subtree and returns nonzero only when that dispatch
fails, so no IPI error can be hidden. After a successful read, retain
CPUMOD_PROFILE_UNKNOWN as the unsupported-profile sentinel. Subtree
creation skips that profile during both initial online enumeration and
later CPUHP online callbacks without blocking module load or CPU hotplug.
Object and sysfs setup failures remain fatal and are propagated.

Keep the dynamically allocated kobject state in a private per-CPU pointer
instead of the CPU device's generic driver-data slot. The online callback
is idempotent when that pointer already records a subtree. The offline
callback clears the private pointer before dropping the kobject reference,
allowing the release callback to retain ownership of the final free.

Register the dynamic CPUHP state with
cpuhp_setup_state_nocalls_cpuslocked(). The nocalls form avoids replaying
startup callbacks for CPUs already initialized by the explicit online
enumeration. On init failure, release the CPU read lock, destroy all
subtrees created so far, and return the error. Module exit removes the
CPUHP state with the matching nocalls helper and tears down all remaining
per-CPU state across possible CPUs.

Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Tushar Dave <tdave@nvidia.com>
Reviewed-by: Kai-Heng Feng <kaihengf@nvidia.com>
Signed-off-by: Koba Ko <kobak@nvidia.com>
---
 arch/arm64/Kconfig                      |  17 +
 arch/arm64/kernel/Makefile              |   1 +
 arch/arm64/kernel/arm_cpumod.c          | 471 ++++++++++++++++++++++++
 arch/arm64/kernel/arm_cpumod_internal.h | 113 ++++++
 4 files changed, 602 insertions(+)
 create mode 100644 arch/arm64/kernel/arm_cpumod.c
 create mode 100644 arch/arm64/kernel/arm_cpumod_internal.h

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b3afe0688919b..5e4ada4884b89 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -2417,6 +2417,23 @@ config ARM64_CONTPTE
 	  bit, for any mappings that meet the size and alignment requirements.
 	  This reduces TLB pressure and improves performance.
 
+config ARM64_CPUMOD
+	tristate "Arm CPU prefetch modulation sysfs controls"
+	help
+	  Expose selected CPU implementation control register fields through
+	  per-CPU sysfs attributes for performance characterization.
+
+	  The exposed controls allow hardware prefetch and cache-management
+	  policy fields to be inspected and adjusted on CPUs where firmware
+	  permits EL1 access to the relevant implementation control registers.
+	  Systems where firmware traps or blocks these writes cannot use this
+	  interface.
+
+	  This interface is intended for controlled performance evaluation and
+	  should not be enabled by default on production systems.
+
+	  If unsure, say N.
+
 endmenu # "Kernel Features"
 
 menu "Boot options"
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index d2690c3ec5288..f180a52284e7d 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -69,6 +69,7 @@ obj-$(CONFIG_VMCORE_INFO)		+= vmcore_info.o
 obj-$(CONFIG_ARM_SDE_INTERFACE)		+= sdei.o
 obj-$(CONFIG_ARM64_PTR_AUTH)		+= pointer_auth.o
 obj-$(CONFIG_ARM64_MPAM)		+= mpam.o
+obj-$(CONFIG_ARM64_CPUMOD)		+= arm_cpumod.o
 obj-$(CONFIG_ARM64_MTE)			+= mte.o
 obj-y					+= vdso-wrap.o
 obj-$(CONFIG_COMPAT_VDSO)		+= vdso32-wrap.o
diff --git a/arch/arm64/kernel/arm_cpumod.c b/arch/arm64/kernel/arm_cpumod.c
new file mode 100644
index 0000000000000..9e64f2c8e7810
--- /dev/null
+++ b/arch/arm64/kernel/arm_cpumod.c
@@ -0,0 +1,471 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ */
+
+#define pr_fmt(fmt) "cpumod: " fmt
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/sysfs.h>
+#include <linux/device.h>
+#include <linux/cpu.h>
+#include <linux/mutex.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+#include <asm/cputype.h>
+#include <asm/sysreg.h>
+#include <linux/bits.h>
+#include "arm_cpumod_internal.h"
+
+#define DRIVER_DESC "CPU Modulation Kernel Module"
+
+/* Disables hardware prefetching.
+ * 0b0 Enables hardware prefetching. This is the default value.
+ * 0b1 Disables hardware prefetching.
+ */
+#define CPUECTLR_PF_DIS				(15)
+#define CPUECTLR_PF_DIS_MSK			BIT_ULL(15)
+
+#define CPUECTLR_CMC_WAYS			(61)
+#define CPUECTLR_CMC_WAYS_MSK			GENMASK_ULL(63, 61)
+
+#define CPUECTLR2_PF_MODE			(11)
+#define CPUECTLR2_PF_MODE_MSK			GENMASK_ULL(14, 11)
+
+#define CPUECTLR2_CBUSY_FILTER_WINDOW		(9)
+#define CPUECTLR2_CBUSY_FILTER_WINDOW_MSK	GENMASK_ULL(10, 9)
+
+#define CPUECTLR2_CBUSY_FILTER_THRESHOLD	(7)
+#define CPUECTLR2_CBUSY_FILTER_THRESHOLD_MSK	GENMASK_ULL(8, 7)
+
+static enum cpuhp_state cpumod_hp_state;
+static DEFINE_PER_CPU(struct cpumod_subsys *, cpumod_subsys);
+
+struct cpumod_attr_call {
+	struct cpumod_subsys *subsys;
+	struct cpumod_attr *attr;
+	u64 value;
+};
+
+struct cpumod_midr_call {
+	u32 midr;
+};
+
+static const char *cpumod_profile_name(enum cpumod_profile profile)
+{
+	switch (profile) {
+	case CPUMOD_PROFILE_GRACE:
+		return "Grace";
+	case CPUMOD_PROFILE_VERA:
+		return "Vera";
+	default:
+		return "unknown";
+	}
+}
+
+/* IMP defined CPU Extended Control Register */
+#define IMP_CPUECTLR_EL1	sys_reg(3, 0, 15, 1, 4)
+#define IMP_CPUECTLR2_EL1	sys_reg(3, 0, 15, 1, 5)
+
+static enum cpumod_profile cpumod_profile_from_midr(u32 midr)
+{
+	switch (midr & MIDR_CPU_MODEL_MASK) {
+	case MIDR_NEOVERSE_V2:
+		return CPUMOD_PROFILE_GRACE;
+	case MIDR_NVIDIA_OLYMPUS:
+		return CPUMOD_PROFILE_VERA;
+	default:
+		return CPUMOD_PROFILE_UNKNOWN;
+	}
+}
+
+static void cpumod_read_midr_remote(void *info)
+{
+	struct cpumod_midr_call *call = info;
+
+	call->midr = read_cpuid_id();
+}
+
+static int cpumod_detect_profile(struct cpumod_subsys *subsys)
+{
+	struct cpumod_midr_call call;
+	int ret;
+
+	ret = smp_call_function_single(subsys->cpu, cpumod_read_midr_remote,
+				       &call, 1);
+	if (ret) {
+		pr_err("Failed to read CPU%u MIDR: %d\n", subsys->cpu, ret);
+		return ret;
+	}
+
+	subsys->profile = cpumod_profile_from_midr(call.midr);
+	if (subsys->profile == CPUMOD_PROFILE_UNKNOWN) {
+		pr_debug("Unsupported CPU%u MIDR 0x%08x\n", subsys->cpu,
+			 call.midr);
+	}
+
+	return 0;
+}
+
+static u64 cpumod_read_reg(enum cpumod_reg_id reg)
+{
+	switch (reg) {
+	case CPUMOD_REG_CPUECTLR:
+		return read_sysreg_s(IMP_CPUECTLR_EL1);
+	case CPUMOD_REG_CPUECTLR2:
+		return read_sysreg_s(IMP_CPUECTLR2_EL1);
+	default:
+		return 0;
+	}
+}
+
+static void cpumod_write_reg(enum cpumod_reg_id reg, u64 value)
+{
+	switch (reg) {
+	case CPUMOD_REG_CPUECTLR:
+		write_sysreg_s(value, IMP_CPUECTLR_EL1);
+		/* The remote callback executes this barrier on the target CPU. */
+		isb();
+		break;
+	case CPUMOD_REG_CPUECTLR2:
+		write_sysreg_s(value, IMP_CPUECTLR2_EL1);
+		/* The remote callback executes this barrier on the target CPU. */
+		isb();
+		break;
+	default:
+		return;
+	}
+}
+
+static u64 cpumod_attr_read_reg(const struct cpumod_attr *attr)
+{
+	return cpumod_attr_unpack_value(attr, cpumod_read_reg(attr->reg));
+}
+
+static void cpumod_attr_write_reg(const struct cpumod_attr *attr, u64 value)
+{
+	u64 reg = cpumod_read_reg(attr->reg);
+	u64 field_value = cpumod_attr_pack_value(attr, value);
+
+	cpumod_write_reg(attr->reg, (reg & ~attr->field_mask) | field_value);
+}
+
+static void cpumod_attr_read_remote(void *info)
+{
+	struct cpumod_attr_call *call = info;
+	u64 *value = cpumod_attr_value_ptr(call->subsys, call->attr);
+
+	*value = cpumod_attr_read_reg(call->attr);
+}
+
+static void cpumod_attr_write_remote(void *info)
+{
+	struct cpumod_attr_call *call = info;
+	struct cpumod_subsys *subsys = call->subsys;
+	struct cpumod_attr *attr = call->attr;
+	u64 readback;
+
+	cpumod_attr_write_reg(attr, call->value);
+	readback = cpumod_attr_read_reg(attr);
+	pr_debug("cpu%u %s: %llx\n",
+		 subsys->cpu,
+		 attr->debug_name,
+		 (unsigned long long)readback);
+}
+
+static ssize_t cpumod_attr_show(struct kobject *kobj,
+				struct kobj_attribute *kattr, char *buf)
+{
+	struct cpumod_subsys *subsys = cpumod_subsys_from_kobj(kobj);
+	struct cpumod_attr *attr = cpumod_attr_from_kobj_attr(kattr);
+	struct cpumod_attr_call call = {
+		.subsys = subsys,
+		.attr = attr,
+	};
+	int ret;
+
+	guard(mutex)(&subsys->lock);
+	ret = smp_call_function_single(subsys->cpu,
+				       cpumod_attr_read_remote, &call, 1);
+	if (ret)
+		return ret;
+
+	ret = sysfs_emit(buf, "%llu\n",
+			 (unsigned long long)*cpumod_attr_value_ptr(subsys, attr));
+
+	return ret;
+}
+
+static ssize_t cpumod_attr_store(struct kobject *kobj,
+				 struct kobj_attribute *kattr,
+				 const char *buf, size_t count)
+{
+	struct cpumod_subsys *subsys = cpumod_subsys_from_kobj(kobj);
+	struct cpumod_attr *attr = cpumod_attr_from_kobj_attr(kattr);
+	struct cpumod_attr_call call = {
+		.subsys = subsys,
+		.attr = attr,
+	};
+	unsigned int val;
+	int ret;
+
+	ret = kstrtouint(buf, 10, &val);
+	if (ret)
+		return -EINVAL;
+	if (!cpumod_attr_value_valid(attr, val))
+		return -EINVAL;
+
+	call.value = cpumod_attr_mask_value(attr, val);
+
+	guard(mutex)(&subsys->lock);
+	ret = smp_call_function_single(subsys->cpu,
+				       cpumod_attr_write_remote, &call, 1);
+	if (ret)
+		return ret;
+	cpumod_attr_set_value(subsys, attr, call.value);
+
+	return count;
+}
+
+static ssize_t affected_cpus_show(struct kobject *kobj,
+				  struct kobj_attribute *attr, char *buf)
+{
+	struct cpumod_subsys *subsys = cpumod_subsys_from_kobj(kobj);
+
+	return sysfs_emit(buf, "%u\n", subsys->cpu);
+}
+
+static struct kobj_attribute affected_cpus_attr = __ATTR_RO(affected_cpus);
+
+static umode_t cpumod_attr_is_visible(struct kobject *kobj,
+				      struct attribute *attr, int unused)
+{
+	struct cpumod_subsys *subsys = cpumod_subsys_from_kobj(kobj);
+	struct cpumod_attr *cpumod_attr;
+
+	if (attr == &affected_cpus_attr.attr)
+		return attr->mode;
+
+	cpumod_attr = container_of(attr, struct cpumod_attr, kattr.attr);
+	if (!cpumod_attr_visible_for_profile(cpumod_attr, subsys->profile))
+		return 0;
+
+	return attr->mode;
+}
+
+#define CPUMOD_ATTR_RW(_name, _field, _reg, _shift, _field_mask,	\
+		       _max_value, _debug_name,			\
+		       _visible_profiles)				\
+static struct cpumod_attr _name##_attr = {				\
+	.kattr = __ATTR(_name, 0644, cpumod_attr_show, cpumod_attr_store), \
+	.value_offset = offsetof(struct cpumod_subsys, _field),	\
+	.reg = _reg,						\
+	.shift = _shift,					\
+	.field_mask = _field_mask,				\
+	.max_value = _max_value,				\
+	.debug_name = _debug_name,				\
+	.visible_profiles = _visible_profiles,			\
+}
+
+CPUMOD_ATTR_RW(pf_dis, pf_dis, CPUMOD_REG_CPUECTLR, CPUECTLR_PF_DIS,
+	       CPUECTLR_PF_DIS_MSK, 1, "PF_DIS",
+	       CPUMOD_PROFILE_VISIBLE_ALL);
+CPUMOD_ATTR_RW(pf_mode, pf_mode, CPUMOD_REG_CPUECTLR2, CPUECTLR2_PF_MODE,
+	       CPUECTLR2_PF_MODE_MSK, 9, "PF_MODE",
+	       CPUMOD_PROFILE_VISIBLE_ALL);
+CPUMOD_ATTR_RW(cbusy_filter_threshold, cbusy_filter_threshold,
+	       CPUMOD_REG_CPUECTLR2, CPUECTLR2_CBUSY_FILTER_THRESHOLD,
+	       CPUECTLR2_CBUSY_FILTER_THRESHOLD_MSK, 3,
+	       "CBUSY_FILTER_THRESHOLD",
+	       CPUMOD_PROFILE_VISIBLE_GRACE);
+CPUMOD_ATTR_RW(cbusy_filter_window, cbusy_filter_window,
+	       CPUMOD_REG_CPUECTLR2, CPUECTLR2_CBUSY_FILTER_WINDOW,
+	       CPUECTLR2_CBUSY_FILTER_WINDOW_MSK, 3,
+	       "CBUSY_FILTER_WINDOW",
+	       CPUMOD_PROFILE_VISIBLE_GRACE);
+CPUMOD_ATTR_RW(cmc_min_ways, cmc_min_ways, CPUMOD_REG_CPUECTLR,
+	       CPUECTLR_CMC_WAYS, CPUECTLR_CMC_WAYS_MSK, 7,
+	       "CMC_MIN_WAYS", CPUMOD_PROFILE_VISIBLE_GRACE);
+CPUMOD_ATTR_RW(l2spr_cmc_max_ways, l2spr_cmc_max_ways, CPUMOD_REG_CPUECTLR,
+	       CPUECTLR_CMC_WAYS, CPUECTLR_CMC_WAYS_MSK, 7,
+	       "L2SPR_CMC_MAX_WAYS",
+	       CPUMOD_PROFILE_VISIBLE_VERA);
+
+#undef CPUMOD_ATTR_RW
+
+static struct attribute *cpumod_attrs[] = {
+	&affected_cpus_attr.attr,
+	&pf_dis_attr.kattr.attr,
+	&pf_mode_attr.kattr.attr,
+	&cbusy_filter_threshold_attr.kattr.attr,
+	&cbusy_filter_window_attr.kattr.attr,
+	&cmc_min_ways_attr.kattr.attr,
+	&l2spr_cmc_max_ways_attr.kattr.attr,
+	NULL,
+};
+
+static const struct attribute_group cpumod_group = {
+	.attrs = cpumod_attrs,
+	.is_visible = cpumod_attr_is_visible,
+};
+
+static const struct attribute_group *cpumod_groups[] = {
+	&cpumod_group,
+	NULL,
+};
+
+static void cpumod_release(struct kobject *kobj)
+{
+	kfree(cpumod_subsys_from_kobj(kobj));
+}
+
+static const struct kobj_type cpumod_ktype = {
+	.release = cpumod_release,
+	.sysfs_ops = &kobj_sysfs_ops,
+	.default_groups = cpumod_groups,
+};
+
+static int cpumod_create_subsys(struct device *cpu_dev, unsigned int cpu)
+{
+	struct cpumod_subsys *subsys;
+	int ret;
+
+	if (per_cpu(cpumod_subsys, cpu))
+		return 0;
+
+	subsys = kzalloc_obj(*subsys, GFP_KERNEL);
+	if (!subsys)
+		return -ENOMEM;
+
+	subsys->cpu = cpu;
+	mutex_init(&subsys->lock);
+	ret = cpumod_detect_profile(subsys);
+	if (ret) {
+		kfree(subsys);
+		return ret;
+	}
+	if (subsys->profile == CPUMOD_PROFILE_UNKNOWN) {
+		kfree(subsys);
+		return 0;
+	}
+
+	pr_debug("CPU%u detected %s profile\n", cpu,
+		 cpumod_profile_name(subsys->profile));
+
+	ret = kobject_init_and_add(&subsys->kobj, &cpumod_ktype,
+				   &cpu_dev->kobj, "cpumod");
+	if (ret) {
+		pr_err("Failed to create cpumod sysfs subtree for CPU%u: %d\n",
+		       cpu, ret);
+		kobject_put(&subsys->kobj);
+		return ret;
+	}
+
+	per_cpu(cpumod_subsys, cpu) = subsys;
+
+	return 0;
+}
+
+static void cpumod_destroy_subsys(unsigned int cpu)
+{
+	struct cpumod_subsys *subsys = per_cpu(cpumod_subsys, cpu);
+
+	if (!subsys)
+		return;
+
+	per_cpu(cpumod_subsys, cpu) = NULL;
+	kobject_put(&subsys->kobj);
+}
+
+static void cpumod_destroy_all_subsys(void)
+{
+	unsigned int cpu;
+
+	for_each_possible_cpu(cpu)
+		cpumod_destroy_subsys(cpu);
+}
+
+static int cpumod_online_cpu(unsigned int cpu)
+{
+	struct device *cpu_dev = get_cpu_device(cpu);
+
+	if (!cpu_dev) {
+		pr_err("Failed to get CPU%u device\n", cpu);
+		return -ENODEV;
+	}
+
+	return cpumod_create_subsys(cpu_dev, cpu);
+}
+
+static int cpumod_offline_cpu(unsigned int cpu)
+{
+	cpumod_destroy_subsys(cpu);
+
+	return 0;
+}
+
+static int __init cpumod_init(void)
+{
+	unsigned int cpu;
+	int ret;
+
+	cpus_read_lock();
+	for_each_online_cpu(cpu) {
+		struct device *cpu_dev = get_cpu_device(cpu);
+
+		if (!cpu_dev) {
+			pr_err("Failed to get CPU%u device\n", cpu);
+			ret = -ENODEV;
+			goto err_unlock;
+		}
+
+		ret = cpumod_create_subsys(cpu_dev, cpu);
+		if (ret)
+			goto err_unlock;
+	}
+
+	ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ONLINE_DYN,
+						   "arm64/cpumod:online",
+						   cpumod_online_cpu,
+						   cpumod_offline_cpu);
+	cpus_read_unlock();
+	if (ret < 0)
+		goto err_destroy;
+
+	cpumod_hp_state = ret;
+	pr_info("module loaded\n");
+
+	return 0;
+
+err_unlock:
+	cpus_read_unlock();
+err_destroy:
+	cpumod_destroy_all_subsys();
+	return ret;
+}
+
+static void __exit cpumod_exit(void)
+{
+	cpuhp_remove_state_nocalls(cpumod_hp_state);
+	cpumod_destroy_all_subsys();
+	pr_info("module unloaded\n");
+}
+
+module_init(cpumod_init);
+module_exit(cpumod_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_VERSION("1.0");
+MODULE_AUTHOR("kobak@nvidia.com");
+MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/arch/arm64/kernel/arm_cpumod_internal.h b/arch/arm64/kernel/arm_cpumod_internal.h
new file mode 100644
index 0000000000000..04bf345dd383a
--- /dev/null
+++ b/arch/arm64/kernel/arm_cpumod_internal.h
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Internal arm_cpumod data structures and helpers.
+ */
+
+#ifndef CPUMOD_INTERNAL_H
+#define CPUMOD_INTERNAL_H
+
+#include <linux/bits.h>
+#include <linux/kernel.h>
+#include <linux/kobject.h>
+#include <linux/mutex.h>
+#include <linux/sysfs.h>
+#include <linux/types.h>
+
+enum cpumod_profile {
+	CPUMOD_PROFILE_GRACE,
+	CPUMOD_PROFILE_VERA,
+	CPUMOD_PROFILE_UNKNOWN,
+};
+
+enum cpumod_reg_id {
+	CPUMOD_REG_CPUECTLR,
+	CPUMOD_REG_CPUECTLR2,
+};
+
+#define CPUMOD_PROFILE_VISIBLE_GRACE	BIT(0)
+#define CPUMOD_PROFILE_VISIBLE_VERA	BIT(1)
+#define CPUMOD_PROFILE_VISIBLE_ALL	(CPUMOD_PROFILE_VISIBLE_GRACE | \
+					 CPUMOD_PROFILE_VISIBLE_VERA)
+
+struct cpumod_subsys {
+	/* Per-CPU instance lock: serializes sysfs read/write for this CPU. */
+	struct mutex lock;
+	struct kobject kobj;
+	unsigned int cpu;
+	enum cpumod_profile profile;
+	u64 pf_dis;
+	u64 pf_mode;
+	u64 cbusy_filter_threshold;
+	u64 cbusy_filter_window;
+	u64 cmc_min_ways;
+	u64 l2spr_cmc_max_ways;
+};
+
+struct cpumod_attr {
+	struct kobj_attribute kattr;
+	size_t value_offset;
+	enum cpumod_reg_id reg;
+	u8 shift;
+	u64 field_mask;
+	u64 max_value;
+	const char *debug_name;
+	u8 visible_profiles;
+};
+
+static inline struct cpumod_subsys *cpumod_subsys_from_kobj(struct kobject *kobj)
+{
+	return container_of(kobj, struct cpumod_subsys, kobj);
+}
+
+static inline struct cpumod_attr *cpumod_attr_from_kobj_attr(struct kobj_attribute *kattr)
+{
+	return container_of(kattr, struct cpumod_attr, kattr);
+}
+
+static inline u64 *cpumod_attr_value_ptr(struct cpumod_subsys *subsys,
+					 const struct cpumod_attr *attr)
+{
+	return (u64 *)((char *)subsys + attr->value_offset);
+}
+
+static inline u64 cpumod_attr_mask_value(const struct cpumod_attr *attr, u64 value)
+{
+	return value & (attr->field_mask >> attr->shift);
+}
+
+static inline bool cpumod_attr_value_valid(const struct cpumod_attr *attr, u64 value)
+{
+	return value <= attr->max_value;
+}
+
+static inline u64 cpumod_attr_unpack_value(const struct cpumod_attr *attr, u64 reg)
+{
+	return (reg & attr->field_mask) >> attr->shift;
+}
+
+static inline u64 cpumod_attr_pack_value(const struct cpumod_attr *attr, u64 value)
+{
+	return (value << attr->shift) & attr->field_mask;
+}
+
+static inline void cpumod_attr_set_value(struct cpumod_subsys *subsys,
+					 const struct cpumod_attr *attr,
+					 u64 value)
+{
+	*cpumod_attr_value_ptr(subsys, attr) = cpumod_attr_mask_value(attr, value);
+}
+
+static inline bool cpumod_attr_visible_for_profile(const struct cpumod_attr *attr,
+						   enum cpumod_profile profile)
+{
+	switch (profile) {
+	case CPUMOD_PROFILE_GRACE:
+		return attr->visible_profiles & CPUMOD_PROFILE_VISIBLE_GRACE;
+	case CPUMOD_PROFILE_VERA:
+		return attr->visible_profiles & CPUMOD_PROFILE_VISIBLE_VERA;
+	default:
+		return false;
+	}
+}
+
+#endif /* CPUMOD_INTERNAL_H */
-- 
2.43.0

[RFC PATCH 2/3] Documentation/arch/arm64: document arm_cpumod

[KobaK]

From: Koba Ko <kobak@nvidia.com>

Document the ABI intent and scope for the new arm64 cpumod interface.
The sysfs directory is per-CPU and is created only for recognized Grace
and Vera profiles:

  /sys/devices/system/cpu/cpuN/cpumod/

CPUs with unsupported MIDRs are skipped without blocking module load or
CPU hotplug and do not receive a cpumod directory.

Describe the common attributes, the Grace- and Vera-specific attributes,
and the accepted range for each writable field. Out-of-range writes are
documented as failing before either cached sysfs state or target CPU
register state is changed.

Document the CPU hotplug lifecycle explicitly: the cpumod directory is
present only while a supported CPU is online, is removed when that CPU
goes offline, and is recreated when it returns online. Reads and writes
therefore require an online target CPU.

Also document the firmware prerequisite: EL1 must be allowed to access
the relevant implementation control registers. The interface remains an
RFC for controlled performance characterization, is not a general
production tuning ABI, and should stay disabled by default on production
systems.

Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Tushar Dave <tdave@nvidia.com>
Reviewed-by: Kai-Heng Feng <kaihengf@nvidia.com>
Signed-off-by: Koba Ko <kobak@nvidia.com>
---
 Documentation/arch/arm64/arm-cpumod.rst | 89 +++++++++++++++++++++++++
 Documentation/arch/arm64/index.rst      |  1 +
 2 files changed, 90 insertions(+)
 create mode 100644 Documentation/arch/arm64/arm-cpumod.rst

diff --git a/Documentation/arch/arm64/arm-cpumod.rst b/Documentation/arch/arm64/arm-cpumod.rst
new file mode 100644
index 0000000000000..7dbe2cd5404a2
--- /dev/null
+++ b/Documentation/arch/arm64/arm-cpumod.rst
@@ -0,0 +1,89 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+====================================
+Arm CPU prefetch modulation controls
+====================================
+
+``ARM64_CPUMOD`` exposes selected CPU implementation control register fields
+through per-CPU sysfs attributes under::
+
+  /sys/devices/system/cpu/cpuN/cpumod/
+
+The ``cpumod`` directory is created only when the CPU MIDR matches a recognized
+Grace or Vera profile. CPUs with unsupported MIDRs are skipped without blocking
+module load or CPU hotplug and do not have a ``cpumod`` directory.
+
+The interface is intended for controlled performance characterization and
+evaluation. It is not intended as a general production tuning ABI, and the
+configuration should remain disabled by default on production systems.
+
+Configuration and placement
+===========================
+
+The controls are built when ``CONFIG_ARM64_CPUMOD`` is enabled. The code lives
+under ``arch/arm64/kernel/`` because the exposed state is CPU implementation
+control state accessed by the arm64 kernel, similar in placement to other
+architecture CPU-facing helpers.
+
+The ABI remains separate from the existing arm64 ``cpu*/regs`` sysfs files:
+``cpumod`` exposes a small set of named, range-checked control fields rather
+than a general raw register dump.
+
+CPU hotplug
+===========
+
+A ``cpumod`` directory is created only for online CPUs with a supported
+profile. The directory is removed when a CPU goes offline and recreated when
+the CPU comes back online, again only when its MIDR matches a supported
+profile. Reads and writes require the target CPU to be online.
+
+Firmware requirement
+====================
+
+The controls require firmware to permit EL1 reads and writes to the relevant
+CPU implementation control registers. On systems where firmware traps or
+blocks those accesses, the interface cannot be used.
+
+Current ABI
+===========
+
+Common attributes:
+
+``affected_cpus``
+  Read-only decimal CPU identifier for the sysfs instance.
+
+``pf_dis``
+  Hardware prefetch disable control. Valid values are ``0`` and ``1``.
+
+``pf_mode``
+  Hardware prefetch aggressiveness mode. Valid values are ``0`` through ``9``.
+  Values ``10`` through ``15`` are reserved and rejected.
+
+Grace-only attributes:
+
+``cbusy_filter_threshold``
+  Valid values are ``0`` through ``3``.
+
+``cbusy_filter_window``
+  Valid values are ``0`` through ``3``.
+
+``cmc_min_ways``
+  Valid values are ``0`` through ``7``.
+
+Vera-only attributes:
+
+``l2spr_cmc_max_ways``
+  Valid values are ``0`` through ``7``.
+
+Invalid writes
+==============
+
+Writes outside the documented range fail with ``-EINVAL`` before the cached
+sysfs state or remote CPU register state is updated.
+
+Open register inventory
+=======================
+
+The first RFC intentionally exposes only the currently validated subset. A
+complete documented-field inventory, including L2CDP, should be reviewed before
+adding further ABI nodes.
diff --git a/Documentation/arch/arm64/index.rst b/Documentation/arch/arm64/index.rst
index 98052b4ef4a1e..9e085ce55d9c4 100644
--- a/Documentation/arch/arm64/index.rst
+++ b/Documentation/arch/arm64/index.rst
@@ -8,6 +8,7 @@ ARM64 Architecture
     :maxdepth: 1
 
     acpi_object_usage
+    arm-cpumod
     amu
     arm-acpi
     arm-cca
-- 
2.43.0

[RFC PATCH 3/3] selftests: arm64: add arm_cpumod kselftest

[KobaK]

From: Koba Ko <kobak@nvidia.com>

Add a kselftest for the arm_cpumod sysfs ABI. The test loads or reuses the
module and selects either one target CPU or a snapshot of all online
CPUs. In single-CPU mode, skip an offline target before module
validation. Observe the per-CPU online attribute first. If it is absent,
immediately require the CPU directory to still exist before treating the
CPU as non-hotpluggable and online. If the attribute exists, it must be
readable and contain 1.

After module load, recheck the online state before classifying a selected
CPU without a cpumod directory as an unsupported profile. Fail if the CPU
went offline or disappeared, log and skip CPUs that remain online without
a directory, and require at least one selected CPU to expose a supported
Grace or Vera profile. Return KSFT_SKIP if none do.

For supported CPUs, recognize the exposed Grace or Vera profile-specific
layout and validate the matching attributes. Only
ARM_CPUMOD_EXPECT_PROFILE supplies an external profile expectation.

Verify read paths by checking that attributes return decimal values and
that affected_cpus identifies the current CPU as a decimal CPU number.
Optional valid-write testing writes each current value back. Optional
invalid-write testing checks selected out-of-range values and confirms
that the prior value remains unchanged.

Both write paths are opt-in. Valid writeback can alter
implementation-defined CPU controls; invalid-write checks exercise
rejection through the same firmware-sensitive interface, but are expected
not to update state. Cover all-online-CPU mode through
/sys/devices/system/cpu/online and, when the test loaded the module,
verify cleanup by removing it and checking that the selected CPU subtrees
disappear. The test does not offline or online CPUs.

Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Tushar Dave <tdave@nvidia.com>
Reviewed-by: Kai-Heng Feng <kaihengf@nvidia.com>
Signed-off-by: Koba Ko <kobak@nvidia.com>
---
 tools/testing/selftests/arm64/Makefile        |   2 +-
 .../selftests/arm64/arm_cpumod/Makefile       |   5 +
 .../selftests/arm64/arm_cpumod/arm_cpumod.sh  | 526 ++++++++++++++++++
 3 files changed, 532 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/arm_cpumod/Makefile
 create mode 100755 tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh

diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile
index e456f3b62fa13..0873be5af0369 100644
--- a/tools/testing/selftests/arm64/Makefile
+++ b/tools/testing/selftests/arm64/Makefile
@@ -4,7 +4,7 @@
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),aarch64 arm64))
-ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs
+	ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs arm_cpumod
 else
 ARM64_SUBTARGETS :=
 endif
diff --git a/tools/testing/selftests/arm64/arm_cpumod/Makefile b/tools/testing/selftests/arm64/arm_cpumod/Makefile
new file mode 100644
index 0000000000000..db0341cdbc621
--- /dev/null
+++ b/tools/testing/selftests/arm64/arm_cpumod/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+TEST_PROGS := arm_cpumod.sh
+
+include ../../lib.mk
diff --git a/tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh b/tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh
new file mode 100755
index 0000000000000..51c620a74c7d4
--- /dev/null
+++ b/tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh
@@ -0,0 +1,526 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+
+readonly KSFT_SKIP=4
+readonly MODULE_NAME="arm_cpumod"
+readonly CPU_ID="${ARM_CPUMOD_CPU:-0}"
+readonly MODULE_PATH="${ARM_CPUMOD_KO:-}"
+readonly ENABLE_WRITES="${ARM_CPUMOD_ENABLE_WRITES:-0}"
+readonly ENABLE_INVALID_WRITES="${ARM_CPUMOD_ENABLE_INVALID_WRITES:-0}"
+readonly ALL_CPUS="${ARM_CPUMOD_ALL_CPUS:-0}"
+readonly EXPECT_PROFILE="${ARM_CPUMOD_EXPECT_PROFILE:-auto}"
+readonly CPU_ONLINE_PATH="/sys/devices/system/cpu/online"
+readonly COMMON_ATTRS=(affected_cpus pf_dis pf_mode)
+readonly COMMON_RW_ATTRS=(pf_dis pf_mode)
+readonly GRACE_ATTRS=(cbusy_filter_threshold cbusy_filter_window cmc_min_ways)
+readonly GRACE_RW_ATTRS=(cbusy_filter_threshold cbusy_filter_window cmc_min_ways)
+readonly VERA_ATTRS=(l2spr_cmc_max_ways)
+readonly VERA_RW_ATTRS=(l2spr_cmc_max_ways)
+
+loaded_by_test=0
+reused_loaded_module=0
+detected_profile=""
+current_cpu="${CPU_ID}"
+verified_cpus=0
+CPU_LIST=()
+
+log()
+{
+	echo "${MODULE_NAME} selftest: $*"
+}
+
+skip()
+{
+	log "$*"
+	exit "${KSFT_SKIP}"
+}
+
+fail()
+{
+	log "$*"
+	exit 1
+}
+
+cpu_dir()
+{
+	local cpu="$1"
+
+	printf '/sys/devices/system/cpu/cpu%s' "${cpu}"
+}
+
+cpu_is_online()
+{
+	local cpu="$1"
+	local dir="$(cpu_dir "${cpu}")"
+	local online="${dir}/online"
+	local value
+
+	if [ ! -e "${online}" ]; then
+		[ -d "${dir}" ] || return 1
+		return 0
+	fi
+
+	[ -r "${online}" ] || return 1
+	value=$(cat "${online}") || return 1
+	[ "${value}" = "1" ]
+}
+
+cpumod_dir()
+{
+	local cpu="$1"
+
+	printf '%s/cpumod' "$(cpu_dir "${cpu}")"
+}
+
+current_cpumod_dir()
+{
+	cpumod_dir "${current_cpu}"
+}
+
+cleanup()
+{
+	if [ "${loaded_by_test}" -eq 1 ]; then
+		rmmod "${MODULE_NAME}" >/dev/null 2>&1 || true
+	fi
+}
+trap cleanup EXIT
+
+require_cmd()
+{
+	command -v "$1" >/dev/null 2>&1 || skip "missing required command: $1"
+}
+
+module_loaded()
+{
+	grep -q "^${MODULE_NAME} " /proc/modules
+}
+
+wait_for_path_state()
+{
+	local expect_present="$1"
+	local target="$2"
+	local i
+
+	for i in $(seq 1 50); do
+		if [ "${expect_present}" = "present" ]; then
+			[ -e "${target}" ] && return 0
+		else
+			[ ! -e "${target}" ] && return 0
+		fi
+		sleep 0.1
+	done
+
+	return 1
+}
+
+read_attr()
+{
+	local attr="$1"
+	local file="$(current_cpumod_dir)/${attr}"
+	local value
+
+	value=$(cat "${file}") || fail "failed to read ${file}"
+	printf '%s' "${value}"
+}
+
+attr_exists()
+{
+	local attr="$1"
+
+	[ -f "$(current_cpumod_dir)/${attr}" ]
+}
+
+require_attr_present()
+{
+	local attr="$1"
+
+	attr_exists "${attr}" || fail "missing sysfs attribute $(current_cpumod_dir)/${attr}"
+}
+
+require_attr_absent()
+{
+	local attr="$1"
+
+	attr_exists "${attr}" && fail "unexpected sysfs attribute $(current_cpumod_dir)/${attr}"
+}
+
+check_decimal_attr()
+{
+	local attr="$1"
+	local value
+
+	value=$(read_attr "${attr}")
+	case "${value}" in
+	''|*[!0-9]*)
+		fail "${attr} returned non-decimal value: ${value}"
+		;;
+	esac
+}
+
+expand_online_cpus()
+{
+	local cpu_list="$1"
+	local part
+	local start
+	local start_num
+	local end
+	local end_num
+	local cpu
+	local parts
+
+	CPU_LIST=()
+	IFS=, read -ra parts <<< "${cpu_list}"
+	for part in "${parts[@]}"; do
+		part="${part//[[:space:]]/}"
+		[ -n "${part}" ] || fail "empty CPU range in ${cpu_list}"
+
+		if [[ "${part}" == *-* ]]; then
+			start="${part%-*}"
+			end="${part#*-}"
+			[[ "${start}" =~ ^[0-9]+$ ]] || fail "invalid CPU range start: ${part}"
+			[[ "${end}" =~ ^[0-9]+$ ]] || fail "invalid CPU range end: ${part}"
+			start_num=$((10#${start}))
+			end_num=$((10#${end}))
+			(( start_num <= end_num )) || fail "invalid descending CPU range: ${part}"
+
+			for ((cpu = start_num; cpu <= end_num; cpu++)); do
+				CPU_LIST+=("${cpu}")
+			done
+		else
+			[[ "${part}" =~ ^[0-9]+$ ]] || fail "invalid CPU entry: ${part}"
+			CPU_LIST+=("$((10#${part}))")
+		fi
+	done
+
+	[ "${#CPU_LIST[@]}" -gt 0 ] || fail "no online CPUs parsed from ${cpu_list}"
+}
+
+select_cpus()
+{
+	if [ "${ALL_CPUS}" = "1" ]; then
+		[ -r "${CPU_ONLINE_PATH}" ] || skip "missing ${CPU_ONLINE_PATH}"
+		expand_online_cpus "$(cat "${CPU_ONLINE_PATH}")"
+	else
+		cpu_is_online "${CPU_ID}" || \
+			skip "cpu${CPU_ID} is offline or its online state is unreadable"
+		CPU_LIST=("${CPU_ID}")
+	fi
+}
+
+detect_profile()
+{
+	local has_grace
+	local has_vera
+	local attr
+	local i
+	local dir="$(current_cpumod_dir)"
+
+	for i in $(seq 1 50); do
+		has_grace=0
+		has_vera=0
+
+		for attr in "${GRACE_ATTRS[@]}"; do
+			if attr_exists "${attr}"; then
+				has_grace=1
+				break
+			fi
+		done
+
+		for attr in "${VERA_ATTRS[@]}"; do
+			if attr_exists "${attr}"; then
+				has_vera=1
+				break
+			fi
+		done
+
+		if [ "${has_grace}" -eq 1 ] && [ "${has_vera}" -eq 1 ]; then
+			fail "mixed Grace/Vera-specific attributes under ${dir}"
+		fi
+
+		if [ "${has_grace}" -eq 1 ]; then
+			detected_profile="grace"
+			break
+		fi
+
+		if [ "${has_vera}" -eq 1 ]; then
+			detected_profile="vera"
+			break
+		fi
+
+		sleep 0.1
+	done
+
+	[ -n "${detected_profile}" ] || \
+		fail "unable to detect Grace or Vera sysfs layout under ${dir}"
+
+	if [ "${EXPECT_PROFILE}" != "auto" ] && [ "${EXPECT_PROFILE}" != "${detected_profile}" ]; then
+		fail "expected ${EXPECT_PROFILE} sysfs layout, detected ${detected_profile}"
+	fi
+}
+
+check_requirements()
+{
+	case "${CPU_ID}" in
+	''|*[!0-9]*)
+		fail "ARM_CPUMOD_CPU must be a decimal CPU index"
+		;;
+	esac
+
+	case "${EXPECT_PROFILE}" in
+	auto|grace|vera)
+		;;
+	*)
+		fail "ARM_CPUMOD_EXPECT_PROFILE must be auto, grace, or vera"
+		;;
+	esac
+
+	case "${ENABLE_WRITES}" in
+	0|1)
+		;;
+	*)
+		fail "ARM_CPUMOD_ENABLE_WRITES must be 0 or 1"
+		;;
+	esac
+
+	case "${ENABLE_INVALID_WRITES}" in
+	0|1)
+		;;
+	*)
+		fail "ARM_CPUMOD_ENABLE_INVALID_WRITES must be 0 or 1"
+		;;
+	esac
+
+	case "${ALL_CPUS}" in
+	0|1)
+		;;
+	*)
+		fail "ARM_CPUMOD_ALL_CPUS must be 0 or 1"
+		;;
+	esac
+
+	[ "$(uname -m)" = "aarch64" ] || skip "requires an arm64 host"
+	[ "$(id -u)" -eq 0 ] || skip "must be run as root"
+	select_cpus
+	for current_cpu in "${CPU_LIST[@]}"; do
+		[ -d "$(cpu_dir "${current_cpu}")" ] || skip "missing CPU directory $(cpu_dir "${current_cpu}")"
+	done
+	require_cmd rmmod
+
+	if [ -n "${MODULE_PATH}" ]; then
+		[ -r "${MODULE_PATH}" ] || skip "ARM_CPUMOD_KO is not readable: ${MODULE_PATH}"
+		require_cmd insmod
+	else
+		require_cmd modprobe
+		modprobe -n "${MODULE_NAME}" >/dev/null 2>&1 || \
+			skip "set ARM_CPUMOD_KO or install ${MODULE_NAME} into /lib/modules"
+	fi
+}
+
+load_module()
+{
+	if module_loaded; then
+		if [ -n "${MODULE_PATH}" ]; then
+			skip "${MODULE_NAME} is already loaded; unload it before using ARM_CPUMOD_KO"
+		fi
+
+		reused_loaded_module=1
+		log "${MODULE_NAME} already loaded, reusing existing instance"
+		return 0
+	fi
+
+	if [ -n "${MODULE_PATH}" ]; then
+		insmod "${MODULE_PATH}" || fail "insmod failed for ${MODULE_PATH}"
+		module_loaded || fail "${MODULE_NAME} did not appear in /proc/modules after insmod"
+	else
+		modprobe "${MODULE_NAME}" || fail "modprobe failed for ${MODULE_NAME}"
+		module_loaded || skip "${MODULE_NAME} did not appear in /proc/modules after modprobe; built-in or non-unloadable setup is unsupported"
+	fi
+
+	loaded_by_test=1
+}
+
+check_sysfs_layout()
+{
+	local attr
+	local expected_cpu
+	local value
+	local dir="$(current_cpumod_dir)"
+
+	wait_for_path_state present "${dir}" || \
+		fail "cpumod sysfs directory did not appear at ${dir}"
+
+	for attr in "${COMMON_ATTRS[@]}"; do
+		require_attr_present "${attr}"
+	done
+
+	detect_profile
+
+	case "${detected_profile}" in
+	grace)
+		for attr in "${GRACE_ATTRS[@]}"; do
+			require_attr_present "${attr}"
+		done
+		for attr in "${VERA_ATTRS[@]}"; do
+			require_attr_absent "${attr}"
+		done
+		;;
+	vera)
+		for attr in "${GRACE_ATTRS[@]}"; do
+			require_attr_absent "${attr}"
+		done
+		for attr in "${VERA_ATTRS[@]}"; do
+			require_attr_present "${attr}"
+		done
+		;;
+	esac
+
+	expected_cpu=$((10#${current_cpu}))
+	value=$(read_attr affected_cpus)
+	[ "${value}" = "${expected_cpu}" ] || \
+		fail "cpu${current_cpu} affected_cpus expected ${expected_cpu}, got ${value}"
+
+	for attr in "${COMMON_RW_ATTRS[@]}"; do
+		check_decimal_attr "${attr}"
+	done
+
+	case "${detected_profile}" in
+	grace)
+		for attr in "${GRACE_RW_ATTRS[@]}"; do
+			check_decimal_attr "${attr}"
+		done
+		;;
+	vera)
+		for attr in "${VERA_RW_ATTRS[@]}"; do
+			check_decimal_attr "${attr}"
+		done
+		;;
+	esac
+}
+
+writeback_same_value()
+{
+	local attr="$1"
+	local before
+	local after
+
+	before=$(read_attr "${attr}")
+	printf '%s\n' "${before}" > "$(current_cpumod_dir)/${attr}" || \
+		fail "failed to write back cpu${current_cpu}/${attr}=${before}"
+	after=$(read_attr "${attr}")
+	[ "${after}" = "${before}" ] || \
+		fail "cpu${current_cpu}/${attr} changed across writeback: before=${before} after=${after}"
+}
+
+check_writeback_same_value()
+{
+	local attr
+
+	[ "${ENABLE_WRITES}" = "1" ] || return 0
+
+	for attr in "${COMMON_RW_ATTRS[@]}"; do
+		writeback_same_value "${attr}"
+	done
+
+	case "${detected_profile}" in
+	grace)
+		for attr in "${GRACE_RW_ATTRS[@]}"; do
+			writeback_same_value "${attr}"
+		done
+		;;
+	vera)
+		for attr in "${VERA_RW_ATTRS[@]}"; do
+			writeback_same_value "${attr}"
+		done
+		;;
+	esac
+}
+
+check_invalid_write()
+{
+	local attr="$1"
+	local value="$2"
+	local before
+	local after
+	local file="$(current_cpumod_dir)/${attr}"
+
+	before=$(read_attr "${attr}")
+	if printf '%s\n' "${value}" > "${file}" 2>/dev/null; then
+		fail "cpu${current_cpu}/${attr} accepted invalid value ${value}"
+	fi
+	after=$(read_attr "${attr}")
+	[ "${after}" = "${before}" ] || \
+		fail "cpu${current_cpu}/${attr} changed after invalid write: before=${before} after=${after}"
+}
+
+check_invalid_writes()
+{
+	[ "${ENABLE_INVALID_WRITES}" = "1" ] || return 0
+
+	check_invalid_write pf_dis 2
+	check_invalid_write pf_mode 10
+	check_invalid_write pf_mode 99
+
+	case "${detected_profile}" in
+	grace)
+		check_invalid_write cbusy_filter_threshold 4
+		check_invalid_write cbusy_filter_window 4
+		check_invalid_write cmc_min_ways 8
+		;;
+	vera)
+		check_invalid_write l2spr_cmc_max_ways 8
+		;;
+	esac
+}
+
+check_cpu()
+{
+	current_cpu="$1"
+	detected_profile=""
+	check_sysfs_layout
+	log "cpu${current_cpu}: detected ${detected_profile} sysfs layout"
+	check_writeback_same_value
+	check_invalid_writes
+}
+
+unload_and_verify_cleanup()
+{
+	local cpu
+	local dir
+
+	if [ "${loaded_by_test}" -ne 1 ]; then
+		log "module was already loaded; leaving it in place and skipping unload cleanup check"
+		return 0
+	fi
+
+	rmmod "${MODULE_NAME}" || fail "rmmod failed for ${MODULE_NAME}"
+	loaded_by_test=0
+	for cpu in "${CPU_LIST[@]}"; do
+		dir="$(cpumod_dir "${cpu}")"
+		wait_for_path_state absent "${dir}" || \
+			fail "${dir} still present after unload"
+	done
+}
+
+check_requirements
+load_module
+for current_cpu in "${CPU_LIST[@]}"; do
+	if [ ! -d "$(cpumod_dir "${current_cpu}")" ]; then
+		cpu_is_online "${current_cpu}" || \
+			fail "cpu${current_cpu} went offline or its online state became unreadable"
+		log "cpu${current_cpu}: no cpumod directory; unsupported CPU profile, skipping"
+		continue
+	fi
+
+	check_cpu "${current_cpu}"
+	verified_cpus=$((verified_cpus + 1))
+done
+unload_and_verify_cleanup
+[ "${verified_cpus}" -gt 0 ] || \
+	skip "no selected CPU exposes a supported cpumod profile"
+
+if [ "${reused_loaded_module}" -eq 1 ]; then
+	log "PASS verified_cpus=${verified_cpus} expect_profile=${EXPECT_PROFILE} writes=${ENABLE_WRITES} invalid_writes=${ENABLE_INVALID_WRITES} all_cpus=${ALL_CPUS} (reused pre-loaded module; unload cleanup check skipped)"
+else
+	log "PASS verified_cpus=${verified_cpus} expect_profile=${EXPECT_PROFILE} writes=${ENABLE_WRITES} invalid_writes=${ENABLE_INVALID_WRITES} all_cpus=${ALL_CPUS}"
+fi
+exit 0
-- 
2.43.0