Re: [PATCH v6 3/9] x86/resctrl: Parse ACPI ERDT table and save CACD cpumask for RMDD domains

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Chenyu,

On 7/25/26 2:22 AM, Chen Yu wrote:
> From: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
> 
> Parse the RMDD subtables within the ERDT ACPI table and their nested
> CACD entries to construct per-domain CPU masks.

The changelog always needs to start with context. Please see "Changelog"
in Documentation/process/maintainer-tip.rst for complete expectations.

> 
> There is one ERDT table per platform. Each RMDD describes one resource

I do not think what "RMDD" stands for has been introduced at this point yet.

> management domain (RMD), also known as an L3 domain, and carries MMIO
> base information for later monitoring support.
> 
> For each RMDD, parse the associated CACD, map its x2APIC IDs to logical

What is "CACD"? Please always expand acronym before its first use.

> CPUs, and save the resulting CPU mask. This mask associates each ERDT
> domain with the CPUs that belong to it and is used later when attaching
> ERDT data to resctrl monitoring domains.

Please let each patch description stand on its own without referring to later
patches in series. If a patch provides capability in preparation for
future changes then you can use language like, 
"Associate every ERDT domain with the CPUs that belong to it to prepare
 for attaching ERDT data to resctrl monitoring domains."



> ---
>  arch/x86/kernel/cpu/resctrl/Makefile   |   1 +
>  arch/x86/kernel/cpu/resctrl/core.c     |  14 +-
>  arch/x86/kernel/cpu/resctrl/erdt.c     | 266 +++++++++++++++++++++++++
>  arch/x86/kernel/cpu/resctrl/internal.h |  31 +++
>  4 files changed, 311 insertions(+), 1 deletion(-)
>  create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/Makefile b/arch/x86/kernel/cpu/resctrl/Makefile
> index 273ddfa30836..2216ee084832 100644
> --- a/arch/x86/kernel/cpu/resctrl/Makefile
> +++ b/arch/x86/kernel/cpu/resctrl/Makefile
> @@ -2,6 +2,7 @@
>  obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
>  obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
>  obj-$(CONFIG_X86_CPU_RESCTRL_INTEL_AET)	+= intel_aet.o
> +obj-$(CONFIG_X86_CPU_RESCTRL)		+= erdt.o
>  obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
>  
>  # To allow define_trace.h's recursive include:
> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index 9c01d2562b7a..23925bcd71d7 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -1013,6 +1013,7 @@ static __init void check_quirks(void)
>  
>  static __init bool get_rdt_resources(void)
>  {
> +	erdt_init();
>  	rdt_alloc_capable = get_rdt_alloc_resources();
>  	rdt_mon_capable = get_rdt_mon_resources();
>  

Functions are not expected to leave dangling state when they return failure. get_rdt_resources()
returning false is considered a failure and now the caller is left to clean up the dangling
state which is not a familiar pattern to use and thus something that can/will trip people.

On top of this this implementation pushes the cleanup very far from even the caller making
this unfamiliar pattern even harder to recognize.

Please let get_rdt_resources() clean up after itself on failure to find any resources. 

> @@ -1114,7 +1115,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c)
>  	}
>  }
>  
> -static int __init resctrl_arch_late_init(void)
> +static int __init __resctrl_arch_late_init(void)
>  {
>  	struct rdt_resource *r;
>  	int state, ret, i;
> @@ -1157,6 +1158,15 @@ static int __init resctrl_arch_late_init(void)
>  	return 0;
>  }
>  
> +static int __init resctrl_arch_late_init(void)
> +{
> +	int ret = __resctrl_arch_late_init();
> +
> +	if (ret)
> +		erdt_exit();
> +	return ret;
> +}

Related to earlier comment on cleanup I find this cleanup to be asymmentrical
and inconsistent with how resctrl usually does cleanup. Why not do cleanup in
(original) resctrl_arch_late_init() to be consistent with other cleanup when
failures are encountered during initialization, for example, cpuhp_remove_state()?
I find that having the cleanup handled where error is encountered is easier to understand.

> +
>  late_initcall(resctrl_arch_late_init);
>  
>  static void __exit resctrl_arch_exit(void)
> @@ -1166,6 +1176,8 @@ static void __exit resctrl_arch_exit(void)
>  	cpuhp_remove_state(rdt_online);
>  
>  	resctrl_exit();
> +
> +	erdt_exit();


Here the cleanup is indeed done in the same place as cleanup of
other init work (cpuhp_remove_state()).

>  }
>  
>  __exitcall(resctrl_arch_exit);
> diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
> new file mode 100644
> index 000000000000..8998cae47090
> --- /dev/null
> +++ b/arch/x86/kernel/cpu/resctrl/erdt.c
> @@ -0,0 +1,266 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Enhanced Resource Director Technology (ERDT)
> + *
> + * Copyright (C) 2026 Intel Corporation
> + *
> + */
> +
> +#define pr_fmt(fmt)     "resctrl: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/overflow.h>
> +#include <linux/resctrl.h>
> +#include <linux/sizes.h>
> +
> +#include <asm/apic.h>
> +
> +#include "internal.h"
> +
> +static LIST_HEAD(domain_info_list);
> +
> +/* True when the ERDT ACPI table describes at least one domain with at least one CPU. */
> +static bool erdt_enabled;
> +
> +#define ERDT_VALID_VERSION		1
> +#define RMDD_FLAG_CPU_L3_DOMAIN		BIT(0)
> +
> +/* Bitmask of valid sub-tables found in the first RMDD, used to ensure all RMDDs match. */
> +static u32 valid_subtbl_mask;
> +
> +/* Domain ID of the first RMDD that established @valid_subtbl_mask, for diagnostics. */
> +static u16 first_rmdd_domain_id;
> +
> +static int erdt_max_rmid;

Could this ever be negative? Could this instead be of same type as the value it is
initialized with? Looks like this will improve type safety with the min_t() usage then
using the accurate and consistent type of both parameters?

I also think a comment describing erdt_max_rmid will be helpful, especially considering
that it has "max" in its name but then its value is determined using the *minimum* of
all domains' RMID?

> +
> +int erdt_get_max_rmid(void)
> +{
> +	return erdt_max_rmid;
> +}
> +
> +static void __iomem *erdt_ioremap(phys_addr_t base, u32 num_pages, const char *desc)
> +{
> +	void __iomem *addr;
> +	size_t size;
> +
> +	if (check_mul_overflow(num_pages, SZ_4K, &size))
> +		return NULL;
> +
> +	addr = ioremap(base, size);

The types seem to target a function with prototype ioremap(phys_addr_t base, size_t size)
but I find arch/x86/include/asm/io.h to declare:
void __iomem *ioremap(resource_size_t offset, unsigned long size);

Since it is this code that determines the type it could just use matching accurate type
from the beginning?

> +	if (!addr)
> +		pr_warn(FW_BUG "ERDT: Failed to map %s at phys addr %pa (size: %u pages)\n",
> +		       desc, &base, num_pages);

Please align to open parenthesis. 

> +
> +	return addr;
> +}
> +
> +static void erdt_iounmap_domain(struct erdt_domain_info *domain)
> +{
> +	for (int i = 0; i < ERDT_MMIO_NUM_TYPES; i++) {
> +		if (domain->base[i]) {
> +			iounmap(domain->base[i]);
> +			domain->base[i] = NULL;
> +		}
> +	}
> +}
> +
> +static void cleanup_one_domain(struct erdt_domain_info *d)
> +{
> +	erdt_iounmap_domain(d);
> +	kfree(d);
> +}
> +
> +/*
> + * Save CACD information for this RMDD:
> + * convert the X2APIC to CPU and save them in a mask.
> + */
> +static __init int cacd_init(struct acpi_subtbl_hdr_16 *subtbl,
> +			    struct erdt_domain_info *domain_info)
> +{
> +	struct acpi_erdt_cacd *cacd = (struct acpi_erdt_cacd *)subtbl;
> +	int num_ids, cpu;

Can num_ids ever be negative? If not, please use unsigned type.

> +
> +	if (cacd->header.length < struct_size(cacd, X2APICIDS, 1)) {
> +		pr_warn(FW_BUG "Invalid x2apicid CACD table\n");
> +		return -EIO;
> +	}
> +
> +	num_ids = (cacd->header.length - sizeof(*cacd)) / sizeof(cacd->X2APICIDS[0]);
> +
> +	for (int i = 0; i < num_ids; i++) {
> +		cpu = topo_lookup_cpuid(cacd->X2APICIDS[i]);
> +		if (cpu < 0) {
> +			pr_warn(FW_BUG "Unknown x2apicid 0x%x\n", cacd->X2APICIDS[i]);
> +			return -EIO;
> +		}
> +
> +		cpumask_set_cpu(cpu, &domain_info->cpu_mask);
> +	}
> +
> +	return 0;
> +}
> +
> +static inline struct acpi_subtbl_hdr_16 *rmdd_subtbl(struct acpi_erdt_rmdd *rmdd)
> +{
> +	return (void *)rmdd + sizeof(*rmdd);
> +}
> +
> +static inline struct acpi_subtbl_hdr_16 *next_subtbl(struct acpi_subtbl_hdr_16 *subtbl)
> +{
> +	return (void *)subtbl + subtbl->length;
> +}
> +
> +static inline bool subtbl_valid(void *end, struct acpi_subtbl_hdr_16 *subtbl)
> +{
> +	/* Ensure the header is within bounds before dereferencing it. */
> +	if ((void *)subtbl + sizeof(*subtbl) > end)
> +		return false;
> +
> +	/* A sub-table must be at least as large as its header. */
> +	if (subtbl->length < sizeof(*subtbl))
> +		return false;
> +
> +	/* The entire sub-table (including body) must fit within the parent. */
> +	if ((void *)subtbl + subtbl->length > end)
> +		return false;
> +
> +	return true;
> +}
> +
> +static __init bool parse_rmdd_table(struct acpi_subtbl_hdr_16 *rmdd_hdr)
> +{
> +	struct erdt_domain_info *domain_info;
> +	struct acpi_subtbl_hdr_16 *subtbl;
> +	struct acpi_erdt_rmdd *rmdd;
> +	u32 subtbl_mask = 0;
> +
> +	if (rmdd_hdr->length < sizeof(*rmdd)) {
> +		pr_warn(FW_BUG "Invalid RMDD length %u bytes\n", rmdd_hdr->length);
> +		return false;
> +	}
> +
> +	rmdd = (struct acpi_erdt_rmdd *)rmdd_hdr;

Could this initialization be done at time of declaration to be consistent with the other
functions parsing tables? The length comparison could then use rmdd->header.length to
match how this check is done in all the other places. Being consistent makes the
code much easier to understand.


> +
> +	/* Quietly ignore non-CPU-based L3 domains */
> +	if (!(rmdd->flags & RMDD_FLAG_CPU_L3_DOMAIN))
> +		return true;
> +
> +	domain_info = kzalloc_obj(*domain_info, GFP_KERNEL);
> +	if (!domain_info)
> +		return false;
> +
> +	domain_info->dom_id = -1;
> +
> +	domain_info->base[ERDT_MMIO_RMDD_CREG] =
> +		erdt_ioremap(rmdd->creg_base, rmdd->creg_size, "RMDD ctrl base");
> +	if (!domain_info->base[ERDT_MMIO_RMDD_CREG])
> +		goto cleanup;
> +
> +	for (subtbl = rmdd_subtbl(rmdd);
> +	     subtbl_valid((void *)rmdd + rmdd->header.length, subtbl);
> +	     subtbl = next_subtbl(subtbl)) {

I find it curious how the implementation varies in how the tables are parsed. For example, here
it uses a for () loop to cycle through the tables while enumerate_erdt_table() uses a while()
loop for what appears to be the same flow. Are they actually different? Why are the two different
patterns needed?

> +		switch (subtbl->type) {
> +		/* An RMDD table has one or more CACD sub-table(s) */
> +		case ACPI_ERDT_TYPE_CACD:
> +			if (cacd_init(subtbl, domain_info))
> +				goto cleanup;
> +
> +			subtbl_mask |= BIT(ACPI_ERDT_TYPE_CACD);
> +			break;
> +		default:
> +			break;
> +		}
> +	}
> +
> +	if (!subtbl_mask)
> +		goto cleanup;
> +
> +	/*
> +	 * Require all RMDDs to support same set of sub-tables
> +	 */
> +	if (!valid_subtbl_mask) {
> +		valid_subtbl_mask = subtbl_mask;
> +		first_rmdd_domain_id = rmdd->domain_id;
> +	} else if (subtbl_mask != valid_subtbl_mask) {
> +		pr_warn(FW_BUG "RMDD %u sub-table set does not match the first RMDD %u\n",
> +			rmdd->domain_id, first_rmdd_domain_id);
> +		goto cleanup;
> +	}
> +
> +	if (!rmdd->max_rmid) {
> +		pr_warn(FW_BUG "Unreasonable RMDD max_rmid %u\n", rmdd->max_rmid);
> +		goto cleanup;
> +	}
> +	domain_info->max_rmid = rmdd->max_rmid;
> +
> +	if (!erdt_max_rmid)
> +		erdt_max_rmid = rmdd->max_rmid;
> +	else
> +		erdt_max_rmid = min_t(int, erdt_max_rmid, rmdd->max_rmid);
> +
> +	list_add(&domain_info->entry, &domain_info_list);
> +
> +	return true;
> +
> +cleanup:
> +	cleanup_one_domain(domain_info);
> +	return false;
> +}
> +
> +void erdt_exit(void)
> +{
> +	struct erdt_domain_info *d, *tmp;
> +
> +	list_for_each_entry_safe(d, tmp, &domain_info_list, entry) {
> +		list_del(&d->entry);
> +		cleanup_one_domain(d);
> +	}
> +	erdt_enabled = false;
> +	valid_subtbl_mask = 0;
> +	first_rmdd_domain_id = 0;

Should erdt_max_rmid be reset also?

> +}
> +
> +static __init int enumerate_erdt_table(struct acpi_table_header *table_hdr)
> +{
> +	struct acpi_table_erdt *erdt = (struct acpi_table_erdt *)table_hdr;
> +	struct acpi_subtbl_hdr_16 *subtbl;
> +	void *table_end;
> +
> +	if (erdt->header.revision != ERDT_VALID_VERSION) {
> +		pr_info("Unsupported ERDT table revision %u (expected %u)\n",
> +			erdt->header.revision, ERDT_VALID_VERSION);
> +		return -EINVAL;
> +	}
> +
> +	if (erdt->header.length < sizeof(*erdt)) {
> +		pr_warn(FW_BUG "ERDT: Invalid table length %u bytes\n", erdt->header.length);
> +		return -EINVAL;
> +	}
> +
> +	subtbl = (void *)erdt + sizeof(struct acpi_table_erdt);

Please use sizeof(*erdt)

> +	table_end = (void *)erdt + erdt->header.length;
> +
> +	while (subtbl_valid(table_end, subtbl)) {
> +		if (subtbl->type == ACPI_ERDT_TYPE_RMDD &&
> +		    !parse_rmdd_table(subtbl))
> +			goto cleanup;
> +
> +		subtbl = next_subtbl(subtbl);
> +	}
> +
> +	if (list_empty(&domain_info_list))
> +		goto cleanup;
> +
> +	erdt_enabled = true;
> +
> +	return 0;
> +
> +cleanup:
> +	erdt_exit();
> +	return -EINVAL;
> +}
> +
> +int __init erdt_init(void)
> +{
> +	return acpi_table_parse(ACPI_SIG_ERDT, enumerate_erdt_table);
> +}
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index e3cfa0c10e92..bdff3ea36e62 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -21,6 +21,33 @@
>  
>  #define RMID_VAL_UNAVAIL		BIT_ULL(62)
>  
> +/*
> + * Index into erdt_domain_info::base[] for each MMIO region.
> + * @ERDT_MMIO_RMDD_CREG: RMDD control register base address
> + */
> +enum erdt_mmio_type {
> +	ERDT_MMIO_RMDD_CREG,
> +	ERDT_MMIO_LAST = ERDT_MMIO_RMDD_CREG
> +};
> +
> +#define ERDT_MMIO_NUM_TYPES	(ERDT_MMIO_LAST + 1)
> +
> +/**
> + * struct erdt_domain_info - Per-domain ERDT information
> + * @base:	Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_* type

I think "type" can be dropped? The enum is already implicitly an "MMIO type"?

> + * @cpu_mask:	CPUs belonging to this resource management domain
> + * @max_rmid:	Maximum RMID supported by this domain
> + * @dom_id:	L3 cache ID shared by all CPUs in this domain (-1 if unset)
> + * @entry:	Links into the global domain_info_list
> + */
> +struct erdt_domain_info {
> +	void __iomem		*base[ERDT_MMIO_NUM_TYPES];
> +	struct cpumask		cpu_mask;
> +	u32			max_rmid;
> +	int			dom_id;
> +	struct list_head	entry;
> +};
> +
>  /*
>   * With the above fields in use 62 bits remain in MSR_IA32_QM_CTR for
>   * data to be returned. The counter width is discovered from the hardware
> @@ -253,4 +280,8 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resour
>  static inline bool intel_handle_aet_option(bool force_off, char *tok) { return false; }
>  #endif
>  
> +int erdt_get_max_rmid(void);
> +int erdt_init(void);
> +void erdt_exit(void);
> +
>  #endif /* _ASM_X86_RESCTRL_INTERNAL_H */

Reinette