From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 950852C11E7 for ; Sun, 2 Aug 2026 16:07:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785686860; cv=none; b=rtoqdUt0WjfQCSsSlB7f0mZ97GTifcemE+b5o2TjK5Y6Y/nDM6SKaqnyQrWjPm8BW+AqOXTGrcysKwTHSiyYhUoRzjyD5hNhGYL1tTHd9cmp4Thyw71DJP0f+lV/BYpqIqmySg5/0GiNJ2DU7hPrKacdwd68fSmxb+PmND3h5lQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785686860; c=relaxed/simple; bh=IMIveCSraRuvB5isCk733gb/irUkO688wrLUNd7EwXM=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type; b=q3tR0yoULV7x6iM79dpVTDcpyv1ZRDBv+l/UYWnqLQyAxW/L6VZt1EdrjtUNbePxYSzefl1edyt9sFO0Xm7JA1hes/JVosXD0U0FMaiLZ0tFHWgJeot27AgDlAA0U70cUOlYGlDEMyG0SC3m2PU5dUNvPpIXKdkKDC50VOZlfok= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nJt+9rPc; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nJt+9rPc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1785686856; x=1817222856; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=IMIveCSraRuvB5isCk733gb/irUkO688wrLUNd7EwXM=; b=nJt+9rPc7hvJBzDgQtNoselXCoQMEZUyLLndUOPRnpwolHm8w4rWU/Iy ToiQD37oBV0y21UE6aGGhMAbGs2twk2s+Mn6ZlrcdxFvOthOMZjCXNnjw V956H+6i8J1yYIni0QJvGOGMr2XCHhbR9KoGVRN5207OffbYlLXQmcLjN FKcZXqgweKnB44RSdFZFxK2Xh/9EpX/8Yz53Br+eCJzqB/P+knN9/MYD3 CMGC+1fSkcDg8ugCos+DC5RhqOdXxo4vznkvYkafAkmZAeL41X0ljyAjv z1fWN6KkF4OvEfeOqyiy987Cw8iT3YtHZpT4o1Js7bHMmF1BzyQLt7bN6 w==; X-CSE-ConnectionGUID: 34C6U0SUSi+3JkILg35ATQ== X-CSE-MsgGUID: BfeC6hz1QyScDV/NE8HOKQ== X-IronPort-AV: E=McAfee;i="6800,10657,11863"; a="97611879" X-IronPort-AV: E=Sophos;i="6.25,200,1779174000"; d="scan'208";a="97611879" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2026 09:07:36 -0700 X-CSE-ConnectionGUID: oqK4h+pnT+y+T5nTqPmC2g== X-CSE-MsgGUID: /ibetMcnT/SeKhCVmWVtDA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,200,1779174000"; d="scan'208";a="261066916" Received: from chenyu-dev.sh.intel.com ([10.239.62.107]) by orviesa007.jf.intel.com with ESMTP; 02 Aug 2026 09:07:33 -0700 From: Chen Yu To: Reinette Chatre , Tony Luck Cc: Ben Horgan , "James Morse" , Dave Martin , Babu Moger , Fenghua Yu , Borislav Petkov , Thomas Gleixner , "Dave Hansen" , Peter Newman , chen.yu@linux.dev, x86@kernel.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 00/31] Introduce region-aware RDT support Date: Sun, 2 Aug 2026 23:57:51 +0800 Message-Id: X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi everyone, This is a PoC patch set posted to support region-aware RDT on Intel platform. Its purpose is to demonstrate what region-aware RDT is and to seek for suggestion from the community on how Linux could support this feature properly. It also allows people to have an early try with this preliminary version if they would like to explore it in advance. This version is not intended for inclusion for upstream in its current shape. This patch set is based on the generic schemata PoC patch set, which introduced multiple controllers per rdt_resource. The multi-controller support is the essential framework for region-aware RDT, as each memory region needs its own set of controls on the same resource. The generic schemata PoC is at: git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v1 Since many details are still under discussion above, current patch set is evolving as well, and will be adjusted as the discussion and the decisions move forward. What is region-aware RDT ======================== Intel hardware supports Region-Aware Memory Bandwidth Allocation (MBA) and Region-Aware Memory Bandwidth Monitoring (MBM). With Region-Aware MBA, independent bandwidth control (throttling) of L3 domain bandwidth to multiple regions is supported, enabling users to dynamically rebalance bandwidth control limits across different memory regions, each of which may have distinct bandwidth, latency, and capacity characteristics. Region-Aware MBM includes the capability to independently track multiple domains that are simultaneously accessing several memory regions. These memory regions correspond to different levels of memory tiers, such as directly attached memory (Tier 1), CXL-attached memory (Tier 2), and CXL accelerator devices with attached memory. Note, the region ID is per socket scope. Intel platform supports up to 4 regions for now. Suppose there are 2 regions in each socket: +------------------------+------------------------+ | +--------+ +--------+ | +--------+ +--------+ | | | L3_00 | | L3_01 | | | L3_02 | | L3_03 | | | +--------+ +--------+ | +--------+ +--------+ | | +-------------------+ | +-------------------+ | | | IMC1 | | | IMC2 | | | +-------------------+ | +-------------------+ | | +-------+ +-------+ | +-------+ +-------+ | | | DDR1 | | CXL1 | | | DDR2 | | CXL2 | | | +-------+ +-------+ | +-------+ +-------+ | | socket0 | socket1 | +------------------------+------------------------+ In above graph, from the perspective of a CPU associated with the L3_00 domain, Region 0 typically represents the local memory region (DDR1), while Region 2 typically represents the remote memory region (DDR2). Similarly, for CPUs attached to L3_00, Region 1 represents the local memory region (CXL1), and Region 3 represents the remote memory region (CXL2). Here the terms "local" and "remote" are defined at the socket level. Take the region aware MBM for example. For the L3_00 domain, the memory bandwidth of Region 0 refers to the data transferred when the L3 miss occurs in L3_00 and the data is refilled from DDR1 - note, data refilled from L3_01 to L3_00 is not counted in. The bandwidth of Region 2, by contrast, refers to the data refilled from DDR2. Similarly, the same calculation logic applies to Region 1 (CXL1) and Region 3 (CXL2). For Region-Aware MBA, when setting values for Region 0 on CPUs attached to L3_00, this configuration controls the traffic generated when data is transferred between cores and L3_00 targeting DDR1. The resctrl interface ===================== Region-aware MBM adds one file per region to each mon_data domain directory. For example, with 4 regions the following files are created: mbm_region0_bytes, mbm_region1_bytes, mbm_region2_bytes and mbm_region3_bytes For region-aware MBA, every region has its own set of controls, and each of them appears as a separate schemata line named "MB_REGION_", where is the region number and is one of OPT, MIN or MAX. Only the control types that the hardware reports in the MARC ACPI sub-table are present. The legacy "MB" control is emulated by the MAX control of every region, so writing "MB" throttles all regions at once, while writing an individual "MB_REGION_MAX" line throttles only that region. Below is an example on a platform with 2 memory regions, adjusting the tier1 local memory bandwidth (usually the DDR) of the second domain: # cat schemata MB:0=100;1=100;2=100;3=100 MB_REGION0_OPT:0=511;1=511;2=511;3=511 MB_REGION0_MIN:0=511;1=511;2=511;3=511 MB_REGION0_MAX:0=511;1=511;2=511;3=511 MB_REGION1_OPT:0=511;1=511;2=511;3=511 MB_REGION1_MIN:0=511;1=511;2=511;3=511 MB_REGION1_MAX:0=511;1=511;2=511;3=511 L3:0=3ff;1=3ff;2=3ff;3=3ff # echo "MB_REGION0_MAX:1=200" > schemata # cat schemata MB:0=100;1=100;2=100;3=100 MB_REGION0_OPT:0=511;1=511;2=511;3=511 MB_REGION0_MIN:0=511;1=511;2=511;3=511 MB_REGION0_MAX:0=511;1=200;2=511;3=511 MB_REGION1_OPT:0=511;1=511;2=511;3=511 MB_REGION1_MIN:0=511;1=511;2=511;3=511 MB_REGION1_MAX:0=511;1=511;2=511;3=511 L3:0=3ff;1=3ff;2=3ff;3=3ff How this patch set is composed ============================== Patches 1 to 9, from "x86/topology: Export topo_lookup_cpuid() for resctrl use" to "x86/resctrl: Add MMIO-based LLC occupancy monitoring support", add the MMIO-based CMT support, which has already been posted at: https://lore.kernel.org/all/cover.1784968626.git.yu.c.chen@intel.com/ They are included here because they have been slightly modified to apply on top of Reinette's generic schemata PoC. Patches 10 to 14, from Revert "x86/resctrl: NOT_FOR_INCLUSION: Example support for multiple controls" to "x86/resctrl: Add emulation controller list to resctrl_ctrl", prepare for region-aware RDT. Patches 15 to 19, from "x86/resctrl: Parse ACPI MMRC table" to "x86/resctrl: Enable the region based events by adding them into the event", add the region-aware MBM support. Patches 20 to 30, from "x86/resctrl: Rename msr_update to hw_update" to the "fs/resctrl: Fix excessive padding in schemata output", add the region-aware MBA support. Patch 31 is the documentation for region-aware RDT. Any comments would be much appreciated. thanks, Chenyu Anil S Keshavamurthy (1): x86/resctrl: Parse ACPI ERDT table and save CACD cpumask for RMDD domains Chen Yu (29): x86/topology: Export topo_lookup_cpuid() for resctrl use x86/resctrl: Require 64-bit x86 for resctrl support x86/resctrl: Attach ACPI ERDT information to L3 mon domain on CPU online x86/resctrl: Parse ACPI CMRC table x86/resctrl: Refactor the monitor read function x86/resctrl: Introduce erdt_cpu_has() and erdt_support() x86/resctrl: Add MMIO-based LLC occupancy monitoring support Revert "x86/resctrl: NOT_FOR_INCLUSION: Example support for multiple controls" x86/resctrl: Rename struct resctrl_membw to struct resctrl_ctrl_scalar x86/resctrl: Rename struct resctrl_cache to struct resctrl_ctrl_bitmap x86/resctrl: Add per-control and per-resource flags x86/resctrl: Add emulation controller list to resctrl_ctrl x86/resctrl: Parse ACPI MMRC table x86/resctrl: Replace "msr" in monitoring data identifiers x86/resctrl: Introduce region aware MBM event definitions x86/resctrl: Introduce memory region based MBM read callback on MMIO space x86/resctrl: Enable the region based events by adding them into the event x86/resctrl: Rename msr_update to hw_update x86/resctrl: Parse ACPI MARC table fs/resctrl: Add region-based control names and resctrl_ctrl_name_region() x86/resctrl: Add region aware MBA controllers x86/resctrl: Attach ACPI ERDT information to ctrl domain on CPU online x86/resctrl: Introduce region-based MBA write implementation on MMIO space x86/resctrl: Allow control writes from any CPU for MMIO controllers x86/resctrl: Enable region-aware MBM/MBA via the RDT_CTRL register x86/resctrl: Emulate the legacy MBA controller via the region MAX controls fs/resctrl: Expose emulation controllers in a resource_schemata subdir fs/resctrl: Fix excessive padding in schemata output x86,fs/resctrl: Update Documentation for region aware RDT Tony Luck (1): fs/resctrl: Do not invoke smp_processor_id() in preemptible context Documentation/filesystems/resctrl.rst | 110 +++ arch/x86/Kconfig | 4 +- arch/x86/include/asm/apic.h | 1 + arch/x86/include/asm/resctrl.h | 11 +- arch/x86/kernel/cpu/resctrl/Makefile | 1 + arch/x86/kernel/cpu/resctrl/core.c | 258 +++---- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 106 ++- arch/x86/kernel/cpu/resctrl/erdt.c | 886 ++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 95 ++- arch/x86/kernel/cpu/resctrl/monitor.c | 82 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 14 +- arch/x86/kernel/cpu/topology.c | 2 +- drivers/resctrl/mpam_resctrl.c | 18 +- fs/resctrl/ctrlmondata.c | 49 +- fs/resctrl/monitor.c | 73 +- fs/resctrl/pseudo_lock.c | 2 +- fs/resctrl/rdtgroup.c | 111 ++- include/linux/resctrl.h | 82 +- include/linux/resctrl_types.h | 21 +- 19 files changed, 1634 insertions(+), 292 deletions(-) create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c -- 2.43.0