Re: [PATCH v3 00/47] arm_mpam: Add KVM/arm64 and resctrl glue code

[Gavin Shan <gshan@redhat.com>]

On 1/13/26 12:58 AM, Ben Horgan wrote:
> This new version of the mpam missing pieces has no major rework from the
> previous version. It's mainly small corrections and code tidying based on
> review and things I spotted along the way. To be able to merge this we need
> review from more people and people to start testing on their platforms and
> giving some Tested-by tags.
> 
> Change list in patches.
> 
> As mentioned in the cover letter for v2, one major departure from the
> previous snapshot branches referenced in the base driver series is that the
> same MPAM setting are used for kernel-space and user-space. There are pros
> and cons of choosing this policy but I think it is the best thing to start
> with as there are AMD plans for adding a resctrl feature to allow a
> different closid/rmid configuration for user-space from kernel space. The
> AMD feature is called PLZA and is mentioned in this lpc slide deck[1]. This
> gives us a path forward to add support for having the EL1 and EL0 MPAM
> partid/pmg configuration differ from each other.
> 
>  From James' cover letter:
> 
> This is the missing piece to make MPAM usable resctrl in user-space. This has
> shed its debugfs code and the read/write 'event configuration' for the monitors
> to make the series smaller.
> 
> This adds the arch code and KVM support first. I anticipate the whole thing
> going via arm64, but if goes via tip instead, the an immutable branch with those
> patches should be easy to do.
> 
> Generally the resctrl glue code works by picking what MPAM features it can expose
> from the MPAM drive, then configuring the structs that back the resctrl helpers.
> If your platform is sufficiently Xeon shaped, you should be able to get L2/L3 CPOR
> bitmaps exposed via resctrl. CSU counters work if they are on/after the L3. MBWU
> counters are considerably more hairy, and depend on hueristics around the topology,
> and a bunch of stuff trying to emulate ABMC.
> If it didn't pick what you wanted it to, please share the debug messages produced
> when enabling dynamic debug and booting with:
> | dyndbg="file mpam_resctrl.c +pl"
> 
> I've not found a platform that can test all the behaviours around the monitors,
> so this is where I'd expect the most bugs.
> 
> The MPAM spec that describes all the system and MMIO registers can be found here:
> https://developer.arm.com/documentation/ddi0598/db/?lang=en
> (Ignored the 'RETIRED' warning - that is just arm moving the documentation around.
>   This document has the best overview)
> 
> 
> Based on v6.19-rc5
> This series can be retrieved from:
> https://gitlab.arm.com/linux-arm/linux-bh.git mpam_resctrl_glue_v3
> 
> v2 can be found at:
> https://lore.kernel.org/linux-arm-kernel/20251219181147.3404071-1-ben.horgan@arm.com/
> 
> rfc can be found at:
> https://lore.kernel.org/linux-arm-kernel/20251205215901.17772-1-james.morse@arm.com/
> 
> [1] https://lpc.events/event/19/contributions/2093/attachments/1958/4172/resctrl%20Microconference%20LPC%202025%20Tokyo.pdf
> 
> Ben Horgan (10):
>    arm_mpam: Use non-atomic bitops when modifying feature bitmap
>    arm64/sysreg: Add MPAMSM_EL1 register
>    KVM: arm64: Preserve host MPAM configuration when changing traps
>    KVM: arm64: Make MPAMSM_EL1 accesses UNDEF
>    arm64: mpam: Initialise and context switch the MPAMSM_EL1 register
>    KVM: arm64: Use kernel-space partid configuration for hypercalls
>    arm_mpam: resctrl: Add rmid index helpers
>    arm_mpam: resctrl: Add kunit test for rmid idx conversions
>    arm_mpam: resctrl: Wait for cacheinfo to be ready
>    arm_mpam: resctrl: Add kunit test for mbw min control generation
> 
> Dave Martin (2):
>    arm_mpam: resctrl: Convert to/from MPAMs fixed-point formats
>    arm_mpam: resctrl: Add kunit test for control format conversions
> 
> James Morse (30):
>    arm64: mpam: Context switch the MPAM registers
>    arm64: mpam: Re-initialise MPAM regs when CPU comes online
>    arm64: mpam: Advertise the CPUs MPAM limits to the driver
>    arm64: mpam: Add cpu_pm notifier to restore MPAM sysregs
>    arm64: mpam: Add helpers to change a task or cpu's MPAM PARTID/PMG
>      values
>    KVM: arm64: Force guest EL1 to use user-space's partid configuration
>    arm_mpam: resctrl: Add boilerplate cpuhp and domain allocation
>    arm_mpam: resctrl: Sort the order of the domain lists
>    arm_mpam: resctrl: Pick the caches we will use as resctrl resources
>    arm_mpam: resctrl: Implement resctrl_arch_reset_all_ctrls()
>    arm_mpam: resctrl: Add resctrl_arch_get_config()
>    arm_mpam: resctrl: Implement helpers to update configuration
>    arm_mpam: resctrl: Add plumbing against arm64 task and cpu hooks
>    arm_mpam: resctrl: Add CDP emulation
>    arm_mpam: resctrl: Add support for 'MB' resource
>    arm_mpam: resctrl: Add support for csu counters
>    arm_mpam: resctrl: Pick classes for use as mbm counters
>    arm_mpam: resctrl: Pre-allocate free running monitors
>    arm_mpam: resctrl: Pre-allocate assignable monitors
>    arm_mpam: resctrl: Add kunit test for ABMC/CDP interactions
>    arm_mpam: resctrl: Add resctrl_arch_config_cntr() for ABMC use
>    arm_mpam: resctrl: Allow resctrl to allocate monitors
>    arm_mpam: resctrl: Add resctrl_arch_rmid_read() and
>      resctrl_arch_reset_rmid()
>    arm_mpam: resctrl: Add resctrl_arch_cntr_read() &
>      resctrl_arch_reset_cntr()
>    arm_mpam: resctrl: Update the rmid reallocation limit
>    arm_mpam: resctrl: Add empty definitions for assorted resctrl
>      functions
>    arm64: mpam: Select ARCH_HAS_CPU_RESCTRL
>    arm_mpam: resctrl: Call resctrl_init() on platforms that can support
>      resctrl
>    arm_mpam: Generate a configuration for min controls
>    arm_mpam: Quirk CMN-650's CSU NRDY behaviour
> 
> Jiapeng Chong (1):
>    arm_mpam: Remove duplicate linux/srcu.h header
> 
> Shanker Donthineni (4):
>    arm_mpam: Add quirk framework
>    arm_mpam: Add workaround for T241-MPAM-1
>    arm_mpam: Add workaround for T241-MPAM-4
>    arm_mpam: Add workaround for T241-MPAM-6
> 
>   Documentation/arch/arm64/silicon-errata.rst |    9 +
>   arch/arm64/Kconfig                          |    6 +-
>   arch/arm64/include/asm/el2_setup.h          |    3 +-
>   arch/arm64/include/asm/mpam.h               |   98 +
>   arch/arm64/include/asm/resctrl.h            |    2 +
>   arch/arm64/include/asm/thread_info.h        |    3 +
>   arch/arm64/kernel/Makefile                  |    1 +
>   arch/arm64/kernel/cpufeature.c              |   21 +-
>   arch/arm64/kernel/mpam.c                    |   58 +
>   arch/arm64/kernel/process.c                 |    7 +
>   arch/arm64/kvm/hyp/include/hyp/switch.h     |   12 +-
>   arch/arm64/kvm/hyp/nvhe/hyp-main.c          |    8 +
>   arch/arm64/kvm/hyp/vhe/sysreg-sr.c          |   13 +
>   arch/arm64/kvm/sys_regs.c                   |    2 +
>   arch/arm64/tools/sysreg                     |    8 +
>   drivers/resctrl/Kconfig                     |    9 +-
>   drivers/resctrl/Makefile                    |    1 +
>   drivers/resctrl/mpam_devices.c              |  306 ++-
>   drivers/resctrl/mpam_internal.h             |  131 +-
>   drivers/resctrl/mpam_resctrl.c              | 1930 +++++++++++++++++++
>   drivers/resctrl/test_mpam_devices.c         |   66 +
>   drivers/resctrl/test_mpam_resctrl.c         |  426 ++++
>   include/linux/arm_mpam.h                    |   32 +
>   23 files changed, 3119 insertions(+), 33 deletions(-)
>   create mode 100644 arch/arm64/include/asm/mpam.h
>   create mode 100644 arch/arm64/include/asm/resctrl.h
>   create mode 100644 arch/arm64/kernel/mpam.c
>   create mode 100644 drivers/resctrl/mpam_resctrl.c
>   create mode 100644 drivers/resctrl/test_mpam_resctrl.c
> 

The L3 cache partitioning and MBW (soft) limiting works fine on NVidia's grace-hopper machine.

Tested-by: Gavin Shan <gshan@redhat.com>

Thanks,
Gavin