Re: [RFC PATCH 31/38] arm_mpam: resctrl: Update the rmid reallocation limit

Fenghua Yu
2025-12-06T00:07:03Z

Hi, James,

On 12/5/25 13:58, James Morse wrote:
> resctrl's limbo code needs to be told when the data left in a cache is
> small enough for the partid+pmg value to be re-allocated.
>
> x86 uses the cache size divided by the number of rmid users the cache
> may have. Do the same, but for the smallest cache, and with the
> number of partid-and-pmg users.
>
> Querying the cache size can't happen until after cacheinfo_sysfs_init()
> has run, so mpam_resctrl_setup() must wait until then.
>
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> drivers/resctrl/mpam_resctrl.c | 54 ++++++++++++++++++++++++++++++++++
> 1 file changed, 54 insertions(+)
>
> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
> index 506063bd3348..ccdf8db742c9 100644
> --- a/drivers/resctrl/mpam_resctrl.c
> +++ b/drivers/resctrl/mpam_resctrl.c
> @@ -16,6 +16,7 @@
> #include <linux/resctrl.h>
> #include <linux/slab.h>
> #include <linux/types.h>
> +#include <linux/wait.h>
>
> #include <asm/mpam.h>
>
> @@ -58,6 +59,13 @@ static bool cdp_enabled;
> */
> static bool resctrl_enabled;
>
> +/*
> + * mpam_resctrl_pick_caches() needs to know the size of the caches. cacheinfo
> + * populates this from a device_initcall(). mpam_resctrl_setup() must wait.
> + */
> +static bool cacheinfo_ready;
> +static DECLARE_WAIT_QUEUE_HEAD(wait_cacheinfo_ready);
> +
> /*
> * L3 local/total may come from different classes - what is the number of MBWU
> * 'on L3'?
> @@ -584,6 +592,38 @@ void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_mon_domain *d,
> reset_mon_cdp_safe(mon, mon_comp, USE_PRE_ALLOCATED, closid, rmid);
> }
>
> +/*
> + * The rmid realloc threshold should be for the smallest cache exposed to
> + * resctrl.
> + */
> +static int update_rmid_limits(struct mpam_class *class)
> +{
> + u32 num_unique_pmg = resctrl_arch_system_num_rmid_idx();
> + struct mpam_props *cprops = &class->props;
> + struct cacheinfo *ci;
> +
> + lockdep_assert_cpus_held();
> +
> + /* Assume cache levels are the same size for all CPUs... */
> + ci = get_cpu_cacheinfo_level(smp_processor_id(), class->level);
> + if (!ci || ci->size == 0) {
> + pr_debug("Could not read cache size for class %u\n",
> + class->level);
> + return -EINVAL;
> + }
> +
> + if (!mpam_has_feature(mpam_feat_msmon_csu, cprops))
> + return 0;
> +
> + if (!resctrl_rmid_realloc_limit ||
> + ci->size < resctrl_rmid_realloc_limit) {
> + resctrl_rmid_realloc_limit = ci->size;
> + resctrl_rmid_realloc_threshold = ci->size / num_unique_pmg;
> + }
> +
> + return 0;
> +}
> +
> static bool cache_has_usable_cpor(struct mpam_class *class)
> {
> struct mpam_props *cprops = &class->props;
> @@ -1025,6 +1065,9 @@ static void mpam_resctrl_pick_counters(void)
> /* CSU counters only make sense on a cache. */
> switch (class->type) {
> case MPAM_CLASS_CACHE:
> + if (update_rmid_limits(class))
> + continue;
> +
> counter_update_class(QOS_L3_OCCUP_EVENT_ID, class);
> return;
> default:
> @@ -1731,6 +1774,8 @@ int mpam_resctrl_setup(void)
> struct mpam_resctrl_res *res;
> struct mpam_resctrl_mon *mon;
>
> + wait_event(wait_cacheinfo_ready, cacheinfo_ready);

This may cause system hang for any hw/fw issue that causes cacheinfo
failure. Instead of hang, is it better to have a timeout wait here? Like
errowait_event_timeout(wait_cache_info_read, cacheinfo_ready, 5 * HZ);
and report failure when cacheinfo is not ready.

Thanks.

-Fenghua