From: Ben Horgan <ben.horgan@arm.com>
To: tizi4590 <ruanjinjie@huawei.com>,
james.morse@arm.com, reinette.chatre@intel.com,
fenghuay@nvidia.com, tony.luck@intel.com, Dave.Martin@arm.com,
babu.moger@amd.com, will@kernel.org, jic23@kernel.org,
gshan@redhat.com, bp@alien8.de, x86@kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 1/2] arm_mpam: resctrl: Use the domain's counter ID
Date: Tue, 17 Nov 2026 11:27:39 +0000 [thread overview]
Message-ID: <a7dc44ee-cf2c-4e2a-9953-4f8ffeffda60@arm.com> (raw)
In-Reply-To: <20261113083506.4129060-2-ruanjinjie@huawei.com>
Hi Jinjie,
On 13/11/2026 08:35, tizi4590 wrote:
> From: Jinjie Ruan <ruanjinjie@huawei.com>
>
> resctrl assigns MBM counters per domain: mbm_L3_assignments takes a
> domain ID, a group can have a counter in one domain and none in
> another, and resctrl_arch_cntr_read() and resctrl_arch_reset_cntr()
> are given the counter to use.
>
> The driver ignores that counter and looks the monitor up in
> mbwu_idx_to_mon[], indexed by CLOSID and RMID.
> resctrl_arch_config_cntr() fills that array without looking at the
> domain, so each group has one monitor for the whole resource.
> Whichever assignment is written last selects the monitor the group
> reads in every domain.
For me, this scenario where we read the wrong counter to be the most
serious bug in the MPAM driver I currently know about. Thanks for
addressing it.
>
> For example, with the counter ID each group uses for mbm_total_bytes
> on each domain and "_" for none:
>
> domain 0 domain 1
> mount, mkdir g1 default=0 g1=1 default=0 g1=1
> unassign both on domain 1 default=0 g1=1 default=_ g1=_
> assign g1 on domain 1 default=0 g1=1 default=_ g1=0
>
> g1 is assigned the lowest free counter on domain 1, counter 0. That
> assignment is the last one written for g1, so it selects the monitor
> g1 reads in every domain. On domain 0, g1 uses counter 1, but reads
> the monitor of counter 0:
>
> counter whose monitor is read on domain 0
> default g1
> expected 0 1
> actual 0 0
>
> That is the default group's monitor.
>
> Translate the counter ID with assigned_counters[], which maps a counter
> ID to a monitor of the class and is valid in every domain,
Ok, sounds reasonable.
and drop
> mbwu_idx_to_mon[] with the lookups in __read_mon() and __reset_mon().
mbwu_idx_to_mon[] does become unused but if we want to ever support the
default mbm_assign_mode then it may become used. A little more on this
below but this can be considered separately from the bug fix.
> With nothing left to record, resctrl_arch_config_cntr() only resets the
> monitors of a counter when it is assigned.
Isn't this a separate optimization?
>
> Nothing consumes USE_PRE_ALLOCATED, the marker that triggered those
> lookups, once they are gone. resctrl only allocates a monitor context
> for an MBM event when resctrl_arch_mbm_cntr_assign_enabled() reports
> false, and this driver always reports true for the L3, so the MBM
> events are always read as assigned counters. Remove the marker, and
> the MBM cases of resctrl_arch_mon_ctx_alloc_no_wait() that returned it.
True but this is a separate, benign, issue with the code.
resctrl_arch_mon_ctx_alloc() is only called for mbm counters when
mbm_assign_mode is default rather than mbm_event. In MPAM, we currently
only support the mbm_event mode, assigning counters. Crossing threads,
but you had an rfc for supporting the software controller on MPAM wher
using the default mbm_assign_mode can be considered. Regardless, please
can you keep the changes in this patch only to those addressing the
particular bug and other changes can be considered as separate patches.
>
> Fixes: a5ff6ef2d73e ("arm_mpam: resctrl: Add resctrl_arch_cntr_read() & resctrl_arch_reset_cntr()")
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Cc: <stable@vger.kernel.org>
> ---
> drivers/resctrl/mpam_internal.h | 15 ---
> drivers/resctrl/mpam_resctrl.c | 195 ++++++++++++--------------------
> 2 files changed, 71 insertions(+), 139 deletions(-)
>
> diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
> index 1f86cc158100..e1348d958d05 100644
> --- a/drivers/resctrl/mpam_internal.h
> +++ b/drivers/resctrl/mpam_internal.h
> @@ -30,14 +30,6 @@ struct platform_device;
> #define PACKED_FOR_KUNIT
> #endif
>
> -/*
> - * This 'mon' values must not alias an actual monitor, so must be larger than
> - * U16_MAX, but not be confused with an errno value, so smaller than
> - * (u32)-SZ_4K.
> - * USE_PRE_ALLOCATED is used to avoid confusion with an actual monitor.
> - */
> -#define USE_PRE_ALLOCATED (U16_MAX + 1)
> -
> static inline bool mpam_is_enabled(void)
> {
> return static_branch_likely(&mpam_enabled);
> @@ -309,10 +301,6 @@ enum mon_filter_options {
> };
>
> struct mon_cfg {
> - /*
> - * mon must be large enough to hold out of range values like
> - * USE_PRE_ALLOCATED
> - */
> u32 mon;
> u8 pmg;
> bool match_pmg;
> @@ -455,9 +443,6 @@ struct mpam_resctrl_res {
> struct mpam_resctrl_mon {
> struct mpam_class *class;
>
> - /* Array of allocated MBWU monitors, indexed by (closid, rmid). */
> - int *mbwu_idx_to_mon;
> -
> /* Array of assigned MBWU monitors, indexed by resctrl's cntr_id. */
> int *assigned_counters;
> };
> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
> index b28c5c04a87c..05c1bf97c43d 100644
> --- a/drivers/resctrl/mpam_resctrl.c
> +++ b/drivers/resctrl/mpam_resctrl.c
> @@ -397,20 +397,13 @@ static int resctrl_arch_mon_ctx_alloc_no_wait(enum resctrl_event_id evtid)
> {
> struct mpam_resctrl_mon *mon = &mpam_resctrl_counters[evtid];
>
> - if (!mpam_is_enabled())
> - return -EINVAL;
> -
> - if (!mon->class)
> + if (!mpam_is_enabled() || !mon->class)
> return -EINVAL;
I don't see any reason for this refactoring.
>
> - switch (evtid) {
> - case QOS_L3_OCCUP_EVENT_ID:
> + if (evtid == QOS_L3_OCCUP_EVENT_ID) {
> /* With CDP, one monitor gets used for both code/data reads */
> return mpam_alloc_csu_mon(mon->class);
> - case QOS_L3_MBM_LOCAL_EVENT_ID:
> - case QOS_L3_MBM_TOTAL_EVENT_ID:
> - return USE_PRE_ALLOCATED;
> - default:
> + } else {
> return -EOPNOTSUPP;
> }
> }
> @@ -464,9 +457,15 @@ void resctrl_arch_mon_ctx_free(struct rdt_resource *r,
> resctrl_arch_mon_ctx_free_no_wait(evtid, mon_idx);
> }
>
> -static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
> - enum mpam_device_features mon_type,
> - int mon_idx,
> +/* The monitor resctrl's counter @cntr_id uses for @cdp_type. */
> +static int mon_for_cntr(struct mpam_resctrl_mon *mon, int cntr_id,
> + enum resctrl_conf_type cdp_type)
> +{
> + return mon->assigned_counters[resctrl_get_config_index(cntr_id, cdp_type)];
> +}
> +
> +static int __read_mon(struct mpam_component *mon_comp,
> + enum mpam_device_features mon_type, int mon_idx,
> enum resctrl_conf_type cdp_type, u32 closid, u32 rmid, u64 *val)
> {
> struct mon_cfg cfg;
> @@ -477,14 +476,6 @@ static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_c
> /* Shift closid to account for CDP */
> closid = resctrl_get_config_index(closid, cdp_type);
>
> - if (mon_idx == USE_PRE_ALLOCATED) {
> - int mbwu_idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> -
> - mon_idx = mon->mbwu_idx_to_mon[mbwu_idx];
> - if (mon_idx == -1)
> - return -ENOENT;
> - }
> -
> if (irqs_disabled()) {
> /* Check if we can access this domain without an IPI */
> return -EIO;
> @@ -500,32 +491,6 @@ static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_c
> return mpam_msmon_read(mon_comp, &cfg, mon_type, val);
> }
>
> -static int read_mon_cdp_safe(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
> - enum mpam_device_features mon_type,
> - int mon_idx, u32 closid, u32 rmid, u64 *val)
> -{
> - if (cdp_enabled) {
> - u64 code_val = 0, data_val = 0;
> - int err;
> -
> - err = __read_mon(mon, mon_comp, mon_type, mon_idx,
> - CDP_CODE, closid, rmid, &code_val);
> - if (err)
> - return err;
> -
> - err = __read_mon(mon, mon_comp, mon_type, mon_idx,
> - CDP_DATA, closid, rmid, &data_val);
> - if (err)
> - return err;
> -
> - *val = code_val + data_val;
> - return 0;
> - }
> -
> - return __read_mon(mon, mon_comp, mon_type, mon_idx,
> - CDP_NONE, closid, rmid, val);
> -}
> -
> /* MBWU when not in ABMC mode (not supported), and CSU counters. */
> int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
> u32 closid, u32 rmid, enum resctrl_event_id eventid,
> @@ -553,13 +518,31 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
>
> mon_type = mpam_feat_msmon_csu;
>
> - return read_mon_cdp_safe(mon, mon_comp, mon_type, mon_idx,
> - closid, rmid, val);
> + if (cdp_enabled) {
> + u64 code_val = 0, data_val = 0;
> + int err;
> +
> + err = __read_mon(mon_comp, mon_type, mon_idx,
> + CDP_CODE, closid, rmid, &code_val);
> + if (err)
> + return err;
> +
> + err = __read_mon(mon_comp, mon_type, mon_idx,
> + CDP_DATA, closid, rmid, &data_val);
> + if (err)
> + return err;
> +
> + *val = code_val + data_val;
> + return 0;
> + }
> +
> + return __read_mon(mon_comp, mon_type, mon_idx,
> + CDP_NONE, closid, rmid, val);
What am I missing? Aren't you just copying the contents of
read_mon_cdp_safe()?
Thanks,
Ben
> }
>
> /* MBWU counters when in ABMC mode */
> int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
> - u32 closid, u32 rmid, int mon_idx,
> + u32 closid, u32 rmid, int cntr_id,
> enum resctrl_event_id eventid, u64 *val)
> {
> struct mpam_resctrl_mon *mon = &mpam_resctrl_counters[eventid];
> @@ -575,43 +558,38 @@ int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
> l3_dom = container_of(d, struct mpam_resctrl_dom, resctrl_mon_dom);
> mon_comp = l3_dom->mon_comp[eventid];
>
> - return read_mon_cdp_safe(mon, mon_comp, mpam_feat_msmon_mbwu,
> - USE_PRE_ALLOCATED, closid, rmid, val);
> -}
> -
> -static void __reset_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
> - int mon_idx,
> - enum resctrl_conf_type cdp_type, u32 closid, u32 rmid)
> -{
> - struct mon_cfg cfg = { };
> -
> - if (!mpam_is_enabled())
> - return;
> + if (cdp_enabled) {
> + u64 code_val = 0, data_val = 0;
> + int err;
>
> - /* Shift closid to account for CDP */
> - closid = resctrl_get_config_index(closid, cdp_type);
> + err = __read_mon(mon_comp, mpam_feat_msmon_mbwu,
> + mon_for_cntr(mon, cntr_id, CDP_CODE),
> + CDP_CODE, closid, rmid, &code_val);
> + if (err)
> + return err;
>
> - if (mon_idx == USE_PRE_ALLOCATED) {
> - int mbwu_idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> + err = __read_mon(mon_comp, mpam_feat_msmon_mbwu,
> + mon_for_cntr(mon, cntr_id, CDP_DATA),
> + CDP_DATA, closid, rmid, &data_val);
> + if (err)
> + return err;
>
> - mon_idx = mon->mbwu_idx_to_mon[mbwu_idx];
> + *val = code_val + data_val;
> + return 0;
> }
>
> - if (mon_idx == -1)
> - return;
> - cfg.mon = mon_idx;
> - mpam_msmon_reset_mbwu(mon_comp, &cfg);
> + return __read_mon(mon_comp, mpam_feat_msmon_mbwu,
> + mon_for_cntr(mon, cntr_id, CDP_NONE),
> + CDP_NONE, closid, rmid, val);
> }
>
> -static void reset_mon_cdp_safe(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
> - int mon_idx, u32 closid, u32 rmid)
> +/* Reset the monitor resctrl's counter @cntr_id uses for @cdp_type. */
> +static void __reset_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
> + int cntr_id, enum resctrl_conf_type cdp_type)
> {
> - if (cdp_enabled) {
> - __reset_mon(mon, mon_comp, mon_idx, CDP_CODE, closid, rmid);
> - __reset_mon(mon, mon_comp, mon_idx, CDP_DATA, closid, rmid);
> - } else {
> - __reset_mon(mon, mon_comp, mon_idx, CDP_NONE, closid, rmid);
> - }
> + struct mon_cfg cfg = { .mon = mon_for_cntr(mon, cntr_id, cdp_type) };
> +
> + mpam_msmon_reset_mbwu(mon_comp, &cfg);
> }
>
> /* Reset an assigned counter */
> @@ -632,7 +610,12 @@ void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d
> l3_dom = container_of(d, struct mpam_resctrl_dom, resctrl_mon_dom);
> mon_comp = l3_dom->mon_comp[eventid];
>
> - reset_mon_cdp_safe(mon, mon_comp, USE_PRE_ALLOCATED, closid, rmid);
> + if (cdp_enabled) {
> + __reset_mon(mon, mon_comp, cntr_id, CDP_CODE);
> + __reset_mon(mon, mon_comp, cntr_id, CDP_DATA);
> + } else {
> + __reset_mon(mon, mon_comp, cntr_id, CDP_NONE);
> + }
> }
>
> /*
> @@ -1170,22 +1153,6 @@ static void mpam_resctrl_pick_counters(void)
> }
> }
>
> -static void __config_cntr(struct mpam_resctrl_mon *mon, u32 cntr_id,
> - enum resctrl_conf_type cdp_type, u32 closid, u32 rmid,
> - bool assign)
> -{
> - /* Same CDP index remap as closid; maps cntr_id to assigned_counters[]. */
> - u32 mbwu_idx, mon_idx = resctrl_get_config_index(cntr_id, cdp_type);
> -
> - closid = resctrl_get_config_index(closid, cdp_type);
> - mbwu_idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> -
> - if (assign)
> - mon->mbwu_idx_to_mon[mbwu_idx] = mon->assigned_counters[mon_idx];
> - else
> - mon->mbwu_idx_to_mon[mbwu_idx] = -1;
> -}
> -
> void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
> enum resctrl_event_id evtid, u32 rmid, u32 closid,
> u32 cntr_id, bool assign)
> @@ -1197,19 +1164,13 @@ void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *
> return;
> }
>
> - if (!mon->mbwu_idx_to_mon || !mon->assigned_counters) {
> - pr_debug("monitor arrays not allocated\n");
> + if (!mon->assigned_counters) {
> + pr_debug("monitor array not allocated\n");
> return;
> }
>
> - if (cdp_enabled) {
> - __config_cntr(mon, cntr_id, CDP_CODE, closid, rmid, assign);
> - __config_cntr(mon, cntr_id, CDP_DATA, closid, rmid, assign);
> - } else {
> - __config_cntr(mon, cntr_id, CDP_NONE, closid, rmid, assign);
> - }
> -
> - resctrl_arch_reset_cntr(r, d, closid, rmid, cntr_id, QOS_L3_MBM_TOTAL_EVENT_ID);
> + if (assign)
> + resctrl_arch_reset_cntr(r, d, closid, rmid, cntr_id, evtid);
> }
>
> static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
> @@ -1296,25 +1257,16 @@ static int mpam_resctrl_pick_domain_id(int cpu, struct mpam_component *comp)
> static int mpam_resctrl_monitor_init_abmc(struct mpam_resctrl_mon *mon)
> {
> struct mpam_resctrl_res *res = &mpam_resctrl_controls[RDT_RESOURCE_L3];
> - size_t num_rmid = resctrl_arch_system_num_rmid_idx();
> struct rdt_resource *l3 = &res->resctrl_res;
> struct mpam_class *class = mon->class;
> u16 num_mbwu_mon;
> int *cntrs;
>
> - int *rmid_array __free(kvfree) = kvmalloc_objs(*rmid_array, num_rmid);
> - if (!rmid_array) {
> - pr_debug("Failed to allocate RMID array\n");
> - return -ENOMEM;
> - }
> - memset(rmid_array, -1, num_rmid * sizeof(*rmid_array));
> -
> num_mbwu_mon = class->props.num_mbwu_mon;
> cntrs = __alloc_mbwu_array(mon->class, num_mbwu_mon);
> if (IS_ERR(cntrs))
> return PTR_ERR(cntrs);
> mon->assigned_counters = cntrs;
> - mon->mbwu_idx_to_mon = no_free_ptr(rmid_array);
>
> l3->mon.mbm_cntr_assignable = true;
> l3->mon.mbm_assign_on_mkdir = true;
> @@ -1943,17 +1895,12 @@ static void mpam_resctrl_teardown_mon(struct mpam_resctrl_mon *mon, struct mpam_
> {
> u32 num_mbwu_mon = class->props.num_mbwu_mon;
>
> - if (!mon->mbwu_idx_to_mon)
> + if (!mon->assigned_counters)
> return;
>
> - if (mon->assigned_counters) {
> - __free_mbwu_mon(class, mon->assigned_counters, num_mbwu_mon);
> - kvfree(mon->assigned_counters);
> - mon->assigned_counters = NULL;
> - }
> -
> - kvfree(mon->mbwu_idx_to_mon);
> - mon->mbwu_idx_to_mon = NULL;
> + __free_mbwu_mon(class, mon->assigned_counters, num_mbwu_mon);
> + kvfree(mon->assigned_counters);
> + mon->assigned_counters = NULL;
> }
>
> void mpam_resctrl_teardown_class(struct mpam_class *class)
next prev parent reply other threads:[~2026-11-17 11:27 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-13 8:35 [PATCH v2 0/2] fs/resctrl: Fix two counter ID bugs tizi4590
2026-11-13 8:35 ` [PATCH v2 1/2] arm_mpam: resctrl: Use the domain's counter ID tizi4590
2026-11-17 11:27 ` Ben Horgan [this message]
2026-11-17 11:48 ` Jinjie Ruan
2026-11-13 8:35 ` [PATCH v2 2/2] fs/resctrl: Give a monitor domain the counters the groups already use tizi4590
2026-11-13 16:14 ` Ben Horgan
2026-11-16 4:02 ` Jinjie Ruan
2026-11-17 3:27 ` Jinjie Ruan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a7dc44ee-cf2c-4e2a-9953-4f8ffeffda60@arm.com \
--to=ben.horgan@arm.com \
--cc=Dave.Martin@arm.com \
--cc=babu.moger@amd.com \
--cc=bp@alien8.de \
--cc=fenghuay@nvidia.com \
--cc=gshan@redhat.com \
--cc=james.morse@arm.com \
--cc=jic23@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=reinette.chatre@intel.com \
--cc=ruanjinjie@huawei.com \
--cc=tony.luck@intel.com \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox