[PATCH v1] fs/resctrl: Allocate rmid_ptrs[] using kvzalloc_objs()

[PATCH v1] fs/resctrl: Allocate rmid_ptrs[] using kvzalloc_objs()

[Ben Horgan]

On an MPAM system the number of entries in rmid_ptrs[] is the number of PARTIDs
multiplied by the number of PMGs. The maximum possible number of PARTIDs is
0x10000 and the maximum number of PMGs is 0x100. On a system pushing the limits
of the MPAM specification this can be large enough to consistently fail the
kzalloc_objs() allocation and hence fail to mount the resctrl filesystem.

Switch to allocating rmid_ptrs[] using kvzalloc_objs() so that large allocations
fall back to vmalloc().

Fixes: 7168ae330e81 ("x86,fs/resctrl: Move the resctrl filesystem code to live in /fs/resctrl")
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
---
I found this by testing the extremes of a software model and don't expect this
to effect real systems.
---
 fs/resctrl/monitor.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index a932a1fea818..b0c588f21de8 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -930,7 +930,7 @@ int setup_rmid_lru_list(void)
 		return 0;
 
 	idx_limit = resctrl_arch_system_num_rmid_idx();
-	rmid_ptrs = kzalloc_objs(struct rmid_entry, idx_limit);
+	rmid_ptrs = kvzalloc_objs(struct rmid_entry, idx_limit);
 	if (!rmid_ptrs)
 		return -ENOMEM;
 
@@ -961,7 +961,7 @@ void free_rmid_lru_list(void)
 		return;
 
 	mutex_lock(&rdtgroup_mutex);
-	kfree(rmid_ptrs);
+	kvfree(rmid_ptrs);
 	rmid_ptrs = NULL;
 	mutex_unlock(&rdtgroup_mutex);
 }
-- 
2.43.0

RE: [PATCH v1] fs/resctrl: Allocate rmid_ptrs[] using kvzalloc_objs()

[Luck, Tony]

> On an MPAM system the number of entries in rmid_ptrs[] is the number of PARTIDs
> multiplied by the number of PMGs. The maximum possible number of PARTIDs is
> 0x10000 and the maximum number of PMGs is 0x100. On a system pushing the limits
> of the MPAM specification this can be large enough to consistently fail the
> kzalloc_objs() allocation and hence fail to mount the resctrl filesystem.

Wow!. 16 million RMIDs * 32 byte per structure = 512MB

I generally don't sweat "big" allocations on servers that have hundreds of Gbytes
of memory. But that seems like a lot. Probably 99.99% of it never used (unless
users regularly use resctrl to create tens of thousands of mon_data directories).

Your fix is fine. So

Reviewed-by: Tony Luck <tony.luck@intel.com>

But maybe at some point resctrl needs some sparse, or on-demand, allocator for
RMIDs?  Or is this a theoretical issue and nobody is likely to build a system that
implements that many PARTIDs and/or PMGs?

-Tony

Re: [PATCH v1] fs/resctrl: Allocate rmid_ptrs[] using kvzalloc_objs()

[Ben Horgan]

Hi Tony,

On 7/10/26 16:42, Luck, Tony wrote:
>> On an MPAM system the number of entries in rmid_ptrs[] is the number of PARTIDs
>> multiplied by the number of PMGs. The maximum possible number of PARTIDs is
>> 0x10000 and the maximum number of PMGs is 0x100. On a system pushing the limits
>> of the MPAM specification this can be large enough to consistently fail the
>> kzalloc_objs() allocation and hence fail to mount the resctrl filesystem.
> 
> Wow!. 16 million RMIDs * 32 byte per structure = 512MB
> 
> I generally don't sweat "big" allocations on servers that have hundreds of Gbytes
> of memory. But that seems like a lot. Probably 99.99% of it never used (unless
> users regularly use resctrl to create tens of thousands of mon_data directories).
> 
> Your fix is fine. So
> 
> Reviewed-by: Tony Luck <tony.luck@intel.com>

Thanks!

> 
> But maybe at some point resctrl needs some sparse, or on-demand, allocator for
> RMIDs?  Or is this a theoretical issue and nobody is likely to build a system that
> implements that many PARTIDs and/or PMGs?

It's a theoretical issue; I don't think anyone will build anything even close to the maximum of 'PMG
x PARTID'. Until the latest version, systems using CHI were limited to 1 bit of PMG. I was just
checking some of the extremes of the MPAM driver with a software model.

Ben

> 
> -Tony

Re: [PATCH v1] fs/resctrl: Allocate rmid_ptrs[] using kvzalloc_objs()

[Reinette Chatre]

Hi Ben,

On 7/10/26 4:23 AM, Ben Horgan wrote:
> On an MPAM system the number of entries in rmid_ptrs[] is the number of PARTIDs
> multiplied by the number of PMGs. The maximum possible number of PARTIDs is
> 0x10000 and the maximum number of PMGs is 0x100. On a system pushing the limits
> of the MPAM specification this can be large enough to consistently fail the
> kzalloc_objs() allocation and hence fail to mount the resctrl filesystem.
> 
> Switch to allocating rmid_ptrs[] using kvzalloc_objs() so that large allocations
> fall back to vmalloc().
> 
> Fixes: 7168ae330e81 ("x86,fs/resctrl: Move the resctrl filesystem code to live in /fs/resctrl")

This is not the patch that added the original allocation but it is the patch
that made it possible for MPAM architecture code to be added. At this point there
is no MPAM support though, could commit 264c285999fc ("arm_mpam: resctrl: Add monitor
initialisation and domain boilerplate") perhaps be more accurate to present when
this change may theoretically be needed?

> Signed-off-by: Ben Horgan <ben.horgan@arm.com>
> ---
Reinette