[PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[James Morse]

Hello!

This is the final series that allows other architectures to implement resctrl.
The last patch just moves the code, and its a bit of a monster. I don't expect
that to get merged as part of this series - we should wait for it to make
less impact on other series. It's included here to show what gets moved, and
that structures/function-prototypes have the right visibility.

Otherwise this series renames functions and moves code around. With the
exception of invalid configurations for the configurable-events, there should
be no changes in behaviour caused by this series.

The driving pattern is to make things like struct rdtgroup private to resctrl.
Features like pseudo-lock aren't going to work on arm64, the ability to disable
it at compile time is added.

After this, I can start posting the MPAM driver to make use of resctrl on arm64.
(What's MPAM? See the cover letter of the first series. [1])

This series is based on Linus' commit 23956900041d and can be retrieved from:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git
mpam/move_to_fs/v1

Sorry for the mid-merge window base, I'm away for a few weeks - this should
rebase trivially onto rc1.

As ever - bugs welcome,
Thanks,

James

[1] https://lore.kernel.org/lkml/20201030161120.227225-1-james.morse@arm.com/

James Morse (31):
  x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no
    monitors
  x86/resctrl: Add a helper to avoid reaching into the arch code
    resource list
  x86/resctrl: Move ctrlval string parsing policy away from the arch
    code
  x86/resctrl: Add helper for setting CPU default properties
  x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()
  x86/resctrl: Export resctrl fs's init function
  x86/resctrl: Wrap resctrl_arch_find_domain() around rdt_find_domain()
  x86/resctrl: Move resctrl types to a separate header
  x86/resctrl: Add a resctrl helper to reset all the resources
  x86/resctrl: Move monitor init work to a resctrl init call
  x86/resctrl: Move monitor exit work to a restrl exit call
  x86/resctrl: Move max_{name,data}_width into resctrl code
  x86/resctrl: Stop using the for_each_*_rdt_resource() walkers
  x86/resctrl: Export the is_mbm_*_enabled() helpers to asm/resctrl.h
  x86/resctrl: Add resctrl_arch_is_evt_configurable() to abstract BMEC
  x86/resctrl: Change mon_event_config_{read,write}() to be arch helpers
  x86/resctrl: Move mbm_cfg_mask to struct rdt_resource
  x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an
    error
  x86/resctrl: Add resctrl_arch_ prefix to pseudo lock functions
  x86/resctrl: Allow an architecture to disable pseudo lock
  x86/resctrl: Make prefetch_disable_bits belong to the arch code
  x86/resctrl: Make resctrl_arch_pseudo_lock_fn() take a plr
  x86/resctrl: Move thread_throttle_mode_init() to be managed by resctrl
  x86/resctrl: Move get_config_index() to a header
  x86/resctrl: Claim get_domain_from_cpu() for resctrl
  x86/resctrl: Describe resctrl's bitmap size assumptions
  x86/resctrl: Rename resctrl_sched_in() to begin resctrl_arch_
  x86/resctrl: Drop __init/__exit on assorted symbols
  fs/resctrl: Add boiler plate for external resctrl code
  x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl
  x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

 MAINTAINERS                               |    2 +
 arch/Kconfig                              |    8 +
 arch/x86/Kconfig                          |    5 +-
 arch/x86/include/asm/resctrl.h            |   45 +-
 arch/x86/kernel/cpu/resctrl/Makefile      |    5 +-
 arch/x86/kernel/cpu/resctrl/core.c        |  119 +-
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  506 +--
 arch/x86/kernel/cpu/resctrl/internal.h    |  433 +--
 arch/x86/kernel/cpu/resctrl/monitor.c     |  813 +----
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1126 +-----
 arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3929 +-------------------
 arch/x86/kernel/process_32.c              |    2 +-
 arch/x86/kernel/process_64.c              |    2 +-
 fs/Kconfig                                |    1 +
 fs/Makefile                               |    1 +
 fs/resctrl/Kconfig                        |   23 +
 fs/resctrl/Makefile                       |    3 +
 fs/resctrl/ctrlmondata.c                  |  527 +++
 fs/resctrl/internal.h                     |  340 ++
 fs/resctrl/monitor.c                      |  843 +++++
 fs/resctrl/psuedo_lock.c                  | 1122 ++++++
 fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
 include/linux/resctrl.h                   |  153 +-
 include/linux/resctrl_types.h             |   98 +
 24 files changed, 7244 insertions(+), 6875 deletions(-)
 create mode 100644 fs/resctrl/Kconfig
 create mode 100644 fs/resctrl/Makefile
 create mode 100644 fs/resctrl/ctrlmondata.c
 create mode 100644 fs/resctrl/internal.h
 create mode 100644 fs/resctrl/monitor.c
 create mode 100644 fs/resctrl/psuedo_lock.c
 create mode 100644 fs/resctrl/rdtgroup.c
 create mode 100644 include/linux/resctrl_types.h

-- 
2.39.2

[PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[James Morse]

commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
searching closid_num_dirty_rmid") added a Kconfig option that causes
resctrl to search for the CLOSID with the fewest dirty cache lines when
creating a new control group. This depends on the values read from the
llc_occupancy counters.

This support missed that some platforms may not have these counters.
This causes a NULL pointer dereference when creating a new control
group as the array was not allocated by dom_data_init().

As this feature isn't necessary on platforms that don't have cache
occupancy monitors, add this to the check that occurs when a new
control group is allocated.

The existing code is not selected by any upstream platform, it makes
no sense to backport this patch to stable.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 011e17efb1a6..1767c1affa60 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -149,7 +149,8 @@ static int closid_alloc(void)
 
 	lockdep_assert_held(&rdtgroup_mutex);
 
-	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
+	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
+	    is_llc_occupancy_enabled()) {
 		cleanest_closid = resctrl_find_cleanest_closid();
 		if (cleanest_closid < 0)
 			return cleanest_closid;
-- 
2.39.2

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> searching closid_num_dirty_rmid") added a Kconfig option that causes

Please see section about "Fixes" tags in section "Ordering of commit tags"
in Documentation/process/maintainer-tip.rst

> resctrl to search for the CLOSID with the fewest dirty cache lines when
> creating a new control group. This depends on the values read from the
> llc_occupancy counters.
> 
> This support missed that some platforms may not have these counters.
> This causes a NULL pointer dereference when creating a new control
> group as the array was not allocated by dom_data_init().
> 
> As this feature isn't necessary on platforms that don't have cache
> occupancy monitors, add this to the check that occurs when a new
> control group is allocated.
> 
> The existing code is not selected by any upstream platform, it makes
> no sense to backport this patch to stable.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 011e17efb1a6..1767c1affa60 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -149,7 +149,8 @@ static int closid_alloc(void)
>  
>  	lockdep_assert_held(&rdtgroup_mutex);
>  
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> +	    is_llc_occupancy_enabled()) {
>  		cleanest_closid = resctrl_find_cleanest_closid();
>  		if (cleanest_closid < 0)
>  			return cleanest_closid;

Patch looks good.

Thank you

Reinette

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Dave Martin]

On Mon, Apr 08, 2024 at 08:13:40PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> > searching closid_num_dirty_rmid") added a Kconfig option that causes
> 
> Please see section about "Fixes" tags in section "Ordering of commit tags"
> in Documentation/process/maintainer-tip.rst

Noted for James' attention.

> > resctrl to search for the CLOSID with the fewest dirty cache lines when
> > creating a new control group. This depends on the values read from the
> > llc_occupancy counters.
> > 
> > This support missed that some platforms may not have these counters.
> > This causes a NULL pointer dereference when creating a new control
> > group as the array was not allocated by dom_data_init().
> > 
> > As this feature isn't necessary on platforms that don't have cache
> > occupancy monitors, add this to the check that occurs when a new
> > control group is allocated.
> > 
> > The existing code is not selected by any upstream platform, it makes
> > no sense to backport this patch to stable.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index 011e17efb1a6..1767c1affa60 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > @@ -149,7 +149,8 @@ static int closid_alloc(void)
> >  
> >  	lockdep_assert_held(&rdtgroup_mutex);
> >  
> > -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> > +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> > +	    is_llc_occupancy_enabled()) {
> >  		cleanest_closid = resctrl_find_cleanest_closid();
> >  		if (cleanest_closid < 0)
> >  			return cleanest_closid;
> 
> Patch looks good.
> 
> Thank you
> 
> Reinette
> 

Noted, thanks.

Cheers
---Dave

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[David Hildenbrand]

On 21.03.24 17:50, James Morse wrote:
> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> searching closid_num_dirty_rmid") added a Kconfig option that causes
> resctrl to search for the CLOSID with the fewest dirty cache lines when
> creating a new control group. This depends on the values read from the
> llc_occupancy counters.
> 
> This support missed that some platforms may not have these counters.
> This causes a NULL pointer dereference when creating a new control
> group as the array was not allocated by dom_data_init().
> 
> As this feature isn't necessary on platforms that don't have cache
> occupancy monitors, add this to the check that occurs when a new
> control group is allocated.
> 
> The existing code is not selected by any upstream platform, it makes
> no sense to backport this patch to stable.
> 

It's weird to not see RESCTRL_RMID_DEPENDS_ON_CLOSID appear in any Kconfig file.
I guess it will all make sense once the refactoring is done :)

As Reinette comments, likely we want here:

Fixes: 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid")

> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 011e17efb1a6..1767c1affa60 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -149,7 +149,8 @@ static int closid_alloc(void)
>   
>   	lockdep_assert_held(&rdtgroup_mutex);
>   
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> +	    is_llc_occupancy_enabled()) {
>   		cleanest_closid = resctrl_find_cleanest_closid();
>   		if (cleanest_closid < 0)
>   			return cleanest_closid;

Reviewed-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Reinette Chatre]

Hi David,

(Thank you very much for taking a look at these)

On 4/9/2024 1:05 AM, David Hildenbrand wrote:
> On 21.03.24 17:50, James Morse wrote:
>> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
>> searching closid_num_dirty_rmid") added a Kconfig option that causes
>> resctrl to search for the CLOSID with the fewest dirty cache lines when
>> creating a new control group. This depends on the values read from the
>> llc_occupancy counters.
>>
>> This support missed that some platforms may not have these counters.
>> This causes a NULL pointer dereference when creating a new control
>> group as the array was not allocated by dom_data_init().
>>
>> As this feature isn't necessary on platforms that don't have cache
>> occupancy monitors, add this to the check that occurs when a new
>> control group is allocated.
>>
>> The existing code is not selected by any upstream platform, it makes
>> no sense to backport this patch to stable.
>>
> 
> It's weird to not see RESCTRL_RMID_DEPENDS_ON_CLOSID appear in any Kconfig file.
> I guess it will all make sense once the refactoring is done :)

This is done later in this series where patch #29, "fs/resctrl: Add boiler
plate for external resctrl code", introduces it to fs/resctrl/Kconfig.

Reinette

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[David Hildenbrand]

On 09.04.24 17:02, Reinette Chatre wrote:
> Hi David,
> 

Hi,

> (Thank you very much for taking a look at these)

I'm planning on reviewing more; as most of resctrl is code I haven't 
really read before, I'm not able to make progress as fast as I normally 
would in core-MM ... :)

> 
> On 4/9/2024 1:05 AM, David Hildenbrand wrote:
>> On 21.03.24 17:50, James Morse wrote:
>>> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
>>> searching closid_num_dirty_rmid") added a Kconfig option that causes
>>> resctrl to search for the CLOSID with the fewest dirty cache lines when
>>> creating a new control group. This depends on the values read from the
>>> llc_occupancy counters.
>>>
>>> This support missed that some platforms may not have these counters.
>>> This causes a NULL pointer dereference when creating a new control
>>> group as the array was not allocated by dom_data_init().
>>>
>>> As this feature isn't necessary on platforms that don't have cache
>>> occupancy monitors, add this to the check that occurs when a new
>>> control group is allocated.
>>>
>>> The existing code is not selected by any upstream platform, it makes
>>> no sense to backport this patch to stable.
>>>
>>
>> It's weird to not see RESCTRL_RMID_DEPENDS_ON_CLOSID appear in any Kconfig file.
>> I guess it will all make sense once the refactoring is done :)
> 
> This is done later in this series where patch #29, "fs/resctrl: Add boiler
> plate for external resctrl code", introduces it to fs/resctrl/Kconfig.

Oh, already in this series, great!

-- 
Cheers,

David / dhildenb

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Dave Martin]

On Tue, Apr 09, 2024 at 05:06:03PM +0200, David Hildenbrand wrote:
> On 09.04.24 17:02, Reinette Chatre wrote:
> > Hi David,
> > 
> 
> Hi,
> 
> > (Thank you very much for taking a look at these)
> 
> I'm planning on reviewing more; as most of resctrl is code I haven't really
> read before, I'm not able to make progress as fast as I normally would in
> core-MM ... :)

[...]

> -- 
> Cheers,
> 
> David / dhildenb

Thanks, all review is appreciated (I'm somewhat in the same boat myself,
but gradually finding my way around...)

Cheers
---Dave

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Dave Martin]

On Tue, Apr 09, 2024 at 10:05:33AM +0200, David Hildenbrand wrote:
> On 21.03.24 17:50, James Morse wrote:
> > commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> > searching closid_num_dirty_rmid") added a Kconfig option that causes
> > resctrl to search for the CLOSID with the fewest dirty cache lines when
> > creating a new control group. This depends on the values read from the
> > llc_occupancy counters.

[...]

> It's weird to not see RESCTRL_RMID_DEPENDS_ON_CLOSID appear in any Kconfig file.
> I guess it will all make sense once the refactoring is done :)

Agreed; a stub Kconfig item could be added, but since the file layout
and naming conventions change after this patch, doing this would
probably just create noise in the series though.

Looking at <linux/kconfig.h> (yikes!), IS_ENABLED() is designed to do
the right thing for non-existing Kconfigs...

If nobody is too concerned about the temporarily dangling IS_ENABLED()s
in this series, I won't propose any change here.


> As Reinette comments, likely we want here:
> 
> Fixes: 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid")

Noted for James' attention.

> 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >   arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
> >   1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index 011e17efb1a6..1767c1affa60 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > @@ -149,7 +149,8 @@ static int closid_alloc(void)
> >   	lockdep_assert_held(&rdtgroup_mutex);
> > -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> > +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> > +	    is_llc_occupancy_enabled()) {
> >   		cleanest_closid = resctrl_find_cleanest_closid();
> >   		if (cleanest_closid < 0)
> >   			return cleanest_closid;
> 
> Reviewed-by: David Hildenbrand <david@redhat.com>
> 
> -- 
> Cheers,
> 
> David / dhildenb

Thanks; noted for James' attention.

Cheers
---Dave

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:13 AM, Dave Martin wrote:
> On Tue, Apr 09, 2024 at 10:05:33AM +0200, David Hildenbrand wrote:
>> On 21.03.24 17:50, James Morse wrote:
>>> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
>>> searching closid_num_dirty_rmid") added a Kconfig option that causes
>>> resctrl to search for the CLOSID with the fewest dirty cache lines when
>>> creating a new control group. This depends on the values read from the
>>> llc_occupancy counters.
> 
> [...]
> 
>> It's weird to not see RESCTRL_RMID_DEPENDS_ON_CLOSID appear in any Kconfig file.
>> I guess it will all make sense once the refactoring is done :)
> 
> Agreed; a stub Kconfig item could be added, but since the file layout
> and naming conventions change after this patch, doing this would
> probably just create noise in the series though.
> 
> Looking at <linux/kconfig.h> (yikes!), IS_ENABLED() is designed to do
> the right thing for non-existing Kconfigs...
> 
> If nobody is too concerned about the temporarily dangling IS_ENABLED()s
> in this series, I won't propose any change here.

I am not concerned about this. Please note that these IS_ENABLED() checks
are not introduced in this series, they were introduced in the previous
portion of this MPAM work that can be found in upstream resctrl.

Reinette

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Dave Martin]

On Thu, Apr 11, 2024 at 10:35:56AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:13 AM, Dave Martin wrote:
> > On Tue, Apr 09, 2024 at 10:05:33AM +0200, David Hildenbrand wrote:
> >> On 21.03.24 17:50, James Morse wrote:
> >>> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> >>> searching closid_num_dirty_rmid") added a Kconfig option that causes
> >>> resctrl to search for the CLOSID with the fewest dirty cache lines when
> >>> creating a new control group. This depends on the values read from the
> >>> llc_occupancy counters.
> > 
> > [...]
> > 
> >> It's weird to not see RESCTRL_RMID_DEPENDS_ON_CLOSID appear in any Kconfig file.
> >> I guess it will all make sense once the refactoring is done :)
> > 
> > Agreed; a stub Kconfig item could be added, but since the file layout
> > and naming conventions change after this patch, doing this would
> > probably just create noise in the series though.
> > 
> > Looking at <linux/kconfig.h> (yikes!), IS_ENABLED() is designed to do
> > the right thing for non-existing Kconfigs...
> > 
> > If nobody is too concerned about the temporarily dangling IS_ENABLED()s
> > in this series, I won't propose any change here.
> 
> I am not concerned about this. Please note that these IS_ENABLED() checks
> are not introduced in this series, they were introduced in the previous
> portion of this MPAM work that can be found in upstream resctrl.
> 
> Reinette
> 

Ah, right.  Thanks for the explanation.

Cheers
---Dave

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Moger, Babu]

On 3/21/24 11:50, James Morse wrote:
> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> searching closid_num_dirty_rmid") added a Kconfig option that causes

This is not true. The Kconfig option is never added. It is added later in
the series. There is also comment
on this https://lore.kernel.org/lkml/ZhecyLQsGZ9Iv8wU@gmail.com/


Shouldn't the Kconfig option added first before doing this change?

> resctrl to search for the CLOSID with the fewest dirty cache lines when
> creating a new control group. This depends on the values read from the
> llc_occupancy counters.
> 
> This support missed that some platforms may not have these counters.
> This causes a NULL pointer dereference when creating a new control
> group as the array was not allocated by dom_data_init().
> 
> As this feature isn't necessary on platforms that don't have cache
> occupancy monitors, add this to the check that occurs when a new
> control group is allocated.
> 
> The existing code is not selected by any upstream platform, it makes
> no sense to backport this patch to stable.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 011e17efb1a6..1767c1affa60 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -149,7 +149,8 @@ static int closid_alloc(void)
>  
>  	lockdep_assert_held(&rdtgroup_mutex);
>  
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> +	    is_llc_occupancy_enabled()) {
>  		cleanest_closid = resctrl_find_cleanest_closid();
>  		if (cleanest_closid < 0)
>  			return cleanest_closid;

-- 
Thanks
Babu Moger

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Dave Martin]

On Mon, Apr 15, 2024 at 03:27:42PM -0500, Moger, Babu wrote:
> 
> 
> On 3/21/24 11:50, James Morse wrote:
> > commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> > searching closid_num_dirty_rmid") added a Kconfig option that causes
> 
> This is not true. The Kconfig option is never added. It is added later in
> the series. There is also comment
> on this https://lore.kernel.org/lkml/ZhecyLQsGZ9Iv8wU@gmail.com/
> 
> 
> Shouldn't the Kconfig option added first before doing this change?
> 
> > resctrl to search for the CLOSID with the fewest dirty cache lines when
> > creating a new control group. This depends on the values read from the
> > llc_occupancy counters.

See David's comments and previous discussion on this patch.

You're right to point out that the description of the original commit
does seem a bit garbled though: CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is
not present in Kconfig here, but already referenced by other code.

We seem to have a consensus that it's OK to have a dangling IS_ENABLED()
so long as the option is added formally to Kconfig later, but it looks
like the commit message here should be reworded.

Does the following make sense?

--8<--

commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
searching closid_num_dirty_rmid") added logic that causes resctrl to
search for the CLOSID with the fewest dirty cache lines when creating a
new control group, if requested by the arch code. This depends on the
values read from the llc_occupancy counters. The logic is applicable to
architectures where the CLOSID effectively forms part of the monitoring
identifier and so do not allow complete freedom to choose an unused
monitoring identifier for a given CLOSID.

-->8--


Although it would probably have been better if the Kconfig option had
been added upstream, this patch does not create that that situation and
the series (taken as a whole) resolves it.

So I am not sure that anything would be solved or improved by changing
the body of this patch, but if people still have concerns then I guess
we can look at it.


> > 
> > This support missed that some platforms may not have these counters.
> > This causes a NULL pointer dereference when creating a new control
> > group as the array was not allocated by dom_data_init().
> > 
> > As this feature isn't necessary on platforms that don't have cache
> > occupancy monitors, add this to the check that occurs when a new
> > control group is allocated.
> > 
> > The existing code is not selected by any upstream platform, it makes
> > no sense to backport this patch to stable.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index 011e17efb1a6..1767c1affa60 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > @@ -149,7 +149,8 @@ static int closid_alloc(void)
> >  
> >  	lockdep_assert_held(&rdtgroup_mutex);
> >  
> > -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> > +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> > +	    is_llc_occupancy_enabled()) {
> >  		cleanest_closid = resctrl_find_cleanest_closid();
> >  		if (cleanest_closid < 0)
> >  			return cleanest_closid;
> 
> -- 
> Thanks
> Babu Moger
> 

[...]

Cheers
---Dave

Re: [PATCH v1 01/31] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors

[Moger, Babu]

Hi Dave,

On 4/16/24 11:13, Dave Martin wrote:
> On Mon, Apr 15, 2024 at 03:27:42PM -0500, Moger, Babu wrote:
>>
>>
>> On 3/21/24 11:50, James Morse wrote:
>>> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
>>> searching closid_num_dirty_rmid") added a Kconfig option that causes
>>
>> This is not true. The Kconfig option is never added. It is added later in
>> the series. There is also comment
>> on this https://lore.kernel.org/lkml/ZhecyLQsGZ9Iv8wU@gmail.com/
>>
>>
>> Shouldn't the Kconfig option added first before doing this change?
>>
>>> resctrl to search for the CLOSID with the fewest dirty cache lines when
>>> creating a new control group. This depends on the values read from the
>>> llc_occupancy counters.
> 
> See David's comments and previous discussion on this patch.
> 
> You're right to point out that the description of the original commit
> does seem a bit garbled though: CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is
> not present in Kconfig here, but already referenced by other code.
> 
> We seem to have a consensus that it's OK to have a dangling IS_ENABLED()
> so long as the option is added formally to Kconfig later, but it looks
> like the commit message here should be reworded.
> 
> Does the following make sense?
> 
> --8<--
> 
> commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by
> searching closid_num_dirty_rmid") added logic that causes resctrl to
> search for the CLOSID with the fewest dirty cache lines when creating a
> new control group, if requested by the arch code. This depends on the
> values read from the llc_occupancy counters. The logic is applicable to
> architectures where the CLOSID effectively forms part of the monitoring
> identifier and so do not allow complete freedom to choose an unused
> monitoring identifier for a given CLOSID.
> 
> -->8--

That looks good. Thanks

> 
> 
> Although it would probably have been better if the Kconfig option had
> been added upstream, this patch does not create that that situation and
> the series (taken as a whole) resolves it.
> 
> So I am not sure that anything would be solved or improved by changing
> the body of this patch, but if people still have concerns then I guess
> we can look at it.

> 
> 
>>>
>>> This support missed that some platforms may not have these counters.
>>> This causes a NULL pointer dereference when creating a new control
>>> group as the array was not allocated by dom_data_init().
>>>
>>> As this feature isn't necessary on platforms that don't have cache
>>> occupancy monitors, add this to the check that occurs when a new
>>> control group is allocated.
>>>
>>> The existing code is not selected by any upstream platform, it makes
>>> no sense to backport this patch to stable.
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>> ---
>>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++-
>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> index 011e17efb1a6..1767c1affa60 100644
>>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> @@ -149,7 +149,8 @@ static int closid_alloc(void)
>>>  
>>>  	lockdep_assert_held(&rdtgroup_mutex);
>>>  
>>> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
>>> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
>>> +	    is_llc_occupancy_enabled()) {
>>>  		cleanest_closid = resctrl_find_cleanest_closid();
>>>  		if (cleanest_closid < 0)
>>>  			return cleanest_closid;
>>
>> -- 
>> Thanks
>> Babu Moger
>>
> 
> [...]
> 
> Cheers
> ---Dave

-- 
Thanks
Babu Moger

[PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[James Morse]

Resctrl occasionally wants to know something about a specific resource,
in these cases it reaches into the arch code's rdt_resources_all[]
array.

Once the filesystem parts of resctrl are moved to /fs/, this means it
will need visibility of the architecture specific struct
resctrl_hw_resource definition, and the array of all resources.
All architectures would also need a r_resctrl member in this struct.

Instead, abstract this via a helper to allow architectures to do
different things here. Move the level enum to the resctrl header and
add a helper to retrieve the struct rdt_resource by 'rid'.

resctrl_arch_get_resource() should not return NULL for any value in
the enum, it may instead return a dummy resource that is
!alloc_enabled && !mon_enabled.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c        | 10 +++++++++-
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  2 +-
 arch/x86/kernel/cpu/resctrl/internal.h    | 10 ----------
 arch/x86/kernel/cpu/resctrl/monitor.c     |  8 ++++----
 arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 15 +++++++--------
 include/linux/resctrl.h                   | 17 +++++++++++++++++
 6 files changed, 38 insertions(+), 24 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 83e40341583e..b773b3bdebe3 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -122,6 +122,14 @@ struct rdt_hw_resource rdt_resources_all[] = {
 	},
 };
 
+struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l)
+{
+	if (l >= RDT_NUM_RESOURCES)
+		return NULL;
+
+	return &rdt_resources_all[l].r_resctrl;
+}
+
 /*
  * cache_alloc_hsw_probe() - Have to probe for Intel haswell server CPUs
  * as they do not have CPUID enumeration support for Cache allocation.
@@ -169,7 +177,7 @@ static inline void cache_alloc_hsw_probe(void)
 bool is_mba_sc(struct rdt_resource *r)
 {
 	if (!r)
-		return rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl.membw.mba_sc;
+		r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
 
 	/*
 	 * The software controller support is only applicable to MBA resource.
diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index 7997b47743a2..788ac0395645 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -599,7 +599,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)
 	domid = md.u.domid;
 	evtid = md.u.evtid;
 
-	r = &rdt_resources_all[resid].r_resctrl;
+	r = resctrl_arch_get_resource(resid);
 	d = rdt_find_domain(r, domid, NULL);
 	if (IS_ERR_OR_NULL(d)) {
 		ret = -ENOENT;
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index c99f26ebe7a6..65990def6c79 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -466,16 +466,6 @@ extern struct rdt_hw_resource rdt_resources_all[];
 extern struct rdtgroup rdtgroup_default;
 extern struct dentry *debugfs_resctrl;
 
-enum resctrl_res_level {
-	RDT_RESOURCE_L3,
-	RDT_RESOURCE_L2,
-	RDT_RESOURCE_MBA,
-	RDT_RESOURCE_SMBA,
-
-	/* Must be the last */
-	RDT_NUM_RESOURCES,
-};
-
 static inline struct rdt_resource *resctrl_inc(struct rdt_resource *res)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(res);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index c34a35ec0f03..06565153ceb2 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -321,7 +321,7 @@ static void limbo_release_entry(struct rmid_entry *entry)
  */
 void __check_limbo(struct rdt_domain *d, bool force_free)
 {
-	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
 	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
 	struct rmid_entry *entry;
 	u32 idx, cur_idx = 1;
@@ -467,7 +467,7 @@ int alloc_rmid(u32 closid)
 
 static void add_rmid_to_limbo(struct rmid_entry *entry)
 {
-	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
 	struct rdt_domain *d;
 	u32 idx;
 
@@ -669,7 +669,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
 	if (!is_mbm_local_enabled())
 		return;
 
-	r_mba = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
+	r_mba = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
 
 	closid = rgrp->closid;
 	rmid = rgrp->mon.rmid;
@@ -839,7 +839,7 @@ void mbm_handle_overflow(struct work_struct *work)
 	if (!resctrl_mounted || !resctrl_arch_mon_capable())
 		goto out_unlock;
 
-	r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+	r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
 	d = container_of(work, struct rdt_domain, mbm_over.work);
 
 	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 1767c1affa60..45372b6a6215 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -2253,7 +2253,7 @@ static void l2_qos_cfg_update(void *arg)
 
 static inline bool is_mba_linear(void)
 {
-	return rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl.membw.delay_linear;
+	return resctrl_arch_get_resource(RDT_RESOURCE_MBA)->membw.delay_linear;
 }
 
 static int set_cache_qos_cfg(int level, bool enable)
@@ -2341,7 +2341,7 @@ static void mba_sc_domain_destroy(struct rdt_resource *r,
  */
 static bool supports_mba_mbps(void)
 {
-	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
 
 	return (is_mbm_local_enabled() &&
 		r->alloc_capable && is_mba_linear());
@@ -2353,7 +2353,7 @@ static bool supports_mba_mbps(void)
  */
 static int set_mba_sc(bool mba_sc)
 {
-	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
 	u32 num_closid = resctrl_arch_get_num_closid(r);
 	struct rdt_domain *d;
 	int i;
@@ -2625,10 +2625,10 @@ static void schemata_list_destroy(void)
 
 static int rdt_get_tree(struct fs_context *fc)
 {
+	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
 	struct rdt_fs_context *ctx = rdt_fc2context(fc);
 	unsigned long flags = RFTYPE_CTRL_BASE;
 	struct rdt_domain *dom;
-	struct rdt_resource *r;
 	int ret;
 
 	cpus_read_lock();
@@ -2701,8 +2701,7 @@ static int rdt_get_tree(struct fs_context *fc)
 		resctrl_mounted = true;
 
 	if (is_mbm_enabled()) {
-		r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
-		list_for_each_entry(dom, &r->domains, list)
+		list_for_each_entry(dom, &l3->domains, list)
 			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
 						   RESCTRL_PICK_ANY_CPU);
 	}
@@ -3878,7 +3877,7 @@ static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
 	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2))
 		seq_puts(seq, ",cdpl2");
 
-	if (is_mba_sc(&rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl))
+	if (is_mba_sc(resctrl_arch_get_resource(RDT_RESOURCE_MBA)))
 		seq_puts(seq, ",mba_MBps");
 
 	if (resctrl_debug)
@@ -4068,7 +4067,7 @@ static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)
 
 void resctrl_offline_cpu(unsigned int cpu)
 {
-	struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
 	struct rdtgroup *rdtgrp;
 	struct rdt_domain *d;
 
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index a365f67131ec..168cc9510069 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -36,6 +36,16 @@ enum resctrl_conf_type {
 	CDP_DATA,
 };
 
+enum resctrl_res_level {
+	RDT_RESOURCE_L3,
+	RDT_RESOURCE_L2,
+	RDT_RESOURCE_MBA,
+	RDT_RESOURCE_SMBA,
+
+	/* Must be the last */
+	RDT_NUM_RESOURCES,
+};
+
 #define CDP_NUM_TYPES	(CDP_DATA + 1)
 
 /*
@@ -190,6 +200,13 @@ struct rdt_resource {
 	bool			cdp_capable;
 };
 
+/*
+ * Get the resource that exists at this level. If the level is not supported
+ * a dummy/not-capable resource can be returned. Levels >= RDT_NUM_RESOURCES
+ * will return NULL.
+ */
+struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l);
+
 /**
  * struct resctrl_schema - configuration abilities of a resource presented to
  *			   user-space
-- 
2.39.2

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> Resctrl occasionally wants to know something about a specific resource,
> in these cases it reaches into the arch code's rdt_resources_all[]
> array.
> 
> Once the filesystem parts of resctrl are moved to /fs/, this means it
> will need visibility of the architecture specific struct
> resctrl_hw_resource definition, and the array of all resources.

rdt_hw_resource?

> All architectures would also need a r_resctrl member in this struct.
> 
> Instead, abstract this via a helper to allow architectures to do
> different things here. Move the level enum to the resctrl header and
> add a helper to retrieve the struct rdt_resource by 'rid'.
> 
> resctrl_arch_get_resource() should not return NULL for any value in
> the enum, it may instead return a dummy resource that is
> !alloc_enabled && !mon_enabled.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---

Patch looks good to me.

Reinette

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[Dave Martin]

On Mon, Apr 08, 2024 at 08:13:58PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > Resctrl occasionally wants to know something about a specific resource,
> > in these cases it reaches into the arch code's rdt_resources_all[]
> > array.
> > 
> > Once the filesystem parts of resctrl are moved to /fs/, this means it
> > will need visibility of the architecture specific struct
> > resctrl_hw_resource definition, and the array of all resources.
> 
> rdt_hw_resource?

Looks like it.  Noted for James' attention.


> > All architectures would also need a r_resctrl member in this struct.
> > 
> > Instead, abstract this via a helper to allow architectures to do
> > different things here. Move the level enum to the resctrl header and
> > add a helper to retrieve the struct rdt_resource by 'rid'.
> > 
> > resctrl_arch_get_resource() should not return NULL for any value in
> > the enum, it may instead return a dummy resource that is
> > !alloc_enabled && !mon_enabled.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> 
> Patch looks good to me.
> 
> Reinette
> 

Noted, thanks.

Cheers
---Dave

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[David Hildenbrand]

On 21.03.24 17:50, James Morse wrote:
> Resctrl occasionally wants to know something about a specific resource,
> in these cases it reaches into the arch code's rdt_resources_all[]
> array.
> 
> Once the filesystem parts of resctrl are moved to /fs/, this means it
> will need visibility of the architecture specific struct
> resctrl_hw_resource definition, and the array of all resources.
> All architectures would also need a r_resctrl member in this struct.
> 
> Instead, abstract this via a helper to allow architectures to do
> different things here. Move the level enum to the resctrl header and
> add a helper to retrieve the struct rdt_resource by 'rid'.
> 
> resctrl_arch_get_resource() should not return NULL for any value in
> the enum, it may instead return a dummy resource that is
> !alloc_enabled && !mon_enabled.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---

Reviewed-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[Dave Martin]

On Tue, Apr 09, 2024 at 10:09:20AM +0200, David Hildenbrand wrote:
> On 21.03.24 17:50, James Morse wrote:
> > Resctrl occasionally wants to know something about a specific resource,
> > in these cases it reaches into the arch code's rdt_resources_all[]
> > array.
> > 
> > Once the filesystem parts of resctrl are moved to /fs/, this means it
> > will need visibility of the architecture specific struct
> > resctrl_hw_resource definition, and the array of all resources.
> > All architectures would also need a r_resctrl member in this struct.
> > 
> > Instead, abstract this via a helper to allow architectures to do
> > different things here. Move the level enum to the resctrl header and
> > add a helper to retrieve the struct rdt_resource by 'rid'.
> > 
> > resctrl_arch_get_resource() should not return NULL for any value in
> > the enum, it may instead return a dummy resource that is
> > !alloc_enabled && !mon_enabled.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> 
> Reviewed-by: David Hildenbrand <david@redhat.com>
> 
> -- 
> Cheers,
> 
> David / dhildenb

Thanks; noted for James' attention.

Cheers
---Dave
> 
> 

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[Moger, Babu]

Hi James/Dave,

On 3/21/24 11:50, James Morse wrote:
> Resctrl occasionally wants to know something about a specific resource,
> in these cases it reaches into the arch code's rdt_resources_all[]
> array.
> 
> Once the filesystem parts of resctrl are moved to /fs/, this means it
> will need visibility of the architecture specific struct
> resctrl_hw_resource definition, and the array of all resources.
> All architectures would also need a r_resctrl member in this struct.
> 
> Instead, abstract this via a helper to allow architectures to do
> different things here. Move the level enum to the resctrl header and
> add a helper to retrieve the struct rdt_resource by 'rid'.
> 
> resctrl_arch_get_resource() should not return NULL for any value in
> the enum, it may instead return a dummy resource that is
> !alloc_enabled && !mon_enabled.

Nit.
You may want to drop the second half of the statement. We don't have a
dummy resource.

> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/core.c        | 10 +++++++++-
>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  2 +-
>  arch/x86/kernel/cpu/resctrl/internal.h    | 10 ----------
>  arch/x86/kernel/cpu/resctrl/monitor.c     |  8 ++++----
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 15 +++++++--------
>  include/linux/resctrl.h                   | 17 +++++++++++++++++
>  6 files changed, 38 insertions(+), 24 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index 83e40341583e..b773b3bdebe3 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -122,6 +122,14 @@ struct rdt_hw_resource rdt_resources_all[] = {
>  	},
>  };
>  
> +struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l)
> +{
> +	if (l >= RDT_NUM_RESOURCES)
> +		return NULL;
> +
> +	return &rdt_resources_all[l].r_resctrl;
> +}
> +
>  /*
>   * cache_alloc_hsw_probe() - Have to probe for Intel haswell server CPUs
>   * as they do not have CPUID enumeration support for Cache allocation.
> @@ -169,7 +177,7 @@ static inline void cache_alloc_hsw_probe(void)
>  bool is_mba_sc(struct rdt_resource *r)
>  {
>  	if (!r)
> -		return rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl.membw.mba_sc;
> +		r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
>  
>  	/*
>  	 * The software controller support is only applicable to MBA resource.
> diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> index 7997b47743a2..788ac0395645 100644
> --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> @@ -599,7 +599,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)
>  	domid = md.u.domid;
>  	evtid = md.u.evtid;
>  
> -	r = &rdt_resources_all[resid].r_resctrl;
> +	r = resctrl_arch_get_resource(resid);
>  	d = rdt_find_domain(r, domid, NULL);
>  	if (IS_ERR_OR_NULL(d)) {
>  		ret = -ENOENT;
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index c99f26ebe7a6..65990def6c79 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -466,16 +466,6 @@ extern struct rdt_hw_resource rdt_resources_all[];
>  extern struct rdtgroup rdtgroup_default;
>  extern struct dentry *debugfs_resctrl;
>  
> -enum resctrl_res_level {
> -	RDT_RESOURCE_L3,
> -	RDT_RESOURCE_L2,
> -	RDT_RESOURCE_MBA,
> -	RDT_RESOURCE_SMBA,
> -
> -	/* Must be the last */
> -	RDT_NUM_RESOURCES,
> -};
> -
>  static inline struct rdt_resource *resctrl_inc(struct rdt_resource *res)
>  {
>  	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(res);
> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index c34a35ec0f03..06565153ceb2 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -321,7 +321,7 @@ static void limbo_release_entry(struct rmid_entry *entry)
>   */
>  void __check_limbo(struct rdt_domain *d, bool force_free)
>  {
> -	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
>  	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
>  	struct rmid_entry *entry;
>  	u32 idx, cur_idx = 1;
> @@ -467,7 +467,7 @@ int alloc_rmid(u32 closid)
>  
>  static void add_rmid_to_limbo(struct rmid_entry *entry)
>  {
> -	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
>  	struct rdt_domain *d;
>  	u32 idx;
>  
> @@ -669,7 +669,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
>  	if (!is_mbm_local_enabled())
>  		return;
>  
> -	r_mba = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
> +	r_mba = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
>  
>  	closid = rgrp->closid;
>  	rmid = rgrp->mon.rmid;
> @@ -839,7 +839,7 @@ void mbm_handle_overflow(struct work_struct *work)
>  	if (!resctrl_mounted || !resctrl_arch_mon_capable())
>  		goto out_unlock;
>  
> -	r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> +	r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
>  	d = container_of(work, struct rdt_domain, mbm_over.work);
>  
>  	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 1767c1affa60..45372b6a6215 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -2253,7 +2253,7 @@ static void l2_qos_cfg_update(void *arg)
>  
>  static inline bool is_mba_linear(void)
>  {
> -	return rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl.membw.delay_linear;
> +	return resctrl_arch_get_resource(RDT_RESOURCE_MBA)->membw.delay_linear;
>  }
>  
>  static int set_cache_qos_cfg(int level, bool enable)
> @@ -2341,7 +2341,7 @@ static void mba_sc_domain_destroy(struct rdt_resource *r,
>   */
>  static bool supports_mba_mbps(void)
>  {
> -	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
>  
>  	return (is_mbm_local_enabled() &&
>  		r->alloc_capable && is_mba_linear());
> @@ -2353,7 +2353,7 @@ static bool supports_mba_mbps(void)
>   */
>  static int set_mba_sc(bool mba_sc)
>  {
> -	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
>  	u32 num_closid = resctrl_arch_get_num_closid(r);
>  	struct rdt_domain *d;
>  	int i;
> @@ -2625,10 +2625,10 @@ static void schemata_list_destroy(void)
>  
>  static int rdt_get_tree(struct fs_context *fc)
>  {
> +	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);

Its is probably best to keep the resource name as r here to be consistent
with other changes.

>  	struct rdt_fs_context *ctx = rdt_fc2context(fc);
>  	unsigned long flags = RFTYPE_CTRL_BASE;
>  	struct rdt_domain *dom;
> -	struct rdt_resource *r;
>  	int ret;
>  
>  	cpus_read_lock();
> @@ -2701,8 +2701,7 @@ static int rdt_get_tree(struct fs_context *fc)
>  		resctrl_mounted = true;
>  
>  	if (is_mbm_enabled()) {
> -		r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> -		list_for_each_entry(dom, &r->domains, list)
> +		list_for_each_entry(dom, &l3->domains, list)
>  			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
>  						   RESCTRL_PICK_ANY_CPU);
>  	}
> @@ -3878,7 +3877,7 @@ static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
>  	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2))
>  		seq_puts(seq, ",cdpl2");
>  
> -	if (is_mba_sc(&rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl))
> +	if (is_mba_sc(resctrl_arch_get_resource(RDT_RESOURCE_MBA)))
>  		seq_puts(seq, ",mba_MBps");
>  
>  	if (resctrl_debug)
> @@ -4068,7 +4067,7 @@ static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)
>  
>  void resctrl_offline_cpu(unsigned int cpu)
>  {
> -	struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> +	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
>  	struct rdtgroup *rdtgrp;
>  	struct rdt_domain *d;
>  
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index a365f67131ec..168cc9510069 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -36,6 +36,16 @@ enum resctrl_conf_type {
>  	CDP_DATA,
>  };
>  
> +enum resctrl_res_level {
> +	RDT_RESOURCE_L3,
> +	RDT_RESOURCE_L2,
> +	RDT_RESOURCE_MBA,
> +	RDT_RESOURCE_SMBA,
> +
> +	/* Must be the last */
> +	RDT_NUM_RESOURCES,
> +};
> +
>  #define CDP_NUM_TYPES	(CDP_DATA + 1)
>  
>  /*
> @@ -190,6 +200,13 @@ struct rdt_resource {
>  	bool			cdp_capable;
>  };
>  
> +/*
> + * Get the resource that exists at this level. If the level is not supported
> + * a dummy/not-capable resource can be returned. Levels >= RDT_NUM_RESOURCES
> + * will return NULL.
> + */
> +struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l);
> +
>  /**
>   * struct resctrl_schema - configuration abilities of a resource presented to
>   *			   user-space

-- 
Thanks
Babu Moger

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[Dave Martin]

On Mon, Apr 15, 2024 at 03:28:18PM -0500, Moger, Babu wrote:
> Hi James/Dave,
> 
> On 3/21/24 11:50, James Morse wrote:
> > Resctrl occasionally wants to know something about a specific resource,
> > in these cases it reaches into the arch code's rdt_resources_all[]
> > array.
> > 
> > Once the filesystem parts of resctrl are moved to /fs/, this means it
> > will need visibility of the architecture specific struct
> > resctrl_hw_resource definition, and the array of all resources.
> > All architectures would also need a r_resctrl member in this struct.
> > 
> > Instead, abstract this via a helper to allow architectures to do
> > different things here. Move the level enum to the resctrl header and
> > add a helper to retrieve the struct rdt_resource by 'rid'.
> > 
> > resctrl_arch_get_resource() should not return NULL for any value in
> > the enum, it may instead return a dummy resource that is
> > !alloc_enabled && !mon_enabled.
> 
> Nit.
> You may want to drop the second half of the statement. We don't have a
> dummy resource.

I guess not, but MPAM will, although I haven't fully understood the
logic.  See:

https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2

(Search for "dummy".)


In any case, the statement above is part of the definition of the new
interface: the resctrl core code is going to explicitly need to cope
with a dummy resource being returned, and the arch code is required
to return a pointer to something and not NULL.

So I would say that it is appropriate (or, at the very least, harmless)
to keep that statement here?

> 
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/core.c        | 10 +++++++++-
> >  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  2 +-
> >  arch/x86/kernel/cpu/resctrl/internal.h    | 10 ----------
> >  arch/x86/kernel/cpu/resctrl/monitor.c     |  8 ++++----
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 15 +++++++--------
> >  include/linux/resctrl.h                   | 17 +++++++++++++++++
> >  6 files changed, 38 insertions(+), 24 deletions(-)
> > 

[...]

> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index 1767c1affa60..45372b6a6215 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c

[...]

> > @@ -2625,10 +2625,10 @@ static void schemata_list_destroy(void)
> >  
> >  static int rdt_get_tree(struct fs_context *fc)
> >  {
> > +	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> 
> Its is probably best to keep the resource name as r here to be consistent
> with other changes.
> 
> >  	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> >  	unsigned long flags = RFTYPE_CTRL_BASE;
> >  	struct rdt_domain *dom;
> > -	struct rdt_resource *r;
> >  	int ret;
> >  
> >  	cpus_read_lock();
> > @@ -2701,8 +2701,7 @@ static int rdt_get_tree(struct fs_context *fc)
> >  		resctrl_mounted = true;
> >  
> >  	if (is_mbm_enabled()) {
> > -		r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> > -		list_for_each_entry(dom, &r->domains, list)
> > +		list_for_each_entry(dom, &l3->domains, list)
> >  			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
> >  						   RESCTRL_PICK_ANY_CPU);
> >  	}
> > @@ -3878,7 +3877,7 @@ static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)

[...]

> Thanks
> Babu Moger

[...]

Yes, this does look a bit odd.

This looks like a no-op change to me -- I think that
resctrl_arch_get_resource() is supposed to be without side-effects,
so I would have expected this to be a one-line change at the assignment
to r, with no particular need for renaming r.

Does that make sense to you, or is there some complexity I'm not
noticing here?

Cheers
---Dave

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[Moger, Babu]

Hi Dave,

On 4/16/24 11:15, Dave Martin wrote:
> On Mon, Apr 15, 2024 at 03:28:18PM -0500, Moger, Babu wrote:
>> Hi James/Dave,
>>
>> On 3/21/24 11:50, James Morse wrote:
>>> Resctrl occasionally wants to know something about a specific resource,
>>> in these cases it reaches into the arch code's rdt_resources_all[]
>>> array.
>>>
>>> Once the filesystem parts of resctrl are moved to /fs/, this means it
>>> will need visibility of the architecture specific struct
>>> resctrl_hw_resource definition, and the array of all resources.
>>> All architectures would also need a r_resctrl member in this struct.
>>>
>>> Instead, abstract this via a helper to allow architectures to do
>>> different things here. Move the level enum to the resctrl header and
>>> add a helper to retrieve the struct rdt_resource by 'rid'.
>>>
>>> resctrl_arch_get_resource() should not return NULL for any value in
>>> the enum, it may instead return a dummy resource that is
>>> !alloc_enabled && !mon_enabled.
>>
>> Nit.
>> You may want to drop the second half of the statement. We don't have a
>> dummy resource.
> 
> I guess not, but MPAM will, although I haven't fully understood the
> logic.  See:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2
> 
> (Search for "dummy".)
> 
> 
> In any case, the statement above is part of the definition of the new
> interface: the resctrl core code is going to explicitly need to cope
> with a dummy resource being returned, and the arch code is required
> to return a pointer to something and not NULL.
> 
> So I would say that it is appropriate (or, at the very least, harmless)
> to keep that statement here?

Ok. fine.
> 
>>
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>> ---
>>>  arch/x86/kernel/cpu/resctrl/core.c        | 10 +++++++++-
>>>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  2 +-
>>>  arch/x86/kernel/cpu/resctrl/internal.h    | 10 ----------
>>>  arch/x86/kernel/cpu/resctrl/monitor.c     |  8 ++++----
>>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 15 +++++++--------
>>>  include/linux/resctrl.h                   | 17 +++++++++++++++++
>>>  6 files changed, 38 insertions(+), 24 deletions(-)
>>>
> 
> [...]
> 
>>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> index 1767c1affa60..45372b6a6215 100644
>>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> 
> [...]
> 
>>> @@ -2625,10 +2625,10 @@ static void schemata_list_destroy(void)
>>>  
>>>  static int rdt_get_tree(struct fs_context *fc)
>>>  {
>>> +	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
>>
>> Its is probably best to keep the resource name as r here to be consistent
>> with other changes.
>>
>>>  	struct rdt_fs_context *ctx = rdt_fc2context(fc);
>>>  	unsigned long flags = RFTYPE_CTRL_BASE;
>>>  	struct rdt_domain *dom;
>>> -	struct rdt_resource *r;
>>>  	int ret;
>>>  
>>>  	cpus_read_lock();
>>> @@ -2701,8 +2701,7 @@ static int rdt_get_tree(struct fs_context *fc)
>>>  		resctrl_mounted = true;
>>>  
>>>  	if (is_mbm_enabled()) {
>>> -		r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
>>> -		list_for_each_entry(dom, &r->domains, list)
>>> +		list_for_each_entry(dom, &l3->domains, list)
>>>  			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
>>>  						   RESCTRL_PICK_ANY_CPU);
>>>  	}
>>> @@ -3878,7 +3877,7 @@ static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
> 
> [...]
> 
>> Thanks
>> Babu Moger
> 
> [...]
> 
> Yes, this does look a bit odd.
> 
> This looks like a no-op change to me -- I think that
> resctrl_arch_get_resource() is supposed to be without side-effects,
> so I would have expected this to be a one-line change at the assignment
> to r, with no particular need for renaming r.
> 
> Does that make sense to you, or is there some complexity I'm not
> noticing here?

No other complexity..  Just keep the variable name as r.

struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);

-- 
Thanks
Babu Moger

Re: [PATCH v1 02/31] x86/resctrl: Add a helper to avoid reaching into the arch code resource list

[Dave Martin]

On Tue, Apr 16, 2024 at 03:44:29PM -0500, Moger, Babu wrote:
> Hi Dave,
> 
> On 4/16/24 11:15, Dave Martin wrote:
> > On Mon, Apr 15, 2024 at 03:28:18PM -0500, Moger, Babu wrote:
> >> Hi James/Dave,
> >>
> >> On 3/21/24 11:50, James Morse wrote:
> >>> Resctrl occasionally wants to know something about a specific resource,
> >>> in these cases it reaches into the arch code's rdt_resources_all[]
> >>> array.
> >>>
> >>> Once the filesystem parts of resctrl are moved to /fs/, this means it
> >>> will need visibility of the architecture specific struct
> >>> resctrl_hw_resource definition, and the array of all resources.
> >>> All architectures would also need a r_resctrl member in this struct.
> >>>
> >>> Instead, abstract this via a helper to allow architectures to do
> >>> different things here. Move the level enum to the resctrl header and
> >>> add a helper to retrieve the struct rdt_resource by 'rid'.
> >>>
> >>> resctrl_arch_get_resource() should not return NULL for any value in
> >>> the enum, it may instead return a dummy resource that is
> >>> !alloc_enabled && !mon_enabled.
> >>
> >> Nit.
> >> You may want to drop the second half of the statement. We don't have a
> >> dummy resource.
> > 
> > I guess not, but MPAM will, although I haven't fully understood the
> > logic.  See:
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2
> > 
> > (Search for "dummy".)
> > 
> > 
> > In any case, the statement above is part of the definition of the new
> > interface: the resctrl core code is going to explicitly need to cope
> > with a dummy resource being returned, and the arch code is required
> > to return a pointer to something and not NULL.
> > 
> > So I would say that it is appropriate (or, at the very least, harmless)
> > to keep that statement here?
> 
> Ok. fine.
> > 
> >>
> >>>
> >>> Signed-off-by: James Morse <james.morse@arm.com>
> >>> ---
> >>>  arch/x86/kernel/cpu/resctrl/core.c        | 10 +++++++++-
> >>>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  2 +-
> >>>  arch/x86/kernel/cpu/resctrl/internal.h    | 10 ----------
> >>>  arch/x86/kernel/cpu/resctrl/monitor.c     |  8 ++++----
> >>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 15 +++++++--------
> >>>  include/linux/resctrl.h                   | 17 +++++++++++++++++
> >>>  6 files changed, 38 insertions(+), 24 deletions(-)
> >>>
> > 
> > [...]
> > 
> >>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> >>> index 1767c1affa60..45372b6a6215 100644
> >>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> >>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > 
> > [...]
> > 
> >>> @@ -2625,10 +2625,10 @@ static void schemata_list_destroy(void)
> >>>  
> >>>  static int rdt_get_tree(struct fs_context *fc)
> >>>  {
> >>> +	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> >>
> >> Its is probably best to keep the resource name as r here to be consistent
> >> with other changes.
> >>
> >>>  	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> >>>  	unsigned long flags = RFTYPE_CTRL_BASE;
> >>>  	struct rdt_domain *dom;
> >>> -	struct rdt_resource *r;
> >>>  	int ret;
> >>>  
> >>>  	cpus_read_lock();
> >>> @@ -2701,8 +2701,7 @@ static int rdt_get_tree(struct fs_context *fc)
> >>>  		resctrl_mounted = true;
> >>>  
> >>>  	if (is_mbm_enabled()) {
> >>> -		r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> >>> -		list_for_each_entry(dom, &r->domains, list)
> >>> +		list_for_each_entry(dom, &l3->domains, list)
> >>>  			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
> >>>  						   RESCTRL_PICK_ANY_CPU);
> >>>  	}
> >>> @@ -3878,7 +3877,7 @@ static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
> > 
> > [...]
> > 
> >> Thanks
> >> Babu Moger
> > 
> > [...]
> > 
> > Yes, this does look a bit odd.
> > 
> > This looks like a no-op change to me -- I think that
> > resctrl_arch_get_resource() is supposed to be without side-effects,
> > so I would have expected this to be a one-line change at the assignment
> > to r, with no particular need for renaming r.
> > 
> > Does that make sense to you, or is there some complexity I'm not
> > noticing here?
> 
> No other complexity..  Just keep the variable name as r.
> 
> struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);

Ack; I went for the minimum-diffstat option in the end, so kept the
declaration and initialiser separate.  If you have a strong view on that
though, please shout.

Cheers
---Dave

[PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[James Morse]

The policy for parsing the configuration values as a string from
user-space is specified by a function pointer the arch code specifies.

These strings are part of resctrl's ABI, and the functions and their
caller both live in the same file. Exporting the parsing functions and
allowing the architecture to choose how a schema is parsed allows an
architecture to get this wrong.

Keep this all in the flesystem parts of resctrl. This should prevent any
architecture's string-parsing behaviour from varying without core code
changes. Use the fflags to spot caches and bandwidth resources, and use
the appropriate helper.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c        |  4 ----
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 28 +++++++++++++++++++----
 arch/x86/kernel/cpu/resctrl/internal.h    | 10 --------
 include/linux/resctrl.h                   |  7 ------
 4 files changed, 23 insertions(+), 26 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index b773b3bdebe3..d07eff7d6750 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -75,7 +75,6 @@ struct rdt_hw_resource rdt_resources_all[] = {
 			.name			= "L3",
 			.cache_level		= 3,
 			.domains		= domain_init(RDT_RESOURCE_L3),
-			.parse_ctrlval		= parse_cbm,
 			.format_str		= "%d=%0*x",
 			.fflags			= RFTYPE_RES_CACHE,
 		},
@@ -89,7 +88,6 @@ struct rdt_hw_resource rdt_resources_all[] = {
 			.name			= "L2",
 			.cache_level		= 2,
 			.domains		= domain_init(RDT_RESOURCE_L2),
-			.parse_ctrlval		= parse_cbm,
 			.format_str		= "%d=%0*x",
 			.fflags			= RFTYPE_RES_CACHE,
 		},
@@ -103,7 +101,6 @@ struct rdt_hw_resource rdt_resources_all[] = {
 			.name			= "MB",
 			.cache_level		= 3,
 			.domains		= domain_init(RDT_RESOURCE_MBA),
-			.parse_ctrlval		= parse_bw,
 			.format_str		= "%d=%*u",
 			.fflags			= RFTYPE_RES_MB,
 		},
@@ -115,7 +112,6 @@ struct rdt_hw_resource rdt_resources_all[] = {
 			.name			= "SMBA",
 			.cache_level		= 3,
 			.domains		= domain_init(RDT_RESOURCE_SMBA),
-			.parse_ctrlval		= parse_bw,
 			.format_str		= "%d=%*u",
 			.fflags			= RFTYPE_RES_MB,
 		},
diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index 788ac0395645..72a651671c68 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -23,6 +23,15 @@
 
 #include "internal.h"
 
+struct rdt_parse_data {
+	struct rdtgroup		*rdtgrp;
+	char			*buf;
+};
+
+typedef int (ctrlval_parser_t)(struct rdt_parse_data *data,
+			       struct resctrl_schema *s,
+			       struct rdt_domain *d);
+
 /*
  * Check whether MBA bandwidth percentage value is correct. The value is
  * checked against the minimum and max bandwidth values specified by the
@@ -59,8 +68,8 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
 	return true;
 }
 
-int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
-	     struct rdt_domain *d)
+static int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
+		    struct rdt_domain *d)
 {
 	struct resctrl_staged_config *cfg;
 	u32 closid = data->rdtgrp->closid;
@@ -138,8 +147,8 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
  * Read one cache bit mask (hex). Check that it is valid for the current
  * resource type.
  */
-int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
-	      struct rdt_domain *d)
+static int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
+		     struct rdt_domain *d)
 {
 	struct rdtgroup *rdtgrp = data->rdtgrp;
 	struct resctrl_staged_config *cfg;
@@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
 	return 0;
 }
 
+static ctrlval_parser_t *get_parser(struct rdt_resource *res)
+{
+	if (res->fflags & RFTYPE_RES_CACHE)
+		return &parse_cbm;
+	else
+		return &parse_bw;
+}
+
 /*
  * For each domain in this resource we expect to find a series of:
  *	id=mask
@@ -204,6 +221,7 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
 static int parse_line(char *line, struct resctrl_schema *s,
 		      struct rdtgroup *rdtgrp)
 {
+	ctrlval_parser_t *parse_ctrlval = get_parser(s->res);
 	enum resctrl_conf_type t = s->conf_type;
 	struct resctrl_staged_config *cfg;
 	struct rdt_resource *r = s->res;
@@ -235,7 +253,7 @@ static int parse_line(char *line, struct resctrl_schema *s,
 		if (d->id == dom_id) {
 			data.buf = dom;
 			data.rdtgrp = rdtgrp;
-			if (r->parse_ctrlval(&data, s, d))
+			if (parse_ctrlval(&data, s, d))
 				return -EINVAL;
 			if (rdtgrp->mode ==  RDT_MODE_PSEUDO_LOCKSETUP) {
 				cfg = &d->staged_config[t];
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 65990def6c79..9048bd32e86f 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -413,11 +413,6 @@ static inline bool is_mbm_event(int e)
 		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
 }
 
-struct rdt_parse_data {
-	struct rdtgroup		*rdtgrp;
-	char			*buf;
-};
-
 /**
  * struct rdt_hw_resource - arch private attributes of a resctrl resource
  * @r_resctrl:		Attributes of the resource used directly by resctrl.
@@ -455,11 +450,6 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *r
 	return container_of(r, struct rdt_hw_resource, r_resctrl);
 }
 
-int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
-	      struct rdt_domain *d);
-int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
-	     struct rdt_domain *d);
-
 extern struct mutex rdtgroup_mutex;
 
 extern struct rdt_hw_resource rdt_resources_all[];
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 168cc9510069..6e87bc95f5ea 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -157,9 +157,6 @@ struct resctrl_membw {
 	u32				*mb_map;
 };
 
-struct rdt_parse_data;
-struct resctrl_schema;
-
 /**
  * struct rdt_resource - attributes of a resctrl resource
  * @rid:		The index of the resource
@@ -174,7 +171,6 @@ struct resctrl_schema;
  * @data_width:		Character width of data when displaying
  * @default_ctrl:	Specifies default cache cbm or memory B/W percent.
  * @format_str:		Per resource format string to show domain value
- * @parse_ctrlval:	Per resource function pointer to parse control values
  * @evt_list:		List of monitoring events
  * @fflags:		flags to choose base and info files
  * @cdp_capable:	Is the CDP feature available on this resource
@@ -192,9 +188,6 @@ struct rdt_resource {
 	int			data_width;
 	u32			default_ctrl;
 	const char		*format_str;
-	int			(*parse_ctrlval)(struct rdt_parse_data *data,
-						 struct resctrl_schema *s,
-						 struct rdt_domain *d);
 	struct list_head	evt_list;
 	unsigned long		fflags;
 	bool			cdp_capable;
-- 
2.39.2

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> The policy for parsing the configuration values as a string from
> user-space is specified by a function pointer the arch code specifies.
> 
> These strings are part of resctrl's ABI, and the functions and their
> caller both live in the same file. Exporting the parsing functions and
> allowing the architecture to choose how a schema is parsed allows an
> architecture to get this wrong.
> 
> Keep this all in the flesystem parts of resctrl. This should prevent any

flesystem -> filesystem

> architecture's string-parsing behaviour from varying without core code
> changes. Use the fflags to spot caches and bandwidth resources, and use
> the appropriate helper.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---

...

> @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
>  	return 0;
>  }
>  
> +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
> +{
> +	if (res->fflags & RFTYPE_RES_CACHE)
> +		return &parse_cbm;
> +	else
> +		return &parse_bw;
> +}

This is borderline ... at minimum it expands what fflags means and how it
is intended to be used and that needs to be documented because it reads:

	* @fflags:		flags to choose base and info files

I am curious why you picked fflags instead of an explicit check against
rid?

Reinette

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[Dave Martin]

On Mon, Apr 08, 2024 at 08:14:47PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > The policy for parsing the configuration values as a string from
> > user-space is specified by a function pointer the arch code specifies.
> > 
> > These strings are part of resctrl's ABI, and the functions and their
> > caller both live in the same file. Exporting the parsing functions and
> > allowing the architecture to choose how a schema is parsed allows an
> > architecture to get this wrong.
> > 
> > Keep this all in the flesystem parts of resctrl. This should prevent any
> 
> flesystem -> filesystem

Noted, thanks.


> > architecture's string-parsing behaviour from varying without core code
> > changes. Use the fflags to spot caches and bandwidth resources, and use
> > the appropriate helper.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> 
> ..
> 
> > @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
> >  	return 0;
> >  }
> >  
> > +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
> > +{
> > +	if (res->fflags & RFTYPE_RES_CACHE)
> > +		return &parse_cbm;
> > +	else
> > +		return &parse_bw;
> > +}
> 
> This is borderline ... at minimum it expands what fflags means and how it
> is intended to be used and that needs to be documented because it reads:
> 
> 	* @fflags:		flags to choose base and info files
> 
> I am curious why you picked fflags instead of an explicit check against
> rid?
> 
> Reinette

Is fflags already somewhat overloaded?  There seem to be a mix of things
that are independent Boolean flags, while other things seem mutually
exclusive or enum-like.

Do we expect RFTYPE_RES_CACHE | RFTYPE_RES_MB ever to make sense,
as David points out?


With MPAM, we could in theory have cache population control and egress
memory bandwidth controls on a single interconnect component.

If that would always be represented through resctrl as two components
with the MB controls considered one level out from the CACHE controls,
then I guess these control types remain mutually exclusive from
resctrl's point of view.

Allowing a single rdt_resource to sprout multiple control types looks
more invasive in the code, even if it logically makes sense in terms of
the hardware.

(I'm guessing that may have already been ruled out?  Apologies if I
seem to be questioning things that were decided already.  That's not
my intention, and James will already have thought about this in any
case...)


Anyway, for this patch, there seem to be a couple of assumptions:

a) get_parser() doesn't get called except for rdt_resources that
represent resource controls (so, fflags = RFTYPE_RES_foo for some "foo",
with no other flags set), and

b) there are exactly two kinds of "foo", so whatever isn't a CACHE is
a BW.

These assumptions seem to hold today (?)

But the semantics of fflags already look a bit complicated, so I can
see why it might be best to avoid anything that may add more
complexity.


If the main aim is to avoid silly copy-paste errors when coding up
resources for a new arch, would it make sense to go for a more low-
tech approach and just bundle up related fields in a macro?

E.g., something like:

#define RDT_RESOURCE_MB_DEFAULTS		\
	.format_str	= "%d=%*u",		\
	.fflags		= RFTYPE_RES_MB,	\
	.parse_ctrlval	= parse_bw

#define RDT_RESOURCE_CACHE_DEFAULTS		\
	.format_str	= "%d=%0*x",		\
	.fflags		= RFTYPE_RES_CACHE,	\
	.parse_ctrlval	= parse_cbm

This isn't particularly pretty, but would at least help avoid accidents
and reduce the amount of explicit boilerplate in the resource
definitions.

Thoughts?

Cheers
---Dave

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[Reinette Chatre]

Hi Dave,

On 4/12/2024 9:16 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:14:47PM -0700, Reinette Chatre wrote:
>> On 3/21/2024 9:50 AM, James Morse wrote:

>>> @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
>>>  	return 0;
>>>  }
>>>  
>>> +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
>>> +{
>>> +	if (res->fflags & RFTYPE_RES_CACHE)
>>> +		return &parse_cbm;
>>> +	else
>>> +		return &parse_bw;
>>> +}
>>
>> This is borderline ... at minimum it expands what fflags means and how it
>> is intended to be used and that needs to be documented because it reads:
>>
>> 	* @fflags:		flags to choose base and info files
>>
>> I am curious why you picked fflags instead of an explicit check against
>> rid?
>>
>> Reinette
> 
> Is fflags already somewhat overloaded?  There seem to be a mix of things
> that are independent Boolean flags, while other things seem mutually
> exclusive or enum-like.
> 
> Do we expect RFTYPE_RES_CACHE | RFTYPE_RES_MB ever to make sense,
> as David points out?
> 
> 
> With MPAM, we could in theory have cache population control and egress
> memory bandwidth controls on a single interconnect component.
> 
> If that would always be represented through resctrl as two components
> with the MB controls considered one level out from the CACHE controls,
> then I guess these control types remain mutually exclusive from
> resctrl's point of view.
> 
> Allowing a single rdt_resource to sprout multiple control types looks
> more invasive in the code, even if it logically makes sense in terms of
> the hardware.
> 
> (I'm guessing that may have already been ruled out?  Apologies if I
> seem to be questioning things that were decided already.  That's not
> my intention, and James will already have thought about this in any
> case...)
> 
> 
> Anyway, for this patch, there seem to be a couple of assumptions:
> 
> a) get_parser() doesn't get called except for rdt_resources that
> represent resource controls (so, fflags = RFTYPE_RES_foo for some "foo",
> with no other flags set), and
> 
> b) there are exactly two kinds of "foo", so whatever isn't a CACHE is
> a BW.
> 
> These assumptions seem to hold today (?)

(c) the parser for user provided data is based on the resource type.

As I understand (c) may not be true for MPAM that supports different
partitioning controls for a single resource. For example, for a cache
MPAM supports portion as well as maximum capacity controls that
I expect would need different parsers (perhaps mapping to different
schemata entries?) from user space but will be used to control the
same resource.

I do now know if the goal is to support this MPAM capability via
resctrl but do accomplish this I wonder if it may not be more appropriate
to associate the parser with the schema entry that is presented to user space.

> But the semantics of fflags already look a bit complicated, so I can
> see why it might be best to avoid anything that may add more
> complexity.

ack.

> If the main aim is to avoid silly copy-paste errors when coding up
> resources for a new arch, would it make sense to go for a more low-
> tech approach and just bundle up related fields in a macro?

I understand this as more than avoiding copy-paste errors. I understand
the goal is to prevent architectures from having architecture specific
parsers.

> 
> E.g., something like:
> 
> #define RDT_RESOURCE_MB_DEFAULTS		\
> 	.format_str	= "%d=%*u",		\
> 	.fflags		= RFTYPE_RES_MB,	\
> 	.parse_ctrlval	= parse_bw
> 
> #define RDT_RESOURCE_CACHE_DEFAULTS		\
> 	.format_str	= "%d=%0*x",		\
> 	.fflags		= RFTYPE_RES_CACHE,	\
> 	.parse_ctrlval	= parse_cbm
> 
> This isn't particularly pretty, but would at least help avoid accidents
> and reduce the amount of explicit boilerplate in the resource
> definitions.
> 
> Thoughts?

I understand the goal of this patch to make the parser something that
the fs code owns. This is done in support of a consistent user interface.
It is not clear how turning this into macros prevents arch code from
still overriding the parser.

You do highlight another point though, shouldn't the fs code own the
format_str also? I do not think we want arch code to control the
print format, this is also something that should be consistent between
all archs and owned by fs code, again perhaps more appropriate for
a schema entry.

Reinette

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[Dave Martin]

On Mon, Apr 15, 2024 at 10:44:34AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/12/2024 9:16 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:14:47PM -0700, Reinette Chatre wrote:
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> 
> >>> @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
> >>>  	return 0;
> >>>  }
> >>>  
> >>> +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
> >>> +{
> >>> +	if (res->fflags & RFTYPE_RES_CACHE)
> >>> +		return &parse_cbm;
> >>> +	else
> >>> +		return &parse_bw;
> >>> +}
> >>
> >> This is borderline ... at minimum it expands what fflags means and how it
> >> is intended to be used and that needs to be documented because it reads:
> >>
> >> 	* @fflags:		flags to choose base and info files
> >>
> >> I am curious why you picked fflags instead of an explicit check against
> >> rid?
> >>
> >> Reinette
> > 
> > Is fflags already somewhat overloaded?  There seem to be a mix of things
> > that are independent Boolean flags, while other things seem mutually
> > exclusive or enum-like.
> > 
> > Do we expect RFTYPE_RES_CACHE | RFTYPE_RES_MB ever to make sense,
> > as David points out?
> > 
> > 
> > With MPAM, we could in theory have cache population control and egress
> > memory bandwidth controls on a single interconnect component.
> > 
> > If that would always be represented through resctrl as two components
> > with the MB controls considered one level out from the CACHE controls,
> > then I guess these control types remain mutually exclusive from
> > resctrl's point of view.
> > 
> > Allowing a single rdt_resource to sprout multiple control types looks
> > more invasive in the code, even if it logically makes sense in terms of
> > the hardware.
> > 
> > (I'm guessing that may have already been ruled out?  Apologies if I
> > seem to be questioning things that were decided already.  That's not
> > my intention, and James will already have thought about this in any
> > case...)
> > 
> > 
> > Anyway, for this patch, there seem to be a couple of assumptions:
> > 
> > a) get_parser() doesn't get called except for rdt_resources that
> > represent resource controls (so, fflags = RFTYPE_RES_foo for some "foo",
> > with no other flags set), and
> > 
> > b) there are exactly two kinds of "foo", so whatever isn't a CACHE is
> > a BW.
> > 
> > These assumptions seem to hold today (?)
> 
> (c) the parser for user provided data is based on the resource type.
> 
> As I understand (c) may not be true for MPAM that supports different
> partitioning controls for a single resource. For example, for a cache
> MPAM supports portion as well as maximum capacity controls that
> I expect would need different parsers (perhaps mapping to different
> schemata entries?) from user space but will be used to control the
> same resource.
> 
> I do now know if the goal is to support this MPAM capability via
> resctrl but do accomplish this I wonder if it may not be more appropriate
> to associate the parser with the schema entry that is presented to user space.
> 
> > But the semantics of fflags already look a bit complicated, so I can
> > see why it might be best to avoid anything that may add more
> > complexity.
> 
> ack.
> 
> > If the main aim is to avoid silly copy-paste errors when coding up
> > resources for a new arch, would it make sense to go for a more low-
> > tech approach and just bundle up related fields in a macro?
> 
> I understand this as more than avoiding copy-paste errors. I understand
> the goal is to prevent architectures from having architecture specific
> parsers.
> 
> > 
> > E.g., something like:
> > 
> > #define RDT_RESOURCE_MB_DEFAULTS		\
> > 	.format_str	= "%d=%*u",		\
> > 	.fflags		= RFTYPE_RES_MB,	\
> > 	.parse_ctrlval	= parse_bw
> > 
> > #define RDT_RESOURCE_CACHE_DEFAULTS		\
> > 	.format_str	= "%d=%0*x",		\
> > 	.fflags		= RFTYPE_RES_CACHE,	\
> > 	.parse_ctrlval	= parse_cbm
> > 
> > This isn't particularly pretty, but would at least help avoid accidents
> > and reduce the amount of explicit boilerplate in the resource
> > definitions.
> > 
> > Thoughts?
> 
> I understand the goal of this patch to make the parser something that
> the fs code owns. This is done in support of a consistent user interface.
> It is not clear how turning this into macros prevents arch code from
> still overriding the parser.
> 
> You do highlight another point though, shouldn't the fs code own the
> format_str also? I do not think we want arch code to control the
> print format, this is also something that should be consistent between
> all archs and owned by fs code, again perhaps more appropriate for
> a schema entry.
> 
> Reinette

Fair points, I guess.

For the print format, I was presuming that this ought to be consistent
with the parse format, so probably a core property too (until/unless
someone comes up with a convincing counterexample).


Would something like the following make sense, if you want a less
informal approach?  (Modulo minor details like naming conventions etc.)


/* In fs/resctrl.c */

struct struct resctrl_ctrl_traits {
	const char		*format_str;
	ctrlval_parser_t	*parse_ctrlval;
};

static const struct resctrl_ctrl_traits resource_traits[]  = {
	[RESTYPE_INVALID] = {},
	[RESTYPE_MB] = {
		.format_str	= "%d=%*u",
		.parse_ctrlval	= parse_bw,
	},
	[RESTYPE_CACHE] = {
		.format_str	= "%d=%0*x",
		.parse_ctrlval	= parse_cbm,
	},
};

static bool is_resource(const struct rdt_resource *r)
{
	return r->fflags & RFTYPE_RES;
}


/* In include/linux/resctrl_types.h */

+#define RFTYPE_RES			BIT(8)
-#define RFTYPE_RES_CACHE		BIT(8)
-#define RFTYPE_RES_MB			BIT(9)

/* For RFTYPE_RES: */
enum resctrl_resource_type {
	RRESTYPE_INVALID,
	RRESTYPE_MB,
	RRESTYPE_CACHE,
};

/* In include/linux/resctrl.h */

struct rdt_resource {
/* ... */

-	const char			*format_str;
+	enum resctrl_resource_type	res_type;

/* ... */
};


(RRESTYPE_INVALID would just be there to catch cases where .res_type is
not assigned.)


James might also have other thoughts about this when he gets back...

In any case, it might make sense to detach this change from this series
if we're making more significant changes in this area than just
splitting the code into core and arch parts.

(Note also, your suggestion about indexing using rid may also work;
I tend to assume that the mapping from rid to resource types may not be
fixed, but maybe I'm being too strongly influenced by MPAM...)

Cheers
---Dave

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[Reinette Chatre]

Hi Dave

On 4/16/2024 9:16 AM, Dave Martin wrote:
> On Mon, Apr 15, 2024 at 10:44:34AM -0700, Reinette Chatre wrote:
>> Hi Dave,
>>
>> On 4/12/2024 9:16 AM, Dave Martin wrote:
>>> On Mon, Apr 08, 2024 at 08:14:47PM -0700, Reinette Chatre wrote:
>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>
>>>>> @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
>>>>>  	return 0;
>>>>>  }
>>>>>  
>>>>> +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
>>>>> +{
>>>>> +	if (res->fflags & RFTYPE_RES_CACHE)
>>>>> +		return &parse_cbm;
>>>>> +	else
>>>>> +		return &parse_bw;
>>>>> +}
>>>>
>>>> This is borderline ... at minimum it expands what fflags means and how it
>>>> is intended to be used and that needs to be documented because it reads:
>>>>
>>>> 	* @fflags:		flags to choose base and info files
>>>>
>>>> I am curious why you picked fflags instead of an explicit check against
>>>> rid?
>>>>
>>>> Reinette
>>>
>>> Is fflags already somewhat overloaded?  There seem to be a mix of things
>>> that are independent Boolean flags, while other things seem mutually
>>> exclusive or enum-like.
>>>
>>> Do we expect RFTYPE_RES_CACHE | RFTYPE_RES_MB ever to make sense,
>>> as David points out?
>>>
>>>
>>> With MPAM, we could in theory have cache population control and egress
>>> memory bandwidth controls on a single interconnect component.
>>>
>>> If that would always be represented through resctrl as two components
>>> with the MB controls considered one level out from the CACHE controls,
>>> then I guess these control types remain mutually exclusive from
>>> resctrl's point of view.
>>>
>>> Allowing a single rdt_resource to sprout multiple control types looks
>>> more invasive in the code, even if it logically makes sense in terms of
>>> the hardware.
>>>
>>> (I'm guessing that may have already been ruled out?  Apologies if I
>>> seem to be questioning things that were decided already.  That's not
>>> my intention, and James will already have thought about this in any
>>> case...)
>>>
>>>
>>> Anyway, for this patch, there seem to be a couple of assumptions:
>>>
>>> a) get_parser() doesn't get called except for rdt_resources that
>>> represent resource controls (so, fflags = RFTYPE_RES_foo for some "foo",
>>> with no other flags set), and
>>>
>>> b) there are exactly two kinds of "foo", so whatever isn't a CACHE is
>>> a BW.
>>>
>>> These assumptions seem to hold today (?)
>>
>> (c) the parser for user provided data is based on the resource type.
>>
>> As I understand (c) may not be true for MPAM that supports different
>> partitioning controls for a single resource. For example, for a cache
>> MPAM supports portion as well as maximum capacity controls that
>> I expect would need different parsers (perhaps mapping to different
>> schemata entries?) from user space but will be used to control the
>> same resource.
>>
>> I do now know if the goal is to support this MPAM capability via
>> resctrl but do accomplish this I wonder if it may not be more appropriate
>> to associate the parser with the schema entry that is presented to user space.
>>
>>> But the semantics of fflags already look a bit complicated, so I can
>>> see why it might be best to avoid anything that may add more
>>> complexity.
>>
>> ack.
>>
>>> If the main aim is to avoid silly copy-paste errors when coding up
>>> resources for a new arch, would it make sense to go for a more low-
>>> tech approach and just bundle up related fields in a macro?
>>
>> I understand this as more than avoiding copy-paste errors. I understand
>> the goal is to prevent architectures from having architecture specific
>> parsers.
>>
>>>
>>> E.g., something like:
>>>
>>> #define RDT_RESOURCE_MB_DEFAULTS		\
>>> 	.format_str	= "%d=%*u",		\
>>> 	.fflags		= RFTYPE_RES_MB,	\
>>> 	.parse_ctrlval	= parse_bw
>>>
>>> #define RDT_RESOURCE_CACHE_DEFAULTS		\
>>> 	.format_str	= "%d=%0*x",		\
>>> 	.fflags		= RFTYPE_RES_CACHE,	\
>>> 	.parse_ctrlval	= parse_cbm
>>>
>>> This isn't particularly pretty, but would at least help avoid accidents
>>> and reduce the amount of explicit boilerplate in the resource
>>> definitions.
>>>
>>> Thoughts?
>>
>> I understand the goal of this patch to make the parser something that
>> the fs code owns. This is done in support of a consistent user interface.
>> It is not clear how turning this into macros prevents arch code from
>> still overriding the parser.
>>
>> You do highlight another point though, shouldn't the fs code own the
>> format_str also? I do not think we want arch code to control the
>> print format, this is also something that should be consistent between
>> all archs and owned by fs code, again perhaps more appropriate for
>> a schema entry.
>>
>> Reinette
> 
> Fair points, I guess.
> 
> For the print format, I was presuming that this ought to be consistent
> with the parse format, so probably a core property too (until/unless
> someone comes up with a convincing counterexample).
> 
> 
> Would something like the following make sense, if you want a less
> informal approach?  (Modulo minor details like naming conventions etc.)
> 
> 
> /* In fs/resctrl.c */
> 
> struct struct resctrl_ctrl_traits {
> 	const char		*format_str;
> 	ctrlval_parser_t	*parse_ctrlval;
> };
> 
> static const struct resctrl_ctrl_traits resource_traits[]  = {
> 	[RESTYPE_INVALID] = {},
> 	[RESTYPE_MB] = {
> 		.format_str	= "%d=%*u",
> 		.parse_ctrlval	= parse_bw,
> 	},
> 	[RESTYPE_CACHE] = {
> 		.format_str	= "%d=%0*x",
> 		.parse_ctrlval	= parse_cbm,
> 	},
> };

It is not obvious to me that another layer is needed here. format_str
and parse_ctrlval can just be members of struct resctrl_schema?

> 
> static bool is_resource(const struct rdt_resource *r)
> {
> 	return r->fflags & RFTYPE_RES;
> }

I do not see the usage of is_resource().

(I think we are now discussing both this patch and patch #30 here)

Here is part relevant to #30:

What I was thinking about was something like below that uses the
enum you introduce later and lets the RF flags stay internal to fs code:

rdtgroup_create_info_dir()
{

	...
	list_for_each_entry(s, &resctrl_schema_all, list) {
		r = s->res;
		if (r->res_type == RRESTYPE_CACHE)
			fflags = RFTYPE_RES_CACHE;
		else if (r->res_type == RRESTYPE_MB)
			fflags = RFTYPE_RES_MB;
		else /* fail */
		
		fflags |= RFTYPE_CTRL_INFO;

		...
	}
	/* same idea for monitor info files */


For this patch the resource type can be used to initialize the schema
entry.

> 
> 
> /* In include/linux/resctrl_types.h */
> 
> +#define RFTYPE_RES			BIT(8)
> -#define RFTYPE_RES_CACHE		BIT(8)
> -#define RFTYPE_RES_MB			BIT(9)

The goal is to not have to expose any of the RFTYPE flags internals to
the architecture. RFTYPE_RES_CACHE and RFTYPE_RES_MB stays, but is
not exposed to arch code. I do not see need for RFTYPE_RES.
All the RFTYPE flags can be defined in fs/resctrl/internal.h

> 
> /* For RFTYPE_RES: */
> enum resctrl_resource_type {
> 	RRESTYPE_INVALID,
> 	RRESTYPE_MB,
> 	RRESTYPE_CACHE,
> };

(I find naming hard ... note the names changed from the beginning of
pseudo code to here where RESTYPE changing to RRESTYPE)

> 
> /* In include/linux/resctrl.h */
> 
> struct rdt_resource {
> /* ... */
> 
> -	const char			*format_str;
> +	enum resctrl_resource_type	res_type;
> 
> /* ... */
> };

Yes. With the above architecture code would only specify if it is
cache or memory via enum resctrl_resource_type and need not know
the individual file flags and can pick how to format and parse
data based on the resource type.

> 
> 
> (RRESTYPE_INVALID would just be there to catch cases where .res_type is
> not assigned.)
> 
> 
> James might also have other thoughts about this when he gets back...
> 
> In any case, it might make sense to detach this change from this series
> if we're making more significant changes in this area than just
> splitting the code into core and arch parts.
> 
> (Note also, your suggestion about indexing using rid may also work;
> I tend to assume that the mapping from rid to resource types may not be
> fixed, but maybe I'm being too strongly influenced by MPAM...)

Reinette

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[James Morse]

Hi Reinette, Dave,

On 18/04/2024 06:34, Reinette Chatre wrote:
> On 4/16/2024 9:16 AM, Dave Martin wrote:
>> On Mon, Apr 15, 2024 at 10:44:34AM -0700, Reinette Chatre wrote:
>>> On 4/12/2024 9:16 AM, Dave Martin wrote:
>>>> On Mon, Apr 08, 2024 at 08:14:47PM -0700, Reinette Chatre wrote:
>>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>
>>>>>> @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
>>>>>>  	return 0;
>>>>>>  }
>>>>>>  
>>>>>> +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
>>>>>> +{
>>>>>> +	if (res->fflags & RFTYPE_RES_CACHE)
>>>>>> +		return &parse_cbm;
>>>>>> +	else
>>>>>> +		return &parse_bw;
>>>>>> +}
>>>>>
>>>>> This is borderline ... at minimum it expands what fflags means and how it
>>>>> is intended to be used and that needs to be documented because it reads:
>>>>>
>>>>> 	* @fflags:		flags to choose base and info files

Hmm, true this is used to day to select which groups of files appear.


>>>>> I am curious why you picked fflags instead of an explicit check against
>>>>> rid?

Simply because it would need to match both L2 and L3 to parse_cbm, I didn't think that
would scale if other cache resources get added. However, with an enum of types we can get
the compiler to bark if an entry is needed here, which is probably good enough.

more background: {
	In the arm world the cache hierarchy isn't something we can reason about. We
	have funny names for where different things converge, (Point of Coherency,
	Point of Unification etc), but its up to the platform designer if/where the
	L2/L3 or even L9 live. The cache topology is fed to the kernel via an ACPI
	table.

	I anticipate a 'System Cache' resource and schema eventually being added to
	resctrl as it looks to be a popular hardware design. These system-cache live
	after the L3 (if there is one).
}


>>>> Is fflags already somewhat overloaded?  There seem to be a mix of things
>>>> that are independent Boolean flags, while other things seem mutually
>>>> exclusive or enum-like.
>>>>
>>>> Do we expect RFTYPE_RES_CACHE | RFTYPE_RES_MB ever to make sense,
>>>> as David points out?
>>>>
>>>>
>>>> With MPAM, we could in theory have cache population control and egress
>>>> memory bandwidth controls on a single interconnect component.
>>>>
>>>> If that would always be represented through resctrl as two components
>>>> with the MB controls considered one level out from the CACHE controls,
>>>> then I guess these control types remain mutually exclusive from
>>>> resctrl's point of view.
>>>>
>>>> Allowing a single rdt_resource to sprout multiple control types looks
>>>> more invasive in the code, even if it logically makes sense in terms of
>>>> the hardware.

MPAM allows this, but it doesn't fit with resctrl. The MPAM drivers resctrl glue code has
a load of stuff to present these as separate resources to resctrl, even if they are the
same piece of hardware underneath.

So far it looks possible to hide this, I don't think its worth changing resctrl's
behaviour to try and cover this.

RFTYPE_RES_CACHE and RFTYPE_RES_MB would remain mutually-exclusive.


>>>> Anyway, for this patch, there seem to be a couple of assumptions:
>>>>
>>>> a) get_parser() doesn't get called except for rdt_resources that
>>>> represent resource controls (so, fflags = RFTYPE_RES_foo for some "foo",
>>>> with no other flags set), and
>>>>
>>>> b) there are exactly two kinds of "foo", so whatever isn't a CACHE is
>>>> a BW.
>>>>
>>>> These assumptions seem to hold today (?)
>>>
>>> (c) the parser for user provided data is based on the resource type.
>>>
>>> As I understand (c) may not be true for MPAM that supports different
>>> partitioning controls for a single resource. For example, for a cache
>>> MPAM supports portion as well as maximum capacity controls that
>>> I expect would need different parsers (perhaps mapping to different
>>> schemata entries?) from user space but will be used to control the
>>> same resource.

Exactly - to maintain compatibility with existing software the driver has to present it as
a totally new thing. I guess it will look something like this:
| L3:0=0xffff;1=0xffff;
| L3_CAP:0=1048576;1=;1048576

Where existing software knows about 'L3', and should ignore 'L3_CAP'.


>>> I do now know if the goal is to support this MPAM capability via
>>> resctrl but do accomplish this I wonder if it may not be more appropriate
>>> to associate the parser with the schema entry that is presented to user space.

Even better.

For Tony's resctrl2 I had mused on exposing to user-space whether the controls were a
bitmap/percentage/MBps-value/raw-number. As there is a parser for the first two (or three)
today I think keying these from something in the schemata makes the most sense.


>>>> But the semantics of fflags already look a bit complicated, so I can
>>>> see why it might be best to avoid anything that may add more
>>>> complexity.
>>>
>>> ack.
>>>
>>>> If the main aim is to avoid silly copy-paste errors when coding up
>>>> resources for a new arch, would it make sense to go for a more low-
>>>> tech approach and just bundle up related fields in a macro?
>>>
>>> I understand this as more than avoiding copy-paste errors. I understand
>>> the goal is to prevent architectures from having architecture specific
>>> parsers.

[...]

>>> You do highlight another point though, shouldn't the fs code own the
>>> format_str also? I do not think we want arch code to control the
>>> print format, this is also something that should be consistent between
>>> all archs and owned by fs code, again perhaps more appropriate for
>>> a schema entry.

Good point ... I've still got that as a "TODO: kill these properties off as they are
derivatives" in the MPAM code.

I agree they should live together. We can also pull in data_width too, as it is calculated
based on the format used here.

Moving default_ctrl is tricky as on AMD platforms the {S,}MBA default value is discovered
from cpuid. But it only makes sense for an architecture to provides this for MBps controls
- bitmaps and percentages have an obvious maximum/default value. Putting that in  struct
resctrl_membw as 'max_bw' makes bw_validate()s use of it clearer.

bw_validate() has always caught me out as it doesn't just parse percentages, but AMDs MBps
values. I don't think this needs changing, but having MBps as a control type will make
this less surprising.

Finally, core.c will end up keeping default_ctrl as an arch-specific thing as its
convenient for the init and reset code.

[...]

> What I was thinking about was something like below that uses the
> enum you introduce later and lets the RF flags stay internal to fs code:
> 
> rdtgroup_create_info_dir()
> {
> 
> 	...
> 	list_for_each_entry(s, &resctrl_schema_all, list) {
> 		r = s->res;
> 		if (r->res_type == RRESTYPE_CACHE)
> 			fflags = RFTYPE_RES_CACHE;
> 		else if (r->res_type == RRESTYPE_MB)
> 			fflags = RFTYPE_RES_MB;
> 		else /* fail */
> 		
> 		fflags |= RFTYPE_CTRL_INFO;
> 
> 		...
> 	}
> 	/* same idea for monitor info files */

Good point, that would let us remove fflags from the arch code too.


> For this patch the resource type can be used to initialize the schema
> entry.
> 
>>
>>
>> /* In include/linux/resctrl_types.h */
>>
>> +#define RFTYPE_RES			BIT(8)
>> -#define RFTYPE_RES_CACHE		BIT(8)
>> -#define RFTYPE_RES_MB			BIT(9)
> 
> The goal is to not have to expose any of the RFTYPE flags internals to
> the architecture. RFTYPE_RES_CACHE and RFTYPE_RES_MB stays, but is
> not exposed to arch code. I do not see need for RFTYPE_RES.
> All the RFTYPE flags can be defined in fs/resctrl/internal.h

Yup, these should stay in internal.h - they got swept up as there are #defines either side
that are needed for MPAM to build.


>> /* For RFTYPE_RES: */
>> enum resctrl_resource_type {
>> 	RRESTYPE_INVALID,
>> 	RRESTYPE_MB,
>> 	RRESTYPE_CACHE,
>> };
> 
> (I find naming hard ... note the names changed from the beginning of
> pseudo code to here where RESTYPE changing to RRESTYPE)

Before I saw this my attempt has:
| enum resctrl_schema_fmt {
| 	RESCTRL_SCHEMA_BITMAP,
| 	RESCTRL_SCHEMA_PERCENTAGE,
| 	RESCTRL_SCHEMA_MBPS,
| };

Invalid as value '0' would catch the arch code missing this - but means any switch over
this enum has to handle it... I'd prefer to leave that out so the compiler can bark about
any place that needs updating when a new control scheme is added.



Thanks,

James

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[David Hildenbrand]

>   /*
>    * Check whether MBA bandwidth percentage value is correct. The value is
>    * checked against the minimum and max bandwidth values specified by the
> @@ -59,8 +68,8 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
>   	return true;
>   }
>   
> -int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
> -	     struct rdt_domain *d)
> +static int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
> +		    struct rdt_domain *d)
>   {
>   	struct resctrl_staged_config *cfg;
>   	u32 closid = data->rdtgrp->closid;
> @@ -138,8 +147,8 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
>    * Read one cache bit mask (hex). Check that it is valid for the current
>    * resource type.
>    */
> -int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
> -	      struct rdt_domain *d)
> +static int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
> +		     struct rdt_domain *d)
>   {
>   	struct rdtgroup *rdtgrp = data->rdtgrp;
>   	struct resctrl_staged_config *cfg;
> @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
>   	return 0;
>   }
>   
> +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
> +{
> +	if (res->fflags & RFTYPE_RES_CACHE)
> +		return &parse_cbm;
> +	else
> +		return &parse_bw;
> +}

Besides what Reinette said, I'd have added here something that would 
fire in case someone adds something unexpected in the future, like

WARN_ON_ONCE(!(res->fflags & (RFTYPE_RES_CACHE|RFTYPE_RES_MB));

At the beginning of the function.


Apart from that, nothing jumped at me.

-- 
Cheers,

David / dhildenb

Re: [PATCH v1 03/31] x86/resctrl: Move ctrlval string parsing policy away from the arch code

[Dave Martin]

Hi,

On Tue, Apr 09, 2024 at 05:13:01PM +0200, David Hildenbrand wrote:

[...]

> > @@ -195,6 +204,14 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
> >   	return 0;
> >   }
> > +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
> > +{
> > +	if (res->fflags & RFTYPE_RES_CACHE)
> > +		return &parse_cbm;
> > +	else
> > +		return &parse_bw;
> > +}
> 
> Besides what Reinette said, I'd have added here something that would fire in
> case someone adds something unexpected in the future, like
> 
> WARN_ON_ONCE(!(res->fflags & (RFTYPE_RES_CACHE|RFTYPE_RES_MB));
> 
> At the beginning of the function.
> 
> 
> Apart from that, nothing jumped at me.
> 
> -- 
> Cheers,
> 
> David / dhildenb
> 

Thanks for that -- I guess that would benefit from discussion; please
see my reply to Reinette on this patch.

Cheers
---Dave

[PATCH v1 04/31] x86/resctrl: Add helper for setting CPU default properties

[James Morse]

rdtgroup_rmdir_ctrl() and rdtgroup_rmdir_mon() set the per-CPU
pqr_state for CPUs that were part of the rmdir()'d group.

Another architecture might not have a 'pqr_state', its hardware may
need the values in a different format. MPAM's equivalent of RMID values
are not unique, and always need the CLOSID to be provided too.

There is only one caller that modifies a single value,
(rdtgroup_rmdir_mon()). MPAM always needs both CLOSID and RMID
for the hardware value as these are written to the same system
register.

As rdtgroup_rmdir_mon() has the CLOSID on hand, only provide a
helper to set both values. These values are read by
__resctrl_sched_in(), but may be written by a different CPU without
any locking, add READ/WRTE_ONCE() to avoid torn values.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/include/asm/resctrl.h         | 14 +++++++++++---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 15 ++++++++++-----
 2 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 12dbd2588ca7..f61382258743 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -4,8 +4,9 @@
 
 #ifdef CONFIG_X86_CPU_RESCTRL
 
-#include <linux/sched.h>
 #include <linux/jump_label.h>
+#include <linux/percpu.h>
+#include <linux/sched.h>
 
 /*
  * This value can never be a valid CLOSID, and is used when mapping a
@@ -96,8 +97,8 @@ static inline void resctrl_arch_disable_mon(void)
 static inline void __resctrl_sched_in(struct task_struct *tsk)
 {
 	struct resctrl_pqr_state *state = this_cpu_ptr(&pqr_state);
-	u32 closid = state->default_closid;
-	u32 rmid = state->default_rmid;
+	u32 closid = READ_ONCE(state->default_closid);
+	u32 rmid = READ_ONCE(state->default_rmid);
 	u32 tmp;
 
 	/*
@@ -132,6 +133,13 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
 	return val * scale;
 }
 
+static inline void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid,
+							    u32 rmid)
+{
+	WRITE_ONCE(per_cpu(pqr_state.default_closid, cpu), closid);
+	WRITE_ONCE(per_cpu(pqr_state.default_rmid, cpu), rmid);
+}
+
 static inline void resctrl_arch_set_closid_rmid(struct task_struct *tsk,
 						u32 closid, u32 rmid)
 {
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 45372b6a6215..5d2c1ce5b6b1 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -3623,14 +3623,18 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
 static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 {
 	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
+	u32 closid, rmid;
 	int cpu;
 
 	/* Give any tasks back to the parent group */
 	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
 
 	/* Update per cpu rmid of the moved CPUs first */
+	closid = rdtgrp->closid;
+	rmid = prdtgrp->mon.rmid;
 	for_each_cpu(cpu, &rdtgrp->cpu_mask)
-		per_cpu(pqr_state.default_rmid, cpu) = prdtgrp->mon.rmid;
+		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
+
 	/*
 	 * Update the MSR on moved CPUs and CPUs which have moved
 	 * task running on them.
@@ -3663,6 +3667,7 @@ static int rdtgroup_ctrl_remove(struct rdtgroup *rdtgrp)
 
 static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 {
+	u32 closid, rmid;
 	int cpu;
 
 	/* Give any tasks back to the default group */
@@ -3673,10 +3678,10 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 		   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
 
 	/* Update per cpu closid and rmid of the moved CPUs first */
-	for_each_cpu(cpu, &rdtgrp->cpu_mask) {
-		per_cpu(pqr_state.default_closid, cpu) = rdtgroup_default.closid;
-		per_cpu(pqr_state.default_rmid, cpu) = rdtgroup_default.mon.rmid;
-	}
+	closid = rdtgroup_default.closid;
+	rmid = rdtgroup_default.mon.rmid;
+	for_each_cpu(cpu, &rdtgrp->cpu_mask)
+		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
 
 	/*
 	 * Update the MSR on moved CPUs and CPUs which have moved
-- 
2.39.2

Re: [PATCH v1 04/31] x86/resctrl: Add helper for setting CPU default properties

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:

> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -3623,14 +3623,18 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
>  static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
>  {
>  	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
> +	u32 closid, rmid;
>  	int cpu;
>  
>  	/* Give any tasks back to the parent group */
>  	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
>  
>  	/* Update per cpu rmid of the moved CPUs first */
> +	closid = rdtgrp->closid;
> +	rmid = prdtgrp->mon.rmid;
>  	for_each_cpu(cpu, &rdtgrp->cpu_mask)
> -		per_cpu(pqr_state.default_rmid, cpu) = prdtgrp->mon.rmid;
> +		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
> +

While I understand that the CLOSIDs are the same, I do think it looks unexpected
for the CLOSID to be set to the CLOSID of the group being removed. Could this
be set to CLOSID of parent group instead?

Reinette

Re: [PATCH v1 04/31] x86/resctrl: Add helper for setting CPU default properties

[Dave Martin]

On Mon, Apr 08, 2024 at 08:15:03PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> 
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > @@ -3623,14 +3623,18 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
> >  static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
> >  {
> >  	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
> > +	u32 closid, rmid;
> >  	int cpu;
> >  
> >  	/* Give any tasks back to the parent group */
> >  	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
> >  
> >  	/* Update per cpu rmid of the moved CPUs first */
> > +	closid = rdtgrp->closid;
> > +	rmid = prdtgrp->mon.rmid;
> >  	for_each_cpu(cpu, &rdtgrp->cpu_mask)
> > -		per_cpu(pqr_state.default_rmid, cpu) = prdtgrp->mon.rmid;
> > +		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
> > +
> 
> While I understand that the CLOSIDs are the same, I do think it looks unexpected
> for the CLOSID to be set to the CLOSID of the group being removed. Could this
> be set to CLOSID of parent group instead?
> 
> Reinette

That seems reasonable.  How about something like this?

-	closid = rdtgrp->closid;
+	closid = prdtgrp->closid; /* no change, but the arch code needs it */

Cheers
---Dave

Re: [PATCH v1 04/31] x86/resctrl: Add helper for setting CPU default properties

[Reinette Chatre]

Hi Dave,

On 4/12/2024 9:13 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:15:03PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>
>>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> @@ -3623,14 +3623,18 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
>>>  static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
>>>  {
>>>  	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
>>> +	u32 closid, rmid;
>>>  	int cpu;
>>>  
>>>  	/* Give any tasks back to the parent group */
>>>  	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
>>>  
>>>  	/* Update per cpu rmid of the moved CPUs first */
>>> +	closid = rdtgrp->closid;
>>> +	rmid = prdtgrp->mon.rmid;
>>>  	for_each_cpu(cpu, &rdtgrp->cpu_mask)
>>> -		per_cpu(pqr_state.default_rmid, cpu) = prdtgrp->mon.rmid;
>>> +		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
>>> +
>>
>> While I understand that the CLOSIDs are the same, I do think it looks unexpected
>> for the CLOSID to be set to the CLOSID of the group being removed. Could this
>> be set to CLOSID of parent group instead?
>>
>> Reinette
> 
> That seems reasonable.  How about something like this?
> 
> -	closid = rdtgrp->closid;
> +	closid = prdtgrp->closid; /* no change, but the arch code needs it */

Looks good. If the comment stays, please do replace the tail comment with a
freestanding comment (for reference you can search for "No tail comments"
in Documentation/process/maintainer-tip.rst). 

Thank you

Reinette

Re: [PATCH v1 04/31] x86/resctrl: Add helper for setting CPU default properties

[Dave Martin]

On Mon, Apr 15, 2024 at 10:45:07AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/12/2024 9:13 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:15:03PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> >>
> >>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> >>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> >>> @@ -3623,14 +3623,18 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
> >>>  static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
> >>>  {
> >>>  	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
> >>> +	u32 closid, rmid;
> >>>  	int cpu;
> >>>  
> >>>  	/* Give any tasks back to the parent group */
> >>>  	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
> >>>  
> >>>  	/* Update per cpu rmid of the moved CPUs first */
> >>> +	closid = rdtgrp->closid;
> >>> +	rmid = prdtgrp->mon.rmid;
> >>>  	for_each_cpu(cpu, &rdtgrp->cpu_mask)
> >>> -		per_cpu(pqr_state.default_rmid, cpu) = prdtgrp->mon.rmid;
> >>> +		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
> >>> +
> >>
> >> While I understand that the CLOSIDs are the same, I do think it looks unexpected
> >> for the CLOSID to be set to the CLOSID of the group being removed. Could this
> >> be set to CLOSID of parent group instead?
> >>
> >> Reinette
> > 
> > That seems reasonable.  How about something like this?
> > 
> > -	closid = rdtgrp->closid;
> > +	closid = prdtgrp->closid; /* no change, but the arch code needs it */
> 
> Looks good. If the comment stays, please do replace the tail comment with a
> freestanding comment (for reference you can search for "No tail comments"
> in Documentation/process/maintainer-tip.rst). 

Ack, noted.

Cheers
---Dave

[PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[James Morse]

update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which
it uses to update the local CPUs default pqr values. This is a problem
once the resctrl parts move out to /fs/, as the arch code cannot
poke around inside struct rdtgroup.

Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults()
to be used as the target of an IPI, and pass the effective CLOSID
and RMID in a new struct.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++----
 include/linux/resctrl.h                | 11 +++++++++++
 2 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 5d2c1ce5b6b1..18f097fce51e 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -341,13 +341,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
  * from update_closid_rmid() is protected against __switch_to() because
  * preemption is disabled.
  */
-static void update_cpu_closid_rmid(void *info)
+void resctrl_arch_sync_cpu_defaults(void *info)
 {
-	struct rdtgroup *r = info;
+	struct resctrl_cpu_sync *r = info;
 
 	if (r) {
 		this_cpu_write(pqr_state.default_closid, r->closid);
-		this_cpu_write(pqr_state.default_rmid, r->mon.rmid);
+		this_cpu_write(pqr_state.default_rmid, r->rmid);
 	}
 
 	/*
@@ -362,11 +362,22 @@ static void update_cpu_closid_rmid(void *info)
  * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
  *
  * Per task closids/rmids must have been set up before calling this function.
+ * @r may be NULL.
  */
 static void
 update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
 {
-	on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1);
+	struct resctrl_cpu_sync defaults;
+	struct resctrl_cpu_sync *defaults_p = NULL;
+
+	if (r) {
+		defaults.closid = r->closid;
+		defaults.rmid = r->mon.rmid;
+		defaults_p = &defaults;
+	}
+
+	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
+			 1);
 }
 
 static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 6e87bc95f5ea..2b79e4159507 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -220,6 +220,17 @@ struct resctrl_schema {
 	u32				num_closid;
 };
 
+struct resctrl_cpu_sync {
+	u32 closid;
+	u32 rmid;
+};
+
+/*
+ * Update and re-load this CPUs defaults. Called via IPI, takes a pointer to
+ * struct resctrl_cpu_sync, or NULL.
+ */
+void resctrl_arch_sync_cpu_defaults(void *info);
+
 /* The number of closid supported by this resource regardless of CDP */
 u32 resctrl_arch_get_num_closid(struct rdt_resource *r);
 int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
-- 
2.39.2

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which
> it uses to update the local CPUs default pqr values. This is a problem
> once the resctrl parts move out to /fs/, as the arch code cannot
> poke around inside struct rdtgroup.
> 
> Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults()
> to be used as the target of an IPI, and pass the effective CLOSID
> and RMID in a new struct.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++----
>  include/linux/resctrl.h                | 11 +++++++++++
>  2 files changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 5d2c1ce5b6b1..18f097fce51e 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -341,13 +341,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
>   * from update_closid_rmid() is protected against __switch_to() because
>   * preemption is disabled.
>   */
> -static void update_cpu_closid_rmid(void *info)
> +void resctrl_arch_sync_cpu_defaults(void *info)
>  {
> -	struct rdtgroup *r = info;
> +	struct resctrl_cpu_sync *r = info;
>  
>  	if (r) {
>  		this_cpu_write(pqr_state.default_closid, r->closid);
> -		this_cpu_write(pqr_state.default_rmid, r->mon.rmid);
> +		this_cpu_write(pqr_state.default_rmid, r->rmid);
>  	}
>  
>  	/*
> @@ -362,11 +362,22 @@ static void update_cpu_closid_rmid(void *info)
>   * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
>   *
>   * Per task closids/rmids must have been set up before calling this function.
> + * @r may be NULL.
>   */
>  static void
>  update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
>  {
> -	on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1);
> +	struct resctrl_cpu_sync defaults;
> +	struct resctrl_cpu_sync *defaults_p = NULL;

Please maintain reverse fir order.

> +
> +	if (r) {
> +		defaults.closid = r->closid;
> +		defaults.rmid = r->mon.rmid;
> +		defaults_p = &defaults;
> +	}
> +
> +	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
> +			 1);
>  }
>  
>  static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index 6e87bc95f5ea..2b79e4159507 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -220,6 +220,17 @@ struct resctrl_schema {
>  	u32				num_closid;
>  };
>  
> +struct resctrl_cpu_sync {
> +	u32 closid;
> +	u32 rmid;
> +};
> +
> +/*
> + * Update and re-load this CPUs defaults. Called via IPI, takes a pointer to

"this CPU's defaults"?

> + * struct resctrl_cpu_sync, or NULL.
> + */

Updating the CPU's defaults is not the primary goal of this function and because
of that I do not think this should be the focus with the main goal (updating
RMID and CLOSID on CPU) ignored. Specifically, this function only updates
the defaults if *info is set but it _always_ ensures CPU is running with
appropriate CLOSID/RMID (which may or may not be from a CPU default).

I think resctrl_arch_sync_cpu_closid_rmid() may be more appropriate
and the comment needs to elaborate what the function does.

> +void resctrl_arch_sync_cpu_defaults(void *info);
> +
>  /* The number of closid supported by this resource regardless of CDP */
>  u32 resctrl_arch_get_num_closid(struct rdt_resource *r);
>  int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);

Reinette

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Dave Martin]

On Mon, Apr 08, 2024 at 08:16:08PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which
> > it uses to update the local CPUs default pqr values. This is a problem
> > once the resctrl parts move out to /fs/, as the arch code cannot
> > poke around inside struct rdtgroup.
> > 
> > Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults()
> > to be used as the target of an IPI, and pass the effective CLOSID
> > and RMID in a new struct.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++----
> >  include/linux/resctrl.h                | 11 +++++++++++
> >  2 files changed, 26 insertions(+), 4 deletions(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index 5d2c1ce5b6b1..18f097fce51e 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > @@ -341,13 +341,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
> >   * from update_closid_rmid() is protected against __switch_to() because
> >   * preemption is disabled.
> >   */
> > -static void update_cpu_closid_rmid(void *info)
> > +void resctrl_arch_sync_cpu_defaults(void *info)
> >  {
> > -	struct rdtgroup *r = info;
> > +	struct resctrl_cpu_sync *r = info;
> >  
> >  	if (r) {
> >  		this_cpu_write(pqr_state.default_closid, r->closid);
> > -		this_cpu_write(pqr_state.default_rmid, r->mon.rmid);
> > +		this_cpu_write(pqr_state.default_rmid, r->rmid);
> >  	}
> >  
> >  	/*
> > @@ -362,11 +362,22 @@ static void update_cpu_closid_rmid(void *info)
> >   * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
> >   *
> >   * Per task closids/rmids must have been set up before calling this function.
> > + * @r may be NULL.
> >   */
> >  static void
> >  update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
> >  {
> > -	on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1);
> > +	struct resctrl_cpu_sync defaults;
> > +	struct resctrl_cpu_sync *defaults_p = NULL;
> 
> Please maintain reverse fir order.

Or, more tersely as follows?

	struct resctrl_cpu_sync defaults, *defaults_p = NULL;

"Reverse fir order" seems to be documented as a preference rather than a
rule.

The declarations can be swapped, but defaults_p is in some sense a weak
pointer to defaults, so it feels a bit strange to declare them backwards.

Alternatively, could we rename defaults_p to p?  Given the size of this
function I don't think that impacts clarity.

I'll wait for your opinion on this.


> > +
> > +	if (r) {
> > +		defaults.closid = r->closid;
> > +		defaults.rmid = r->mon.rmid;
> > +		defaults_p = &defaults;
> > +	}
> > +
> > +	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
> > +			 1);
> >  }
> >  
> >  static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
> > diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> > index 6e87bc95f5ea..2b79e4159507 100644
> > --- a/include/linux/resctrl.h
> > +++ b/include/linux/resctrl.h
> > @@ -220,6 +220,17 @@ struct resctrl_schema {
> >  	u32				num_closid;
> >  };
> >  
> > +struct resctrl_cpu_sync {
> > +	u32 closid;
> > +	u32 rmid;
> > +};
> > +
> > +/*
> > + * Update and re-load this CPUs defaults. Called via IPI, takes a pointer to
> 
> "this CPU's defaults"?

Ack (also in the commit message).

> 
> > + * struct resctrl_cpu_sync, or NULL.
> > + */
> 
> Updating the CPU's defaults is not the primary goal of this function and because
> of that I do not think this should be the focus with the main goal (updating
> RMID and CLOSID on CPU) ignored. Specifically, this function only updates
> the defaults if *info is set but it _always_ ensures CPU is running with
> appropriate CLOSID/RMID (which may or may not be from a CPU default).
> 
> I think resctrl_arch_sync_cpu_closid_rmid() may be more appropriate
> and the comment needs to elaborate what the function does.
>
> > +void resctrl_arch_sync_cpu_defaults(void *info);

That seems reasonable, and follows the original naming and what the
code does:

What about:

/**
 * resctrl_arch_sync_cpu_defaults() - Refresh the CPU's CLOSID and RMID.
 *				      Call via IPI.
 * @info:	If non-NULL, a pointer to a struct resctrl_cpu_sync specifying
 *		the new CLOSID and RMID for tasks in the default resctrl ctrl
 *		and mon group when running on this CPU.  If NULL, the default
 *		CLOSID and RMID are not changed.
 *
 * This is how reassignment of CPUs and/or tasks to different resctrl groups
 * is propagated when requested by the resctrl fs core code.
 *
 * This function should typically record the per-cpu defaults specified by
 * @info (if any), and then reconfigure the CPU's hardware CLOSID and RMID
 * for subsequent execution based on @current, in the same way as during a
 * task switch.
 */

...?


Cheers
---Dave

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Reinette Chatre]

Hi Dave,

On 4/12/2024 9:12 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:16:08PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which
>>> it uses to update the local CPUs default pqr values. This is a problem
>>> once the resctrl parts move out to /fs/, as the arch code cannot
>>> poke around inside struct rdtgroup.
>>>
>>> Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults()
>>> to be used as the target of an IPI, and pass the effective CLOSID
>>> and RMID in a new struct.
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>> ---
>>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++----
>>>  include/linux/resctrl.h                | 11 +++++++++++
>>>  2 files changed, 26 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> index 5d2c1ce5b6b1..18f097fce51e 100644
>>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>>> @@ -341,13 +341,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
>>>   * from update_closid_rmid() is protected against __switch_to() because
>>>   * preemption is disabled.
>>>   */
>>> -static void update_cpu_closid_rmid(void *info)
>>> +void resctrl_arch_sync_cpu_defaults(void *info)
>>>  {
>>> -	struct rdtgroup *r = info;
>>> +	struct resctrl_cpu_sync *r = info;
>>>  
>>>  	if (r) {
>>>  		this_cpu_write(pqr_state.default_closid, r->closid);
>>> -		this_cpu_write(pqr_state.default_rmid, r->mon.rmid);
>>> +		this_cpu_write(pqr_state.default_rmid, r->rmid);
>>>  	}
>>>  
>>>  	/*
>>> @@ -362,11 +362,22 @@ static void update_cpu_closid_rmid(void *info)
>>>   * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
>>>   *
>>>   * Per task closids/rmids must have been set up before calling this function.
>>> + * @r may be NULL.
>>>   */
>>>  static void
>>>  update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
>>>  {
>>> -	on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1);
>>> +	struct resctrl_cpu_sync defaults;
>>> +	struct resctrl_cpu_sync *defaults_p = NULL;
>>
>> Please maintain reverse fir order.
> 
> Or, more tersely as follows?
> 
> 	struct resctrl_cpu_sync defaults, *defaults_p = NULL;

Sure.

> 
> "Reverse fir order" seems to be documented as a preference rather than a
> rule.

This does not seem to be a place that warrants an exception to this
preference. Note how this function is not consistent with any other
in the file.

> The declarations can be swapped, but defaults_p is in some sense a weak
> pointer to defaults, so it feels a bit strange to declare them backwards.
> 
> Alternatively, could we rename defaults_p to p?  Given the size of this
> function I don't think that impacts clarity.

Do you imply that this would maintain the order in this patch? It does
not look to me that it would but I may be looking wrong.

sidenote: the "on_each_cpu_mask()" in update_closid_rmid() can be on
one line.

> 
> I'll wait for your opinion on this.
> 
> 

...

>>> + * struct resctrl_cpu_sync, or NULL.
>>> + */
>>
>> Updating the CPU's defaults is not the primary goal of this function and because
>> of that I do not think this should be the focus with the main goal (updating
>> RMID and CLOSID on CPU) ignored. Specifically, this function only updates
>> the defaults if *info is set but it _always_ ensures CPU is running with
>> appropriate CLOSID/RMID (which may or may not be from a CPU default).
>>
>> I think resctrl_arch_sync_cpu_closid_rmid() may be more appropriate
>> and the comment needs to elaborate what the function does.
>>
>>> +void resctrl_arch_sync_cpu_defaults(void *info);
> 
> That seems reasonable, and follows the original naming and what the
> code does:
> 
> What about:
> 
> /**
>  * resctrl_arch_sync_cpu_defaults() - Refresh the CPU's CLOSID and RMID.
>  *				      Call via IPI.

Did you intend to change function name?

How about "Refresh the CPU's ..." -> "Refresh this CPU's ..." I think it
makes it more obvious how this function is called.

>  * @info:	If non-NULL, a pointer to a struct resctrl_cpu_sync specifying
>  *		the new CLOSID and RMID for tasks in the default resctrl ctrl
>  *		and mon group when running on this CPU.  If NULL, the default
>  *		CLOSID and RMID are not changed.

"If NULL, this CPU is not re-assigned to a different group." ?

>  *
>  * This is how reassignment of CPUs and/or tasks to different resctrl groups
>  * is propagated when requested by the resctrl fs core code.

Could you please use imperative tone here?  For example, "Propagates reassignment
of CPUs and/or tasks to different resctrl groups."

>  *
>  * This function should typically record the per-cpu defaults specified by

"should" sounds like there may be cases when this is not done? Maybe just
"Records the per-CPU defaults specified ..."

>  * @info (if any), and then reconfigure the CPU's hardware CLOSID and RMID
>  * for subsequent execution based on @current, in the same way as during a
>  * task switch.
>  */
> 
> ...?

Reinette

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Dave Martin]

On Mon, Apr 15, 2024 at 10:47:55AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/12/2024 9:12 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:16:08PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> >>> update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which
> >>> it uses to update the local CPUs default pqr values. This is a problem
> >>> once the resctrl parts move out to /fs/, as the arch code cannot
> >>> poke around inside struct rdtgroup.
> >>>
> >>> Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults()
> >>> to be used as the target of an IPI, and pass the effective CLOSID
> >>> and RMID in a new struct.
> >>>
> >>> Signed-off-by: James Morse <james.morse@arm.com>
> >>> ---
> >>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++----
> >>>  include/linux/resctrl.h                | 11 +++++++++++
> >>>  2 files changed, 26 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> >>> index 5d2c1ce5b6b1..18f097fce51e 100644
> >>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> >>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> >>> @@ -341,13 +341,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
> >>>   * from update_closid_rmid() is protected against __switch_to() because
> >>>   * preemption is disabled.
> >>>   */
> >>> -static void update_cpu_closid_rmid(void *info)
> >>> +void resctrl_arch_sync_cpu_defaults(void *info)
> >>>  {
> >>> -	struct rdtgroup *r = info;
> >>> +	struct resctrl_cpu_sync *r = info;
> >>>  
> >>>  	if (r) {
> >>>  		this_cpu_write(pqr_state.default_closid, r->closid);
> >>> -		this_cpu_write(pqr_state.default_rmid, r->mon.rmid);
> >>> +		this_cpu_write(pqr_state.default_rmid, r->rmid);
> >>>  	}
> >>>  
> >>>  	/*
> >>> @@ -362,11 +362,22 @@ static void update_cpu_closid_rmid(void *info)
> >>>   * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
> >>>   *
> >>>   * Per task closids/rmids must have been set up before calling this function.
> >>> + * @r may be NULL.
> >>>   */
> >>>  static void
> >>>  update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
> >>>  {
> >>> -	on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1);
> >>> +	struct resctrl_cpu_sync defaults;
> >>> +	struct resctrl_cpu_sync *defaults_p = NULL;
> >>
> >> Please maintain reverse fir order.
> > 
> > Or, more tersely as follows?
> > 
> > 	struct resctrl_cpu_sync defaults, *defaults_p = NULL;
> 
> Sure.

[*]

> > 
> > "Reverse fir order" seems to be documented as a preference rather than a
> > rule.
> 
> This does not seem to be a place that warrants an exception to this
> preference. Note how this function is not consistent with any other
> in the file.

Ack (just bikeshedding here TBH).

> 
> > The declarations can be swapped, but defaults_p is in some sense a weak
> > pointer to defaults, so it feels a bit strange to declare them backwards.
> > 
> > Alternatively, could we rename defaults_p to p?  Given the size of this
> > function I don't think that impacts clarity.

[...]

> > 
> > I'll wait for your opinion on this.
> > 
> > 
> 
> Do you imply that this would maintain the order in this patch? It does
> not look to me that it would but I may be looking wrong.

I'm not sure without looking again, but since this discussion is not a
good use of your time I'll just go ahead and implement the change at
[*] above, while restoring referse FIR order, if that is good for you.

> 
> sidenote: the "on_each_cpu_mask()" in update_closid_rmid() can be on
> one line.

I guess that might have been split to stick to the 80-char limit.

Due the the small size of this function, shall I just rename defaults_p to p?
Alternatively, there are already a few non-printk lines over 80 chars, so
maybe we can tolerate one more here?

> 
> ..
> 
> >>> + * struct resctrl_cpu_sync, or NULL.
> >>> + */
> >>
> >> Updating the CPU's defaults is not the primary goal of this function and because
> >> of that I do not think this should be the focus with the main goal (updating
> >> RMID and CLOSID on CPU) ignored. Specifically, this function only updates
> >> the defaults if *info is set but it _always_ ensures CPU is running with
> >> appropriate CLOSID/RMID (which may or may not be from a CPU default).
> >>
> >> I think resctrl_arch_sync_cpu_closid_rmid() may be more appropriate
> >> and the comment needs to elaborate what the function does.
> >>
> >>> +void resctrl_arch_sync_cpu_defaults(void *info);
> > 
> > That seems reasonable, and follows the original naming and what the
> > code does:
> > 
> > What about:
> > 
> > /**
> >  * resctrl_arch_sync_cpu_defaults() - Refresh the CPU's CLOSID and RMID.
> >  *				      Call via IPI.
> 
> Did you intend to change function name?

Er, yes, I meant to use your suggestion here, so:
resctrl_arch_sync_cpu_closid_rmid().

Also, Babu Moger's suggestion to rename struct resctrl_cpu_sync
to resctrl_cpu_defaults seems good, since that accurately describes what
is specified in the struct (and what is *not* specified if NULL is
passed).

> 
> How about "Refresh the CPU's ..." -> "Refresh this CPU's ..." I think it
> makes it more obvious how this function is called.

Agreed.

> 
> >  * @info:	If non-NULL, a pointer to a struct resctrl_cpu_sync specifying
> >  *		the new CLOSID and RMID for tasks in the default resctrl ctrl
> >  *		and mon group when running on this CPU.  If NULL, the default
> >  *		CLOSID and RMID are not changed.
> 
> "If NULL, this CPU is not re-assigned to a different group." ?

Agreed.

> >  *
> >  * This is how reassignment of CPUs and/or tasks to different resctrl groups
> >  * is propagated when requested by the resctrl fs core code.
> 
> Could you please use imperative tone here?  For example, "Propagates reassignment
> of CPUs and/or tasks to different resctrl groups."

Yes, that's better (and shorter).

> 
> >  *
> >  * This function should typically record the per-cpu defaults specified by
> 
> "should" sounds like there may be cases when this is not done? Maybe just
> "Records the per-CPU defaults specified ..."

I didn't want to pre-judge what implementation-specific cruft the arch
code needs here, so I was intentionally vague.  But the arch would need
to put the CPU defaults into effect somehow or other, so yes, I think
your text is better here.

I'll make a note of those changes.

[...]

Cheers
---Dave

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Reinette Chatre]

Hi Dave,

On 4/16/2024 9:16 AM, Dave Martin wrote:
> On Mon, Apr 15, 2024 at 10:47:55AM -0700, Reinette Chatre wrote:
>> On 4/12/2024 9:12 AM, Dave Martin wrote:
>>> On Mon, Apr 08, 2024 at 08:16:08PM -0700, Reinette Chatre wrote:
>>>> On 3/21/2024 9:50 AM, James Morse wrote:
...

>> Do you imply that this would maintain the order in this patch? It does
>> not look to me that it would but I may be looking wrong.
> 
> I'm not sure without looking again, but since this discussion is not a
> good use of your time I'll just go ahead and implement the change at
> [*] above, while restoring referse FIR order, if that is good for you.
> 
>>
>> sidenote: the "on_each_cpu_mask()" in update_closid_rmid() can be on
>> one line.
> 
> I guess that might have been split to stick to the 80-char limit.
> 
> Due the the small size of this function, shall I just rename defaults_p to p?
> Alternatively, there are already a few non-printk lines over 80 chars, so
> maybe we can tolerate one more here?

80 chars are not enforced so strictly that it impacts readability. You
may refer to how update_task_closid_rmid() looks for more confidence in/
motivation for placing this on one line.

> 
>>
>> ..
>>
>>>>> + * struct resctrl_cpu_sync, or NULL.
>>>>> + */
>>>>
>>>> Updating the CPU's defaults is not the primary goal of this function and because
>>>> of that I do not think this should be the focus with the main goal (updating
>>>> RMID and CLOSID on CPU) ignored. Specifically, this function only updates
>>>> the defaults if *info is set but it _always_ ensures CPU is running with
>>>> appropriate CLOSID/RMID (which may or may not be from a CPU default).
>>>>
>>>> I think resctrl_arch_sync_cpu_closid_rmid() may be more appropriate
>>>> and the comment needs to elaborate what the function does.
>>>>
>>>>> +void resctrl_arch_sync_cpu_defaults(void *info);
>>>
>>> That seems reasonable, and follows the original naming and what the
>>> code does:
>>>
>>> What about:
>>>
>>> /**
>>>  * resctrl_arch_sync_cpu_defaults() - Refresh the CPU's CLOSID and RMID.
>>>  *				      Call via IPI.
>>
>> Did you intend to change function name?
> 
> Er, yes, I meant to use your suggestion here, so:
> resctrl_arch_sync_cpu_closid_rmid().
> 

I'm a bit confused here when comparing with your response in [1] mentioning
a change to another name. 

[1] https://lore.kernel.org/lkml/Zh6kgs1%2fbji1P1Hl@e133380.arm.com/

> Also, Babu Moger's suggestion to rename struct resctrl_cpu_sync
> to resctrl_cpu_defaults seems good, since that accurately describes what
> is specified in the struct (and what is *not* specified if NULL is
> passed).

Sounds good, yes.

Reinette

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Dave Martin]

On Wed, Apr 17, 2024 at 10:12:35PM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/16/2024 9:16 AM, Dave Martin wrote:
> > On Mon, Apr 15, 2024 at 10:47:55AM -0700, Reinette Chatre wrote:
> >> On 4/12/2024 9:12 AM, Dave Martin wrote:
> >>> On Mon, Apr 08, 2024 at 08:16:08PM -0700, Reinette Chatre wrote:
> >>>> On 3/21/2024 9:50 AM, James Morse wrote:
> ..
> 
> >> Do you imply that this would maintain the order in this patch? It does
> >> not look to me that it would but I may be looking wrong.
> > 
> > I'm not sure without looking again, but since this discussion is not a
> > good use of your time I'll just go ahead and implement the change at
> > [*] above, while restoring referse FIR order, if that is good for you.
> > 
> >>
> >> sidenote: the "on_each_cpu_mask()" in update_closid_rmid() can be on
> >> one line.
> > 
> > I guess that might have been split to stick to the 80-char limit.
> > 
> > Due the the small size of this function, shall I just rename defaults_p to p?
> > Alternatively, there are already a few non-printk lines over 80 chars, so
> > maybe we can tolerate one more here?
> 
> 80 chars are not enforced so strictly that it impacts readability. You
> may refer to how update_task_closid_rmid() looks for more confidence in/
> motivation for placing this on one line.

Agreed.

(I did in fact rename default_p to p in my fixup, which shortens the
affected line as a side-effect anyway.)


<aside>

Although probably out of scope for this series, I wonder whether these
two paths can be combined?

update_task_closid_rmid() selects the cross_call target by task, where
update_closid_rmid() selects the cross_call target(s) by cpu.  But the
backend work that the arch code needs to do seems basically the same:
possibly update the the CPU default group membership, the reconfigure
the MSRs for the running task to ensure that they aren't stale (with a
possible optimisation not to bother if we sure that the MSRs are not
stale for the task actually running, or if we know they wouldn't be
changed by the write).

Even the check to see whether the right task is running seems somewhat
redundant: we already paid the cost of taking the IPI, and we have to
cope with spurious, idempotent updates to the MSRs anyway since this is
all racy.

Is there a high overhead to writing the MSRs on x86?

For arm64, the relevant system register only affects EL0 (i.e.,
userspace) execution, so we defer synchronisation of a whole bunch of
stuff until the return to userspace.

</aside>


> 
> > 
> >>
> >> ..
> >>
> >>>>> + * struct resctrl_cpu_sync, or NULL.
> >>>>> + */
> >>>>
> >>>> Updating the CPU's defaults is not the primary goal of this function and because
> >>>> of that I do not think this should be the focus with the main goal (updating
> >>>> RMID and CLOSID on CPU) ignored. Specifically, this function only updates
> >>>> the defaults if *info is set but it _always_ ensures CPU is running with
> >>>> appropriate CLOSID/RMID (which may or may not be from a CPU default).
> >>>>
> >>>> I think resctrl_arch_sync_cpu_closid_rmid() may be more appropriate
> >>>> and the comment needs to elaborate what the function does.
> >>>>
> >>>>> +void resctrl_arch_sync_cpu_defaults(void *info);
> >>>
> >>> That seems reasonable, and follows the original naming and what the
> >>> code does:
> >>>
> >>> What about:
> >>>
> >>> /**
> >>>  * resctrl_arch_sync_cpu_defaults() - Refresh the CPU's CLOSID and RMID.
> >>>  *				      Call via IPI.
> >>
> >> Did you intend to change function name?
> > 
> > Er, yes, I meant to use your suggestion here, so:
> > resctrl_arch_sync_cpu_closid_rmid().
> > 
> 
> I'm a bit confused here when comparing with your response in [1] mentioning
> a change to another name. 
> 
> [1] https://lore.kernel.org/lkml/Zh6kgs1%2fbji1P1Hl@e133380.arm.com/

My bad (sorry Babu!).

I read that suggestion carelessly and assumed it was aligned with
Reinette's.

The most important thing seems to be to transfer the "defaults" from the
name of the function to the name of the struct, since the struct is
about defaults (and only about defaults), while the function is about
defaults and the running task.

To avoid extra busy-work, I'll stick with
resctrl_arch_sync_cpu_closid_rmid() for now, but I don't mind changing
it if people prefer.


> > Also, Babu Moger's suggestion to rename struct resctrl_cpu_sync
> > to resctrl_cpu_defaults seems good, since that accurately describes what
> > is specified in the struct (and what is *not* specified if NULL is
> > passed).
> 
> Sounds good, yes.
> 
> Reinette
> 

Cheers
---Dave

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Reinette Chatre]

Hi Dave,

On 4/18/2024 8:21 AM, Dave Martin wrote:
> On Wed, Apr 17, 2024 at 10:12:35PM -0700, Reinette Chatre wrote:
>> On 4/16/2024 9:16 AM, Dave Martin wrote:
>>> On Mon, Apr 15, 2024 at 10:47:55AM -0700, Reinette Chatre wrote:
>>>> On 4/12/2024 9:12 AM, Dave Martin wrote:
>>>>> On Mon, Apr 08, 2024 at 08:16:08PM -0700, Reinette Chatre wrote:
>>>>>> On 3/21/2024 9:50 AM, James Morse wrote:
...


> <aside>
> 
> Although probably out of scope for this series, I wonder whether these
> two paths can be combined?
> 
> update_task_closid_rmid() selects the cross_call target by task, where
> update_closid_rmid() selects the cross_call target(s) by cpu.  But the
> backend work that the arch code needs to do seems basically the same:
> possibly update the the CPU default group membership, the reconfigure
> the MSRs for the running task to ensure that they aren't stale (with a
> possible optimisation not to bother if we sure that the MSRs are not
> stale for the task actually running, or if we know they wouldn't be
> changed by the write).
> 
> Even the check to see whether the right task is running seems somewhat
> redundant: we already paid the cost of taking the IPI, and we have to
> cope with spurious, idempotent updates to the MSRs anyway since this is
> all racy.
> 
> Is there a high overhead to writing the MSRs on x86?

The MSRs do not all have the same overhead. MSR_IA32_PQR_ASSOC is intended to
be updated quickly but it is not free and I'd prefer to keep avoiding
unnecessary updates where possible.

> 
> For arm64, the relevant system register only affects EL0 (i.e.,
> userspace) execution, so we defer synchronisation of a whole bunch of
> stuff until the return to userspace.
> 
> </aside>
> 
> 
>>
>>>
>>>>
>>>> ..
>>>>
>>>>>>> + * struct resctrl_cpu_sync, or NULL.
>>>>>>> + */
>>>>>>
>>>>>> Updating the CPU's defaults is not the primary goal of this function and because
>>>>>> of that I do not think this should be the focus with the main goal (updating
>>>>>> RMID and CLOSID on CPU) ignored. Specifically, this function only updates
>>>>>> the defaults if *info is set but it _always_ ensures CPU is running with
>>>>>> appropriate CLOSID/RMID (which may or may not be from a CPU default).
>>>>>>
>>>>>> I think resctrl_arch_sync_cpu_closid_rmid() may be more appropriate
>>>>>> and the comment needs to elaborate what the function does.
>>>>>>
>>>>>>> +void resctrl_arch_sync_cpu_defaults(void *info);
>>>>>
>>>>> That seems reasonable, and follows the original naming and what the
>>>>> code does:
>>>>>
>>>>> What about:
>>>>>
>>>>> /**
>>>>>  * resctrl_arch_sync_cpu_defaults() - Refresh the CPU's CLOSID and RMID.
>>>>>  *				      Call via IPI.
>>>>
>>>> Did you intend to change function name?
>>>
>>> Er, yes, I meant to use your suggestion here, so:
>>> resctrl_arch_sync_cpu_closid_rmid().
>>>
>>
>> I'm a bit confused here when comparing with your response in [1] mentioning
>> a change to another name. 
>>
>> [1] https://lore.kernel.org/lkml/Zh6kgs1%2fbji1P1Hl@e133380.arm.com/
> 
> My bad (sorry Babu!).
> 
> I read that suggestion carelessly and assumed it was aligned with
> Reinette's.
> 
> The most important thing seems to be to transfer the "defaults" from the
> name of the function to the name of the struct, since the struct is
> about defaults (and only about defaults), while the function is about
> defaults and the running task.
> 
> To avoid extra busy-work, I'll stick with
> resctrl_arch_sync_cpu_closid_rmid() for now, but I don't mind changing
> it if people prefer.

resctrl_arch_sync_cpu_closid_rmid() sounds good to me.

Reinette

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Moger, Babu]

Hi James/Dave,

On 3/21/24 11:50, James Morse wrote:
> update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which
> it uses to update the local CPUs default pqr values. This is a problem
> once the resctrl parts move out to /fs/, as the arch code cannot
> poke around inside struct rdtgroup.
> 
> Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults()
> to be used as the target of an IPI, and pass the effective CLOSID
> and RMID in a new struct.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++----
>  include/linux/resctrl.h                | 11 +++++++++++
>  2 files changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 5d2c1ce5b6b1..18f097fce51e 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -341,13 +341,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
>   * from update_closid_rmid() is protected against __switch_to() because
>   * preemption is disabled.
>   */
> -static void update_cpu_closid_rmid(void *info)
> +void resctrl_arch_sync_cpu_defaults(void *info)

How about keeping the name similar to the old name?

resctrl_arch_update_cpu_defaults

>  {
> -	struct rdtgroup *r = info;
> +	struct resctrl_cpu_sync *r = info;
>  
>  	if (r) {
>  		this_cpu_write(pqr_state.default_closid, r->closid);
> -		this_cpu_write(pqr_state.default_rmid, r->mon.rmid);
> +		this_cpu_write(pqr_state.default_rmid, r->rmid);
>  	}
>  
>  	/*
> @@ -362,11 +362,22 @@ static void update_cpu_closid_rmid(void *info)
>   * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
>   *
>   * Per task closids/rmids must have been set up before calling this function.
> + * @r may be NULL.
>   */
>  static void
>  update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
>  {
> -	on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1);
> +	struct resctrl_cpu_sync defaults;
> +	struct resctrl_cpu_sync *defaults_p = NULL;
> +
> +	if (r) {
> +		defaults.closid = r->closid;
> +		defaults.rmid = r->mon.rmid;
> +		defaults_p = &defaults;
> +	}
> +
> +	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
> +			 1);
>  }
>  
>  static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index 6e87bc95f5ea..2b79e4159507 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -220,6 +220,17 @@ struct resctrl_schema {
>  	u32				num_closid;
>  };
>  
> +struct resctrl_cpu_sync {

How about changing it to  resctrl_cpu_defaults?

> +	u32 closid;
> +	u32 rmid;
> +};
> +
> +/*
> + * Update and re-load this CPUs defaults. Called via IPI, takes a pointer to
> + * struct resctrl_cpu_sync, or NULL.
> + */
> +void resctrl_arch_sync_cpu_defaults(void *info);
> +
>  /* The number of closid supported by this resource regardless of CDP */
>  u32 resctrl_arch_get_num_closid(struct rdt_resource *r);
>  int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);

-- 
Thanks
Babu Moger

Re: [PATCH v1 05/31] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid()

[Dave Martin]

On Mon, Apr 15, 2024 at 03:40:46PM -0500, Moger, Babu wrote:
> Hi James/Dave,
> 
> On 3/21/24 11:50, James Morse wrote:
> > update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which
> > it uses to update the local CPUs default pqr values. This is a problem
> > once the resctrl parts move out to /fs/, as the arch code cannot
> > poke around inside struct rdtgroup.
> > 
> > Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults()
> > to be used as the target of an IPI, and pass the effective CLOSID
> > and RMID in a new struct.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++----
> >  include/linux/resctrl.h                | 11 +++++++++++
> >  2 files changed, 26 insertions(+), 4 deletions(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index 5d2c1ce5b6b1..18f097fce51e 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > @@ -341,13 +341,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
> >   * from update_closid_rmid() is protected against __switch_to() because
> >   * preemption is disabled.
> >   */
> > -static void update_cpu_closid_rmid(void *info)
> > +void resctrl_arch_sync_cpu_defaults(void *info)
> 
> How about keeping the name similar to the old name?
> 
> resctrl_arch_update_cpu_defaults

Ack (Reinette made a similar comment.)

> 
> >  {
> > -	struct rdtgroup *r = info;
> > +	struct resctrl_cpu_sync *r = info;
> >  
> >  	if (r) {
> >  		this_cpu_write(pqr_state.default_closid, r->closid);
> > -		this_cpu_write(pqr_state.default_rmid, r->mon.rmid);
> > +		this_cpu_write(pqr_state.default_rmid, r->rmid);
> >  	}
> >  
> >  	/*

[...]

> > diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> > index 6e87bc95f5ea..2b79e4159507 100644
> > --- a/include/linux/resctrl.h
> > +++ b/include/linux/resctrl.h
> > @@ -220,6 +220,17 @@ struct resctrl_schema {
> >  	u32				num_closid;
> >  };
> >  
> > +struct resctrl_cpu_sync {
> 
> How about changing it to  resctrl_cpu_defaults?
> 
> > +	u32 closid;
> > +	u32 rmid;
> > +};

[...]

> -- 
> Thanks
> Babu Moger
> 

Yes, your name describes what the struct means, so renaming it as per
your suggestion does make sense.

I'll make a note.

Cheers
---Dave

[PATCH v1 06/31] x86/resctrl: Export resctrl fs's init function

[James Morse]

rdtgroup_init() needs exporting so that arch code can call it once
it lives in core code. As this is one of the few functions we export,
rename it to have the resctrl in the name. The same goes for the exit
call.

x86's arch code init functions for RDT are renamed to have an arch
prefix to make it clear these are part of the architecture code.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c     | 12 ++++++------
 arch/x86/kernel/cpu/resctrl/internal.h |  3 ---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c |  8 ++++----
 include/linux/resctrl.h                |  3 +++
 4 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index d07eff7d6750..b4e7d655e242 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -938,7 +938,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c)
 	}
 }
 
-static int __init resctrl_late_init(void)
+static int __init resctrl_arch_late_init(void)
 {
 	struct rdt_resource *r;
 	int state, ret;
@@ -963,7 +963,7 @@ static int __init resctrl_late_init(void)
 	if (state < 0)
 		return state;
 
-	ret = rdtgroup_init();
+	ret = resctrl_init();
 	if (ret) {
 		cpuhp_remove_state(state);
 		return ret;
@@ -979,18 +979,18 @@ static int __init resctrl_late_init(void)
 	return 0;
 }
 
-late_initcall(resctrl_late_init);
+late_initcall(resctrl_arch_late_init);
 
-static void __exit resctrl_exit(void)
+static void __exit resctrl_arch_exit(void)
 {
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
 
 	cpuhp_remove_state(rdt_online);
 
-	rdtgroup_exit();
+	resctrl_exit();
 
 	if (r->mon_capable)
 		rdt_put_mon_l3_config();
 }
 
-__exitcall(resctrl_exit);
+__exitcall(resctrl_arch_exit);
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 9048bd32e86f..7c073298aabf 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -300,9 +300,6 @@ extern struct list_head rdt_all_groups;
 
 extern int max_name_width, max_data_width;
 
-int __init rdtgroup_init(void);
-void __exit rdtgroup_exit(void);
-
 /**
  * struct rftype - describe each file in the resctrl file system
  * @name:	File name
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 18f097fce51e..1a49c9918f8d 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -4116,14 +4116,14 @@ void resctrl_offline_cpu(unsigned int cpu)
 }
 
 /*
- * rdtgroup_init - rdtgroup initialization
+ * resctrl_init - resctrl filesystem initialization
  *
  * Setup resctrl file system including set up root, create mount point,
- * register rdtgroup filesystem, and initialize files under root directory.
+ * register resctrl filesystem, and initialize files under root directory.
  *
  * Return: 0 on success or -errno
  */
-int __init rdtgroup_init(void)
+int __init resctrl_init(void)
 {
 	int ret = 0;
 
@@ -4171,7 +4171,7 @@ int __init rdtgroup_init(void)
 	return ret;
 }
 
-void __exit rdtgroup_exit(void)
+void __exit resctrl_exit(void)
 {
 	debugfs_remove_recursive(debugfs_resctrl);
 	unregister_filesystem(&rdt_fs_type);
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 2b79e4159507..f6a4b75f8122 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -325,4 +325,7 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_domain *d);
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
+int __init resctrl_init(void);
+void __exit resctrl_exit(void);
+
 #endif /* _RESCTRL_H */
-- 
2.39.2

RE: [PATCH v1 06/31] x86/resctrl: Export resctrl fs's init function

[Shaopeng Tan (Fujitsu)]

Hello James,

> rdtgroup_init() needs exporting so that arch code can call it once it lives in core
> code. As this is one of the few functions we export, rename it to have the resctrl
> in the name. The same goes for the exit call.
> 
> x86's arch code init functions for RDT are renamed to have an arch prefix to
> make it clear these are part of the architecture code.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/core.c     | 12 ++++++------
>  arch/x86/kernel/cpu/resctrl/internal.h |  3 ---
> arch/x86/kernel/cpu/resctrl/rdtgroup.c |  8 ++++----
>  include/linux/resctrl.h                |  3 +++
>  4 files changed, 13 insertions(+), 13 deletions(-)

./fs/resctrl/monitor.c
746          * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and
747          * are always allocated. These are used for the rdtgroup_default
748          * control group, which will be setup later in rdtgroup_init(). 
"rdtgroup_init()" is still in the comments.


Best regards,
Shaopeng TAN

> diff --git a/arch/x86/kernel/cpu/resctrl/core.c
> b/arch/x86/kernel/cpu/resctrl/core.c
> index d07eff7d6750..b4e7d655e242 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -938,7 +938,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c)
>  	}
>  }
> 
> -static int __init resctrl_late_init(void)
> +static int __init resctrl_arch_late_init(void)
>  {
>  	struct rdt_resource *r;
>  	int state, ret;
> @@ -963,7 +963,7 @@ static int __init resctrl_late_init(void)
>  	if (state < 0)
>  		return state;
> 
> -	ret = rdtgroup_init();
> +	ret = resctrl_init();
>  	if (ret) {
>  		cpuhp_remove_state(state);
>  		return ret;
> @@ -979,18 +979,18 @@ static int __init resctrl_late_init(void)
>  	return 0;
>  }
> 
> -late_initcall(resctrl_late_init);
> +late_initcall(resctrl_arch_late_init);
> 
> -static void __exit resctrl_exit(void)
> +static void __exit resctrl_arch_exit(void)
>  {
>  	struct rdt_resource *r =
> &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> 
>  	cpuhp_remove_state(rdt_online);
> 
> -	rdtgroup_exit();
> +	resctrl_exit();
> 
>  	if (r->mon_capable)
>  		rdt_put_mon_l3_config();
>  }
> 
> -__exitcall(resctrl_exit);
> +__exitcall(resctrl_arch_exit);
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h
> b/arch/x86/kernel/cpu/resctrl/internal.h
> index 9048bd32e86f..7c073298aabf 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -300,9 +300,6 @@ extern struct list_head rdt_all_groups;
> 
>  extern int max_name_width, max_data_width;
> 
> -int __init rdtgroup_init(void);
> -void __exit rdtgroup_exit(void);
> -
>  /**
>   * struct rftype - describe each file in the resctrl file system
>   * @name:	File name
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 18f097fce51e..1a49c9918f8d 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -4116,14 +4116,14 @@ void resctrl_offline_cpu(unsigned int cpu)  }
> 
>  /*
> - * rdtgroup_init - rdtgroup initialization
> + * resctrl_init - resctrl filesystem initialization
>   *
>   * Setup resctrl file system including set up root, create mount point,
> - * register rdtgroup filesystem, and initialize files under root directory.
> + * register resctrl filesystem, and initialize files under root directory.
>   *
>   * Return: 0 on success or -errno
>   */
> -int __init rdtgroup_init(void)
> +int __init resctrl_init(void)
>  {
>  	int ret = 0;
> 
> @@ -4171,7 +4171,7 @@ int __init rdtgroup_init(void)
>  	return ret;
>  }
> 
> -void __exit rdtgroup_exit(void)
> +void __exit resctrl_exit(void)
>  {
>  	debugfs_remove_recursive(debugfs_resctrl);
>  	unregister_filesystem(&rdt_fs_type);
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index
> 2b79e4159507..f6a4b75f8122 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -325,4 +325,7 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r,
> struct rdt_domain *d);  extern unsigned int resctrl_rmid_realloc_threshold;
> extern unsigned int resctrl_rmid_realloc_limit;
> 
> +int __init resctrl_init(void);
> +void __exit resctrl_exit(void);
> +
>  #endif /* _RESCTRL_H */
> --
> 2.39.2

Re: [PATCH v1 06/31] x86/resctrl: Export resctrl fs's init function

[Dave Martin]

On Wed, Apr 03, 2024 at 07:51:54AM +0000, Shaopeng Tan (Fujitsu) wrote:
> Hello James,
> 
> > rdtgroup_init() needs exporting so that arch code can call it once it lives in core
> > code. As this is one of the few functions we export, rename it to have the resctrl
> > in the name. The same goes for the exit call.
> > 
> > x86's arch code init functions for RDT are renamed to have an arch prefix to
> > make it clear these are part of the architecture code.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/core.c     | 12 ++++++------
> >  arch/x86/kernel/cpu/resctrl/internal.h |  3 ---
> > arch/x86/kernel/cpu/resctrl/rdtgroup.c |  8 ++++----
> >  include/linux/resctrl.h                |  3 +++
> >  4 files changed, 13 insertions(+), 13 deletions(-)
> 
> /fs/resctrl/monitor.c
> 746          * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and
> 747          * are always allocated. These are used for the rdtgroup_default
> 748          * control group, which will be setup later in rdtgroup_init(). 
> "rdtgroup_init()" is still in the comments.
> 
> 
> Best regards,
> Shaopeng TAN
> 

[...]

Noted for James' attention, thanks.

Cheers
---Dave

Re: [PATCH v1 06/31] x86/resctrl: Export resctrl fs's init function

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> rdtgroup_init() needs exporting so that arch code can call it once
> it lives in core code. As this is one of the few functions we export,

Please do not impersonate code.

> rename it to have the resctrl in the name. The same goes for the exit

"to have the resctrl" -> "to have resctrl"

> call.
> 
> x86's arch code init functions for RDT are renamed to have an arch
> prefix to make it clear these are part of the architecture code.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---

Reinette

Re: [PATCH v1 06/31] x86/resctrl: Export resctrl fs's init function

[Dave Martin]

On Mon, Apr 08, 2024 at 08:16:32PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > rdtgroup_init() needs exporting so that arch code can call it once
> > it lives in core code. As this is one of the few functions we export,
> 
> Please do not impersonate code.

Noted.  Any objection to rewording this as follows?

"As this is one of the few functions exported from the resctrl common code, [...]"


> > rename it to have the resctrl in the name. The same goes for the exit
> 
> "to have the resctrl" -> "to have resctrl"

Noted for James' attention.

[...]

Cheers
---Dave

Re: [PATCH v1 06/31] x86/resctrl: Export resctrl fs's init function

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:14 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:16:32PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> rdtgroup_init() needs exporting so that arch code can call it once
>>> it lives in core code. As this is one of the few functions we export,
>>
>> Please do not impersonate code.
> 
> Noted.  Any objection to rewording this as follows?
> 
> "As this is one of the few functions exported from the resctrl common code, [...]"

Sounds good. Thank you.

Reinette

[PATCH v1 07/31] x86/resctrl: Wrap resctrl_arch_find_domain() around rdt_find_domain()

[James Morse]

rdt_find_domain() finds a domain given a resource and a cache-id.
It's not quite right for the resctrl arch API as it also returns the
position to insert a new domain, which is needed when bringing a
domain online in the arch code.

Wrap rdt_find_domain() in a another function resctrl_arch_find_domain()
so we avoid the unnecessary argument outside the arch code.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c        | 9 +++++++--
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +-
 arch/x86/kernel/cpu/resctrl/internal.h    | 2 --
 include/linux/resctrl.h                   | 2 ++
 4 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index b4e7d655e242..06dc12b08afc 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -405,8 +405,8 @@ void rdt_ctrl_update(void *arg)
  * caller, return the first domain whose id is bigger than the input id.
  * The domain list is sorted by id in ascending order.
  */
-struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,
-				   struct list_head **pos)
+static struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,
+					  struct list_head **pos)
 {
 	struct rdt_domain *d;
 	struct list_head *l;
@@ -430,6 +430,11 @@ struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,
 	return NULL;
 }
 
+struct rdt_domain *resctrl_arch_find_domain(struct rdt_resource *r, int id)
+{
+	return rdt_find_domain(r, id, NULL);
+}
+
 static void setup_default_ctrlval(struct rdt_resource *r, u32 *dc)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index 72a651671c68..3603ade95f1d 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -618,7 +618,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)
 	evtid = md.u.evtid;
 
 	r = resctrl_arch_get_resource(resid);
-	d = rdt_find_domain(r, domid, NULL);
+	d = resctrl_arch_find_domain(r, domid);
 	if (IS_ERR_OR_NULL(d)) {
 		ret = -ENOENT;
 		goto out;
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 7c073298aabf..32ade929ea1b 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -533,8 +533,6 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn);
 int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name);
 int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
 			     umode_t mask);
-struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,
-				   struct list_head **pos);
 ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
 				char *buf, size_t nbytes, loff_t off);
 int rdtgroup_schemata_show(struct kernfs_open_file *of,
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index f6a4b75f8122..c5fcbb524136 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -233,6 +233,8 @@ void resctrl_arch_sync_cpu_defaults(void *info);
 
 /* The number of closid supported by this resource regardless of CDP */
 u32 resctrl_arch_get_num_closid(struct rdt_resource *r);
+
+struct rdt_domain *resctrl_arch_find_domain(struct rdt_resource *r, int id);
 int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
 
 /*
-- 
2.39.2

Re: [PATCH v1 07/31] x86/resctrl: Wrap resctrl_arch_find_domain() around rdt_find_domain()

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> rdt_find_domain() finds a domain given a resource and a cache-id.
> It's not quite right for the resctrl arch API as it also returns the
> position to insert a new domain, which is needed when bringing a
> domain online in the arch code.
> 
> Wrap rdt_find_domain() in a another function resctrl_arch_find_domain()

"in a another" -> "in another"

> so we avoid the unnecessary argument outside the arch code.

Please do not impersonate code.

> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---

Reinette

Re: [PATCH v1 07/31] x86/resctrl: Wrap resctrl_arch_find_domain() around rdt_find_domain()

[Dave Martin]

On Mon, Apr 08, 2024 at 08:17:12PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > rdt_find_domain() finds a domain given a resource and a cache-id.
> > It's not quite right for the resctrl arch API as it also returns the
> > position to insert a new domain, which is needed when bringing a
> > domain online in the arch code.
> > 
> > Wrap rdt_find_domain() in a another function resctrl_arch_find_domain()
> 
> "in a another" -> "in another"

Noted for James' attention.


> > so we avoid the unnecessary argument outside the arch code.
> 
> Please do not impersonate code.

[...]

> Reinette

Noted.

Cheers
---Dave

[PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[James Morse]

To avoid sticky problems in the mpam glue code, move the resctrl
enums into a separate header.

This lets the arch code declare prototypes that use these enums without
creating a loop via asm<->linux resctrl.h The same logic applies to the
monitor-configuration defines, move these too.

The maintainers entry for these headers was missed when resctrl.h
was created. Add a wildcard entry to match both resctrl.h and
resctrl_types.h.

Signed-off-by: James Morse <james.morse@arm.com>
---
internal.h lacks a copyright notice so there is nothing to preserve
when creating a new file...
---
 MAINTAINERS                            |  1 +
 arch/x86/kernel/cpu/resctrl/internal.h | 24 ---------
 include/linux/resctrl.h                | 35 +------------
 include/linux/resctrl_types.h          | 68 ++++++++++++++++++++++++++
 4 files changed, 70 insertions(+), 58 deletions(-)
 create mode 100644 include/linux/resctrl_types.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 43b39956694a..5621dd823e79 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18543,6 +18543,7 @@ S:	Supported
 F:	Documentation/arch/x86/resctrl*
 F:	arch/x86/include/asm/resctrl.h
 F:	arch/x86/kernel/cpu/resctrl/
+F:	include/linux/resctrl*.h
 F:	tools/testing/selftests/resctrl/
 
 READ-COPY UPDATE (RCU)
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 32ade929ea1b..031948322eab 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -32,30 +32,6 @@
  */
 #define MBM_CNTR_WIDTH_OFFSET_MAX (62 - MBM_CNTR_WIDTH_BASE)
 
-/* Reads to Local DRAM Memory */
-#define READS_TO_LOCAL_MEM		BIT(0)
-
-/* Reads to Remote DRAM Memory */
-#define READS_TO_REMOTE_MEM		BIT(1)
-
-/* Non-Temporal Writes to Local Memory */
-#define NON_TEMP_WRITE_TO_LOCAL_MEM	BIT(2)
-
-/* Non-Temporal Writes to Remote Memory */
-#define NON_TEMP_WRITE_TO_REMOTE_MEM	BIT(3)
-
-/* Reads to Local Memory the system identifies as "Slow Memory" */
-#define READS_TO_LOCAL_S_MEM		BIT(4)
-
-/* Reads to Remote Memory the system identifies as "Slow Memory" */
-#define READS_TO_REMOTE_S_MEM		BIT(5)
-
-/* Dirty Victims to All Types of Memory */
-#define DIRTY_VICTIMS_TO_ALL_MEM	BIT(6)
-
-/* Max event bits supported */
-#define MAX_EVT_CONFIG_BITS		GENMASK(6, 0)
-
 /**
  * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that
  *			        aren't marked nohz_full
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index c5fcbb524136..b0ee7256e095 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -5,6 +5,7 @@
 #include <linux/kernel.h>
 #include <linux/list.h>
 #include <linux/pid.h>
+#include <linux/resctrl_types.h>
 
 /* CLOSID, RMID value used by the default control group */
 #define RESCTRL_RESERVED_CLOSID		0
@@ -24,40 +25,6 @@ int proc_resctrl_show(struct seq_file *m,
 /* max value for struct rdt_domain's mbps_val */
 #define MBA_MAX_MBPS   U32_MAX
 
-/**
- * enum resctrl_conf_type - The type of configuration.
- * @CDP_NONE:	No prioritisation, both code and data are controlled or monitored.
- * @CDP_CODE:	Configuration applies to instruction fetches.
- * @CDP_DATA:	Configuration applies to reads and writes.
- */
-enum resctrl_conf_type {
-	CDP_NONE,
-	CDP_CODE,
-	CDP_DATA,
-};
-
-enum resctrl_res_level {
-	RDT_RESOURCE_L3,
-	RDT_RESOURCE_L2,
-	RDT_RESOURCE_MBA,
-	RDT_RESOURCE_SMBA,
-
-	/* Must be the last */
-	RDT_NUM_RESOURCES,
-};
-
-#define CDP_NUM_TYPES	(CDP_DATA + 1)
-
-/*
- * Event IDs, the values match those used to program IA32_QM_EVTSEL before
- * reading IA32_QM_CTR on RDT systems.
- */
-enum resctrl_event_id {
-	QOS_L3_OCCUP_EVENT_ID		= 0x01,
-	QOS_L3_MBM_TOTAL_EVENT_ID	= 0x02,
-	QOS_L3_MBM_LOCAL_EVENT_ID	= 0x03,
-};
-
 /**
  * struct resctrl_staged_config - parsed configuration to be applied
  * @new_ctrl:		new ctrl value to be loaded
diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
new file mode 100644
index 000000000000..4788bd95dac6
--- /dev/null
+++ b/include/linux/resctrl_types.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2024 Arm Ltd.
+ * Based on arch/x86/kernel/cpu/resctrl/internal.h
+ */
+
+#ifndef __LINUX_RESCTRL_TYPES_H
+#define __LINUX_RESCTRL_TYPES_H
+
+/* Reads to Local DRAM Memory */
+#define READS_TO_LOCAL_MEM		BIT(0)
+
+/* Reads to Remote DRAM Memory */
+#define READS_TO_REMOTE_MEM		BIT(1)
+
+/* Non-Temporal Writes to Local Memory */
+#define NON_TEMP_WRITE_TO_LOCAL_MEM	BIT(2)
+
+/* Non-Temporal Writes to Remote Memory */
+#define NON_TEMP_WRITE_TO_REMOTE_MEM	BIT(3)
+
+/* Reads to Local Memory the system identifies as "Slow Memory" */
+#define READS_TO_LOCAL_S_MEM		BIT(4)
+
+/* Reads to Remote Memory the system identifies as "Slow Memory" */
+#define READS_TO_REMOTE_S_MEM		BIT(5)
+
+/* Dirty Victims to All Types of Memory */
+#define DIRTY_VICTIMS_TO_ALL_MEM	BIT(6)
+
+/* Max event bits supported */
+#define MAX_EVT_CONFIG_BITS		GENMASK(6, 0)
+
+/**
+ * enum resctrl_conf_type - The type of configuration.
+ * @CDP_NONE:	No prioritisation, both code and data are controlled or monitored.
+ * @CDP_CODE:	Configuration applies to instruction fetches.
+ * @CDP_DATA:	Configuration applies to reads and writes.
+ */
+enum resctrl_conf_type {
+	CDP_NONE,
+	CDP_CODE,
+	CDP_DATA,
+};
+
+enum resctrl_res_level {
+	RDT_RESOURCE_L3,
+	RDT_RESOURCE_L2,
+	RDT_RESOURCE_MBA,
+	RDT_RESOURCE_SMBA,
+
+	/* Must be the last */
+	RDT_NUM_RESOURCES,
+};
+
+#define CDP_NUM_TYPES	(CDP_DATA + 1)
+
+/*
+ * Event IDs, the values match those used to program IA32_QM_EVTSEL before
+ * reading IA32_QM_CTR on RDT systems.
+ */
+enum resctrl_event_id {
+	QOS_L3_OCCUP_EVENT_ID		= 0x01,
+	QOS_L3_MBM_TOTAL_EVENT_ID	= 0x02,
+	QOS_L3_MBM_LOCAL_EVENT_ID	= 0x03,
+};
+
+#endif /* __LINUX_RESCTRL_TYPES_H */
-- 
2.39.2

Re: [PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> To avoid sticky problems in the mpam glue code, move the resctrl
> enums into a separate header.

Could you please elaborate so that "sticky problems in the mpam glue code" is
specific about what problems are avoided?

> 
> This lets the arch code declare prototypes that use these enums without
> creating a loop via asm<->linux resctrl.h The same logic applies to the
> monitor-configuration defines, move these too.
> 
> The maintainers entry for these headers was missed when resctrl.h
> was created. Add a wildcard entry to match both resctrl.h and
> resctrl_types.h.
> 
> Signed-off-by: James Morse <james.morse@arm.com>

...

> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> new file mode 100644
> index 000000000000..4788bd95dac6
> --- /dev/null
> +++ b/include/linux/resctrl_types.h
> @@ -0,0 +1,68 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2024 Arm Ltd.
> + * Based on arch/x86/kernel/cpu/resctrl/internal.h
> + */

Could this header please explain how this file is intended to be used?
What is it intended to contain?

Reinette

Re: [PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[Dave Martin]

On Mon, Apr 08, 2024 at 08:18:00PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > To avoid sticky problems in the mpam glue code, move the resctrl
> > enums into a separate header.
> 
> Could you please elaborate so that "sticky problems in the mpam glue code" is
> specific about what problems are avoided?

Maybe just delete the the sticky clause, and leave:

	Move the resctrl enums into a separate header.

...since the next paragraph explains the rationale?

> > 
> > This lets the arch code declare prototypes that use these enums without
> > creating a loop via asm<->linux resctrl.h The same logic applies to the
> > monitor-configuration defines, move these too.
> > 
> > The maintainers entry for these headers was missed when resctrl.h
> > was created. Add a wildcard entry to match both resctrl.h and
> > resctrl_types.h.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> 
> ..
> 
> > diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> > new file mode 100644
> > index 000000000000..4788bd95dac6
> > --- /dev/null
> > +++ b/include/linux/resctrl_types.h
> > @@ -0,0 +1,68 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2024 Arm Ltd.
> > + * Based on arch/x86/kernel/cpu/resctrl/internal.h
> > + */
> 
> Could this header please explain how this file is intended to be used?
> What is it intended to contain?
> 
> Reinette

Maybe something like the following?

 * Resctrl types and constants needed for inline function definitions in
 * the arch-specific <asm/resctrl.h> headers.

(James might have had other things in mind, but this is what it looks
like to me...)

Cheers
---Dave

Re: [PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[Reinette Chatre]

Hi Dave,

On 4/12/2024 9:17 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:18:00PM -0700, Reinette Chatre wrote:
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> To avoid sticky problems in the mpam glue code, move the resctrl
>>> enums into a separate header.
>>
>> Could you please elaborate so that "sticky problems in the mpam glue code" is
>> specific about what problems are avoided?
> 
> Maybe just delete the the sticky clause, and leave:
> 
> 	Move the resctrl enums into a separate header.
> 
> ...since the next paragraph explains the rationale?

In the x86 area changelogs start with a context paragraph to
provide reader with foundation to parse the subsequent problem and
solution statements (that are also expected to be in separate
paragraphs in that order). 

>>> This lets the arch code declare prototypes that use these enums without
>>> creating a loop via asm<->linux resctrl.h The same logic applies to the
>>> monitor-configuration defines, move these too.
>>>
>>> The maintainers entry for these headers was missed when resctrl.h
>>> was created. Add a wildcard entry to match both resctrl.h and
>>> resctrl_types.h.
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>
>> ..
>>
>>> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
>>> new file mode 100644
>>> index 000000000000..4788bd95dac6
>>> --- /dev/null
>>> +++ b/include/linux/resctrl_types.h
>>> @@ -0,0 +1,68 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Copyright (C) 2024 Arm Ltd.
>>> + * Based on arch/x86/kernel/cpu/resctrl/internal.h
>>> + */
>>
>> Could this header please explain how this file is intended to be used?
>> What is it intended to contain?
>>
>> Reinette
> 
> Maybe something like the following?
> 
>  * Resctrl types and constants needed for inline function definitions in
>  * the arch-specific <asm/resctrl.h> headers.
> 

ok.

Reinette

Re: [PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[Dave Martin]

On Mon, Apr 15, 2024 at 11:03:05AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/12/2024 9:17 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:18:00PM -0700, Reinette Chatre wrote:
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> >>> To avoid sticky problems in the mpam glue code, move the resctrl
> >>> enums into a separate header.
> >>
> >> Could you please elaborate so that "sticky problems in the mpam glue code" is
> >> specific about what problems are avoided?
> > 
> > Maybe just delete the the sticky clause, and leave:
> > 
> > 	Move the resctrl enums into a separate header.
> > 
> > ...since the next paragraph explains the rationale?
> 
> In the x86 area changelogs start with a context paragraph to
> provide reader with foundation to parse the subsequent problem and
> solution statements (that are also expected to be in separate
> paragraphs in that order). 

Acknowledged; I was hoping to avoid a rewrite, but ...
> 
> >>> This lets the arch code declare prototypes that use these enums without
> >>> creating a loop via asm<->linux resctrl.h The same logic applies to the
> >>> monitor-configuration defines, move these too.

[...]

OK, how about the following:

--8<--

When resctrl is fully factored into core and per-arch code, each arch
will need to use some resctrl common definitions in order to define its
own specialisations and helpers.  Following conventional practice,
it would be desirable to put the dependent arch definitions in an
<asm/resctrl.h> header that is included by the common <linux/resctrl.h>
header.  However, this can make it awkward to avoid a circular
dependency between <linux/resctrl.h> and the arch header.

To avoid solving this issue via conditional inclusion tricks that are
likely to be tricky to maintain, move the affected common types and
constants into a new header that does not need to depend on
<linux/resctrl.h> or on the arch headers.

The same logic applies to the monitor-configuration defines, move
these too.

-->8--

Then:

> >>>
> >>> The maintainers entry for these headers was missed when resctrl.h
> >>> was created. Add a wildcard entry to match both resctrl.h and
> >>> resctrl_types.h.
> >>>
> >>> Signed-off-by: James Morse <james.morse@arm.com>

The last paragraph looks like it can stand as-is.

Does that look OK?


[...]

> >>> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> >>> new file mode 100644
> >>> index 000000000000..4788bd95dac6
> >>> --- /dev/null
> >>> +++ b/include/linux/resctrl_types.h
> >>> @@ -0,0 +1,68 @@
> >>> +/* SPDX-License-Identifier: GPL-2.0 */
> >>> +/*
> >>> + * Copyright (C) 2024 Arm Ltd.
> >>> + * Based on arch/x86/kernel/cpu/resctrl/internal.h
> >>> + */
> >>
> >> Could this header please explain how this file is intended to be used?
> >> What is it intended to contain?
> >>
> >> Reinette
> > 
> > Maybe something like the following?
> > 
> >  * Resctrl types and constants needed for inline function definitions in
> >  * the arch-specific <asm/resctrl.h> headers.
> > 
> 
> ok.
> 
> Reinette
> 

OK, I'll propose to add that.

Cheers
---Dave

Re: [PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[Reinette Chatre]

Hi Dave,

On 4/16/2024 9:19 AM, Dave Martin wrote:
> On Mon, Apr 15, 2024 at 11:03:05AM -0700, Reinette Chatre wrote:
>> Hi Dave,
>>
>> On 4/12/2024 9:17 AM, Dave Martin wrote:
>>> On Mon, Apr 08, 2024 at 08:18:00PM -0700, Reinette Chatre wrote:
>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>>> To avoid sticky problems in the mpam glue code, move the resctrl
>>>>> enums into a separate header.
>>>>
>>>> Could you please elaborate so that "sticky problems in the mpam glue code" is
>>>> specific about what problems are avoided?
>>>
>>> Maybe just delete the the sticky clause, and leave:
>>>
>>> 	Move the resctrl enums into a separate header.
>>>
>>> ...since the next paragraph explains the rationale?
>>
>> In the x86 area changelogs start with a context paragraph to
>> provide reader with foundation to parse the subsequent problem and
>> solution statements (that are also expected to be in separate
>> paragraphs in that order). 
> 
> Acknowledged; I was hoping to avoid a rewrite, but ...
>>
>>>>> This lets the arch code declare prototypes that use these enums without
>>>>> creating a loop via asm<->linux resctrl.h The same logic applies to the
>>>>> monitor-configuration defines, move these too.
> 
> [...]
> 
> OK, how about the following:
> 
> --8<--
> 
> When resctrl is fully factored into core and per-arch code, each arch
> will need to use some resctrl common definitions in order to define its
> own specialisations and helpers.  Following conventional practice,

specializations

> it would be desirable to put the dependent arch definitions in an
> <asm/resctrl.h> header that is included by the common <linux/resctrl.h>
> header.  However, this can make it awkward to avoid a circular
> dependency between <linux/resctrl.h> and the arch header.
> 
> To avoid solving this issue via conditional inclusion tricks that are
> likely to be tricky to maintain, move the affected common types and

To help with motivation please be specific (somebody may interpret above
that it may not be tricky to maintain). So just ... "that are difficult
to maintain ...".

> constants into a new header that does not need to depend on
> <linux/resctrl.h> or on the arch headers.
> 
> The same logic applies to the monitor-configuration defines, move
> these too.
> 
> -->8--
> 

This explains the motivation for this file well, but its contents
is not obvious to me and after reading [1] I am more weary of including
code before use. Not all of these definitions are needed
by the end of this series so there needs to be a good motivation for
making things global without any visible user.

I do understand that in the next stage of this work we may need to deal
with potentially three subsystems when making changes and there is thus 
a strong motivation for laying a good foundation now. I do not think this
should be an excuse to not be diligent in ensuring the changes are
required.

> Then:
> 
>>>>>
>>>>> The maintainers entry for these headers was missed when resctrl.h
>>>>> was created. Add a wildcard entry to match both resctrl.h and
>>>>> resctrl_types.h.
>>>>>
>>>>> Signed-off-by: James Morse <james.morse@arm.com>
> 
> The last paragraph looks like it can stand as-is.
> 
> Does that look OK?

Yes.

Reinette

[1] https://lore.kernel.org/lkml/ZhecyLQsGZ9Iv8wU@gmail.com/

Re: [PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[Dave Martin]

On Wed, Apr 17, 2024 at 10:15:57PM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/16/2024 9:19 AM, Dave Martin wrote:
> > On Mon, Apr 15, 2024 at 11:03:05AM -0700, Reinette Chatre wrote:
> >> Hi Dave,
> >>
> >> On 4/12/2024 9:17 AM, Dave Martin wrote:
> >>> On Mon, Apr 08, 2024 at 08:18:00PM -0700, Reinette Chatre wrote:
> >>>> On 3/21/2024 9:50 AM, James Morse wrote:
> >>>>> To avoid sticky problems in the mpam glue code, move the resctrl
> >>>>> enums into a separate header.
> >>>>
> >>>> Could you please elaborate so that "sticky problems in the mpam glue code" is
> >>>> specific about what problems are avoided?
> >>>
> >>> Maybe just delete the the sticky clause, and leave:
> >>>
> >>> 	Move the resctrl enums into a separate header.
> >>>
> >>> ...since the next paragraph explains the rationale?
> >>
> >> In the x86 area changelogs start with a context paragraph to
> >> provide reader with foundation to parse the subsequent problem and
> >> solution statements (that are also expected to be in separate
> >> paragraphs in that order). 
> > 
> > Acknowledged; I was hoping to avoid a rewrite, but ...
> >>
> >>>>> This lets the arch code declare prototypes that use these enums without
> >>>>> creating a loop via asm<->linux resctrl.h The same logic applies to the
> >>>>> monitor-configuration defines, move these too.
> > 
> > [...]
> > 
> > OK, how about the following:
> > 
> > --8<--
> > 
> > When resctrl is fully factored into core and per-arch code, each arch
> > will need to use some resctrl common definitions in order to define its
> > own specialisations and helpers.  Following conventional practice,
> 
> specializations

Debatable, but OK, fine.

> > it would be desirable to put the dependent arch definitions in an
> > <asm/resctrl.h> header that is included by the common <linux/resctrl.h>
> > header.  However, this can make it awkward to avoid a circular
> > dependency between <linux/resctrl.h> and the arch header.
> > 
> > To avoid solving this issue via conditional inclusion tricks that are
> > likely to be tricky to maintain, move the affected common types and
> 
> To help with motivation please be specific (somebody may interpret above
> that it may not be tricky to maintain). So just ... "that are difficult
> to maintain ...".

Rather than the text encouraging questions about whether there are
reasonable alternative approaches, perhaps this can just be, simply:

"To avoid such dependencies, move the affected types into a new
header [...]"

?

> 
> > constants into a new header that does not need to depend on
> > <linux/resctrl.h> or on the arch headers.
> > 
> > The same logic applies to the monitor-configuration defines, move
> > these too.
> > 
> > -->8--
> > 
> 
> This explains the motivation for this file well, but its contents
> is not obvious to me and after reading [1] I am more weary of including
> code before use. Not all of these definitions are needed
> by the end of this series so there needs to be a good motivation for
> making things global without any visible user.

That's fair.  I guess we need to review the contents of this header and
make sure that everything that's here really should be here.

However, this is not user ABI and there are only 1.5 users of this
interface (given that MPAM is not yet merged).  So, the penalty for
not getting this quite right and fixing it later seems low.

If you agree that adding this header is appropriate, are you OK with
some post-merge cleanup, or do you think it's essential to sanitise this
fully up-front?

> 
> I do understand that in the next stage of this work we may need to deal
> with potentially three subsystems when making changes and there is thus 
> a strong motivation for laying a good foundation now. I do not think this
> should be an excuse to not be diligent in ensuring the changes are
> required.
> 
> > Then:
> > 
> >>>>>
> >>>>> The maintainers entry for these headers was missed when resctrl.h
> >>>>> was created. Add a wildcard entry to match both resctrl.h and
> >>>>> resctrl_types.h.
> >>>>>
> >>>>> Signed-off-by: James Morse <james.morse@arm.com>
> > 
> > The last paragraph looks like it can stand as-is.
> > 
> > Does that look OK?
> 
> Yes.
> 
> Reinette
> 
> [1] https://lore.kernel.org/lkml/ZhecyLQsGZ9Iv8wU@gmail.com/
>  

Cheers
---Dave

Re: [PATCH v1 08/31] x86/resctrl: Move resctrl types to a separate header

[Reinette Chatre]

Hi Dave,

On 4/18/2024 8:25 AM, Dave Martin wrote:
> On Wed, Apr 17, 2024 at 10:15:57PM -0700, Reinette Chatre wrote:
>> On 4/16/2024 9:19 AM, Dave Martin wrote:
>>> On Mon, Apr 15, 2024 at 11:03:05AM -0700, Reinette Chatre wrote:
>>>> On 4/12/2024 9:17 AM, Dave Martin wrote:
>>>>> On Mon, Apr 08, 2024 at 08:18:00PM -0700, Reinette Chatre
>>>>> wrote:
>>>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>>>>> To avoid sticky problems in the mpam glue code, move the
>>>>>>> resctrl enums into a separate header.
>>>>>> 
>>>>>> Could you please elaborate so that "sticky problems in the
>>>>>> mpam glue code" is specific about what problems are
>>>>>> avoided?
>>>>> 
>>>>> Maybe just delete the the sticky clause, and leave:
>>>>> 
>>>>> Move the resctrl enums into a separate header.
>>>>> 
>>>>> ...since the next paragraph explains the rationale?
>>>> 
>>>> In the x86 area changelogs start with a context paragraph to 
>>>> provide reader with foundation to parse the subsequent problem
>>>> and solution statements (that are also expected to be in
>>>> separate paragraphs in that order).
>>> 
>>> Acknowledged; I was hoping to avoid a rewrite, but ...
>>>> 
>>>>>>> This lets the arch code declare prototypes that use these
>>>>>>> enums without creating a loop via asm<->linux resctrl.h
>>>>>>> The same logic applies to the monitor-configuration
>>>>>>> defines, move these too.
>>> 
>>> [...]
>>> 
>>> OK, how about the following:
>>> 
>>> --8<--
>>> 
>>> When resctrl is fully factored into core and per-arch code, each
>>> arch will need to use some resctrl common definitions in order to
>>> define its own specialisations and helpers.  Following
>>> conventional practice,
>> 
>> specializations
> 
> Debatable, but OK, fine.

ah British spelling, apologies.

> 
>>> it would be desirable to put the dependent arch definitions in
>>> an <asm/resctrl.h> header that is included by the common
>>> <linux/resctrl.h> header.  However, this can make it awkward to
>>> avoid a circular dependency between <linux/resctrl.h> and the
>>> arch header.
>>> 
>>> To avoid solving this issue via conditional inclusion tricks that
>>> are likely to be tricky to maintain, move the affected common
>>> types and
>> 
>> To help with motivation please be specific (somebody may interpret
>> above that it may not be tricky to maintain). So just ... "that are
>> difficult to maintain ...".
> 
> Rather than the text encouraging questions about whether there are 
> reasonable alternative approaches, perhaps this can just be, simply:
> 
> "To avoid such dependencies, move the affected types into a new 
> header [...]"
> 
> ?

Sure.

> 
>> 
>>> constants into a new header that does not need to depend on 
>>> <linux/resctrl.h> or on the arch headers.
>>> 
>>> The same logic applies to the monitor-configuration defines,
>>> move these too.
>>> 
>>> -->8--
>>> 
>> 
>> This explains the motivation for this file well, but its contents 
>> is not obvious to me and after reading [1] I am more weary of
>> including code before use. Not all of these definitions are needed 
>> by the end of this series so there needs to be a good motivation
>> for making things global without any visible user.
> 
> That's fair.  I guess we need to review the contents of this header
> and make sure that everything that's here really should be here.
> 
> However, this is not user ABI and there are only 1.5 users of this 
> interface (given that MPAM is not yet merged).  So, the penalty for 
> not getting this quite right and fixing it later seems low.
> 
> If you agree that adding this header is appropriate, are you OK with 
> some post-merge cleanup, or do you think it's essential to sanitise
> this fully up-front?
> 

I think you may have sent this before your response to patch #17 where you
are talking about keeping some definitions x86 specific until their usage is
clear.

I understand this is not user ABI and as I also stated previously I recognize
that these changes are easier now than later when changes need to cross two
subsystems. I do not think the goal should be to have the perfect header file
but I would like to understand why each definition in it needs to be global.

Unfortunately, based on learnings during the four year history of this work,
I do not have confidence that there will be post-merge cleanup. 

Reinette

[PATCH v1 09/31] x86/resctrl: Add a resctrl helper to reset all the resources

[James Morse]

On umount(), resctrl resets each resource back to its default
configuration. It only ever does this for all resources in one go.

reset_all_ctrls() is architecture specific as it works with struct
rdt_hw_resource.

Add an architecture helper to reset all resources.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/include/asm/resctrl.h         |  2 ++
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +++++++++++-----
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index f61382258743..5f6a5375bb4a 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -15,6 +15,8 @@
  */
 #define X86_RESCTRL_EMPTY_CLOSID         ((u32)~0)
 
+void resctrl_arch_reset_resources(void);
+
 /**
  * struct resctrl_pqr_state - State cache for the PQR MSR
  * @cur_rmid:		The cached Resource Monitoring ID
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 1a49c9918f8d..13c24cb18d76 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -2859,6 +2859,14 @@ static int reset_all_ctrls(struct rdt_resource *r)
 	return 0;
 }
 
+void resctrl_arch_reset_resources(void)
+{
+	struct rdt_resource *r;
+
+	for_each_capable_rdt_resource(r)
+		reset_all_ctrls(r);
+}
+
 /*
  * Move tasks from one to the other group. If @from is NULL, then all tasks
  * in the systems are moved unconditionally (used for teardown).
@@ -2968,16 +2976,14 @@ static void rmdir_all_sub(void)
 
 static void rdt_kill_sb(struct super_block *sb)
 {
-	struct rdt_resource *r;
-
 	cpus_read_lock();
 	mutex_lock(&rdtgroup_mutex);
 
 	rdt_disable_ctx();
 
-	/*Put everything back to default values. */
-	for_each_alloc_capable_rdt_resource(r)
-		reset_all_ctrls(r);
+	/* Put everything back to default values. */
+	resctrl_arch_reset_resources();
+
 	rmdir_all_sub();
 	rdt_pseudo_lock_release();
 	rdtgroup_default.mode = RDT_MODE_SHAREABLE;
-- 
2.39.2

RE: [PATCH v1 09/31] x86/resctrl: Add a resctrl helper to reset all the resources

[Shaopeng Tan (Fujitsu)]

Hello James,

> On umount(), resctrl resets each resource back to its default configuration. It
> only ever does this for all resources in one go.
> 
> reset_all_ctrls() is architecture specific as it works with struct
> rdt_hw_resource.
> 
> Add an architecture helper to reset all resources.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/include/asm/resctrl.h         |  2 ++
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +++++++++++-----
>  2 files changed, 13 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
> index f61382258743..5f6a5375bb4a 100644
> --- a/arch/x86/include/asm/resctrl.h
> +++ b/arch/x86/include/asm/resctrl.h
> @@ -15,6 +15,8 @@
>   */
>  #define X86_RESCTRL_EMPTY_CLOSID         ((u32)~0)
> 
> +void resctrl_arch_reset_resources(void);
> +
>  /**
>   * struct resctrl_pqr_state - State cache for the PQR MSR
>   * @cur_rmid:		The cached Resource Monitoring ID
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 1a49c9918f8d..13c24cb18d76 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -2859,6 +2859,14 @@ static int reset_all_ctrls(struct rdt_resource *r)
>  	return 0;
>  }
> 
> +void resctrl_arch_reset_resources(void)
> +{
> +	struct rdt_resource *r;
> +
> +	for_each_capable_rdt_resource(r)
Could you explain why not "for_each_alloc_capable_rdt_resource(r)"?

> +		reset_all_ctrls(r);
> +}
> +
>  /*
>   * Move tasks from one to the other group. If @from is NULL, then all tasks
>   * in the systems are moved unconditionally (used for teardown).
> @@ -2968,16 +2976,14 @@ static void rmdir_all_sub(void)
> 
>  static void rdt_kill_sb(struct super_block *sb)  {
> -	struct rdt_resource *r;
> -
>  	cpus_read_lock();
>  	mutex_lock(&rdtgroup_mutex);
> 
>  	rdt_disable_ctx();
> 
> -	/*Put everything back to default values. */
> -	for_each_alloc_capable_rdt_resource(r)
This was "for_each_alloc_capable_rdt_resource(r)".

Best regards, 
Shaopeng TAN

> -		reset_all_ctrls(r);
> +	/* Put everything back to default values. */
> +	resctrl_arch_reset_resources();
> +
>  	rmdir_all_sub();
>  	rdt_pseudo_lock_release();
>  	rdtgroup_default.mode = RDT_MODE_SHAREABLE;
> --
> 2.39.2

Re: [PATCH v1 09/31] x86/resctrl: Add a resctrl helper to reset all the resources

[Dave Martin]

On Wed, Apr 03, 2024 at 07:52:42AM +0000, Shaopeng Tan (Fujitsu) wrote:
> Hello James,
> 
> > On umount(), resctrl resets each resource back to its default configuration. It
> > only ever does this for all resources in one go.
> > 
> > reset_all_ctrls() is architecture specific as it works with struct
> > rdt_hw_resource.
> > 
> > Add an architecture helper to reset all resources.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/include/asm/resctrl.h         |  2 ++
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +++++++++++-----
> >  2 files changed, 13 insertions(+), 5 deletions(-)
> > 
> > diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
> > index f61382258743..5f6a5375bb4a 100644
> > --- a/arch/x86/include/asm/resctrl.h
> > +++ b/arch/x86/include/asm/resctrl.h
> > @@ -15,6 +15,8 @@
> >   */
> >  #define X86_RESCTRL_EMPTY_CLOSID         ((u32)~0)
> > 
> > +void resctrl_arch_reset_resources(void);
> > +
> >  /**
> >   * struct resctrl_pqr_state - State cache for the PQR MSR
> >   * @cur_rmid:		The cached Resource Monitoring ID
> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index 1a49c9918f8d..13c24cb18d76 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > @@ -2859,6 +2859,14 @@ static int reset_all_ctrls(struct rdt_resource *r)
> >  	return 0;
> >  }
> > 
> > +void resctrl_arch_reset_resources(void)
> > +{
> > +	struct rdt_resource *r;
> > +
> > +	for_each_capable_rdt_resource(r)
> Could you explain why not "for_each_alloc_capable_rdt_resource(r)"?
> 
> > +		reset_all_ctrls(r);
> > +}
> > +
> >  /*
> >   * Move tasks from one to the other group. If @from is NULL, then all tasks
> >   * in the systems are moved unconditionally (used for teardown).
> > @@ -2968,16 +2976,14 @@ static void rmdir_all_sub(void)
> > 
> >  static void rdt_kill_sb(struct super_block *sb)  {
> > -	struct rdt_resource *r;
> > -
> >  	cpus_read_lock();
> >  	mutex_lock(&rdtgroup_mutex);
> > 
> >  	rdt_disable_ctx();
> > 
> > -	/*Put everything back to default values. */
> > -	for_each_alloc_capable_rdt_resource(r)
> This was "for_each_alloc_capable_rdt_resource(r)".
> 
> Best regards, 
> Shaopeng TAN

Noted for James to take a look at, thanks.

If this was intentional, it probably needs more explanation -- it looks
quite likely to have been unintentional, though.

Cheers
---Dave

[PATCH v1 10/31] x86/resctrl: Move monitor init work to a resctrl init call

[James Morse]

rdt_get_mon_l3_config() is called from the architecture's
resctrl_arch_late_init(), and initialises both architecture specific
fields, such as hw_res->mon_scale and resctrl filesystem fields
by calling dom_data_init().

To separate the filesystem and architecture parts of resctrl, this
function needs splitting up.

Add resctrl_mon_resource_init() to do the filesystem specific work,
and call it from resctrl_init(). This runs later, but is still before
the filesystem is mounted and the rmid_ptrs[] array can be used.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/internal.h |  1 +
 arch/x86/kernel/cpu/resctrl/monitor.c  | 24 +++++++++++++++++-------
 arch/x86/kernel/cpu/resctrl/rdtgroup.c |  4 ++++
 3 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 031948322eab..7a0c74779c53 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -540,6 +540,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg);
 void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
 		    int evtid, int first);
+int resctrl_mon_resource_init(void);
 void mbm_setup_overflow_handler(struct rdt_domain *dom,
 				unsigned long delay_ms,
 				int exclude_cpu);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 06565153ceb2..929ec1430b45 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -1003,12 +1003,28 @@ static void l3_mon_evt_init(struct rdt_resource *r)
 		list_add_tail(&mbm_local_event.list, &r->evt_list);
 }
 
+int resctrl_mon_resource_init(void)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+	int ret;
+
+	if (!r->mon_capable)
+		return 0;
+
+	ret = dom_data_init(r);
+	if (ret)
+		return ret;
+
+	l3_mon_evt_init(r);
+
+	return 0;
+}
+
 int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 {
 	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	unsigned int threshold;
-	int ret;
 
 	resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size * 1024;
 	hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale;
@@ -1036,10 +1052,6 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 	 */
 	resctrl_rmid_realloc_threshold = resctrl_arch_round_mon_val(threshold);
 
-	ret = dom_data_init(r);
-	if (ret)
-		return ret;
-
 	if (rdt_cpu_has(X86_FEATURE_BMEC)) {
 		u32 eax, ebx, ecx, edx;
 
@@ -1057,8 +1069,6 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 		}
 	}
 
-	l3_mon_evt_init(r);
-
 	r->mon_capable = true;
 
 	return 0;
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 13c24cb18d76..7a9696f53f2b 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -4138,6 +4138,10 @@ int __init resctrl_init(void)
 
 	rdtgroup_setup_default();
 
+	ret = resctrl_mon_resource_init();
+	if (ret)
+		return ret;
+
 	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
 	if (ret)
 		return ret;
-- 
2.39.2

Re: [PATCH v1 10/31] x86/resctrl: Move monitor init work to a resctrl init call

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:

...

> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 13c24cb18d76..7a9696f53f2b 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -4138,6 +4138,10 @@ int __init resctrl_init(void)
>  
>  	rdtgroup_setup_default();
>  
> +	ret = resctrl_mon_resource_init();
> +	if (ret)
> +		return ret;
> +
>  	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
>  	if (ret)
>  		return ret;

Should the mon data be cleaned up if sysfs_create_mount_point() fails?

Reinette 

Re: [PATCH v1 10/31] x86/resctrl: Move monitor init work to a resctrl init call

[Shawn Wang]

Hi James,

On 3/22/24 12:50 AM, James Morse wrote:
> rdt_get_mon_l3_config() is called from the architecture's
> resctrl_arch_late_init(), and initialises both architecture specific
> fields, such as hw_res->mon_scale and resctrl filesystem fields
> by calling dom_data_init().
> 
> To separate the filesystem and architecture parts of resctrl, this
> function needs splitting up.
> 
> Add resctrl_mon_resource_init() to do the filesystem specific work,
> and call it from resctrl_init(). This runs later, but is still before
> the filesystem is mounted and the rmid_ptrs[] array can be used.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>   arch/x86/kernel/cpu/resctrl/internal.h |  1 +
>   arch/x86/kernel/cpu/resctrl/monitor.c  | 24 +++++++++++++++++-------
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c |  4 ++++
>   3 files changed, 22 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index 031948322eab..7a0c74779c53 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -540,6 +540,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg);
>   void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
>   		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
>   		    int evtid, int first);
> +int resctrl_mon_resource_init(void);
>   void mbm_setup_overflow_handler(struct rdt_domain *dom,
>   				unsigned long delay_ms,
>   				int exclude_cpu);
> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index 06565153ceb2..929ec1430b45 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -1003,12 +1003,28 @@ static void l3_mon_evt_init(struct rdt_resource *r)
>   		list_add_tail(&mbm_local_event.list, &r->evt_list);
>   }
>   
> +int resctrl_mon_resource_init(void)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	int ret;
> +
> +	if (!r->mon_capable)
> +		return 0;
> +
> +	ret = dom_data_init(r);
> +	if (ret)
> +		return ret;
> +
> +	l3_mon_evt_init(r);
> +
> +	return 0;
> +}
> +

Now x86 platform defaults to all monitoring features on the L3 cache, but some monitoring features may also be on the L2 cache or memory controller according to the MPAM spec. For example, the memory bandwidth monitors could be on the memory controller.

Will resctrl_mon_resource_init() consider this scenario?

Best Regards,
Shawn

Re: [PATCH v1 10/31] x86/resctrl: Move monitor init work to a resctrl init call

[Dave Martin]

On Wed, Apr 10, 2024 at 10:57:30AM +0800, Shawn Wang wrote:
> Hi James,
> 
> On 3/22/24 12:50 AM, James Morse wrote:
> > rdt_get_mon_l3_config() is called from the architecture's
> > resctrl_arch_late_init(), and initialises both architecture specific
> > fields, such as hw_res->mon_scale and resctrl filesystem fields
> > by calling dom_data_init().
> > 
> > To separate the filesystem and architecture parts of resctrl, this
> > function needs splitting up.
> > 
> > Add resctrl_mon_resource_init() to do the filesystem specific work,
> > and call it from resctrl_init(). This runs later, but is still before
> > the filesystem is mounted and the rmid_ptrs[] array can be used.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >   arch/x86/kernel/cpu/resctrl/internal.h |  1 +
> >   arch/x86/kernel/cpu/resctrl/monitor.c  | 24 +++++++++++++++++-------
> >   arch/x86/kernel/cpu/resctrl/rdtgroup.c |  4 ++++
> >   3 files changed, 22 insertions(+), 7 deletions(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> > index 031948322eab..7a0c74779c53 100644
> > --- a/arch/x86/kernel/cpu/resctrl/internal.h
> > +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> > @@ -540,6 +540,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg);
> >   void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
> >   		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
> >   		    int evtid, int first);
> > +int resctrl_mon_resource_init(void);
> >   void mbm_setup_overflow_handler(struct rdt_domain *dom,
> >   				unsigned long delay_ms,
> >   				int exclude_cpu);
> > diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> > index 06565153ceb2..929ec1430b45 100644
> > --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> > +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> > @@ -1003,12 +1003,28 @@ static void l3_mon_evt_init(struct rdt_resource *r)
> >   		list_add_tail(&mbm_local_event.list, &r->evt_list);
> >   }
> > +int resctrl_mon_resource_init(void)
> > +{
> > +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> > +	int ret;
> > +
> > +	if (!r->mon_capable)
> > +		return 0;
> > +
> > +	ret = dom_data_init(r);
> > +	if (ret)
> > +		return ret;
> > +
> > +	l3_mon_evt_init(r);
> > +
> > +	return 0;
> > +}
> > +
> 
> Now x86 platform defaults to all monitoring features on the L3 cache, but some monitoring features may also be on the L2 cache or memory controller according to the MPAM spec. For example, the memory bandwidth monitors could be on the memory controller.
> 
> Will resctrl_mon_resource_init() consider this scenario?
> 
> Best Regards,
> Shawn

My understanding is limited here, so this will probably have to wait
until James gets back for a full answer.

The resctrl userspace filesystem interface looks like it can describe
monitors at multiple levels of the memory system, so my assumption would
be that it should be possible to wire it up.

This series focuses on refactoring and does not try to add new
functionality, though.

Cheers
---Dave

Re: [PATCH v1 10/31] x86/resctrl: Move monitor init work to a resctrl init call

[James Morse]

Hi Shawn,

On 10/04/2024 03:57, Shawn Wang wrote:
> On 3/22/24 12:50 AM, James Morse wrote:
>> rdt_get_mon_l3_config() is called from the architecture's
>> resctrl_arch_late_init(), and initialises both architecture specific
>> fields, such as hw_res->mon_scale and resctrl filesystem fields
>> by calling dom_data_init().
>>
>> To separate the filesystem and architecture parts of resctrl, this
>> function needs splitting up.
>>
>> Add resctrl_mon_resource_init() to do the filesystem specific work,
>> and call it from resctrl_init(). This runs later, but is still before
>> the filesystem is mounted and the rmid_ptrs[] array can be used.


> Now x86 platform defaults to all monitoring features on the L3 cache, but some monitoring
> features may also be on the L2 cache or memory controller according to the MPAM spec. For
> example, the memory bandwidth monitors could be on the memory controller.
> 
> Will resctrl_mon_resource_init() consider this scenario?

Nope.

The aim here is to get existing user-space software working on machines with MPAM.
You can build an MPAM machine with monitors on the L1, L2 and L5 - but resctrl can't
expose them, and there is no existing software that makes use of them.

The result is the MPAM driver plays fast and loose with some of this stuff to try and slot
the machine into a Xeon shaped hole.

I'd like to support this in the future, but it will need user visible filesystem changes -
its not worth discussing now.
I strongly suspect that declaring all monitors platform-specific, and plumbing them out
via perf is the best thing for everyone...



Thanks,

James

[PATCH v1 11/31] x86/resctrl: Move monitor exit work to a restrl exit call

[James Morse]

rdt_put_mon_l3_config() is called via the architecture's
resctrl_arch_exit() call, and appears to free the rmid_ptrs[]
and closid_num_dirty_rmid[] arrays. In reality this code is marked
__exit, and is removed by the linker as resctl can't be built
as a module.

MPAM can make use of this code from its error interrupt handler,
a later patch drops all the __init/__exit annotations.

To separate the filesystem and architecture parts of resctrl,
this free()ing work needs to be triggered by the filesystem,
as these structures belong to the filesystem code.

Rename rdt_put_mon_l3_config() resctrl_mon_resource_exit()
and call it from resctrl_exit(). The kfree() is currently
dependent on r->mon_capable. resctrl_mon_resource_init()
takes no arguments, so resctrl_mon_resource_exit() shouldn't
take any either. Add the check to dom_data_exit(), making it
take the resource as an argument. This makes it more symmetrical
with dom_data_init().

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c     |  5 -----
 arch/x86/kernel/cpu/resctrl/internal.h |  2 +-
 arch/x86/kernel/cpu/resctrl/monitor.c  | 12 ++++++++----
 arch/x86/kernel/cpu/resctrl/rdtgroup.c |  2 ++
 4 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 06dc12b08afc..9a09a64bbb7f 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -988,14 +988,9 @@ late_initcall(resctrl_arch_late_init);
 
 static void __exit resctrl_arch_exit(void)
 {
-	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
-
 	cpuhp_remove_state(rdt_online);
 
 	resctrl_exit();
-
-	if (r->mon_capable)
-		rdt_put_mon_l3_config();
 }
 
 __exitcall(resctrl_arch_exit);
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 7a0c74779c53..01fcd4ef26ca 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -533,7 +533,7 @@ void closid_free(int closid);
 int alloc_rmid(u32 closid);
 void free_rmid(u32 closid, u32 rmid);
 int rdt_get_mon_l3_config(struct rdt_resource *r);
-void __exit rdt_put_mon_l3_config(void);
+void __exit resctrl_mon_resource_exit(void);
 bool __init rdt_cpu_has(int flag);
 void mon_event_count(void *info);
 int rdtgroup_mondata_show(struct seq_file *m, void *arg);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 929ec1430b45..2a1cbd4de6ee 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -954,10 +954,12 @@ static int dom_data_init(struct rdt_resource *r)
 	return err;
 }
 
-static void __exit dom_data_exit(void)
+static void __exit dom_data_exit(struct rdt_resource *r)
 {
-	mutex_lock(&rdtgroup_mutex);
+	if (!r->mon_capable)
+		return;
 
+	mutex_lock(&rdtgroup_mutex);
 	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
 		kfree(closid_num_dirty_rmid);
 		closid_num_dirty_rmid = NULL;
@@ -1074,9 +1076,11 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 	return 0;
 }
 
-void __exit rdt_put_mon_l3_config(void)
+void __exit resctrl_mon_resource_exit(void)
 {
-	dom_data_exit();
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+
+	dom_data_exit(r);
 }
 
 void __init intel_rdt_mbm_apply_quirk(void)
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 7a9696f53f2b..6cf4ebe9c058 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -4186,4 +4186,6 @@ void __exit resctrl_exit(void)
 	debugfs_remove_recursive(debugfs_resctrl);
 	unregister_filesystem(&rdt_fs_type);
 	sysfs_remove_mount_point(fs_kobj, "resctrl");
+
+	resctrl_mon_resource_exit();
 }
-- 
2.39.2

Re: [PATCH v1 11/31] x86/resctrl: Move monitor exit work to a restrl exit call

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> rdt_put_mon_l3_config() is called via the architecture's
> resctrl_arch_exit() call, and appears to free the rmid_ptrs[]
> and closid_num_dirty_rmid[] arrays. In reality this code is marked
> __exit, and is removed by the linker as resctl can't be built

resctl -> resctrl

> as a module.
> 
> MPAM can make use of this code from its error interrupt handler,
> a later patch drops all the __init/__exit annotations.

Reminder:
https://lore.kernel.org/lkml/85f8756a-23a0-444e-b37d-a7817b600f46@intel.com/

Reinette

Re: [PATCH v1 11/31] x86/resctrl: Move monitor exit work to a restrl exit call

[Dave Martin]

On Mon, Apr 08, 2024 at 08:19:00PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > rdt_put_mon_l3_config() is called via the architecture's
> > resctrl_arch_exit() call, and appears to free the rmid_ptrs[]
> > and closid_num_dirty_rmid[] arrays. In reality this code is marked
> > __exit, and is removed by the linker as resctl can't be built
> 
> resctl -> resctrl

Noted, thanks (also, there is "restrl" in the subject line.)

> 
> > as a module.
> > 
> > MPAM can make use of this code from its error interrupt handler,
> > a later patch drops all the __init/__exit annotations.
> 
> Reminder:
> https://lore.kernel.org/lkml/85f8756a-23a0-444e-b37d-a7817b600f46@intel.com/

The "Drop __init/__exit on assorted symbols" patch speaks for itself,
I guess. I think it's probably sufficient for now to comfirm that this
patch is deliberately not changing the annotations for now.

Does the following work?

--8<--

Since there is no immediate need to change them, leave the __exit
annotations as-is.  This will need to be revisited as and when there is
a need to call these functions other than at __exit time.

-->8--

Cheers
---Dave

Re: [PATCH v1 11/31] x86/resctrl: Move monitor exit work to a restrl exit call

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:15 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:19:00PM -0700, Reinette Chatre wrote:
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> rdt_put_mon_l3_config() is called via the architecture's
>>> resctrl_arch_exit() call, and appears to free the rmid_ptrs[]
>>> and closid_num_dirty_rmid[] arrays. In reality this code is marked
>>> __exit, and is removed by the linker as resctl can't be built
>>
>> resctl -> resctrl
> 
> Noted, thanks (also, there is "restrl" in the subject line.)

Thank you for catching that.

> 
>>
>>> as a module.
>>>
>>> MPAM can make use of this code from its error interrupt handler,
>>> a later patch drops all the __init/__exit annotations.
>>
>> Reminder:
>> https://lore.kernel.org/lkml/85f8756a-23a0-444e-b37d-a7817b600f46@intel.com/
> 
> The "Drop __init/__exit on assorted symbols" patch speaks for itself,
> I guess. I think it's probably sufficient for now to comfirm that this
> patch is deliberately not changing the annotations for now.
> 
> Does the following work?
> 
> --8<--
> 
> Since there is no immediate need to change them, leave the __exit
> annotations as-is.  This will need to be revisited as and when there is
> a need to call these functions other than at __exit time.
> 
> -->8--

Sounds good. I find a big focus on the __init/__exit annotations
distracting. As I understand this has nothing to do with the goal of
this patch, which is to move code from architecture side to filesystem side. 

Reinette

[PATCH v1 12/31] x86/resctrl: Move max_{name,data}_width into resctrl code

[James Morse]

max_name_width and max_data_width are used to pad the strings in the
resctrl schemata file.

This should be part of the fs code as it influences the user-space
interface, but currently max_data_width is generated by the arch init code.
max_name_width is already managed by schemata_list_add().

Move the variables and max_data_width's initialisation code to rdtgroup.c.
There is no need for an extra rdt_init_padding() helper as the length
of the name can be considered when schemata_list_add() creates each
schema entry.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c     | 22 ----------------------
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++++++++++++
 2 files changed, 12 insertions(+), 22 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 9a09a64bbb7f..9551ca4a6480 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -44,12 +44,6 @@ static DEFINE_MUTEX(domain_list_lock);
  */
 DEFINE_PER_CPU(struct resctrl_pqr_state, pqr_state);
 
-/*
- * Used to store the max resource name width and max resource data width
- * to display the schemata in a tabular format
- */
-int max_name_width, max_data_width;
-
 /*
  * Global boolean for rdt_alloc which is true if any
  * resource allocation is enabled.
@@ -648,20 +642,6 @@ static int resctrl_arch_offline_cpu(unsigned int cpu)
 	return 0;
 }
 
-/*
- * Choose a width for the resource name and resource data based on the
- * resource that has widest name and cbm.
- */
-static __init void rdt_init_padding(void)
-{
-	struct rdt_resource *r;
-
-	for_each_alloc_capable_rdt_resource(r) {
-		if (r->data_width > max_data_width)
-			max_data_width = r->data_width;
-	}
-}
-
 enum {
 	RDT_FLAG_CMT,
 	RDT_FLAG_MBM_TOTAL,
@@ -959,8 +939,6 @@ static int __init resctrl_arch_late_init(void)
 	if (!get_rdt_resources())
 		return -ENODEV;
 
-	rdt_init_padding();
-
 	state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
 				  "x86/resctrl/cat:online:",
 				  resctrl_arch_online_cpu,
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 6cf4ebe9c058..e736e4d20f63 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -58,6 +58,12 @@ static struct kernfs_node *kn_mongrp;
 /* Kernel fs node for "mon_data" directory under root */
 static struct kernfs_node *kn_mondata;
 
+/*
+ * Used to store the max resource name width and max resource data width
+ * to display the schemata in a tabular format
+ */
+int max_name_width, max_data_width;
+
 static struct seq_buf last_cmd_status;
 static char last_cmd_status_buf[512];
 
@@ -2595,6 +2601,12 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
 	if (cl > max_name_width)
 		max_name_width = cl;
 
+	/*
+	 * Choose a width for the resource data based on the resource that has
+	 * widest name and cbm.
+	 */
+	max_data_width = max(max_data_width, r->data_width);
+
 	INIT_LIST_HEAD(&s->list);
 	list_add(&s->list, &resctrl_schema_all);
 
-- 
2.39.2

Re: [PATCH v1 12/31] x86/resctrl: Move max_{name,data}_width into resctrl code

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> @@ -2595,6 +2601,12 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
>  	if (cl > max_name_width)
>  		max_name_width = cl;
>  
> +	/*
> +	 * Choose a width for the resource data based on the resource that has
> +	 * widest name and cbm.

Please check series to ensure upper case is used for acronyms.

> +	 */
> +	max_data_width = max(max_data_width, r->data_width);
> +
>  	INIT_LIST_HEAD(&s->list);
>  	list_add(&s->list, &resctrl_schema_all);
>  

Reinette

Re: [PATCH v1 12/31] x86/resctrl: Move max_{name,data}_width into resctrl code

[Dave Martin]

On Mon, Apr 08, 2024 at 08:19:15PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > @@ -2595,6 +2601,12 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
> >  	if (cl > max_name_width)
> >  		max_name_width = cl;
> >  
> > +	/*
> > +	 * Choose a width for the resource data based on the resource that has
> > +	 * widest name and cbm.
> 
> Please check series to ensure upper case is used for acronyms.

[...]

> Reinette

This patch is just moving existing code around AFAICT.  See:
commit de016df88f23 ("x86/intel_rdt: Update schemata read to show data in tabular format")

Since no new usage of any term is being introduced here, can it be
left as-is?

There seem to be other uses of "cbm" with this sense in the resctrl
code already.

Cheers
---Dave

Re: [PATCH v1 12/31] x86/resctrl: Move max_{name,data}_width into resctrl code

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:15 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:19:15PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> @@ -2595,6 +2601,12 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
>>>  	if (cl > max_name_width)
>>>  		max_name_width = cl;
>>>  
>>> +	/*
>>> +	 * Choose a width for the resource data based on the resource that has
>>> +	 * widest name and cbm.
>>
>> Please check series to ensure upper case is used for acronyms.
> 
> [...]
> 
>> Reinette
> 
> This patch is just moving existing code around AFAICT.  See:
> commit de016df88f23 ("x86/intel_rdt: Update schemata read to show data in tabular format")
> 
> Since no new usage of any term is being introduced here, can it be
> left as-is?
> 
> There seem to be other uses of "cbm" with this sense in the resctrl
> code already.

I am not asking to change all existing usages of these terms but in
any new changes, please use upper case for acronyms.

Reinette

Re: [PATCH v1 12/31] x86/resctrl: Move max_{name,data}_width into resctrl code

[Dave Martin]

Hi Reinette,

On Thu, Apr 11, 2024 at 10:38:20AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:15 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:19:15PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> >>> @@ -2595,6 +2601,12 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
> >>>  	if (cl > max_name_width)
> >>>  		max_name_width = cl;
> >>>  
> >>> +	/*
> >>> +	 * Choose a width for the resource data based on the resource that has
> >>> +	 * widest name and cbm.
> >>
> >> Please check series to ensure upper case is used for acronyms.
> > 
> > [...]
> > 
> >> Reinette
> > 
> > This patch is just moving existing code around AFAICT.  See:
> > commit de016df88f23 ("x86/intel_rdt: Update schemata read to show data in tabular format")
> > 
> > Since no new usage of any term is being introduced here, can it be
> > left as-is?
> > 
> > There seem to be other uses of "cbm" with this sense in the resctrl
> > code already.
> 
> I am not asking to change all existing usages of these terms but in
> any new changes, please use upper case for acronyms.

While there is a general argument to be made here, it sounds from this
like you are not requesting a change to this patch; can you confirm?

Cheers
---Dave

Re: [PATCH v1 12/31] x86/resctrl: Move max_{name,data}_width into resctrl code

[Reinette Chatre]

Hi Dave,

On 4/17/2024 7:41 AM, Dave Martin wrote:
> On Thu, Apr 11, 2024 at 10:38:20AM -0700, Reinette Chatre wrote:
>> On 4/11/2024 7:15 AM, Dave Martin wrote:
>>> On Mon, Apr 08, 2024 at 08:19:15PM -0700, Reinette Chatre wrote:
>>>> Hi James,
>>>>
>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>>> @@ -2595,6 +2601,12 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
>>>>>  	if (cl > max_name_width)
>>>>>  		max_name_width = cl;
>>>>>  
>>>>> +	/*
>>>>> +	 * Choose a width for the resource data based on the resource that has
>>>>> +	 * widest name and cbm.
>>>>
>>>> Please check series to ensure upper case is used for acronyms.
>>>
>>> [...]
>>>
>>>> Reinette
>>>
>>> This patch is just moving existing code around AFAICT.  See:
>>> commit de016df88f23 ("x86/intel_rdt: Update schemata read to show data in tabular format")
>>>
>>> Since no new usage of any term is being introduced here, can it be
>>> left as-is?
>>>
>>> There seem to be other uses of "cbm" with this sense in the resctrl
>>> code already.
>>
>> I am not asking to change all existing usages of these terms but in
>> any new changes, please use upper case for acronyms.
> 
> While there is a general argument to be made here, it sounds from this
> like you are not requesting a change to this patch; can you confirm?

Sorry for confusion. I do think in a small change like this it is a good
opportunity to make sure the style is clean. Since this changes the code
anyway, it might as well ensure the style is clean. So, yes, could
you please use CBM instead of cbm.

The final patch of this series is in a different category altogether and I
do not think style changes will be appropriate in it.

Reinette

Re: [PATCH v1 12/31] x86/resctrl: Move max_{name,data}_width into resctrl code

[Dave Martin]

On Wed, Apr 17, 2024 at 10:16:45PM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/17/2024 7:41 AM, Dave Martin wrote:
> > On Thu, Apr 11, 2024 at 10:38:20AM -0700, Reinette Chatre wrote:
> >> On 4/11/2024 7:15 AM, Dave Martin wrote:
> >>> On Mon, Apr 08, 2024 at 08:19:15PM -0700, Reinette Chatre wrote:
> >>>> Hi James,
> >>>>
> >>>> On 3/21/2024 9:50 AM, James Morse wrote:
> >>>>> @@ -2595,6 +2601,12 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
> >>>>>  	if (cl > max_name_width)
> >>>>>  		max_name_width = cl;
> >>>>>  
> >>>>> +	/*
> >>>>> +	 * Choose a width for the resource data based on the resource that has
> >>>>> +	 * widest name and cbm.
> >>>>
> >>>> Please check series to ensure upper case is used for acronyms.
> >>>
> >>> [...]
> >>>
> >>>> Reinette
> >>>
> >>> This patch is just moving existing code around AFAICT.  See:
> >>> commit de016df88f23 ("x86/intel_rdt: Update schemata read to show data in tabular format")
> >>>
> >>> Since no new usage of any term is being introduced here, can it be
> >>> left as-is?
> >>>
> >>> There seem to be other uses of "cbm" with this sense in the resctrl
> >>> code already.
> >>
> >> I am not asking to change all existing usages of these terms but in
> >> any new changes, please use upper case for acronyms.
> > 
> > While there is a general argument to be made here, it sounds from this
> > like you are not requesting a change to this patch; can you confirm?
> 
> Sorry for confusion. I do think in a small change like this it is a good
> opportunity to make sure the style is clean. Since this changes the code
> anyway, it might as well ensure the style is clean. So, yes, could
> you please use CBM instead of cbm.

OK; I had thought that we might be introducing a new inconsistency here
by making such a change, but looking at upstream, "CBM" is prevalent in
comments in the preexisting x86 code.  I should have checked that before;
apologies for the unnecessary back-and-forth on this.

So, sure, it makes sense to change it.

Noted.

Cheers
---Dave

[PATCH v1 13/31] x86/resctrl: Stop using the for_each_*_rdt_resource() walkers

[James Morse]

The for_each_*_rdt_resource() helpers walk the architectures array
of structures, using the resctrl visible part as an iterator. These
became over-complex when the structures were split into a
filesystem and architecture-specific struct. This approach avoided
the need to touch every call site.

Once the filesystem parts of resctrl are moved to /fs/, both the
architecture's resource array, and the definition of those structures
is no longer accessible. To support resctrl, each architecture would
have to provide equally complex macros.

Change the resctrl code that uses these to walk through the resource_level
enum and check the mon/alloc capable flags instead. Instances in core.c,
and resctrl_arch_reset_resources() remain part of x86's architecture
specific code.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c |  7 +++++-
 arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 30 +++++++++++++++++++----
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
index 884b88e25141..f2315a50ea4f 100644
--- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
+++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
@@ -840,6 +840,7 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm
 bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
 {
 	cpumask_var_t cpu_with_psl;
+	enum resctrl_res_level i;
 	struct rdt_resource *r;
 	struct rdt_domain *d_i;
 	bool ret = false;
@@ -854,7 +855,11 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
 	 * First determine which cpus have pseudo-locked regions
 	 * associated with them.
 	 */
-	for_each_alloc_capable_rdt_resource(r) {
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->alloc_capable)
+			continue;
+
 		list_for_each_entry(d_i, &r->domains, list) {
 			if (d_i->plr)
 				cpumask_or(cpu_with_psl, cpu_with_psl,
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index e736e4d20f63..3f16e7854411 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -98,12 +98,17 @@ void rdt_last_cmd_printf(const char *fmt, ...)
 
 void rdt_staged_configs_clear(void)
 {
+	enum resctrl_res_level i;
 	struct rdt_resource *r;
 	struct rdt_domain *dom;
 
 	lockdep_assert_held(&rdtgroup_mutex);
 
-	for_each_alloc_capable_rdt_resource(r) {
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->alloc_capable)
+			continue;
+
 		list_for_each_entry(dom, &r->domains, list)
 			memset(dom->staged_config, 0, sizeof(dom->staged_config));
 	}
@@ -2181,6 +2186,7 @@ static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
 
 static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
 {
+	enum resctrl_res_level i;
 	struct resctrl_schema *s;
 	struct rdt_resource *r;
 	unsigned long fflags;
@@ -2205,8 +2211,12 @@ static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
 			goto out_destroy;
 	}
 
-	for_each_mon_capable_rdt_resource(r) {
-		fflags = r->fflags | RFTYPE_MON_INFO;
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->mon_capable)
+			continue;
+
+		fflags =  r->fflags | RFTYPE_MON_INFO;
 		sprintf(name, "%s_MON", r->name);
 		ret = rdtgroup_mkdir_info_resdir(r, name, fflags);
 		if (ret)
@@ -2615,10 +2625,15 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
 
 static int schemata_list_create(void)
 {
+	enum resctrl_res_level i;
 	struct rdt_resource *r;
 	int ret = 0;
 
-	for_each_alloc_capable_rdt_resource(r) {
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->alloc_capable)
+			continue;
+
 		if (resctrl_arch_get_cdp_enabled(r->rid)) {
 			ret = schemata_list_add(r, CDP_CODE);
 			if (ret)
@@ -3166,6 +3181,7 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
 			     struct rdtgroup *prgrp,
 			     struct kernfs_node **dest_kn)
 {
+	enum resctrl_res_level i;
 	struct rdt_resource *r;
 	struct kernfs_node *kn;
 	int ret;
@@ -3184,7 +3200,11 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
 	 * Create the subdirectories for each domain. Note that all events
 	 * in a domain like L3 are grouped into a resource whose domain is L3
 	 */
-	for_each_mon_capable_rdt_resource(r) {
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->mon_capable)
+			continue;
+
 		ret = mkdir_mondata_subdir_alldom(kn, r, prgrp);
 		if (ret)
 			goto out_destroy;
-- 
2.39.2

Re: [PATCH v1 13/31] x86/resctrl: Stop using the for_each_*_rdt_resource() walkers

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> The for_each_*_rdt_resource() helpers walk the architectures array

architecture's ?

> of structures, using the resctrl visible part as an iterator. These
> became over-complex when the structures were split into a
> filesystem and architecture-specific struct. This approach avoided
> the need to touch every call site.
> 
> Once the filesystem parts of resctrl are moved to /fs/, both the
> architecture's resource array, and the definition of those structures
> is no longer accessible. To support resctrl, each architecture would
> have to provide equally complex macros.
> 
> Change the resctrl code that uses these to walk through the resource_level
> enum and check the mon/alloc capable flags instead. Instances in core.c,
> and resctrl_arch_reset_resources() remain part of x86's architecture
> specific code.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/pseudo_lock.c |  7 +++++-
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 30 +++++++++++++++++++----
>  2 files changed, 31 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> index 884b88e25141..f2315a50ea4f 100644
> --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> @@ -840,6 +840,7 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm
>  bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
>  {
>  	cpumask_var_t cpu_with_psl;
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	struct rdt_domain *d_i;
>  	bool ret = false;
> @@ -854,7 +855,11 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
>  	 * First determine which cpus have pseudo-locked regions
>  	 * associated with them.
>  	 */
> -	for_each_alloc_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
>  		list_for_each_entry(d_i, &r->domains, list) {
>  			if (d_i->plr)
>  				cpumask_or(cpu_with_psl, cpu_with_psl,
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index e736e4d20f63..3f16e7854411 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -98,12 +98,17 @@ void rdt_last_cmd_printf(const char *fmt, ...)
>  
>  void rdt_staged_configs_clear(void)
>  {
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	struct rdt_domain *dom;
>  
>  	lockdep_assert_held(&rdtgroup_mutex);
>  
> -	for_each_alloc_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
>  		list_for_each_entry(dom, &r->domains, list)
>  			memset(dom->staged_config, 0, sizeof(dom->staged_config));
>  	}
> @@ -2181,6 +2186,7 @@ static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
>  
>  static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
>  {
> +	enum resctrl_res_level i;
>  	struct resctrl_schema *s;
>  	struct rdt_resource *r;
>  	unsigned long fflags;
> @@ -2205,8 +2211,12 @@ static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
>  			goto out_destroy;
>  	}
>  
> -	for_each_mon_capable_rdt_resource(r) {
> -		fflags = r->fflags | RFTYPE_MON_INFO;
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->mon_capable)
> +			continue;
> +
> +		fflags =  r->fflags | RFTYPE_MON_INFO;

Please check spacing.

>  		sprintf(name, "%s_MON", r->name);
>  		ret = rdtgroup_mkdir_info_resdir(r, name, fflags);
>  		if (ret)
> @@ -2615,10 +2625,15 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
>  
>  static int schemata_list_create(void)
>  {
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	int ret = 0;
>  
> -	for_each_alloc_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
>  		if (resctrl_arch_get_cdp_enabled(r->rid)) {
>  			ret = schemata_list_add(r, CDP_CODE);
>  			if (ret)
> @@ -3166,6 +3181,7 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
>  			     struct rdtgroup *prgrp,
>  			     struct kernfs_node **dest_kn)
>  {
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	struct kernfs_node *kn;
>  	int ret;
> @@ -3184,7 +3200,11 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
>  	 * Create the subdirectories for each domain. Note that all events
>  	 * in a domain like L3 are grouped into a resource whose domain is L3
>  	 */
> -	for_each_mon_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->mon_capable)
> +			continue;
> +
>  		ret = mkdir_mondata_subdir_alldom(kn, r, prgrp);
>  		if (ret)
>  			goto out_destroy;

Reinette

Re: [PATCH v1 13/31] x86/resctrl: Stop using the for_each_*_rdt_resource() walkers

[Dave Martin]

On Mon, Apr 08, 2024 at 08:19:40PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > The for_each_*_rdt_resource() helpers walk the architectures array
> 
> architecture's ?

Noted.

> > of structures, using the resctrl visible part as an iterator. These
> > became over-complex when the structures were split into a
> > filesystem and architecture-specific struct. This approach avoided
> > the need to touch every call site.
> > 
> > Once the filesystem parts of resctrl are moved to /fs/, both the
> > architecture's resource array, and the definition of those structures
> > is no longer accessible. To support resctrl, each architecture would
> > have to provide equally complex macros.
> > 
> > Change the resctrl code that uses these to walk through the resource_level
> > enum and check the mon/alloc capable flags instead. Instances in core.c,
> > and resctrl_arch_reset_resources() remain part of x86's architecture
> > specific code.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/pseudo_lock.c |  7 +++++-
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 30 +++++++++++++++++++----
> >  2 files changed, 31 insertions(+), 6 deletions(-)

[...]

> > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > index e736e4d20f63..3f16e7854411 100644
> > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c

[...]

> > @@ -2205,8 +2211,12 @@ static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
> >  			goto out_destroy;
> >  	}
> >  
> > -	for_each_mon_capable_rdt_resource(r) {
> > -		fflags = r->fflags | RFTYPE_MON_INFO;
> > +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> > +		r = resctrl_arch_get_resource(i);
> > +		if (!r->mon_capable)
> > +			continue;
> > +
> > +		fflags =  r->fflags | RFTYPE_MON_INFO;
> 
> Please check spacing.

[...]

> Reinette

Noted, thanks.


Cheers
---Dave

[PATCH v1 14/31] x86/resctrl: Export the is_mbm_*_enabled() helpers to asm/resctrl.h

[James Morse]

The architecture specific parts of resctrl have helpers to hide accesses
to the rdt_mon_features bitmap.

Once the filesystem parts of resctrl are moved, these can no longer live
in internal.h. Once these are exposed to the wider kernel, they should
have a 'resctrl_arch_' prefix, to fit the rest of the arch<->fs interface.

Move and rename the helpers that touch rdt_mon_features directly.
is_mbm_event() and is_mbm_enabled() are only called from rdtgroup.c,
so can be moved into that file.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/include/asm/resctrl.h         | 17 +++++++++++
 arch/x86/kernel/cpu/resctrl/core.c     |  4 +--
 arch/x86/kernel/cpu/resctrl/internal.h | 27 -----------------
 arch/x86/kernel/cpu/resctrl/monitor.c  | 18 ++++++------
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 40 +++++++++++++++++---------
 5 files changed, 54 insertions(+), 52 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 5f6a5375bb4a..50407e83d0ca 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -7,6 +7,7 @@
 #include <linux/jump_label.h>
 #include <linux/percpu.h>
 #include <linux/sched.h>
+#include <linux/resctrl_types.h>
 
 /*
  * This value can never be a valid CLOSID, and is used when mapping a
@@ -43,6 +44,7 @@ DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state);
 
 extern bool rdt_alloc_capable;
 extern bool rdt_mon_capable;
+extern unsigned int rdt_mon_features;
 
 DECLARE_STATIC_KEY_FALSE(rdt_enable_key);
 DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
@@ -82,6 +84,21 @@ static inline void resctrl_arch_disable_mon(void)
 	static_branch_dec_cpuslocked(&rdt_enable_key);
 }
 
+static inline bool resctrl_arch_is_llc_occupancy_enabled(void)
+{
+	return (rdt_mon_features & (1 << QOS_L3_OCCUP_EVENT_ID));
+}
+
+static inline bool resctrl_arch_is_mbm_total_enabled(void)
+{
+	return (rdt_mon_features & (1 << QOS_L3_MBM_TOTAL_EVENT_ID));
+}
+
+static inline bool resctrl_arch_is_mbm_local_enabled(void)
+{
+	return (rdt_mon_features & (1 << QOS_L3_MBM_LOCAL_EVENT_ID));
+}
+
 /*
  * __resctrl_sched_in() - Writes the task's CLOSid/RMID to IA32_PQR_MSR
  *
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 9551ca4a6480..a4b5ade6e291 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -481,13 +481,13 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)
 {
 	size_t tsize;
 
-	if (is_mbm_total_enabled()) {
+	if (resctrl_arch_is_mbm_total_enabled()) {
 		tsize = sizeof(*hw_dom->arch_mbm_total);
 		hw_dom->arch_mbm_total = kcalloc(num_rmid, tsize, GFP_KERNEL);
 		if (!hw_dom->arch_mbm_total)
 			return -ENOMEM;
 	}
-	if (is_mbm_local_enabled()) {
+	if (resctrl_arch_is_mbm_local_enabled()) {
 		tsize = sizeof(*hw_dom->arch_mbm_local);
 		hw_dom->arch_mbm_local = kcalloc(num_rmid, tsize, GFP_KERNEL);
 		if (!hw_dom->arch_mbm_local) {
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 01fcd4ef26ca..edbccc79246f 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -130,7 +130,6 @@ struct rmid_read {
 	void			*arch_mon_ctx;
 };
 
-extern unsigned int rdt_mon_features;
 extern struct list_head resctrl_schema_all;
 extern bool resctrl_mounted;
 
@@ -360,32 +359,6 @@ struct msr_param {
 	u32			high;
 };
 
-static inline bool is_llc_occupancy_enabled(void)
-{
-	return (rdt_mon_features & (1 << QOS_L3_OCCUP_EVENT_ID));
-}
-
-static inline bool is_mbm_total_enabled(void)
-{
-	return (rdt_mon_features & (1 << QOS_L3_MBM_TOTAL_EVENT_ID));
-}
-
-static inline bool is_mbm_local_enabled(void)
-{
-	return (rdt_mon_features & (1 << QOS_L3_MBM_LOCAL_EVENT_ID));
-}
-
-static inline bool is_mbm_enabled(void)
-{
-	return (is_mbm_total_enabled() || is_mbm_local_enabled());
-}
-
-static inline bool is_mbm_event(int e)
-{
-	return (e >= QOS_L3_MBM_TOTAL_EVENT_ID &&
-		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
-}
-
 /**
  * struct rdt_hw_resource - arch private attributes of a resctrl resource
  * @r_resctrl:		Attributes of the resource used directly by resctrl.
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 2a1cbd4de6ee..c9be2d0819c0 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -251,11 +251,11 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_domain *d)
 {
 	struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d);
 
-	if (is_mbm_total_enabled())
+	if (resctrl_arch_is_mbm_total_enabled())
 		memset(hw_dom->arch_mbm_total, 0,
 		       sizeof(*hw_dom->arch_mbm_total) * r->num_rmid);
 
-	if (is_mbm_local_enabled())
+	if (resctrl_arch_is_mbm_local_enabled())
 		memset(hw_dom->arch_mbm_local, 0,
 		       sizeof(*hw_dom->arch_mbm_local) * r->num_rmid);
 }
@@ -514,7 +514,7 @@ void free_rmid(u32 closid, u32 rmid)
 
 	entry = __rmid_entry(idx);
 
-	if (is_llc_occupancy_enabled())
+	if (resctrl_arch_is_llc_occupancy_enabled())
 		add_rmid_to_limbo(entry);
 	else
 		list_add_tail(&entry->list, &rmid_free_lru);
@@ -666,7 +666,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
 	struct list_head *head;
 	struct rdtgroup *entry;
 
-	if (!is_mbm_local_enabled())
+	if (!resctrl_arch_is_mbm_local_enabled())
 		return;
 
 	r_mba = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
@@ -735,7 +735,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,
 	 * This is protected from concurrent reads from user
 	 * as both the user and we hold the global mutex.
 	 */
-	if (is_mbm_total_enabled()) {
+	if (resctrl_arch_is_mbm_total_enabled()) {
 		rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID;
 		rr.val = 0;
 		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
@@ -749,7 +749,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,
 
 		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
 	}
-	if (is_mbm_local_enabled()) {
+	if (resctrl_arch_is_mbm_local_enabled()) {
 		rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID;
 		rr.val = 0;
 		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
@@ -997,11 +997,11 @@ static void l3_mon_evt_init(struct rdt_resource *r)
 {
 	INIT_LIST_HEAD(&r->evt_list);
 
-	if (is_llc_occupancy_enabled())
+	if (resctrl_arch_is_llc_occupancy_enabled())
 		list_add_tail(&llc_occupancy_event.list, &r->evt_list);
-	if (is_mbm_total_enabled())
+	if (resctrl_arch_is_mbm_total_enabled())
 		list_add_tail(&mbm_total_event.list, &r->evt_list);
-	if (is_mbm_local_enabled())
+	if (resctrl_arch_is_mbm_local_enabled())
 		list_add_tail(&mbm_local_event.list, &r->evt_list);
 }
 
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 3f16e7854411..8285b916289c 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -114,6 +114,18 @@ void rdt_staged_configs_clear(void)
 	}
 }
 
+static bool resctrl_is_mbm_enabled(void)
+{
+	return (resctrl_arch_is_mbm_total_enabled() ||
+		resctrl_arch_is_mbm_local_enabled());
+}
+
+static bool resctrl_is_mbm_event(int e)
+{
+	return (e >= QOS_L3_MBM_TOTAL_EVENT_ID &&
+		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
+}
+
 /*
  * Trivial allocator for CLOSIDs. Since h/w only supports a small number,
  * we can keep a bitmap of free CLOSIDs in a single integer.
@@ -161,7 +173,7 @@ static int closid_alloc(void)
 	lockdep_assert_held(&rdtgroup_mutex);
 
 	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
-	    is_llc_occupancy_enabled()) {
+	    resctrl_arch_is_llc_occupancy_enabled()) {
 		cleanest_closid = resctrl_find_cleanest_closid();
 		if (cleanest_closid < 0)
 			return cleanest_closid;
@@ -2370,7 +2382,7 @@ static bool supports_mba_mbps(void)
 {
 	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
 
-	return (is_mbm_local_enabled() &&
+	return (resctrl_arch_is_mbm_local_enabled() &&
 		r->alloc_capable && is_mba_linear());
 }
 
@@ -2738,7 +2750,7 @@ static int rdt_get_tree(struct fs_context *fc)
 	if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable())
 		resctrl_mounted = true;
 
-	if (is_mbm_enabled()) {
+	if (resctrl_is_mbm_enabled()) {
 		list_for_each_entry(dom, &l3->domains, list)
 			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
 						   RESCTRL_PICK_ANY_CPU);
@@ -3107,7 +3119,7 @@ static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,
 		if (ret)
 			goto out_destroy;
 
-		if (is_mbm_event(mevt->evtid))
+		if (resctrl_is_mbm_event(mevt->evtid))
 			mon_event_read(&rr, r, d, prgrp, mevt->evtid, true);
 	}
 	kernfs_activate(kn);
@@ -4006,9 +4018,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)
 	if (resctrl_mounted && resctrl_arch_mon_capable())
 		rmdir_mondata_subdir_allrdtgrp(r, d->id);
 
-	if (is_mbm_enabled())
+	if (resctrl_is_mbm_enabled())
 		cancel_delayed_work(&d->mbm_over);
-	if (is_llc_occupancy_enabled() && has_busy_rmid(d)) {
+	if (resctrl_arch_is_llc_occupancy_enabled() && has_busy_rmid(d)) {
 		/*
 		 * When a package is going down, forcefully
 		 * decrement rmid->ebusy. There is no way to know
@@ -4032,12 +4044,12 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
 	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
 	size_t tsize;
 
-	if (is_llc_occupancy_enabled()) {
+	if (resctrl_arch_is_llc_occupancy_enabled()) {
 		d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL);
 		if (!d->rmid_busy_llc)
 			return -ENOMEM;
 	}
-	if (is_mbm_total_enabled()) {
+	if (resctrl_arch_is_mbm_total_enabled()) {
 		tsize = sizeof(*d->mbm_total);
 		d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL);
 		if (!d->mbm_total) {
@@ -4045,7 +4057,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
 			return -ENOMEM;
 		}
 	}
-	if (is_mbm_local_enabled()) {
+	if (resctrl_arch_is_mbm_local_enabled()) {
 		tsize = sizeof(*d->mbm_local);
 		d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL);
 		if (!d->mbm_local) {
@@ -4077,13 +4089,13 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)
 	if (err)
 		goto out_unlock;
 
-	if (is_mbm_enabled()) {
+	if (resctrl_is_mbm_enabled()) {
 		INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow);
 		mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL,
 					   RESCTRL_PICK_ANY_CPU);
 	}
 
-	if (is_llc_occupancy_enabled())
+	if (resctrl_arch_is_llc_occupancy_enabled())
 		INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo);
 
 	/*
@@ -4138,12 +4150,12 @@ void resctrl_offline_cpu(unsigned int cpu)
 
 	d = get_domain_from_cpu(cpu, l3);
 	if (d) {
-		if (is_mbm_enabled() && cpu == d->mbm_work_cpu) {
+		if (resctrl_is_mbm_enabled() && cpu == d->mbm_work_cpu) {
 			cancel_delayed_work(&d->mbm_over);
 			mbm_setup_overflow_handler(d, 0, cpu);
 		}
-		if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu &&
-		    has_busy_rmid(d)) {
+		if (resctrl_arch_is_llc_occupancy_enabled() &&
+		    cpu == d->cqm_work_cpu && has_busy_rmid(d)) {
 			cancel_delayed_work(&d->cqm_limbo);
 			cqm_setup_limbo_handler(d, 0, cpu);
 		}
-- 
2.39.2

Re: [PATCH v1 14/31] x86/resctrl: Export the is_mbm_*_enabled() helpers to asm/resctrl.h

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> The architecture specific parts of resctrl have helpers to hide accesses
> to the rdt_mon_features bitmap.
> 
> Once the filesystem parts of resctrl are moved, these can no longer live
> in internal.h. Once these are exposed to the wider kernel, they should
> have a 'resctrl_arch_' prefix, to fit the rest of the arch<->fs interface.
> 
> Move and rename the helpers that touch rdt_mon_features directly.
> is_mbm_event() and is_mbm_enabled() are only called from rdtgroup.c,
> so can be moved into that file.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/include/asm/resctrl.h         | 17 +++++++++++
>  arch/x86/kernel/cpu/resctrl/core.c     |  4 +--
>  arch/x86/kernel/cpu/resctrl/internal.h | 27 -----------------
>  arch/x86/kernel/cpu/resctrl/monitor.c  | 18 ++++++------
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 40 +++++++++++++++++---------
>  5 files changed, 54 insertions(+), 52 deletions(-)
> 
> diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
> index 5f6a5375bb4a..50407e83d0ca 100644
> --- a/arch/x86/include/asm/resctrl.h
> +++ b/arch/x86/include/asm/resctrl.h
> @@ -7,6 +7,7 @@
>  #include <linux/jump_label.h>
>  #include <linux/percpu.h>
>  #include <linux/sched.h>
> +#include <linux/resctrl_types.h>
>  

Can this be alphabetical?

Reinette

Re: [PATCH v1 14/31] x86/resctrl: Export the is_mbm_*_enabled() helpers to asm/resctrl.h

[Dave Martin]

On Mon, Apr 08, 2024 at 08:20:12PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > The architecture specific parts of resctrl have helpers to hide accesses
> > to the rdt_mon_features bitmap.
> > 
> > Once the filesystem parts of resctrl are moved, these can no longer live
> > in internal.h. Once these are exposed to the wider kernel, they should
> > have a 'resctrl_arch_' prefix, to fit the rest of the arch<->fs interface.
> > 
> > Move and rename the helpers that touch rdt_mon_features directly.
> > is_mbm_event() and is_mbm_enabled() are only called from rdtgroup.c,
> > so can be moved into that file.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/include/asm/resctrl.h         | 17 +++++++++++
> >  arch/x86/kernel/cpu/resctrl/core.c     |  4 +--
> >  arch/x86/kernel/cpu/resctrl/internal.h | 27 -----------------
> >  arch/x86/kernel/cpu/resctrl/monitor.c  | 18 ++++++------
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 40 +++++++++++++++++---------
> >  5 files changed, 54 insertions(+), 52 deletions(-)
> > 
> > diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
> > index 5f6a5375bb4a..50407e83d0ca 100644
> > --- a/arch/x86/include/asm/resctrl.h
> > +++ b/arch/x86/include/asm/resctrl.h
> > @@ -7,6 +7,7 @@
> >  #include <linux/jump_label.h>
> >  #include <linux/percpu.h>
> >  #include <linux/sched.h>
> > +#include <linux/resctrl_types.h>
> >  
> 
> Can this be alphabetical?

I'd guess so, looking at it.

Noted.

Cheers
---Dave

[PATCH v1 15/31] x86/resctrl: Add resctrl_arch_is_evt_configurable() to abstract BMEC

[James Morse]

When BMEC is supported the resctrl event can be configured in a number
of ways. This depends on architecture support. rdt_get_mon_l3_config()
modifies the struct mon_evt and calls mbm_config_rftype_init() to create
the files that allow the configuration.

Splitting this into separate architecture and filesystem parts would
require the struct mon_evt and mbm_config_rftype_init() to be exposed.

Instead, add resctrl_arch_is_evt_configurable(), and use this from
resctrl_mon_resource_init() to initialise struct mon_evt and call
mbm_config_rftype_init().
resctrl_arch_is_evt_configurable() calls rdt_cpu_has() so it doesn't
obviously benefit from being inlined. Putting it in core.c will allow
rdt_cpu_has() to eventually become static.

resctrl_arch_is_evt_configurable() uses rdt_cpu_has() from
resctrl_mon_resource_init(), which isn't marked __init. In addition,
MPAM needs to initialise resctrl late. Drop the __init on the relevant
functions.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c     | 19 +++++++++++++++++--
 arch/x86/kernel/cpu/resctrl/internal.h |  4 ++--
 arch/x86/kernel/cpu/resctrl/monitor.c  | 18 +++++++++---------
 arch/x86/kernel/cpu/resctrl/rdtgroup.c |  2 +-
 include/linux/resctrl.h                |  2 ++
 5 files changed, 31 insertions(+), 14 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index a4b5ade6e291..824a94593e9d 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -667,7 +667,7 @@ struct rdt_options {
 	bool	force_off, force_on;
 };
 
-static struct rdt_options rdt_options[]  __initdata = {
+static struct rdt_options rdt_options[]  __ro_after_init = {
 	RDT_OPT(RDT_FLAG_CMT,	    "cmt",	X86_FEATURE_CQM_OCCUP_LLC),
 	RDT_OPT(RDT_FLAG_MBM_TOTAL, "mbmtotal", X86_FEATURE_CQM_MBM_TOTAL),
 	RDT_OPT(RDT_FLAG_MBM_LOCAL, "mbmlocal", X86_FEATURE_CQM_MBM_LOCAL),
@@ -707,7 +707,7 @@ static int __init set_rdt_options(char *str)
 }
 __setup("rdt", set_rdt_options);
 
-bool __init rdt_cpu_has(int flag)
+bool rdt_cpu_has(int flag)
 {
 	bool ret = boot_cpu_has(flag);
 	struct rdt_options *o;
@@ -727,6 +727,21 @@ bool __init rdt_cpu_has(int flag)
 	return ret;
 }
 
+bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt)
+{
+	if (!rdt_cpu_has(X86_FEATURE_BMEC))
+		return false;
+
+	switch (evt) {
+	case QOS_L3_MBM_TOTAL_EVENT_ID:
+		return rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL);
+	case QOS_L3_MBM_LOCAL_EVENT_ID:
+		return rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL);
+	default:
+		return false;
+	}
+}
+
 static __init bool get_mem_config(void)
 {
 	struct rdt_hw_resource *hw_res = &rdt_resources_all[RDT_RESOURCE_MBA];
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index edbccc79246f..46370eafb00f 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -507,7 +507,7 @@ int alloc_rmid(u32 closid);
 void free_rmid(u32 closid, u32 rmid);
 int rdt_get_mon_l3_config(struct rdt_resource *r);
 void __exit resctrl_mon_resource_exit(void);
-bool __init rdt_cpu_has(int flag);
+bool rdt_cpu_has(int flag);
 void mon_event_count(void *info);
 int rdtgroup_mondata_show(struct seq_file *m, void *arg);
 void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
@@ -527,7 +527,7 @@ bool has_busy_rmid(struct rdt_domain *d);
 void __check_limbo(struct rdt_domain *d, bool force_free);
 void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
 void __init thread_throttle_mode_init(void);
-void __init mbm_config_rftype_init(const char *config);
+void mbm_config_rftype_init(const char *config);
 void rdt_staged_configs_clear(void);
 bool closid_allocated(unsigned int closid);
 int resctrl_find_cleanest_closid(void);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index c9be2d0819c0..ccb85c61b43b 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -1019,6 +1019,15 @@ int resctrl_mon_resource_init(void)
 
 	l3_mon_evt_init(r);
 
+	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_TOTAL_EVENT_ID)) {
+		mbm_total_event.configurable = true;
+		mbm_config_rftype_init("mbm_total_bytes_config");
+	}
+	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_LOCAL_EVENT_ID)) {
+		mbm_local_event.configurable = true;
+		mbm_config_rftype_init("mbm_local_bytes_config");
+	}
+
 	return 0;
 }
 
@@ -1060,15 +1069,6 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 		/* Detect list of bandwidth sources that can be tracked */
 		cpuid_count(0x80000020, 3, &eax, &ebx, &ecx, &edx);
 		hw_res->mbm_cfg_mask = ecx & MAX_EVT_CONFIG_BITS;
-
-		if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {
-			mbm_total_event.configurable = true;
-			mbm_config_rftype_init("mbm_total_bytes_config");
-		}
-		if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL)) {
-			mbm_local_event.configurable = true;
-			mbm_config_rftype_init("mbm_local_bytes_config");
-		}
 	}
 
 	r->mon_capable = true;
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 8285b916289c..2d6f4e0d3656 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -2068,7 +2068,7 @@ void __init thread_throttle_mode_init(void)
 	rft->fflags = RFTYPE_CTRL_INFO | RFTYPE_RES_MB;
 }
 
-void __init mbm_config_rftype_init(const char *config)
+void mbm_config_rftype_init(const char *config)
 {
 	struct rftype *rft;
 
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index b0ee7256e095..bfc63e8219e5 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -204,6 +204,8 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *r);
 struct rdt_domain *resctrl_arch_find_domain(struct rdt_resource *r, int id);
 int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
 
+bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
+
 /*
  * Update the ctrl_val and apply this config right now.
  * Must be called on one of the domain's CPUs.
-- 
2.39.2

[PATCH v1 16/31] x86/resctrl: Change mon_event_config_{read,write}() to be arch helpers

[James Morse]

mon_event_config_{read,write}() are called via IPI and access model
specific registers to do their work.

To support another architecture, this needs abstracting.

Rename mon_event_config_{read,write}() to have a resctrl_arch_ prefix,
and move their struct mon_config_info parameter into the restrl_types
header. This allows another architecture to supply an implementation
of these.

As struct mon_config_info is now exposed globally, give it a 'resctrl_'
prefix. MPAM systems need access to the domain to do this work, add
the resource and domain to struct resctrl_mon_config_info.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 34 +++++++++++++-------------
 include/linux/resctrl.h                |  9 +++++++
 2 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 2d6f4e0d3656..e76018687117 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -1580,11 +1580,6 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
 	return ret;
 }
 
-struct mon_config_info {
-	u32 evtid;
-	u32 mon_config;
-};
-
 #define INVALID_CONFIG_INDEX   UINT_MAX
 
 /**
@@ -1609,9 +1604,9 @@ static inline unsigned int mon_event_config_index_get(u32 evtid)
 	}
 }
 
-static void mon_event_config_read(void *info)
+void resctrl_arch_mon_event_config_read(void *info)
 {
-	struct mon_config_info *mon_info = info;
+	struct resctrl_mon_config_info *mon_info = info;
 	unsigned int index;
 	u64 msrval;
 
@@ -1626,14 +1621,15 @@ static void mon_event_config_read(void *info)
 	mon_info->mon_config = msrval & MAX_EVT_CONFIG_BITS;
 }
 
-static void mondata_config_read(struct rdt_domain *d, struct mon_config_info *mon_info)
+static void mondata_config_read(struct resctrl_mon_config_info *mon_info)
 {
-	smp_call_function_any(&d->cpu_mask, mon_event_config_read, mon_info, 1);
+	smp_call_function_any(&mon_info->d->cpu_mask,
+			      resctrl_arch_mon_event_config_read, mon_info, 1);
 }
 
 static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid)
 {
-	struct mon_config_info mon_info = {0};
+	struct resctrl_mon_config_info mon_info = {0};
 	struct rdt_domain *dom;
 	bool sep = false;
 
@@ -1644,9 +1640,11 @@ static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid
 		if (sep)
 			seq_puts(s, ";");
 
-		memset(&mon_info, 0, sizeof(struct mon_config_info));
+		memset(&mon_info, 0, sizeof(struct resctrl_mon_config_info));
+		mon_info.r = r;
+		mon_info.d = dom;
 		mon_info.evtid = evtid;
-		mondata_config_read(dom, &mon_info);
+		mondata_config_read(&mon_info);
 
 		seq_printf(s, "%d=0x%02x", dom->id, mon_info.mon_config);
 		sep = true;
@@ -1679,9 +1677,9 @@ static int mbm_local_bytes_config_show(struct kernfs_open_file *of,
 	return 0;
 }
 
-static void mon_event_config_write(void *info)
+void resctrl_arch_mon_event_config_write(void *info)
 {
-	struct mon_config_info *mon_info = info;
+	struct resctrl_mon_config_info *mon_info = info;
 	unsigned int index;
 
 	index = mon_event_config_index_get(mon_info->evtid);
@@ -1695,14 +1693,16 @@ static void mon_event_config_write(void *info)
 static void mbm_config_write_domain(struct rdt_resource *r,
 				    struct rdt_domain *d, u32 evtid, u32 val)
 {
-	struct mon_config_info mon_info = {0};
+	struct resctrl_mon_config_info mon_info = {0};
 
 	/*
 	 * Read the current config value first. If both are the same then
 	 * no need to write it again.
 	 */
+	mon_info.r = r;
+	mon_info.d = d;
 	mon_info.evtid = evtid;
-	mondata_config_read(d, &mon_info);
+	mondata_config_read(&mon_info);
 	if (mon_info.mon_config == val)
 		return;
 
@@ -1714,7 +1714,7 @@ static void mbm_config_write_domain(struct rdt_resource *r,
 	 * are scoped at the domain level. Writing any of these MSRs
 	 * on one CPU is observed by all the CPUs in the domain.
 	 */
-	smp_call_function_any(&d->cpu_mask, mon_event_config_write,
+	smp_call_function_any(&d->cpu_mask, resctrl_arch_mon_event_config_write,
 			      &mon_info, 1);
 
 	/*
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index bfc63e8219e5..975b80102fbe 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -192,6 +192,13 @@ struct resctrl_cpu_sync {
 	u32 rmid;
 };
 
+struct resctrl_mon_config_info {
+	struct rdt_resource *r;
+	struct rdt_domain   *d;
+	u32                  evtid;
+	u32                  mon_config;
+};
+
 /*
  * Update and re-load this CPUs defaults. Called via IPI, takes a pointer to
  * struct resctrl_cpu_sync, or NULL.
@@ -205,6 +212,8 @@ struct rdt_domain *resctrl_arch_find_domain(struct rdt_resource *r, int id);
 int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
 
 bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
+void resctrl_arch_mon_event_config_write(void *info);
+void resctrl_arch_mon_event_config_read(void *info);
 
 /*
  * Update the ctrl_val and apply this config right now.
-- 
2.39.2

Re: [PATCH v1 16/31] x86/resctrl: Change mon_event_config_{read,write}() to be arch helpers

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> mon_event_config_{read,write}() are called via IPI and access model
> specific registers to do their work.
> 
> To support another architecture, this needs abstracting.
> 
> Rename mon_event_config_{read,write}() to have a resctrl_arch_ prefix,
> and move their struct mon_config_info parameter into the restrl_types

Looks like this change is actually moving the struct into include/linux/resctrl.h,
not resctrl_types.h.

> header. This allows another architecture to supply an implementation
> of these.
> 
> As struct mon_config_info is now exposed globally, give it a 'resctrl_'
> prefix. MPAM systems need access to the domain to do this work, add
> the resource and domain to struct resctrl_mon_config_info.
> 
> Signed-off-by: James Morse <james.morse@arm.com>

...

> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index bfc63e8219e5..975b80102fbe 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -192,6 +192,13 @@ struct resctrl_cpu_sync {
>  	u32 rmid;
>  };
>  
> +struct resctrl_mon_config_info {
> +	struct rdt_resource *r;
> +	struct rdt_domain   *d;
> +	u32                  evtid;
> +	u32                  mon_config;
> +};
> +

Please use tabs consistently in this file.

Reinette

Re: [PATCH v1 16/31] x86/resctrl: Change mon_event_config_{read,write}() to be arch helpers

[Dave Martin]

On Mon, Apr 08, 2024 at 08:20:41PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > mon_event_config_{read,write}() are called via IPI and access model
> > specific registers to do their work.
> > 
> > To support another architecture, this needs abstracting.
> > 
> > Rename mon_event_config_{read,write}() to have a resctrl_arch_ prefix,
> > and move their struct mon_config_info parameter into the restrl_types
> 
> Looks like this change is actually moving the struct into include/linux/resctrl.h,
> not resctrl_types.h.

Noted.

> 
> > header. This allows another architecture to supply an implementation
> > of these.
> > 
> > As struct mon_config_info is now exposed globally, give it a 'resctrl_'
> > prefix. MPAM systems need access to the domain to do this work, add
> > the resource and domain to struct resctrl_mon_config_info.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> 
> ..
> 
> > diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> > index bfc63e8219e5..975b80102fbe 100644
> > --- a/include/linux/resctrl.h
> > +++ b/include/linux/resctrl.h
> > @@ -192,6 +192,13 @@ struct resctrl_cpu_sync {
> >  	u32 rmid;
> >  };
> >  
> > +struct resctrl_mon_config_info {
> > +	struct rdt_resource *r;
> > +	struct rdt_domain   *d;
> > +	u32                  evtid;
> > +	u32                  mon_config;
> > +};
> > +
> 
> Please use tabs consistently in this file.
> 
> Reinette
> 

Noted; I have presumed that you mean tabs-only indentation before the
member declarator, and to line up the declarators (including the *),
e.g.:

struct resctrl_mon_config_info {
	struct rdt_resource	*r;
	struct rdt_domain	*d;
	u32			evtid;
	u32			mon_config;
};

(But if that's not what you meant, please shout!)

Cheers
---Dave

[PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[James Morse]

The mbm_cfg_mask field lists the bits that user-space can set when
configuring an event. This value is output via the last_cmd_status
file.

Once the filesystem parts of resctrl are moved to live in /fs/, the
struct rdt_hw_resource is inaccessible to the filesystem code. Because
this value is output to user-space, it has to be accessible to the
filesystem code.

Move it to struct rdt_resource.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/internal.h | 3 ---
 arch/x86/kernel/cpu/resctrl/monitor.c  | 2 +-
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 ++---
 include/linux/resctrl.h                | 3 +++
 4 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 46370eafb00f..238b81d3f64a 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -371,8 +371,6 @@ struct msr_param {
  * @msr_update:		Function pointer to update QOS MSRs
  * @mon_scale:		cqm counter * mon_scale = occupancy in bytes
  * @mbm_width:		Monitor width, to detect and correct for overflow.
- * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
- *			Monitoring Event Configuration (BMEC) is supported.
  * @cdp_enabled:	CDP state of this resource
  *
  * Members of this structure are either private to the architecture
@@ -387,7 +385,6 @@ struct rdt_hw_resource {
 				 struct rdt_resource *r);
 	unsigned int		mon_scale;
 	unsigned int		mbm_width;
-	unsigned int		mbm_cfg_mask;
 	bool			cdp_enabled;
 };
 
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index ccb85c61b43b..287fb0a5f060 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -1068,7 +1068,7 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 
 		/* Detect list of bandwidth sources that can be tracked */
 		cpuid_count(0x80000020, 3, &eax, &ebx, &ecx, &edx);
-		hw_res->mbm_cfg_mask = ecx & MAX_EVT_CONFIG_BITS;
+		r->mbm_cfg_mask = ecx & MAX_EVT_CONFIG_BITS;
 	}
 
 	r->mon_capable = true;
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index e76018687117..3d3a839eba6b 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -1731,7 +1731,6 @@ static void mbm_config_write_domain(struct rdt_resource *r,
 
 static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
 {
-	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	char *dom_str = NULL, *id_str;
 	unsigned long dom_id, val;
 	struct rdt_domain *d;
@@ -1758,9 +1757,9 @@ static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
 	}
 
 	/* Value from user cannot be more than the supported set of events */
-	if ((val & hw_res->mbm_cfg_mask) != val) {
+	if ((val & r->mbm_cfg_mask) != val) {
 		rdt_last_cmd_printf("Invalid event configuration: max valid mask is 0x%02x\n",
-				    hw_res->mbm_cfg_mask);
+				    r->mbm_cfg_mask);
 		return -EINVAL;
 	}
 
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 975b80102fbe..8a7367d1ce45 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -140,6 +140,8 @@ struct resctrl_membw {
  * @format_str:		Per resource format string to show domain value
  * @evt_list:		List of monitoring events
  * @fflags:		flags to choose base and info files
+ * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
+ *			Monitoring Event Configuration (BMEC) is supported.
  * @cdp_capable:	Is the CDP feature available on this resource
  */
 struct rdt_resource {
@@ -157,6 +159,7 @@ struct rdt_resource {
 	const char		*format_str;
 	struct list_head	evt_list;
 	unsigned long		fflags;
+	unsigned int		mbm_cfg_mask;
 	bool			cdp_capable;
 };
 
-- 
2.39.2

Re: [PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> The mbm_cfg_mask field lists the bits that user-space can set when
> configuring an event. This value is output via the last_cmd_status
> file.
> 
> Once the filesystem parts of resctrl are moved to live in /fs/, the
> struct rdt_hw_resource is inaccessible to the filesystem code. Because
> this value is output to user-space, it has to be accessible to the
> filesystem code.
> 
> Move it to struct rdt_resource.
> 
> Signed-off-by: James Morse <james.morse@arm.com>

...

> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index 975b80102fbe..8a7367d1ce45 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -140,6 +140,8 @@ struct resctrl_membw {
>   * @format_str:		Per resource format string to show domain value
>   * @evt_list:		List of monitoring events
>   * @fflags:		flags to choose base and info files
> + * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
> + *			Monitoring Event Configuration (BMEC) is supported.

BMEC is an AMD term. If MPAM is planning to use this member then this comment
should be made generic.

>   * @cdp_capable:	Is the CDP feature available on this resource
>   */
>  struct rdt_resource {
> @@ -157,6 +159,7 @@ struct rdt_resource {
>  	const char		*format_str;
>  	struct list_head	evt_list;
>  	unsigned long		fflags;
> +	unsigned int		mbm_cfg_mask;
>  	bool			cdp_capable;
>  };
>  

Reinette

Re: [PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[Dave Martin]

On Mon, Apr 08, 2024 at 08:21:24PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > The mbm_cfg_mask field lists the bits that user-space can set when
> > configuring an event. This value is output via the last_cmd_status
> > file.
> > 
> > Once the filesystem parts of resctrl are moved to live in /fs/, the
> > struct rdt_hw_resource is inaccessible to the filesystem code. Because
> > this value is output to user-space, it has to be accessible to the
> > filesystem code.
> > 
> > Move it to struct rdt_resource.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> 
> ..
> 
> > diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> > index 975b80102fbe..8a7367d1ce45 100644
> > --- a/include/linux/resctrl.h
> > +++ b/include/linux/resctrl.h
> > @@ -140,6 +140,8 @@ struct resctrl_membw {
> >   * @format_str:		Per resource format string to show domain value
> >   * @evt_list:		List of monitoring events
> >   * @fflags:		flags to choose base and info files
> > + * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
> > + *			Monitoring Event Configuration (BMEC) is supported.

[...]

> Reinette
> 
> BMEC is an AMD term. If MPAM is planning to use this member then this comment
> should be made generic.

MPAM can (at least) filter reads and/or writes, and it looks like James
is expecting to support this.

However, it probably does make sense to keep the comments neutral in a
common header.

Would the following work?

	* @ mbm_cfg_mask:	Arch-specific event configuration flags



I think that's broad enough to cover all bases, but I'll wait for your
response.

Cheers
---Dave

Re: [PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:16 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:21:24PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> The mbm_cfg_mask field lists the bits that user-space can set when
>>> configuring an event. This value is output via the last_cmd_status
>>> file.
>>>
>>> Once the filesystem parts of resctrl are moved to live in /fs/, the
>>> struct rdt_hw_resource is inaccessible to the filesystem code. Because
>>> this value is output to user-space, it has to be accessible to the
>>> filesystem code.
>>>
>>> Move it to struct rdt_resource.
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>
>> ..
>>
>>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
>>> index 975b80102fbe..8a7367d1ce45 100644
>>> --- a/include/linux/resctrl.h
>>> +++ b/include/linux/resctrl.h
>>> @@ -140,6 +140,8 @@ struct resctrl_membw {
>>>   * @format_str:		Per resource format string to show domain value
>>>   * @evt_list:		List of monitoring events
>>>   * @fflags:		flags to choose base and info files
>>> + * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
>>> + *			Monitoring Event Configuration (BMEC) is supported.
> 
> [...]
> 
>> Reinette
>>
>> BMEC is an AMD term. If MPAM is planning to use this member then this comment
>> should be made generic.
> 
> MPAM can (at least) filter reads and/or writes, and it looks like James
> is expecting to support this.
> 
> However, it probably does make sense to keep the comments neutral in a
> common header.
> 
> Would the following work?
> 
> 	* @ mbm_cfg_mask:	Arch-specific event configuration flags
> 
> 
> 
> I think that's broad enough to cover all bases, but I'll wait for your
> response.

Since this is exposed to user space, ideally all architectures would use
the same bits to mean the same thing. I assumed that is what James intended
by placing the existing (AMD BMEC) bits in the global resctrl_types.h.

Reinette

Re: [PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[Dave Martin]

On Thu, Apr 11, 2024 at 10:39:06AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:16 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:21:24PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> >>> The mbm_cfg_mask field lists the bits that user-space can set when
> >>> configuring an event. This value is output via the last_cmd_status
> >>> file.
> >>>
> >>> Once the filesystem parts of resctrl are moved to live in /fs/, the
> >>> struct rdt_hw_resource is inaccessible to the filesystem code. Because
> >>> this value is output to user-space, it has to be accessible to the
> >>> filesystem code.
> >>>
> >>> Move it to struct rdt_resource.
> >>>
> >>> Signed-off-by: James Morse <james.morse@arm.com>
> >>
> >> ..
> >>
> >>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> >>> index 975b80102fbe..8a7367d1ce45 100644
> >>> --- a/include/linux/resctrl.h
> >>> +++ b/include/linux/resctrl.h
> >>> @@ -140,6 +140,8 @@ struct resctrl_membw {
> >>>   * @format_str:		Per resource format string to show domain value
> >>>   * @evt_list:		List of monitoring events
> >>>   * @fflags:		flags to choose base and info files
> >>> + * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
> >>> + *			Monitoring Event Configuration (BMEC) is supported.
> > 
> > [...]
> > 
> >> Reinette
> >>
> >> BMEC is an AMD term. If MPAM is planning to use this member then this comment
> >> should be made generic.
> > 
> > MPAM can (at least) filter reads and/or writes, and it looks like James
> > is expecting to support this.
> > 
> > However, it probably does make sense to keep the comments neutral in a
> > common header.
> > 
> > Would the following work?
> > 
> > 	* @ mbm_cfg_mask:	Arch-specific event configuration flags
> > 
> > 
> > 
> > I think that's broad enough to cover all bases, but I'll wait for your
> > response.
> 
> Since this is exposed to user space, ideally all architectures would use
> the same bits to mean the same thing. I assumed that is what James intended
> by placing the existing (AMD BMEC) bits in the global resctrl_types.h.
> 
> Reinette

Maybe, but the bits as defined by AMD BMEC look rather architecture and
bus specific, and I am suspicious that there is no guaranteed clean
mapping between MPAM's config and BMEC's config.

MPAM currently just has "reads" and "writes" (or both), though as for
BMEC, the meanings of these are not fully defined.  I suppose finer
filtering granularity might be supported in future (at least, it is not
explicitly ruled out).

James' current approach seems to be to pick a single BMEC flag that's
in the right sort of area for each MPAM bit (though not equivalent) and
translate that bit across to drive a corresponding the MPAM bit.  But
I'd say that this is arch-specific configuration masquerading as
generic configuration IMHO and not really generic at all.

See "untested: arm_mpam: resctrl: Allow monitors to be configured"
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/commit/?h=mpam/snapshot/v6.7-rc2&id=db0ac51f60675b6c4a54ccd24fa7198ec321c56d

I guess this needs discussion with James, since there will have been
additional thought process behind all this that is not captured; either
way, I guess it could be resolved after this series, but it will need a
decision before the MPAM support is merged (or at least, before MPAM
exposes userspace support for event configuration upstream).

(If this has already been discussed and James' current approach has
already been agreed as the least worst option, then I guess I can live
with it; I just find it icky, and it looks odd to have AMD specifics in
a common header.)

Cheers
---Dave

Re: [PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[Reinette Chatre]

Hi Dave,

On 4/17/2024 7:41 AM, Dave Martin wrote:
> On Thu, Apr 11, 2024 at 10:39:06AM -0700, Reinette Chatre wrote:
>> Hi Dave,
>>
>> On 4/11/2024 7:16 AM, Dave Martin wrote:
>>> On Mon, Apr 08, 2024 at 08:21:24PM -0700, Reinette Chatre wrote:
>>>> Hi James,
>>>>
>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>>> The mbm_cfg_mask field lists the bits that user-space can set when
>>>>> configuring an event. This value is output via the last_cmd_status
>>>>> file.
>>>>>
>>>>> Once the filesystem parts of resctrl are moved to live in /fs/, the
>>>>> struct rdt_hw_resource is inaccessible to the filesystem code. Because
>>>>> this value is output to user-space, it has to be accessible to the
>>>>> filesystem code.
>>>>>
>>>>> Move it to struct rdt_resource.
>>>>>
>>>>> Signed-off-by: James Morse <james.morse@arm.com>
>>>>
>>>> ..
>>>>
>>>>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
>>>>> index 975b80102fbe..8a7367d1ce45 100644
>>>>> --- a/include/linux/resctrl.h
>>>>> +++ b/include/linux/resctrl.h
>>>>> @@ -140,6 +140,8 @@ struct resctrl_membw {
>>>>>   * @format_str:		Per resource format string to show domain value
>>>>>   * @evt_list:		List of monitoring events
>>>>>   * @fflags:		flags to choose base and info files
>>>>> + * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
>>>>> + *			Monitoring Event Configuration (BMEC) is supported.
>>>
>>> [...]
>>>
>>>> Reinette
>>>>
>>>> BMEC is an AMD term. If MPAM is planning to use this member then this comment
>>>> should be made generic.
>>>
>>> MPAM can (at least) filter reads and/or writes, and it looks like James
>>> is expecting to support this.
>>>
>>> However, it probably does make sense to keep the comments neutral in a
>>> common header.
>>>
>>> Would the following work?
>>>
>>> 	* @ mbm_cfg_mask:	Arch-specific event configuration flags
>>>
>>>
>>>
>>> I think that's broad enough to cover all bases, but I'll wait for your
>>> response.
>>
>> Since this is exposed to user space, ideally all architectures would use
>> the same bits to mean the same thing. I assumed that is what James intended
>> by placing the existing (AMD BMEC) bits in the global resctrl_types.h.
>>
>> Reinette
> 
> Maybe, but the bits as defined by AMD BMEC look rather architecture and
> bus specific, and I am suspicious that there is no guaranteed clean
> mapping between MPAM's config and BMEC's config.
> 
> MPAM currently just has "reads" and "writes" (or both), though as for
> BMEC, the meanings of these are not fully defined.  I suppose finer
> filtering granularity might be supported in future (at least, it is not
> explicitly ruled out).
> 
> James' current approach seems to be to pick a single BMEC flag that's
> in the right sort of area for each MPAM bit (though not equivalent) and
> translate that bit across to drive a corresponding the MPAM bit.  But
> I'd say that this is arch-specific configuration masquerading as
> generic configuration IMHO and not really generic at all.
> 
> See "untested: arm_mpam: resctrl: Allow monitors to be configured"
> https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/commit/?h=mpam/snapshot/v6.7-rc2&id=db0ac51f60675b6c4a54ccd24fa7198ec321c56d
> 
> I guess this needs discussion with James, since there will have been
> additional thought process behind all this that is not captured; either
> way, I guess it could be resolved after this series, but it will need a
> decision before the MPAM support is merged (or at least, before MPAM
> exposes userspace support for event configuration upstream).
> 
> (If this has already been discussed and James' current approach has
> already been agreed as the least worst option, then I guess I can live
> with it; I just find it icky, and it looks odd to have AMD specifics in
> a common header.)

I am not aware of such a discussion.

Sounds like a motivation to delay this portion of the changes in patch #8.

Reinette

Re: [PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[Dave Martin]

On Wed, Apr 17, 2024 at 10:18:48PM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/17/2024 7:41 AM, Dave Martin wrote:
> > On Thu, Apr 11, 2024 at 10:39:06AM -0700, Reinette Chatre wrote:
> >> Hi Dave,
> >>
> >> On 4/11/2024 7:16 AM, Dave Martin wrote:
> >>> On Mon, Apr 08, 2024 at 08:21:24PM -0700, Reinette Chatre wrote:
> >>>> Hi James,
> >>>>
> >>>> On 3/21/2024 9:50 AM, James Morse wrote:
> >>>>> The mbm_cfg_mask field lists the bits that user-space can set when
> >>>>> configuring an event. This value is output via the last_cmd_status
> >>>>> file.
> >>>>>
> >>>>> Once the filesystem parts of resctrl are moved to live in /fs/, the
> >>>>> struct rdt_hw_resource is inaccessible to the filesystem code. Because
> >>>>> this value is output to user-space, it has to be accessible to the
> >>>>> filesystem code.
> >>>>>
> >>>>> Move it to struct rdt_resource.
> >>>>>
> >>>>> Signed-off-by: James Morse <james.morse@arm.com>
> >>>>
> >>>> ..
> >>>>
> >>>>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> >>>>> index 975b80102fbe..8a7367d1ce45 100644
> >>>>> --- a/include/linux/resctrl.h
> >>>>> +++ b/include/linux/resctrl.h
> >>>>> @@ -140,6 +140,8 @@ struct resctrl_membw {
> >>>>>   * @format_str:		Per resource format string to show domain value
> >>>>>   * @evt_list:		List of monitoring events
> >>>>>   * @fflags:		flags to choose base and info files
> >>>>> + * @mbm_cfg_mask:	Bandwidth sources that can be tracked when Bandwidth
> >>>>> + *			Monitoring Event Configuration (BMEC) is supported.
> >>>
> >>> [...]
> >>>
> >>>> Reinette
> >>>>
> >>>> BMEC is an AMD term. If MPAM is planning to use this member then this comment
> >>>> should be made generic.
> >>>
> >>> MPAM can (at least) filter reads and/or writes, and it looks like James
> >>> is expecting to support this.
> >>>
> >>> However, it probably does make sense to keep the comments neutral in a
> >>> common header.
> >>>
> >>> Would the following work?
> >>>
> >>> 	* @ mbm_cfg_mask:	Arch-specific event configuration flags
> >>>
> >>>
> >>>
> >>> I think that's broad enough to cover all bases, but I'll wait for your
> >>> response.
> >>
> >> Since this is exposed to user space, ideally all architectures would use
> >> the same bits to mean the same thing. I assumed that is what James intended
> >> by placing the existing (AMD BMEC) bits in the global resctrl_types.h.
> >>
> >> Reinette
> > 
> > Maybe, but the bits as defined by AMD BMEC look rather architecture and
> > bus specific, and I am suspicious that there is no guaranteed clean
> > mapping between MPAM's config and BMEC's config.
> > 
> > MPAM currently just has "reads" and "writes" (or both), though as for
> > BMEC, the meanings of these are not fully defined.  I suppose finer
> > filtering granularity might be supported in future (at least, it is not
> > explicitly ruled out).
> > 
> > James' current approach seems to be to pick a single BMEC flag that's
> > in the right sort of area for each MPAM bit (though not equivalent) and
> > translate that bit across to drive a corresponding the MPAM bit.  But
> > I'd say that this is arch-specific configuration masquerading as
> > generic configuration IMHO and not really generic at all.
> > 
> > See "untested: arm_mpam: resctrl: Allow monitors to be configured"
> > https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/commit/?h=mpam/snapshot/v6.7-rc2&id=db0ac51f60675b6c4a54ccd24fa7198ec321c56d
> > 
> > I guess this needs discussion with James, since there will have been
> > additional thought process behind all this that is not captured; either
> > way, I guess it could be resolved after this series, but it will need a
> > decision before the MPAM support is merged (or at least, before MPAM
> > exposes userspace support for event configuration upstream).
> > 
> > (If this has already been discussed and James' current approach has
> > already been agreed as the least worst option, then I guess I can live
> > with it; I just find it icky, and it looks odd to have AMD specifics in
> > a common header.)
> 
> I am not aware of such a discussion.
> 
> Sounds like a motivation to delay this portion of the changes in patch #8.
> 
> Reinette


Ack, I'll discuss this with James.

I guess the thing to do will be to keep the affected definitions in the
x86 headers for now, and carry the exports in James' MPAM branch until
we figure out whether it really makes sense to share them.

Cheers
---Dave

Re: [PATCH v1 17/31] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource

[James Morse]

Hi guys,

On 18/04/2024 16:26, Dave Martin wrote:
> On Wed, Apr 17, 2024 at 10:18:48PM -0700, Reinette Chatre wrote:
>> On 4/17/2024 7:41 AM, Dave Martin wrote:
>>> On Thu, Apr 11, 2024 at 10:39:06AM -0700, Reinette Chatre wrote:
>>>> On 4/11/2024 7:16 AM, Dave Martin wrote:
>>>>> On Mon, Apr 08, 2024 at 08:21:24PM -0700, Reinette Chatre wrote:
>>>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>>>>> The mbm_cfg_mask field lists the bits that user-space can set when
>>>>>>> configuring an event. This value is output via the last_cmd_status
>>>>>>> file.
>>>>>>>
>>>>>>> Once the filesystem parts of resctrl are moved to live in /fs/, the
>>>>>>> struct rdt_hw_resource is inaccessible to the filesystem code. Because
>>>>>>> this value is output to user-space, it has to be accessible to the
>>>>>>> filesystem code.
>>>>>>>
>>>>>>> Move it to struct rdt_resource.

>>> Maybe, but the bits as defined by AMD BMEC look rather architecture and
>>> bus specific, and I am suspicious that there is no guaranteed clean
>>> mapping between MPAM's config and BMEC's config.
>>>
>>> MPAM currently just has "reads" and "writes" (or both), though as for
>>> BMEC, the meanings of these are not fully defined.  I suppose finer
>>> filtering granularity might be supported in future (at least, it is not
>>> explicitly ruled out).
>>>
>>> James' current approach seems to be to pick a single BMEC flag that's
>>> in the right sort of area for each MPAM bit (though not equivalent) and
>>> translate that bit across to drive a corresponding the MPAM bit.  But
>>> I'd say that this is arch-specific configuration masquerading as
>>> generic configuration IMHO and not really generic at all.
>>>
>>> See "untested: arm_mpam: resctrl: Allow monitors to be configured"
>>> https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/commit/?h=mpam/snapshot/v6.7-rc2&id=db0ac51f60675b6c4a54ccd24fa7198ec321c56d
>>>
>>> I guess this needs discussion with James, since there will have been
>>> additional thought process behind all this that is not captured; either
>>> way, I guess it could be resolved after this series, but it will need a
>>> decision before the MPAM support is merged (or at least, before MPAM
>>> exposes userspace support for event configuration upstream).
>>>
>>> (If this has already been discussed and James' current approach has
>>> already been agreed as the least worst option, then I guess I can live
>>> with it; I just find it icky, and it looks odd to have AMD specifics in
>>> a common header.)
>>
>> I am not aware of such a discussion.
>>
>> Sounds like a motivation to delay this portion of the changes in patch #8.
>>
>> Reinette
> 
> 
> Ack, I'll discuss this with James.
> 
> I guess the thing to do will be to keep the affected definitions in the
> x86 headers for now, and carry the exports in James' MPAM branch until
> we figure out whether it really makes sense to share them.

I'm afraid this ship has already sailed. Nothing about the 'mbm_total_bytes_config'
section of Documentation/arch/x86/resctrl.rst says that this interface is AMD specific. It
just happens not to be supported on Intel parts, meaning no-one can tell the difference today.

Because the documentation doesn't say "AMD only" or "consult the lid of the box for the
meaning of the bits" - I expect user-space can expect this to work in the same way on any
platform that supports these files. As such, it has become resctrl's interface to this
stuff. We certainly don't want two different ways of doing this.


As MPAM can configure the monitors like this, I'm stuck between a rock and a hard place. I
can't invent something new - because existing user-space expects BMEC, and making these
bits generic is being questioned because its secretly AMD specific.


Mapping the BMEC^Wresctrl bits to what MPAM supports is my best attempt at supporting
this. The MPAM driver is all about fitting MPAM into a shape that can be exposed via
resctrl, so this is hardly out of keeping. (I could make the same argument about the
MBA percentages...)

The same argument holds for X86's event numbers. Some agreement on IDs for events is
needed between the arch and fs code - and we may as well stick with the X86 numbers today
- its certainly more convenient for the X86 arch code. Where arm64/MPAM or any other
architecture extends this, I'd hope to do it via perf where numbered events are already
understood to be platform specific, and there is tooling to support this.

For configurable events on non-AMD architectures? We can kick that into the perf weeds -
but we need somewhere to hang the fs/resctrl code that drives these files. I think
'is_AMD()' is detestable, and as these bits are uapi, they should be exposed for all
architectures that support resctrl.


Thanks,

James

[PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[James Morse]

resctrl_arch_mon_event_config_write() writes a bitmap of events provided
by user-space into the configuration register for the monitors.

This assumes that all architectures support all the features each bit
corresponds to.

MPAM can filter monitors based on read, write, or both, but there are
many more options in the existing bitmap. To allow this interface to
work for machines with MPAM, allow the architecture helper to return
an error if an incompatible bitmap is set.

When valid values are provided, there is no change in behaviour. If
an invalid value is provided, currently it is silently ignored, but
last_cmd_status is updated. After this change, the parser will stop
at the first invalid value and return an error to user-space. This
matches the way changes to the schemata file are made.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 20 ++++++++++++++++----
 include/linux/resctrl.h                |  1 +
 2 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 3d3a839eba6b..56a0bfdc11f7 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -1685,13 +1685,16 @@ void resctrl_arch_mon_event_config_write(void *info)
 	index = mon_event_config_index_get(mon_info->evtid);
 	if (index == INVALID_CONFIG_INDEX) {
 		pr_warn_once("Invalid event id %d\n", mon_info->evtid);
+		mon_info->err = -EINVAL;
 		return;
 	}
 	wrmsr(MSR_IA32_EVT_CFG_BASE + index, mon_info->mon_config, 0);
+
+	mon_info->err = 0;
 }
 
-static void mbm_config_write_domain(struct rdt_resource *r,
-				    struct rdt_domain *d, u32 evtid, u32 val)
+static int mbm_config_write_domain(struct rdt_resource *r,
+				   struct rdt_domain *d, u32 evtid, u32 val)
 {
 	struct resctrl_mon_config_info mon_info = {0};
 
@@ -1704,7 +1707,7 @@ static void mbm_config_write_domain(struct rdt_resource *r,
 	mon_info.evtid = evtid;
 	mondata_config_read(&mon_info);
 	if (mon_info.mon_config == val)
-		return;
+		return 0;
 
 	mon_info.mon_config = val;
 
@@ -1716,6 +1719,10 @@ static void mbm_config_write_domain(struct rdt_resource *r,
 	 */
 	smp_call_function_any(&d->cpu_mask, resctrl_arch_mon_event_config_write,
 			      &mon_info, 1);
+	if (mon_info.err) {
+		rdt_last_cmd_puts("Invalid event configuration\n");
+		return mon_info.err;
+	}
 
 	/*
 	 * When an Event Configuration is changed, the bandwidth counters
@@ -1727,6 +1734,8 @@ static void mbm_config_write_domain(struct rdt_resource *r,
 	 * mbm_local and mbm_total counts for all the RMIDs.
 	 */
 	resctrl_arch_reset_rmid_all(r, d);
+
+	return 0;
 }
 
 static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
@@ -1734,6 +1743,7 @@ static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
 	char *dom_str = NULL, *id_str;
 	unsigned long dom_id, val;
 	struct rdt_domain *d;
+	int err;
 
 	/* Walking r->domains, ensure it can't race with cpuhp */
 	lockdep_assert_cpus_held();
@@ -1765,7 +1775,9 @@ static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
 
 	list_for_each_entry(d, &r->domains, list) {
 		if (d->id == dom_id) {
-			mbm_config_write_domain(r, d, evtid, val);
+			err = mbm_config_write_domain(r, d, evtid, val);
+			if (err)
+				return err;
 			goto next;
 		}
 	}
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 8a7367d1ce45..6705d7960dfd 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -200,6 +200,7 @@ struct resctrl_mon_config_info {
 	struct rdt_domain   *d;
 	u32                  evtid;
 	u32                  mon_config;
+	int                  err;
 };
 
 /*
-- 
2.39.2

Re: [PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> resctrl_arch_mon_event_config_write() writes a bitmap of events provided
> by user-space into the configuration register for the monitors.
> 
> This assumes that all architectures support all the features each bit
> corresponds to.
> 
> MPAM can filter monitors based on read, write, or both, but there are
> many more options in the existing bitmap. To allow this interface to
> work for machines with MPAM, allow the architecture helper to return
> an error if an incompatible bitmap is set.
> 
> When valid values are provided, there is no change in behaviour. If
> an invalid value is provided, currently it is silently ignored, but
> last_cmd_status is updated. After this change, the parser will stop
> at the first invalid value and return an error to user-space. This
> matches the way changes to the schemata file are made.
> 

Is this needed? With move of mbm_cfg_mask to rdt_resource I expect
MPAM would use it to set what the valid values are. With that done,
when user space provides a value, mon_config_write() compares user
provided value against mbm_cfg_mask and will already return early
(before attempting to write to hardware) with error
if value is not supported. This seems to accomplish the goal of this
patch?

> Signed-off-by: James Morse <james.morse@arm.com>
> ---

...
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index 8a7367d1ce45..6705d7960dfd 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -200,6 +200,7 @@ struct resctrl_mon_config_info {
>  	struct rdt_domain   *d;
>  	u32                  evtid;
>  	u32                  mon_config;
> +	int                  err;
>  };

Please take care to use consistent spacing.

Reinette

Re: [PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[Dave Martin]

On Mon, Apr 08, 2024 at 08:23:36PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > resctrl_arch_mon_event_config_write() writes a bitmap of events provided
> > by user-space into the configuration register for the monitors.
> > 
> > This assumes that all architectures support all the features each bit
> > corresponds to.
> > 
> > MPAM can filter monitors based on read, write, or both, but there are
> > many more options in the existing bitmap. To allow this interface to
> > work for machines with MPAM, allow the architecture helper to return
> > an error if an incompatible bitmap is set.
> > 
> > When valid values are provided, there is no change in behaviour. If
> > an invalid value is provided, currently it is silently ignored, but
> > last_cmd_status is updated. After this change, the parser will stop
> > at the first invalid value and return an error to user-space. This
> > matches the way changes to the schemata file are made.
> > 
> 
> Is this needed? With move of mbm_cfg_mask to rdt_resource I expect
> MPAM would use it to set what the valid values are. With that done,
> when user space provides a value, mon_config_write() compares user
> provided value against mbm_cfg_mask and will already return early
> (before attempting to write to hardware) with error
> if value is not supported. This seems to accomplish the goal of this
> patch?

This sounds plausible.

In a recent snapshot of James' MPAM code, it looks like we could be
initialising rdt_resource::mbm_cfg_mask when setting up the rdt_resource
struct for resctrl, though in fact this information is captured
differently right now.  I'm sure why (though James may have a
reason). [1]

I don't see an obvious reason though why we couldn't set mbm_cfg_mask
and detect bad config values globally in mon_config_write(), the same as
for the existing AMD BMEC case.

Nothing in the MPAM architecture stops hardware vendors from randomly
implementing different capabilities in different components of the
system, but provided that we only expose the globally supported subset
of event filtering capabilities to resctrl this approach looks workable.
This consistent with the James' MPAM code deals with other feature
mismatches across the system today.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2#n730

> 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> 
> ..
> > diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> > index 8a7367d1ce45..6705d7960dfd 100644
> > --- a/include/linux/resctrl.h
> > +++ b/include/linux/resctrl.h
> > @@ -200,6 +200,7 @@ struct resctrl_mon_config_info {
> >  	struct rdt_domain   *d;
> >  	u32                  evtid;
> >  	u32                  mon_config;
> > +	int                  err;
> >  };
> 
> Please take care to use consistent spacing.
> 
> Reinette

Noted FWIW (though I guess this code might change or go away).

Cheers
---Dave

Re: [PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:17 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:23:36PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> resctrl_arch_mon_event_config_write() writes a bitmap of events provided
>>> by user-space into the configuration register for the monitors.
>>>
>>> This assumes that all architectures support all the features each bit
>>> corresponds to.
>>>
>>> MPAM can filter monitors based on read, write, or both, but there are
>>> many more options in the existing bitmap. To allow this interface to
>>> work for machines with MPAM, allow the architecture helper to return
>>> an error if an incompatible bitmap is set.
>>>
>>> When valid values are provided, there is no change in behaviour. If
>>> an invalid value is provided, currently it is silently ignored, but
>>> last_cmd_status is updated. After this change, the parser will stop
>>> at the first invalid value and return an error to user-space. This
>>> matches the way changes to the schemata file are made.
>>>
>>
>> Is this needed? With move of mbm_cfg_mask to rdt_resource I expect
>> MPAM would use it to set what the valid values are. With that done,
>> when user space provides a value, mon_config_write() compares user
>> provided value against mbm_cfg_mask and will already return early
>> (before attempting to write to hardware) with error
>> if value is not supported. This seems to accomplish the goal of this
>> patch?
> 
> This sounds plausible.
> 
> In a recent snapshot of James' MPAM code, it looks like we could be
> initialising rdt_resource::mbm_cfg_mask when setting up the rdt_resource
> struct for resctrl, though in fact this information is captured
> differently right now.  I'm sure why (though James may have a
> reason). [1]
> 
> I don't see an obvious reason though why we couldn't set mbm_cfg_mask
> and detect bad config values globally in mon_config_write(), the same as
> for the existing AMD BMEC case.
> 
> Nothing in the MPAM architecture stops hardware vendors from randomly
> implementing different capabilities in different components of the
> system, but provided that we only expose the globally supported subset
> of event filtering capabilities to resctrl this approach looks workable.
> This consistent with the James' MPAM code deals with other feature
> mismatches across the system today.
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2#n730

My response was based on what I understood from the goal of this change
as described by the changelog. The patch does not appear to match with
the goals stated in changelog.

As I understand the patch it aims to detect when there is an invalid
event id. It is not possible for this scenario to occur because this code
is always called with a valid event id.

Reinette

Re: [PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[Dave Martin]

Hi Rainette,

On Thu, Apr 11, 2024 at 10:39:37AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:17 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:23:36PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> >>> resctrl_arch_mon_event_config_write() writes a bitmap of events provided
> >>> by user-space into the configuration register for the monitors.
> >>>
> >>> This assumes that all architectures support all the features each bit
> >>> corresponds to.
> >>>
> >>> MPAM can filter monitors based on read, write, or both, but there are
> >>> many more options in the existing bitmap. To allow this interface to
> >>> work for machines with MPAM, allow the architecture helper to return
> >>> an error if an incompatible bitmap is set.
> >>>
> >>> When valid values are provided, there is no change in behaviour. If
> >>> an invalid value is provided, currently it is silently ignored, but
> >>> last_cmd_status is updated. After this change, the parser will stop
> >>> at the first invalid value and return an error to user-space. This
> >>> matches the way changes to the schemata file are made.
> >>>
> >>
> >> Is this needed? With move of mbm_cfg_mask to rdt_resource I expect
> >> MPAM would use it to set what the valid values are. With that done,
> >> when user space provides a value, mon_config_write() compares user
> >> provided value against mbm_cfg_mask and will already return early
> >> (before attempting to write to hardware) with error
> >> if value is not supported. This seems to accomplish the goal of this
> >> patch?
> > 
> > This sounds plausible.
> > 
> > In a recent snapshot of James' MPAM code, it looks like we could be
> > initialising rdt_resource::mbm_cfg_mask when setting up the rdt_resource
> > struct for resctrl, though in fact this information is captured
> > differently right now.  I'm sure why (though James may have a
> > reason). [1]
> > 
> > I don't see an obvious reason though why we couldn't set mbm_cfg_mask
> > and detect bad config values globally in mon_config_write(), the same as
> > for the existing AMD BMEC case.
> > 
> > Nothing in the MPAM architecture stops hardware vendors from randomly
> > implementing different capabilities in different components of the
> > system, but provided that we only expose the globally supported subset
> > of event filtering capabilities to resctrl this approach looks workable.
> > This consistent with the James' MPAM code deals with other feature
> > mismatches across the system today.
> > 
> > [1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2#n730
> 
> My response was based on what I understood from the goal of this change
> as described by the changelog. The patch does not appear to match with
> the goals stated in changelog.
> 
> As I understand the patch it aims to detect when there is an invalid
> event id. It is not possible for this scenario to occur because this code
> is always called with a valid event id.
> 
> Reinette

I guess this will need discussion with James.  FWIW, my impression was
that the real goal of this patch was to allow a bad event config to be
detected at cross-call time and reported asynchronously.  Changes
elsewhere look to be there just to make error reporting consistent for
other existing paths too.

Either way, I agree that we really ought be be able to detect and
report "bad event config" errors synchronously, unless I'm missing
something.  I'll discuss with James whether we should be dropping this
patch.

(For MPAM, I suppose some componets grouped as a single resctrl resource
could support event configuration and some not, but I'd hope that we can
just not expose event configurability to resctrl at all in that case.
A sane system design should not be affected.)

Cheers
---Dave
> 
> 
> 
> 
> 

Re: [PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[Reinette Chatre]

Hi Dave,

On 4/17/2024 7:42 AM, Dave Martin wrote:
> Hi Rainette,
> 
> On Thu, Apr 11, 2024 at 10:39:37AM -0700, Reinette Chatre wrote:
>> Hi Dave,
>>
>> On 4/11/2024 7:17 AM, Dave Martin wrote:
>>> On Mon, Apr 08, 2024 at 08:23:36PM -0700, Reinette Chatre wrote:
>>>> Hi James,
>>>>
>>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>>> resctrl_arch_mon_event_config_write() writes a bitmap of events provided
>>>>> by user-space into the configuration register for the monitors.
>>>>>
>>>>> This assumes that all architectures support all the features each bit
>>>>> corresponds to.
>>>>>
>>>>> MPAM can filter monitors based on read, write, or both, but there are
>>>>> many more options in the existing bitmap. To allow this interface to
>>>>> work for machines with MPAM, allow the architecture helper to return
>>>>> an error if an incompatible bitmap is set.
>>>>>
>>>>> When valid values are provided, there is no change in behaviour. If
>>>>> an invalid value is provided, currently it is silently ignored, but
>>>>> last_cmd_status is updated. After this change, the parser will stop
>>>>> at the first invalid value and return an error to user-space. This
>>>>> matches the way changes to the schemata file are made.
>>>>>
>>>>
>>>> Is this needed? With move of mbm_cfg_mask to rdt_resource I expect
>>>> MPAM would use it to set what the valid values are. With that done,
>>>> when user space provides a value, mon_config_write() compares user
>>>> provided value against mbm_cfg_mask and will already return early
>>>> (before attempting to write to hardware) with error
>>>> if value is not supported. This seems to accomplish the goal of this
>>>> patch?
>>>
>>> This sounds plausible.
>>>
>>> In a recent snapshot of James' MPAM code, it looks like we could be
>>> initialising rdt_resource::mbm_cfg_mask when setting up the rdt_resource
>>> struct for resctrl, though in fact this information is captured
>>> differently right now.  I'm sure why (though James may have a
>>> reason). [1]
>>>
>>> I don't see an obvious reason though why we couldn't set mbm_cfg_mask
>>> and detect bad config values globally in mon_config_write(), the same as
>>> for the existing AMD BMEC case.
>>>
>>> Nothing in the MPAM architecture stops hardware vendors from randomly
>>> implementing different capabilities in different components of the
>>> system, but provided that we only expose the globally supported subset
>>> of event filtering capabilities to resctrl this approach looks workable.
>>> This consistent with the James' MPAM code deals with other feature
>>> mismatches across the system today.
>>>
>>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2#n730
>>
>> My response was based on what I understood from the goal of this change
>> as described by the changelog. The patch does not appear to match with
>> the goals stated in changelog.
>>
>> As I understand the patch it aims to detect when there is an invalid
>> event id. It is not possible for this scenario to occur because this code
>> is always called with a valid event id.
>>
>> Reinette
> 
> I guess this will need discussion with James.  FWIW, my impression was
> that the real goal of this patch was to allow a bad event config to be
> detected at cross-call time and reported asynchronously.  Changes
> elsewhere look to be there just to make error reporting consistent for
> other existing paths too.

How do you interpret "bad event config"?

As I understand it, this patch only sets an error in one scenario:

	 	index = mon_event_config_index_get(mon_info->evtid);
	 	if (index == INVALID_CONFIG_INDEX) {
 			pr_warn_once("Invalid event id %d\n", mon_info->evtid);
			mon_info->err = -EINVAL;
	 		return;
 		}

When will mon_info->evtid be anything but QOS_L3_MBM_TOTAL_EVENT_ID or
QOS_L3_MBM_LOCAL_EVENT_ID? 

Reinette

Re: [PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[Dave Martin]

On Wed, Apr 17, 2024 at 10:19:31PM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/17/2024 7:42 AM, Dave Martin wrote:
> > Hi Rainette,
> > 
> > On Thu, Apr 11, 2024 at 10:39:37AM -0700, Reinette Chatre wrote:
> >> Hi Dave,
> >>
> >> On 4/11/2024 7:17 AM, Dave Martin wrote:
> >>> On Mon, Apr 08, 2024 at 08:23:36PM -0700, Reinette Chatre wrote:
> >>>> Hi James,
> >>>>
> >>>> On 3/21/2024 9:50 AM, James Morse wrote:
> >>>>> resctrl_arch_mon_event_config_write() writes a bitmap of events provided
> >>>>> by user-space into the configuration register for the monitors.
> >>>>>
> >>>>> This assumes that all architectures support all the features each bit
> >>>>> corresponds to.
> >>>>>
> >>>>> MPAM can filter monitors based on read, write, or both, but there are
> >>>>> many more options in the existing bitmap. To allow this interface to
> >>>>> work for machines with MPAM, allow the architecture helper to return
> >>>>> an error if an incompatible bitmap is set.
> >>>>>
> >>>>> When valid values are provided, there is no change in behaviour. If
> >>>>> an invalid value is provided, currently it is silently ignored, but
> >>>>> last_cmd_status is updated. After this change, the parser will stop
> >>>>> at the first invalid value and return an error to user-space. This
> >>>>> matches the way changes to the schemata file are made.
> >>>>>
> >>>>
> >>>> Is this needed? With move of mbm_cfg_mask to rdt_resource I expect
> >>>> MPAM would use it to set what the valid values are. With that done,
> >>>> when user space provides a value, mon_config_write() compares user
> >>>> provided value against mbm_cfg_mask and will already return early
> >>>> (before attempting to write to hardware) with error
> >>>> if value is not supported. This seems to accomplish the goal of this
> >>>> patch?
> >>>
> >>> This sounds plausible.
> >>>
> >>> In a recent snapshot of James' MPAM code, it looks like we could be
> >>> initialising rdt_resource::mbm_cfg_mask when setting up the rdt_resource
> >>> struct for resctrl, though in fact this information is captured
> >>> differently right now.  I'm sure why (though James may have a
> >>> reason). [1]
> >>>
> >>> I don't see an obvious reason though why we couldn't set mbm_cfg_mask
> >>> and detect bad config values globally in mon_config_write(), the same as
> >>> for the existing AMD BMEC case.
> >>>
> >>> Nothing in the MPAM architecture stops hardware vendors from randomly
> >>> implementing different capabilities in different components of the
> >>> system, but provided that we only expose the globally supported subset
> >>> of event filtering capabilities to resctrl this approach looks workable.
> >>> This consistent with the James' MPAM code deals with other feature
> >>> mismatches across the system today.
> >>>
> >>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2#n730
> >>
> >> My response was based on what I understood from the goal of this change
> >> as described by the changelog. The patch does not appear to match with
> >> the goals stated in changelog.
> >>
> >> As I understand the patch it aims to detect when there is an invalid
> >> event id. It is not possible for this scenario to occur because this code
> >> is always called with a valid event id.
> >>
> >> Reinette
> > 
> > I guess this will need discussion with James.  FWIW, my impression was
> > that the real goal of this patch was to allow a bad event config to be
> > detected at cross-call time and reported asynchronously.  Changes
> > elsewhere look to be there just to make error reporting consistent for
> > other existing paths too.
> 
> How do you interpret "bad event config"?
> 
> As I understand it, this patch only sets an error in one scenario:
> 
> 	 	index = mon_event_config_index_get(mon_info->evtid);
> 	 	if (index == INVALID_CONFIG_INDEX) {
>  			pr_warn_once("Invalid event id %d\n", mon_info->evtid);
> 			mon_info->err = -EINVAL;
> 	 		return;
>  		}
> 
> When will mon_info->evtid be anything but QOS_L3_MBM_TOTAL_EVENT_ID or
> QOS_L3_MBM_LOCAL_EVENT_ID? 
> 
> Reinette

I don't know; my reading of this was that since there was a pr_warn()
already, and since James was adding the capability to return an error,
he figured that a suitable error ought to be returned in this case.

But the real reason for the error return mechanism seems to be
resctrl_arch_mon_event_config_write() in the MPAM code, here:

https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/commit/?h=mpam/snapshot/v6.7-rc2&id=db0ac51f60675b6c4a54ccd24fa7198ec321c56d

I agree though that if we set mbm_cfg_mask in the rdt_resource at probe
time, the code in mon_config_write() ought to catch such cases cleanly
before making the cross-call.  So maybe the new mechanism isn't needed.

I think I need to discuss this with James, to figure out if there's any
reason why that wouldn't work.

Cheers
---Dave

Re: [PATCH v1 18/31] x86/resctrl: Allow resctrl_arch_mon_event_config_write() to return an error

[James Morse]

Hi Reinette,

On 09/04/2024 04:23, Reinette Chatre wrote:
> On 3/21/2024 9:50 AM, James Morse wrote:
>> resctrl_arch_mon_event_config_write() writes a bitmap of events provided
>> by user-space into the configuration register for the monitors.
>>
>> This assumes that all architectures support all the features each bit
>> corresponds to.
>>
>> MPAM can filter monitors based on read, write, or both, but there are
>> many more options in the existing bitmap. To allow this interface to
>> work for machines with MPAM, allow the architecture helper to return
>> an error if an incompatible bitmap is set.
>>
>> When valid values are provided, there is no change in behaviour. If
>> an invalid value is provided, currently it is silently ignored, but
>> last_cmd_status is updated. After this change, the parser will stop
>> at the first invalid value and return an error to user-space. This
>> matches the way changes to the schemata file are made.
>>
> 
> Is this needed? With move of mbm_cfg_mask to rdt_resource I expect
> MPAM would use it to set what the valid values are. With that done,
> when user space provides a value, mon_config_write() compares user
> provided value against mbm_cfg_mask and will already return early
> (before attempting to write to hardware) with error
> if value is not supported. This seems to accomplish the goal of this
> patch?

Aha, this is done in mon_config_write(), not mbm_config_write_domain(). I'd missed that.
I'll drop this patch.

Thanks!

James

[PATCH v1 19/31] x86/resctrl: Add resctrl_arch_ prefix to pseudo lock functions

[James Morse]

resctrl's pseudo lock has some copy-to-cache and measurement
functions that are micro-architecture specific. pseudo_lock_fn()
is not at all portable. Label these 'resctrl_arch_' so they stay
under /arch/x86.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/include/asm/resctrl.h            |  5 ++++
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 36 ++++++++++++-----------
 2 files changed, 24 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 50407e83d0ca..a88af68f9fe2 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -211,6 +211,11 @@ static inline void *resctrl_arch_mon_ctx_alloc(struct rdt_resource *r, int evtid
 static inline void resctrl_arch_mon_ctx_free(struct rdt_resource *r, int evtid,
 					     void *ctx) { };
 
+u64 resctrl_arch_get_prefetch_disable_bits(void);
+int resctrl_arch_pseudo_lock_fn(void *_rdtgrp);
+int resctrl_arch_measure_cycles_lat_fn(void *_plr);
+int resctrl_arch_measure_l2_residency(void *_plr);
+int resctrl_arch_measure_l3_residency(void *_plr);
 void resctrl_cpu_detect(struct cpuinfo_x86 *c);
 
 #else
diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
index f2315a50ea4f..856beb6f668b 100644
--- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
+++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
@@ -62,7 +62,8 @@ static const struct class pseudo_lock_class = {
 };
 
 /**
- * get_prefetch_disable_bits - prefetch disable bits of supported platforms
+ * resctrl_arch_get_prefetch_disable_bits - prefetch disable bits of supported
+ *                                          platforms
  * @void: It takes no parameters.
  *
  * Capture the list of platforms that have been validated to support
@@ -76,13 +77,13 @@ static const struct class pseudo_lock_class = {
  * in the SDM.
  *
  * When adding a platform here also add support for its cache events to
- * measure_cycles_perf_fn()
+ * resctrl_arch_measure_l*_residency()
  *
  * Return:
  * If platform is supported, the bits to disable hardware prefetchers, 0
  * if platform is not supported.
  */
-static u64 get_prefetch_disable_bits(void)
+u64 resctrl_arch_get_prefetch_disable_bits(void)
 {
 	if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL ||
 	    boot_cpu_data.x86 != 6)
@@ -410,7 +411,7 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp)
 }
 
 /**
- * pseudo_lock_fn - Load kernel memory into cache
+ * resctrl_arch_pseudo_lock_fn - Load kernel memory into cache
  * @_rdtgrp: resource group to which pseudo-lock region belongs
  *
  * This is the core pseudo-locking flow.
@@ -428,7 +429,7 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp)
  *
  * Return: 0. Waiter on waitqueue will be woken on completion.
  */
-static int pseudo_lock_fn(void *_rdtgrp)
+int resctrl_arch_pseudo_lock_fn(void *_rdtgrp)
 {
 	struct rdtgroup *rdtgrp = _rdtgrp;
 	struct pseudo_lock_region *plr = rdtgrp->plr;
@@ -714,7 +715,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
 	 * Not knowing the bits to disable prefetching implies that this
 	 * platform does not support Cache Pseudo-Locking.
 	 */
-	prefetch_disable_bits = get_prefetch_disable_bits();
+	prefetch_disable_bits = resctrl_arch_get_prefetch_disable_bits();
 	if (prefetch_disable_bits == 0) {
 		rdt_last_cmd_puts("Pseudo-locking not supported\n");
 		return -EINVAL;
@@ -879,7 +880,8 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
 }
 
 /**
- * measure_cycles_lat_fn - Measure cycle latency to read pseudo-locked memory
+ * resctrl_arch_measure_cycles_lat_fn - Measure cycle latency to read
+ *                                      pseudo-locked memory
  * @_plr: pseudo-lock region to measure
  *
  * There is no deterministic way to test if a memory region is cached. One
@@ -892,7 +894,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
  *
  * Return: 0. Waiter on waitqueue will be woken on completion.
  */
-static int measure_cycles_lat_fn(void *_plr)
+int resctrl_arch_measure_cycles_lat_fn(void *_plr)
 {
 	struct pseudo_lock_region *plr = _plr;
 	u32 saved_low, saved_high;
@@ -1076,7 +1078,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr,
 	return 0;
 }
 
-static int measure_l2_residency(void *_plr)
+int resctrl_arch_measure_l2_residency(void *_plr)
 {
 	struct pseudo_lock_region *plr = _plr;
 	struct residency_counts counts = {0};
@@ -1114,7 +1116,7 @@ static int measure_l2_residency(void *_plr)
 	return 0;
 }
 
-static int measure_l3_residency(void *_plr)
+int resctrl_arch_measure_l3_residency(void *_plr)
 {
 	struct pseudo_lock_region *plr = _plr;
 	struct residency_counts counts = {0};
@@ -1212,18 +1214,18 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel)
 	plr->cpu = cpu;
 
 	if (sel == 1)
-		thread = kthread_create_on_node(measure_cycles_lat_fn, plr,
-						cpu_to_node(cpu),
+		thread = kthread_create_on_node(resctrl_arch_measure_cycles_lat_fn,
+						plr, cpu_to_node(cpu),
 						"pseudo_lock_measure/%u",
 						cpu);
 	else if (sel == 2)
-		thread = kthread_create_on_node(measure_l2_residency, plr,
-						cpu_to_node(cpu),
+		thread = kthread_create_on_node(resctrl_arch_measure_l2_residency,
+						plr, cpu_to_node(cpu),
 						"pseudo_lock_measure/%u",
 						cpu);
 	else if (sel == 3)
-		thread = kthread_create_on_node(measure_l3_residency, plr,
-						cpu_to_node(cpu),
+		thread = kthread_create_on_node(resctrl_arch_measure_l3_residency,
+						plr, cpu_to_node(cpu),
 						"pseudo_lock_measure/%u",
 						cpu);
 	else
@@ -1322,7 +1324,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
 
 	plr->thread_done = 0;
 
-	thread = kthread_create_on_node(pseudo_lock_fn, rdtgrp,
+	thread = kthread_create_on_node(resctrl_arch_pseudo_lock_fn, rdtgrp,
 					cpu_to_node(plr->cpu),
 					"pseudo_lock/%u", plr->cpu);
 	if (IS_ERR(thread)) {
-- 
2.39.2

[PATCH v1 20/31] x86/resctrl: Allow an architecture to disable pseudo lock

[James Morse]

Pseudo-lock relies on knowledge of the micro-architecture to disable
prefetchers etc.

On arm64 these controls are typically secure only, meaning linux can't
access them. Arm's cache-lockdown feature works in a very different
way. Resctrl's pseudo-lock isn't going to be used on arm64 platforms.

Add a Kconfig symbol that can be selected by the architecture. This
enables or disables building of the psuedo_lock.c file, and replaces
the functions with stubs. An additional IS_ENABLED() check is needed
in rdtgroup_mode_write() so that attempting to enable pseudo-lock
reports an "Unknown or unsupported mode" to user-space.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/Kconfig                       |  7 ++++
 arch/x86/kernel/cpu/resctrl/Makefile   |  5 +--
 arch/x86/kernel/cpu/resctrl/internal.h | 48 +++++++++++++++++++++-----
 arch/x86/kernel/cpu/resctrl/rdtgroup.c |  3 +-
 4 files changed, 52 insertions(+), 11 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 7aed87cbf386..e071e564452e 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -481,6 +481,7 @@ config X86_CPU_RESCTRL
 	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
 	select KERNFS
 	select PROC_CPU_RESCTRL		if PROC_FS
+	select RESCTRL_FS_PSEUDO_LOCK
 	help
 	  Enable x86 CPU resource control support.
 
@@ -506,6 +507,12 @@ config X86_FRED
 	  ring transitions and exception/interrupt handling if the
 	  system supports.
 
+config RESCTRL_FS_PSEUDO_LOCK
+	bool
+	help
+	  Software mechanism to pin data in a cache portion using
+	  micro-architecture specific knowledge.
+
 if X86_32
 config X86_BIGSMP
 	bool "Support for big SMP systems with more than 8 CPUs"
diff --git a/arch/x86/kernel/cpu/resctrl/Makefile b/arch/x86/kernel/cpu/resctrl/Makefile
index 4a06c37b9cf1..0c13b0befd8a 100644
--- a/arch/x86/kernel/cpu/resctrl/Makefile
+++ b/arch/x86/kernel/cpu/resctrl/Makefile
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_X86_CPU_RESCTRL)	+= core.o rdtgroup.o monitor.o
-obj-$(CONFIG_X86_CPU_RESCTRL)	+= ctrlmondata.o pseudo_lock.o
+obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
+obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
+obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
 CFLAGS_pseudo_lock.o = -I$(src)
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 238b81d3f64a..d6db15839dc4 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -489,14 +489,6 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d,
 				  unsigned long cbm);
 enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);
 int rdtgroup_tasks_assigned(struct rdtgroup *r);
-int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
-int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp);
-bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm);
-bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d);
-int rdt_pseudo_lock_init(void);
-void rdt_pseudo_lock_release(void);
-int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
-void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
 struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);
 int closids_supported(void);
 void closid_free(int closid);
@@ -529,4 +521,44 @@ void rdt_staged_configs_clear(void);
 bool closid_allocated(unsigned int closid);
 int resctrl_find_cleanest_closid(void);
 
+#ifdef CONFIG_RESCTRL_FS_PSEUDO_LOCK
+int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
+int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp);
+bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm);
+bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d);
+int rdt_pseudo_lock_init(void);
+void rdt_pseudo_lock_release(void);
+int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
+void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
+#else
+static inline int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
+{
+	return false;
+}
+
+static inline bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
+{
+	return false;
+}
+
+static inline int rdt_pseudo_lock_init(void) { return 0; }
+static inline void rdt_pseudo_lock_release(void) { }
+static inline int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp) { }
+#endif /* CONFIG_RESCTRL_FS_PSEUDO_LOCK */
+
 #endif /* _ASM_X86_RESCTRL_INTERNAL_H */
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 56a0bfdc11f7..9275d6f8a74e 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -1452,7 +1452,8 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
 				goto out;
 		}
 		rdtgrp->mode = RDT_MODE_EXCLUSIVE;
-	} else if (!strcmp(buf, "pseudo-locksetup")) {
+	} else if (IS_ENABLED(CONFIG_RESCTRL_FS_PSEUDO_LOCK) &&
+		   !strcmp(buf, "pseudo-locksetup")) {
 		ret = rdtgroup_locksetup_enter(rdtgrp);
 		if (ret)
 			goto out;
-- 
2.39.2

Re: [PATCH v1 20/31] x86/resctrl: Allow an architecture to disable pseudo lock

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> Pseudo-lock relies on knowledge of the micro-architecture to disable
> prefetchers etc.
> 
> On arm64 these controls are typically secure only, meaning linux can't
> access them. Arm's cache-lockdown feature works in a very different
> way. Resctrl's pseudo-lock isn't going to be used on arm64 platforms.
> 
> Add a Kconfig symbol that can be selected by the architecture. This
> enables or disables building of the psuedo_lock.c file, and replaces

pseudo_lock.c

> the functions with stubs. An additional IS_ENABLED() check is needed
> in rdtgroup_mode_write() so that attempting to enable pseudo-lock
> reports an "Unknown or unsupported mode" to user-space.
> 

I am missing something here. It is not obvious to me why the IS_ENABLED()
check is needed. Wouldn't rdtgroup_locksetup_enter()
return -EOPNOTSUPP if CONFIG_RESCTRL_FS_PSEUDO_LOCK is not enabled?

Reinette

Re: [PATCH v1 20/31] x86/resctrl: Allow an architecture to disable pseudo lock

[Dave Martin]

On Mon, Apr 08, 2024 at 08:24:12PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > Pseudo-lock relies on knowledge of the micro-architecture to disable
> > prefetchers etc.
> > 
> > On arm64 these controls are typically secure only, meaning linux can't
> > access them. Arm's cache-lockdown feature works in a very different
> > way. Resctrl's pseudo-lock isn't going to be used on arm64 platforms.
> > 
> > Add a Kconfig symbol that can be selected by the architecture. This
> > enables or disables building of the psuedo_lock.c file, and replaces
> 
> pseudo_lock.c

Noted.

> > the functions with stubs. An additional IS_ENABLED() check is needed
> > in rdtgroup_mode_write() so that attempting to enable pseudo-lock
> > reports an "Unknown or unsupported mode" to user-space.
> > 
> 
> I am missing something here. It is not obvious to me why the IS_ENABLED()
> check is needed. Wouldn't rdtgroup_locksetup_enter()
> return -EOPNOTSUPP if CONFIG_RESCTRL_FS_PSEUDO_LOCK is not enabled?
> 
> Reinette
> 

Hmm, if I've understood all this correctly, then it looks like the
existing code in rdtgroup_mode_write() relies on the dispatched
function (rdtgroup_locksetup_enter() etc.) to do an appropriate
rdt_last_cmd_puts() on failure.  If no function is called at all and
the requested mode change is not a no-op or otherwise trivially
successful, then it looks like we're supposed to fall into the else
clause.

I'd guess James' intent here was to use the fallback else {} to write
a suitable status string, while keeping the stub functions as trivial
as possible.

Just taking the IS_ENABLED() away would result in error return from the
write(), but no suitable last_cmd_status string.

For consistency with the existing x86 implementation, I wonder whether
we should put a suitable rdt_last_cmd_puts() in the stub for
rdtgroup_locksetup_enter().

There might be other ways to refactor or simplify this, though.

Thoughts?

Cheers
---Dave

Re: [PATCH v1 20/31] x86/resctrl: Allow an architecture to disable pseudo lock

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:17 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:24:12PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> Pseudo-lock relies on knowledge of the micro-architecture to disable
>>> prefetchers etc.
>>>
>>> On arm64 these controls are typically secure only, meaning linux can't
>>> access them. Arm's cache-lockdown feature works in a very different
>>> way. Resctrl's pseudo-lock isn't going to be used on arm64 platforms.
>>>
>>> Add a Kconfig symbol that can be selected by the architecture. This
>>> enables or disables building of the psuedo_lock.c file, and replaces
>>
>> pseudo_lock.c
> 
> Noted.
> 
>>> the functions with stubs. An additional IS_ENABLED() check is needed
>>> in rdtgroup_mode_write() so that attempting to enable pseudo-lock
>>> reports an "Unknown or unsupported mode" to user-space.
>>>
>>
>> I am missing something here. It is not obvious to me why the IS_ENABLED()
>> check is needed. Wouldn't rdtgroup_locksetup_enter()
>> return -EOPNOTSUPP if CONFIG_RESCTRL_FS_PSEUDO_LOCK is not enabled?
>>
>> Reinette
>>
> 
> Hmm, if I've understood all this correctly, then it looks like the
> existing code in rdtgroup_mode_write() relies on the dispatched
> function (rdtgroup_locksetup_enter() etc.) to do an appropriate
> rdt_last_cmd_puts() on failure.  If no function is called at all and
> the requested mode change is not a no-op or otherwise trivially
> successful, then it looks like we're supposed to fall into the else
> clause.
> 
> I'd guess James' intent here was to use the fallback else {} to write
> a suitable status string, while keeping the stub functions as trivial
> as possible.
> 
> Just taking the IS_ENABLED() away would result in error return from the
> write(), but no suitable last_cmd_status string.
> 
> For consistency with the existing x86 implementation, I wonder whether
> we should put a suitable rdt_last_cmd_puts() in the stub for
> rdtgroup_locksetup_enter().
> 
> There might be other ways to refactor or simplify this, though.
> 
> Thoughts?

Thank you for digging into this. It was not obvious to me that
the changelog referred to the last_cmd_status string. I do
not think this warrants making the stubs more complicated.

Reinette

Re: [PATCH v1 20/31] x86/resctrl: Allow an architecture to disable pseudo lock

[Dave Martin]

Hi Reinette,

On Thu, Apr 11, 2024 at 10:40:03AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:17 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:24:12PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:50 AM, James Morse wrote:
> >>> Pseudo-lock relies on knowledge of the micro-architecture to disable
> >>> prefetchers etc.
> >>>
> >>> On arm64 these controls are typically secure only, meaning linux can't
> >>> access them. Arm's cache-lockdown feature works in a very different
> >>> way. Resctrl's pseudo-lock isn't going to be used on arm64 platforms.
> >>>
> >>> Add a Kconfig symbol that can be selected by the architecture. This
> >>> enables or disables building of the psuedo_lock.c file, and replaces
> >>
> >> pseudo_lock.c
> > 
> > Noted.
> > 
> >>> the functions with stubs. An additional IS_ENABLED() check is needed
> >>> in rdtgroup_mode_write() so that attempting to enable pseudo-lock
> >>> reports an "Unknown or unsupported mode" to user-space.
> >>>
> >>
> >> I am missing something here. It is not obvious to me why the IS_ENABLED()
> >> check is needed. Wouldn't rdtgroup_locksetup_enter()
> >> return -EOPNOTSUPP if CONFIG_RESCTRL_FS_PSEUDO_LOCK is not enabled?
> >>
> >> Reinette
> >>
> > 
> > Hmm, if I've understood all this correctly, then it looks like the
> > existing code in rdtgroup_mode_write() relies on the dispatched
> > function (rdtgroup_locksetup_enter() etc.) to do an appropriate
> > rdt_last_cmd_puts() on failure.  If no function is called at all and
> > the requested mode change is not a no-op or otherwise trivially
> > successful, then it looks like we're supposed to fall into the else
> > clause.
> > 
> > I'd guess James' intent here was to use the fallback else {} to write
> > a suitable status string, while keeping the stub functions as trivial
> > as possible.
> > 
> > Just taking the IS_ENABLED() away would result in error return from the
> > write(), but no suitable last_cmd_status string.
> > 
> > For consistency with the existing x86 implementation, I wonder whether
> > we should put a suitable rdt_last_cmd_puts() in the stub for
> > rdtgroup_locksetup_enter().
> > 
> > There might be other ways to refactor or simplify this, though.
> > 
> > Thoughts?
> 
> Thank you for digging into this. It was not obvious to me that
> the changelog referred to the last_cmd_status string. I do
> not think this warrants making the stubs more complicated.
> 
> Reinette
> 

OK, I'll leave this as-is for now.

Cheers
---Dave

[PATCH v1 21/31] x86/resctrl: Make prefetch_disable_bits belong to the arch code

[James Morse]

prefetch_disable_bits is set by rdtgroup_locksetup_enter() from a
value provided by the architecture, but is largely read by other
architecture helpers.

Instead of exporting this value, make
resctrl_arch_get_prefetch_disable_bits() set it so that the other
arch-code helpers can use the cached-value.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
index 856beb6f668b..5a66e3b2c2ea 100644
--- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
+++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
@@ -85,6 +85,8 @@ static const struct class pseudo_lock_class = {
  */
 u64 resctrl_arch_get_prefetch_disable_bits(void)
 {
+	prefetch_disable_bits = 0;
+
 	if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL ||
 	    boot_cpu_data.x86 != 6)
 		return 0;
@@ -100,7 +102,8 @@ u64 resctrl_arch_get_prefetch_disable_bits(void)
 		 * 3    DCU IP Prefetcher Disable (R/W)
 		 * 63:4 Reserved
 		 */
-		return 0xF;
+		prefetch_disable_bits = 0xF;
+		break;
 	case INTEL_FAM6_ATOM_GOLDMONT:
 	case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
 		/*
@@ -111,10 +114,11 @@ u64 resctrl_arch_get_prefetch_disable_bits(void)
 		 * 2     DCU Hardware Prefetcher Disable (R/W)
 		 * 63:3  Reserved
 		 */
-		return 0x5;
+		prefetch_disable_bits = 0x5;
+		break;
 	}
 
-	return 0;
+	return prefetch_disable_bits;
 }
 
 /**
@@ -715,8 +719,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
 	 * Not knowing the bits to disable prefetching implies that this
 	 * platform does not support Cache Pseudo-Locking.
 	 */
-	prefetch_disable_bits = resctrl_arch_get_prefetch_disable_bits();
-	if (prefetch_disable_bits == 0) {
+	if (resctrl_arch_get_prefetch_disable_bits() == 0) {
 		rdt_last_cmd_puts("Pseudo-locking not supported\n");
 		return -EINVAL;
 	}
-- 
2.39.2

[PATCH v1 22/31] x86/resctrl: Make resctrl_arch_pseudo_lock_fn() take a plr

[James Morse]

resctrl_arch_pseudo_lock_fn() has architecture specific behaviour,
and takes a struct rdtgroup as an argument.

After the filesystem code moves to /fs/, the definition of struct
rdtgroup will not be available to the architecture code.

The only reason resctrl_arch_pseudo_lock_fn() wants the rdtgroup is
for the CLOSID. Embed that in the pseudo_lock_region as a hw_closid,
and move the definition of struct pseudo_lock_region to resctrl.h.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/include/asm/resctrl.h            |  2 +-
 arch/x86/kernel/cpu/resctrl/internal.h    | 37 ---------------------
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 13 ++++----
 include/linux/resctrl.h                   | 39 +++++++++++++++++++++++
 4 files changed, 47 insertions(+), 44 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index a88af68f9fe2..9940398e367e 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -212,7 +212,7 @@ static inline void resctrl_arch_mon_ctx_free(struct rdt_resource *r, int evtid,
 					     void *ctx) { };
 
 u64 resctrl_arch_get_prefetch_disable_bits(void);
-int resctrl_arch_pseudo_lock_fn(void *_rdtgrp);
+int resctrl_arch_pseudo_lock_fn(void *_plr);
 int resctrl_arch_measure_cycles_lat_fn(void *_plr);
 int resctrl_arch_measure_l2_residency(void *_plr);
 int resctrl_arch_measure_l3_residency(void *_plr);
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index d6db15839dc4..be4e8f31b127 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -182,43 +182,6 @@ struct mongroup {
 	u32			rmid;
 };
 
-/**
- * struct pseudo_lock_region - pseudo-lock region information
- * @s:			Resctrl schema for the resource to which this
- *			pseudo-locked region belongs
- * @d:			RDT domain to which this pseudo-locked region
- *			belongs
- * @cbm:		bitmask of the pseudo-locked region
- * @lock_thread_wq:	waitqueue used to wait on the pseudo-locking thread
- *			completion
- * @thread_done:	variable used by waitqueue to test if pseudo-locking
- *			thread completed
- * @cpu:		core associated with the cache on which the setup code
- *			will be run
- * @line_size:		size of the cache lines
- * @size:		size of pseudo-locked region in bytes
- * @kmem:		the kernel memory associated with pseudo-locked region
- * @minor:		minor number of character device associated with this
- *			region
- * @debugfs_dir:	pointer to this region's directory in the debugfs
- *			filesystem
- * @pm_reqs:		Power management QoS requests related to this region
- */
-struct pseudo_lock_region {
-	struct resctrl_schema	*s;
-	struct rdt_domain	*d;
-	u32			cbm;
-	wait_queue_head_t	lock_thread_wq;
-	int			thread_done;
-	int			cpu;
-	unsigned int		line_size;
-	unsigned int		size;
-	void			*kmem;
-	unsigned int		minor;
-	struct dentry		*debugfs_dir;
-	struct list_head	pm_reqs;
-};
-
 /**
  * struct rdtgroup - store rdtgroup's data in resctrl file system.
  * @kn:				kernfs node
diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
index 5a66e3b2c2ea..ba51ab1f70e6 100644
--- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
+++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
@@ -416,7 +416,7 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp)
 
 /**
  * resctrl_arch_pseudo_lock_fn - Load kernel memory into cache
- * @_rdtgrp: resource group to which pseudo-lock region belongs
+ * @_plr: the pseudo-lock region descriptor
  *
  * This is the core pseudo-locking flow.
  *
@@ -433,10 +433,9 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp)
  *
  * Return: 0. Waiter on waitqueue will be woken on completion.
  */
-int resctrl_arch_pseudo_lock_fn(void *_rdtgrp)
+int resctrl_arch_pseudo_lock_fn(void *_plr)
 {
-	struct rdtgroup *rdtgrp = _rdtgrp;
-	struct pseudo_lock_region *plr = rdtgrp->plr;
+	struct pseudo_lock_region *plr = _plr;
 	u32 rmid_p, closid_p;
 	unsigned long i;
 	u64 saved_msr;
@@ -496,7 +495,8 @@ int resctrl_arch_pseudo_lock_fn(void *_rdtgrp)
 	 * pseudo-locked followed by reading of kernel memory to load it
 	 * into the cache.
 	 */
-	__wrmsr(MSR_IA32_PQR_ASSOC, rmid_p, rdtgrp->closid);
+	__wrmsr(MSR_IA32_PQR_ASSOC, rmid_p, plr->closid);
+
 	/*
 	 * Cache was flushed earlier. Now access kernel memory to read it
 	 * into cache region associated with just activated plr->closid.
@@ -1327,7 +1327,8 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
 
 	plr->thread_done = 0;
 
-	thread = kthread_create_on_node(resctrl_arch_pseudo_lock_fn, rdtgrp,
+	plr->closid = rdtgrp->closid;
+	thread = kthread_create_on_node(resctrl_arch_pseudo_lock_fn, plr,
 					cpu_to_node(plr->cpu),
 					"pseudo_lock/%u", plr->cpu);
 	if (IS_ERR(thread)) {
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 6705d7960dfd..3de5bc63ace0 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -25,6 +25,45 @@ int proc_resctrl_show(struct seq_file *m,
 /* max value for struct rdt_domain's mbps_val */
 #define MBA_MAX_MBPS   U32_MAX
 
+/**
+ * struct pseudo_lock_region - pseudo-lock region information
+ * @s:			Resctrl schema for the resource to which this
+ *			pseudo-locked region belongs
+ * @closid:		The closid that this pseudo-locked region uses
+ * @d:			RDT domain to which this pseudo-locked region
+ *			belongs
+ * @cbm:		bitmask of the pseudo-locked region
+ * @lock_thread_wq:	waitqueue used to wait on the pseudo-locking thread
+ *			completion
+ * @thread_done:	variable used by waitqueue to test if pseudo-locking
+ *			thread completed
+ * @cpu:		core associated with the cache on which the setup code
+ *			will be run
+ * @line_size:		size of the cache lines
+ * @size:		size of pseudo-locked region in bytes
+ * @kmem:		the kernel memory associated with pseudo-locked region
+ * @minor:		minor number of character device associated with this
+ *			region
+ * @debugfs_dir:	pointer to this region's directory in the debugfs
+ *			filesystem
+ * @pm_reqs:		Power management QoS requests related to this region
+ */
+struct pseudo_lock_region {
+	struct resctrl_schema	*s;
+	u32			closid;
+	struct rdt_domain	*d;
+	u32			cbm;
+	wait_queue_head_t	lock_thread_wq;
+	int			thread_done;
+	int			cpu;
+	unsigned int		line_size;
+	unsigned int		size;
+	void			*kmem;
+	unsigned int		minor;
+	struct dentry		*debugfs_dir;
+	struct list_head	pm_reqs;
+};
+
 /**
  * struct resctrl_staged_config - parsed configuration to be applied
  * @new_ctrl:		new ctrl value to be loaded
-- 
2.39.2

Re: [PATCH v1 22/31] x86/resctrl: Make resctrl_arch_pseudo_lock_fn() take a plr

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> resctrl_arch_pseudo_lock_fn() has architecture specific behaviour,
> and takes a struct rdtgroup as an argument.
> 
> After the filesystem code moves to /fs/, the definition of struct
> rdtgroup will not be available to the architecture code.
> 
> The only reason resctrl_arch_pseudo_lock_fn() wants the rdtgroup is
> for the CLOSID. Embed that in the pseudo_lock_region as a hw_closid,

Above creates expectation that the new member will be named hw_closid,
but that is not what the code does.

> and move the definition of struct pseudo_lock_region to resctrl.h.
> 
> Signed-off-by: James Morse <james.morse@arm.com>

...

> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index 6705d7960dfd..3de5bc63ace0 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -25,6 +25,45 @@ int proc_resctrl_show(struct seq_file *m,
>  /* max value for struct rdt_domain's mbps_val */
>  #define MBA_MAX_MBPS   U32_MAX
>  
> +/**
> + * struct pseudo_lock_region - pseudo-lock region information
> + * @s:			Resctrl schema for the resource to which this
> + *			pseudo-locked region belongs
> + * @closid:		The closid that this pseudo-locked region uses
> + * @d:			RDT domain to which this pseudo-locked region
> + *			belongs
> + * @cbm:		bitmask of the pseudo-locked region
> + * @lock_thread_wq:	waitqueue used to wait on the pseudo-locking thread
> + *			completion
> + * @thread_done:	variable used by waitqueue to test if pseudo-locking
> + *			thread completed
> + * @cpu:		core associated with the cache on which the setup code
> + *			will be run
> + * @line_size:		size of the cache lines
> + * @size:		size of pseudo-locked region in bytes
> + * @kmem:		the kernel memory associated with pseudo-locked region
> + * @minor:		minor number of character device associated with this
> + *			region
> + * @debugfs_dir:	pointer to this region's directory in the debugfs
> + *			filesystem
> + * @pm_reqs:		Power management QoS requests related to this region
> + */
> +struct pseudo_lock_region {
> +	struct resctrl_schema	*s;
> +	u32			closid;
> +	struct rdt_domain	*d;
> +	u32			cbm;
> +	wait_queue_head_t	lock_thread_wq;
> +	int			thread_done;
> +	int			cpu;
> +	unsigned int		line_size;
> +	unsigned int		size;
> +	void			*kmem;
> +	unsigned int		minor;
> +	struct dentry		*debugfs_dir;
> +	struct list_head	pm_reqs;
> +};
> +
>  /**
>   * struct resctrl_staged_config - parsed configuration to be applied
>   * @new_ctrl:		new ctrl value to be loaded


Reinette

Re: [PATCH v1 22/31] x86/resctrl: Make resctrl_arch_pseudo_lock_fn() take a plr

[Dave Martin]

On Mon, Apr 08, 2024 at 08:24:35PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > resctrl_arch_pseudo_lock_fn() has architecture specific behaviour,
> > and takes a struct rdtgroup as an argument.
> > 
> > After the filesystem code moves to /fs/, the definition of struct
> > rdtgroup will not be available to the architecture code.
> > 
> > The only reason resctrl_arch_pseudo_lock_fn() wants the rdtgroup is
> > for the CLOSID. Embed that in the pseudo_lock_region as a hw_closid,
> 
> Above creates expectation that the new member will be named hw_closid,
> but that is not what the code does.

I'll flag this for review, but I'd guess that this can probably just be
"closid".  I'll make a note to consider what needs to change to make
things consistent between the patch and commit message.

James might have had other ideas, connected with the remapping done for
CDP emulation causing the resctrl closid being different from the actual
value used by the hardware, at least for MPAM (see my response on
patch 24).  I don't fully understand how this works for x86 though.

So long as functionality is unaffected, and this patch is introducing no
new confusion that wasn't there beforehand, the exact name may not
matter too much(?)

Did you have other concerns here?

Cheers
---Dave

Re: [PATCH v1 22/31] x86/resctrl: Make resctrl_arch_pseudo_lock_fn() take a plr

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:38 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:24:35PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> resctrl_arch_pseudo_lock_fn() has architecture specific behaviour,
>>> and takes a struct rdtgroup as an argument.
>>>
>>> After the filesystem code moves to /fs/, the definition of struct
>>> rdtgroup will not be available to the architecture code.
>>>
>>> The only reason resctrl_arch_pseudo_lock_fn() wants the rdtgroup is
>>> for the CLOSID. Embed that in the pseudo_lock_region as a hw_closid,
>>
>> Above creates expectation that the new member will be named hw_closid,
>> but that is not what the code does.
> 
> I'll flag this for review, but I'd guess that this can probably just be
> "closid".  I'll make a note to consider what needs to change to make
> things consistent between the patch and commit message.
> 
> James might have had other ideas, connected with the remapping done for
> CDP emulation causing the resctrl closid being different from the actual
> value used by the hardware, at least for MPAM (see my response on
> patch 24).  I don't fully understand how this works for x86 though.
> 
> So long as functionality is unaffected, and this patch is introducing no
> new confusion that wasn't there beforehand, the exact name may not
> matter too much(?)

closid sounds good. It may be a good match for what is expected to be in
general/fs code. 

> 
> Did you have other concerns here?

No.

Reinette

[PATCH v1 23/31] x86/resctrl: Move thread_throttle_mode_init() to be managed by resctrl

[James Morse]

thread_throttle_mode_init() is called from the architecture specific code
to make the 'thread_throttle_mode' file visible. The architecture specific
code has already set the membw.throttle_mode in the rdt_resource.

This doesn't need to be specific to the architecture, the throttle_mode
can be used by resctrl to determine if the 'thread_throttle_mode' file
should be visible.

Call thread_throttle_mode_init() from resctrl_setup(), check the
membw.throttle_mode on the MBA resource. This avoids publishing an
extra function between the architecture and filesystem code.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c     | 1 -
 arch/x86/kernel/cpu/resctrl/internal.h | 1 -
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 9 ++++++++-
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 824a94593e9d..7cdecd8ac00b 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -227,7 +227,6 @@ static bool __get_mem_config_intel(struct rdt_resource *r)
 		r->membw.throttle_mode = THREAD_THROTTLE_PER_THREAD;
 	else
 		r->membw.throttle_mode = THREAD_THROTTLE_MAX;
-	thread_throttle_mode_init();
 
 	r->alloc_capable = true;
 
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index be4e8f31b127..e849d4407769 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -478,7 +478,6 @@ void cqm_handle_limbo(struct work_struct *work);
 bool has_busy_rmid(struct rdt_domain *d);
 void __check_limbo(struct rdt_domain *d, bool force_free);
 void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
-void __init thread_throttle_mode_init(void);
 void mbm_config_rftype_init(const char *config);
 void rdt_staged_configs_clear(void);
 bool closid_allocated(unsigned int closid);
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 9275d6f8a74e..702a94fad6db 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -2069,10 +2069,15 @@ static struct rftype *rdtgroup_get_rftype_by_name(const char *name)
 	return NULL;
 }
 
-void __init thread_throttle_mode_init(void)
+static void __init thread_throttle_mode_init(void)
 {
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
 	struct rftype *rft;
 
+	if (!r->alloc_capable ||
+	    r->membw.throttle_mode == THREAD_THROTTLE_UNDEFINED)
+		return;
+
 	rft = rdtgroup_get_rftype_by_name("thread_throttle_mode");
 	if (!rft)
 		return;
@@ -4194,6 +4199,8 @@ int __init resctrl_init(void)
 
 	rdtgroup_setup_default();
 
+	thread_throttle_mode_init();
+
 	ret = resctrl_mon_resource_init();
 	if (ret)
 		return ret;
-- 
2.39.2

[PATCH v1 24/31] x86/resctrl: Move get_config_index() to a header

[James Morse]

get_config_index() is used by the architecture specific code to map a
CLOSID+type pair to an index in the configuration arrays.

MPAM needs to do this too to preserve the ABI to user-space, there is
no reason to do it differently.

Move the helper to a header file.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 19 +++----------------
 include/linux/resctrl.h                   | 15 +++++++++++++++
 2 files changed, 18 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index 3603ade95f1d..b4627ae19291 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -277,19 +277,6 @@ static int parse_line(char *line, struct resctrl_schema *s,
 	return -EINVAL;
 }
 
-static u32 get_config_index(u32 closid, enum resctrl_conf_type type)
-{
-	switch (type) {
-	default:
-	case CDP_NONE:
-		return closid;
-	case CDP_CODE:
-		return closid * 2 + 1;
-	case CDP_DATA:
-		return closid * 2;
-	}
-}
-
 static bool apply_config(struct rdt_hw_domain *hw_dom,
 			 struct resctrl_staged_config *cfg, u32 idx,
 			 cpumask_var_t cpu_mask)
@@ -311,7 +298,7 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d);
-	u32 idx = get_config_index(closid, t);
+	u32 idx = resctrl_get_config_index(closid, t);
 	struct msr_param msr_param;
 
 	if (!cpumask_test_cpu(smp_processor_id(), &d->cpu_mask))
@@ -351,7 +338,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
 			if (!cfg->have_new_ctrl)
 				continue;
 
-			idx = get_config_index(closid, t);
+			idx = resctrl_get_config_index(closid, t);
 			if (!apply_config(hw_dom, cfg, idx, cpu_mask))
 				continue;
 
@@ -476,7 +463,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
 			    u32 closid, enum resctrl_conf_type type)
 {
 	struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d);
-	u32 idx = get_config_index(closid, type);
+	u32 idx = resctrl_get_config_index(closid, type);
 
 	return hw_dom->ctrl_val[idx];
 }
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 3de5bc63ace0..73c111963433 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -258,6 +258,21 @@ bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
 void resctrl_arch_mon_event_config_write(void *info);
 void resctrl_arch_mon_event_config_read(void *info);
 
+/* For use by arch code to remap resctrl's smaller CDP CLOSID range */
+static inline u32 resctrl_get_config_index(u32 closid,
+					   enum resctrl_conf_type type)
+{
+	switch (type) {
+	default:
+	case CDP_NONE:
+		return closid;
+	case CDP_CODE:
+			return (closid * 2) + 1;
+	case CDP_DATA:
+			return (closid * 2);
+	}
+}
+
 /*
  * Update the ctrl_val and apply this config right now.
  * Must be called on one of the domain's CPUs.
-- 
2.39.2

Re: [PATCH v1 24/31] x86/resctrl: Move get_config_index() to a header

[Reinette Chatre]

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> get_config_index() is used by the architecture specific code to map a
> CLOSID+type pair to an index in the configuration arrays.
> 
> MPAM needs to do this too to preserve the ABI to user-space, there is
> no reason to do it differently.
> 
> Move the helper to a header file.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 19 +++----------------
>  include/linux/resctrl.h                   | 15 +++++++++++++++
>  2 files changed, 18 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> index 3603ade95f1d..b4627ae19291 100644
> --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> @@ -277,19 +277,6 @@ static int parse_line(char *line, struct resctrl_schema *s,
>  	return -EINVAL;
>  }
>  
> -static u32 get_config_index(u32 closid, enum resctrl_conf_type type)
> -{
> -	switch (type) {
> -	default:
> -	case CDP_NONE:
> -		return closid;
> -	case CDP_CODE:
> -		return closid * 2 + 1;
> -	case CDP_DATA:
> -		return closid * 2;
> -	}
> -}
> -
>  static bool apply_config(struct rdt_hw_domain *hw_dom,
>  			 struct resctrl_staged_config *cfg, u32 idx,
>  			 cpumask_var_t cpu_mask)
> @@ -311,7 +298,7 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,
>  {
>  	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
>  	struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d);
> -	u32 idx = get_config_index(closid, t);
> +	u32 idx = resctrl_get_config_index(closid, t);
>  	struct msr_param msr_param;
>  
>  	if (!cpumask_test_cpu(smp_processor_id(), &d->cpu_mask))
> @@ -351,7 +338,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
>  			if (!cfg->have_new_ctrl)
>  				continue;
>  
> -			idx = get_config_index(closid, t);
> +			idx = resctrl_get_config_index(closid, t);
>  			if (!apply_config(hw_dom, cfg, idx, cpu_mask))
>  				continue;
>  
> @@ -476,7 +463,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
>  			    u32 closid, enum resctrl_conf_type type)
>  {
>  	struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d);
> -	u32 idx = get_config_index(closid, type);
> +	u32 idx = resctrl_get_config_index(closid, type);
>  
>  	return hw_dom->ctrl_val[idx];
>  }
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index 3de5bc63ace0..73c111963433 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -258,6 +258,21 @@ bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
>  void resctrl_arch_mon_event_config_write(void *info);
>  void resctrl_arch_mon_event_config_read(void *info);
>  
> +/* For use by arch code to remap resctrl's smaller CDP CLOSID range */
> +static inline u32 resctrl_get_config_index(u32 closid,
> +					   enum resctrl_conf_type type)
> +{
> +	switch (type) {
> +	default:
> +	case CDP_NONE:
> +		return closid;
> +	case CDP_CODE:
> +			return (closid * 2) + 1;
> +	case CDP_DATA:
> +			return (closid * 2);
> +	}
> +}

(please check the tabs)

This change is unexpected to me. Could you please elaborate how
MPAM's variant of CDP works?

Thank you very much.

Reinette

Re: [PATCH v1 24/31] x86/resctrl: Move get_config_index() to a header

[Dave Martin]

On Mon, Apr 08, 2024 at 08:25:26PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> > get_config_index() is used by the architecture specific code to map a
> > CLOSID+type pair to an index in the configuration arrays.
> > 
> > MPAM needs to do this too to preserve the ABI to user-space, there is
> > no reason to do it differently.
> > 
> > Move the helper to a header file.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 19 +++----------------
> >  include/linux/resctrl.h                   | 15 +++++++++++++++
> >  2 files changed, 18 insertions(+), 16 deletions(-)

[...]

> > diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> > index 3de5bc63ace0..73c111963433 100644
> > --- a/include/linux/resctrl.h
> > +++ b/include/linux/resctrl.h
> > @@ -258,6 +258,21 @@ bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
> >  void resctrl_arch_mon_event_config_write(void *info);
> >  void resctrl_arch_mon_event_config_read(void *info);
> >  
> > +/* For use by arch code to remap resctrl's smaller CDP CLOSID range */
> > +static inline u32 resctrl_get_config_index(u32 closid,
> > +					   enum resctrl_conf_type type)
> > +{
> > +	switch (type) {
> > +	default:
> > +	case CDP_NONE:
> > +		return closid;
> > +	case CDP_CODE:
> > +			return (closid * 2) + 1;
> > +	case CDP_DATA:
> > +			return (closid * 2);
> > +	}
> > +}
> 
> (please check the tabs)

Noted.  I also see that redundant parentheses seem spuriously added
compared with the original version of this moved code.  I can make a
note to drop them if you prefer.

> This change is unexpected to me. Could you please elaborate how
> MPAM's variant of CDP works?
> 
> Thank you very much.
> 
> Reinette

Note: I haven't discussed this specifically with James, so the following
is my best guess at the rationale...  With that in mind:

For MPAM, CDP isn't a special mode; instead, the PARTIDs for
instructions and data are always configured independently in the CPU.
If resctrl is not configured for CDP, we simply program the same PARTID
value both for instructions and data on task switch.

For a given resctrl control group we could pick two random unrelated
PARTIDs, but there seems to be no advantage in doing that since resctrl
enables cdp globally or not, and we would require more effort to
translate resctrl closids to PARTIDs if we didn't pair the IDs up
systematically.

(See [1], [2] in James' snapshot, which illustrate how he proposes
to do it.)


So, we may as well stick with the same scheme already established for
x86: nothing forces us to do that, but it looks simpler than the
alternatives.  I think that's the idea, anyway.

Then, if the same scheme is used by multiple arches (and 100% of the
arches currently known to resctrl), it probably makes sense to share the
definition of the mapping at least as a default for arches that don't
have their own different ways of doing it.

Does this make sense?

I can recommend adding some of this rationale to the commit message
if it helps (and assuming I'm right!)

Cheers
---Dave


[1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_resctrl.c?h=mpam/snapshot/v6.7-rc2#n206
[2] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/arch/arm64/include/asm/mpam.h?h=mpam/snapshot/v6.7-rc2#n146

Re: [PATCH v1 24/31] x86/resctrl: Move get_config_index() to a header

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:25 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:25:26PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:50 AM, James Morse wrote:
>>> get_config_index() is used by the architecture specific code to map a
>>> CLOSID+type pair to an index in the configuration arrays.
>>>
>>> MPAM needs to do this too to preserve the ABI to user-space, there is
>>> no reason to do it differently.
>>>
>>> Move the helper to a header file.
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>> ---
>>>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 19 +++----------------
>>>  include/linux/resctrl.h                   | 15 +++++++++++++++
>>>  2 files changed, 18 insertions(+), 16 deletions(-)
> 
> [...]
> 
>>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
>>> index 3de5bc63ace0..73c111963433 100644
>>> --- a/include/linux/resctrl.h
>>> +++ b/include/linux/resctrl.h
>>> @@ -258,6 +258,21 @@ bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
>>>  void resctrl_arch_mon_event_config_write(void *info);
>>>  void resctrl_arch_mon_event_config_read(void *info);
>>>  
>>> +/* For use by arch code to remap resctrl's smaller CDP CLOSID range */
>>> +static inline u32 resctrl_get_config_index(u32 closid,
>>> +					   enum resctrl_conf_type type)
>>> +{
>>> +	switch (type) {
>>> +	default:
>>> +	case CDP_NONE:
>>> +		return closid;
>>> +	case CDP_CODE:
>>> +			return (closid * 2) + 1;
>>> +	case CDP_DATA:
>>> +			return (closid * 2);
>>> +	}
>>> +}
>>
>> (please check the tabs)
> 
> Noted.  I also see that redundant parentheses seem spuriously added
> compared with the original version of this moved code.  I can make a
> note to drop them if you prefer.
> 
>> This change is unexpected to me. Could you please elaborate how
>> MPAM's variant of CDP works?
>>
>> Thank you very much.
>>
>> Reinette
> 
> Note: I haven't discussed this specifically with James, so the following
> is my best guess at the rationale...  With that in mind:
> 
> For MPAM, CDP isn't a special mode; instead, the PARTIDs for
> instructions and data are always configured independently in the CPU.
> If resctrl is not configured for CDP, we simply program the same PARTID
> value both for instructions and data on task switch.
> 
> For a given resctrl control group we could pick two random unrelated
> PARTIDs, but there seems to be no advantage in doing that since resctrl
> enables cdp globally or not, and we would require more effort to
> translate resctrl closids to PARTIDs if we didn't pair the IDs up
> systematically.
> 
> (See [1], [2] in James' snapshot, which illustrate how he proposes
> to do it.)
> 
> 
> So, we may as well stick with the same scheme already established for
> x86: nothing forces us to do that, but it looks simpler than the
> alternatives.  I think that's the idea, anyway.
> 
> Then, if the same scheme is used by multiple arches (and 100% of the
> arches currently known to resctrl), it probably makes sense to share the
> definition of the mapping at least as a default for arches that don't
> have their own different ways of doing it.
> 
> Does this make sense?

It does, thank you very much.

> 
> I can recommend adding some of this rationale to the commit message
> if it helps (and assuming I'm right!)

Sounds good, thank you.

Reinette

Re: [PATCH v1 24/31] x86/resctrl: Move get_config_index() to a header

[James Morse]

Hi guys,

On 11/04/2024 18:41, Reinette Chatre wrote:
> On 4/11/2024 7:25 AM, Dave Martin wrote:
>> On Mon, Apr 08, 2024 at 08:25:26PM -0700, Reinette Chatre wrote:
>>> On 3/21/2024 9:50 AM, James Morse wrote:
>>>> get_config_index() is used by the architecture specific code to map a
>>>> CLOSID+type pair to an index in the configuration arrays.
>>>>
>>>> MPAM needs to do this too to preserve the ABI to user-space, there is
>>>> no reason to do it differently.
>>>>
>>>> Move the helper to a header file.

>>>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
>>>> index 3de5bc63ace0..73c111963433 100644
>>>> --- a/include/linux/resctrl.h
>>>> +++ b/include/linux/resctrl.h
>>>> @@ -258,6 +258,21 @@ bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
>>>>  void resctrl_arch_mon_event_config_write(void *info);
>>>>  void resctrl_arch_mon_event_config_read(void *info);
>>>>  
>>>> +/* For use by arch code to remap resctrl's smaller CDP CLOSID range */
>>>> +static inline u32 resctrl_get_config_index(u32 closid,
>>>> +					   enum resctrl_conf_type type)
>>>> +{
>>>> +	switch (type) {
>>>> +	default:
>>>> +	case CDP_NONE:
>>>> +		return closid;
>>>> +	case CDP_CODE:
>>>> +			return (closid * 2) + 1;
>>>> +	case CDP_DATA:
>>>> +			return (closid * 2);
>>>> +	}
>>>> +}
>>>
>>> (please check the tabs)
>>
>> Noted.  I also see that redundant parentheses seem spuriously added
>> compared with the original version of this moved code.  I can make a
>> note to drop them if you prefer.
>>
>>> This change is unexpected to me. Could you please elaborate how
>>> MPAM's variant of CDP works?
>>>
>>> Thank you very much.
>>>
>>> Reinette
>>
>> Note: I haven't discussed this specifically with James, so the following
>> is my best guess at the rationale...  With that in mind:
>>
>> For MPAM, CDP isn't a special mode; instead, the PARTIDs for
>> instructions and data are always configured independently in the CPU.
>> If resctrl is not configured for CDP, we simply program the same PARTID
>> value both for instructions and data on task switch.
>>
>> For a given resctrl control group we could pick two random unrelated
>> PARTIDs, but there seems to be no advantage in doing that since resctrl
>> enables cdp globally or not, and we would require more effort to
>> translate resctrl closids to PARTIDs if we didn't pair the IDs up
>> systematically.
>>
>> (See [1], [2] in James' snapshot, which illustrate how he proposes
>> to do it.)
>>
>>
>> So, we may as well stick with the same scheme already established for
>> x86: nothing forces us to do that, but it looks simpler than the
>> alternatives.  I think that's the idea, anyway.
>>
>> Then, if the same scheme is used by multiple arches (and 100% of the
>> arches currently known to resctrl), it probably makes sense to share the
>> definition of the mapping at least as a default for arches that don't
>> have their own different ways of doing it.
>>
>> Does this make sense?
> 
> It does, thank you very much.

Thanks for the summary Dave - spot on!

There are some additional headaches for MPAM to provide the counters when CDP emulation is
enabled: because two CLOSID/PARTID are in use, two counters have to be allocated and read,
and their counters summed. This is all done behind the scenes in the MPAM driver.

MBA gets even more fun - as there is no MBA_CODE/MBA_DATA - but whatever MPAM is using to
back the MBA resource sees two CLOSID/PARTID. The result is the MPAM driver has to replay
configuration changes to the second CLOSID/PARTID. Again, this is done behind the scenes
in the MPAM driver.


(and as we're on this topic: I've had requests to make the counters available for code and
data separately - I don't intend to do this in resctrl as it wouldn't be portable to x86.
I'm looking at doing this with the perf pmu stuff)


Thanks,

James

[PATCH v1 25/31] x86/resctrl: Claim get_domain_from_cpu() for resctrl

[James Morse]

get_domain_from_cpu() is a handy helper that both the arch code and
resctrl need to use. Rename it resctrl_get_domain_from_cpu() so it
gets moved out to /fs, and exported back to the arch code.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/kernel/cpu/resctrl/core.c     | 15 +--------------
 arch/x86/kernel/cpu/resctrl/internal.h |  1 -
 arch/x86/kernel/cpu/resctrl/monitor.c  |  2 +-
 arch/x86/kernel/cpu/resctrl/rdtgroup.c |  2 +-
 include/linux/resctrl.h                | 19 +++++++++++++++++++
 5 files changed, 22 insertions(+), 17 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 7cdecd8ac00b..f94ad04023c3 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -355,19 +355,6 @@ cat_wrmsr(struct rdt_domain *d, struct msr_param *m, struct rdt_resource *r)
 		wrmsrl(hw_res->msr_base + i, hw_dom->ctrl_val[i]);
 }
 
-struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r)
-{
-	struct rdt_domain *d;
-
-	list_for_each_entry(d, &r->domains, list) {
-		/* Find the domain that contains this CPU */
-		if (cpumask_test_cpu(cpu, &d->cpu_mask))
-			return d;
-	}
-
-	return NULL;
-}
-
 u32 resctrl_arch_get_num_closid(struct rdt_resource *r)
 {
 	return resctrl_to_arch_res(r)->num_closid;
@@ -381,7 +368,7 @@ void rdt_ctrl_update(void *arg)
 	int cpu = smp_processor_id();
 	struct rdt_domain *d;
 
-	d = get_domain_from_cpu(cpu, r);
+	d = resctrl_get_domain_from_cpu(cpu, r);
 	if (d) {
 		hw_res->msr_update(d, m, r);
 		return;
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e849d4407769..3a3962736061 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -452,7 +452,6 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d,
 				  unsigned long cbm);
 enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);
 int rdtgroup_tasks_assigned(struct rdtgroup *r);
-struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);
 int closids_supported(void);
 void closid_free(int closid);
 int alloc_rmid(u32 closid);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 287fb0a5f060..8b316d9acc3b 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -676,7 +676,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
 	idx = resctrl_arch_rmid_idx_encode(closid, rmid);
 	pmbm_data = &dom_mbm->mbm_local[idx];
 
-	dom_mba = get_domain_from_cpu(smp_processor_id(), r_mba);
+	dom_mba = resctrl_get_domain_from_cpu(smp_processor_id(), r_mba);
 	if (!dom_mba) {
 		pr_warn_once("Failure to get domain for MBA update\n");
 		return;
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 702a94fad6db..085fb9c2333a 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -4165,7 +4165,7 @@ void resctrl_offline_cpu(unsigned int cpu)
 	if (!l3->mon_capable)
 		goto out_unlock;
 
-	d = get_domain_from_cpu(cpu, l3);
+	d = resctrl_get_domain_from_cpu(cpu, l3);
 	if (d) {
 		if (resctrl_is_mbm_enabled() && cpu == d->mbm_work_cpu) {
 			cancel_delayed_work(&d->mbm_over);
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 73c111963433..84420253dc05 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -2,6 +2,7 @@
 #ifndef _RESCTRL_H
 #define _RESCTRL_H
 
+#include <linux/cpu.h>
 #include <linux/kernel.h>
 #include <linux/list.h>
 #include <linux/pid.h>
@@ -273,6 +274,24 @@ static inline u32 resctrl_get_config_index(u32 closid,
 	}
 }
 
+/*
+ * Caller must hold the cpuhp read lock to prevent the struct rdt_domain being
+ * freed.
+ */
+static inline struct rdt_domain *
+resctrl_get_domain_from_cpu(int cpu, struct rdt_resource *r)
+{
+	struct rdt_domain *d;
+
+	list_for_each_entry_rcu(d, &r->domains, list) {
+		/* Find the domain that contains this CPU */
+		if (cpumask_test_cpu(cpu, &d->cpu_mask))
+			return d;
+	}
+
+	return NULL;
+}
+
 /*
  * Update the ctrl_val and apply this config right now.
  * Must be called on one of the domain's CPUs.
-- 
2.39.2

[PATCH v1 26/31] x86/resctrl: Describe resctrl's bitmap size assumptions

[James Morse]

resctrl operates on configuration bitmaps and a bitmap of allocated
CLOSID, both are stored in a u32.

MPAM supports configuration/portion bitmaps and PARTIDs larger
than will fit in a u32.

Add some preprocessor values that make it clear why MPAM clamps
some of these values. This will make it easier to find code related
to these values if this resctrl behaviour ever changes.

Signed-off-by: James Morse <james.morse@arm.com>
---
 include/linux/resctrl.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 84420253dc05..f463fb949677 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -26,6 +26,17 @@ int proc_resctrl_show(struct seq_file *m,
 /* max value for struct rdt_domain's mbps_val */
 #define MBA_MAX_MBPS   U32_MAX
 
+/*
+ * Resctrl uses a u32 as a closid bitmap. The maximum closid is 32.
+ */
+#define RESCTRL_MAX_CLOSID		32
+
+/*
+ * Resctrl uses u32 to hold the user-space config. The maximum bitmap size is
+ * 32.
+ */
+#define RESCTRL_MAX_CBM			32
+
 /**
  * struct pseudo_lock_region - pseudo-lock region information
  * @s:			Resctrl schema for the resource to which this
-- 
2.39.2

[PATCH v1 27/31] x86/resctrl: Rename resctrl_sched_in() to begin resctrl_arch_

[James Morse]

resctrl_sched_in() loads the architecture specific CPU MSRs with the
CLOSID and RMID values. This function was named before resctrl was
split to have architecture specific code, and generic filesystem code.

This function is obviously architecture specific, but does not begin
with 'resctrl_arch_', making it the odd one out in the functions an
architecture needs to support to enable resctrl.

Rename it for concistency. This is purely cosmetic.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/include/asm/resctrl.h         |  4 ++--
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++++++------
 arch/x86/kernel/process_32.c           |  2 +-
 arch/x86/kernel/process_64.c           |  2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 9940398e367e..491342f56811 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -177,7 +177,7 @@ static inline bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 ignored,
 	return READ_ONCE(tsk->rmid) == rmid;
 }
 
-static inline void resctrl_sched_in(struct task_struct *tsk)
+static inline void resctrl_arch_sched_in(struct task_struct *tsk)
 {
 	if (static_branch_likely(&rdt_enable_key))
 		__resctrl_sched_in(tsk);
@@ -220,7 +220,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c);
 
 #else
 
-static inline void resctrl_sched_in(struct task_struct *tsk) {}
+static inline void resctrl_arch_sched_in(struct task_struct *tsk) {}
 static inline void resctrl_cpu_detect(struct cpuinfo_x86 *c) {}
 
 #endif /* CONFIG_X86_CPU_RESCTRL */
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 085fb9c2333a..218aebd6387f 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -359,7 +359,7 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
 }
 
 /*
- * This is safe against resctrl_sched_in() called from __switch_to()
+ * This is safe against resctrl_arch_sched_in() called from __switch_to()
  * because __switch_to() is executed with interrupts disabled. A local call
  * from update_closid_rmid() is protected against __switch_to() because
  * preemption is disabled.
@@ -378,7 +378,7 @@ void resctrl_arch_sync_cpu_defaults(void *info)
 	 * executing task might have its own closid selected. Just reuse
 	 * the context switch code.
 	 */
-	resctrl_sched_in(current);
+	resctrl_arch_sched_in(current);
 }
 
 /*
@@ -605,7 +605,7 @@ static void _update_task_closid_rmid(void *task)
 	 * Otherwise, the MSR is updated when the task is scheduled in.
 	 */
 	if (task == current)
-		resctrl_sched_in(task);
+		resctrl_arch_sched_in(task);
 }
 
 static void update_task_closid_rmid(struct task_struct *t)
@@ -663,7 +663,7 @@ static int __rdtgroup_move_task(struct task_struct *tsk,
 	 * Ensure the task's closid and rmid are written before determining if
 	 * the task is current that will decide if it will be interrupted.
 	 * This pairs with the full barrier between the rq->curr update and
-	 * resctrl_sched_in() during context switch.
+	 * resctrl_arch_sched_in() during context switch.
 	 */
 	smp_mb();
 
@@ -2946,8 +2946,8 @@ static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,
 			/*
 			 * Order the closid/rmid stores above before the loads
 			 * in task_curr(). This pairs with the full barrier
-			 * between the rq->curr update and resctrl_sched_in()
-			 * during context switch.
+			 * between the rq->curr update and
+			 * resctrl_arch_sched_in() during context switch.
 			 */
 			smp_mb();
 
diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c
index 0917c7f25720..8697b02dabf1 100644
--- a/arch/x86/kernel/process_32.c
+++ b/arch/x86/kernel/process_32.c
@@ -211,7 +211,7 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
 	switch_fpu_finish(next_p);
 
 	/* Load the Intel cache allocation PQR MSR. */
-	resctrl_sched_in(next_p);
+	resctrl_arch_sched_in(next_p);
 
 	return prev_p;
 }
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index 7062b84dd467..d1cf885e8930 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -707,7 +707,7 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
 	}
 
 	/* Load the Intel cache allocation PQR MSR. */
-	resctrl_sched_in(next_p);
+	resctrl_arch_sched_in(next_p);
 
 	return prev_p;
 }
-- 
2.39.2

Re: [PATCH v1 27/31] x86/resctrl: Rename resctrl_sched_in() to begin resctrl_arch_

[Reinette Chatre]

Hi James,

In subject, did you intend to say "to begin with"?

On 3/21/2024 9:51 AM, James Morse wrote:
> resctrl_sched_in() loads the architecture specific CPU MSRs with the
> CLOSID and RMID values. This function was named before resctrl was
> split to have architecture specific code, and generic filesystem code.
> 
> This function is obviously architecture specific, but does not begin
> with 'resctrl_arch_', making it the odd one out in the functions an
> architecture needs to support to enable resctrl.
> 
> Rename it for concistency. This is purely cosmetic.

concistency -> consistency

Reinette

Re: [PATCH v1 27/31] x86/resctrl: Rename resctrl_sched_in() to begin resctrl_arch_

[Dave Martin]

On Mon, Apr 08, 2024 at 08:26:04PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> In subject, did you intend to say "to begin with"?

I'd say the subject is likely to be abbreviated rather than incorrect,
but it certainly does no harm to clarify it.  I'll make a note.

> On 3/21/2024 9:51 AM, James Morse wrote:
> > resctrl_sched_in() loads the architecture specific CPU MSRs with the
> > CLOSID and RMID values. This function was named before resctrl was
> > split to have architecture specific code, and generic filesystem code.
> > 
> > This function is obviously architecture specific, but does not begin
> > with 'resctrl_arch_', making it the odd one out in the functions an
> > architecture needs to support to enable resctrl.
> > 
> > Rename it for concistency. This is purely cosmetic.
> 
> concistency -> consistency

Noted.

Cheers
---Dave

Re: [PATCH v1 27/31] x86/resctrl: Rename resctrl_sched_in() to begin resctrl_arch_

[Moger, Babu]

On 3/21/24 11:51, James Morse wrote:
> resctrl_sched_in() loads the architecture specific CPU MSRs with the
> CLOSID and RMID values. This function was named before resctrl was
> split to have architecture specific code, and generic filesystem code.
> 
> This function is obviously architecture specific, but does not begin
> with 'resctrl_arch_', making it the odd one out in the functions an
> architecture needs to support to enable resctrl.
> 
> Rename it for concistency. This is purely cosmetic.

s/concistency/consistency

> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/include/asm/resctrl.h         |  4 ++--
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++++++------
>  arch/x86/kernel/process_32.c           |  2 +-
>  arch/x86/kernel/process_64.c           |  2 +-
>  4 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
> index 9940398e367e..491342f56811 100644
> --- a/arch/x86/include/asm/resctrl.h
> +++ b/arch/x86/include/asm/resctrl.h
> @@ -177,7 +177,7 @@ static inline bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 ignored,
>  	return READ_ONCE(tsk->rmid) == rmid;
>  }
>  
> -static inline void resctrl_sched_in(struct task_struct *tsk)
> +static inline void resctrl_arch_sched_in(struct task_struct *tsk)
>  {
>  	if (static_branch_likely(&rdt_enable_key))
>  		__resctrl_sched_in(tsk);
> @@ -220,7 +220,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c);
>  
>  #else
>  
> -static inline void resctrl_sched_in(struct task_struct *tsk) {}
> +static inline void resctrl_arch_sched_in(struct task_struct *tsk) {}
>  static inline void resctrl_cpu_detect(struct cpuinfo_x86 *c) {}
>  
>  #endif /* CONFIG_X86_CPU_RESCTRL */
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 085fb9c2333a..218aebd6387f 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -359,7 +359,7 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
>  }
>  
>  /*
> - * This is safe against resctrl_sched_in() called from __switch_to()
> + * This is safe against resctrl_arch_sched_in() called from __switch_to()
>   * because __switch_to() is executed with interrupts disabled. A local call
>   * from update_closid_rmid() is protected against __switch_to() because
>   * preemption is disabled.
> @@ -378,7 +378,7 @@ void resctrl_arch_sync_cpu_defaults(void *info)
>  	 * executing task might have its own closid selected. Just reuse
>  	 * the context switch code.
>  	 */
> -	resctrl_sched_in(current);
> +	resctrl_arch_sched_in(current);
>  }
>  
>  /*
> @@ -605,7 +605,7 @@ static void _update_task_closid_rmid(void *task)
>  	 * Otherwise, the MSR is updated when the task is scheduled in.
>  	 */
>  	if (task == current)
> -		resctrl_sched_in(task);
> +		resctrl_arch_sched_in(task);
>  }
>  
>  static void update_task_closid_rmid(struct task_struct *t)
> @@ -663,7 +663,7 @@ static int __rdtgroup_move_task(struct task_struct *tsk,
>  	 * Ensure the task's closid and rmid are written before determining if
>  	 * the task is current that will decide if it will be interrupted.
>  	 * This pairs with the full barrier between the rq->curr update and
> -	 * resctrl_sched_in() during context switch.
> +	 * resctrl_arch_sched_in() during context switch.
>  	 */
>  	smp_mb();
>  
> @@ -2946,8 +2946,8 @@ static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,
>  			/*
>  			 * Order the closid/rmid stores above before the loads
>  			 * in task_curr(). This pairs with the full barrier
> -			 * between the rq->curr update and resctrl_sched_in()
> -			 * during context switch.
> +			 * between the rq->curr update and
> +			 * resctrl_arch_sched_in() during context switch.
>  			 */
>  			smp_mb();
>  
> diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c
> index 0917c7f25720..8697b02dabf1 100644
> --- a/arch/x86/kernel/process_32.c
> +++ b/arch/x86/kernel/process_32.c
> @@ -211,7 +211,7 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
>  	switch_fpu_finish(next_p);
>  
>  	/* Load the Intel cache allocation PQR MSR. */
> -	resctrl_sched_in(next_p);
> +	resctrl_arch_sched_in(next_p);
>  
>  	return prev_p;
>  }
> diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
> index 7062b84dd467..d1cf885e8930 100644
> --- a/arch/x86/kernel/process_64.c
> +++ b/arch/x86/kernel/process_64.c
> @@ -707,7 +707,7 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
>  	}
>  
>  	/* Load the Intel cache allocation PQR MSR. */
> -	resctrl_sched_in(next_p);
> +	resctrl_arch_sched_in(next_p);
>  
>  	return prev_p;
>  }

-- 
Thanks
Babu Moger

[PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[James Morse]

Because ARM's MPAM controls are probed using MMIO, resctrl can't be
initialised until enough CPUs are online to have determined the
system-wide supported num_closid. Arm64 also supports 'late onlined
secondaries', where only a subset of CPUs are online during boot.

These two combine to mean the MPAM driver may not be able to initialise
resctrl until user-space has brought 'enough' CPUs online.

To allow MPAM to initialise resctrl after __init text has been free'd,
remove all the __init markings from resctrl.

The existing __exit markings cause these functions to be removed by the
linker as it has never been possible to build resctrl as a module. MPAM
has an error interrupt which causes the driver to reset and disable
itself. Remove the __exit markings to allow the MPAM driver to tear down
resctrl when an error occurs.

Signed-off-by: James Morse <james.morse@arm.com>
---
If 'late onlined secondaries' is an alien concept, I can add a worked
example to the commit message:
If a system has two L3 caches, but during boot only CPU-0 is online,
then no CPU is able to probe the features of the second L3 cache.
It's not until user-space brings other CPUs online that the MPAM
driver can finally get a glimpse of all the hardware to determine
what properties the system has.
---
 arch/x86/kernel/cpu/resctrl/internal.h | 2 +-
 arch/x86/kernel/cpu/resctrl/monitor.c  | 4 ++--
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++----
 include/linux/resctrl.h                | 4 ++--
 4 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 3a3962736061..56218193a8ba 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -457,7 +457,7 @@ void closid_free(int closid);
 int alloc_rmid(u32 closid);
 void free_rmid(u32 closid, u32 rmid);
 int rdt_get_mon_l3_config(struct rdt_resource *r);
-void __exit resctrl_mon_resource_exit(void);
+void resctrl_mon_resource_exit(void);
 bool rdt_cpu_has(int flag);
 void mon_event_count(void *info);
 int rdtgroup_mondata_show(struct seq_file *m, void *arg);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 8b316d9acc3b..7e6fca138cb7 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -954,7 +954,7 @@ static int dom_data_init(struct rdt_resource *r)
 	return err;
 }
 
-static void __exit dom_data_exit(struct rdt_resource *r)
+static void dom_data_exit(struct rdt_resource *r)
 {
 	if (!r->mon_capable)
 		return;
@@ -1076,7 +1076,7 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 	return 0;
 }
 
-void __exit resctrl_mon_resource_exit(void)
+void resctrl_mon_resource_exit(void)
 {
 	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
 
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 218aebd6387f..1425a33d201d 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -2069,7 +2069,7 @@ static struct rftype *rdtgroup_get_rftype_by_name(const char *name)
 	return NULL;
 }
 
-static void __init thread_throttle_mode_init(void)
+static void thread_throttle_mode_init(void)
 {
 	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
 	struct rftype *rft;
@@ -3997,7 +3997,7 @@ static void rdtgroup_destroy_root(void)
 	rdtgroup_default.kn = NULL;
 }
 
-static void __init rdtgroup_setup_default(void)
+static void rdtgroup_setup_default(void)
 {
 	mutex_lock(&rdtgroup_mutex);
 
@@ -4190,7 +4190,7 @@ void resctrl_offline_cpu(unsigned int cpu)
  *
  * Return: 0 on success or -errno
  */
-int __init resctrl_init(void)
+int resctrl_init(void)
 {
 	int ret = 0;
 
@@ -4244,7 +4244,7 @@ int __init resctrl_init(void)
 	return ret;
 }
 
-void __exit resctrl_exit(void)
+void resctrl_exit(void)
 {
 	debugfs_remove_recursive(debugfs_resctrl);
 	unregister_filesystem(&rdt_fs_type);
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index f463fb949677..5da55e58f229 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -393,7 +393,7 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_domain *d);
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-int __init resctrl_init(void);
-void __exit resctrl_exit(void);
+int resctrl_init(void);
+void resctrl_exit(void);
 
 #endif /* _RESCTRL_H */
-- 
2.39.2

Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Reinette Chatre]

Hi James,

On 3/21/2024 9:51 AM, James Morse wrote:
> Because ARM's MPAM controls are probed using MMIO, resctrl can't be
> initialised until enough CPUs are online to have determined the
> system-wide supported num_closid. Arm64 also supports 'late onlined
> secondaries', where only a subset of CPUs are online during boot.
> 
> These two combine to mean the MPAM driver may not be able to initialise
> resctrl until user-space has brought 'enough' CPUs online.
> 
> To allow MPAM to initialise resctrl after __init text has been free'd,
> remove all the __init markings from resctrl.
> 
> The existing __exit markings cause these functions to be removed by the
> linker as it has never been possible to build resctrl as a module. MPAM
> has an error interrupt which causes the driver to reset and disable
> itself. Remove the __exit markings to allow the MPAM driver to tear down
> resctrl when an error occurs.

Obviously for the reasons you state this code has never been exercised.
Were you able to test this error interrupt flow yet?

Reinette

Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Dave Martin]

On Mon, Apr 08, 2024 at 08:32:36PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:51 AM, James Morse wrote:
> > Because ARM's MPAM controls are probed using MMIO, resctrl can't be
> > initialised until enough CPUs are online to have determined the
> > system-wide supported num_closid. Arm64 also supports 'late onlined
> > secondaries', where only a subset of CPUs are online during boot.
> > 
> > These two combine to mean the MPAM driver may not be able to initialise
> > resctrl until user-space has brought 'enough' CPUs online.
> > 
> > To allow MPAM to initialise resctrl after __init text has been free'd,
> > remove all the __init markings from resctrl.
> > 
> > The existing __exit markings cause these functions to be removed by the
> > linker as it has never been possible to build resctrl as a module. MPAM
> > has an error interrupt which causes the driver to reset and disable
> > itself. Remove the __exit markings to allow the MPAM driver to tear down
> > resctrl when an error occurs.
> 
> Obviously for the reasons you state this code has never been exercised.
> Were you able to test this error interrupt flow yet?
> 
> Reinette
> 

I think this will have to wait for James to respond.

There is code to tear down resctrl in response to an MPAM error interrupt,
but I don't know how it has been exercised so far (if at all).

https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_devices.c?h=mpam/snapshot/v6.7-rc2#n1940


Cheers
---Dave

Re: [EXTERNAL] Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Amit Singh Tomar]

Hi Dave, Reinette,

> On Mon, Apr 08, 2024 at 08:32:36PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:51 AM, James Morse wrote:
>>> Because ARM's MPAM controls are probed using MMIO, resctrl can't be
>>> initialised until enough CPUs are online to have determined the
>>> system-wide supported num_closid. Arm64 also supports 'late onlined
>>> secondaries', where only a subset of CPUs are online during boot.
>>>
>>> These two combine to mean the MPAM driver may not be able to initialise
>>> resctrl until user-space has brought 'enough' CPUs online.
>>>
>>> To allow MPAM to initialise resctrl after __init text has been free'd,
>>> remove all the __init markings from resctrl.
>>>
>>> The existing __exit markings cause these functions to be removed by the
>>> linker as it has never been possible to build resctrl as a module. MPAM
>>> has an error interrupt which causes the driver to reset and disable
>>> itself. Remove the __exit markings to allow the MPAM driver to tear down
>>> resctrl when an error occurs.
>>
>> Obviously for the reasons you state this code has never been exercised.
>> Were you able to test this error interrupt flow yet?
>>
>> Reinette
>>
> 
> I think this will have to wait for James to respond.
> 
> There is code to tear down resctrl in response to an MPAM error interrupt,
> but I don't know how it has been exercised so far (if at all).

We are managed to test the MPAM error interrupt (on the platform that 
supports MPAM interrupts on software errors). For instance programming
more resource control groups (part IDs) than available, and It appears 
to correctly remove the "resctrl" mount point (though mount command 
still shows resctrl on /sys/fs/resctrl type resctrl (rw,relatime)
), but

# mount -t resctrl resctrl /sys/fs/resctrl
mount: /sys/fs/resctrl: mount point does not exist.

Additionally, a question regarding this, Is a complete system restart 
necessary to regain the mount?

Thanks
-Amit

Re: [EXTERNAL] Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Dave Martin]

On Thu, Apr 11, 2024 at 09:21:38PM +0530, Amit Singh Tomar wrote:
> Hi Dave, Reinette,
> 
> > On Mon, Apr 08, 2024 at 08:32:36PM -0700, Reinette Chatre wrote:
> > > Hi James,
> > > 
> > > On 3/21/2024 9:51 AM, James Morse wrote:
> > > > Because ARM's MPAM controls are probed using MMIO, resctrl can't be
> > > > initialised until enough CPUs are online to have determined the
> > > > system-wide supported num_closid. Arm64 also supports 'late onlined
> > > > secondaries', where only a subset of CPUs are online during boot.
> > > > 
> > > > These two combine to mean the MPAM driver may not be able to initialise
> > > > resctrl until user-space has brought 'enough' CPUs online.
> > > > 
> > > > To allow MPAM to initialise resctrl after __init text has been free'd,
> > > > remove all the __init markings from resctrl.
> > > > 
> > > > The existing __exit markings cause these functions to be removed by the
> > > > linker as it has never been possible to build resctrl as a module. MPAM
> > > > has an error interrupt which causes the driver to reset and disable
> > > > itself. Remove the __exit markings to allow the MPAM driver to tear down
> > > > resctrl when an error occurs.
> > > 
> > > Obviously for the reasons you state this code has never been exercised.
> > > Were you able to test this error interrupt flow yet?
> > > 
> > > Reinette
> > > 
> > 
> > I think this will have to wait for James to respond.
> > 
> > There is code to tear down resctrl in response to an MPAM error interrupt,
> > but I don't know how it has been exercised so far (if at all).
> 
> We are managed to test the MPAM error interrupt (on the platform that
> supports MPAM interrupts on software errors). For instance programming
> more resource control groups (part IDs) than available, and It appears to
> correctly remove the "resctrl" mount point (though mount command still shows
> resctrl on /sys/fs/resctrl type resctrl (rw,relatime)
> ), but

Thanks for trying this out!

Is it possible to unmount resctrl once the system is in this state?

> # mount -t resctrl resctrl /sys/fs/resctrl
> mount: /sys/fs/resctrl: mount point does not exist.

What if you now try to mount resctrl somewhere else, e.g.:

# mount -t resctrl resctrl /mnt

I'm guessing this _should_ fail if you weren't able to unmount resctrl,
since resctrl seems to forbid multiple mount instances.

I'm not sure what the best behaviour is here.  Leaving resctrl "half-
mounted" might be a good thing: at this point the system is in a semi-
bad state we want to make sure it can't be remounted.  Unregistering the
resctrl filesystem from the fs core feels cleaner if feasible though.

Leaving an impossible unmount operation for init to do during reboot/
shutdown feels unfortunate.

We might have to look at what other filesystems do in this area.

The mount machinery does provide other ways of getting into broken,
impossible situations from userspace, so this doesn't feel like an
entirely new problem.

> 
> Additionally, a question regarding this, Is a complete system restart
> necessary to regain the mount?
> 
> Thanks
> -Amit

I think James will need to comment on this, but I think that yes, it
is probably appropriate to require a reboot.  I think an MPAM error
interrupt should only happen if the software did something wrong, so
it's a bit like hitting a BUG(): we don't promise that everything works
100% properly until the system is restarted.  Misbehaviour should be
contained to MPAM though.

Cheers
---Dave

Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Amit Singh Tomar]

Hi Dave,

> Is it possible to unmount resctrl once the system is in this state?
No, it can't be unmounted, as there is no mount exist.

>> # mount -t resctrl resctrl /sys/fs/resctrl
>> mount: /sys/fs/resctrl: mount point does not exist.
> 
> What if you now try to mount resctrl somewhere else, e.g.:
> 
> # mount -t resctrl resctrl /mnt
root@localhost:~# mount -t resctrl resctrl /test
mount: /test: unknown filesystem type 'resctrl'.

> 
> I'm guessing this _should_ fail if you weren't able to unmount resctrl,
> since resctrl seems to forbid multiple mount instances.
> 
> I'm not sure what the best behaviour is here.  Leaving resctrl "half-
> mounted" might be a good thing: at this point the system is in a semi-
> bad state we want to make sure it can't be remounted.  Unregistering the
> resctrl filesystem from the fs core feels cleaner if feasible though.
> 
> Leaving an impossible unmount operation for init to do during reboot/
> shutdown feels unfortunate.
> 
> We might have to look at what other filesystems do in this area.
> 
> The mount machinery does provide other ways of getting into broken,
> impossible situations from userspace, so this doesn't feel like an
> entirely new problem.
> 
>>
>> Additionally, a question regarding this, Is a complete system restart
>> necessary to regain the mount?
>>
>> Thanks
>> -Amit
> 
> I think James will need to comment on this, but I think that yes, it
> is probably appropriate to require a reboot.  I think an MPAM error
> interrupt should only happen if the software did something wrong, so
> it's a bit like hitting a BUG(): we don't promise that everything works
> 100% properly until the system is restarted.  Misbehaviour should be
> contained to MPAM though.
> 
if "resctrl" is nonfunctional in this state, then this comment[1] here 
does *not* make sense.

"restore any modified controls to their reset values."

Thanks
-Amit

[1]: 
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_devices.c?h=mpam/snapshot/v6.7-rc2#n2228

root@localhost:~# mount
tmpfs on /run/user/0 type tmpfs 
(rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
resctrl on /sys/fs/resctrl type resctrl (rw,relatime)

root@localhost:~# devmem msc_addr 32 0x9999
[  687.096276] mpam: error irq from msc:1 'PARTID_SEL_Range', 
partid:39321, pmg: 0, ris: 0

root@localhost:~# mount
tmpfs on /run/user/0 type tmpfs 
(rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
resctrl on /sys/fs/resctrl type resctrl (rw,relatime)

root@localhost:~# umount resctrl
umount: /sys/fs/resctrl: no mount point specified.

root@localhost:~# mount
tmpfs on /run/user/0 type tmpfs 
(rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)

root@localhost:~# mount -t resctrl resctrl /test
mount: /test: unknown filesystem type 'resctrl'.

Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Dave Martin]

On Tue, Apr 30, 2024 at 12:43:05PM +0530, Amit Singh Tomar wrote:
> Hi Dave,
> 
> > Is it possible to unmount resctrl once the system is in this state?
> No, it can't be unmounted, as there is no mount exist.

I see.


> > > # mount -t resctrl resctrl /sys/fs/resctrl
> > > mount: /sys/fs/resctrl: mount point does not exist.
> > 
> > What if you now try to mount resctrl somewhere else, e.g.:
> > 
> > # mount -t resctrl resctrl /mnt
> root@localhost:~# mount -t resctrl resctrl /test
> mount: /test: unknown filesystem type 'resctrl'.

Oh, right, so the resctrl filesystem gets unregistered in that
case...

> > 
> > I'm guessing this _should_ fail if you weren't able to unmount resctrl,
> > since resctrl seems to forbid multiple mount instances.
> > 
> > I'm not sure what the best behaviour is here.  Leaving resctrl "half-
> > mounted" might be a good thing: at this point the system is in a semi-
> > bad state we want to make sure it can't be remounted.  Unregistering the
> > resctrl filesystem from the fs core feels cleaner if feasible though.
> > 
> > Leaving an impossible unmount operation for init to do during reboot/
> > shutdown feels unfortunate.
> > 
> > We might have to look at what other filesystems do in this area.
> > 
> > The mount machinery does provide other ways of getting into broken,
> > impossible situations from userspace, so this doesn't feel like an
> > entirely new problem.
> > 
> > > 
> > > Additionally, a question regarding this, Is a complete system restart
> > > necessary to regain the mount?
> > > 
> > > Thanks
> > > -Amit
> > 
> > I think James will need to comment on this, but I think that yes, it
> > is probably appropriate to require a reboot.  I think an MPAM error
> > interrupt should only happen if the software did something wrong, so
> > it's a bit like hitting a BUG(): we don't promise that everything works
> > 100% properly until the system is restarted.  Misbehaviour should be
> > contained to MPAM though.
> > 
> if "resctrl" is nonfunctional in this state, then this comment[1] here does
> *not* make sense.
> 
> "restore any modified controls to their reset values."

Can you clarify what you mean here?

I think it makes sense to clean up the MPAM hardware as well as we can
in these situations, even if we can't be certain what went wrong.

[final comments below]

> Thanks
> -Amit
> 
> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_devices.c?h=mpam/snapshot/v6.7-rc2#n2228
> 
> root@localhost:~# mount
> tmpfs on /run/user/0 type tmpfs
> (rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
> resctrl on /sys/fs/resctrl type resctrl (rw,relatime)
> 
> root@localhost:~# devmem msc_addr 32 0x9999
> [  687.096276] mpam: error irq from msc:1 'PARTID_SEL_Range', partid:39321,
> pmg: 0, ris: 0
> 
> root@localhost:~# mount
> tmpfs on /run/user/0 type tmpfs
> (rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
> resctrl on /sys/fs/resctrl type resctrl (rw,relatime)
> 
> root@localhost:~# umount resctrl
> umount: /sys/fs/resctrl: no mount point specified.
> 
> root@localhost:~# mount
> tmpfs on /run/user/0 type tmpfs
> (rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
> 
> root@localhost:~# mount -t resctrl resctrl /test
> mount: /test: unknown filesystem type 'resctrl'.


Thanks for trying this out.

I guess the behaviour here might want a bit more thought.

I'm not too keen on us leaving a defective mount in the namespace,
with a nonexistent mount pount.  I'm wondering whether things like
systemd may get confused by this...

Cheers
---Dave

Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Amit Singh Tomar]

>>> I think James will need to comment on this, but I think that yes, it
>>> is probably appropriate to require a reboot.  I think an MPAM error
>>> interrupt should only happen if the software did something wrong, so
>>> it's a bit like hitting a BUG(): we don't promise that everything works
>>> 100% properly until the system is restarted.  Misbehaviour should be
>>> contained to MPAM though.
>>>
>> if "resctrl" is nonfunctional in this state, then this comment[1] here does
>> *not* make sense.
>>
>> "restore any modified controls to their reset values."
> 
> Can you clarify what you mean here?

What I meant was, What's the rationale behind restoring the modified 
controls, if user is going to restart the system anyways (in order to 
use MPAM again), but later realized that it is needed so that *non* MPAM 
loads (user may still want to run other things even after MPAM error 
interrupt) would not have any adverse effect with modified controls.

Therefore, taking my statement back.

> 
> I think it makes sense to clean up the MPAM hardware as well as we can
> in these situations, even if we can't be certain what went wrong.
> 
> [final comments below]
> 
>> Thanks
>> -Amit
>>
>> [1]: https://urldefense.proofpoint.com/v2/url?u=https-3A__git.kernel.org_pub_scm_linux_kernel_git_morse_linux.git_tree_drivers_platform_mpam_mpam-5Fdevices.c-3Fh-3Dmpam_snapshot_v6.7-2Drc2-23n2228&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=V_GK7jRuCHDErm6txmgDK1-MbUihtnSQ3gPgB-A-JKU&m=DzR4EYX-356bYvqcrD5mYQBzLmDppMaRaHx6yjN7nRE5GH7nogtw6VtDZchmqb_q&s=SKpVy4sPg3dbFJGfMfUGoo252IHOrHrLfcv5f0sCmm0&e=
>>
>> root@localhost:~# mount
>> tmpfs on /run/user/0 type tmpfs
>> (rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
>> resctrl on /sys/fs/resctrl type resctrl (rw,relatime)
>>
>> root@localhost:~# devmem msc_addr 32 0x9999
>> [  687.096276] mpam: error irq from msc:1 'PARTID_SEL_Range', partid:39321,
>> pmg: 0, ris: 0
>>
>> root@localhost:~# mount
>> tmpfs on /run/user/0 type tmpfs
>> (rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
>> resctrl on /sys/fs/resctrl type resctrl (rw,relatime)
>>
>> root@localhost:~# umount resctrl
>> umount: /sys/fs/resctrl: no mount point specified.
>>
>> root@localhost:~# mount
>> tmpfs on /run/user/0 type tmpfs
>> (rw,nosuid,nodev,relatime,size=32923772k,nr_inodes=8230943,mode=700)
>>
>> root@localhost:~# mount -t resctrl resctrl /test
>> mount: /test: unknown filesystem type 'resctrl'.
> 
> 
> Thanks for trying this out.
> 
> I guess the behaviour here might want a bit more thought.
> 
> I'm not too keen on us leaving a defective mount in the namespace,
> with a nonexistent mount pount.  I'm wondering whether things like
> systemd may get confused by this...
> 
> Cheers
> ---Dave

Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[Dave Martin]

On Wed, May 01, 2024 at 09:51:51PM +0530, Amit Singh Tomar wrote:
> 
> > > > I think James will need to comment on this, but I think that yes, it
> > > > is probably appropriate to require a reboot.  I think an MPAM error
> > > > interrupt should only happen if the software did something wrong, so
> > > > it's a bit like hitting a BUG(): we don't promise that everything works
> > > > 100% properly until the system is restarted.  Misbehaviour should be
> > > > contained to MPAM though.
> > > > 
> > > if "resctrl" is nonfunctional in this state, then this comment[1] here does
> > > *not* make sense.
> > > 
> > > "restore any modified controls to their reset values."
> > 
> > Can you clarify what you mean here?
> 
> What I meant was, What's the rationale behind restoring the modified
> controls, if user is going to restart the system anyways (in order to use
> MPAM again), but later realized that it is needed so that *non* MPAM loads
> (user may still want to run other things even after MPAM error interrupt)
> would not have any adverse effect with modified controls.
> 
> Therefore, taking my statement back.

Ack: we can't force the system to restart without losing data.  Really,
the decision about when and whether to attempt a graceful shutdown or
reboot should be left to userspace.  But until userspace does shut down
the system, we do our best to behave as if the broken part of the system
(MPAM) were not present at all.

[...]

Cheers
---Dave

Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[James Morse]

Hi guys,

On 02/05/2024 16:58, Dave Martin wrote:
> On Wed, May 01, 2024 at 09:51:51PM +0530, Amit Singh Tomar wrote:
>>>>> I think James will need to comment on this, but I think that yes, it
>>>>> is probably appropriate to require a reboot.  I think an MPAM error
>>>>> interrupt should only happen if the software did something wrong, so
>>>>> it's a bit like hitting a BUG(): we don't promise that everything works
>>>>> 100% properly until the system is restarted.  Misbehaviour should be
>>>>> contained to MPAM though.

Indeed - all the reasons for the MPAM error interrupt being triggered indicate a software
bug, so re-mounting resctrl with the same buggy code isn't going to fix anything.


>>>> if "resctrl" is nonfunctional in this state, then this comment[1] here does
>>>> *not* make sense.
>>>>
>>>> "restore any modified controls to their reset values."

The MPAM driver goes on to reset all the MPAM hardware to the best of its ability.
These means everything gets set back to 100%, so its as if MPAM is not implemented.
This is better than throttling the wrong task because an out-of-range PARTID for
${important_task} is using the configuration of ${background_process}...


>>> Can you clarify what you mean here?
>>
>> What I meant was, What's the rationale behind restoring the modified
>> controls, if user is going to restart the system anyways (in order to use
>> MPAM again),  but later realized that it is needed so that *non* MPAM loads>> (user may still want to run other things even after MPAM error interrupt)
>> would not have any adverse effect with modified controls.
>>
>> Therefore, taking my statement back.
> 
> Ack: we can't force the system to restart without losing data.  Really,
> the decision about when and whether to attempt a graceful shutdown or
> reboot should be left to userspace.  But until userspace does shut down
> the system, we do our best to behave as if the broken part of the system
> (MPAM) were not present at all.

Dave's systemd choking on this angle is interesting - I'll go experiment with this.

The alternative here is to delete the __exit text completely as it can't be run, and
instead get MPAM's error interrupt to disable the static-keys and return -EIO for every
call into the arch code.
I didn't do this as its likely to cause extra churn to ensure that every arch helper can
propagate errors back to user-space, and this seemed like a good (re-)use of existing code.

The third option was to not do anything in MPAM, and just print a message to say bad
things might be happening. Given its extra work for hardware to detect the error
conditions, I previously assumed no-one would do this, and hardware would just 'go wrong'
instead... but as someone has built this, it would be good to try and react to it.


Thanks,

James

Re: [EXTERNAL] Re: [PATCH v1 28/31] x86/resctrl: Drop __init/__exit on assorted symbols

[James Morse]

Hi Amit, Reinette,

On 11/04/2024 16:51, Amit Singh Tomar wrote:
>> On Mon, Apr 08, 2024 at 08:32:36PM -0700, Reinette Chatre wrote:
>>> On 3/21/2024 9:51 AM, James Morse wrote:
>>>> Because ARM's MPAM controls are probed using MMIO, resctrl can't be
>>>> initialised until enough CPUs are online to have determined the
>>>> system-wide supported num_closid. Arm64 also supports 'late onlined
>>>> secondaries', where only a subset of CPUs are online during boot.
>>>>
>>>> These two combine to mean the MPAM driver may not be able to initialise
>>>> resctrl until user-space has brought 'enough' CPUs online.
>>>>
>>>> To allow MPAM to initialise resctrl after __init text has been free'd,
>>>> remove all the __init markings from resctrl.
>>>>
>>>> The existing __exit markings cause these functions to be removed by the
>>>> linker as it has never been possible to build resctrl as a module. MPAM
>>>> has an error interrupt which causes the driver to reset and disable
>>>> itself. Remove the __exit markings to allow the MPAM driver to tear down
>>>> resctrl when an error occurs.
>>>
>>> Obviously for the reasons you state this code has never been exercised.
>>> Were you able to test this error interrupt flow yet?

>> I think this will have to wait for James to respond.
>>
>> There is code to tear down resctrl in response to an MPAM error interrupt,
>> but I don't know how it has been exercised so far (if at all).

Previously I saw one or two kernfs structures left behind. (to discover this you had to
leave a shell with its CWD in the filesystem), but it looks like those issues have been
solved.

Dave points out that resctrl_exit() removing the sysfs mount point means the filesystem
can't be umount()ed. Systemd doesn't seem to care today, but might choke on this in the
future.

I think the right thing to do here is get resctrl_exit() to call rdtgroup_destroy_root(),
and drop sysfs_remove_mount_point(). This creates a bit of asymmetry, but if resctrl were
a module the mount-point stuff would be done in module init/exit - only we don't have a
module to unload, so the asymmetry is to be expected. I don't think its worth adding new
__exit text that we know will never be used for the sake of symmetry.

With this change, triggering the interrupt makes all the files under resctrl disappear, I
can then umount() the filesystem, but not re-mount it.


The aim here is for the arch code to be able to say "this is broken, I can't support
resctrl" with minimum changes to the existing code.

There are a couple of vanishingly unlikely corner cases that need tightening up: e.g. the
rmid_ptrs[] array disappears, a syscall could get blocked on the rdtgroup_mutex while the
teardown happens, once it gains the lock it discoverers a surprise NULL pointer.

Fixing these can wait until after the code is moved as these things can't happen on x86.
(patches are in the 'extra's branch of the mpam tree:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/log/?h=mpam/snapshot%2bextras/v6.10-rc1
)


> We are managed to test the MPAM error interrupt (on the platform that supports MPAM
> interrupts on software errors). For instance programming
> more resource control groups (part IDs) than available, and It appears to correctly remove
> the "resctrl" mount point (though mount command still shows resctrl on /sys/fs/resctrl
> type resctrl (rw,relatime)
> ), but
> 
> # mount -t resctrl resctrl /sys/fs/resctrl
> mount: /sys/fs/resctrl: mount point does not exist.
> 
> Additionally, a question regarding this, Is a complete system restart necessary to regain
> the mount?

It is - but you are likely to hit the same software bug again. The story here is about
keeping the machine running without penalising the wrong task. I think its acceptable for
programs driving resctrl to go wrong, provided the workload doesn't run with the wrong
configuration.


Thanks,

James

[PATCH v1 29/31] fs/resctrl: Add boiler plate for external resctrl code

[James Morse]

Add Makefile and Kconfig for fs/resctrl. Add ARCH_HAS_CPU_RESCTRL
for the common parts of the resctrl interface and make X86_CPU_RESCTRL
depend on this.

Signed-off-by: James Morse <james.morse@arm.com>
---
 MAINTAINERS              |  1 +
 arch/Kconfig             |  8 ++++++++
 arch/x86/Kconfig         | 10 +++-------
 fs/Kconfig               |  1 +
 fs/Makefile              |  1 +
 fs/resctrl/Kconfig       | 23 +++++++++++++++++++++++
 fs/resctrl/Makefile      |  3 +++
 fs/resctrl/ctrlmondata.c |  0
 fs/resctrl/internal.h    |  0
 fs/resctrl/monitor.c     |  0
 fs/resctrl/psuedo_lock.c |  0
 fs/resctrl/rdtgroup.c    |  0
 include/linux/resctrl.h  |  4 ++++
 13 files changed, 44 insertions(+), 7 deletions(-)
 create mode 100644 fs/resctrl/Kconfig
 create mode 100644 fs/resctrl/Makefile
 create mode 100644 fs/resctrl/ctrlmondata.c
 create mode 100644 fs/resctrl/internal.h
 create mode 100644 fs/resctrl/monitor.c
 create mode 100644 fs/resctrl/psuedo_lock.c
 create mode 100644 fs/resctrl/rdtgroup.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 5621dd823e79..c49090e9c777 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18543,6 +18543,7 @@ S:	Supported
 F:	Documentation/arch/x86/resctrl*
 F:	arch/x86/include/asm/resctrl.h
 F:	arch/x86/kernel/cpu/resctrl/
+F:	fs/resctrl/
 F:	include/linux/resctrl*.h
 F:	tools/testing/selftests/resctrl/
 
diff --git a/arch/Kconfig b/arch/Kconfig
index fd18b7db2c77..131d874d6738 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1406,6 +1406,14 @@ config STRICT_MODULE_RWX
 config ARCH_HAS_PHYS_TO_DMA
 	bool
 
+config ARCH_HAS_CPU_RESCTRL
+	bool
+	help
+	  The 'resctrl' filesystem allows CPU controls of shared resources
+	  such as caches and memory bandwidth to be configured. An architecture
+	  selects this if it provides the arch-specific hooks for the filesystem
+	  and needs the per-task CLOSID/RMID properties.
+
 config HAVE_ARCH_COMPILER_H
 	bool
 	help
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index e071e564452e..cb043543f088 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -479,8 +479,10 @@ config GOLDFISH
 config X86_CPU_RESCTRL
 	bool "x86 CPU resource control support"
 	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
+	depends on MISC_FILESYSTEMS
 	select KERNFS
-	select PROC_CPU_RESCTRL		if PROC_FS
+	select ARCH_HAS_CPU_RESCTRL
+	select RESCTRL_FS
 	select RESCTRL_FS_PSEUDO_LOCK
 	help
 	  Enable x86 CPU resource control support.
@@ -507,12 +509,6 @@ config X86_FRED
 	  ring transitions and exception/interrupt handling if the
 	  system supports.
 
-config RESCTRL_FS_PSEUDO_LOCK
-	bool
-	help
-	  Software mechanism to pin data in a cache portion using
-	  micro-architecture specific knowledge.
-
 if X86_32
 config X86_BIGSMP
 	bool "Support for big SMP systems with more than 8 CPUs"
diff --git a/fs/Kconfig b/fs/Kconfig
index a46b0cbc4d8f..d8a36383b6dc 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -331,6 +331,7 @@ source "fs/omfs/Kconfig"
 source "fs/hpfs/Kconfig"
 source "fs/qnx4/Kconfig"
 source "fs/qnx6/Kconfig"
+source "fs/resctrl/Kconfig"
 source "fs/romfs/Kconfig"
 source "fs/pstore/Kconfig"
 source "fs/sysv/Kconfig"
diff --git a/fs/Makefile b/fs/Makefile
index 6ecc9b0a53f2..da6e2d028722 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -129,3 +129,4 @@ obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
 obj-$(CONFIG_EROFS_FS)		+= erofs/
 obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
 obj-$(CONFIG_ZONEFS_FS)		+= zonefs/
+obj-$(CONFIG_RESCTRL_FS)	+= resctrl/
diff --git a/fs/resctrl/Kconfig b/fs/resctrl/Kconfig
new file mode 100644
index 000000000000..36a1ddbe6c21
--- /dev/null
+++ b/fs/resctrl/Kconfig
@@ -0,0 +1,23 @@
+config RESCTRL_FS
+	bool "CPU Resource Control Filesystem (resctrl)"
+	depends on ARCH_HAS_CPU_RESCTRL
+	select KERNFS
+	select PROC_CPU_RESCTRL               if PROC_FS
+	help
+	  Resctrl is a filesystem interface
+	  to control allocation and
+	  monitoring of system resources
+	  used by the CPUs.
+
+config RESCTRL_FS_PSEUDO_LOCK
+	bool
+	help
+          Software mechanism to pin data in a cache portion using
+          micro-architecture specific knowledge.
+
+config RESCTRL_RMID_DEPENDS_ON_CLOSID
+	bool
+	help
+	  Enable by the architecture when the RMID values depend on the CLOSID.
+	  This causes the closid allocator to search for CLOSID with clean
+	  RMID.
diff --git a/fs/resctrl/Makefile b/fs/resctrl/Makefile
new file mode 100644
index 000000000000..10fcfb0fdb10
--- /dev/null
+++ b/fs/resctrl/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_RESCTRL_FS)		+= rdtgroup.o ctrlmondata.o monitor.o
+obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= psuedo_lock.o
diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/fs/resctrl/psuedo_lock.c b/fs/resctrl/psuedo_lock.c
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 5da55e58f229..f786ffceeda3 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -8,6 +8,10 @@
 #include <linux/pid.h>
 #include <linux/resctrl_types.h>
 
+#ifdef CONFIG_ARCH_HAS_CPU_RESCTRL
+#include <asm/resctrl.h>
+#endif
+
 /* CLOSID, RMID value used by the default control group */
 #define RESCTRL_RESERVED_CLOSID		0
 #define RESCTRL_RESERVED_RMID		0
-- 
2.39.2

Re: [PATCH v1 29/31] fs/resctrl: Add boiler plate for external resctrl code

[Reinette Chatre]

Hi James,

On 3/21/2024 9:51 AM, James Morse wrote:
> Add Makefile and Kconfig for fs/resctrl. Add ARCH_HAS_CPU_RESCTRL
> for the common parts of the resctrl interface and make X86_CPU_RESCTRL
> depend on this.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  MAINTAINERS              |  1 +
>  arch/Kconfig             |  8 ++++++++
>  arch/x86/Kconfig         | 10 +++-------
>  fs/Kconfig               |  1 +
>  fs/Makefile              |  1 +
>  fs/resctrl/Kconfig       | 23 +++++++++++++++++++++++
>  fs/resctrl/Makefile      |  3 +++
>  fs/resctrl/ctrlmondata.c |  0
>  fs/resctrl/internal.h    |  0
>  fs/resctrl/monitor.c     |  0
>  fs/resctrl/psuedo_lock.c |  0
>  fs/resctrl/rdtgroup.c    |  0
>  include/linux/resctrl.h  |  4 ++++
>  13 files changed, 44 insertions(+), 7 deletions(-)
>  create mode 100644 fs/resctrl/Kconfig
>  create mode 100644 fs/resctrl/Makefile
>  create mode 100644 fs/resctrl/ctrlmondata.c
>  create mode 100644 fs/resctrl/internal.h
>  create mode 100644 fs/resctrl/monitor.c
>  create mode 100644 fs/resctrl/psuedo_lock.c
>  create mode 100644 fs/resctrl/rdtgroup.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 5621dd823e79..c49090e9c777 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -18543,6 +18543,7 @@ S:	Supported
>  F:	Documentation/arch/x86/resctrl*
>  F:	arch/x86/include/asm/resctrl.h
>  F:	arch/x86/kernel/cpu/resctrl/
> +F:	fs/resctrl/
>  F:	include/linux/resctrl*.h
>  F:	tools/testing/selftests/resctrl/
>  
> diff --git a/arch/Kconfig b/arch/Kconfig
> index fd18b7db2c77..131d874d6738 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -1406,6 +1406,14 @@ config STRICT_MODULE_RWX
>  config ARCH_HAS_PHYS_TO_DMA
>  	bool
>  
> +config ARCH_HAS_CPU_RESCTRL
> +	bool
> +	help
> +	  The 'resctrl' filesystem allows CPU controls of shared resources
> +	  such as caches and memory bandwidth to be configured. An architecture
> +	  selects this if it provides the arch-specific hooks for the filesystem
> +	  and needs the per-task CLOSID/RMID properties.

Should it mention monitoring capabilities?

> +
>  config HAVE_ARCH_COMPILER_H
>  	bool
>  	help
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index e071e564452e..cb043543f088 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -479,8 +479,10 @@ config GOLDFISH
>  config X86_CPU_RESCTRL
>  	bool "x86 CPU resource control support"
>  	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
> +	depends on MISC_FILESYSTEMS
>  	select KERNFS

Do both X86_CPU_RESCTRL and RESCTRL_FS need to select KERNFS?

> -	select PROC_CPU_RESCTRL		if PROC_FS
> +	select ARCH_HAS_CPU_RESCTRL
> +	select RESCTRL_FS
>  	select RESCTRL_FS_PSEUDO_LOCK
>  	help
>  	  Enable x86 CPU resource control support.
> @@ -507,12 +509,6 @@ config X86_FRED
>  	  ring transitions and exception/interrupt handling if the
>  	  system supports.
>  
> -config RESCTRL_FS_PSEUDO_LOCK
> -	bool
> -	help
> -	  Software mechanism to pin data in a cache portion using
> -	  micro-architecture specific knowledge.
> -
>  if X86_32
>  config X86_BIGSMP
>  	bool "Support for big SMP systems with more than 8 CPUs"
> diff --git a/fs/Kconfig b/fs/Kconfig
> index a46b0cbc4d8f..d8a36383b6dc 100644
> --- a/fs/Kconfig
> +++ b/fs/Kconfig
> @@ -331,6 +331,7 @@ source "fs/omfs/Kconfig"
>  source "fs/hpfs/Kconfig"
>  source "fs/qnx4/Kconfig"
>  source "fs/qnx6/Kconfig"
> +source "fs/resctrl/Kconfig"
>  source "fs/romfs/Kconfig"
>  source "fs/pstore/Kconfig"
>  source "fs/sysv/Kconfig"
> diff --git a/fs/Makefile b/fs/Makefile
> index 6ecc9b0a53f2..da6e2d028722 100644
> --- a/fs/Makefile
> +++ b/fs/Makefile
> @@ -129,3 +129,4 @@ obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
>  obj-$(CONFIG_EROFS_FS)		+= erofs/
>  obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
>  obj-$(CONFIG_ZONEFS_FS)		+= zonefs/
> +obj-$(CONFIG_RESCTRL_FS)	+= resctrl/
> diff --git a/fs/resctrl/Kconfig b/fs/resctrl/Kconfig
> new file mode 100644
> index 000000000000..36a1ddbe6c21
> --- /dev/null
> +++ b/fs/resctrl/Kconfig

Could you please review the contents of this file for
appropriate line length and consistent tab usage?

> @@ -0,0 +1,23 @@
> +config RESCTRL_FS
> +	bool "CPU Resource Control Filesystem (resctrl)"
> +	depends on ARCH_HAS_CPU_RESCTRL
> +	select KERNFS
> +	select PROC_CPU_RESCTRL               if PROC_FS
> +	help
> +	  Resctrl is a filesystem interface
> +	  to control allocation and
> +	  monitoring of system resources
> +	  used by the CPUs.
> +
> +config RESCTRL_FS_PSEUDO_LOCK
> +	bool
> +	help
> +          Software mechanism to pin data in a cache portion using
> +          micro-architecture specific knowledge.
> +
> +config RESCTRL_RMID_DEPENDS_ON_CLOSID
> +	bool
> +	help
> +	  Enable by the architecture when the RMID values depend on the CLOSID.
> +	  This causes the closid allocator to search for CLOSID with clean
> +	  RMID.
> diff --git a/fs/resctrl/Makefile b/fs/resctrl/Makefile
> new file mode 100644
> index 000000000000..10fcfb0fdb10
> --- /dev/null
> +++ b/fs/resctrl/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +obj-$(CONFIG_RESCTRL_FS)		+= rdtgroup.o ctrlmondata.o monitor.o
> +obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= psuedo_lock.o
> diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/fs/resctrl/psuedo_lock.c b/fs/resctrl/psuedo_lock.c

pseudo_lock.c

> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index 5da55e58f229..f786ffceeda3 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -8,6 +8,10 @@
>  #include <linux/pid.h>
>  #include <linux/resctrl_types.h>
>  
> +#ifdef CONFIG_ARCH_HAS_CPU_RESCTRL
> +#include <asm/resctrl.h>
> +#endif
> +
>  /* CLOSID, RMID value used by the default control group */
>  #define RESCTRL_RESERVED_CLOSID		0
>  #define RESCTRL_RESERVED_RMID		0

Reinette

Re: [PATCH v1 29/31] fs/resctrl: Add boiler plate for external resctrl code

[Dave Martin]

On Mon, Apr 08, 2024 at 08:41:04PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:51 AM, James Morse wrote:
> > Add Makefile and Kconfig for fs/resctrl. Add ARCH_HAS_CPU_RESCTRL
> > for the common parts of the resctrl interface and make X86_CPU_RESCTRL
> > depend on this.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  MAINTAINERS              |  1 +
> >  arch/Kconfig             |  8 ++++++++
> >  arch/x86/Kconfig         | 10 +++-------
> >  fs/Kconfig               |  1 +
> >  fs/Makefile              |  1 +
> >  fs/resctrl/Kconfig       | 23 +++++++++++++++++++++++
> >  fs/resctrl/Makefile      |  3 +++
> >  fs/resctrl/ctrlmondata.c |  0
> >  fs/resctrl/internal.h    |  0
> >  fs/resctrl/monitor.c     |  0
> >  fs/resctrl/psuedo_lock.c |  0
> >  fs/resctrl/rdtgroup.c    |  0
> >  include/linux/resctrl.h  |  4 ++++
> >  13 files changed, 44 insertions(+), 7 deletions(-)
> >  create mode 100644 fs/resctrl/Kconfig
> >  create mode 100644 fs/resctrl/Makefile
> >  create mode 100644 fs/resctrl/ctrlmondata.c
> >  create mode 100644 fs/resctrl/internal.h
> >  create mode 100644 fs/resctrl/monitor.c
> >  create mode 100644 fs/resctrl/psuedo_lock.c
> >  create mode 100644 fs/resctrl/rdtgroup.c
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 5621dd823e79..c49090e9c777 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -18543,6 +18543,7 @@ S:	Supported
> >  F:	Documentation/arch/x86/resctrl*
> >  F:	arch/x86/include/asm/resctrl.h
> >  F:	arch/x86/kernel/cpu/resctrl/
> > +F:	fs/resctrl/
> >  F:	include/linux/resctrl*.h
> >  F:	tools/testing/selftests/resctrl/
> >  
> > diff --git a/arch/Kconfig b/arch/Kconfig
> > index fd18b7db2c77..131d874d6738 100644
> > --- a/arch/Kconfig
> > +++ b/arch/Kconfig
> > @@ -1406,6 +1406,14 @@ config STRICT_MODULE_RWX
> >  config ARCH_HAS_PHYS_TO_DMA
> >  	bool
> >  
> > +config ARCH_HAS_CPU_RESCTRL
> > +	bool
> > +	help
> > +	  The 'resctrl' filesystem allows CPU controls of shared resources
> > +	  such as caches and memory bandwidth to be configured. An architecture
> > +	  selects this if it provides the arch-specific hooks for the filesystem
> > +	  and needs the per-task CLOSID/RMID properties.
> 
> Should it mention monitoring capabilities?

Probably, although I wonder whether it is better to describe this just
once, under RESCTRL_FS.  Does it makes sense to have something
like this here?

	  An architecture selects this option to indicate that the necessary
	  hooks are provided to support the common memory system usage
	  monitoring and control interfaces provided by the 'resctrl'
	  filesystem (see RESCTRL_FS).

If so, I can propose this.

(Details on what gets added to task_struct is maybe unnecessarily low-
level to bother with here...)

> > +
> >  config HAVE_ARCH_COMPILER_H
> >  	bool
> >  	help
> > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> > index e071e564452e..cb043543f088 100644
> > --- a/arch/x86/Kconfig
> > +++ b/arch/x86/Kconfig
> > @@ -479,8 +479,10 @@ config GOLDFISH
> >  config X86_CPU_RESCTRL
> >  	bool "x86 CPU resource control support"
> >  	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
> > +	depends on MISC_FILESYSTEMS
> >  	select KERNFS
> 
> Do both X86_CPU_RESCTRL and RESCTRL_FS need to select KERNFS?

Hmmm, hopefully the arch backend doesn't need to re-depend on KERNFS.
I'll note that for review.

(If not, we can probably drop filesystem-related #includes from the
remaining x86 arch code too...)

[...]

> > diff --git a/fs/Kconfig b/fs/Kconfig
> > index a46b0cbc4d8f..d8a36383b6dc 100644
> > --- a/fs/Kconfig
> > +++ b/fs/Kconfig
> > @@ -331,6 +331,7 @@ source "fs/omfs/Kconfig"
> >  source "fs/hpfs/Kconfig"
> >  source "fs/qnx4/Kconfig"
> >  source "fs/qnx6/Kconfig"
> > +source "fs/resctrl/Kconfig"
> >  source "fs/romfs/Kconfig"
> >  source "fs/pstore/Kconfig"
> >  source "fs/sysv/Kconfig"
> > diff --git a/fs/Makefile b/fs/Makefile
> > index 6ecc9b0a53f2..da6e2d028722 100644
> > --- a/fs/Makefile
> > +++ b/fs/Makefile
> > @@ -129,3 +129,4 @@ obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
> >  obj-$(CONFIG_EROFS_FS)		+= erofs/
> >  obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
> >  obj-$(CONFIG_ZONEFS_FS)		+= zonefs/
> > +obj-$(CONFIG_RESCTRL_FS)	+= resctrl/
> > diff --git a/fs/resctrl/Kconfig b/fs/resctrl/Kconfig
> > new file mode 100644
> > index 000000000000..36a1ddbe6c21
> > --- /dev/null
> > +++ b/fs/resctrl/Kconfig
> 
> Could you please review the contents of this file for
> appropriate line length and consistent tab usage?

Noted.

> > @@ -0,0 +1,23 @@
> > +config RESCTRL_FS
> > +	bool "CPU Resource Control Filesystem (resctrl)"
> > +	depends on ARCH_HAS_CPU_RESCTRL
> > +	select KERNFS
> > +	select PROC_CPU_RESCTRL               if PROC_FS
> > +	help
> > +	  Resctrl is a filesystem interface
> > +	  to control allocation and
> > +	  monitoring of system resources
> > +	  used by the CPUs.

(Not quite a haiku, but I don't know how many syllables "resctrl"
counts as...)

Since this is the Kconfig user's primary knob for enabling resctrl,
maybe flesh this out and make it a bit more generic and newbie-friendly?
Something like:

	  Some architectures provide hardware facilities to group tasks and
	  monitor and control their usage of memory system resources such as
	  caches and memory bandwidth.  Examples of such facilities include
	  Intel's Resource Director Technology (Intel(R) RDT) and AMD's
	  Platform Quality of Service (AMD QoS).

	  If your system has the necessary support and you want to be able to
	  assign tasks to groups and manipulate the associated resource
	  monitors and controls from userspace, say Y here to get a mountable
	  'resctrl' filesystem that lets you do just that.

	  If nothing mounts or prods the 'resctrl' filesystem, resource
	  controls and monitors are left in a quiescent, permissive state.

	  If unsure, it is safe to say Y.

	  See <file:Documentation/arch/x86/resctrl.rst> for more information.

I'm assuming that just enabling this option doesn't introduce
significant overheads.  For MPAM I'm pretty sure it doesn't,
but if this is a concern that we could go for "If unsure, say N."

If this looks OK, I can propose it to James.

> > +
> > +config RESCTRL_FS_PSEUDO_LOCK
> > +	bool
> > +	help
> > +          Software mechanism to pin data in a cache portion using
> > +          micro-architecture specific knowledge.
> > +
> > +config RESCTRL_RMID_DEPENDS_ON_CLOSID
> > +	bool
> > +	help
> > +	  Enable by the architecture when the RMID values depend on the CLOSID.
> > +	  This causes the closid allocator to search for CLOSID with clean
> > +	  RMID.
> > diff --git a/fs/resctrl/Makefile b/fs/resctrl/Makefile
> > new file mode 100644
> > index 000000000000..10fcfb0fdb10
> > --- /dev/null
> > +++ b/fs/resctrl/Makefile
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +obj-$(CONFIG_RESCTRL_FS)		+= rdtgroup.o ctrlmondata.o monitor.o
> > +obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= psuedo_lock.o
> > diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
> > new file mode 100644
> > index 000000000000..e69de29bb2d1
> > diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
> > new file mode 100644
> > index 000000000000..e69de29bb2d1
> > diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> > new file mode 100644
> > index 000000000000..e69de29bb2d1
> > diff --git a/fs/resctrl/psuedo_lock.c b/fs/resctrl/psuedo_lock.c
> 
> pseudo_lock.c

[...]

> Reinette

Noted here, and in the Makefile snippet above.

Cheers
---Dave

Re: [PATCH v1 29/31] fs/resctrl: Add boiler plate for external resctrl code

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:27 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:41:04PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:51 AM, James Morse wrote:
>>> Add Makefile and Kconfig for fs/resctrl. Add ARCH_HAS_CPU_RESCTRL
>>> for the common parts of the resctrl interface and make X86_CPU_RESCTRL
>>> depend on this.
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>> ---
>>>  MAINTAINERS              |  1 +
>>>  arch/Kconfig             |  8 ++++++++
>>>  arch/x86/Kconfig         | 10 +++-------
>>>  fs/Kconfig               |  1 +
>>>  fs/Makefile              |  1 +
>>>  fs/resctrl/Kconfig       | 23 +++++++++++++++++++++++
>>>  fs/resctrl/Makefile      |  3 +++
>>>  fs/resctrl/ctrlmondata.c |  0
>>>  fs/resctrl/internal.h    |  0
>>>  fs/resctrl/monitor.c     |  0
>>>  fs/resctrl/psuedo_lock.c |  0
>>>  fs/resctrl/rdtgroup.c    |  0
>>>  include/linux/resctrl.h  |  4 ++++
>>>  13 files changed, 44 insertions(+), 7 deletions(-)
>>>  create mode 100644 fs/resctrl/Kconfig
>>>  create mode 100644 fs/resctrl/Makefile
>>>  create mode 100644 fs/resctrl/ctrlmondata.c
>>>  create mode 100644 fs/resctrl/internal.h
>>>  create mode 100644 fs/resctrl/monitor.c
>>>  create mode 100644 fs/resctrl/psuedo_lock.c
>>>  create mode 100644 fs/resctrl/rdtgroup.c
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 5621dd823e79..c49090e9c777 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -18543,6 +18543,7 @@ S:	Supported
>>>  F:	Documentation/arch/x86/resctrl*
>>>  F:	arch/x86/include/asm/resctrl.h
>>>  F:	arch/x86/kernel/cpu/resctrl/
>>> +F:	fs/resctrl/
>>>  F:	include/linux/resctrl*.h
>>>  F:	tools/testing/selftests/resctrl/
>>>  
>>> diff --git a/arch/Kconfig b/arch/Kconfig
>>> index fd18b7db2c77..131d874d6738 100644
>>> --- a/arch/Kconfig
>>> +++ b/arch/Kconfig
>>> @@ -1406,6 +1406,14 @@ config STRICT_MODULE_RWX
>>>  config ARCH_HAS_PHYS_TO_DMA
>>>  	bool
>>>  
>>> +config ARCH_HAS_CPU_RESCTRL
>>> +	bool
>>> +	help
>>> +	  The 'resctrl' filesystem allows CPU controls of shared resources
>>> +	  such as caches and memory bandwidth to be configured. An architecture
>>> +	  selects this if it provides the arch-specific hooks for the filesystem
>>> +	  and needs the per-task CLOSID/RMID properties.
>>
>> Should it mention monitoring capabilities?
> 
> Probably, although I wonder whether it is better to describe this just
> once, under RESCTRL_FS.  Does it makes sense to have something
> like this here?
> 
> 	  An architecture selects this option to indicate that the necessary
> 	  hooks are provided to support the common memory system usage
> 	  monitoring and control interfaces provided by the 'resctrl'
> 	  filesystem (see RESCTRL_FS).

This looks good to me.

> 
> If so, I can propose this.
> 
> (Details on what gets added to task_struct is maybe unnecessarily low-
> level to bother with here...)
> 
>>> +
>>>  config HAVE_ARCH_COMPILER_H
>>>  	bool
>>>  	help
>>> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
>>> index e071e564452e..cb043543f088 100644
>>> --- a/arch/x86/Kconfig
>>> +++ b/arch/x86/Kconfig
>>> @@ -479,8 +479,10 @@ config GOLDFISH
>>>  config X86_CPU_RESCTRL
>>>  	bool "x86 CPU resource control support"
>>>  	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
>>> +	depends on MISC_FILESYSTEMS
>>>  	select KERNFS
>>
>> Do both X86_CPU_RESCTRL and RESCTRL_FS need to select KERNFS?
> 
> Hmmm, hopefully the arch backend doesn't need to re-depend on KERNFS.
> I'll note that for review.
> 
> (If not, we can probably drop filesystem-related #includes from the
> remaining x86 arch code too...)
> 
> [...]
> 
>>> diff --git a/fs/Kconfig b/fs/Kconfig
>>> index a46b0cbc4d8f..d8a36383b6dc 100644
>>> --- a/fs/Kconfig
>>> +++ b/fs/Kconfig
>>> @@ -331,6 +331,7 @@ source "fs/omfs/Kconfig"
>>>  source "fs/hpfs/Kconfig"
>>>  source "fs/qnx4/Kconfig"
>>>  source "fs/qnx6/Kconfig"
>>> +source "fs/resctrl/Kconfig"
>>>  source "fs/romfs/Kconfig"
>>>  source "fs/pstore/Kconfig"
>>>  source "fs/sysv/Kconfig"
>>> diff --git a/fs/Makefile b/fs/Makefile
>>> index 6ecc9b0a53f2..da6e2d028722 100644
>>> --- a/fs/Makefile
>>> +++ b/fs/Makefile
>>> @@ -129,3 +129,4 @@ obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
>>>  obj-$(CONFIG_EROFS_FS)		+= erofs/
>>>  obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
>>>  obj-$(CONFIG_ZONEFS_FS)		+= zonefs/
>>> +obj-$(CONFIG_RESCTRL_FS)	+= resctrl/
>>> diff --git a/fs/resctrl/Kconfig b/fs/resctrl/Kconfig
>>> new file mode 100644
>>> index 000000000000..36a1ddbe6c21
>>> --- /dev/null
>>> +++ b/fs/resctrl/Kconfig
>>
>> Could you please review the contents of this file for
>> appropriate line length and consistent tab usage?
> 
> Noted.
> 
>>> @@ -0,0 +1,23 @@
>>> +config RESCTRL_FS
>>> +	bool "CPU Resource Control Filesystem (resctrl)"
>>> +	depends on ARCH_HAS_CPU_RESCTRL
>>> +	select KERNFS
>>> +	select PROC_CPU_RESCTRL               if PROC_FS
>>> +	help
>>> +	  Resctrl is a filesystem interface
>>> +	  to control allocation and
>>> +	  monitoring of system resources
>>> +	  used by the CPUs.
> 
> (Not quite a haiku, but I don't know how many syllables "resctrl"
> counts as...)
> 
> Since this is the Kconfig user's primary knob for enabling resctrl,
> maybe flesh this out and make it a bit more generic and newbie-friendly?
> Something like:
> 
> 	  Some architectures provide hardware facilities to group tasks and
> 	  monitor and control their usage of memory system resources such as
> 	  caches and memory bandwidth.  Examples of such facilities include
> 	  Intel's Resource Director Technology (Intel(R) RDT) and AMD's
> 	  Platform Quality of Service (AMD QoS).

Nit: We should double check with AMD how they want to refer to their
feature. Their contribution to the resctrl docs used the term you provide
but their spec uses PQOS.

Do you expect this snippet to be updated when MPAM full support lands or
does resctrl have enough support at this point to include a mention of MPAM?

> 
> 	  If your system has the necessary support and you want to be able to
> 	  assign tasks to groups and manipulate the associated resource
> 	  monitors and controls from userspace, say Y here to get a mountable
> 	  'resctrl' filesystem that lets you do just that.
> 
> 	  If nothing mounts or prods the 'resctrl' filesystem, resource
> 	  controls and monitors are left in a quiescent, permissive state.

Well written, thank you.

> 
> 	  If unsure, it is safe to say Y.
> 
> 	  See <file:Documentation/arch/x86/resctrl.rst> for more information.
> 
> I'm assuming that just enabling this option doesn't introduce
> significant overheads.  For MPAM I'm pretty sure it doesn't,
> but if this is a concern that we could go for "If unsure, say N."

I would vote for N for when folks are not sure.

> 
> If this looks OK, I can propose it to James.
> 

Reinette

Re: [PATCH v1 29/31] fs/resctrl: Add boiler plate for external resctrl code

[Dave Martin]

On Thu, Apr 11, 2024 at 10:42:48AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:27 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:41:04PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:51 AM, James Morse wrote:
> >>> Add Makefile and Kconfig for fs/resctrl. Add ARCH_HAS_CPU_RESCTRL
> >>> for the common parts of the resctrl interface and make X86_CPU_RESCTRL
> >>> depend on this.
> >>>
> >>> Signed-off-by: James Morse <james.morse@arm.com>
> >>> ---
> >>>  MAINTAINERS              |  1 +
> >>>  arch/Kconfig             |  8 ++++++++
> >>>  arch/x86/Kconfig         | 10 +++-------
> >>>  fs/Kconfig               |  1 +
> >>>  fs/Makefile              |  1 +
> >>>  fs/resctrl/Kconfig       | 23 +++++++++++++++++++++++
> >>>  fs/resctrl/Makefile      |  3 +++
> >>>  fs/resctrl/ctrlmondata.c |  0
> >>>  fs/resctrl/internal.h    |  0
> >>>  fs/resctrl/monitor.c     |  0
> >>>  fs/resctrl/psuedo_lock.c |  0
> >>>  fs/resctrl/rdtgroup.c    |  0
> >>>  include/linux/resctrl.h  |  4 ++++
> >>>  13 files changed, 44 insertions(+), 7 deletions(-)
> >>>  create mode 100644 fs/resctrl/Kconfig
> >>>  create mode 100644 fs/resctrl/Makefile
> >>>  create mode 100644 fs/resctrl/ctrlmondata.c
> >>>  create mode 100644 fs/resctrl/internal.h
> >>>  create mode 100644 fs/resctrl/monitor.c
> >>>  create mode 100644 fs/resctrl/psuedo_lock.c
> >>>  create mode 100644 fs/resctrl/rdtgroup.c
> >>>
> >>> diff --git a/MAINTAINERS b/MAINTAINERS
> >>> index 5621dd823e79..c49090e9c777 100644
> >>> --- a/MAINTAINERS
> >>> +++ b/MAINTAINERS
> >>> @@ -18543,6 +18543,7 @@ S:	Supported
> >>>  F:	Documentation/arch/x86/resctrl*
> >>>  F:	arch/x86/include/asm/resctrl.h
> >>>  F:	arch/x86/kernel/cpu/resctrl/
> >>> +F:	fs/resctrl/
> >>>  F:	include/linux/resctrl*.h
> >>>  F:	tools/testing/selftests/resctrl/
> >>>  
> >>> diff --git a/arch/Kconfig b/arch/Kconfig
> >>> index fd18b7db2c77..131d874d6738 100644
> >>> --- a/arch/Kconfig
> >>> +++ b/arch/Kconfig
> >>> @@ -1406,6 +1406,14 @@ config STRICT_MODULE_RWX
> >>>  config ARCH_HAS_PHYS_TO_DMA
> >>>  	bool
> >>>  
> >>> +config ARCH_HAS_CPU_RESCTRL
> >>> +	bool
> >>> +	help
> >>> +	  The 'resctrl' filesystem allows CPU controls of shared resources
> >>> +	  such as caches and memory bandwidth to be configured. An architecture
> >>> +	  selects this if it provides the arch-specific hooks for the filesystem
> >>> +	  and needs the per-task CLOSID/RMID properties.
> >>
> >> Should it mention monitoring capabilities?
> > 
> > Probably, although I wonder whether it is better to describe this just
> > once, under RESCTRL_FS.  Does it makes sense to have something
> > like this here?
> > 
> > 	  An architecture selects this option to indicate that the necessary
> > 	  hooks are provided to support the common memory system usage
> > 	  monitoring and control interfaces provided by the 'resctrl'
> > 	  filesystem (see RESCTRL_FS).
> 
> This looks good to me.

Noted, thanks.

> 
> > 
> > If so, I can propose this.
> > 
> > (Details on what gets added to task_struct is maybe unnecessarily low-
> > level to bother with here...)
> > 
> >>> +
> >>>  config HAVE_ARCH_COMPILER_H
> >>>  	bool
> >>>  	help
> >>> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> >>> index e071e564452e..cb043543f088 100644
> >>> --- a/arch/x86/Kconfig
> >>> +++ b/arch/x86/Kconfig
> >>> @@ -479,8 +479,10 @@ config GOLDFISH
> >>>  config X86_CPU_RESCTRL
> >>>  	bool "x86 CPU resource control support"
> >>>  	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
> >>> +	depends on MISC_FILESYSTEMS
> >>>  	select KERNFS
> >>
> >> Do both X86_CPU_RESCTRL and RESCTRL_FS need to select KERNFS?
> > 
> > Hmmm, hopefully the arch backend doesn't need to re-depend on KERNFS.
> > I'll note that for review.
> > 
> > (If not, we can probably drop filesystem-related #includes from the
> > remaining x86 arch code too...)
> > 
> > [...]

(I still need to look at this.)

> > 
> >>> diff --git a/fs/Kconfig b/fs/Kconfig
> >>> index a46b0cbc4d8f..d8a36383b6dc 100644
> >>> --- a/fs/Kconfig
> >>> +++ b/fs/Kconfig
> >>> @@ -331,6 +331,7 @@ source "fs/omfs/Kconfig"
> >>>  source "fs/hpfs/Kconfig"
> >>>  source "fs/qnx4/Kconfig"
> >>>  source "fs/qnx6/Kconfig"
> >>> +source "fs/resctrl/Kconfig"
> >>>  source "fs/romfs/Kconfig"
> >>>  source "fs/pstore/Kconfig"
> >>>  source "fs/sysv/Kconfig"
> >>> diff --git a/fs/Makefile b/fs/Makefile
> >>> index 6ecc9b0a53f2..da6e2d028722 100644
> >>> --- a/fs/Makefile
> >>> +++ b/fs/Makefile
> >>> @@ -129,3 +129,4 @@ obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
> >>>  obj-$(CONFIG_EROFS_FS)		+= erofs/
> >>>  obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
> >>>  obj-$(CONFIG_ZONEFS_FS)		+= zonefs/
> >>> +obj-$(CONFIG_RESCTRL_FS)	+= resctrl/
> >>> diff --git a/fs/resctrl/Kconfig b/fs/resctrl/Kconfig
> >>> new file mode 100644
> >>> index 000000000000..36a1ddbe6c21
> >>> --- /dev/null
> >>> +++ b/fs/resctrl/Kconfig
> >>
> >> Could you please review the contents of this file for
> >> appropriate line length and consistent tab usage?
> > 
> > Noted.
> > 
> >>> @@ -0,0 +1,23 @@
> >>> +config RESCTRL_FS
> >>> +	bool "CPU Resource Control Filesystem (resctrl)"
> >>> +	depends on ARCH_HAS_CPU_RESCTRL
> >>> +	select KERNFS
> >>> +	select PROC_CPU_RESCTRL               if PROC_FS
> >>> +	help
> >>> +	  Resctrl is a filesystem interface
> >>> +	  to control allocation and
> >>> +	  monitoring of system resources
> >>> +	  used by the CPUs.
> > 
> > (Not quite a haiku, but I don't know how many syllables "resctrl"
> > counts as...)
> > 
> > Since this is the Kconfig user's primary knob for enabling resctrl,
> > maybe flesh this out and make it a bit more generic and newbie-friendly?
> > Something like:
> > 
> > 	  Some architectures provide hardware facilities to group tasks and
> > 	  monitor and control their usage of memory system resources such as
> > 	  caches and memory bandwidth.  Examples of such facilities include
> > 	  Intel's Resource Director Technology (Intel(R) RDT) and AMD's
> > 	  Platform Quality of Service (AMD QoS).
> 
> Nit: We should double check with AMD how they want to refer to their
> feature. Their contribution to the resctrl docs used the term you provide
> but their spec uses PQOS.
> 
> Do you expect this snippet to be updated when MPAM full support lands or
> does resctrl have enough support at this point to include a mention of MPAM?

Probably yes, even if only to make the point that this is generic and
relevant for more than one architecture.

For this series, MPAM is not known to the kernel, so I didn't want to
give the impression that there was any support for it (yet).

> > 
> > 	  If your system has the necessary support and you want to be able to
> > 	  assign tasks to groups and manipulate the associated resource
> > 	  monitors and controls from userspace, say Y here to get a mountable
> > 	  'resctrl' filesystem that lets you do just that.
> > 
> > 	  If nothing mounts or prods the 'resctrl' filesystem, resource
> > 	  controls and monitors are left in a quiescent, permissive state.
> 
> Well written, thank you.

Thanks for the once-over; I'll propose this to James for inclusion.

> 
> > 
> > 	  If unsure, it is safe to say Y.
> > 
> > 	  See <file:Documentation/arch/x86/resctrl.rst> for more information.
> > 
> > I'm assuming that just enabling this option doesn't introduce
> > significant overheads.  For MPAM I'm pretty sure it doesn't,
> > but if this is a concern that we could go for "If unsure, say N."
> 
> I would vote for N for when folks are not sure.

OK, fair enough.  Noted.

Cheers
---Dave

Re: [PATCH v1 29/31] fs/resctrl: Add boiler plate for external resctrl code

[Dave Martin]

Hi Babu,

[see below]

On Thu, Apr 11, 2024 at 10:42:48AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:27 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:41:04PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:51 AM, James Morse wrote:
> >>> Add Makefile and Kconfig for fs/resctrl. Add ARCH_HAS_CPU_RESCTRL
> >>> for the common parts of the resctrl interface and make X86_CPU_RESCTRL
> >>> depend on this.
> >>>
> >>> Signed-off-by: James Morse <james.morse@arm.com>

[...]

> >>> diff --git a/fs/resctrl/Kconfig b/fs/resctrl/Kconfig
> >>> new file mode 100644
> >>> index 000000000000..36a1ddbe6c21
> >>> --- /dev/null
> >>> +++ b/fs/resctrl/Kconfig
> >>
> >> Could you please review the contents of this file for
> >> appropriate line length and consistent tab usage?
> > 
> > Noted.
> > 
> >>> @@ -0,0 +1,23 @@
> >>> +config RESCTRL_FS
> >>> +	bool "CPU Resource Control Filesystem (resctrl)"
> >>> +	depends on ARCH_HAS_CPU_RESCTRL
> >>> +	select KERNFS
> >>> +	select PROC_CPU_RESCTRL               if PROC_FS
> >>> +	help
> >>> +	  Resctrl is a filesystem interface
> >>> +	  to control allocation and
> >>> +	  monitoring of system resources
> >>> +	  used by the CPUs.
> > 
> > (Not quite a haiku, but I don't know how many syllables "resctrl"
> > counts as...)
> > 
> > Since this is the Kconfig user's primary knob for enabling resctrl,
> > maybe flesh this out and make it a bit more generic and newbie-friendly?
> > Something like:
> > 
> > 	  Some architectures provide hardware facilities to group tasks and
> > 	  monitor and control their usage of memory system resources such as
> > 	  caches and memory bandwidth.  Examples of such facilities include
> > 	  Intel's Resource Director Technology (Intel(R) RDT) and AMD's
> > 	  Platform Quality of Service (AMD QoS).
> 
> Nit: We should double check with AMD how they want to refer to their
> feature. Their contribution to the resctrl docs used the term you provide
> but their spec uses PQOS.

Babu, do you have a view on this?

Initially I just pasted this across from the existing text in
resctrl.rst and arch/x86/Kconfig.

Looking at the AMD's spec though, I'd probably follow Reinette's
suggestion and go for:

	[...] AMD's Platform Quality of Service (PQOS)

but I'm happy to take your recommendation if you think AMD would prefer
different text here.

Cheers
---Dave

[PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[James Morse]

Once the filesystem parts of resctrl move to fs/resctrl, it cannot rely
on definitions in x86's internal.h.

Move definitions in internal.h that need to be shared between the
filesystem and architecture code to header files that fs/resctrl can
include.

Doing this separately means the filesystem code only moves between files
of the same name, instead of having these changes mixed in too.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/x86/include/asm/resctrl.h         |  3 +++
 arch/x86/kernel/cpu/resctrl/core.c     |  5 ++++
 arch/x86/kernel/cpu/resctrl/internal.h | 36 --------------------------
 include/linux/resctrl.h                |  3 +++
 include/linux/resctrl_types.h          | 30 +++++++++++++++++++++
 5 files changed, 41 insertions(+), 36 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 491342f56811..746431c66fc4 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -218,6 +218,9 @@ int resctrl_arch_measure_l2_residency(void *_plr);
 int resctrl_arch_measure_l3_residency(void *_plr);
 void resctrl_cpu_detect(struct cpuinfo_x86 *c);
 
+bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level l);
+int resctrl_arch_set_cdp_enabled(enum resctrl_res_level l, bool enable);
+
 #else
 
 static inline void resctrl_arch_sched_in(struct task_struct *tsk) {}
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index f94ad04023c3..c0fb2e22e110 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -306,6 +306,11 @@ static void rdt_get_cdp_l2_config(void)
 	rdt_get_cdp_config(RDT_RESOURCE_L2);
 }
 
+bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level l)
+{
+	return rdt_resources_all[l].cdp_enabled;
+}
+
 static void
 mba_wrmsr_amd(struct rdt_domain *d, struct msr_param *m, struct rdt_resource *r)
 {
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 56218193a8ba..0f7e3f10941b 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -15,12 +15,6 @@
 
 #define L2_QOS_CDP_ENABLE		0x01ULL
 
-#define CQM_LIMBOCHECK_INTERVAL	1000
-
-#define MBM_CNTR_WIDTH_BASE		24
-#define MBM_OVERFLOW_INTERVAL		1000
-#define MAX_MBA_BW			100u
-#define MBA_IS_LINEAR			0x4
 #define MBM_CNTR_WIDTH_OFFSET_AMD	20
 
 #define RMID_VAL_ERROR			BIT_ULL(63)
@@ -210,29 +204,6 @@ struct rdtgroup {
 	struct pseudo_lock_region	*plr;
 };
 
-/* rdtgroup.flags */
-#define	RDT_DELETED		1
-
-/* rftype.flags */
-#define RFTYPE_FLAGS_CPUS_LIST	1
-
-/*
- * Define the file type flags for base and info directories.
- */
-#define RFTYPE_INFO			BIT(0)
-#define RFTYPE_BASE			BIT(1)
-#define RFTYPE_CTRL			BIT(4)
-#define RFTYPE_MON			BIT(5)
-#define RFTYPE_TOP			BIT(6)
-#define RFTYPE_RES_CACHE		BIT(8)
-#define RFTYPE_RES_MB			BIT(9)
-#define RFTYPE_DEBUG			BIT(10)
-#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
-#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
-#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
-#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
-#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
-
 /* List of all resource groups */
 extern struct list_head rdt_all_groups;
 
@@ -370,13 +341,6 @@ static inline struct rdt_resource *resctrl_inc(struct rdt_resource *res)
 	return &hw_res->r_resctrl;
 }
 
-static inline bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level l)
-{
-	return rdt_resources_all[l].cdp_enabled;
-}
-
-int resctrl_arch_set_cdp_enabled(enum resctrl_res_level l, bool enable);
-
 /*
  * To return the common struct rdt_resource, which is contained in struct
  * rdt_hw_resource, walk the resctrl member of struct rdt_hw_resource.
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index f786ffceeda3..00cc0457af50 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -41,6 +41,9 @@ int proc_resctrl_show(struct seq_file *m,
  */
 #define RESCTRL_MAX_CBM			32
 
+extern unsigned int resctrl_rmid_realloc_limit;
+extern unsigned int resctrl_rmid_realloc_threshold;
+
 /**
  * struct pseudo_lock_region - pseudo-lock region information
  * @s:			Resctrl schema for the resource to which this
diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
index 4788bd95dac6..fe0b10b589c0 100644
--- a/include/linux/resctrl_types.h
+++ b/include/linux/resctrl_types.h
@@ -7,6 +7,36 @@
 #ifndef __LINUX_RESCTRL_TYPES_H
 #define __LINUX_RESCTRL_TYPES_H
 
+#define CQM_LIMBOCHECK_INTERVAL	1000
+
+#define MBM_CNTR_WIDTH_BASE		24
+#define MBM_OVERFLOW_INTERVAL		1000
+#define MAX_MBA_BW			100u
+#define MBA_IS_LINEAR			0x4
+
+/* rdtgroup.flags */
+#define	RDT_DELETED		1
+
+/* rftype.flags */
+#define RFTYPE_FLAGS_CPUS_LIST	1
+
+/*
+ * Define the file type flags for base and info directories.
+ */
+#define RFTYPE_INFO			BIT(0)
+#define RFTYPE_BASE			BIT(1)
+#define RFTYPE_CTRL			BIT(4)
+#define RFTYPE_MON			BIT(5)
+#define RFTYPE_TOP			BIT(6)
+#define RFTYPE_RES_CACHE		BIT(8)
+#define RFTYPE_RES_MB			BIT(9)
+#define RFTYPE_DEBUG			BIT(10)
+#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
+#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
+#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
+#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
+#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
+
 /* Reads to Local DRAM Memory */
 #define READS_TO_LOCAL_MEM		BIT(0)
 
-- 
2.39.2

RE: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Shaopeng Tan (Fujitsu)]

Hello James

> Once the filesystem parts of resctrl move to fs/resctrl, it cannot rely on
> definitions in x86's internal.h.
> 
> Move definitions in internal.h that need to be shared between the filesystem
> and architecture code to header files that fs/resctrl can include.
> 
> Doing this separately means the filesystem code only moves between files of
> the same name, instead of having these changes mixed in too.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/include/asm/resctrl.h         |  3 +++
>  arch/x86/kernel/cpu/resctrl/core.c     |  5 ++++
>  arch/x86/kernel/cpu/resctrl/internal.h | 36 --------------------------
>  include/linux/resctrl.h                |  3 +++
>  include/linux/resctrl_types.h          | 30
> +++++++++++++++++++++
>  5 files changed, 41 insertions(+), 36 deletions(-)
> 
> diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
> index 491342f56811..746431c66fc4 100644
> --- a/arch/x86/include/asm/resctrl.h
> +++ b/arch/x86/include/asm/resctrl.h
> @@ -218,6 +218,9 @@ int resctrl_arch_measure_l2_residency(void *_plr);  int
> resctrl_arch_measure_l3_residency(void *_plr);  void
> resctrl_cpu_detect(struct cpuinfo_x86 *c);
> 
> +bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level l); int
> +resctrl_arch_set_cdp_enabled(enum resctrl_res_level l, bool enable);
> +
>  #else
> 
>  static inline void resctrl_arch_sched_in(struct task_struct *tsk) {} diff --git
> a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index f94ad04023c3..c0fb2e22e110 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -306,6 +306,11 @@ static void rdt_get_cdp_l2_config(void)
>  	rdt_get_cdp_config(RDT_RESOURCE_L2);
>  }
> 
> +bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level l) {
> +	return rdt_resources_all[l].cdp_enabled; }
> +
>  static void
>  mba_wrmsr_amd(struct rdt_domain *d, struct msr_param *m, struct
> rdt_resource *r)  { diff --git a/arch/x86/kernel/cpu/resctrl/internal.h
> b/arch/x86/kernel/cpu/resctrl/internal.h
> index 56218193a8ba..0f7e3f10941b 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -15,12 +15,6 @@
> 
>  #define L2_QOS_CDP_ENABLE		0x01ULL
> 
> -#define CQM_LIMBOCHECK_INTERVAL	1000
> -
> -#define MBM_CNTR_WIDTH_BASE		24
> -#define MBM_OVERFLOW_INTERVAL		1000
> -#define MAX_MBA_BW			100u
> -#define MBA_IS_LINEAR			0x4
>  #define MBM_CNTR_WIDTH_OFFSET_AMD	20
> 
>  #define RMID_VAL_ERROR			BIT_ULL(63)
> @@ -210,29 +204,6 @@ struct rdtgroup {
>  	struct pseudo_lock_region	*plr;
>  };
> 
> -/* rdtgroup.flags */
> -#define	RDT_DELETED		1
> -
> -/* rftype.flags */
> -#define RFTYPE_FLAGS_CPUS_LIST	1
> -
> -/*
> - * Define the file type flags for base and info directories.
> - */
> -#define RFTYPE_INFO			BIT(0)
> -#define RFTYPE_BASE			BIT(1)
> -#define RFTYPE_CTRL			BIT(4)
> -#define RFTYPE_MON			BIT(5)
> -#define RFTYPE_TOP			BIT(6)
> -#define RFTYPE_RES_CACHE		BIT(8)
> -#define RFTYPE_RES_MB			BIT(9)
> -#define RFTYPE_DEBUG			BIT(10)
> -#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
> -#define RFTYPE_MON_INFO			(RFTYPE_INFO |
> RFTYPE_MON)
> -#define RFTYPE_TOP_INFO			(RFTYPE_INFO |
> RFTYPE_TOP)
> -#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
> -#define RFTYPE_MON_BASE			(RFTYPE_BASE |
> RFTYPE_MON)
> -
>  /* List of all resource groups */
>  extern struct list_head rdt_all_groups;
> 
> @@ -370,13 +341,6 @@ static inline struct rdt_resource *resctrl_inc(struct
> rdt_resource *res)
>  	return &hw_res->r_resctrl;
>  }
> 
> -static inline bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level l) -{
> -	return rdt_resources_all[l].cdp_enabled;
> -}
> -
> -int resctrl_arch_set_cdp_enabled(enum resctrl_res_level l, bool enable);
> -
>  /*
>   * To return the common struct rdt_resource, which is contained in struct
>   * rdt_hw_resource, walk the resctrl member of struct rdt_hw_resource.
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index
> f786ffceeda3..00cc0457af50 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -41,6 +41,9 @@ int proc_resctrl_show(struct seq_file *m,
>   */
>  #define RESCTRL_MAX_CBM			32
> 
> +extern unsigned int resctrl_rmid_realloc_limit; extern unsigned int
> +resctrl_rmid_realloc_threshold;
> +

These two variables has been defined.
 44 extern unsigned int resctrl_rmid_realloc_limit;
 45 extern unsigned int resctrl_rmid_realloc_threshold;
400 extern unsigned int resctrl_rmid_realloc_threshold;
401 extern unsigned int resctrl_rmid_realloc_limit;

Best regards,
Shaopeng TAN

>  /**
>   * struct pseudo_lock_region - pseudo-lock region information
>   * @s:			Resctrl schema for the resource to which this
> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> index 4788bd95dac6..fe0b10b589c0 100644
> --- a/include/linux/resctrl_types.h
> +++ b/include/linux/resctrl_types.h
> @@ -7,6 +7,36 @@
>  #ifndef __LINUX_RESCTRL_TYPES_H
>  #define __LINUX_RESCTRL_TYPES_H
> 
> +#define CQM_LIMBOCHECK_INTERVAL	1000
> +
> +#define MBM_CNTR_WIDTH_BASE		24
> +#define MBM_OVERFLOW_INTERVAL		1000
> +#define MAX_MBA_BW			100u
> +#define MBA_IS_LINEAR			0x4
> +
> +/* rdtgroup.flags */
> +#define	RDT_DELETED		1
> +
> +/* rftype.flags */
> +#define RFTYPE_FLAGS_CPUS_LIST	1
> +
> +/*
> + * Define the file type flags for base and info directories.
> + */
> +#define RFTYPE_INFO			BIT(0)
> +#define RFTYPE_BASE			BIT(1)
> +#define RFTYPE_CTRL			BIT(4)
> +#define RFTYPE_MON			BIT(5)
> +#define RFTYPE_TOP			BIT(6)
> +#define RFTYPE_RES_CACHE		BIT(8)
> +#define RFTYPE_RES_MB			BIT(9)
> +#define RFTYPE_DEBUG			BIT(10)
> +#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
> +#define RFTYPE_MON_INFO			(RFTYPE_INFO |
> RFTYPE_MON)
> +#define RFTYPE_TOP_INFO			(RFTYPE_INFO |
> RFTYPE_TOP)
> +#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
> +#define RFTYPE_MON_BASE			(RFTYPE_BASE |
> RFTYPE_MON)
> +
>  /* Reads to Local DRAM Memory */
>  #define READS_TO_LOCAL_MEM		BIT(0)
> 
> --
> 2.39.2

Re: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Dave Martin]

On Thu, Apr 04, 2024 at 07:43:03AM +0000, Shaopeng Tan (Fujitsu) wrote:
> Hello James
> 
> > Once the filesystem parts of resctrl move to fs/resctrl, it cannot rely on
> > definitions in x86's internal.h.
> > 
> > Move definitions in internal.h that need to be shared between the filesystem
> > and architecture code to header files that fs/resctrl can include.
> > 
> > Doing this separately means the filesystem code only moves between files of
> > the same name, instead of having these changes mixed in too.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> >  arch/x86/include/asm/resctrl.h         |  3 +++
> >  arch/x86/kernel/cpu/resctrl/core.c     |  5 ++++
> >  arch/x86/kernel/cpu/resctrl/internal.h | 36 --------------------------
> >  include/linux/resctrl.h                |  3 +++
> >  include/linux/resctrl_types.h          | 30
> > +++++++++++++++++++++
> >  5 files changed, 41 insertions(+), 36 deletions(-)

[...]

> > diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index
> > f786ffceeda3..00cc0457af50 100644
> > --- a/include/linux/resctrl.h
> > +++ b/include/linux/resctrl.h
> > @@ -41,6 +41,9 @@ int proc_resctrl_show(struct seq_file *m,
> >   */
> >  #define RESCTRL_MAX_CBM			32
> > 
> > +extern unsigned int resctrl_rmid_realloc_limit; extern unsigned int
> > +resctrl_rmid_realloc_threshold;
> > +

[FYI, your mailer or editor seems to have messed this patch up a bit in
your reply...]

> These two variables has been defined.
>  44 extern unsigned int resctrl_rmid_realloc_limit;
>  45 extern unsigned int resctrl_rmid_realloc_threshold;
> 400 extern unsigned int resctrl_rmid_realloc_threshold;
> 401 extern unsigned int resctrl_rmid_realloc_limit;
> 
> Best regards,
> Shaopeng TAN

[...]

Strange.  This looks unintentional and there don't seem to be any
relevant #ifdefs or references to these variables in the header that
might justify reordering these declarations.

I'll propose to James that the addition of these duplicates be reverted
out, so long as it doesn't break anything.

Cheers
---Dave

Re: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Reinette Chatre]

Hi James,

On 3/21/2024 9:51 AM, James Morse wrote:
...
> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> index 4788bd95dac6..fe0b10b589c0 100644
> --- a/include/linux/resctrl_types.h
> +++ b/include/linux/resctrl_types.h
> @@ -7,6 +7,36 @@
>  #ifndef __LINUX_RESCTRL_TYPES_H
>  #define __LINUX_RESCTRL_TYPES_H
>  
> +#define CQM_LIMBOCHECK_INTERVAL	1000
> +
> +#define MBM_CNTR_WIDTH_BASE		24
> +#define MBM_OVERFLOW_INTERVAL		1000
> +#define MAX_MBA_BW			100u
> +#define MBA_IS_LINEAR			0x4
> +
> +/* rdtgroup.flags */
> +#define	RDT_DELETED		1
> +
> +/* rftype.flags */
> +#define RFTYPE_FLAGS_CPUS_LIST	1
> +
> +/*
> + * Define the file type flags for base and info directories.
> + */
> +#define RFTYPE_INFO			BIT(0)
> +#define RFTYPE_BASE			BIT(1)
> +#define RFTYPE_CTRL			BIT(4)
> +#define RFTYPE_MON			BIT(5)
> +#define RFTYPE_TOP			BIT(6)
> +#define RFTYPE_RES_CACHE		BIT(8)
> +#define RFTYPE_RES_MB			BIT(9)
> +#define RFTYPE_DEBUG			BIT(10)
> +#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
> +#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
> +#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
> +#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
> +#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
> +
>  /* Reads to Local DRAM Memory */
>  #define READS_TO_LOCAL_MEM		BIT(0)
>  

Not all these new seem to belong in this file. Could you please confirm?

For example:
Earlier in series it was mentioned that struct rdtgroup is private to the
fs so having RDT_DELETED is unexpected as it implies access to struct rdtgroup.

CQM_LIMBOCHECK_INTERVAL seems private to the fs code, so too
RFTYPE_FLAGS_CPUS_LIST.

Reinette

Re: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Dave Martin]

On Mon, Apr 08, 2024 at 08:42:00PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:51 AM, James Morse wrote:
> ..
> > diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> > index 4788bd95dac6..fe0b10b589c0 100644
> > --- a/include/linux/resctrl_types.h
> > +++ b/include/linux/resctrl_types.h
> > @@ -7,6 +7,36 @@
> >  #ifndef __LINUX_RESCTRL_TYPES_H
> >  #define __LINUX_RESCTRL_TYPES_H
> >  
> > +#define CQM_LIMBOCHECK_INTERVAL	1000
> > +
> > +#define MBM_CNTR_WIDTH_BASE		24
> > +#define MBM_OVERFLOW_INTERVAL		1000
> > +#define MAX_MBA_BW			100u
> > +#define MBA_IS_LINEAR			0x4
> > +
> > +/* rdtgroup.flags */
> > +#define	RDT_DELETED		1
> > +
> > +/* rftype.flags */
> > +#define RFTYPE_FLAGS_CPUS_LIST	1
> > +
> > +/*
> > + * Define the file type flags for base and info directories.
> > + */
> > +#define RFTYPE_INFO			BIT(0)
> > +#define RFTYPE_BASE			BIT(1)
> > +#define RFTYPE_CTRL			BIT(4)
> > +#define RFTYPE_MON			BIT(5)
> > +#define RFTYPE_TOP			BIT(6)
> > +#define RFTYPE_RES_CACHE		BIT(8)
> > +#define RFTYPE_RES_MB			BIT(9)
> > +#define RFTYPE_DEBUG			BIT(10)
> > +#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
> > +#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
> > +#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
> > +#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
> > +#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
> > +
> >  /* Reads to Local DRAM Memory */
> >  #define READS_TO_LOCAL_MEM		BIT(0)
> >  
> 
> Not all these new seem to belong in this file. Could you please confirm?
> 
> For example:
> Earlier in series it was mentioned that struct rdtgroup is private to the
> fs so having RDT_DELETED is unexpected as it implies access to struct rdtgroup.
> 
> CQM_LIMBOCHECK_INTERVAL seems private to the fs code, so too
> RFTYPE_FLAGS_CPUS_LIST.
> 
> Reinette
> 

I'll flag this for James to review.

These have to be moved out of the x86 private headers, but you're right
that some of them seem logically private to the resctrl core.

I guess some of these could move to fs/resctrl/internal.h?

OTOH, might it be preferable to keep all the flag definitions for a
given member together for ease of maintenance, even if some are for
resctrl internal use only?

Cheers
---Dave

Re: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:28 AM, Dave Martin wrote:
> On Mon, Apr 08, 2024 at 08:42:00PM -0700, Reinette Chatre wrote:
>> Hi James,
>>
>> On 3/21/2024 9:51 AM, James Morse wrote:
>> ..
>>> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
>>> index 4788bd95dac6..fe0b10b589c0 100644
>>> --- a/include/linux/resctrl_types.h
>>> +++ b/include/linux/resctrl_types.h
>>> @@ -7,6 +7,36 @@
>>>  #ifndef __LINUX_RESCTRL_TYPES_H
>>>  #define __LINUX_RESCTRL_TYPES_H
>>>  
>>> +#define CQM_LIMBOCHECK_INTERVAL	1000
>>> +
>>> +#define MBM_CNTR_WIDTH_BASE		24
>>> +#define MBM_OVERFLOW_INTERVAL		1000
>>> +#define MAX_MBA_BW			100u
>>> +#define MBA_IS_LINEAR			0x4
>>> +
>>> +/* rdtgroup.flags */
>>> +#define	RDT_DELETED		1
>>> +
>>> +/* rftype.flags */
>>> +#define RFTYPE_FLAGS_CPUS_LIST	1
>>> +
>>> +/*
>>> + * Define the file type flags for base and info directories.
>>> + */
>>> +#define RFTYPE_INFO			BIT(0)
>>> +#define RFTYPE_BASE			BIT(1)
>>> +#define RFTYPE_CTRL			BIT(4)
>>> +#define RFTYPE_MON			BIT(5)
>>> +#define RFTYPE_TOP			BIT(6)
>>> +#define RFTYPE_RES_CACHE		BIT(8)
>>> +#define RFTYPE_RES_MB			BIT(9)
>>> +#define RFTYPE_DEBUG			BIT(10)
>>> +#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
>>> +#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
>>> +#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
>>> +#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
>>> +#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
>>> +
>>>  /* Reads to Local DRAM Memory */
>>>  #define READS_TO_LOCAL_MEM		BIT(0)
>>>  
>>
>> Not all these new seem to belong in this file. Could you please confirm?
>>
>> For example:
>> Earlier in series it was mentioned that struct rdtgroup is private to the
>> fs so having RDT_DELETED is unexpected as it implies access to struct rdtgroup.
>>
>> CQM_LIMBOCHECK_INTERVAL seems private to the fs code, so too
>> RFTYPE_FLAGS_CPUS_LIST.
>>
>> Reinette
>>
> 
> I'll flag this for James to review.
> 
> These have to be moved out of the x86 private headers, but you're right
> that some of them seem logically private to the resctrl core.
> 
> I guess some of these could move to fs/resctrl/internal.h?

It looks to me that way.

> 
> OTOH, might it be preferable to keep all the flag definitions for a
> given member together for ease of maintenance, even if some are for
> resctrl internal use only?

Indeed, those RFTYPE flags really seem to be fs code but I agree that
architectures' use of RFTYPE_RES_CACHE and RFTYPE_RES_MB does make this
complicated and having these in a central place is reasonable to me.

Reinette

Re: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Dave Martin]

On Thu, Apr 11, 2024 at 10:43:55AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:28 AM, Dave Martin wrote:
> > On Mon, Apr 08, 2024 at 08:42:00PM -0700, Reinette Chatre wrote:
> >> Hi James,
> >>
> >> On 3/21/2024 9:51 AM, James Morse wrote:
> >> ..
> >>> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> >>> index 4788bd95dac6..fe0b10b589c0 100644
> >>> --- a/include/linux/resctrl_types.h
> >>> +++ b/include/linux/resctrl_types.h
> >>> @@ -7,6 +7,36 @@
> >>>  #ifndef __LINUX_RESCTRL_TYPES_H
> >>>  #define __LINUX_RESCTRL_TYPES_H
> >>>  
> >>> +#define CQM_LIMBOCHECK_INTERVAL	1000
> >>> +
> >>> +#define MBM_CNTR_WIDTH_BASE		24
> >>> +#define MBM_OVERFLOW_INTERVAL		1000
> >>> +#define MAX_MBA_BW			100u
> >>> +#define MBA_IS_LINEAR			0x4
> >>> +
> >>> +/* rdtgroup.flags */
> >>> +#define	RDT_DELETED		1
> >>> +
> >>> +/* rftype.flags */
> >>> +#define RFTYPE_FLAGS_CPUS_LIST	1
> >>> +
> >>> +/*
> >>> + * Define the file type flags for base and info directories.
> >>> + */
> >>> +#define RFTYPE_INFO			BIT(0)
> >>> +#define RFTYPE_BASE			BIT(1)
> >>> +#define RFTYPE_CTRL			BIT(4)
> >>> +#define RFTYPE_MON			BIT(5)
> >>> +#define RFTYPE_TOP			BIT(6)
> >>> +#define RFTYPE_RES_CACHE		BIT(8)
> >>> +#define RFTYPE_RES_MB			BIT(9)
> >>> +#define RFTYPE_DEBUG			BIT(10)
> >>> +#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
> >>> +#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
> >>> +#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
> >>> +#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
> >>> +#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
> >>> +
> >>>  /* Reads to Local DRAM Memory */
> >>>  #define READS_TO_LOCAL_MEM		BIT(0)
> >>>  
> >>
> >> Not all these new seem to belong in this file. Could you please confirm?
> >>
> >> For example:
> >> Earlier in series it was mentioned that struct rdtgroup is private to the
> >> fs so having RDT_DELETED is unexpected as it implies access to struct rdtgroup.
> >>
> >> CQM_LIMBOCHECK_INTERVAL seems private to the fs code, so too
> >> RFTYPE_FLAGS_CPUS_LIST.
> >>
> >> Reinette
> >>
> > 
> > I'll flag this for James to review.
> > 
> > These have to be moved out of the x86 private headers, but you're right
> > that some of them seem logically private to the resctrl core.
> > 
> > I guess some of these could move to fs/resctrl/internal.h?
> 
> It looks to me that way.
> 
> > 
> > OTOH, might it be preferable to keep all the flag definitions for a
> > given member together for ease of maintenance, even if some are for
> > resctrl internal use only?
> 
> Indeed, those RFTYPE flags really seem to be fs code but I agree that
> architectures' use of RFTYPE_RES_CACHE and RFTYPE_RES_MB does make this
> complicated and having these in a central place is reasonable to me.
> 
> Reinette

Maybe we could split these into two groups, and clearly comment the ones
that have no user outside resctrl as internal use only?

That's not as clean as removing those definitions from a shared header,
but at least would help document the issue until/unless a better
solution is found...

Cheers
---Dave

Re: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Reinette Chatre]

Hi Dave,

On 4/12/2024 9:20 AM, Dave Martin wrote:
> On Thu, Apr 11, 2024 at 10:43:55AM -0700, Reinette Chatre wrote:
>> Hi Dave,
>>
>> On 4/11/2024 7:28 AM, Dave Martin wrote:
>>> On Mon, Apr 08, 2024 at 08:42:00PM -0700, Reinette Chatre wrote:
>>>> Hi James,
>>>>
>>>> On 3/21/2024 9:51 AM, James Morse wrote:
>>>> ..
>>>>> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
>>>>> index 4788bd95dac6..fe0b10b589c0 100644
>>>>> --- a/include/linux/resctrl_types.h
>>>>> +++ b/include/linux/resctrl_types.h
>>>>> @@ -7,6 +7,36 @@
>>>>>  #ifndef __LINUX_RESCTRL_TYPES_H
>>>>>  #define __LINUX_RESCTRL_TYPES_H
>>>>>  
>>>>> +#define CQM_LIMBOCHECK_INTERVAL	1000
>>>>> +
>>>>> +#define MBM_CNTR_WIDTH_BASE		24
>>>>> +#define MBM_OVERFLOW_INTERVAL		1000
>>>>> +#define MAX_MBA_BW			100u
>>>>> +#define MBA_IS_LINEAR			0x4
>>>>> +
>>>>> +/* rdtgroup.flags */
>>>>> +#define	RDT_DELETED		1
>>>>> +
>>>>> +/* rftype.flags */
>>>>> +#define RFTYPE_FLAGS_CPUS_LIST	1
>>>>> +
>>>>> +/*
>>>>> + * Define the file type flags for base and info directories.
>>>>> + */
>>>>> +#define RFTYPE_INFO			BIT(0)
>>>>> +#define RFTYPE_BASE			BIT(1)
>>>>> +#define RFTYPE_CTRL			BIT(4)
>>>>> +#define RFTYPE_MON			BIT(5)
>>>>> +#define RFTYPE_TOP			BIT(6)
>>>>> +#define RFTYPE_RES_CACHE		BIT(8)
>>>>> +#define RFTYPE_RES_MB			BIT(9)
>>>>> +#define RFTYPE_DEBUG			BIT(10)
>>>>> +#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
>>>>> +#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
>>>>> +#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
>>>>> +#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
>>>>> +#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
>>>>> +
>>>>>  /* Reads to Local DRAM Memory */
>>>>>  #define READS_TO_LOCAL_MEM		BIT(0)
>>>>>  
>>>>
>>>> Not all these new seem to belong in this file. Could you please confirm?
>>>>
>>>> For example:
>>>> Earlier in series it was mentioned that struct rdtgroup is private to the
>>>> fs so having RDT_DELETED is unexpected as it implies access to struct rdtgroup.
>>>>
>>>> CQM_LIMBOCHECK_INTERVAL seems private to the fs code, so too
>>>> RFTYPE_FLAGS_CPUS_LIST.
>>>>
>>>> Reinette
>>>>
>>>
>>> I'll flag this for James to review.
>>>
>>> These have to be moved out of the x86 private headers, but you're right
>>> that some of them seem logically private to the resctrl core.
>>>
>>> I guess some of these could move to fs/resctrl/internal.h?
>>
>> It looks to me that way.
>>
>>>
>>> OTOH, might it be preferable to keep all the flag definitions for a
>>> given member together for ease of maintenance, even if some are for
>>> resctrl internal use only?
>>
>> Indeed, those RFTYPE flags really seem to be fs code but I agree that
>> architectures' use of RFTYPE_RES_CACHE and RFTYPE_RES_MB does make this
>> complicated and having these in a central place is reasonable to me.
>>
>> Reinette
> 
> Maybe we could split these into two groups, and clearly comment the ones
> that have no user outside resctrl as internal use only?

Another option to consider, which I think you hinted about earlier, is
to add an enum that maps to the RFTYPE_RES_CACHE and RFTYPE_RES_MB
flags. Just like, for example, RDTCTRL_GROUP maps to RFTYPE_CTRL.
The new enum can then be used similar to enum rdt_group_type to pick the
appropriate files based on RFTYPE_RES_CACHE or RFTYPE_RES_MB. 

> 
> That's not as clean as removing those definitions from a shared header,
> but at least would help document the issue until/unless a better
> solution is found...
> 
> Cheers
> ---Dave

Re: [PATCH v1 30/31] x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl

[Dave Martin]

On Mon, Apr 15, 2024 at 11:03:32AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/12/2024 9:20 AM, Dave Martin wrote:
> > On Thu, Apr 11, 2024 at 10:43:55AM -0700, Reinette Chatre wrote:
> >> Hi Dave,
> >>
> >> On 4/11/2024 7:28 AM, Dave Martin wrote:
> >>> On Mon, Apr 08, 2024 at 08:42:00PM -0700, Reinette Chatre wrote:
> >>>> Hi James,
> >>>>
> >>>> On 3/21/2024 9:51 AM, James Morse wrote:
> >>>> ..
> >>>>> diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h
> >>>>> index 4788bd95dac6..fe0b10b589c0 100644
> >>>>> --- a/include/linux/resctrl_types.h
> >>>>> +++ b/include/linux/resctrl_types.h
> >>>>> @@ -7,6 +7,36 @@
> >>>>>  #ifndef __LINUX_RESCTRL_TYPES_H
> >>>>>  #define __LINUX_RESCTRL_TYPES_H
> >>>>>  
> >>>>> +#define CQM_LIMBOCHECK_INTERVAL	1000
> >>>>> +
> >>>>> +#define MBM_CNTR_WIDTH_BASE		24
> >>>>> +#define MBM_OVERFLOW_INTERVAL		1000
> >>>>> +#define MAX_MBA_BW			100u
> >>>>> +#define MBA_IS_LINEAR			0x4
> >>>>> +
> >>>>> +/* rdtgroup.flags */
> >>>>> +#define	RDT_DELETED		1
> >>>>> +
> >>>>> +/* rftype.flags */
> >>>>> +#define RFTYPE_FLAGS_CPUS_LIST	1
> >>>>> +
> >>>>> +/*
> >>>>> + * Define the file type flags for base and info directories.
> >>>>> + */
> >>>>> +#define RFTYPE_INFO			BIT(0)
> >>>>> +#define RFTYPE_BASE			BIT(1)
> >>>>> +#define RFTYPE_CTRL			BIT(4)
> >>>>> +#define RFTYPE_MON			BIT(5)
> >>>>> +#define RFTYPE_TOP			BIT(6)
> >>>>> +#define RFTYPE_RES_CACHE		BIT(8)
> >>>>> +#define RFTYPE_RES_MB			BIT(9)
> >>>>> +#define RFTYPE_DEBUG			BIT(10)
> >>>>> +#define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
> >>>>> +#define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
> >>>>> +#define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
> >>>>> +#define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
> >>>>> +#define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
> >>>>> +
> >>>>>  /* Reads to Local DRAM Memory */
> >>>>>  #define READS_TO_LOCAL_MEM		BIT(0)
> >>>>>  
> >>>>
> >>>> Not all these new seem to belong in this file. Could you please confirm?
> >>>>
> >>>> For example:
> >>>> Earlier in series it was mentioned that struct rdtgroup is private to the
> >>>> fs so having RDT_DELETED is unexpected as it implies access to struct rdtgroup.
> >>>>
> >>>> CQM_LIMBOCHECK_INTERVAL seems private to the fs code, so too
> >>>> RFTYPE_FLAGS_CPUS_LIST.
> >>>>
> >>>> Reinette
> >>>>
> >>>
> >>> I'll flag this for James to review.
> >>>
> >>> These have to be moved out of the x86 private headers, but you're right
> >>> that some of them seem logically private to the resctrl core.
> >>>
> >>> I guess some of these could move to fs/resctrl/internal.h?
> >>
> >> It looks to me that way.
> >>
> >>>
> >>> OTOH, might it be preferable to keep all the flag definitions for a
> >>> given member together for ease of maintenance, even if some are for
> >>> resctrl internal use only?
> >>
> >> Indeed, those RFTYPE flags really seem to be fs code but I agree that
> >> architectures' use of RFTYPE_RES_CACHE and RFTYPE_RES_MB does make this
> >> complicated and having these in a central place is reasonable to me.
> >>
> >> Reinette
> > 
> > Maybe we could split these into two groups, and clearly comment the ones
> > that have no user outside resctrl as internal use only?
> 
> Another option to consider, which I think you hinted about earlier, is
> to add an enum that maps to the RFTYPE_RES_CACHE and RFTYPE_RES_MB
> flags. Just like, for example, RDTCTRL_GROUP maps to RFTYPE_CTRL.
> The new enum can then be used similar to enum rdt_group_type to pick the
> appropriate files based on RFTYPE_RES_CACHE or RFTYPE_RES_MB. 
> 
> > 
> > That's not as clean as removing those definitions from a shared header,
> > but at least would help document the issue until/unless a better
> > solution is found...
> > 
> > Cheers
> > ---Dave

(Note, I seem to have responded to that suggestion via my reply on patch 3,
rather than here:

"x86/resctrl: Move ctrlval string parsing policy away from the arch code"
https://lore.kernel.org/lkml/Zh6kRMkqVpu0Km4l@e133380.arm.com/ )

Cheers
---Dave

[PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[James Morse]

resctrl is linux's defacto interface for managing cache and bandwidth
policies for groups of tasks.

To allow other architectures to make use of this pseudo filesystem,
move it live in /fs/resctrl instead of /arch/x86.

This move leaves behind the parts of resctrl that form the architecture
interface for x86.

Signed-off-by: James Morse <james.morse@arm.com>
---
Discussion needed on how/when to merge this, as it would conflict with
all outstanding series. It's probably worth deferring to some opportune
time, but is included here for illustration.
---
 arch/x86/kernel/cpu/resctrl/core.c        |   15 -
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  505 ---
 arch/x86/kernel/cpu/resctrl/internal.h    |  310 --
 arch/x86/kernel/cpu/resctrl/monitor.c     |  821 -----
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1093 ------
 arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3994 --------------------
 fs/resctrl/ctrlmondata.c                  |  527 +++
 fs/resctrl/internal.h                     |  340 ++
 fs/resctrl/monitor.c                      |  843 +++++
 fs/resctrl/psuedo_lock.c                  | 1122 ++++++
 fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
 11 files changed, 6845 insertions(+), 6738 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index c0fb2e22e110..8ddfebd5f008 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -164,21 +164,6 @@ static inline void cache_alloc_hsw_probe(void)
 	rdt_alloc_capable = true;
 }
 
-bool is_mba_sc(struct rdt_resource *r)
-{
-	if (!r)
-		r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
-
-	/*
-	 * The software controller support is only applicable to MBA resource.
-	 * Make sure to check for resource type.
-	 */
-	if (r->rid != RDT_RESOURCE_MBA)
-		return false;
-
-	return r->membw.mba_sc;
-}
-
 /*
  * rdt_get_mb_table() - get a mapping of bandwidth(b/w) percentage values
  * exposed to user interface and the h/w understandable delay values.
diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index b4627ae19291..c5c3eaea27b6 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -23,260 +23,6 @@
 
 #include "internal.h"
 
-struct rdt_parse_data {
-	struct rdtgroup		*rdtgrp;
-	char			*buf;
-};
-
-typedef int (ctrlval_parser_t)(struct rdt_parse_data *data,
-			       struct resctrl_schema *s,
-			       struct rdt_domain *d);
-
-/*
- * Check whether MBA bandwidth percentage value is correct. The value is
- * checked against the minimum and max bandwidth values specified by the
- * hardware. The allocated bandwidth percentage is rounded to the next
- * control step available on the hardware.
- */
-static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
-{
-	unsigned long bw;
-	int ret;
-
-	/*
-	 * Only linear delay values is supported for current Intel SKUs.
-	 */
-	if (!r->membw.delay_linear && r->membw.arch_needs_linear) {
-		rdt_last_cmd_puts("No support for non-linear MB domains\n");
-		return false;
-	}
-
-	ret = kstrtoul(buf, 10, &bw);
-	if (ret) {
-		rdt_last_cmd_printf("Non-decimal digit in MB value %s\n", buf);
-		return false;
-	}
-
-	if ((bw < r->membw.min_bw || bw > r->default_ctrl) &&
-	    !is_mba_sc(r)) {
-		rdt_last_cmd_printf("MB value %ld out of range [%d,%d]\n", bw,
-				    r->membw.min_bw, r->default_ctrl);
-		return false;
-	}
-
-	*data = roundup(bw, (unsigned long)r->membw.bw_gran);
-	return true;
-}
-
-static int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
-		    struct rdt_domain *d)
-{
-	struct resctrl_staged_config *cfg;
-	u32 closid = data->rdtgrp->closid;
-	struct rdt_resource *r = s->res;
-	unsigned long bw_val;
-
-	cfg = &d->staged_config[s->conf_type];
-	if (cfg->have_new_ctrl) {
-		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
-		return -EINVAL;
-	}
-
-	if (!bw_validate(data->buf, &bw_val, r))
-		return -EINVAL;
-
-	if (is_mba_sc(r)) {
-		d->mbps_val[closid] = bw_val;
-		return 0;
-	}
-
-	cfg->new_ctrl = bw_val;
-	cfg->have_new_ctrl = true;
-
-	return 0;
-}
-
-/*
- * Check whether a cache bit mask is valid.
- * On Intel CPUs, non-contiguous 1s value support is indicated by CPUID:
- *   - CPUID.0x10.1:ECX[3]: L3 non-contiguous 1s value supported if 1
- *   - CPUID.0x10.2:ECX[3]: L2 non-contiguous 1s value supported if 1
- *
- * Haswell does not support a non-contiguous 1s value and additionally
- * requires at least two bits set.
- * AMD allows non-contiguous bitmasks.
- */
-static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
-{
-	unsigned long first_bit, zero_bit, val;
-	unsigned int cbm_len = r->cache.cbm_len;
-	int ret;
-
-	ret = kstrtoul(buf, 16, &val);
-	if (ret) {
-		rdt_last_cmd_printf("Non-hex character in the mask %s\n", buf);
-		return false;
-	}
-
-	if ((r->cache.min_cbm_bits > 0 && val == 0) || val > r->default_ctrl) {
-		rdt_last_cmd_puts("Mask out of range\n");
-		return false;
-	}
-
-	first_bit = find_first_bit(&val, cbm_len);
-	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
-
-	/* Are non-contiguous bitmasks allowed? */
-	if (!r->cache.arch_has_sparse_bitmasks &&
-	    (find_next_bit(&val, cbm_len, zero_bit) < cbm_len)) {
-		rdt_last_cmd_printf("The mask %lx has non-consecutive 1-bits\n", val);
-		return false;
-	}
-
-	if ((zero_bit - first_bit) < r->cache.min_cbm_bits) {
-		rdt_last_cmd_printf("Need at least %d bits in the mask\n",
-				    r->cache.min_cbm_bits);
-		return false;
-	}
-
-	*data = val;
-	return true;
-}
-
-/*
- * Read one cache bit mask (hex). Check that it is valid for the current
- * resource type.
- */
-static int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
-		     struct rdt_domain *d)
-{
-	struct rdtgroup *rdtgrp = data->rdtgrp;
-	struct resctrl_staged_config *cfg;
-	struct rdt_resource *r = s->res;
-	u32 cbm_val;
-
-	cfg = &d->staged_config[s->conf_type];
-	if (cfg->have_new_ctrl) {
-		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
-		return -EINVAL;
-	}
-
-	/*
-	 * Cannot set up more than one pseudo-locked region in a cache
-	 * hierarchy.
-	 */
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
-	    rdtgroup_pseudo_locked_in_hierarchy(d)) {
-		rdt_last_cmd_puts("Pseudo-locked region in hierarchy\n");
-		return -EINVAL;
-	}
-
-	if (!cbm_validate(data->buf, &cbm_val, r))
-		return -EINVAL;
-
-	if ((rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
-	     rdtgrp->mode == RDT_MODE_SHAREABLE) &&
-	    rdtgroup_cbm_overlaps_pseudo_locked(d, cbm_val)) {
-		rdt_last_cmd_puts("CBM overlaps with pseudo-locked region\n");
-		return -EINVAL;
-	}
-
-	/*
-	 * The CBM may not overlap with the CBM of another closid if
-	 * either is exclusive.
-	 */
-	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, true)) {
-		rdt_last_cmd_puts("Overlaps with exclusive group\n");
-		return -EINVAL;
-	}
-
-	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, false)) {
-		if (rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
-		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-			rdt_last_cmd_puts("Overlaps with other group\n");
-			return -EINVAL;
-		}
-	}
-
-	cfg->new_ctrl = cbm_val;
-	cfg->have_new_ctrl = true;
-
-	return 0;
-}
-
-static ctrlval_parser_t *get_parser(struct rdt_resource *res)
-{
-	if (res->fflags & RFTYPE_RES_CACHE)
-		return &parse_cbm;
-	else
-		return &parse_bw;
-}
-
-/*
- * For each domain in this resource we expect to find a series of:
- *	id=mask
- * separated by ";". The "id" is in decimal, and must match one of
- * the "id"s for this resource.
- */
-static int parse_line(char *line, struct resctrl_schema *s,
-		      struct rdtgroup *rdtgrp)
-{
-	ctrlval_parser_t *parse_ctrlval = get_parser(s->res);
-	enum resctrl_conf_type t = s->conf_type;
-	struct resctrl_staged_config *cfg;
-	struct rdt_resource *r = s->res;
-	struct rdt_parse_data data;
-	char *dom = NULL, *id;
-	struct rdt_domain *d;
-	unsigned long dom_id;
-
-	/* Walking r->domains, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
-	    (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)) {
-		rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
-		return -EINVAL;
-	}
-
-next:
-	if (!line || line[0] == '\0')
-		return 0;
-	dom = strsep(&line, ";");
-	id = strsep(&dom, "=");
-	if (!dom || kstrtoul(id, 10, &dom_id)) {
-		rdt_last_cmd_puts("Missing '=' or non-numeric domain\n");
-		return -EINVAL;
-	}
-	dom = strim(dom);
-	list_for_each_entry(d, &r->domains, list) {
-		if (d->id == dom_id) {
-			data.buf = dom;
-			data.rdtgrp = rdtgrp;
-			if (parse_ctrlval(&data, s, d))
-				return -EINVAL;
-			if (rdtgrp->mode ==  RDT_MODE_PSEUDO_LOCKSETUP) {
-				cfg = &d->staged_config[t];
-				/*
-				 * In pseudo-locking setup mode and just
-				 * parsed a valid CBM that should be
-				 * pseudo-locked. Only one locked region per
-				 * resource group and domain so just do
-				 * the required initialization for single
-				 * region and return.
-				 */
-				rdtgrp->plr->s = s;
-				rdtgrp->plr->d = d;
-				rdtgrp->plr->cbm = cfg->new_ctrl;
-				d->plr = rdtgrp->plr;
-				return 0;
-			}
-			goto next;
-		}
-	}
-	return -EINVAL;
-}
-
 static bool apply_config(struct rdt_hw_domain *hw_dom,
 			 struct resctrl_staged_config *cfg, u32 idx,
 			 cpumask_var_t cpu_mask)
@@ -365,100 +111,6 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
 	return 0;
 }
 
-static int rdtgroup_parse_resource(char *resname, char *tok,
-				   struct rdtgroup *rdtgrp)
-{
-	struct resctrl_schema *s;
-
-	list_for_each_entry(s, &resctrl_schema_all, list) {
-		if (!strcmp(resname, s->name) && rdtgrp->closid < s->num_closid)
-			return parse_line(tok, s, rdtgrp);
-	}
-	rdt_last_cmd_printf("Unknown or unsupported resource name '%s'\n", resname);
-	return -EINVAL;
-}
-
-ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
-				char *buf, size_t nbytes, loff_t off)
-{
-	struct resctrl_schema *s;
-	struct rdtgroup *rdtgrp;
-	struct rdt_resource *r;
-	char *tok, *resname;
-	int ret = 0;
-
-	/* Valid input requires a trailing newline */
-	if (nbytes == 0 || buf[nbytes - 1] != '\n')
-		return -EINVAL;
-	buf[nbytes - 1] = '\0';
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (!rdtgrp) {
-		rdtgroup_kn_unlock(of->kn);
-		return -ENOENT;
-	}
-	rdt_last_cmd_clear();
-
-	/*
-	 * No changes to pseudo-locked region allowed. It has to be removed
-	 * and re-created instead.
-	 */
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
-		ret = -EINVAL;
-		rdt_last_cmd_puts("Resource group is pseudo-locked\n");
-		goto out;
-	}
-
-	rdt_staged_configs_clear();
-
-	while ((tok = strsep(&buf, "\n")) != NULL) {
-		resname = strim(strsep(&tok, ":"));
-		if (!tok) {
-			rdt_last_cmd_puts("Missing ':'\n");
-			ret = -EINVAL;
-			goto out;
-		}
-		if (tok[0] == '\0') {
-			rdt_last_cmd_printf("Missing '%s' value\n", resname);
-			ret = -EINVAL;
-			goto out;
-		}
-		ret = rdtgroup_parse_resource(resname, tok, rdtgrp);
-		if (ret)
-			goto out;
-	}
-
-	list_for_each_entry(s, &resctrl_schema_all, list) {
-		r = s->res;
-
-		/*
-		 * Writes to mba_sc resources update the software controller,
-		 * not the control MSR.
-		 */
-		if (is_mba_sc(r))
-			continue;
-
-		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
-		if (ret)
-			goto out;
-	}
-
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-		/*
-		 * If pseudo-locking fails we keep the resource group in
-		 * mode RDT_MODE_PSEUDO_LOCKSETUP with its class of service
-		 * active and updated for just the domain the pseudo-locked
-		 * region was requested for.
-		 */
-		ret = rdtgroup_pseudo_lock_create(rdtgrp);
-	}
-
-out:
-	rdt_staged_configs_clear();
-	rdtgroup_kn_unlock(of->kn);
-	return ret ?: nbytes;
-}
-
 u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
 			    u32 closid, enum resctrl_conf_type type)
 {
@@ -467,160 +119,3 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
 
 	return hw_dom->ctrl_val[idx];
 }
-
-static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int closid)
-{
-	struct rdt_resource *r = schema->res;
-	struct rdt_domain *dom;
-	bool sep = false;
-	u32 ctrl_val;
-
-	/* Walking r->domains, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-	seq_printf(s, "%*s:", max_name_width, schema->name);
-	list_for_each_entry(dom, &r->domains, list) {
-		if (sep)
-			seq_puts(s, ";");
-
-		if (is_mba_sc(r))
-			ctrl_val = dom->mbps_val[closid];
-		else
-			ctrl_val = resctrl_arch_get_config(r, dom, closid,
-							   schema->conf_type);
-
-		seq_printf(s, r->format_str, dom->id, max_data_width,
-			   ctrl_val);
-		sep = true;
-	}
-	seq_puts(s, "\n");
-}
-
-int rdtgroup_schemata_show(struct kernfs_open_file *of,
-			   struct seq_file *s, void *v)
-{
-	struct resctrl_schema *schema;
-	struct rdtgroup *rdtgrp;
-	int ret = 0;
-	u32 closid;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (rdtgrp) {
-		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-			list_for_each_entry(schema, &resctrl_schema_all, list) {
-				seq_printf(s, "%s:uninitialized\n", schema->name);
-			}
-		} else if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
-			if (!rdtgrp->plr->d) {
-				rdt_last_cmd_clear();
-				rdt_last_cmd_puts("Cache domain offline\n");
-				ret = -ENODEV;
-			} else {
-				seq_printf(s, "%s:%d=%x\n",
-					   rdtgrp->plr->s->res->name,
-					   rdtgrp->plr->d->id,
-					   rdtgrp->plr->cbm);
-			}
-		} else {
-			closid = rdtgrp->closid;
-			list_for_each_entry(schema, &resctrl_schema_all, list) {
-				if (closid < schema->num_closid)
-					show_doms(s, schema, closid);
-			}
-		}
-	} else {
-		ret = -ENOENT;
-	}
-	rdtgroup_kn_unlock(of->kn);
-	return ret;
-}
-
-static int smp_mon_event_count(void *arg)
-{
-	mon_event_count(arg);
-
-	return 0;
-}
-
-void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
-		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
-		    int evtid, int first)
-{
-	int cpu;
-
-	/* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-	/*
-	 * Setup the parameters to pass to mon_event_count() to read the data.
-	 */
-	rr->rgrp = rdtgrp;
-	rr->evtid = evtid;
-	rr->r = r;
-	rr->d = d;
-	rr->val = 0;
-	rr->first = first;
-	rr->arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, evtid);
-	if (IS_ERR(rr->arch_mon_ctx)) {
-		rr->err = -EINVAL;
-		return;
-	}
-
-	cpu = cpumask_any_housekeeping(&d->cpu_mask, RESCTRL_PICK_ANY_CPU);
-
-	/*
-	 * cpumask_any_housekeeping() prefers housekeeping CPUs, but
-	 * are all the CPUs nohz_full? If yes, pick a CPU to IPI.
-	 * MPAM's resctrl_arch_rmid_read() is unable to read the
-	 * counters on some platforms if its called in IRQ context.
-	 */
-	if (tick_nohz_full_cpu(cpu))
-		smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1);
-	else
-		smp_call_on_cpu(cpu, smp_mon_event_count, rr, false);
-
-	resctrl_arch_mon_ctx_free(r, evtid, rr->arch_mon_ctx);
-}
-
-int rdtgroup_mondata_show(struct seq_file *m, void *arg)
-{
-	struct kernfs_open_file *of = m->private;
-	u32 resid, evtid, domid;
-	struct rdtgroup *rdtgrp;
-	struct rdt_resource *r;
-	union mon_data_bits md;
-	struct rdt_domain *d;
-	struct rmid_read rr;
-	int ret = 0;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (!rdtgrp) {
-		ret = -ENOENT;
-		goto out;
-	}
-
-	md.priv = of->kn->priv;
-	resid = md.u.rid;
-	domid = md.u.domid;
-	evtid = md.u.evtid;
-
-	r = resctrl_arch_get_resource(resid);
-	d = resctrl_arch_find_domain(r, domid);
-	if (IS_ERR_OR_NULL(d)) {
-		ret = -ENOENT;
-		goto out;
-	}
-
-	mon_event_read(&rr, r, d, rdtgrp, evtid, false);
-
-	if (rr.err == -EIO)
-		seq_puts(m, "Error\n");
-	else if (rr.err == -EINVAL)
-		seq_puts(m, "Unavailable\n");
-	else
-		seq_printf(m, "%llu\n", rr.val);
-
-out:
-	rdtgroup_kn_unlock(of->kn);
-	return ret;
-}
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 0f7e3f10941b..bf3538992667 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -26,227 +26,6 @@
  */
 #define MBM_CNTR_WIDTH_OFFSET_MAX (62 - MBM_CNTR_WIDTH_BASE)
 
-/**
- * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that
- *			        aren't marked nohz_full
- * @mask:	The mask to pick a CPU from.
- * @exclude_cpu:The CPU to avoid picking.
- *
- * Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping
- * CPUs that don't use nohz_full, these are preferred. Pass
- * RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs.
- *
- * When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available.
- */
-static inline unsigned int
-cpumask_any_housekeeping(const struct cpumask *mask, int exclude_cpu)
-{
-	unsigned int cpu, hk_cpu;
-
-	if (exclude_cpu == RESCTRL_PICK_ANY_CPU)
-		cpu = cpumask_any(mask);
-	else
-		cpu = cpumask_any_but(mask, exclude_cpu);
-
-	if (!IS_ENABLED(CONFIG_NO_HZ_FULL))
-		return cpu;
-
-	/* If the CPU picked isn't marked nohz_full nothing more needs doing. */
-	if (cpu < nr_cpu_ids && !tick_nohz_full_cpu(cpu))
-		return cpu;
-
-	/* Try to find a CPU that isn't nohz_full to use in preference */
-	hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask);
-	if (hk_cpu == exclude_cpu)
-		hk_cpu = cpumask_nth_andnot(1, mask, tick_nohz_full_mask);
-
-	if (hk_cpu < nr_cpu_ids)
-		cpu = hk_cpu;
-
-	return cpu;
-}
-
-struct rdt_fs_context {
-	struct kernfs_fs_context	kfc;
-	bool				enable_cdpl2;
-	bool				enable_cdpl3;
-	bool				enable_mba_mbps;
-	bool				enable_debug;
-};
-
-static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
-{
-	struct kernfs_fs_context *kfc = fc->fs_private;
-
-	return container_of(kfc, struct rdt_fs_context, kfc);
-}
-
-/**
- * struct mon_evt - Entry in the event list of a resource
- * @evtid:		event id
- * @name:		name of the event
- * @configurable:	true if the event is configurable
- * @list:		entry in &rdt_resource->evt_list
- */
-struct mon_evt {
-	enum resctrl_event_id	evtid;
-	char			*name;
-	bool			configurable;
-	struct list_head	list;
-};
-
-/**
- * union mon_data_bits - Monitoring details for each event file
- * @priv:              Used to store monitoring event data in @u
- *                     as kernfs private data
- * @rid:               Resource id associated with the event file
- * @evtid:             Event id associated with the event file
- * @domid:             The domain to which the event file belongs
- * @u:                 Name of the bit fields struct
- */
-union mon_data_bits {
-	void *priv;
-	struct {
-		unsigned int rid		: 10;
-		enum resctrl_event_id evtid	: 8;
-		unsigned int domid		: 14;
-	} u;
-};
-
-struct rmid_read {
-	struct rdtgroup		*rgrp;
-	struct rdt_resource	*r;
-	struct rdt_domain	*d;
-	enum resctrl_event_id	evtid;
-	bool			first;
-	int			err;
-	u64			val;
-	void			*arch_mon_ctx;
-};
-
-extern struct list_head resctrl_schema_all;
-extern bool resctrl_mounted;
-
-enum rdt_group_type {
-	RDTCTRL_GROUP = 0,
-	RDTMON_GROUP,
-	RDT_NUM_GROUP,
-};
-
-/**
- * enum rdtgrp_mode - Mode of a RDT resource group
- * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations
- * @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed
- * @RDT_MODE_PSEUDO_LOCKSETUP: Resource group will be used for Pseudo-Locking
- * @RDT_MODE_PSEUDO_LOCKED: No sharing of this resource group's allocations
- *                          allowed AND the allocations are Cache Pseudo-Locked
- * @RDT_NUM_MODES: Total number of modes
- *
- * The mode of a resource group enables control over the allowed overlap
- * between allocations associated with different resource groups (classes
- * of service). User is able to modify the mode of a resource group by
- * writing to the "mode" resctrl file associated with the resource group.
- *
- * The "shareable", "exclusive", and "pseudo-locksetup" modes are set by
- * writing the appropriate text to the "mode" file. A resource group enters
- * "pseudo-locked" mode after the schemata is written while the resource
- * group is in "pseudo-locksetup" mode.
- */
-enum rdtgrp_mode {
-	RDT_MODE_SHAREABLE = 0,
-	RDT_MODE_EXCLUSIVE,
-	RDT_MODE_PSEUDO_LOCKSETUP,
-	RDT_MODE_PSEUDO_LOCKED,
-
-	/* Must be last */
-	RDT_NUM_MODES,
-};
-
-/**
- * struct mongroup - store mon group's data in resctrl fs.
- * @mon_data_kn:		kernfs node for the mon_data directory
- * @parent:			parent rdtgrp
- * @crdtgrp_list:		child rdtgroup node list
- * @rmid:			rmid for this rdtgroup
- */
-struct mongroup {
-	struct kernfs_node	*mon_data_kn;
-	struct rdtgroup		*parent;
-	struct list_head	crdtgrp_list;
-	u32			rmid;
-};
-
-/**
- * struct rdtgroup - store rdtgroup's data in resctrl file system.
- * @kn:				kernfs node
- * @rdtgroup_list:		linked list for all rdtgroups
- * @closid:			closid for this rdtgroup
- * @cpu_mask:			CPUs assigned to this rdtgroup
- * @flags:			status bits
- * @waitcount:			how many cpus expect to find this
- *				group when they acquire rdtgroup_mutex
- * @type:			indicates type of this rdtgroup - either
- *				monitor only or ctrl_mon group
- * @mon:			mongroup related data
- * @mode:			mode of resource group
- * @plr:			pseudo-locked region
- */
-struct rdtgroup {
-	struct kernfs_node		*kn;
-	struct list_head		rdtgroup_list;
-	u32				closid;
-	struct cpumask			cpu_mask;
-	int				flags;
-	atomic_t			waitcount;
-	enum rdt_group_type		type;
-	struct mongroup			mon;
-	enum rdtgrp_mode		mode;
-	struct pseudo_lock_region	*plr;
-};
-
-/* List of all resource groups */
-extern struct list_head rdt_all_groups;
-
-extern int max_name_width, max_data_width;
-
-/**
- * struct rftype - describe each file in the resctrl file system
- * @name:	File name
- * @mode:	Access mode
- * @kf_ops:	File operations
- * @flags:	File specific RFTYPE_FLAGS_* flags
- * @fflags:	File specific RFTYPE_* flags
- * @seq_show:	Show content of the file
- * @write:	Write to the file
- */
-struct rftype {
-	char			*name;
-	umode_t			mode;
-	const struct kernfs_ops	*kf_ops;
-	unsigned long		flags;
-	unsigned long		fflags;
-
-	int (*seq_show)(struct kernfs_open_file *of,
-			struct seq_file *sf, void *v);
-	/*
-	 * write() is the generic write callback which maps directly to
-	 * kernfs write operation and overrides all other operations.
-	 * Maximum write size is determined by ->max_write_len.
-	 */
-	ssize_t (*write)(struct kernfs_open_file *of,
-			 char *buf, size_t nbytes, loff_t off);
-};
-
-/**
- * struct mbm_state - status for each MBM counter in each domain
- * @prev_bw_bytes: Previous bytes value read for bandwidth calculation
- * @prev_bw:	The most recent bandwidth in MBps
- */
-struct mbm_state {
-	u64	prev_bw_bytes;
-	u32	prev_bw;
-};
-
 /**
  * struct arch_mbm_state - values used to compute resctrl_arch_rmid_read()s
  *			   return value.
@@ -327,11 +106,7 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *r
 	return container_of(r, struct rdt_hw_resource, r_resctrl);
 }
 
-extern struct mutex rdtgroup_mutex;
-
 extern struct rdt_hw_resource rdt_resources_all[];
-extern struct rdtgroup rdtgroup_default;
-extern struct dentry *debugfs_resctrl;
 
 static inline struct rdt_resource *resctrl_inc(struct rdt_resource *res)
 {
@@ -395,95 +170,10 @@ union cpuid_0x10_x_edx {
 	unsigned int full;
 };
 
-void rdt_last_cmd_clear(void);
-void rdt_last_cmd_puts(const char *s);
-__printf(1, 2)
-void rdt_last_cmd_printf(const char *fmt, ...);
-
 void rdt_ctrl_update(void *arg);
-struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
-void rdtgroup_kn_unlock(struct kernfs_node *kn);
-int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name);
-int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
-			     umode_t mask);
-ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
-				char *buf, size_t nbytes, loff_t off);
-int rdtgroup_schemata_show(struct kernfs_open_file *of,
-			   struct seq_file *s, void *v);
-bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
-			   unsigned long cbm, int closid, bool exclusive);
-unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d,
-				  unsigned long cbm);
-enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);
-int rdtgroup_tasks_assigned(struct rdtgroup *r);
-int closids_supported(void);
-void closid_free(int closid);
-int alloc_rmid(u32 closid);
-void free_rmid(u32 closid, u32 rmid);
 int rdt_get_mon_l3_config(struct rdt_resource *r);
-void resctrl_mon_resource_exit(void);
 bool rdt_cpu_has(int flag);
-void mon_event_count(void *info);
-int rdtgroup_mondata_show(struct seq_file *m, void *arg);
-void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
-		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
-		    int evtid, int first);
-int resctrl_mon_resource_init(void);
-void mbm_setup_overflow_handler(struct rdt_domain *dom,
-				unsigned long delay_ms,
-				int exclude_cpu);
-void mbm_handle_overflow(struct work_struct *work);
 void __init intel_rdt_mbm_apply_quirk(void);
-bool is_mba_sc(struct rdt_resource *r);
-void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
-			     int exclude_cpu);
-void cqm_handle_limbo(struct work_struct *work);
-bool has_busy_rmid(struct rdt_domain *d);
-void __check_limbo(struct rdt_domain *d, bool force_free);
 void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
-void mbm_config_rftype_init(const char *config);
-void rdt_staged_configs_clear(void);
-bool closid_allocated(unsigned int closid);
-int resctrl_find_cleanest_closid(void);
-
-#ifdef CONFIG_RESCTRL_FS_PSEUDO_LOCK
-int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
-int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp);
-bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm);
-bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d);
-int rdt_pseudo_lock_init(void);
-void rdt_pseudo_lock_release(void);
-int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
-void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
-#else
-static inline int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
-{
-	return -EOPNOTSUPP;
-}
-
-static inline int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
-{
-	return -EOPNOTSUPP;
-}
-
-static inline bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
-{
-	return false;
-}
-
-static inline bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
-{
-	return false;
-}
-
-static inline int rdt_pseudo_lock_init(void) { return 0; }
-static inline void rdt_pseudo_lock_release(void) { }
-static inline int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
-{
-	return -EOPNOTSUPP;
-}
-
-static inline void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp) { }
-#endif /* CONFIG_RESCTRL_FS_PSEUDO_LOCK */
 
 #endif /* _ASM_X86_RESCTRL_INTERNAL_H */
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 7e6fca138cb7..02fb9d87479a 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -25,53 +25,6 @@
 
 #include "internal.h"
 
-/**
- * struct rmid_entry - dirty tracking for all RMID.
- * @closid:	The CLOSID for this entry.
- * @rmid:	The RMID for this entry.
- * @busy:	The number of domains with cached data using this RMID.
- * @list:	Member of the rmid_free_lru list when busy == 0.
- *
- * Depending on the architecture the correct monitor is accessed using
- * both @closid and @rmid, or @rmid only.
- *
- * Take the rdtgroup_mutex when accessing.
- */
-struct rmid_entry {
-	u32				closid;
-	u32				rmid;
-	int				busy;
-	struct list_head		list;
-};
-
-/*
- * @rmid_free_lru - A least recently used list of free RMIDs
- *     These RMIDs are guaranteed to have an occupancy less than the
- *     threshold occupancy
- */
-static LIST_HEAD(rmid_free_lru);
-
-/*
- * @closid_num_dirty_rmid    The number of dirty RMID each CLOSID has.
- *     Only allocated when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined.
- *     Indexed by CLOSID. Protected by rdtgroup_mutex.
- */
-static u32 *closid_num_dirty_rmid;
-
-/*
- * @rmid_limbo_count - count of currently unused but (potentially)
- *     dirty RMIDs.
- *     This counts RMIDs that no one is currently using but that
- *     may have a occupancy value > resctrl_rmid_realloc_threshold. User can
- *     change the threshold occupancy value.
- */
-static unsigned int rmid_limbo_count;
-
-/*
- * @rmid_entry - The entry in the limbo and free lists.
- */
-static struct rmid_entry	*rmid_ptrs;
-
 /*
  * Global boolean for rdt_monitor which is true if any
  * resource monitoring is enabled.
@@ -83,17 +36,6 @@ bool rdt_mon_capable;
  */
 unsigned int rdt_mon_features;
 
-/*
- * This is the threshold cache occupancy in bytes at which we will consider an
- * RMID available for re-allocation.
- */
-unsigned int resctrl_rmid_realloc_threshold;
-
-/*
- * This is the maximum value for the reallocation threshold, in bytes.
- */
-unsigned int resctrl_rmid_realloc_limit;
-
 #define CF(cf)	((unsigned long)(1048576 * (cf) + 0.5))
 
 /*
@@ -157,33 +99,6 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)
 	return val;
 }
 
-/*
- * x86 and arm64 differ in their handling of monitoring.
- * x86's RMID are independent numbers, there is only one source of traffic
- * with an RMID value of '1'.
- * arm64's PMG extends the PARTID/CLOSID space, there are multiple sources of
- * traffic with a PMG value of '1', one for each CLOSID, meaning the RMID
- * value is no longer unique.
- * To account for this, resctrl uses an index. On x86 this is just the RMID,
- * on arm64 it encodes the CLOSID and RMID. This gives a unique number.
- *
- * The domain's rmid_busy_llc and rmid_ptrs[] are sized by index. The arch code
- * must accept an attempt to read every index.
- */
-static inline struct rmid_entry *__rmid_entry(u32 idx)
-{
-	struct rmid_entry *entry;
-	u32 closid, rmid;
-
-	entry = &rmid_ptrs[idx];
-	resctrl_arch_rmid_idx_decode(idx, &closid, &rmid);
-
-	WARN_ON_ONCE(entry->closid != closid);
-	WARN_ON_ONCE(entry->rmid != rmid);
-
-	return entry;
-}
-
 static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)
 {
 	u64 msr_val;
@@ -302,735 +217,6 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,
 	return 0;
 }
 
-static void limbo_release_entry(struct rmid_entry *entry)
-{
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	rmid_limbo_count--;
-	list_add_tail(&entry->list, &rmid_free_lru);
-
-	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
-		closid_num_dirty_rmid[entry->closid]--;
-}
-
-/*
- * Check the RMIDs that are marked as busy for this domain. If the
- * reported LLC occupancy is below the threshold clear the busy bit and
- * decrement the count. If the busy count gets to zero on an RMID, we
- * free the RMID
- */
-void __check_limbo(struct rdt_domain *d, bool force_free)
-{
-	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
-	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
-	struct rmid_entry *entry;
-	u32 idx, cur_idx = 1;
-	void *arch_mon_ctx;
-	bool rmid_dirty;
-	u64 val = 0;
-
-	arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID);
-	if (IS_ERR(arch_mon_ctx)) {
-		pr_warn_ratelimited("Failed to allocate monitor context: %ld",
-				    PTR_ERR(arch_mon_ctx));
-		return;
-	}
-
-	/*
-	 * Skip RMID 0 and start from RMID 1 and check all the RMIDs that
-	 * are marked as busy for occupancy < threshold. If the occupancy
-	 * is less than the threshold decrement the busy counter of the
-	 * RMID and move it to the free list when the counter reaches 0.
-	 */
-	for (;;) {
-		idx = find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx);
-		if (idx >= idx_limit)
-			break;
-
-		entry = __rmid_entry(idx);
-		if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid,
-					   QOS_L3_OCCUP_EVENT_ID, &val,
-					   arch_mon_ctx)) {
-			rmid_dirty = true;
-		} else {
-			rmid_dirty = (val >= resctrl_rmid_realloc_threshold);
-		}
-
-		if (force_free || !rmid_dirty) {
-			clear_bit(idx, d->rmid_busy_llc);
-			if (!--entry->busy)
-				limbo_release_entry(entry);
-		}
-		cur_idx = idx + 1;
-	}
-
-	resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx);
-}
-
-bool has_busy_rmid(struct rdt_domain *d)
-{
-	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
-
-	return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit;
-}
-
-static struct rmid_entry *resctrl_find_free_rmid(u32 closid)
-{
-	struct rmid_entry *itr;
-	u32 itr_idx, cmp_idx;
-
-	if (list_empty(&rmid_free_lru))
-		return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC);
-
-	list_for_each_entry(itr, &rmid_free_lru, list) {
-		/*
-		 * Get the index of this free RMID, and the index it would need
-		 * to be if it were used with this CLOSID.
-		 * If the CLOSID is irrelevant on this architecture, the two
-		 * index values are always the same on every entry and thus the
-		 * very first entry will be returned.
-		 */
-		itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid);
-		cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid);
-
-		if (itr_idx == cmp_idx)
-			return itr;
-	}
-
-	return ERR_PTR(-ENOSPC);
-}
-
-/**
- * resctrl_find_cleanest_closid() - Find a CLOSID where all the associated
- *                                  RMID are clean, or the CLOSID that has
- *                                  the most clean RMID.
- *
- * MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocated CLOSID
- * may not be able to allocate clean RMID. To avoid this the allocator will
- * choose the CLOSID with the most clean RMID.
- *
- * When the CLOSID and RMID are independent numbers, the first free CLOSID will
- * be returned.
- */
-int resctrl_find_cleanest_closid(void)
-{
-	u32 cleanest_closid = ~0;
-	int i = 0;
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
-		return -EIO;
-
-	for (i = 0; i < closids_supported(); i++) {
-		int num_dirty;
-
-		if (closid_allocated(i))
-			continue;
-
-		num_dirty = closid_num_dirty_rmid[i];
-		if (num_dirty == 0)
-			return i;
-
-		if (cleanest_closid == ~0)
-			cleanest_closid = i;
-
-		if (num_dirty < closid_num_dirty_rmid[cleanest_closid])
-			cleanest_closid = i;
-	}
-
-	if (cleanest_closid == ~0)
-		return -ENOSPC;
-
-	return cleanest_closid;
-}
-
-/*
- * For MPAM the RMID value is not unique, and has to be considered with
- * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which
- * allows all domains to be managed by a single free list.
- * Each domain also has a rmid_busy_llc to reduce the work of the limbo handler.
- */
-int alloc_rmid(u32 closid)
-{
-	struct rmid_entry *entry;
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	entry = resctrl_find_free_rmid(closid);
-	if (IS_ERR(entry))
-		return PTR_ERR(entry);
-
-	list_del(&entry->list);
-	return entry->rmid;
-}
-
-static void add_rmid_to_limbo(struct rmid_entry *entry)
-{
-	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
-	struct rdt_domain *d;
-	u32 idx;
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	/* Walking r->domains, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-	idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid);
-
-	entry->busy = 0;
-	list_for_each_entry(d, &r->domains, list) {
-		/*
-		 * For the first limbo RMID in the domain,
-		 * setup up the limbo worker.
-		 */
-		if (!has_busy_rmid(d))
-			cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL,
-						RESCTRL_PICK_ANY_CPU);
-		set_bit(idx, d->rmid_busy_llc);
-		entry->busy++;
-	}
-
-	rmid_limbo_count++;
-	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
-		closid_num_dirty_rmid[entry->closid]++;
-}
-
-void free_rmid(u32 closid, u32 rmid)
-{
-	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
-	struct rmid_entry *entry;
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	/*
-	 * Do not allow the default rmid to be free'd. Comparing by index
-	 * allows architectures that ignore the closid parameter to avoid an
-	 * unnecessary check.
-	 */
-	if (idx == resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
-						RESCTRL_RESERVED_RMID))
-		return;
-
-	entry = __rmid_entry(idx);
-
-	if (resctrl_arch_is_llc_occupancy_enabled())
-		add_rmid_to_limbo(entry);
-	else
-		list_add_tail(&entry->list, &rmid_free_lru);
-}
-
-static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid,
-				       u32 rmid, enum resctrl_event_id evtid)
-{
-	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
-
-	switch (evtid) {
-	case QOS_L3_MBM_TOTAL_EVENT_ID:
-		return &d->mbm_total[idx];
-	case QOS_L3_MBM_LOCAL_EVENT_ID:
-		return &d->mbm_local[idx];
-	default:
-		return NULL;
-	}
-}
-
-static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr)
-{
-	struct mbm_state *m;
-	u64 tval = 0;
-
-	if (rr->first) {
-		resctrl_arch_reset_rmid(rr->r, rr->d, closid, rmid, rr->evtid);
-		m = get_mbm_state(rr->d, closid, rmid, rr->evtid);
-		if (m)
-			memset(m, 0, sizeof(struct mbm_state));
-		return 0;
-	}
-
-	rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid,
-					 &tval, rr->arch_mon_ctx);
-	if (rr->err)
-		return rr->err;
-
-	rr->val += tval;
-
-	return 0;
-}
-
-/*
- * mbm_bw_count() - Update bw count from values previously read by
- *		    __mon_event_count().
- * @closid:	The closid used to identify the cached mbm_state.
- * @rmid:	The rmid used to identify the cached mbm_state.
- * @rr:		The struct rmid_read populated by __mon_event_count().
- *
- * Supporting function to calculate the memory bandwidth
- * and delta bandwidth in MBps. The chunks value previously read by
- * __mon_event_count() is compared with the chunks value from the previous
- * invocation. This must be called once per second to maintain values in MBps.
- */
-static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr)
-{
-	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
-	struct mbm_state *m = &rr->d->mbm_local[idx];
-	u64 cur_bw, bytes, cur_bytes;
-
-	cur_bytes = rr->val;
-	bytes = cur_bytes - m->prev_bw_bytes;
-	m->prev_bw_bytes = cur_bytes;
-
-	cur_bw = bytes / SZ_1M;
-
-	m->prev_bw = cur_bw;
-}
-
-/*
- * This is scheduled by mon_event_read() to read the CQM/MBM counters
- * on a domain.
- */
-void mon_event_count(void *info)
-{
-	struct rdtgroup *rdtgrp, *entry;
-	struct rmid_read *rr = info;
-	struct list_head *head;
-	int ret;
-
-	rdtgrp = rr->rgrp;
-
-	ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr);
-
-	/*
-	 * For Ctrl groups read data from child monitor groups and
-	 * add them together. Count events which are read successfully.
-	 * Discard the rmid_read's reporting errors.
-	 */
-	head = &rdtgrp->mon.crdtgrp_list;
-
-	if (rdtgrp->type == RDTCTRL_GROUP) {
-		list_for_each_entry(entry, head, mon.crdtgrp_list) {
-			if (__mon_event_count(entry->closid, entry->mon.rmid,
-					      rr) == 0)
-				ret = 0;
-		}
-	}
-
-	/*
-	 * __mon_event_count() calls for newly created monitor groups may
-	 * report -EINVAL/Unavailable if the monitor hasn't seen any traffic.
-	 * Discard error if any of the monitor event reads succeeded.
-	 */
-	if (ret == 0)
-		rr->err = 0;
-}
-
-/*
- * Feedback loop for MBA software controller (mba_sc)
- *
- * mba_sc is a feedback loop where we periodically read MBM counters and
- * adjust the bandwidth percentage values via the IA32_MBA_THRTL_MSRs so
- * that:
- *
- *   current bandwidth(cur_bw) < user specified bandwidth(user_bw)
- *
- * This uses the MBM counters to measure the bandwidth and MBA throttle
- * MSRs to control the bandwidth for a particular rdtgrp. It builds on the
- * fact that resctrl rdtgroups have both monitoring and control.
- *
- * The frequency of the checks is 1s and we just tag along the MBM overflow
- * timer. Having 1s interval makes the calculation of bandwidth simpler.
- *
- * Although MBA's goal is to restrict the bandwidth to a maximum, there may
- * be a need to increase the bandwidth to avoid unnecessarily restricting
- * the L2 <-> L3 traffic.
- *
- * Since MBA controls the L2 external bandwidth where as MBM measures the
- * L3 external bandwidth the following sequence could lead to such a
- * situation.
- *
- * Consider an rdtgroup which had high L3 <-> memory traffic in initial
- * phases -> mba_sc kicks in and reduced bandwidth percentage values -> but
- * after some time rdtgroup has mostly L2 <-> L3 traffic.
- *
- * In this case we may restrict the rdtgroup's L2 <-> L3 traffic as its
- * throttle MSRs already have low percentage values.  To avoid
- * unnecessarily restricting such rdtgroups, we also increase the bandwidth.
- */
-static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
-{
-	u32 closid, rmid, cur_msr_val, new_msr_val;
-	struct mbm_state *pmbm_data, *cmbm_data;
-	struct rdt_resource *r_mba;
-	struct rdt_domain *dom_mba;
-	u32 cur_bw, user_bw, idx;
-	struct list_head *head;
-	struct rdtgroup *entry;
-
-	if (!resctrl_arch_is_mbm_local_enabled())
-		return;
-
-	r_mba = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
-
-	closid = rgrp->closid;
-	rmid = rgrp->mon.rmid;
-	idx = resctrl_arch_rmid_idx_encode(closid, rmid);
-	pmbm_data = &dom_mbm->mbm_local[idx];
-
-	dom_mba = resctrl_get_domain_from_cpu(smp_processor_id(), r_mba);
-	if (!dom_mba) {
-		pr_warn_once("Failure to get domain for MBA update\n");
-		return;
-	}
-
-	cur_bw = pmbm_data->prev_bw;
-	user_bw = dom_mba->mbps_val[closid];
-
-	/* MBA resource doesn't support CDP */
-	cur_msr_val = resctrl_arch_get_config(r_mba, dom_mba, closid, CDP_NONE);
-
-	/*
-	 * For Ctrl groups read data from child monitor groups.
-	 */
-	head = &rgrp->mon.crdtgrp_list;
-	list_for_each_entry(entry, head, mon.crdtgrp_list) {
-		cmbm_data = &dom_mbm->mbm_local[entry->mon.rmid];
-		cur_bw += cmbm_data->prev_bw;
-	}
-
-	/*
-	 * Scale up/down the bandwidth linearly for the ctrl group.  The
-	 * bandwidth step is the bandwidth granularity specified by the
-	 * hardware.
-	 * Always increase throttling if current bandwidth is above the
-	 * target set by user.
-	 * But avoid thrashing up and down on every poll by checking
-	 * whether a decrease in throttling is likely to push the group
-	 * back over target. E.g. if currently throttling to 30% of bandwidth
-	 * on a system with 10% granularity steps, check whether moving to
-	 * 40% would go past the limit by multiplying current bandwidth by
-	 * "(30 + 10) / 30".
-	 */
-	if (cur_msr_val > r_mba->membw.min_bw && user_bw < cur_bw) {
-		new_msr_val = cur_msr_val - r_mba->membw.bw_gran;
-	} else if (cur_msr_val < MAX_MBA_BW &&
-		   (user_bw > (cur_bw * (cur_msr_val + r_mba->membw.min_bw) / cur_msr_val))) {
-		new_msr_val = cur_msr_val + r_mba->membw.bw_gran;
-	} else {
-		return;
-	}
-
-	resctrl_arch_update_one(r_mba, dom_mba, closid, CDP_NONE, new_msr_val);
-}
-
-static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,
-		       u32 closid, u32 rmid)
-{
-	struct rmid_read rr;
-
-	rr.first = false;
-	rr.r = r;
-	rr.d = d;
-
-	/*
-	 * This is protected from concurrent reads from user
-	 * as both the user and we hold the global mutex.
-	 */
-	if (resctrl_arch_is_mbm_total_enabled()) {
-		rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID;
-		rr.val = 0;
-		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
-		if (IS_ERR(rr.arch_mon_ctx)) {
-			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
-					    PTR_ERR(rr.arch_mon_ctx));
-			return;
-		}
-
-		__mon_event_count(closid, rmid, &rr);
-
-		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
-	}
-	if (resctrl_arch_is_mbm_local_enabled()) {
-		rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID;
-		rr.val = 0;
-		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
-		if (IS_ERR(rr.arch_mon_ctx)) {
-			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
-					    PTR_ERR(rr.arch_mon_ctx));
-			return;
-		}
-
-		__mon_event_count(closid, rmid, &rr);
-
-		/*
-		 * Call the MBA software controller only for the
-		 * control groups and when user has enabled
-		 * the software controller explicitly.
-		 */
-		if (is_mba_sc(NULL))
-			mbm_bw_count(closid, rmid, &rr);
-
-		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
-	}
-}
-
-/*
- * Handler to scan the limbo list and move the RMIDs
- * to free list whose occupancy < threshold_occupancy.
- */
-void cqm_handle_limbo(struct work_struct *work)
-{
-	unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL);
-	struct rdt_domain *d;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	d = container_of(work, struct rdt_domain, cqm_limbo.work);
-
-	__check_limbo(d, false);
-
-	if (has_busy_rmid(d)) {
-		d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
-							   RESCTRL_PICK_ANY_CPU);
-		schedule_delayed_work_on(d->cqm_work_cpu, &d->cqm_limbo,
-					 delay);
-	}
-
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-}
-
-/**
- * cqm_setup_limbo_handler() - Schedule the limbo handler to run for this
- *                             domain.
- * @dom:           The domain the limbo handler should run for.
- * @delay_ms:      How far in the future the handler should run.
- * @exclude_cpu:   Which CPU the handler should not run on,
- *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
- */
-void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
-			     int exclude_cpu)
-{
-	unsigned long delay = msecs_to_jiffies(delay_ms);
-	int cpu;
-
-	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
-	dom->cqm_work_cpu = cpu;
-
-	if (cpu < nr_cpu_ids)
-		schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay);
-}
-
-void mbm_handle_overflow(struct work_struct *work)
-{
-	unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL);
-	struct rdtgroup *prgrp, *crgrp;
-	struct list_head *head;
-	struct rdt_resource *r;
-	struct rdt_domain *d;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	/*
-	 * If the filesystem has been unmounted this work no longer needs to
-	 * run.
-	 */
-	if (!resctrl_mounted || !resctrl_arch_mon_capable())
-		goto out_unlock;
-
-	r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
-	d = container_of(work, struct rdt_domain, mbm_over.work);
-
-	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
-		mbm_update(r, d, prgrp->closid, prgrp->mon.rmid);
-
-		head = &prgrp->mon.crdtgrp_list;
-		list_for_each_entry(crgrp, head, mon.crdtgrp_list)
-			mbm_update(r, d, crgrp->closid, crgrp->mon.rmid);
-
-		if (is_mba_sc(NULL))
-			update_mba_bw(prgrp, d);
-	}
-
-	/*
-	 * Re-check for housekeeping CPUs. This allows the overflow handler to
-	 * move off a nohz_full CPU quickly.
-	 */
-	d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
-						   RESCTRL_PICK_ANY_CPU);
-	schedule_delayed_work_on(d->mbm_work_cpu, &d->mbm_over, delay);
-
-out_unlock:
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-}
-
-/**
- * mbm_setup_overflow_handler() - Schedule the overflow handler to run for this
- *                                domain.
- * @dom:           The domain the overflow handler should run for.
- * @delay_ms:      How far in the future the handler should run.
- * @exclude_cpu:   Which CPU the handler should not run on,
- *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
- */
-void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms,
-				int exclude_cpu)
-{
-	unsigned long delay = msecs_to_jiffies(delay_ms);
-	int cpu;
-
-	/*
-	 * When a domain comes online there is no guarantee the filesystem is
-	 * mounted. If not, there is no need to catch counter overflow.
-	 */
-	if (!resctrl_mounted || !resctrl_arch_mon_capable())
-		return;
-	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
-	dom->mbm_work_cpu = cpu;
-
-	if (cpu < nr_cpu_ids)
-		schedule_delayed_work_on(cpu, &dom->mbm_over, delay);
-}
-
-static int dom_data_init(struct rdt_resource *r)
-{
-	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
-	u32 num_closid = resctrl_arch_get_num_closid(r);
-	struct rmid_entry *entry = NULL;
-	int err = 0, i;
-	u32 idx;
-
-	mutex_lock(&rdtgroup_mutex);
-	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
-		u32 *tmp;
-
-		/*
-		 * If the architecture hasn't provided a sanitised value here,
-		 * this may result in larger arrays than necessary. Resctrl will
-		 * use a smaller system wide value based on the resources in
-		 * use.
-		 */
-		tmp = kcalloc(num_closid, sizeof(*tmp), GFP_KERNEL);
-		if (!tmp) {
-			err = -ENOMEM;
-			goto out_unlock;
-		}
-
-		closid_num_dirty_rmid = tmp;
-	}
-
-	rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL);
-	if (!rmid_ptrs) {
-		if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
-			kfree(closid_num_dirty_rmid);
-			closid_num_dirty_rmid = NULL;
-		}
-		err = -ENOMEM;
-		goto out_unlock;
-	}
-
-	for (i = 0; i < idx_limit; i++) {
-		entry = &rmid_ptrs[i];
-		INIT_LIST_HEAD(&entry->list);
-
-		resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid);
-		list_add_tail(&entry->list, &rmid_free_lru);
-	}
-
-	/*
-	 * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and
-	 * are always allocated. These are used for the rdtgroup_default
-	 * control group, which will be setup later in rdtgroup_init().
-	 */
-	idx = resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
-					   RESCTRL_RESERVED_RMID);
-	entry = __rmid_entry(idx);
-	list_del(&entry->list);
-
-out_unlock:
-	mutex_unlock(&rdtgroup_mutex);
-
-	return err;
-}
-
-static void dom_data_exit(struct rdt_resource *r)
-{
-	if (!r->mon_capable)
-		return;
-
-	mutex_lock(&rdtgroup_mutex);
-	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
-		kfree(closid_num_dirty_rmid);
-		closid_num_dirty_rmid = NULL;
-	}
-
-	kfree(rmid_ptrs);
-	rmid_ptrs = NULL;
-
-	mutex_unlock(&rdtgroup_mutex);
-}
-
-static struct mon_evt llc_occupancy_event = {
-	.name		= "llc_occupancy",
-	.evtid		= QOS_L3_OCCUP_EVENT_ID,
-};
-
-static struct mon_evt mbm_total_event = {
-	.name		= "mbm_total_bytes",
-	.evtid		= QOS_L3_MBM_TOTAL_EVENT_ID,
-};
-
-static struct mon_evt mbm_local_event = {
-	.name		= "mbm_local_bytes",
-	.evtid		= QOS_L3_MBM_LOCAL_EVENT_ID,
-};
-
-/*
- * Initialize the event list for the resource.
- *
- * Note that MBM events are also part of RDT_RESOURCE_L3 resource
- * because as per the SDM the total and local memory bandwidth
- * are enumerated as part of L3 monitoring.
- */
-static void l3_mon_evt_init(struct rdt_resource *r)
-{
-	INIT_LIST_HEAD(&r->evt_list);
-
-	if (resctrl_arch_is_llc_occupancy_enabled())
-		list_add_tail(&llc_occupancy_event.list, &r->evt_list);
-	if (resctrl_arch_is_mbm_total_enabled())
-		list_add_tail(&mbm_total_event.list, &r->evt_list);
-	if (resctrl_arch_is_mbm_local_enabled())
-		list_add_tail(&mbm_local_event.list, &r->evt_list);
-}
-
-int resctrl_mon_resource_init(void)
-{
-	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
-	int ret;
-
-	if (!r->mon_capable)
-		return 0;
-
-	ret = dom_data_init(r);
-	if (ret)
-		return ret;
-
-	l3_mon_evt_init(r);
-
-	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_TOTAL_EVENT_ID)) {
-		mbm_total_event.configurable = true;
-		mbm_config_rftype_init("mbm_total_bytes_config");
-	}
-	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_LOCAL_EVENT_ID)) {
-		mbm_local_event.configurable = true;
-		mbm_config_rftype_init("mbm_local_bytes_config");
-	}
-
-	return 0;
-}
-
 int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 {
 	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
@@ -1076,13 +262,6 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 	return 0;
 }
 
-void resctrl_mon_resource_exit(void)
-{
-	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
-
-	dom_data_exit(r);
-}
-
 void __init intel_rdt_mbm_apply_quirk(void)
 {
 	int cf_index;
diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
index ba51ab1f70e6..ba1596afee10 100644
--- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
+++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
@@ -39,28 +39,6 @@
  */
 static u64 prefetch_disable_bits;
 
-/*
- * Major number assigned to and shared by all devices exposing
- * pseudo-locked regions.
- */
-static unsigned int pseudo_lock_major;
-static unsigned long pseudo_lock_minor_avail = GENMASK(MINORBITS, 0);
-
-static char *pseudo_lock_devnode(const struct device *dev, umode_t *mode)
-{
-	const struct rdtgroup *rdtgrp;
-
-	rdtgrp = dev_get_drvdata(dev);
-	if (mode)
-		*mode = 0600;
-	return kasprintf(GFP_KERNEL, "pseudo_lock/%s", rdtgrp->kn->name);
-}
-
-static const struct class pseudo_lock_class = {
-	.name = "pseudo_lock",
-	.devnode = pseudo_lock_devnode,
-};
-
 /**
  * resctrl_arch_get_prefetch_disable_bits - prefetch disable bits of supported
  *                                          platforms
@@ -121,299 +99,6 @@ u64 resctrl_arch_get_prefetch_disable_bits(void)
 	return prefetch_disable_bits;
 }
 
-/**
- * pseudo_lock_minor_get - Obtain available minor number
- * @minor: Pointer to where new minor number will be stored
- *
- * A bitmask is used to track available minor numbers. Here the next free
- * minor number is marked as unavailable and returned.
- *
- * Return: 0 on success, <0 on failure.
- */
-static int pseudo_lock_minor_get(unsigned int *minor)
-{
-	unsigned long first_bit;
-
-	first_bit = find_first_bit(&pseudo_lock_minor_avail, MINORBITS);
-
-	if (first_bit == MINORBITS)
-		return -ENOSPC;
-
-	__clear_bit(first_bit, &pseudo_lock_minor_avail);
-	*minor = first_bit;
-
-	return 0;
-}
-
-/**
- * pseudo_lock_minor_release - Return minor number to available
- * @minor: The minor number made available
- */
-static void pseudo_lock_minor_release(unsigned int minor)
-{
-	__set_bit(minor, &pseudo_lock_minor_avail);
-}
-
-/**
- * region_find_by_minor - Locate a pseudo-lock region by inode minor number
- * @minor: The minor number of the device representing pseudo-locked region
- *
- * When the character device is accessed we need to determine which
- * pseudo-locked region it belongs to. This is done by matching the minor
- * number of the device to the pseudo-locked region it belongs.
- *
- * Minor numbers are assigned at the time a pseudo-locked region is associated
- * with a cache instance.
- *
- * Return: On success return pointer to resource group owning the pseudo-locked
- *         region, NULL on failure.
- */
-static struct rdtgroup *region_find_by_minor(unsigned int minor)
-{
-	struct rdtgroup *rdtgrp, *rdtgrp_match = NULL;
-
-	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
-		if (rdtgrp->plr && rdtgrp->plr->minor == minor) {
-			rdtgrp_match = rdtgrp;
-			break;
-		}
-	}
-	return rdtgrp_match;
-}
-
-/**
- * struct pseudo_lock_pm_req - A power management QoS request list entry
- * @list:	Entry within the @pm_reqs list for a pseudo-locked region
- * @req:	PM QoS request
- */
-struct pseudo_lock_pm_req {
-	struct list_head list;
-	struct dev_pm_qos_request req;
-};
-
-static void pseudo_lock_cstates_relax(struct pseudo_lock_region *plr)
-{
-	struct pseudo_lock_pm_req *pm_req, *next;
-
-	list_for_each_entry_safe(pm_req, next, &plr->pm_reqs, list) {
-		dev_pm_qos_remove_request(&pm_req->req);
-		list_del(&pm_req->list);
-		kfree(pm_req);
-	}
-}
-
-/**
- * pseudo_lock_cstates_constrain - Restrict cores from entering C6
- * @plr: Pseudo-locked region
- *
- * To prevent the cache from being affected by power management entering
- * C6 has to be avoided. This is accomplished by requesting a latency
- * requirement lower than lowest C6 exit latency of all supported
- * platforms as found in the cpuidle state tables in the intel_idle driver.
- * At this time it is possible to do so with a single latency requirement
- * for all supported platforms.
- *
- * Since Goldmont is supported, which is affected by X86_BUG_MONITOR,
- * the ACPI latencies need to be considered while keeping in mind that C2
- * may be set to map to deeper sleep states. In this case the latency
- * requirement needs to prevent entering C2 also.
- *
- * Return: 0 on success, <0 on failure
- */
-static int pseudo_lock_cstates_constrain(struct pseudo_lock_region *plr)
-{
-	struct pseudo_lock_pm_req *pm_req;
-	int cpu;
-	int ret;
-
-	for_each_cpu(cpu, &plr->d->cpu_mask) {
-		pm_req = kzalloc(sizeof(*pm_req), GFP_KERNEL);
-		if (!pm_req) {
-			rdt_last_cmd_puts("Failure to allocate memory for PM QoS\n");
-			ret = -ENOMEM;
-			goto out_err;
-		}
-		ret = dev_pm_qos_add_request(get_cpu_device(cpu),
-					     &pm_req->req,
-					     DEV_PM_QOS_RESUME_LATENCY,
-					     30);
-		if (ret < 0) {
-			rdt_last_cmd_printf("Failed to add latency req CPU%d\n",
-					    cpu);
-			kfree(pm_req);
-			ret = -1;
-			goto out_err;
-		}
-		list_add(&pm_req->list, &plr->pm_reqs);
-	}
-
-	return 0;
-
-out_err:
-	pseudo_lock_cstates_relax(plr);
-	return ret;
-}
-
-/**
- * pseudo_lock_region_clear - Reset pseudo-lock region data
- * @plr: pseudo-lock region
- *
- * All content of the pseudo-locked region is reset - any memory allocated
- * freed.
- *
- * Return: void
- */
-static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)
-{
-	plr->size = 0;
-	plr->line_size = 0;
-	kfree(plr->kmem);
-	plr->kmem = NULL;
-	plr->s = NULL;
-	if (plr->d)
-		plr->d->plr = NULL;
-	plr->d = NULL;
-	plr->cbm = 0;
-	plr->debugfs_dir = NULL;
-}
-
-/**
- * pseudo_lock_region_init - Initialize pseudo-lock region information
- * @plr: pseudo-lock region
- *
- * Called after user provided a schemata to be pseudo-locked. From the
- * schemata the &struct pseudo_lock_region is on entry already initialized
- * with the resource, domain, and capacity bitmask. Here the information
- * required for pseudo-locking is deduced from this data and &struct
- * pseudo_lock_region initialized further. This information includes:
- * - size in bytes of the region to be pseudo-locked
- * - cache line size to know the stride with which data needs to be accessed
- *   to be pseudo-locked
- * - a cpu associated with the cache instance on which the pseudo-locking
- *   flow can be executed
- *
- * Return: 0 on success, <0 on failure. Descriptive error will be written
- * to last_cmd_status buffer.
- */
-static int pseudo_lock_region_init(struct pseudo_lock_region *plr)
-{
-	struct cpu_cacheinfo *ci;
-	int ret;
-	int i;
-
-	/* Pick the first cpu we find that is associated with the cache. */
-	plr->cpu = cpumask_first(&plr->d->cpu_mask);
-
-	if (!cpu_online(plr->cpu)) {
-		rdt_last_cmd_printf("CPU %u associated with cache not online\n",
-				    plr->cpu);
-		ret = -ENODEV;
-		goto out_region;
-	}
-
-	ci = get_cpu_cacheinfo(plr->cpu);
-
-	plr->size = rdtgroup_cbm_to_size(plr->s->res, plr->d, plr->cbm);
-
-	for (i = 0; i < ci->num_leaves; i++) {
-		if (ci->info_list[i].level == plr->s->res->cache_level) {
-			plr->line_size = ci->info_list[i].coherency_line_size;
-			return 0;
-		}
-	}
-
-	ret = -1;
-	rdt_last_cmd_puts("Unable to determine cache line size\n");
-out_region:
-	pseudo_lock_region_clear(plr);
-	return ret;
-}
-
-/**
- * pseudo_lock_init - Initialize a pseudo-lock region
- * @rdtgrp: resource group to which new pseudo-locked region will belong
- *
- * A pseudo-locked region is associated with a resource group. When this
- * association is created the pseudo-locked region is initialized. The
- * details of the pseudo-locked region are not known at this time so only
- * allocation is done and association established.
- *
- * Return: 0 on success, <0 on failure
- */
-static int pseudo_lock_init(struct rdtgroup *rdtgrp)
-{
-	struct pseudo_lock_region *plr;
-
-	plr = kzalloc(sizeof(*plr), GFP_KERNEL);
-	if (!plr)
-		return -ENOMEM;
-
-	init_waitqueue_head(&plr->lock_thread_wq);
-	INIT_LIST_HEAD(&plr->pm_reqs);
-	rdtgrp->plr = plr;
-	return 0;
-}
-
-/**
- * pseudo_lock_region_alloc - Allocate kernel memory that will be pseudo-locked
- * @plr: pseudo-lock region
- *
- * Initialize the details required to set up the pseudo-locked region and
- * allocate the contiguous memory that will be pseudo-locked to the cache.
- *
- * Return: 0 on success, <0 on failure.  Descriptive error will be written
- * to last_cmd_status buffer.
- */
-static int pseudo_lock_region_alloc(struct pseudo_lock_region *plr)
-{
-	int ret;
-
-	ret = pseudo_lock_region_init(plr);
-	if (ret < 0)
-		return ret;
-
-	/*
-	 * We do not yet support contiguous regions larger than
-	 * KMALLOC_MAX_SIZE.
-	 */
-	if (plr->size > KMALLOC_MAX_SIZE) {
-		rdt_last_cmd_puts("Requested region exceeds maximum size\n");
-		ret = -E2BIG;
-		goto out_region;
-	}
-
-	plr->kmem = kzalloc(plr->size, GFP_KERNEL);
-	if (!plr->kmem) {
-		rdt_last_cmd_puts("Unable to allocate memory\n");
-		ret = -ENOMEM;
-		goto out_region;
-	}
-
-	ret = 0;
-	goto out;
-out_region:
-	pseudo_lock_region_clear(plr);
-out:
-	return ret;
-}
-
-/**
- * pseudo_lock_free - Free a pseudo-locked region
- * @rdtgrp: resource group to which pseudo-locked region belonged
- *
- * The pseudo-locked region's resources have already been released, or not
- * yet created at this point. Now it can be freed and disassociated from the
- * resource group.
- *
- * Return: void
- */
-static void pseudo_lock_free(struct rdtgroup *rdtgrp)
-{
-	pseudo_lock_region_clear(rdtgrp->plr);
-	kfree(rdtgrp->plr);
-	rdtgrp->plr = NULL;
-}
-
 /**
  * resctrl_arch_pseudo_lock_fn - Load kernel memory into cache
  * @_plr: the pseudo-lock region descriptor
@@ -543,345 +228,6 @@ int resctrl_arch_pseudo_lock_fn(void *_plr)
 	return 0;
 }
 
-/**
- * rdtgroup_monitor_in_progress - Test if monitoring in progress
- * @rdtgrp: resource group being queried
- *
- * Return: 1 if monitor groups have been created for this resource
- * group, 0 otherwise.
- */
-static int rdtgroup_monitor_in_progress(struct rdtgroup *rdtgrp)
-{
-	return !list_empty(&rdtgrp->mon.crdtgrp_list);
-}
-
-/**
- * rdtgroup_locksetup_user_restrict - Restrict user access to group
- * @rdtgrp: resource group needing access restricted
- *
- * A resource group used for cache pseudo-locking cannot have cpus or tasks
- * assigned to it. This is communicated to the user by restricting access
- * to all the files that can be used to make such changes.
- *
- * Permissions restored with rdtgroup_locksetup_user_restore()
- *
- * Return: 0 on success, <0 on failure. If a failure occurs during the
- * restriction of access an attempt will be made to restore permissions but
- * the state of the mode of these files will be uncertain when a failure
- * occurs.
- */
-static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)
-{
-	int ret;
-
-	ret = rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
-	if (ret)
-		return ret;
-
-	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
-	if (ret)
-		goto err_tasks;
-
-	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
-	if (ret)
-		goto err_cpus;
-
-	if (resctrl_arch_mon_capable()) {
-		ret = rdtgroup_kn_mode_restrict(rdtgrp, "mon_groups");
-		if (ret)
-			goto err_cpus_list;
-	}
-
-	ret = 0;
-	goto out;
-
-err_cpus_list:
-	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
-err_cpus:
-	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
-err_tasks:
-	rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
-out:
-	return ret;
-}
-
-/**
- * rdtgroup_locksetup_user_restore - Restore user access to group
- * @rdtgrp: resource group needing access restored
- *
- * Restore all file access previously removed using
- * rdtgroup_locksetup_user_restrict()
- *
- * Return: 0 on success, <0 on failure.  If a failure occurs during the
- * restoration of access an attempt will be made to restrict permissions
- * again but the state of the mode of these files will be uncertain when
- * a failure occurs.
- */
-static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)
-{
-	int ret;
-
-	ret = rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
-	if (ret)
-		return ret;
-
-	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
-	if (ret)
-		goto err_tasks;
-
-	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
-	if (ret)
-		goto err_cpus;
-
-	if (resctrl_arch_mon_capable()) {
-		ret = rdtgroup_kn_mode_restore(rdtgrp, "mon_groups", 0777);
-		if (ret)
-			goto err_cpus_list;
-	}
-
-	ret = 0;
-	goto out;
-
-err_cpus_list:
-	rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
-err_cpus:
-	rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
-err_tasks:
-	rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
-out:
-	return ret;
-}
-
-/**
- * rdtgroup_locksetup_enter - Resource group enters locksetup mode
- * @rdtgrp: resource group requested to enter locksetup mode
- *
- * A resource group enters locksetup mode to reflect that it would be used
- * to represent a pseudo-locked region and is in the process of being set
- * up to do so. A resource group used for a pseudo-locked region would
- * lose the closid associated with it so we cannot allow it to have any
- * tasks or cpus assigned nor permit tasks or cpus to be assigned in the
- * future. Monitoring of a pseudo-locked region is not allowed either.
- *
- * The above and more restrictions on a pseudo-locked region are checked
- * for and enforced before the resource group enters the locksetup mode.
- *
- * Returns: 0 if the resource group successfully entered locksetup mode, <0
- * on failure. On failure the last_cmd_status buffer is updated with text to
- * communicate details of failure to the user.
- */
-int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
-{
-	int ret;
-
-	/*
-	 * The default resource group can neither be removed nor lose the
-	 * default closid associated with it.
-	 */
-	if (rdtgrp == &rdtgroup_default) {
-		rdt_last_cmd_puts("Cannot pseudo-lock default group\n");
-		return -EINVAL;
-	}
-
-	/*
-	 * Cache Pseudo-locking not supported when CDP is enabled.
-	 *
-	 * Some things to consider if you would like to enable this
-	 * support (using L3 CDP as example):
-	 * - When CDP is enabled two separate resources are exposed,
-	 *   L3DATA and L3CODE, but they are actually on the same cache.
-	 *   The implication for pseudo-locking is that if a
-	 *   pseudo-locked region is created on a domain of one
-	 *   resource (eg. L3CODE), then a pseudo-locked region cannot
-	 *   be created on that same domain of the other resource
-	 *   (eg. L3DATA). This is because the creation of a
-	 *   pseudo-locked region involves a call to wbinvd that will
-	 *   affect all cache allocations on particular domain.
-	 * - Considering the previous, it may be possible to only
-	 *   expose one of the CDP resources to pseudo-locking and
-	 *   hide the other. For example, we could consider to only
-	 *   expose L3DATA and since the L3 cache is unified it is
-	 *   still possible to place instructions there are execute it.
-	 * - If only one region is exposed to pseudo-locking we should
-	 *   still keep in mind that availability of a portion of cache
-	 *   for pseudo-locking should take into account both resources.
-	 *   Similarly, if a pseudo-locked region is created in one
-	 *   resource, the portion of cache used by it should be made
-	 *   unavailable to all future allocations from both resources.
-	 */
-	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3) ||
-	    resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2)) {
-		rdt_last_cmd_puts("CDP enabled\n");
-		return -EINVAL;
-	}
-
-	/*
-	 * Not knowing the bits to disable prefetching implies that this
-	 * platform does not support Cache Pseudo-Locking.
-	 */
-	if (resctrl_arch_get_prefetch_disable_bits() == 0) {
-		rdt_last_cmd_puts("Pseudo-locking not supported\n");
-		return -EINVAL;
-	}
-
-	if (rdtgroup_monitor_in_progress(rdtgrp)) {
-		rdt_last_cmd_puts("Monitoring in progress\n");
-		return -EINVAL;
-	}
-
-	if (rdtgroup_tasks_assigned(rdtgrp)) {
-		rdt_last_cmd_puts("Tasks assigned to resource group\n");
-		return -EINVAL;
-	}
-
-	if (!cpumask_empty(&rdtgrp->cpu_mask)) {
-		rdt_last_cmd_puts("CPUs assigned to resource group\n");
-		return -EINVAL;
-	}
-
-	if (rdtgroup_locksetup_user_restrict(rdtgrp)) {
-		rdt_last_cmd_puts("Unable to modify resctrl permissions\n");
-		return -EIO;
-	}
-
-	ret = pseudo_lock_init(rdtgrp);
-	if (ret) {
-		rdt_last_cmd_puts("Unable to init pseudo-lock region\n");
-		goto out_release;
-	}
-
-	/*
-	 * If this system is capable of monitoring a rmid would have been
-	 * allocated when the control group was created. This is not needed
-	 * anymore when this group would be used for pseudo-locking. This
-	 * is safe to call on platforms not capable of monitoring.
-	 */
-	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
-
-	ret = 0;
-	goto out;
-
-out_release:
-	rdtgroup_locksetup_user_restore(rdtgrp);
-out:
-	return ret;
-}
-
-/**
- * rdtgroup_locksetup_exit - resource group exist locksetup mode
- * @rdtgrp: resource group
- *
- * When a resource group exits locksetup mode the earlier restrictions are
- * lifted.
- *
- * Return: 0 on success, <0 on failure
- */
-int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
-{
-	int ret;
-
-	if (resctrl_arch_mon_capable()) {
-		ret = alloc_rmid(rdtgrp->closid);
-		if (ret < 0) {
-			rdt_last_cmd_puts("Out of RMIDs\n");
-			return ret;
-		}
-		rdtgrp->mon.rmid = ret;
-	}
-
-	ret = rdtgroup_locksetup_user_restore(rdtgrp);
-	if (ret) {
-		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
-		return ret;
-	}
-
-	pseudo_lock_free(rdtgrp);
-	return 0;
-}
-
-/**
- * rdtgroup_cbm_overlaps_pseudo_locked - Test if CBM or portion is pseudo-locked
- * @d: RDT domain
- * @cbm: CBM to test
- *
- * @d represents a cache instance and @cbm a capacity bitmask that is
- * considered for it. Determine if @cbm overlaps with any existing
- * pseudo-locked region on @d.
- *
- * @cbm is unsigned long, even if only 32 bits are used, to make the
- * bitmap functions work correctly.
- *
- * Return: true if @cbm overlaps with pseudo-locked region on @d, false
- * otherwise.
- */
-bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
-{
-	unsigned int cbm_len;
-	unsigned long cbm_b;
-
-	if (d->plr) {
-		cbm_len = d->plr->s->res->cache.cbm_len;
-		cbm_b = d->plr->cbm;
-		if (bitmap_intersects(&cbm, &cbm_b, cbm_len))
-			return true;
-	}
-	return false;
-}
-
-/**
- * rdtgroup_pseudo_locked_in_hierarchy - Pseudo-locked region in cache hierarchy
- * @d: RDT domain under test
- *
- * The setup of a pseudo-locked region affects all cache instances within
- * the hierarchy of the region. It is thus essential to know if any
- * pseudo-locked regions exist within a cache hierarchy to prevent any
- * attempts to create new pseudo-locked regions in the same hierarchy.
- *
- * Return: true if a pseudo-locked region exists in the hierarchy of @d or
- *         if it is not possible to test due to memory allocation issue,
- *         false otherwise.
- */
-bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
-{
-	cpumask_var_t cpu_with_psl;
-	enum resctrl_res_level i;
-	struct rdt_resource *r;
-	struct rdt_domain *d_i;
-	bool ret = false;
-
-	/* Walking r->domains, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-	if (!zalloc_cpumask_var(&cpu_with_psl, GFP_KERNEL))
-		return true;
-
-	/*
-	 * First determine which cpus have pseudo-locked regions
-	 * associated with them.
-	 */
-	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
-		r = resctrl_arch_get_resource(i);
-		if (!r->alloc_capable)
-			continue;
-
-		list_for_each_entry(d_i, &r->domains, list) {
-			if (d_i->plr)
-				cpumask_or(cpu_with_psl, cpu_with_psl,
-					   &d_i->cpu_mask);
-		}
-	}
-
-	/*
-	 * Next test if new pseudo-locked region would intersect with
-	 * existing region.
-	 */
-	if (cpumask_intersects(&d->cpu_mask, cpu_with_psl))
-		ret = true;
-
-	free_cpumask_var(cpu_with_psl);
-	return ret;
-}
-
 /**
  * resctrl_arch_measure_cycles_lat_fn - Measure cycle latency to read
  *                                      pseudo-locked memory
@@ -1174,442 +520,3 @@ int resctrl_arch_measure_l3_residency(void *_plr)
 	wake_up_interruptible(&plr->lock_thread_wq);
 	return 0;
 }
-
-/**
- * pseudo_lock_measure_cycles - Trigger latency measure to pseudo-locked region
- * @rdtgrp: Resource group to which the pseudo-locked region belongs.
- * @sel: Selector of which measurement to perform on a pseudo-locked region.
- *
- * The measurement of latency to access a pseudo-locked region should be
- * done from a cpu that is associated with that pseudo-locked region.
- * Determine which cpu is associated with this region and start a thread on
- * that cpu to perform the measurement, wait for that thread to complete.
- *
- * Return: 0 on success, <0 on failure
- */
-static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel)
-{
-	struct pseudo_lock_region *plr = rdtgrp->plr;
-	struct task_struct *thread;
-	unsigned int cpu;
-	int ret = -1;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	if (rdtgrp->flags & RDT_DELETED) {
-		ret = -ENODEV;
-		goto out;
-	}
-
-	if (!plr->d) {
-		ret = -ENODEV;
-		goto out;
-	}
-
-	plr->thread_done = 0;
-	cpu = cpumask_first(&plr->d->cpu_mask);
-	if (!cpu_online(cpu)) {
-		ret = -ENODEV;
-		goto out;
-	}
-
-	plr->cpu = cpu;
-
-	if (sel == 1)
-		thread = kthread_create_on_node(resctrl_arch_measure_cycles_lat_fn,
-						plr, cpu_to_node(cpu),
-						"pseudo_lock_measure/%u",
-						cpu);
-	else if (sel == 2)
-		thread = kthread_create_on_node(resctrl_arch_measure_l2_residency,
-						plr, cpu_to_node(cpu),
-						"pseudo_lock_measure/%u",
-						cpu);
-	else if (sel == 3)
-		thread = kthread_create_on_node(resctrl_arch_measure_l3_residency,
-						plr, cpu_to_node(cpu),
-						"pseudo_lock_measure/%u",
-						cpu);
-	else
-		goto out;
-
-	if (IS_ERR(thread)) {
-		ret = PTR_ERR(thread);
-		goto out;
-	}
-	kthread_bind(thread, cpu);
-	wake_up_process(thread);
-
-	ret = wait_event_interruptible(plr->lock_thread_wq,
-				       plr->thread_done == 1);
-	if (ret < 0)
-		goto out;
-
-	ret = 0;
-
-out:
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-	return ret;
-}
-
-static ssize_t pseudo_lock_measure_trigger(struct file *file,
-					   const char __user *user_buf,
-					   size_t count, loff_t *ppos)
-{
-	struct rdtgroup *rdtgrp = file->private_data;
-	size_t buf_size;
-	char buf[32];
-	int ret;
-	int sel;
-
-	buf_size = min(count, (sizeof(buf) - 1));
-	if (copy_from_user(buf, user_buf, buf_size))
-		return -EFAULT;
-
-	buf[buf_size] = '\0';
-	ret = kstrtoint(buf, 10, &sel);
-	if (ret == 0) {
-		if (sel != 1 && sel != 2 && sel != 3)
-			return -EINVAL;
-		ret = debugfs_file_get(file->f_path.dentry);
-		if (ret)
-			return ret;
-		ret = pseudo_lock_measure_cycles(rdtgrp, sel);
-		if (ret == 0)
-			ret = count;
-		debugfs_file_put(file->f_path.dentry);
-	}
-
-	return ret;
-}
-
-static const struct file_operations pseudo_measure_fops = {
-	.write = pseudo_lock_measure_trigger,
-	.open = simple_open,
-	.llseek = default_llseek,
-};
-
-/**
- * rdtgroup_pseudo_lock_create - Create a pseudo-locked region
- * @rdtgrp: resource group to which pseudo-lock region belongs
- *
- * Called when a resource group in the pseudo-locksetup mode receives a
- * valid schemata that should be pseudo-locked. Since the resource group is
- * in pseudo-locksetup mode the &struct pseudo_lock_region has already been
- * allocated and initialized with the essential information. If a failure
- * occurs the resource group remains in the pseudo-locksetup mode with the
- * &struct pseudo_lock_region associated with it, but cleared from all
- * information and ready for the user to re-attempt pseudo-locking by
- * writing the schemata again.
- *
- * Return: 0 if the pseudo-locked region was successfully pseudo-locked, <0
- * on failure. Descriptive error will be written to last_cmd_status buffer.
- */
-int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
-{
-	struct pseudo_lock_region *plr = rdtgrp->plr;
-	struct task_struct *thread;
-	unsigned int new_minor;
-	struct device *dev;
-	int ret;
-
-	ret = pseudo_lock_region_alloc(plr);
-	if (ret < 0)
-		return ret;
-
-	ret = pseudo_lock_cstates_constrain(plr);
-	if (ret < 0) {
-		ret = -EINVAL;
-		goto out_region;
-	}
-
-	plr->thread_done = 0;
-
-	plr->closid = rdtgrp->closid;
-	thread = kthread_create_on_node(resctrl_arch_pseudo_lock_fn, plr,
-					cpu_to_node(plr->cpu),
-					"pseudo_lock/%u", plr->cpu);
-	if (IS_ERR(thread)) {
-		ret = PTR_ERR(thread);
-		rdt_last_cmd_printf("Locking thread returned error %d\n", ret);
-		goto out_cstates;
-	}
-
-	kthread_bind(thread, plr->cpu);
-	wake_up_process(thread);
-
-	ret = wait_event_interruptible(plr->lock_thread_wq,
-				       plr->thread_done == 1);
-	if (ret < 0) {
-		/*
-		 * If the thread does not get on the CPU for whatever
-		 * reason and the process which sets up the region is
-		 * interrupted then this will leave the thread in runnable
-		 * state and once it gets on the CPU it will dereference
-		 * the cleared, but not freed, plr struct resulting in an
-		 * empty pseudo-locking loop.
-		 */
-		rdt_last_cmd_puts("Locking thread interrupted\n");
-		goto out_cstates;
-	}
-
-	ret = pseudo_lock_minor_get(&new_minor);
-	if (ret < 0) {
-		rdt_last_cmd_puts("Unable to obtain a new minor number\n");
-		goto out_cstates;
-	}
-
-	/*
-	 * Unlock access but do not release the reference. The
-	 * pseudo-locked region will still be here on return.
-	 *
-	 * The mutex has to be released temporarily to avoid a potential
-	 * deadlock with the mm->mmap_lock which is obtained in the
-	 * device_create() and debugfs_create_dir() callpath below as well as
-	 * before the mmap() callback is called.
-	 */
-	mutex_unlock(&rdtgroup_mutex);
-
-	if (!IS_ERR_OR_NULL(debugfs_resctrl)) {
-		plr->debugfs_dir = debugfs_create_dir(rdtgrp->kn->name,
-						      debugfs_resctrl);
-		if (!IS_ERR_OR_NULL(plr->debugfs_dir))
-			debugfs_create_file("pseudo_lock_measure", 0200,
-					    plr->debugfs_dir, rdtgrp,
-					    &pseudo_measure_fops);
-	}
-
-	dev = device_create(&pseudo_lock_class, NULL,
-			    MKDEV(pseudo_lock_major, new_minor),
-			    rdtgrp, "%s", rdtgrp->kn->name);
-
-	mutex_lock(&rdtgroup_mutex);
-
-	if (IS_ERR(dev)) {
-		ret = PTR_ERR(dev);
-		rdt_last_cmd_printf("Failed to create character device: %d\n",
-				    ret);
-		goto out_debugfs;
-	}
-
-	/* We released the mutex - check if group was removed while we did so */
-	if (rdtgrp->flags & RDT_DELETED) {
-		ret = -ENODEV;
-		goto out_device;
-	}
-
-	plr->minor = new_minor;
-
-	rdtgrp->mode = RDT_MODE_PSEUDO_LOCKED;
-	closid_free(rdtgrp->closid);
-	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0444);
-	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0444);
-
-	ret = 0;
-	goto out;
-
-out_device:
-	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, new_minor));
-out_debugfs:
-	debugfs_remove_recursive(plr->debugfs_dir);
-	pseudo_lock_minor_release(new_minor);
-out_cstates:
-	pseudo_lock_cstates_relax(plr);
-out_region:
-	pseudo_lock_region_clear(plr);
-out:
-	return ret;
-}
-
-/**
- * rdtgroup_pseudo_lock_remove - Remove a pseudo-locked region
- * @rdtgrp: resource group to which the pseudo-locked region belongs
- *
- * The removal of a pseudo-locked region can be initiated when the resource
- * group is removed from user space via a "rmdir" from userspace or the
- * unmount of the resctrl filesystem. On removal the resource group does
- * not go back to pseudo-locksetup mode before it is removed, instead it is
- * removed directly. There is thus asymmetry with the creation where the
- * &struct pseudo_lock_region is removed here while it was not created in
- * rdtgroup_pseudo_lock_create().
- *
- * Return: void
- */
-void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp)
-{
-	struct pseudo_lock_region *plr = rdtgrp->plr;
-
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-		/*
-		 * Default group cannot be a pseudo-locked region so we can
-		 * free closid here.
-		 */
-		closid_free(rdtgrp->closid);
-		goto free;
-	}
-
-	pseudo_lock_cstates_relax(plr);
-	debugfs_remove_recursive(rdtgrp->plr->debugfs_dir);
-	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, plr->minor));
-	pseudo_lock_minor_release(plr->minor);
-
-free:
-	pseudo_lock_free(rdtgrp);
-}
-
-static int pseudo_lock_dev_open(struct inode *inode, struct file *filp)
-{
-	struct rdtgroup *rdtgrp;
-
-	mutex_lock(&rdtgroup_mutex);
-
-	rdtgrp = region_find_by_minor(iminor(inode));
-	if (!rdtgrp) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -ENODEV;
-	}
-
-	filp->private_data = rdtgrp;
-	atomic_inc(&rdtgrp->waitcount);
-	/* Perform a non-seekable open - llseek is not supported */
-	filp->f_mode &= ~(FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE);
-
-	mutex_unlock(&rdtgroup_mutex);
-
-	return 0;
-}
-
-static int pseudo_lock_dev_release(struct inode *inode, struct file *filp)
-{
-	struct rdtgroup *rdtgrp;
-
-	mutex_lock(&rdtgroup_mutex);
-	rdtgrp = filp->private_data;
-	WARN_ON(!rdtgrp);
-	if (!rdtgrp) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -ENODEV;
-	}
-	filp->private_data = NULL;
-	atomic_dec(&rdtgrp->waitcount);
-	mutex_unlock(&rdtgroup_mutex);
-	return 0;
-}
-
-static int pseudo_lock_dev_mremap(struct vm_area_struct *area)
-{
-	/* Not supported */
-	return -EINVAL;
-}
-
-static const struct vm_operations_struct pseudo_mmap_ops = {
-	.mremap = pseudo_lock_dev_mremap,
-};
-
-static int pseudo_lock_dev_mmap(struct file *filp, struct vm_area_struct *vma)
-{
-	unsigned long vsize = vma->vm_end - vma->vm_start;
-	unsigned long off = vma->vm_pgoff << PAGE_SHIFT;
-	struct pseudo_lock_region *plr;
-	struct rdtgroup *rdtgrp;
-	unsigned long physical;
-	unsigned long psize;
-
-	mutex_lock(&rdtgroup_mutex);
-
-	rdtgrp = filp->private_data;
-	WARN_ON(!rdtgrp);
-	if (!rdtgrp) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -ENODEV;
-	}
-
-	plr = rdtgrp->plr;
-
-	if (!plr->d) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -ENODEV;
-	}
-
-	/*
-	 * Task is required to run with affinity to the cpus associated
-	 * with the pseudo-locked region. If this is not the case the task
-	 * may be scheduled elsewhere and invalidate entries in the
-	 * pseudo-locked region.
-	 */
-	if (!cpumask_subset(current->cpus_ptr, &plr->d->cpu_mask)) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -EINVAL;
-	}
-
-	physical = __pa(plr->kmem) >> PAGE_SHIFT;
-	psize = plr->size - off;
-
-	if (off > plr->size) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -ENOSPC;
-	}
-
-	/*
-	 * Ensure changes are carried directly to the memory being mapped,
-	 * do not allow copy-on-write mapping.
-	 */
-	if (!(vma->vm_flags & VM_SHARED)) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -EINVAL;
-	}
-
-	if (vsize > psize) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -ENOSPC;
-	}
-
-	memset(plr->kmem + off, 0, vsize);
-
-	if (remap_pfn_range(vma, vma->vm_start, physical + vma->vm_pgoff,
-			    vsize, vma->vm_page_prot)) {
-		mutex_unlock(&rdtgroup_mutex);
-		return -EAGAIN;
-	}
-	vma->vm_ops = &pseudo_mmap_ops;
-	mutex_unlock(&rdtgroup_mutex);
-	return 0;
-}
-
-static const struct file_operations pseudo_lock_dev_fops = {
-	.owner =	THIS_MODULE,
-	.llseek =	no_llseek,
-	.read =		NULL,
-	.write =	NULL,
-	.open =		pseudo_lock_dev_open,
-	.release =	pseudo_lock_dev_release,
-	.mmap =		pseudo_lock_dev_mmap,
-};
-
-int rdt_pseudo_lock_init(void)
-{
-	int ret;
-
-	ret = register_chrdev(0, "pseudo_lock", &pseudo_lock_dev_fops);
-	if (ret < 0)
-		return ret;
-
-	pseudo_lock_major = ret;
-
-	ret = class_register(&pseudo_lock_class);
-	if (ret) {
-		unregister_chrdev(pseudo_lock_major, "pseudo_lock");
-		return ret;
-	}
-
-	return 0;
-}
-
-void rdt_pseudo_lock_release(void)
-{
-	class_unregister(&pseudo_lock_class);
-	unregister_chrdev(pseudo_lock_major, "pseudo_lock");
-	pseudo_lock_major = 0;
-}
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 1425a33d201d..fe3952514add 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -12,22 +12,8 @@
 
 #define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
 
-#include <linux/cacheinfo.h>
 #include <linux/cpu.h>
-#include <linux/debugfs.h>
-#include <linux/fs.h>
-#include <linux/fs_parser.h>
-#include <linux/sysfs.h>
-#include <linux/kernfs.h>
-#include <linux/seq_buf.h>
-#include <linux/seq_file.h>
-#include <linux/sched/signal.h>
-#include <linux/sched/task.h>
 #include <linux/slab.h>
-#include <linux/task_work.h>
-#include <linux/user_namespace.h>
-
-#include <uapi/linux/magic.h>
 
 #include <asm/resctrl.h>
 #include "internal.h"
@@ -36,328 +22,6 @@ DEFINE_STATIC_KEY_FALSE(rdt_enable_key);
 DEFINE_STATIC_KEY_FALSE(rdt_mon_enable_key);
 DEFINE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
 
-/* Mutex to protect rdtgroup access. */
-DEFINE_MUTEX(rdtgroup_mutex);
-
-static struct kernfs_root *rdt_root;
-struct rdtgroup rdtgroup_default;
-LIST_HEAD(rdt_all_groups);
-
-/* list of entries for the schemata file */
-LIST_HEAD(resctrl_schema_all);
-
-/* The filesystem can only be mounted once. */
-bool resctrl_mounted;
-
-/* Kernel fs node for "info" directory under root */
-static struct kernfs_node *kn_info;
-
-/* Kernel fs node for "mon_groups" directory under root */
-static struct kernfs_node *kn_mongrp;
-
-/* Kernel fs node for "mon_data" directory under root */
-static struct kernfs_node *kn_mondata;
-
-/*
- * Used to store the max resource name width and max resource data width
- * to display the schemata in a tabular format
- */
-int max_name_width, max_data_width;
-
-static struct seq_buf last_cmd_status;
-static char last_cmd_status_buf[512];
-
-static int rdtgroup_setup_root(struct rdt_fs_context *ctx);
-static void rdtgroup_destroy_root(void);
-
-struct dentry *debugfs_resctrl;
-
-static bool resctrl_debug;
-
-void rdt_last_cmd_clear(void)
-{
-	lockdep_assert_held(&rdtgroup_mutex);
-	seq_buf_clear(&last_cmd_status);
-}
-
-void rdt_last_cmd_puts(const char *s)
-{
-	lockdep_assert_held(&rdtgroup_mutex);
-	seq_buf_puts(&last_cmd_status, s);
-}
-
-void rdt_last_cmd_printf(const char *fmt, ...)
-{
-	va_list ap;
-
-	va_start(ap, fmt);
-	lockdep_assert_held(&rdtgroup_mutex);
-	seq_buf_vprintf(&last_cmd_status, fmt, ap);
-	va_end(ap);
-}
-
-void rdt_staged_configs_clear(void)
-{
-	enum resctrl_res_level i;
-	struct rdt_resource *r;
-	struct rdt_domain *dom;
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
-		r = resctrl_arch_get_resource(i);
-		if (!r->alloc_capable)
-			continue;
-
-		list_for_each_entry(dom, &r->domains, list)
-			memset(dom->staged_config, 0, sizeof(dom->staged_config));
-	}
-}
-
-static bool resctrl_is_mbm_enabled(void)
-{
-	return (resctrl_arch_is_mbm_total_enabled() ||
-		resctrl_arch_is_mbm_local_enabled());
-}
-
-static bool resctrl_is_mbm_event(int e)
-{
-	return (e >= QOS_L3_MBM_TOTAL_EVENT_ID &&
-		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
-}
-
-/*
- * Trivial allocator for CLOSIDs. Since h/w only supports a small number,
- * we can keep a bitmap of free CLOSIDs in a single integer.
- *
- * Using a global CLOSID across all resources has some advantages and
- * some drawbacks:
- * + We can simply set current's closid to assign a task to a resource
- *   group.
- * + Context switch code can avoid extra memory references deciding which
- *   CLOSID to load into the PQR_ASSOC MSR
- * - We give up some options in configuring resource groups across multi-socket
- *   systems.
- * - Our choices on how to configure each resource become progressively more
- *   limited as the number of resources grows.
- */
-static unsigned long closid_free_map;
-static int closid_free_map_len;
-
-int closids_supported(void)
-{
-	return closid_free_map_len;
-}
-
-static void closid_init(void)
-{
-	struct resctrl_schema *s;
-	u32 rdt_min_closid = 32;
-
-	/* Compute rdt_min_closid across all resources */
-	list_for_each_entry(s, &resctrl_schema_all, list)
-		rdt_min_closid = min(rdt_min_closid, s->num_closid);
-
-	closid_free_map = BIT_MASK(rdt_min_closid) - 1;
-
-	/* RESCTRL_RESERVED_CLOSID is always reserved for the default group */
-	__clear_bit(RESCTRL_RESERVED_CLOSID, &closid_free_map);
-	closid_free_map_len = rdt_min_closid;
-}
-
-static int closid_alloc(void)
-{
-	int cleanest_closid;
-	u32 closid;
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
-	    resctrl_arch_is_llc_occupancy_enabled()) {
-		cleanest_closid = resctrl_find_cleanest_closid();
-		if (cleanest_closid < 0)
-			return cleanest_closid;
-		closid = cleanest_closid;
-	} else {
-		closid = ffs(closid_free_map);
-		if (closid == 0)
-			return -ENOSPC;
-		closid--;
-	}
-	__clear_bit(closid, &closid_free_map);
-
-	return closid;
-}
-
-void closid_free(int closid)
-{
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	__set_bit(closid, &closid_free_map);
-}
-
-/**
- * closid_allocated - test if provided closid is in use
- * @closid: closid to be tested
- *
- * Return: true if @closid is currently associated with a resource group,
- * false if @closid is free
- */
-bool closid_allocated(unsigned int closid)
-{
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	return !test_bit(closid, &closid_free_map);
-}
-
-/**
- * rdtgroup_mode_by_closid - Return mode of resource group with closid
- * @closid: closid if the resource group
- *
- * Each resource group is associated with a @closid. Here the mode
- * of a resource group can be queried by searching for it using its closid.
- *
- * Return: mode as &enum rdtgrp_mode of resource group with closid @closid
- */
-enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)
-{
-	struct rdtgroup *rdtgrp;
-
-	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
-		if (rdtgrp->closid == closid)
-			return rdtgrp->mode;
-	}
-
-	return RDT_NUM_MODES;
-}
-
-static const char * const rdt_mode_str[] = {
-	[RDT_MODE_SHAREABLE]		= "shareable",
-	[RDT_MODE_EXCLUSIVE]		= "exclusive",
-	[RDT_MODE_PSEUDO_LOCKSETUP]	= "pseudo-locksetup",
-	[RDT_MODE_PSEUDO_LOCKED]	= "pseudo-locked",
-};
-
-/**
- * rdtgroup_mode_str - Return the string representation of mode
- * @mode: the resource group mode as &enum rdtgroup_mode
- *
- * Return: string representation of valid mode, "unknown" otherwise
- */
-static const char *rdtgroup_mode_str(enum rdtgrp_mode mode)
-{
-	if (mode < RDT_MODE_SHAREABLE || mode >= RDT_NUM_MODES)
-		return "unknown";
-
-	return rdt_mode_str[mode];
-}
-
-/* set uid and gid of rdtgroup dirs and files to that of the creator */
-static int rdtgroup_kn_set_ugid(struct kernfs_node *kn)
-{
-	struct iattr iattr = { .ia_valid = ATTR_UID | ATTR_GID,
-				.ia_uid = current_fsuid(),
-				.ia_gid = current_fsgid(), };
-
-	if (uid_eq(iattr.ia_uid, GLOBAL_ROOT_UID) &&
-	    gid_eq(iattr.ia_gid, GLOBAL_ROOT_GID))
-		return 0;
-
-	return kernfs_setattr(kn, &iattr);
-}
-
-static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
-{
-	struct kernfs_node *kn;
-	int ret;
-
-	kn = __kernfs_create_file(parent_kn, rft->name, rft->mode,
-				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID,
-				  0, rft->kf_ops, rft, NULL, NULL);
-	if (IS_ERR(kn))
-		return PTR_ERR(kn);
-
-	ret = rdtgroup_kn_set_ugid(kn);
-	if (ret) {
-		kernfs_remove(kn);
-		return ret;
-	}
-
-	return 0;
-}
-
-static int rdtgroup_seqfile_show(struct seq_file *m, void *arg)
-{
-	struct kernfs_open_file *of = m->private;
-	struct rftype *rft = of->kn->priv;
-
-	if (rft->seq_show)
-		return rft->seq_show(of, m, arg);
-	return 0;
-}
-
-static ssize_t rdtgroup_file_write(struct kernfs_open_file *of, char *buf,
-				   size_t nbytes, loff_t off)
-{
-	struct rftype *rft = of->kn->priv;
-
-	if (rft->write)
-		return rft->write(of, buf, nbytes, off);
-
-	return -EINVAL;
-}
-
-static const struct kernfs_ops rdtgroup_kf_single_ops = {
-	.atomic_write_len	= PAGE_SIZE,
-	.write			= rdtgroup_file_write,
-	.seq_show		= rdtgroup_seqfile_show,
-};
-
-static const struct kernfs_ops kf_mondata_ops = {
-	.atomic_write_len	= PAGE_SIZE,
-	.seq_show		= rdtgroup_mondata_show,
-};
-
-static bool is_cpu_list(struct kernfs_open_file *of)
-{
-	struct rftype *rft = of->kn->priv;
-
-	return rft->flags & RFTYPE_FLAGS_CPUS_LIST;
-}
-
-static int rdtgroup_cpus_show(struct kernfs_open_file *of,
-			      struct seq_file *s, void *v)
-{
-	struct rdtgroup *rdtgrp;
-	struct cpumask *mask;
-	int ret = 0;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-
-	if (rdtgrp) {
-		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
-			if (!rdtgrp->plr->d) {
-				rdt_last_cmd_clear();
-				rdt_last_cmd_puts("Cache domain offline\n");
-				ret = -ENODEV;
-			} else {
-				mask = &rdtgrp->plr->d->cpu_mask;
-				seq_printf(s, is_cpu_list(of) ?
-					   "%*pbl\n" : "%*pb\n",
-					   cpumask_pr_args(mask));
-			}
-		} else {
-			seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
-				   cpumask_pr_args(&rdtgrp->cpu_mask));
-		}
-	} else {
-		ret = -ENOENT;
-	}
-	rdtgroup_kn_unlock(of->kn);
-
-	return ret;
-}
-
 /*
  * This is safe against resctrl_arch_sched_in() called from __switch_to()
  * because __switch_to() is executed with interrupts disabled. A local call
@@ -381,1206 +45,6 @@ void resctrl_arch_sync_cpu_defaults(void *info)
 	resctrl_arch_sched_in(current);
 }
 
-/*
- * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
- *
- * Per task closids/rmids must have been set up before calling this function.
- * @r may be NULL.
- */
-static void
-update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
-{
-	struct resctrl_cpu_sync defaults;
-	struct resctrl_cpu_sync *defaults_p = NULL;
-
-	if (r) {
-		defaults.closid = r->closid;
-		defaults.rmid = r->mon.rmid;
-		defaults_p = &defaults;
-	}
-
-	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
-			 1);
-}
-
-static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
-			  cpumask_var_t tmpmask)
-{
-	struct rdtgroup *prgrp = rdtgrp->mon.parent, *crgrp;
-	struct list_head *head;
-
-	/* Check whether cpus belong to parent ctrl group */
-	cpumask_andnot(tmpmask, newmask, &prgrp->cpu_mask);
-	if (!cpumask_empty(tmpmask)) {
-		rdt_last_cmd_puts("Can only add CPUs to mongroup that belong to parent\n");
-		return -EINVAL;
-	}
-
-	/* Check whether cpus are dropped from this group */
-	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
-	if (!cpumask_empty(tmpmask)) {
-		/* Give any dropped cpus to parent rdtgroup */
-		cpumask_or(&prgrp->cpu_mask, &prgrp->cpu_mask, tmpmask);
-		update_closid_rmid(tmpmask, prgrp);
-	}
-
-	/*
-	 * If we added cpus, remove them from previous group that owned them
-	 * and update per-cpu rmid
-	 */
-	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
-	if (!cpumask_empty(tmpmask)) {
-		head = &prgrp->mon.crdtgrp_list;
-		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
-			if (crgrp == rdtgrp)
-				continue;
-			cpumask_andnot(&crgrp->cpu_mask, &crgrp->cpu_mask,
-				       tmpmask);
-		}
-		update_closid_rmid(tmpmask, rdtgrp);
-	}
-
-	/* Done pushing/pulling - update this group with new mask */
-	cpumask_copy(&rdtgrp->cpu_mask, newmask);
-
-	return 0;
-}
-
-static void cpumask_rdtgrp_clear(struct rdtgroup *r, struct cpumask *m)
-{
-	struct rdtgroup *crgrp;
-
-	cpumask_andnot(&r->cpu_mask, &r->cpu_mask, m);
-	/* update the child mon group masks as well*/
-	list_for_each_entry(crgrp, &r->mon.crdtgrp_list, mon.crdtgrp_list)
-		cpumask_and(&crgrp->cpu_mask, &r->cpu_mask, &crgrp->cpu_mask);
-}
-
-static int cpus_ctrl_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
-			   cpumask_var_t tmpmask, cpumask_var_t tmpmask1)
-{
-	struct rdtgroup *r, *crgrp;
-	struct list_head *head;
-
-	/* Check whether cpus are dropped from this group */
-	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
-	if (!cpumask_empty(tmpmask)) {
-		/* Can't drop from default group */
-		if (rdtgrp == &rdtgroup_default) {
-			rdt_last_cmd_puts("Can't drop CPUs from default group\n");
-			return -EINVAL;
-		}
-
-		/* Give any dropped cpus to rdtgroup_default */
-		cpumask_or(&rdtgroup_default.cpu_mask,
-			   &rdtgroup_default.cpu_mask, tmpmask);
-		update_closid_rmid(tmpmask, &rdtgroup_default);
-	}
-
-	/*
-	 * If we added cpus, remove them from previous group and
-	 * the prev group's child groups that owned them
-	 * and update per-cpu closid/rmid.
-	 */
-	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
-	if (!cpumask_empty(tmpmask)) {
-		list_for_each_entry(r, &rdt_all_groups, rdtgroup_list) {
-			if (r == rdtgrp)
-				continue;
-			cpumask_and(tmpmask1, &r->cpu_mask, tmpmask);
-			if (!cpumask_empty(tmpmask1))
-				cpumask_rdtgrp_clear(r, tmpmask1);
-		}
-		update_closid_rmid(tmpmask, rdtgrp);
-	}
-
-	/* Done pushing/pulling - update this group with new mask */
-	cpumask_copy(&rdtgrp->cpu_mask, newmask);
-
-	/*
-	 * Clear child mon group masks since there is a new parent mask
-	 * now and update the rmid for the cpus the child lost.
-	 */
-	head = &rdtgrp->mon.crdtgrp_list;
-	list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
-		cpumask_and(tmpmask, &rdtgrp->cpu_mask, &crgrp->cpu_mask);
-		update_closid_rmid(tmpmask, rdtgrp);
-		cpumask_clear(&crgrp->cpu_mask);
-	}
-
-	return 0;
-}
-
-static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of,
-				   char *buf, size_t nbytes, loff_t off)
-{
-	cpumask_var_t tmpmask, newmask, tmpmask1;
-	struct rdtgroup *rdtgrp;
-	int ret;
-
-	if (!buf)
-		return -EINVAL;
-
-	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
-		return -ENOMEM;
-	if (!zalloc_cpumask_var(&newmask, GFP_KERNEL)) {
-		free_cpumask_var(tmpmask);
-		return -ENOMEM;
-	}
-	if (!zalloc_cpumask_var(&tmpmask1, GFP_KERNEL)) {
-		free_cpumask_var(tmpmask);
-		free_cpumask_var(newmask);
-		return -ENOMEM;
-	}
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (!rdtgrp) {
-		ret = -ENOENT;
-		goto unlock;
-	}
-
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
-	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-		ret = -EINVAL;
-		rdt_last_cmd_puts("Pseudo-locking in progress\n");
-		goto unlock;
-	}
-
-	if (is_cpu_list(of))
-		ret = cpulist_parse(buf, newmask);
-	else
-		ret = cpumask_parse(buf, newmask);
-
-	if (ret) {
-		rdt_last_cmd_puts("Bad CPU list/mask\n");
-		goto unlock;
-	}
-
-	/* check that user didn't specify any offline cpus */
-	cpumask_andnot(tmpmask, newmask, cpu_online_mask);
-	if (!cpumask_empty(tmpmask)) {
-		ret = -EINVAL;
-		rdt_last_cmd_puts("Can only assign online CPUs\n");
-		goto unlock;
-	}
-
-	if (rdtgrp->type == RDTCTRL_GROUP)
-		ret = cpus_ctrl_write(rdtgrp, newmask, tmpmask, tmpmask1);
-	else if (rdtgrp->type == RDTMON_GROUP)
-		ret = cpus_mon_write(rdtgrp, newmask, tmpmask);
-	else
-		ret = -EINVAL;
-
-unlock:
-	rdtgroup_kn_unlock(of->kn);
-	free_cpumask_var(tmpmask);
-	free_cpumask_var(newmask);
-	free_cpumask_var(tmpmask1);
-
-	return ret ?: nbytes;
-}
-
-/**
- * rdtgroup_remove - the helper to remove resource group safely
- * @rdtgrp: resource group to remove
- *
- * On resource group creation via a mkdir, an extra kernfs_node reference is
- * taken to ensure that the rdtgroup structure remains accessible for the
- * rdtgroup_kn_unlock() calls where it is removed.
- *
- * Drop the extra reference here, then free the rdtgroup structure.
- *
- * Return: void
- */
-static void rdtgroup_remove(struct rdtgroup *rdtgrp)
-{
-	kernfs_put(rdtgrp->kn);
-	kfree(rdtgrp);
-}
-
-static void _update_task_closid_rmid(void *task)
-{
-	/*
-	 * If the task is still current on this CPU, update PQR_ASSOC MSR.
-	 * Otherwise, the MSR is updated when the task is scheduled in.
-	 */
-	if (task == current)
-		resctrl_arch_sched_in(task);
-}
-
-static void update_task_closid_rmid(struct task_struct *t)
-{
-	if (IS_ENABLED(CONFIG_SMP) && task_curr(t))
-		smp_call_function_single(task_cpu(t), _update_task_closid_rmid, t, 1);
-	else
-		_update_task_closid_rmid(t);
-}
-
-static bool task_in_rdtgroup(struct task_struct *tsk, struct rdtgroup *rdtgrp)
-{
-	u32 closid, rmid = rdtgrp->mon.rmid;
-
-	if (rdtgrp->type == RDTCTRL_GROUP)
-		closid = rdtgrp->closid;
-	else if (rdtgrp->type == RDTMON_GROUP)
-		closid = rdtgrp->mon.parent->closid;
-	else
-		return false;
-
-	return resctrl_arch_match_closid(tsk, closid) &&
-	       resctrl_arch_match_rmid(tsk, closid, rmid);
-}
-
-static int __rdtgroup_move_task(struct task_struct *tsk,
-				struct rdtgroup *rdtgrp)
-{
-	/* If the task is already in rdtgrp, no need to move the task. */
-	if (task_in_rdtgroup(tsk, rdtgrp))
-		return 0;
-
-	/*
-	 * Set the task's closid/rmid before the PQR_ASSOC MSR can be
-	 * updated by them.
-	 *
-	 * For ctrl_mon groups, move both closid and rmid.
-	 * For monitor groups, can move the tasks only from
-	 * their parent CTRL group.
-	 */
-	if (rdtgrp->type == RDTMON_GROUP &&
-	    !resctrl_arch_match_closid(tsk, rdtgrp->mon.parent->closid)) {
-		rdt_last_cmd_puts("Can't move task to different control group\n");
-		return -EINVAL;
-	}
-
-	if (rdtgrp->type == RDTMON_GROUP)
-		resctrl_arch_set_closid_rmid(tsk, rdtgrp->mon.parent->closid,
-					     rdtgrp->mon.rmid);
-	else
-		resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid,
-					     rdtgrp->mon.rmid);
-
-	/*
-	 * Ensure the task's closid and rmid are written before determining if
-	 * the task is current that will decide if it will be interrupted.
-	 * This pairs with the full barrier between the rq->curr update and
-	 * resctrl_arch_sched_in() during context switch.
-	 */
-	smp_mb();
-
-	/*
-	 * By now, the task's closid and rmid are set. If the task is current
-	 * on a CPU, the PQR_ASSOC MSR needs to be updated to make the resource
-	 * group go into effect. If the task is not current, the MSR will be
-	 * updated when the task is scheduled in.
-	 */
-	update_task_closid_rmid(tsk);
-
-	return 0;
-}
-
-static bool is_closid_match(struct task_struct *t, struct rdtgroup *r)
-{
-	return (resctrl_arch_alloc_capable() && (r->type == RDTCTRL_GROUP) &&
-		resctrl_arch_match_closid(t, r->closid));
-}
-
-static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r)
-{
-	return (resctrl_arch_mon_capable() && (r->type == RDTMON_GROUP) &&
-		resctrl_arch_match_rmid(t, r->mon.parent->closid,
-					r->mon.rmid));
-}
-
-/**
- * rdtgroup_tasks_assigned - Test if tasks have been assigned to resource group
- * @r: Resource group
- *
- * Return: 1 if tasks have been assigned to @r, 0 otherwise
- */
-int rdtgroup_tasks_assigned(struct rdtgroup *r)
-{
-	struct task_struct *p, *t;
-	int ret = 0;
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	rcu_read_lock();
-	for_each_process_thread(p, t) {
-		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
-			ret = 1;
-			break;
-		}
-	}
-	rcu_read_unlock();
-
-	return ret;
-}
-
-static int rdtgroup_task_write_permission(struct task_struct *task,
-					  struct kernfs_open_file *of)
-{
-	const struct cred *tcred = get_task_cred(task);
-	const struct cred *cred = current_cred();
-	int ret = 0;
-
-	/*
-	 * Even if we're attaching all tasks in the thread group, we only
-	 * need to check permissions on one of them.
-	 */
-	if (!uid_eq(cred->euid, GLOBAL_ROOT_UID) &&
-	    !uid_eq(cred->euid, tcred->uid) &&
-	    !uid_eq(cred->euid, tcred->suid)) {
-		rdt_last_cmd_printf("No permission to move task %d\n", task->pid);
-		ret = -EPERM;
-	}
-
-	put_cred(tcred);
-	return ret;
-}
-
-static int rdtgroup_move_task(pid_t pid, struct rdtgroup *rdtgrp,
-			      struct kernfs_open_file *of)
-{
-	struct task_struct *tsk;
-	int ret;
-
-	rcu_read_lock();
-	if (pid) {
-		tsk = find_task_by_vpid(pid);
-		if (!tsk) {
-			rcu_read_unlock();
-			rdt_last_cmd_printf("No task %d\n", pid);
-			return -ESRCH;
-		}
-	} else {
-		tsk = current;
-	}
-
-	get_task_struct(tsk);
-	rcu_read_unlock();
-
-	ret = rdtgroup_task_write_permission(tsk, of);
-	if (!ret)
-		ret = __rdtgroup_move_task(tsk, rdtgrp);
-
-	put_task_struct(tsk);
-	return ret;
-}
-
-static ssize_t rdtgroup_tasks_write(struct kernfs_open_file *of,
-				    char *buf, size_t nbytes, loff_t off)
-{
-	struct rdtgroup *rdtgrp;
-	char *pid_str;
-	int ret = 0;
-	pid_t pid;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (!rdtgrp) {
-		rdtgroup_kn_unlock(of->kn);
-		return -ENOENT;
-	}
-	rdt_last_cmd_clear();
-
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
-	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-		ret = -EINVAL;
-		rdt_last_cmd_puts("Pseudo-locking in progress\n");
-		goto unlock;
-	}
-
-	while (buf && buf[0] != '\0' && buf[0] != '\n') {
-		pid_str = strim(strsep(&buf, ","));
-
-		if (kstrtoint(pid_str, 0, &pid)) {
-			rdt_last_cmd_printf("Task list parsing error pid %s\n", pid_str);
-			ret = -EINVAL;
-			break;
-		}
-
-		if (pid < 0) {
-			rdt_last_cmd_printf("Invalid pid %d\n", pid);
-			ret = -EINVAL;
-			break;
-		}
-
-		ret = rdtgroup_move_task(pid, rdtgrp, of);
-		if (ret) {
-			rdt_last_cmd_printf("Error while processing task %d\n", pid);
-			break;
-		}
-	}
-
-unlock:
-	rdtgroup_kn_unlock(of->kn);
-
-	return ret ?: nbytes;
-}
-
-static void show_rdt_tasks(struct rdtgroup *r, struct seq_file *s)
-{
-	struct task_struct *p, *t;
-	pid_t pid;
-
-	rcu_read_lock();
-	for_each_process_thread(p, t) {
-		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
-			pid = task_pid_vnr(t);
-			if (pid)
-				seq_printf(s, "%d\n", pid);
-		}
-	}
-	rcu_read_unlock();
-}
-
-static int rdtgroup_tasks_show(struct kernfs_open_file *of,
-			       struct seq_file *s, void *v)
-{
-	struct rdtgroup *rdtgrp;
-	int ret = 0;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (rdtgrp)
-		show_rdt_tasks(rdtgrp, s);
-	else
-		ret = -ENOENT;
-	rdtgroup_kn_unlock(of->kn);
-
-	return ret;
-}
-
-static int rdtgroup_closid_show(struct kernfs_open_file *of,
-				struct seq_file *s, void *v)
-{
-	struct rdtgroup *rdtgrp;
-	int ret = 0;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (rdtgrp)
-		seq_printf(s, "%u\n", rdtgrp->closid);
-	else
-		ret = -ENOENT;
-	rdtgroup_kn_unlock(of->kn);
-
-	return ret;
-}
-
-static int rdtgroup_rmid_show(struct kernfs_open_file *of,
-			      struct seq_file *s, void *v)
-{
-	struct rdtgroup *rdtgrp;
-	int ret = 0;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (rdtgrp)
-		seq_printf(s, "%u\n", rdtgrp->mon.rmid);
-	else
-		ret = -ENOENT;
-	rdtgroup_kn_unlock(of->kn);
-
-	return ret;
-}
-
-#ifdef CONFIG_PROC_CPU_RESCTRL
-
-/*
- * A task can only be part of one resctrl control group and of one monitor
- * group which is associated to that control group.
- *
- * 1)   res:
- *      mon:
- *
- *    resctrl is not available.
- *
- * 2)   res:/
- *      mon:
- *
- *    Task is part of the root resctrl control group, and it is not associated
- *    to any monitor group.
- *
- * 3)  res:/
- *     mon:mon0
- *
- *    Task is part of the root resctrl control group and monitor group mon0.
- *
- * 4)  res:group0
- *     mon:
- *
- *    Task is part of resctrl control group group0, and it is not associated
- *    to any monitor group.
- *
- * 5) res:group0
- *    mon:mon1
- *
- *    Task is part of resctrl control group group0 and monitor group mon1.
- */
-int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,
-		      struct pid *pid, struct task_struct *tsk)
-{
-	struct rdtgroup *rdtg;
-	int ret = 0;
-
-	mutex_lock(&rdtgroup_mutex);
-
-	/* Return empty if resctrl has not been mounted. */
-	if (!resctrl_mounted) {
-		seq_puts(s, "res:\nmon:\n");
-		goto unlock;
-	}
-
-	list_for_each_entry(rdtg, &rdt_all_groups, rdtgroup_list) {
-		struct rdtgroup *crg;
-
-		/*
-		 * Task information is only relevant for shareable
-		 * and exclusive groups.
-		 */
-		if (rdtg->mode != RDT_MODE_SHAREABLE &&
-		    rdtg->mode != RDT_MODE_EXCLUSIVE)
-			continue;
-
-		if (!resctrl_arch_match_closid(tsk, rdtg->closid))
-			continue;
-
-		seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "",
-			   rdtg->kn->name);
-		seq_puts(s, "mon:");
-		list_for_each_entry(crg, &rdtg->mon.crdtgrp_list,
-				    mon.crdtgrp_list) {
-			if (!resctrl_arch_match_rmid(tsk, crg->mon.parent->closid,
-						     crg->mon.rmid))
-				continue;
-			seq_printf(s, "%s", crg->kn->name);
-			break;
-		}
-		seq_putc(s, '\n');
-		goto unlock;
-	}
-	/*
-	 * The above search should succeed. Otherwise return
-	 * with an error.
-	 */
-	ret = -ENOENT;
-unlock:
-	mutex_unlock(&rdtgroup_mutex);
-
-	return ret;
-}
-#endif
-
-static int rdt_last_cmd_status_show(struct kernfs_open_file *of,
-				    struct seq_file *seq, void *v)
-{
-	int len;
-
-	mutex_lock(&rdtgroup_mutex);
-	len = seq_buf_used(&last_cmd_status);
-	if (len)
-		seq_printf(seq, "%.*s", len, last_cmd_status_buf);
-	else
-		seq_puts(seq, "ok\n");
-	mutex_unlock(&rdtgroup_mutex);
-	return 0;
-}
-
-static int rdt_num_closids_show(struct kernfs_open_file *of,
-				struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-
-	seq_printf(seq, "%u\n", s->num_closid);
-	return 0;
-}
-
-static int rdt_default_ctrl_show(struct kernfs_open_file *of,
-			     struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	seq_printf(seq, "%x\n", r->default_ctrl);
-	return 0;
-}
-
-static int rdt_min_cbm_bits_show(struct kernfs_open_file *of,
-			     struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	seq_printf(seq, "%u\n", r->cache.min_cbm_bits);
-	return 0;
-}
-
-static int rdt_shareable_bits_show(struct kernfs_open_file *of,
-				   struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	seq_printf(seq, "%x\n", r->cache.shareable_bits);
-	return 0;
-}
-
-/*
- * rdt_bit_usage_show - Display current usage of resources
- *
- * A domain is a shared resource that can now be allocated differently. Here
- * we display the current regions of the domain as an annotated bitmask.
- * For each domain of this resource its allocation bitmask
- * is annotated as below to indicate the current usage of the corresponding bit:
- *   0 - currently unused
- *   X - currently available for sharing and used by software and hardware
- *   H - currently used by hardware only but available for software use
- *   S - currently used and shareable by software only
- *   E - currently used exclusively by one resource group
- *   P - currently pseudo-locked by one resource group
- */
-static int rdt_bit_usage_show(struct kernfs_open_file *of,
-			      struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	/*
-	 * Use unsigned long even though only 32 bits are used to ensure
-	 * test_bit() is used safely.
-	 */
-	unsigned long sw_shareable = 0, hw_shareable = 0;
-	unsigned long exclusive = 0, pseudo_locked = 0;
-	struct rdt_resource *r = s->res;
-	struct rdt_domain *dom;
-	int i, hwb, swb, excl, psl;
-	enum rdtgrp_mode mode;
-	bool sep = false;
-	u32 ctrl_val;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-	hw_shareable = r->cache.shareable_bits;
-	list_for_each_entry(dom, &r->domains, list) {
-		if (sep)
-			seq_putc(seq, ';');
-		sw_shareable = 0;
-		exclusive = 0;
-		seq_printf(seq, "%d=", dom->id);
-		for (i = 0; i < closids_supported(); i++) {
-			if (!closid_allocated(i))
-				continue;
-			ctrl_val = resctrl_arch_get_config(r, dom, i,
-							   s->conf_type);
-			mode = rdtgroup_mode_by_closid(i);
-			switch (mode) {
-			case RDT_MODE_SHAREABLE:
-				sw_shareable |= ctrl_val;
-				break;
-			case RDT_MODE_EXCLUSIVE:
-				exclusive |= ctrl_val;
-				break;
-			case RDT_MODE_PSEUDO_LOCKSETUP:
-			/*
-			 * RDT_MODE_PSEUDO_LOCKSETUP is possible
-			 * here but not included since the CBM
-			 * associated with this CLOSID in this mode
-			 * is not initialized and no task or cpu can be
-			 * assigned this CLOSID.
-			 */
-				break;
-			case RDT_MODE_PSEUDO_LOCKED:
-			case RDT_NUM_MODES:
-				WARN(1,
-				     "invalid mode for closid %d\n", i);
-				break;
-			}
-		}
-		for (i = r->cache.cbm_len - 1; i >= 0; i--) {
-			pseudo_locked = dom->plr ? dom->plr->cbm : 0;
-			hwb = test_bit(i, &hw_shareable);
-			swb = test_bit(i, &sw_shareable);
-			excl = test_bit(i, &exclusive);
-			psl = test_bit(i, &pseudo_locked);
-			if (hwb && swb)
-				seq_putc(seq, 'X');
-			else if (hwb && !swb)
-				seq_putc(seq, 'H');
-			else if (!hwb && swb)
-				seq_putc(seq, 'S');
-			else if (excl)
-				seq_putc(seq, 'E');
-			else if (psl)
-				seq_putc(seq, 'P');
-			else /* Unused bits remain */
-				seq_putc(seq, '0');
-		}
-		sep = true;
-	}
-	seq_putc(seq, '\n');
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-	return 0;
-}
-
-static int rdt_min_bw_show(struct kernfs_open_file *of,
-			     struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	seq_printf(seq, "%u\n", r->membw.min_bw);
-	return 0;
-}
-
-static int rdt_num_rmids_show(struct kernfs_open_file *of,
-			      struct seq_file *seq, void *v)
-{
-	struct rdt_resource *r = of->kn->parent->priv;
-
-	seq_printf(seq, "%d\n", r->num_rmid);
-
-	return 0;
-}
-
-static int rdt_mon_features_show(struct kernfs_open_file *of,
-				 struct seq_file *seq, void *v)
-{
-	struct rdt_resource *r = of->kn->parent->priv;
-	struct mon_evt *mevt;
-
-	list_for_each_entry(mevt, &r->evt_list, list) {
-		seq_printf(seq, "%s\n", mevt->name);
-		if (mevt->configurable)
-			seq_printf(seq, "%s_config\n", mevt->name);
-	}
-
-	return 0;
-}
-
-static int rdt_bw_gran_show(struct kernfs_open_file *of,
-			     struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	seq_printf(seq, "%u\n", r->membw.bw_gran);
-	return 0;
-}
-
-static int rdt_delay_linear_show(struct kernfs_open_file *of,
-			     struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	seq_printf(seq, "%u\n", r->membw.delay_linear);
-	return 0;
-}
-
-static int max_threshold_occ_show(struct kernfs_open_file *of,
-				  struct seq_file *seq, void *v)
-{
-	seq_printf(seq, "%u\n", resctrl_rmid_realloc_threshold);
-
-	return 0;
-}
-
-static int rdt_thread_throttle_mode_show(struct kernfs_open_file *of,
-					 struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	if (r->membw.throttle_mode == THREAD_THROTTLE_PER_THREAD)
-		seq_puts(seq, "per-thread\n");
-	else
-		seq_puts(seq, "max\n");
-
-	return 0;
-}
-
-static ssize_t max_threshold_occ_write(struct kernfs_open_file *of,
-				       char *buf, size_t nbytes, loff_t off)
-{
-	unsigned int bytes;
-	int ret;
-
-	ret = kstrtouint(buf, 0, &bytes);
-	if (ret)
-		return ret;
-
-	if (bytes > resctrl_rmid_realloc_limit)
-		return -EINVAL;
-
-	resctrl_rmid_realloc_threshold = resctrl_arch_round_mon_val(bytes);
-
-	return nbytes;
-}
-
-/*
- * rdtgroup_mode_show - Display mode of this resource group
- */
-static int rdtgroup_mode_show(struct kernfs_open_file *of,
-			      struct seq_file *s, void *v)
-{
-	struct rdtgroup *rdtgrp;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (!rdtgrp) {
-		rdtgroup_kn_unlock(of->kn);
-		return -ENOENT;
-	}
-
-	seq_printf(s, "%s\n", rdtgroup_mode_str(rdtgrp->mode));
-
-	rdtgroup_kn_unlock(of->kn);
-	return 0;
-}
-
-static enum resctrl_conf_type resctrl_peer_type(enum resctrl_conf_type my_type)
-{
-	switch (my_type) {
-	case CDP_CODE:
-		return CDP_DATA;
-	case CDP_DATA:
-		return CDP_CODE;
-	default:
-	case CDP_NONE:
-		return CDP_NONE;
-	}
-}
-
-static int rdt_has_sparse_bitmasks_show(struct kernfs_open_file *of,
-					struct seq_file *seq, void *v)
-{
-	struct resctrl_schema *s = of->kn->parent->priv;
-	struct rdt_resource *r = s->res;
-
-	seq_printf(seq, "%u\n", r->cache.arch_has_sparse_bitmasks);
-
-	return 0;
-}
-
-/**
- * __rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other
- * @r: Resource to which domain instance @d belongs.
- * @d: The domain instance for which @closid is being tested.
- * @cbm: Capacity bitmask being tested.
- * @closid: Intended closid for @cbm.
- * @type: CDP type of @r.
- * @exclusive: Only check if overlaps with exclusive resource groups
- *
- * Checks if provided @cbm intended to be used for @closid on domain
- * @d overlaps with any other closids or other hardware usage associated
- * with this domain. If @exclusive is true then only overlaps with
- * resource groups in exclusive mode will be considered. If @exclusive
- * is false then overlaps with any resource group or hardware entities
- * will be considered.
- *
- * @cbm is unsigned long, even if only 32 bits are used, to make the
- * bitmap functions work correctly.
- *
- * Return: false if CBM does not overlap, true if it does.
- */
-static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
-				    unsigned long cbm, int closid,
-				    enum resctrl_conf_type type, bool exclusive)
-{
-	enum rdtgrp_mode mode;
-	unsigned long ctrl_b;
-	int i;
-
-	/* Check for any overlap with regions used by hardware directly */
-	if (!exclusive) {
-		ctrl_b = r->cache.shareable_bits;
-		if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len))
-			return true;
-	}
-
-	/* Check for overlap with other resource groups */
-	for (i = 0; i < closids_supported(); i++) {
-		ctrl_b = resctrl_arch_get_config(r, d, i, type);
-		mode = rdtgroup_mode_by_closid(i);
-		if (closid_allocated(i) && i != closid &&
-		    mode != RDT_MODE_PSEUDO_LOCKSETUP) {
-			if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len)) {
-				if (exclusive) {
-					if (mode == RDT_MODE_EXCLUSIVE)
-						return true;
-					continue;
-				}
-				return true;
-			}
-		}
-	}
-
-	return false;
-}
-
-/**
- * rdtgroup_cbm_overlaps - Does CBM overlap with other use of hardware
- * @s: Schema for the resource to which domain instance @d belongs.
- * @d: The domain instance for which @closid is being tested.
- * @cbm: Capacity bitmask being tested.
- * @closid: Intended closid for @cbm.
- * @exclusive: Only check if overlaps with exclusive resource groups
- *
- * Resources that can be allocated using a CBM can use the CBM to control
- * the overlap of these allocations. rdtgroup_cmb_overlaps() is the test
- * for overlap. Overlap test is not limited to the specific resource for
- * which the CBM is intended though - when dealing with CDP resources that
- * share the underlying hardware the overlap check should be performed on
- * the CDP resource sharing the hardware also.
- *
- * Refer to description of __rdtgroup_cbm_overlaps() for the details of the
- * overlap test.
- *
- * Return: true if CBM overlap detected, false if there is no overlap
- */
-bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
-			   unsigned long cbm, int closid, bool exclusive)
-{
-	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
-	struct rdt_resource *r = s->res;
-
-	if (__rdtgroup_cbm_overlaps(r, d, cbm, closid, s->conf_type,
-				    exclusive))
-		return true;
-
-	if (!resctrl_arch_get_cdp_enabled(r->rid))
-		return false;
-	return  __rdtgroup_cbm_overlaps(r, d, cbm, closid, peer_type, exclusive);
-}
-
-/**
- * rdtgroup_mode_test_exclusive - Test if this resource group can be exclusive
- * @rdtgrp: Resource group identified through its closid.
- *
- * An exclusive resource group implies that there should be no sharing of
- * its allocated resources. At the time this group is considered to be
- * exclusive this test can determine if its current schemata supports this
- * setting by testing for overlap with all other resource groups.
- *
- * Return: true if resource group can be exclusive, false if there is overlap
- * with allocations of other resource groups and thus this resource group
- * cannot be exclusive.
- */
-static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
-{
-	int closid = rdtgrp->closid;
-	struct resctrl_schema *s;
-	struct rdt_resource *r;
-	bool has_cache = false;
-	struct rdt_domain *d;
-	u32 ctrl;
-
-	/* Walking r->domains, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-	list_for_each_entry(s, &resctrl_schema_all, list) {
-		r = s->res;
-		if (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)
-			continue;
-		has_cache = true;
-		list_for_each_entry(d, &r->domains, list) {
-			ctrl = resctrl_arch_get_config(r, d, closid,
-						       s->conf_type);
-			if (rdtgroup_cbm_overlaps(s, d, ctrl, closid, false)) {
-				rdt_last_cmd_puts("Schemata overlaps\n");
-				return false;
-			}
-		}
-	}
-
-	if (!has_cache) {
-		rdt_last_cmd_puts("Cannot be exclusive without CAT/CDP\n");
-		return false;
-	}
-
-	return true;
-}
-
-/*
- * rdtgroup_mode_write - Modify the resource group's mode
- */
-static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
-				   char *buf, size_t nbytes, loff_t off)
-{
-	struct rdtgroup *rdtgrp;
-	enum rdtgrp_mode mode;
-	int ret = 0;
-
-	/* Valid input requires a trailing newline */
-	if (nbytes == 0 || buf[nbytes - 1] != '\n')
-		return -EINVAL;
-	buf[nbytes - 1] = '\0';
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (!rdtgrp) {
-		rdtgroup_kn_unlock(of->kn);
-		return -ENOENT;
-	}
-
-	rdt_last_cmd_clear();
-
-	mode = rdtgrp->mode;
-
-	if ((!strcmp(buf, "shareable") && mode == RDT_MODE_SHAREABLE) ||
-	    (!strcmp(buf, "exclusive") && mode == RDT_MODE_EXCLUSIVE) ||
-	    (!strcmp(buf, "pseudo-locksetup") &&
-	     mode == RDT_MODE_PSEUDO_LOCKSETUP) ||
-	    (!strcmp(buf, "pseudo-locked") && mode == RDT_MODE_PSEUDO_LOCKED))
-		goto out;
-
-	if (mode == RDT_MODE_PSEUDO_LOCKED) {
-		rdt_last_cmd_puts("Cannot change pseudo-locked group\n");
-		ret = -EINVAL;
-		goto out;
-	}
-
-	if (!strcmp(buf, "shareable")) {
-		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-			ret = rdtgroup_locksetup_exit(rdtgrp);
-			if (ret)
-				goto out;
-		}
-		rdtgrp->mode = RDT_MODE_SHAREABLE;
-	} else if (!strcmp(buf, "exclusive")) {
-		if (!rdtgroup_mode_test_exclusive(rdtgrp)) {
-			ret = -EINVAL;
-			goto out;
-		}
-		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-			ret = rdtgroup_locksetup_exit(rdtgrp);
-			if (ret)
-				goto out;
-		}
-		rdtgrp->mode = RDT_MODE_EXCLUSIVE;
-	} else if (IS_ENABLED(CONFIG_RESCTRL_FS_PSEUDO_LOCK) &&
-		   !strcmp(buf, "pseudo-locksetup")) {
-		ret = rdtgroup_locksetup_enter(rdtgrp);
-		if (ret)
-			goto out;
-		rdtgrp->mode = RDT_MODE_PSEUDO_LOCKSETUP;
-	} else {
-		rdt_last_cmd_puts("Unknown or unsupported mode\n");
-		ret = -EINVAL;
-	}
-
-out:
-	rdtgroup_kn_unlock(of->kn);
-	return ret ?: nbytes;
-}
-
-/**
- * rdtgroup_cbm_to_size - Translate CBM to size in bytes
- * @r: RDT resource to which @d belongs.
- * @d: RDT domain instance.
- * @cbm: bitmask for which the size should be computed.
- *
- * The bitmask provided associated with the RDT domain instance @d will be
- * translated into how many bytes it represents. The size in bytes is
- * computed by first dividing the total cache size by the CBM length to
- * determine how many bytes each bit in the bitmask represents. The result
- * is multiplied with the number of bits set in the bitmask.
- *
- * @cbm is unsigned long, even if only 32 bits are used to make the
- * bitmap functions work correctly.
- */
-unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,
-				  struct rdt_domain *d, unsigned long cbm)
-{
-	struct cpu_cacheinfo *ci;
-	unsigned int size = 0;
-	int num_b, i;
-
-	num_b = bitmap_weight(&cbm, r->cache.cbm_len);
-	ci = get_cpu_cacheinfo(cpumask_any(&d->cpu_mask));
-	for (i = 0; i < ci->num_leaves; i++) {
-		if (ci->info_list[i].level == r->cache_level) {
-			size = ci->info_list[i].size / r->cache.cbm_len * num_b;
-			break;
-		}
-	}
-
-	return size;
-}
-
-/*
- * rdtgroup_size_show - Display size in bytes of allocated regions
- *
- * The "size" file mirrors the layout of the "schemata" file, printing the
- * size in bytes of each region instead of the capacity bitmask.
- */
-static int rdtgroup_size_show(struct kernfs_open_file *of,
-			      struct seq_file *s, void *v)
-{
-	struct resctrl_schema *schema;
-	enum resctrl_conf_type type;
-	struct rdtgroup *rdtgrp;
-	struct rdt_resource *r;
-	struct rdt_domain *d;
-	unsigned int size;
-	int ret = 0;
-	u32 closid;
-	bool sep;
-	u32 ctrl;
-
-	rdtgrp = rdtgroup_kn_lock_live(of->kn);
-	if (!rdtgrp) {
-		rdtgroup_kn_unlock(of->kn);
-		return -ENOENT;
-	}
-
-	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
-		if (!rdtgrp->plr->d) {
-			rdt_last_cmd_clear();
-			rdt_last_cmd_puts("Cache domain offline\n");
-			ret = -ENODEV;
-		} else {
-			seq_printf(s, "%*s:", max_name_width,
-				   rdtgrp->plr->s->name);
-			size = rdtgroup_cbm_to_size(rdtgrp->plr->s->res,
-						    rdtgrp->plr->d,
-						    rdtgrp->plr->cbm);
-			seq_printf(s, "%d=%u\n", rdtgrp->plr->d->id, size);
-		}
-		goto out;
-	}
-
-	closid = rdtgrp->closid;
-
-	list_for_each_entry(schema, &resctrl_schema_all, list) {
-		r = schema->res;
-		type = schema->conf_type;
-		sep = false;
-		seq_printf(s, "%*s:", max_name_width, schema->name);
-		list_for_each_entry(d, &r->domains, list) {
-			if (sep)
-				seq_putc(s, ';');
-			if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
-				size = 0;
-			} else {
-				if (is_mba_sc(r))
-					ctrl = d->mbps_val[closid];
-				else
-					ctrl = resctrl_arch_get_config(r, d,
-								       closid,
-								       type);
-				if (r->rid == RDT_RESOURCE_MBA ||
-				    r->rid == RDT_RESOURCE_SMBA)
-					size = ctrl;
-				else
-					size = rdtgroup_cbm_to_size(r, d, ctrl);
-			}
-			seq_printf(s, "%d=%u", d->id, size);
-			sep = true;
-		}
-		seq_putc(s, '\n');
-	}
-
-out:
-	rdtgroup_kn_unlock(of->kn);
-
-	return ret;
-}
-
 #define INVALID_CONFIG_INDEX   UINT_MAX
 
 /**
@@ -1622,62 +86,6 @@ void resctrl_arch_mon_event_config_read(void *info)
 	mon_info->mon_config = msrval & MAX_EVT_CONFIG_BITS;
 }
 
-static void mondata_config_read(struct resctrl_mon_config_info *mon_info)
-{
-	smp_call_function_any(&mon_info->d->cpu_mask,
-			      resctrl_arch_mon_event_config_read, mon_info, 1);
-}
-
-static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid)
-{
-	struct resctrl_mon_config_info mon_info = {0};
-	struct rdt_domain *dom;
-	bool sep = false;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	list_for_each_entry(dom, &r->domains, list) {
-		if (sep)
-			seq_puts(s, ";");
-
-		memset(&mon_info, 0, sizeof(struct resctrl_mon_config_info));
-		mon_info.r = r;
-		mon_info.d = dom;
-		mon_info.evtid = evtid;
-		mondata_config_read(&mon_info);
-
-		seq_printf(s, "%d=0x%02x", dom->id, mon_info.mon_config);
-		sep = true;
-	}
-	seq_puts(s, "\n");
-
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-
-	return 0;
-}
-
-static int mbm_total_bytes_config_show(struct kernfs_open_file *of,
-				       struct seq_file *seq, void *v)
-{
-	struct rdt_resource *r = of->kn->parent->priv;
-
-	mbm_config_show(seq, r, QOS_L3_MBM_TOTAL_EVENT_ID);
-
-	return 0;
-}
-
-static int mbm_local_bytes_config_show(struct kernfs_open_file *of,
-				       struct seq_file *seq, void *v)
-{
-	struct rdt_resource *r = of->kn->parent->priv;
-
-	mbm_config_show(seq, r, QOS_L3_MBM_LOCAL_EVENT_ID);
-
-	return 0;
-}
-
 void resctrl_arch_mon_event_config_write(void *info)
 {
 	struct resctrl_mon_config_info *mon_info = info;
@@ -1694,605 +102,6 @@ void resctrl_arch_mon_event_config_write(void *info)
 	mon_info->err = 0;
 }
 
-static int mbm_config_write_domain(struct rdt_resource *r,
-				   struct rdt_domain *d, u32 evtid, u32 val)
-{
-	struct resctrl_mon_config_info mon_info = {0};
-
-	/*
-	 * Read the current config value first. If both are the same then
-	 * no need to write it again.
-	 */
-	mon_info.r = r;
-	mon_info.d = d;
-	mon_info.evtid = evtid;
-	mondata_config_read(&mon_info);
-	if (mon_info.mon_config == val)
-		return 0;
-
-	mon_info.mon_config = val;
-
-	/*
-	 * Update MSR_IA32_EVT_CFG_BASE MSR on one of the CPUs in the
-	 * domain. The MSRs offset from MSR MSR_IA32_EVT_CFG_BASE
-	 * are scoped at the domain level. Writing any of these MSRs
-	 * on one CPU is observed by all the CPUs in the domain.
-	 */
-	smp_call_function_any(&d->cpu_mask, resctrl_arch_mon_event_config_write,
-			      &mon_info, 1);
-	if (mon_info.err) {
-		rdt_last_cmd_puts("Invalid event configuration\n");
-		return mon_info.err;
-	}
-
-	/*
-	 * When an Event Configuration is changed, the bandwidth counters
-	 * for all RMIDs and Events will be cleared by the hardware. The
-	 * hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for
-	 * every RMID on the next read to any event for every RMID.
-	 * Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62)
-	 * cleared while it is tracked by the hardware. Clear the
-	 * mbm_local and mbm_total counts for all the RMIDs.
-	 */
-	resctrl_arch_reset_rmid_all(r, d);
-
-	return 0;
-}
-
-static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
-{
-	char *dom_str = NULL, *id_str;
-	unsigned long dom_id, val;
-	struct rdt_domain *d;
-	int err;
-
-	/* Walking r->domains, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-next:
-	if (!tok || tok[0] == '\0')
-		return 0;
-
-	/* Start processing the strings for each domain */
-	dom_str = strim(strsep(&tok, ";"));
-	id_str = strsep(&dom_str, "=");
-
-	if (!id_str || kstrtoul(id_str, 10, &dom_id)) {
-		rdt_last_cmd_puts("Missing '=' or non-numeric domain id\n");
-		return -EINVAL;
-	}
-
-	if (!dom_str || kstrtoul(dom_str, 16, &val)) {
-		rdt_last_cmd_puts("Non-numeric event configuration value\n");
-		return -EINVAL;
-	}
-
-	/* Value from user cannot be more than the supported set of events */
-	if ((val & r->mbm_cfg_mask) != val) {
-		rdt_last_cmd_printf("Invalid event configuration: max valid mask is 0x%02x\n",
-				    r->mbm_cfg_mask);
-		return -EINVAL;
-	}
-
-	list_for_each_entry(d, &r->domains, list) {
-		if (d->id == dom_id) {
-			err = mbm_config_write_domain(r, d, evtid, val);
-			if (err)
-				return err;
-			goto next;
-		}
-	}
-
-	return -EINVAL;
-}
-
-static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of,
-					    char *buf, size_t nbytes,
-					    loff_t off)
-{
-	struct rdt_resource *r = of->kn->parent->priv;
-	int ret;
-
-	/* Valid input requires a trailing newline */
-	if (nbytes == 0 || buf[nbytes - 1] != '\n')
-		return -EINVAL;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	rdt_last_cmd_clear();
-
-	buf[nbytes - 1] = '\0';
-
-	ret = mon_config_write(r, buf, QOS_L3_MBM_TOTAL_EVENT_ID);
-
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-
-	return ret ?: nbytes;
-}
-
-static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of,
-					    char *buf, size_t nbytes,
-					    loff_t off)
-{
-	struct rdt_resource *r = of->kn->parent->priv;
-	int ret;
-
-	/* Valid input requires a trailing newline */
-	if (nbytes == 0 || buf[nbytes - 1] != '\n')
-		return -EINVAL;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	rdt_last_cmd_clear();
-
-	buf[nbytes - 1] = '\0';
-
-	ret = mon_config_write(r, buf, QOS_L3_MBM_LOCAL_EVENT_ID);
-
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-
-	return ret ?: nbytes;
-}
-
-/* rdtgroup information files for one cache resource. */
-static struct rftype res_common_files[] = {
-	{
-		.name		= "last_cmd_status",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_last_cmd_status_show,
-		.fflags		= RFTYPE_TOP_INFO,
-	},
-	{
-		.name		= "num_closids",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_num_closids_show,
-		.fflags		= RFTYPE_CTRL_INFO,
-	},
-	{
-		.name		= "mon_features",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_mon_features_show,
-		.fflags		= RFTYPE_MON_INFO,
-	},
-	{
-		.name		= "num_rmids",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_num_rmids_show,
-		.fflags		= RFTYPE_MON_INFO,
-	},
-	{
-		.name		= "cbm_mask",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_default_ctrl_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
-	},
-	{
-		.name		= "min_cbm_bits",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_min_cbm_bits_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
-	},
-	{
-		.name		= "shareable_bits",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_shareable_bits_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
-	},
-	{
-		.name		= "bit_usage",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_bit_usage_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
-	},
-	{
-		.name		= "min_bandwidth",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_min_bw_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
-	},
-	{
-		.name		= "bandwidth_gran",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_bw_gran_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
-	},
-	{
-		.name		= "delay_linear",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_delay_linear_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
-	},
-	/*
-	 * Platform specific which (if any) capabilities are provided by
-	 * thread_throttle_mode. Defer "fflags" initialization to platform
-	 * discovery.
-	 */
-	{
-		.name		= "thread_throttle_mode",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_thread_throttle_mode_show,
-	},
-	{
-		.name		= "max_threshold_occupancy",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.write		= max_threshold_occ_write,
-		.seq_show	= max_threshold_occ_show,
-		.fflags		= RFTYPE_MON_INFO | RFTYPE_RES_CACHE,
-	},
-	{
-		.name		= "mbm_total_bytes_config",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= mbm_total_bytes_config_show,
-		.write		= mbm_total_bytes_config_write,
-	},
-	{
-		.name		= "mbm_local_bytes_config",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= mbm_local_bytes_config_show,
-		.write		= mbm_local_bytes_config_write,
-	},
-	{
-		.name		= "cpus",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.write		= rdtgroup_cpus_write,
-		.seq_show	= rdtgroup_cpus_show,
-		.fflags		= RFTYPE_BASE,
-	},
-	{
-		.name		= "cpus_list",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.write		= rdtgroup_cpus_write,
-		.seq_show	= rdtgroup_cpus_show,
-		.flags		= RFTYPE_FLAGS_CPUS_LIST,
-		.fflags		= RFTYPE_BASE,
-	},
-	{
-		.name		= "tasks",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.write		= rdtgroup_tasks_write,
-		.seq_show	= rdtgroup_tasks_show,
-		.fflags		= RFTYPE_BASE,
-	},
-	{
-		.name		= "mon_hw_id",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdtgroup_rmid_show,
-		.fflags		= RFTYPE_MON_BASE | RFTYPE_DEBUG,
-	},
-	{
-		.name		= "schemata",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.write		= rdtgroup_schemata_write,
-		.seq_show	= rdtgroup_schemata_show,
-		.fflags		= RFTYPE_CTRL_BASE,
-	},
-	{
-		.name		= "mode",
-		.mode		= 0644,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.write		= rdtgroup_mode_write,
-		.seq_show	= rdtgroup_mode_show,
-		.fflags		= RFTYPE_CTRL_BASE,
-	},
-	{
-		.name		= "size",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdtgroup_size_show,
-		.fflags		= RFTYPE_CTRL_BASE,
-	},
-	{
-		.name		= "sparse_masks",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdt_has_sparse_bitmasks_show,
-		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
-	},
-	{
-		.name		= "ctrl_hw_id",
-		.mode		= 0444,
-		.kf_ops		= &rdtgroup_kf_single_ops,
-		.seq_show	= rdtgroup_closid_show,
-		.fflags		= RFTYPE_CTRL_BASE | RFTYPE_DEBUG,
-	},
-
-};
-
-static int rdtgroup_add_files(struct kernfs_node *kn, unsigned long fflags)
-{
-	struct rftype *rfts, *rft;
-	int ret, len;
-
-	rfts = res_common_files;
-	len = ARRAY_SIZE(res_common_files);
-
-	lockdep_assert_held(&rdtgroup_mutex);
-
-	if (resctrl_debug)
-		fflags |= RFTYPE_DEBUG;
-
-	for (rft = rfts; rft < rfts + len; rft++) {
-		if (rft->fflags && ((fflags & rft->fflags) == rft->fflags)) {
-			ret = rdtgroup_add_file(kn, rft);
-			if (ret)
-				goto error;
-		}
-	}
-
-	return 0;
-error:
-	pr_warn("Failed to add %s, err=%d\n", rft->name, ret);
-	while (--rft >= rfts) {
-		if ((fflags & rft->fflags) == rft->fflags)
-			kernfs_remove_by_name(kn, rft->name);
-	}
-	return ret;
-}
-
-static struct rftype *rdtgroup_get_rftype_by_name(const char *name)
-{
-	struct rftype *rfts, *rft;
-	int len;
-
-	rfts = res_common_files;
-	len = ARRAY_SIZE(res_common_files);
-
-	for (rft = rfts; rft < rfts + len; rft++) {
-		if (!strcmp(rft->name, name))
-			return rft;
-	}
-
-	return NULL;
-}
-
-static void thread_throttle_mode_init(void)
-{
-	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
-	struct rftype *rft;
-
-	if (!r->alloc_capable ||
-	    r->membw.throttle_mode == THREAD_THROTTLE_UNDEFINED)
-		return;
-
-	rft = rdtgroup_get_rftype_by_name("thread_throttle_mode");
-	if (!rft)
-		return;
-
-	rft->fflags = RFTYPE_CTRL_INFO | RFTYPE_RES_MB;
-}
-
-void mbm_config_rftype_init(const char *config)
-{
-	struct rftype *rft;
-
-	rft = rdtgroup_get_rftype_by_name(config);
-	if (rft)
-		rft->fflags = RFTYPE_MON_INFO | RFTYPE_RES_CACHE;
-}
-
-/**
- * rdtgroup_kn_mode_restrict - Restrict user access to named resctrl file
- * @r: The resource group with which the file is associated.
- * @name: Name of the file
- *
- * The permissions of named resctrl file, directory, or link are modified
- * to not allow read, write, or execute by any user.
- *
- * WARNING: This function is intended to communicate to the user that the
- * resctrl file has been locked down - that it is not relevant to the
- * particular state the system finds itself in. It should not be relied
- * on to protect from user access because after the file's permissions
- * are restricted the user can still change the permissions using chmod
- * from the command line.
- *
- * Return: 0 on success, <0 on failure.
- */
-int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name)
-{
-	struct iattr iattr = {.ia_valid = ATTR_MODE,};
-	struct kernfs_node *kn;
-	int ret = 0;
-
-	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
-	if (!kn)
-		return -ENOENT;
-
-	switch (kernfs_type(kn)) {
-	case KERNFS_DIR:
-		iattr.ia_mode = S_IFDIR;
-		break;
-	case KERNFS_FILE:
-		iattr.ia_mode = S_IFREG;
-		break;
-	case KERNFS_LINK:
-		iattr.ia_mode = S_IFLNK;
-		break;
-	}
-
-	ret = kernfs_setattr(kn, &iattr);
-	kernfs_put(kn);
-	return ret;
-}
-
-/**
- * rdtgroup_kn_mode_restore - Restore user access to named resctrl file
- * @r: The resource group with which the file is associated.
- * @name: Name of the file
- * @mask: Mask of permissions that should be restored
- *
- * Restore the permissions of the named file. If @name is a directory the
- * permissions of its parent will be used.
- *
- * Return: 0 on success, <0 on failure.
- */
-int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
-			     umode_t mask)
-{
-	struct iattr iattr = {.ia_valid = ATTR_MODE,};
-	struct kernfs_node *kn, *parent;
-	struct rftype *rfts, *rft;
-	int ret, len;
-
-	rfts = res_common_files;
-	len = ARRAY_SIZE(res_common_files);
-
-	for (rft = rfts; rft < rfts + len; rft++) {
-		if (!strcmp(rft->name, name))
-			iattr.ia_mode = rft->mode & mask;
-	}
-
-	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
-	if (!kn)
-		return -ENOENT;
-
-	switch (kernfs_type(kn)) {
-	case KERNFS_DIR:
-		parent = kernfs_get_parent(kn);
-		if (parent) {
-			iattr.ia_mode |= parent->mode;
-			kernfs_put(parent);
-		}
-		iattr.ia_mode |= S_IFDIR;
-		break;
-	case KERNFS_FILE:
-		iattr.ia_mode |= S_IFREG;
-		break;
-	case KERNFS_LINK:
-		iattr.ia_mode |= S_IFLNK;
-		break;
-	}
-
-	ret = kernfs_setattr(kn, &iattr);
-	kernfs_put(kn);
-	return ret;
-}
-
-static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
-				      unsigned long fflags)
-{
-	struct kernfs_node *kn_subdir;
-	int ret;
-
-	kn_subdir = kernfs_create_dir(kn_info, name,
-				      kn_info->mode, priv);
-	if (IS_ERR(kn_subdir))
-		return PTR_ERR(kn_subdir);
-
-	ret = rdtgroup_kn_set_ugid(kn_subdir);
-	if (ret)
-		return ret;
-
-	ret = rdtgroup_add_files(kn_subdir, fflags);
-	if (!ret)
-		kernfs_activate(kn_subdir);
-
-	return ret;
-}
-
-static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
-{
-	enum resctrl_res_level i;
-	struct resctrl_schema *s;
-	struct rdt_resource *r;
-	unsigned long fflags;
-	char name[32];
-	int ret;
-
-	/* create the directory */
-	kn_info = kernfs_create_dir(parent_kn, "info", parent_kn->mode, NULL);
-	if (IS_ERR(kn_info))
-		return PTR_ERR(kn_info);
-
-	ret = rdtgroup_add_files(kn_info, RFTYPE_TOP_INFO);
-	if (ret)
-		goto out_destroy;
-
-	/* loop over enabled controls, these are all alloc_capable */
-	list_for_each_entry(s, &resctrl_schema_all, list) {
-		r = s->res;
-		fflags = r->fflags | RFTYPE_CTRL_INFO;
-		ret = rdtgroup_mkdir_info_resdir(s, s->name, fflags);
-		if (ret)
-			goto out_destroy;
-	}
-
-	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
-		r = resctrl_arch_get_resource(i);
-		if (!r->mon_capable)
-			continue;
-
-		fflags =  r->fflags | RFTYPE_MON_INFO;
-		sprintf(name, "%s_MON", r->name);
-		ret = rdtgroup_mkdir_info_resdir(r, name, fflags);
-		if (ret)
-			goto out_destroy;
-	}
-
-	ret = rdtgroup_kn_set_ugid(kn_info);
-	if (ret)
-		goto out_destroy;
-
-	kernfs_activate(kn_info);
-
-	return 0;
-
-out_destroy:
-	kernfs_remove(kn_info);
-	return ret;
-}
-
-static int
-mongroup_create_dir(struct kernfs_node *parent_kn, struct rdtgroup *prgrp,
-		    char *name, struct kernfs_node **dest_kn)
-{
-	struct kernfs_node *kn;
-	int ret;
-
-	/* create the directory */
-	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
-	if (IS_ERR(kn))
-		return PTR_ERR(kn);
-
-	if (dest_kn)
-		*dest_kn = kn;
-
-	ret = rdtgroup_kn_set_ugid(kn);
-	if (ret)
-		goto out_destroy;
-
-	kernfs_activate(kn);
-
-	return 0;
-
-out_destroy:
-	kernfs_remove(kn);
-	return ret;
-}
-
 static void l3_qos_cfg_update(void *arg)
 {
 	bool *enable = arg;
@@ -2307,11 +116,6 @@ static void l2_qos_cfg_update(void *arg)
 	wrmsrl(MSR_IA32_L2_QOS_CFG, *enable ? L2_QOS_CDP_ENABLE : 0ULL);
 }
 
-static inline bool is_mba_linear(void)
-{
-	return resctrl_arch_get_resource(RDT_RESOURCE_MBA)->membw.delay_linear;
-}
-
 static int set_cache_qos_cfg(int level, bool enable)
 {
 	void (*update)(void *arg);
@@ -2367,66 +171,6 @@ void rdt_domain_reconfigure_cdp(struct rdt_resource *r)
 		l3_qos_cfg_update(&hw_res->cdp_enabled);
 }
 
-static int mba_sc_domain_allocate(struct rdt_resource *r, struct rdt_domain *d)
-{
-	u32 num_closid = resctrl_arch_get_num_closid(r);
-	int cpu = cpumask_any(&d->cpu_mask);
-	int i;
-
-	d->mbps_val = kcalloc_node(num_closid, sizeof(*d->mbps_val),
-				   GFP_KERNEL, cpu_to_node(cpu));
-	if (!d->mbps_val)
-		return -ENOMEM;
-
-	for (i = 0; i < num_closid; i++)
-		d->mbps_val[i] = MBA_MAX_MBPS;
-
-	return 0;
-}
-
-static void mba_sc_domain_destroy(struct rdt_resource *r,
-				  struct rdt_domain *d)
-{
-	kfree(d->mbps_val);
-	d->mbps_val = NULL;
-}
-
-/*
- * MBA software controller is supported only if
- * MBM is supported and MBA is in linear scale.
- */
-static bool supports_mba_mbps(void)
-{
-	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
-
-	return (resctrl_arch_is_mbm_local_enabled() &&
-		r->alloc_capable && is_mba_linear());
-}
-
-/*
- * Enable or disable the MBA software controller
- * which helps user specify bandwidth in MBps.
- */
-static int set_mba_sc(bool mba_sc)
-{
-	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
-	u32 num_closid = resctrl_arch_get_num_closid(r);
-	struct rdt_domain *d;
-	int i;
-
-	if (!supports_mba_mbps() || mba_sc == is_mba_sc(r))
-		return -EINVAL;
-
-	r->membw.mba_sc = mba_sc;
-
-	list_for_each_entry(d, &r->domains, list) {
-		for (i = 0; i < num_closid; i++)
-			d->mbps_val[i] = MBA_MAX_MBPS;
-	}
-
-	return 0;
-}
-
 static int cdp_enable(int level)
 {
 	struct rdt_resource *r_l = &rdt_resources_all[level].r_resctrl;
@@ -2467,414 +211,6 @@ int resctrl_arch_set_cdp_enabled(enum resctrl_res_level l, bool enable)
 	return 0;
 }
 
-/*
- * We don't allow rdtgroup directories to be created anywhere
- * except the root directory. Thus when looking for the rdtgroup
- * structure for a kernfs node we are either looking at a directory,
- * in which case the rdtgroup structure is pointed at by the "priv"
- * field, otherwise we have a file, and need only look to the parent
- * to find the rdtgroup.
- */
-static struct rdtgroup *kernfs_to_rdtgroup(struct kernfs_node *kn)
-{
-	if (kernfs_type(kn) == KERNFS_DIR) {
-		/*
-		 * All the resource directories use "kn->priv"
-		 * to point to the "struct rdtgroup" for the
-		 * resource. "info" and its subdirectories don't
-		 * have rdtgroup structures, so return NULL here.
-		 */
-		if (kn == kn_info || kn->parent == kn_info)
-			return NULL;
-		else
-			return kn->priv;
-	} else {
-		return kn->parent->priv;
-	}
-}
-
-static void rdtgroup_kn_get(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
-{
-	atomic_inc(&rdtgrp->waitcount);
-	kernfs_break_active_protection(kn);
-}
-
-static void rdtgroup_kn_put(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
-{
-	if (atomic_dec_and_test(&rdtgrp->waitcount) &&
-	    (rdtgrp->flags & RDT_DELETED)) {
-		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
-		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
-			rdtgroup_pseudo_lock_remove(rdtgrp);
-		kernfs_unbreak_active_protection(kn);
-		rdtgroup_remove(rdtgrp);
-	} else {
-		kernfs_unbreak_active_protection(kn);
-	}
-}
-
-struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn)
-{
-	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
-
-	if (!rdtgrp)
-		return NULL;
-
-	rdtgroup_kn_get(rdtgrp, kn);
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	/* Was this group deleted while we waited? */
-	if (rdtgrp->flags & RDT_DELETED)
-		return NULL;
-
-	return rdtgrp;
-}
-
-void rdtgroup_kn_unlock(struct kernfs_node *kn)
-{
-	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
-
-	if (!rdtgrp)
-		return;
-
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-
-	rdtgroup_kn_put(rdtgrp, kn);
-}
-
-static int mkdir_mondata_all(struct kernfs_node *parent_kn,
-			     struct rdtgroup *prgrp,
-			     struct kernfs_node **mon_data_kn);
-
-static void rdt_disable_ctx(void)
-{
-	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
-	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
-	set_mba_sc(false);
-
-	resctrl_debug = false;
-}
-
-static int rdt_enable_ctx(struct rdt_fs_context *ctx)
-{
-	int ret = 0;
-
-	if (ctx->enable_cdpl2) {
-		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, true);
-		if (ret)
-			goto out_done;
-	}
-
-	if (ctx->enable_cdpl3) {
-		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, true);
-		if (ret)
-			goto out_cdpl2;
-	}
-
-	if (ctx->enable_mba_mbps) {
-		ret = set_mba_sc(true);
-		if (ret)
-			goto out_cdpl3;
-	}
-
-	if (ctx->enable_debug)
-		resctrl_debug = true;
-
-	return 0;
-
-out_cdpl3:
-	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
-out_cdpl2:
-	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
-out_done:
-	return ret;
-}
-
-static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type)
-{
-	struct resctrl_schema *s;
-	const char *suffix = "";
-	int ret, cl;
-
-	s = kzalloc(sizeof(*s), GFP_KERNEL);
-	if (!s)
-		return -ENOMEM;
-
-	s->res = r;
-	s->num_closid = resctrl_arch_get_num_closid(r);
-	if (resctrl_arch_get_cdp_enabled(r->rid))
-		s->num_closid /= 2;
-
-	s->conf_type = type;
-	switch (type) {
-	case CDP_CODE:
-		suffix = "CODE";
-		break;
-	case CDP_DATA:
-		suffix = "DATA";
-		break;
-	case CDP_NONE:
-		suffix = "";
-		break;
-	}
-
-	ret = snprintf(s->name, sizeof(s->name), "%s%s", r->name, suffix);
-	if (ret >= sizeof(s->name)) {
-		kfree(s);
-		return -EINVAL;
-	}
-
-	cl = strlen(s->name);
-
-	/*
-	 * If CDP is supported by this resource, but not enabled,
-	 * include the suffix. This ensures the tabular format of the
-	 * schemata file does not change between mounts of the filesystem.
-	 */
-	if (r->cdp_capable && !resctrl_arch_get_cdp_enabled(r->rid))
-		cl += 4;
-
-	if (cl > max_name_width)
-		max_name_width = cl;
-
-	/*
-	 * Choose a width for the resource data based on the resource that has
-	 * widest name and cbm.
-	 */
-	max_data_width = max(max_data_width, r->data_width);
-
-	INIT_LIST_HEAD(&s->list);
-	list_add(&s->list, &resctrl_schema_all);
-
-	return 0;
-}
-
-static int schemata_list_create(void)
-{
-	enum resctrl_res_level i;
-	struct rdt_resource *r;
-	int ret = 0;
-
-	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
-		r = resctrl_arch_get_resource(i);
-		if (!r->alloc_capable)
-			continue;
-
-		if (resctrl_arch_get_cdp_enabled(r->rid)) {
-			ret = schemata_list_add(r, CDP_CODE);
-			if (ret)
-				break;
-
-			ret = schemata_list_add(r, CDP_DATA);
-		} else {
-			ret = schemata_list_add(r, CDP_NONE);
-		}
-
-		if (ret)
-			break;
-	}
-
-	return ret;
-}
-
-static void schemata_list_destroy(void)
-{
-	struct resctrl_schema *s, *tmp;
-
-	list_for_each_entry_safe(s, tmp, &resctrl_schema_all, list) {
-		list_del(&s->list);
-		kfree(s);
-	}
-}
-
-static int rdt_get_tree(struct fs_context *fc)
-{
-	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
-	struct rdt_fs_context *ctx = rdt_fc2context(fc);
-	unsigned long flags = RFTYPE_CTRL_BASE;
-	struct rdt_domain *dom;
-	int ret;
-
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-	/*
-	 * resctrl file system can only be mounted once.
-	 */
-	if (resctrl_mounted) {
-		ret = -EBUSY;
-		goto out;
-	}
-
-	ret = rdtgroup_setup_root(ctx);
-	if (ret)
-		goto out;
-
-	ret = rdt_enable_ctx(ctx);
-	if (ret)
-		goto out_root;
-
-	ret = schemata_list_create();
-	if (ret) {
-		schemata_list_destroy();
-		goto out_ctx;
-	}
-
-	closid_init();
-
-	if (resctrl_arch_mon_capable())
-		flags |= RFTYPE_MON;
-
-	ret = rdtgroup_add_files(rdtgroup_default.kn, flags);
-	if (ret)
-		goto out_schemata_free;
-
-	kernfs_activate(rdtgroup_default.kn);
-
-	ret = rdtgroup_create_info_dir(rdtgroup_default.kn);
-	if (ret < 0)
-		goto out_schemata_free;
-
-	if (resctrl_arch_mon_capable()) {
-		ret = mongroup_create_dir(rdtgroup_default.kn,
-					  &rdtgroup_default, "mon_groups",
-					  &kn_mongrp);
-		if (ret < 0)
-			goto out_info;
-
-		ret = mkdir_mondata_all(rdtgroup_default.kn,
-					&rdtgroup_default, &kn_mondata);
-		if (ret < 0)
-			goto out_mongrp;
-		rdtgroup_default.mon.mon_data_kn = kn_mondata;
-	}
-
-	ret = rdt_pseudo_lock_init();
-	if (ret)
-		goto out_mondata;
-
-	ret = kernfs_get_tree(fc);
-	if (ret < 0)
-		goto out_psl;
-
-	if (resctrl_arch_alloc_capable())
-		resctrl_arch_enable_alloc();
-	if (resctrl_arch_mon_capable())
-		resctrl_arch_enable_mon();
-
-	if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable())
-		resctrl_mounted = true;
-
-	if (resctrl_is_mbm_enabled()) {
-		list_for_each_entry(dom, &l3->domains, list)
-			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
-						   RESCTRL_PICK_ANY_CPU);
-	}
-
-	goto out;
-
-out_psl:
-	rdt_pseudo_lock_release();
-out_mondata:
-	if (resctrl_arch_mon_capable())
-		kernfs_remove(kn_mondata);
-out_mongrp:
-	if (resctrl_arch_mon_capable())
-		kernfs_remove(kn_mongrp);
-out_info:
-	kernfs_remove(kn_info);
-out_schemata_free:
-	schemata_list_destroy();
-out_ctx:
-	rdt_disable_ctx();
-out_root:
-	rdtgroup_destroy_root();
-out:
-	rdt_last_cmd_clear();
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-	return ret;
-}
-
-enum rdt_param {
-	Opt_cdp,
-	Opt_cdpl2,
-	Opt_mba_mbps,
-	Opt_debug,
-	nr__rdt_params
-};
-
-static const struct fs_parameter_spec rdt_fs_parameters[] = {
-	fsparam_flag("cdp",		Opt_cdp),
-	fsparam_flag("cdpl2",		Opt_cdpl2),
-	fsparam_flag("mba_MBps",	Opt_mba_mbps),
-	fsparam_flag("debug",		Opt_debug),
-	{}
-};
-
-static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
-{
-	struct rdt_fs_context *ctx = rdt_fc2context(fc);
-	struct fs_parse_result result;
-	int opt;
-
-	opt = fs_parse(fc, rdt_fs_parameters, param, &result);
-	if (opt < 0)
-		return opt;
-
-	switch (opt) {
-	case Opt_cdp:
-		ctx->enable_cdpl3 = true;
-		return 0;
-	case Opt_cdpl2:
-		ctx->enable_cdpl2 = true;
-		return 0;
-	case Opt_mba_mbps:
-		if (!supports_mba_mbps())
-			return -EINVAL;
-		ctx->enable_mba_mbps = true;
-		return 0;
-	case Opt_debug:
-		ctx->enable_debug = true;
-		return 0;
-	}
-
-	return -EINVAL;
-}
-
-static void rdt_fs_context_free(struct fs_context *fc)
-{
-	struct rdt_fs_context *ctx = rdt_fc2context(fc);
-
-	kernfs_free_fs_context(fc);
-	kfree(ctx);
-}
-
-static const struct fs_context_operations rdt_fs_context_ops = {
-	.free		= rdt_fs_context_free,
-	.parse_param	= rdt_parse_param,
-	.get_tree	= rdt_get_tree,
-};
-
-static int rdt_init_fs_context(struct fs_context *fc)
-{
-	struct rdt_fs_context *ctx;
-
-	ctx = kzalloc(sizeof(struct rdt_fs_context), GFP_KERNEL);
-	if (!ctx)
-		return -ENOMEM;
-
-	ctx->kfc.magic = RDTGROUP_SUPER_MAGIC;
-	fc->fs_private = &ctx->kfc;
-	fc->ops = &rdt_fs_context_ops;
-	put_user_ns(fc->user_ns);
-	fc->user_ns = get_user_ns(&init_user_ns);
-	fc->global = true;
-	return 0;
-}
-
 static int reset_all_ctrls(struct rdt_resource *r)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
@@ -2922,1333 +258,3 @@ void resctrl_arch_reset_resources(void)
 	for_each_capable_rdt_resource(r)
 		reset_all_ctrls(r);
 }
-
-/*
- * Move tasks from one to the other group. If @from is NULL, then all tasks
- * in the systems are moved unconditionally (used for teardown).
- *
- * If @mask is not NULL the cpus on which moved tasks are running are set
- * in that mask so the update smp function call is restricted to affected
- * cpus.
- */
-static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,
-				 struct cpumask *mask)
-{
-	struct task_struct *p, *t;
-
-	read_lock(&tasklist_lock);
-	for_each_process_thread(p, t) {
-		if (!from || is_closid_match(t, from) ||
-		    is_rmid_match(t, from)) {
-			resctrl_arch_set_closid_rmid(t, to->closid,
-						     to->mon.rmid);
-
-			/*
-			 * Order the closid/rmid stores above before the loads
-			 * in task_curr(). This pairs with the full barrier
-			 * between the rq->curr update and
-			 * resctrl_arch_sched_in() during context switch.
-			 */
-			smp_mb();
-
-			/*
-			 * If the task is on a CPU, set the CPU in the mask.
-			 * The detection is inaccurate as tasks might move or
-			 * schedule before the smp function call takes place.
-			 * In such a case the function call is pointless, but
-			 * there is no other side effect.
-			 */
-			if (IS_ENABLED(CONFIG_SMP) && mask && task_curr(t))
-				cpumask_set_cpu(task_cpu(t), mask);
-		}
-	}
-	read_unlock(&tasklist_lock);
-}
-
-static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp)
-{
-	struct rdtgroup *sentry, *stmp;
-	struct list_head *head;
-
-	head = &rdtgrp->mon.crdtgrp_list;
-	list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) {
-		free_rmid(sentry->closid, sentry->mon.rmid);
-		list_del(&sentry->mon.crdtgrp_list);
-
-		if (atomic_read(&sentry->waitcount) != 0)
-			sentry->flags = RDT_DELETED;
-		else
-			rdtgroup_remove(sentry);
-	}
-}
-
-/*
- * Forcibly remove all of subdirectories under root.
- */
-static void rmdir_all_sub(void)
-{
-	struct rdtgroup *rdtgrp, *tmp;
-
-	/* Move all tasks to the default resource group */
-	rdt_move_group_tasks(NULL, &rdtgroup_default, NULL);
-
-	list_for_each_entry_safe(rdtgrp, tmp, &rdt_all_groups, rdtgroup_list) {
-		/* Free any child rmids */
-		free_all_child_rdtgrp(rdtgrp);
-
-		/* Remove each rdtgroup other than root */
-		if (rdtgrp == &rdtgroup_default)
-			continue;
-
-		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
-		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
-			rdtgroup_pseudo_lock_remove(rdtgrp);
-
-		/*
-		 * Give any CPUs back to the default group. We cannot copy
-		 * cpu_online_mask because a CPU might have executed the
-		 * offline callback already, but is still marked online.
-		 */
-		cpumask_or(&rdtgroup_default.cpu_mask,
-			   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
-
-		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
-
-		kernfs_remove(rdtgrp->kn);
-		list_del(&rdtgrp->rdtgroup_list);
-
-		if (atomic_read(&rdtgrp->waitcount) != 0)
-			rdtgrp->flags = RDT_DELETED;
-		else
-			rdtgroup_remove(rdtgrp);
-	}
-	/* Notify online CPUs to update per cpu storage and PQR_ASSOC MSR */
-	update_closid_rmid(cpu_online_mask, &rdtgroup_default);
-
-	kernfs_remove(kn_info);
-	kernfs_remove(kn_mongrp);
-	kernfs_remove(kn_mondata);
-}
-
-static void rdt_kill_sb(struct super_block *sb)
-{
-	cpus_read_lock();
-	mutex_lock(&rdtgroup_mutex);
-
-	rdt_disable_ctx();
-
-	/* Put everything back to default values. */
-	resctrl_arch_reset_resources();
-
-	rmdir_all_sub();
-	rdt_pseudo_lock_release();
-	rdtgroup_default.mode = RDT_MODE_SHAREABLE;
-	schemata_list_destroy();
-	rdtgroup_destroy_root();
-	if (resctrl_arch_alloc_capable())
-		resctrl_arch_disable_alloc();
-	if (resctrl_arch_mon_capable())
-		resctrl_arch_disable_mon();
-	resctrl_mounted = false;
-	kernfs_kill_sb(sb);
-	mutex_unlock(&rdtgroup_mutex);
-	cpus_read_unlock();
-}
-
-static struct file_system_type rdt_fs_type = {
-	.name			= "resctrl",
-	.init_fs_context	= rdt_init_fs_context,
-	.parameters		= rdt_fs_parameters,
-	.kill_sb		= rdt_kill_sb,
-};
-
-static int mon_addfile(struct kernfs_node *parent_kn, const char *name,
-		       void *priv)
-{
-	struct kernfs_node *kn;
-	int ret = 0;
-
-	kn = __kernfs_create_file(parent_kn, name, 0444,
-				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, 0,
-				  &kf_mondata_ops, priv, NULL, NULL);
-	if (IS_ERR(kn))
-		return PTR_ERR(kn);
-
-	ret = rdtgroup_kn_set_ugid(kn);
-	if (ret) {
-		kernfs_remove(kn);
-		return ret;
-	}
-
-	return ret;
-}
-
-/*
- * Remove all subdirectories of mon_data of ctrl_mon groups
- * and monitor groups with given domain id.
- */
-static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
-					   unsigned int dom_id)
-{
-	struct rdtgroup *prgrp, *crgrp;
-	char name[32];
-
-	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
-		sprintf(name, "mon_%s_%02d", r->name, dom_id);
-		kernfs_remove_by_name(prgrp->mon.mon_data_kn, name);
-
-		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
-			kernfs_remove_by_name(crgrp->mon.mon_data_kn, name);
-	}
-}
-
-static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,
-				struct rdt_domain *d,
-				struct rdt_resource *r, struct rdtgroup *prgrp)
-{
-	union mon_data_bits priv;
-	struct kernfs_node *kn;
-	struct mon_evt *mevt;
-	struct rmid_read rr;
-	char name[32];
-	int ret;
-
-	sprintf(name, "mon_%s_%02d", r->name, d->id);
-	/* create the directory */
-	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
-	if (IS_ERR(kn))
-		return PTR_ERR(kn);
-
-	ret = rdtgroup_kn_set_ugid(kn);
-	if (ret)
-		goto out_destroy;
-
-	if (WARN_ON(list_empty(&r->evt_list))) {
-		ret = -EPERM;
-		goto out_destroy;
-	}
-
-	priv.u.rid = r->rid;
-	priv.u.domid = d->id;
-	list_for_each_entry(mevt, &r->evt_list, list) {
-		priv.u.evtid = mevt->evtid;
-		ret = mon_addfile(kn, mevt->name, priv.priv);
-		if (ret)
-			goto out_destroy;
-
-		if (resctrl_is_mbm_event(mevt->evtid))
-			mon_event_read(&rr, r, d, prgrp, mevt->evtid, true);
-	}
-	kernfs_activate(kn);
-	return 0;
-
-out_destroy:
-	kernfs_remove(kn);
-	return ret;
-}
-
-/*
- * Add all subdirectories of mon_data for "ctrl_mon" groups
- * and "monitor" groups with given domain id.
- */
-static void mkdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
-					   struct rdt_domain *d)
-{
-	struct kernfs_node *parent_kn;
-	struct rdtgroup *prgrp, *crgrp;
-	struct list_head *head;
-
-	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
-		parent_kn = prgrp->mon.mon_data_kn;
-		mkdir_mondata_subdir(parent_kn, d, r, prgrp);
-
-		head = &prgrp->mon.crdtgrp_list;
-		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
-			parent_kn = crgrp->mon.mon_data_kn;
-			mkdir_mondata_subdir(parent_kn, d, r, crgrp);
-		}
-	}
-}
-
-static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn,
-				       struct rdt_resource *r,
-				       struct rdtgroup *prgrp)
-{
-	struct rdt_domain *dom;
-	int ret;
-
-	/* Walking r->domains, ensure it can't race with cpuhp */
-	lockdep_assert_cpus_held();
-
-	list_for_each_entry(dom, &r->domains, list) {
-		ret = mkdir_mondata_subdir(parent_kn, dom, r, prgrp);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-
-/*
- * This creates a directory mon_data which contains the monitored data.
- *
- * mon_data has one directory for each domain which are named
- * in the format mon_<domain_name>_<domain_id>. For ex: A mon_data
- * with L3 domain looks as below:
- * ./mon_data:
- * mon_L3_00
- * mon_L3_01
- * mon_L3_02
- * ...
- *
- * Each domain directory has one file per event:
- * ./mon_L3_00/:
- * llc_occupancy
- *
- */
-static int mkdir_mondata_all(struct kernfs_node *parent_kn,
-			     struct rdtgroup *prgrp,
-			     struct kernfs_node **dest_kn)
-{
-	enum resctrl_res_level i;
-	struct rdt_resource *r;
-	struct kernfs_node *kn;
-	int ret;
-
-	/*
-	 * Create the mon_data directory first.
-	 */
-	ret = mongroup_create_dir(parent_kn, prgrp, "mon_data", &kn);
-	if (ret)
-		return ret;
-
-	if (dest_kn)
-		*dest_kn = kn;
-
-	/*
-	 * Create the subdirectories for each domain. Note that all events
-	 * in a domain like L3 are grouped into a resource whose domain is L3
-	 */
-	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
-		r = resctrl_arch_get_resource(i);
-		if (!r->mon_capable)
-			continue;
-
-		ret = mkdir_mondata_subdir_alldom(kn, r, prgrp);
-		if (ret)
-			goto out_destroy;
-	}
-
-	return 0;
-
-out_destroy:
-	kernfs_remove(kn);
-	return ret;
-}
-
-/**
- * cbm_ensure_valid - Enforce validity on provided CBM
- * @_val:	Candidate CBM
- * @r:		RDT resource to which the CBM belongs
- *
- * The provided CBM represents all cache portions available for use. This
- * may be represented by a bitmap that does not consist of contiguous ones
- * and thus be an invalid CBM.
- * Here the provided CBM is forced to be a valid CBM by only considering
- * the first set of contiguous bits as valid and clearing all bits.
- * The intention here is to provide a valid default CBM with which a new
- * resource group is initialized. The user can follow this with a
- * modification to the CBM if the default does not satisfy the
- * requirements.
- */
-static u32 cbm_ensure_valid(u32 _val, struct rdt_resource *r)
-{
-	unsigned int cbm_len = r->cache.cbm_len;
-	unsigned long first_bit, zero_bit;
-	unsigned long val = _val;
-
-	if (!val)
-		return 0;
-
-	first_bit = find_first_bit(&val, cbm_len);
-	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
-
-	/* Clear any remaining bits to ensure contiguous region */
-	bitmap_clear(&val, zero_bit, cbm_len - zero_bit);
-	return (u32)val;
-}
-
-/*
- * Initialize cache resources per RDT domain
- *
- * Set the RDT domain up to start off with all usable allocations. That is,
- * all shareable and unused bits. All-zero CBM is invalid.
- */
-static int __init_one_rdt_domain(struct rdt_domain *d, struct resctrl_schema *s,
-				 u32 closid)
-{
-	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
-	enum resctrl_conf_type t = s->conf_type;
-	struct resctrl_staged_config *cfg;
-	struct rdt_resource *r = s->res;
-	u32 used_b = 0, unused_b = 0;
-	unsigned long tmp_cbm;
-	enum rdtgrp_mode mode;
-	u32 peer_ctl, ctrl_val;
-	int i;
-
-	cfg = &d->staged_config[t];
-	cfg->have_new_ctrl = false;
-	cfg->new_ctrl = r->cache.shareable_bits;
-	used_b = r->cache.shareable_bits;
-	for (i = 0; i < closids_supported(); i++) {
-		if (closid_allocated(i) && i != closid) {
-			mode = rdtgroup_mode_by_closid(i);
-			if (mode == RDT_MODE_PSEUDO_LOCKSETUP)
-				/*
-				 * ctrl values for locksetup aren't relevant
-				 * until the schemata is written, and the mode
-				 * becomes RDT_MODE_PSEUDO_LOCKED.
-				 */
-				continue;
-			/*
-			 * If CDP is active include peer domain's
-			 * usage to ensure there is no overlap
-			 * with an exclusive group.
-			 */
-			if (resctrl_arch_get_cdp_enabled(r->rid))
-				peer_ctl = resctrl_arch_get_config(r, d, i,
-								   peer_type);
-			else
-				peer_ctl = 0;
-			ctrl_val = resctrl_arch_get_config(r, d, i,
-							   s->conf_type);
-			used_b |= ctrl_val | peer_ctl;
-			if (mode == RDT_MODE_SHAREABLE)
-				cfg->new_ctrl |= ctrl_val | peer_ctl;
-		}
-	}
-	if (d->plr && d->plr->cbm > 0)
-		used_b |= d->plr->cbm;
-	unused_b = used_b ^ (BIT_MASK(r->cache.cbm_len) - 1);
-	unused_b &= BIT_MASK(r->cache.cbm_len) - 1;
-	cfg->new_ctrl |= unused_b;
-	/*
-	 * Force the initial CBM to be valid, user can
-	 * modify the CBM based on system availability.
-	 */
-	cfg->new_ctrl = cbm_ensure_valid(cfg->new_ctrl, r);
-	/*
-	 * Assign the u32 CBM to an unsigned long to ensure that
-	 * bitmap_weight() does not access out-of-bound memory.
-	 */
-	tmp_cbm = cfg->new_ctrl;
-	if (bitmap_weight(&tmp_cbm, r->cache.cbm_len) < r->cache.min_cbm_bits) {
-		rdt_last_cmd_printf("No space on %s:%d\n", s->name, d->id);
-		return -ENOSPC;
-	}
-	cfg->have_new_ctrl = true;
-
-	return 0;
-}
-
-/*
- * Initialize cache resources with default values.
- *
- * A new RDT group is being created on an allocation capable (CAT)
- * supporting system. Set this group up to start off with all usable
- * allocations.
- *
- * If there are no more shareable bits available on any domain then
- * the entire allocation will fail.
- */
-static int rdtgroup_init_cat(struct resctrl_schema *s, u32 closid)
-{
-	struct rdt_domain *d;
-	int ret;
-
-	list_for_each_entry(d, &s->res->domains, list) {
-		ret = __init_one_rdt_domain(d, s, closid);
-		if (ret < 0)
-			return ret;
-	}
-
-	return 0;
-}
-
-/* Initialize MBA resource with default values. */
-static void rdtgroup_init_mba(struct rdt_resource *r, u32 closid)
-{
-	struct resctrl_staged_config *cfg;
-	struct rdt_domain *d;
-
-	list_for_each_entry(d, &r->domains, list) {
-		if (is_mba_sc(r)) {
-			d->mbps_val[closid] = MBA_MAX_MBPS;
-			continue;
-		}
-
-		cfg = &d->staged_config[CDP_NONE];
-		cfg->new_ctrl = r->default_ctrl;
-		cfg->have_new_ctrl = true;
-	}
-}
-
-/* Initialize the RDT group's allocations. */
-static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
-{
-	struct resctrl_schema *s;
-	struct rdt_resource *r;
-	int ret = 0;
-
-	rdt_staged_configs_clear();
-
-	list_for_each_entry(s, &resctrl_schema_all, list) {
-		r = s->res;
-		if (r->rid == RDT_RESOURCE_MBA ||
-		    r->rid == RDT_RESOURCE_SMBA) {
-			rdtgroup_init_mba(r, rdtgrp->closid);
-			if (is_mba_sc(r))
-				continue;
-		} else {
-			ret = rdtgroup_init_cat(s, rdtgrp->closid);
-			if (ret < 0)
-				goto out;
-		}
-
-		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
-		if (ret < 0) {
-			rdt_last_cmd_puts("Failed to initialize allocations\n");
-			goto out;
-		}
-
-	}
-
-	rdtgrp->mode = RDT_MODE_SHAREABLE;
-
-out:
-	rdt_staged_configs_clear();
-	return ret;
-}
-
-static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)
-{
-	int ret;
-
-	if (!resctrl_arch_mon_capable())
-		return 0;
-
-	ret = alloc_rmid(rdtgrp->closid);
-	if (ret < 0) {
-		rdt_last_cmd_puts("Out of RMIDs\n");
-		return ret;
-	}
-	rdtgrp->mon.rmid = ret;
-
-	ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn);
-	if (ret) {
-		rdt_last_cmd_puts("kernfs subdir error\n");
-		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
-		return ret;
-	}
-
-	return 0;
-}
-
-static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp)
-{
-	if (resctrl_arch_mon_capable())
-		free_rmid(rgrp->closid, rgrp->mon.rmid);
-}
-
-static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,
-			     const char *name, umode_t mode,
-			     enum rdt_group_type rtype, struct rdtgroup **r)
-{
-	struct rdtgroup *prdtgrp, *rdtgrp;
-	unsigned long files = 0;
-	struct kernfs_node *kn;
-	int ret;
-
-	prdtgrp = rdtgroup_kn_lock_live(parent_kn);
-	if (!prdtgrp) {
-		ret = -ENODEV;
-		goto out_unlock;
-	}
-
-	if (rtype == RDTMON_GROUP &&
-	    (prdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
-	     prdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)) {
-		ret = -EINVAL;
-		rdt_last_cmd_puts("Pseudo-locking in progress\n");
-		goto out_unlock;
-	}
-
-	/* allocate the rdtgroup. */
-	rdtgrp = kzalloc(sizeof(*rdtgrp), GFP_KERNEL);
-	if (!rdtgrp) {
-		ret = -ENOSPC;
-		rdt_last_cmd_puts("Kernel out of memory\n");
-		goto out_unlock;
-	}
-	*r = rdtgrp;
-	rdtgrp->mon.parent = prdtgrp;
-	rdtgrp->type = rtype;
-	INIT_LIST_HEAD(&rdtgrp->mon.crdtgrp_list);
-
-	/* kernfs creates the directory for rdtgrp */
-	kn = kernfs_create_dir(parent_kn, name, mode, rdtgrp);
-	if (IS_ERR(kn)) {
-		ret = PTR_ERR(kn);
-		rdt_last_cmd_puts("kernfs create error\n");
-		goto out_free_rgrp;
-	}
-	rdtgrp->kn = kn;
-
-	/*
-	 * kernfs_remove() will drop the reference count on "kn" which
-	 * will free it. But we still need it to stick around for the
-	 * rdtgroup_kn_unlock(kn) call. Take one extra reference here,
-	 * which will be dropped by kernfs_put() in rdtgroup_remove().
-	 */
-	kernfs_get(kn);
-
-	ret = rdtgroup_kn_set_ugid(kn);
-	if (ret) {
-		rdt_last_cmd_puts("kernfs perm error\n");
-		goto out_destroy;
-	}
-
-	if (rtype == RDTCTRL_GROUP) {
-		files = RFTYPE_BASE | RFTYPE_CTRL;
-		if (resctrl_arch_mon_capable())
-			files |= RFTYPE_MON;
-	} else {
-		files = RFTYPE_BASE | RFTYPE_MON;
-	}
-
-	ret = rdtgroup_add_files(kn, files);
-	if (ret) {
-		rdt_last_cmd_puts("kernfs fill error\n");
-		goto out_destroy;
-	}
-
-	/*
-	 * The caller unlocks the parent_kn upon success.
-	 */
-	return 0;
-
-out_destroy:
-	kernfs_put(rdtgrp->kn);
-	kernfs_remove(rdtgrp->kn);
-out_free_rgrp:
-	kfree(rdtgrp);
-out_unlock:
-	rdtgroup_kn_unlock(parent_kn);
-	return ret;
-}
-
-static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp)
-{
-	kernfs_remove(rgrp->kn);
-	rdtgroup_remove(rgrp);
-}
-
-/*
- * Create a monitor group under "mon_groups" directory of a control
- * and monitor group(ctrl_mon). This is a resource group
- * to monitor a subset of tasks and cpus in its parent ctrl_mon group.
- */
-static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn,
-			      const char *name, umode_t mode)
-{
-	struct rdtgroup *rdtgrp, *prgrp;
-	int ret;
-
-	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTMON_GROUP, &rdtgrp);
-	if (ret)
-		return ret;
-
-	prgrp = rdtgrp->mon.parent;
-	rdtgrp->closid = prgrp->closid;
-
-	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
-	if (ret) {
-		mkdir_rdt_prepare_clean(rdtgrp);
-		goto out_unlock;
-	}
-
-	kernfs_activate(rdtgrp->kn);
-
-	/*
-	 * Add the rdtgrp to the list of rdtgrps the parent
-	 * ctrl_mon group has to track.
-	 */
-	list_add_tail(&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list);
-
-out_unlock:
-	rdtgroup_kn_unlock(parent_kn);
-	return ret;
-}
-
-/*
- * These are rdtgroups created under the root directory. Can be used
- * to allocate and monitor resources.
- */
-static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,
-				   const char *name, umode_t mode)
-{
-	struct rdtgroup *rdtgrp;
-	struct kernfs_node *kn;
-	u32 closid;
-	int ret;
-
-	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTCTRL_GROUP, &rdtgrp);
-	if (ret)
-		return ret;
-
-	kn = rdtgrp->kn;
-	ret = closid_alloc();
-	if (ret < 0) {
-		rdt_last_cmd_puts("Out of CLOSIDs\n");
-		goto out_common_fail;
-	}
-	closid = ret;
-	ret = 0;
-
-	rdtgrp->closid = closid;
-
-	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
-	if (ret)
-		goto out_closid_free;
-
-	kernfs_activate(rdtgrp->kn);
-
-	ret = rdtgroup_init_alloc(rdtgrp);
-	if (ret < 0)
-		goto out_rmid_free;
-
-	list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups);
-
-	if (resctrl_arch_mon_capable()) {
-		/*
-		 * Create an empty mon_groups directory to hold the subset
-		 * of tasks and cpus to monitor.
-		 */
-		ret = mongroup_create_dir(kn, rdtgrp, "mon_groups", NULL);
-		if (ret) {
-			rdt_last_cmd_puts("kernfs subdir error\n");
-			goto out_del_list;
-		}
-	}
-
-	goto out_unlock;
-
-out_del_list:
-	list_del(&rdtgrp->rdtgroup_list);
-out_rmid_free:
-	mkdir_rdt_prepare_rmid_free(rdtgrp);
-out_closid_free:
-	closid_free(closid);
-out_common_fail:
-	mkdir_rdt_prepare_clean(rdtgrp);
-out_unlock:
-	rdtgroup_kn_unlock(parent_kn);
-	return ret;
-}
-
-/*
- * We allow creating mon groups only with in a directory called "mon_groups"
- * which is present in every ctrl_mon group. Check if this is a valid
- * "mon_groups" directory.
- *
- * 1. The directory should be named "mon_groups".
- * 2. The mon group itself should "not" be named "mon_groups".
- *   This makes sure "mon_groups" directory always has a ctrl_mon group
- *   as parent.
- */
-static bool is_mon_groups(struct kernfs_node *kn, const char *name)
-{
-	return (!strcmp(kn->name, "mon_groups") &&
-		strcmp(name, "mon_groups"));
-}
-
-static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
-			  umode_t mode)
-{
-	/* Do not accept '\n' to avoid unparsable situation. */
-	if (strchr(name, '\n'))
-		return -EINVAL;
-
-	/*
-	 * If the parent directory is the root directory and RDT
-	 * allocation is supported, add a control and monitoring
-	 * subdirectory
-	 */
-	if (resctrl_arch_alloc_capable() && parent_kn == rdtgroup_default.kn)
-		return rdtgroup_mkdir_ctrl_mon(parent_kn, name, mode);
-
-	/*
-	 * If RDT monitoring is supported and the parent directory is a valid
-	 * "mon_groups" directory, add a monitoring subdirectory.
-	 */
-	if (resctrl_arch_mon_capable() && is_mon_groups(parent_kn, name))
-		return rdtgroup_mkdir_mon(parent_kn, name, mode);
-
-	return -EPERM;
-}
-
-static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
-{
-	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
-	u32 closid, rmid;
-	int cpu;
-
-	/* Give any tasks back to the parent group */
-	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
-
-	/* Update per cpu rmid of the moved CPUs first */
-	closid = rdtgrp->closid;
-	rmid = prdtgrp->mon.rmid;
-	for_each_cpu(cpu, &rdtgrp->cpu_mask)
-		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
-
-	/*
-	 * Update the MSR on moved CPUs and CPUs which have moved
-	 * task running on them.
-	 */
-	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
-	update_closid_rmid(tmpmask, NULL);
-
-	rdtgrp->flags = RDT_DELETED;
-	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
-
-	/*
-	 * Remove the rdtgrp from the parent ctrl_mon group's list
-	 */
-	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
-	list_del(&rdtgrp->mon.crdtgrp_list);
-
-	kernfs_remove(rdtgrp->kn);
-
-	return 0;
-}
-
-static int rdtgroup_ctrl_remove(struct rdtgroup *rdtgrp)
-{
-	rdtgrp->flags = RDT_DELETED;
-	list_del(&rdtgrp->rdtgroup_list);
-
-	kernfs_remove(rdtgrp->kn);
-	return 0;
-}
-
-static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
-{
-	u32 closid, rmid;
-	int cpu;
-
-	/* Give any tasks back to the default group */
-	rdt_move_group_tasks(rdtgrp, &rdtgroup_default, tmpmask);
-
-	/* Give any CPUs back to the default group */
-	cpumask_or(&rdtgroup_default.cpu_mask,
-		   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
-
-	/* Update per cpu closid and rmid of the moved CPUs first */
-	closid = rdtgroup_default.closid;
-	rmid = rdtgroup_default.mon.rmid;
-	for_each_cpu(cpu, &rdtgrp->cpu_mask)
-		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
-
-	/*
-	 * Update the MSR on moved CPUs and CPUs which have moved
-	 * task running on them.
-	 */
-	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
-	update_closid_rmid(tmpmask, NULL);
-
-	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
-	closid_free(rdtgrp->closid);
-
-	rdtgroup_ctrl_remove(rdtgrp);
-
-	/*
-	 * Free all the child monitor group rmids.
-	 */
-	free_all_child_rdtgrp(rdtgrp);
-
-	return 0;
-}
-
-static int rdtgroup_rmdir(struct kernfs_node *kn)
-{
-	struct kernfs_node *parent_kn = kn->parent;
-	struct rdtgroup *rdtgrp;
-	cpumask_var_t tmpmask;
-	int ret = 0;
-
-	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
-		return -ENOMEM;
-
-	rdtgrp = rdtgroup_kn_lock_live(kn);
-	if (!rdtgrp) {
-		ret = -EPERM;
-		goto out;
-	}
-
-	/*
-	 * If the rdtgroup is a ctrl_mon group and parent directory
-	 * is the root directory, remove the ctrl_mon group.
-	 *
-	 * If the rdtgroup is a mon group and parent directory
-	 * is a valid "mon_groups" directory, remove the mon group.
-	 */
-	if (rdtgrp->type == RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn &&
-	    rdtgrp != &rdtgroup_default) {
-		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
-		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
-			ret = rdtgroup_ctrl_remove(rdtgrp);
-		} else {
-			ret = rdtgroup_rmdir_ctrl(rdtgrp, tmpmask);
-		}
-	} else if (rdtgrp->type == RDTMON_GROUP &&
-		 is_mon_groups(parent_kn, kn->name)) {
-		ret = rdtgroup_rmdir_mon(rdtgrp, tmpmask);
-	} else {
-		ret = -EPERM;
-	}
-
-out:
-	rdtgroup_kn_unlock(kn);
-	free_cpumask_var(tmpmask);
-	return ret;
-}
-
-/**
- * mongrp_reparent() - replace parent CTRL_MON group of a MON group
- * @rdtgrp:		the MON group whose parent should be replaced
- * @new_prdtgrp:	replacement parent CTRL_MON group for @rdtgrp
- * @cpus:		cpumask provided by the caller for use during this call
- *
- * Replaces the parent CTRL_MON group for a MON group, resulting in all member
- * tasks' CLOSID immediately changing to that of the new parent group.
- * Monitoring data for the group is unaffected by this operation.
- */
-static void mongrp_reparent(struct rdtgroup *rdtgrp,
-			    struct rdtgroup *new_prdtgrp,
-			    cpumask_var_t cpus)
-{
-	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
-
-	WARN_ON(rdtgrp->type != RDTMON_GROUP);
-	WARN_ON(new_prdtgrp->type != RDTCTRL_GROUP);
-
-	/* Nothing to do when simply renaming a MON group. */
-	if (prdtgrp == new_prdtgrp)
-		return;
-
-	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
-	list_move_tail(&rdtgrp->mon.crdtgrp_list,
-		       &new_prdtgrp->mon.crdtgrp_list);
-
-	rdtgrp->mon.parent = new_prdtgrp;
-	rdtgrp->closid = new_prdtgrp->closid;
-
-	/* Propagate updated closid to all tasks in this group. */
-	rdt_move_group_tasks(rdtgrp, rdtgrp, cpus);
-
-	update_closid_rmid(cpus, NULL);
-}
-
-static int rdtgroup_rename(struct kernfs_node *kn,
-			   struct kernfs_node *new_parent, const char *new_name)
-{
-	struct rdtgroup *new_prdtgrp;
-	struct rdtgroup *rdtgrp;
-	cpumask_var_t tmpmask;
-	int ret;
-
-	rdtgrp = kernfs_to_rdtgroup(kn);
-	new_prdtgrp = kernfs_to_rdtgroup(new_parent);
-	if (!rdtgrp || !new_prdtgrp)
-		return -ENOENT;
-
-	/* Release both kernfs active_refs before obtaining rdtgroup mutex. */
-	rdtgroup_kn_get(rdtgrp, kn);
-	rdtgroup_kn_get(new_prdtgrp, new_parent);
-
-	mutex_lock(&rdtgroup_mutex);
-
-	rdt_last_cmd_clear();
-
-	/*
-	 * Don't allow kernfs_to_rdtgroup() to return a parent rdtgroup if
-	 * either kernfs_node is a file.
-	 */
-	if (kernfs_type(kn) != KERNFS_DIR ||
-	    kernfs_type(new_parent) != KERNFS_DIR) {
-		rdt_last_cmd_puts("Source and destination must be directories");
-		ret = -EPERM;
-		goto out;
-	}
-
-	if ((rdtgrp->flags & RDT_DELETED) || (new_prdtgrp->flags & RDT_DELETED)) {
-		ret = -ENOENT;
-		goto out;
-	}
-
-	if (rdtgrp->type != RDTMON_GROUP || !kn->parent ||
-	    !is_mon_groups(kn->parent, kn->name)) {
-		rdt_last_cmd_puts("Source must be a MON group\n");
-		ret = -EPERM;
-		goto out;
-	}
-
-	if (!is_mon_groups(new_parent, new_name)) {
-		rdt_last_cmd_puts("Destination must be a mon_groups subdirectory\n");
-		ret = -EPERM;
-		goto out;
-	}
-
-	/*
-	 * If the MON group is monitoring CPUs, the CPUs must be assigned to the
-	 * current parent CTRL_MON group and therefore cannot be assigned to
-	 * the new parent, making the move illegal.
-	 */
-	if (!cpumask_empty(&rdtgrp->cpu_mask) &&
-	    rdtgrp->mon.parent != new_prdtgrp) {
-		rdt_last_cmd_puts("Cannot move a MON group that monitors CPUs\n");
-		ret = -EPERM;
-		goto out;
-	}
-
-	/*
-	 * Allocate the cpumask for use in mongrp_reparent() to avoid the
-	 * possibility of failing to allocate it after kernfs_rename() has
-	 * succeeded.
-	 */
-	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL)) {
-		ret = -ENOMEM;
-		goto out;
-	}
-
-	/*
-	 * Perform all input validation and allocations needed to ensure
-	 * mongrp_reparent() will succeed before calling kernfs_rename(),
-	 * otherwise it would be necessary to revert this call if
-	 * mongrp_reparent() failed.
-	 */
-	ret = kernfs_rename(kn, new_parent, new_name);
-	if (!ret)
-		mongrp_reparent(rdtgrp, new_prdtgrp, tmpmask);
-
-	free_cpumask_var(tmpmask);
-
-out:
-	mutex_unlock(&rdtgroup_mutex);
-	rdtgroup_kn_put(rdtgrp, kn);
-	rdtgroup_kn_put(new_prdtgrp, new_parent);
-	return ret;
-}
-
-static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
-{
-	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3))
-		seq_puts(seq, ",cdp");
-
-	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2))
-		seq_puts(seq, ",cdpl2");
-
-	if (is_mba_sc(resctrl_arch_get_resource(RDT_RESOURCE_MBA)))
-		seq_puts(seq, ",mba_MBps");
-
-	if (resctrl_debug)
-		seq_puts(seq, ",debug");
-
-	return 0;
-}
-
-static struct kernfs_syscall_ops rdtgroup_kf_syscall_ops = {
-	.mkdir		= rdtgroup_mkdir,
-	.rmdir		= rdtgroup_rmdir,
-	.rename		= rdtgroup_rename,
-	.show_options	= rdtgroup_show_options,
-};
-
-static int rdtgroup_setup_root(struct rdt_fs_context *ctx)
-{
-	rdt_root = kernfs_create_root(&rdtgroup_kf_syscall_ops,
-				      KERNFS_ROOT_CREATE_DEACTIVATED |
-				      KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK,
-				      &rdtgroup_default);
-	if (IS_ERR(rdt_root))
-		return PTR_ERR(rdt_root);
-
-	ctx->kfc.root = rdt_root;
-	rdtgroup_default.kn = kernfs_root_to_node(rdt_root);
-
-	return 0;
-}
-
-static void rdtgroup_destroy_root(void)
-{
-	kernfs_destroy_root(rdt_root);
-	rdtgroup_default.kn = NULL;
-}
-
-static void rdtgroup_setup_default(void)
-{
-	mutex_lock(&rdtgroup_mutex);
-
-	rdtgroup_default.closid = RESCTRL_RESERVED_CLOSID;
-	rdtgroup_default.mon.rmid = RESCTRL_RESERVED_RMID;
-	rdtgroup_default.type = RDTCTRL_GROUP;
-	INIT_LIST_HEAD(&rdtgroup_default.mon.crdtgrp_list);
-
-	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
-
-	mutex_unlock(&rdtgroup_mutex);
-}
-
-static void domain_destroy_mon_state(struct rdt_domain *d)
-{
-	bitmap_free(d->rmid_busy_llc);
-	kfree(d->mbm_total);
-	kfree(d->mbm_local);
-}
-
-void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)
-{
-	mutex_lock(&rdtgroup_mutex);
-
-	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA)
-		mba_sc_domain_destroy(r, d);
-
-	if (!r->mon_capable)
-		goto out_unlock;
-
-	/*
-	 * If resctrl is mounted, remove all the
-	 * per domain monitor data directories.
-	 */
-	if (resctrl_mounted && resctrl_arch_mon_capable())
-		rmdir_mondata_subdir_allrdtgrp(r, d->id);
-
-	if (resctrl_is_mbm_enabled())
-		cancel_delayed_work(&d->mbm_over);
-	if (resctrl_arch_is_llc_occupancy_enabled() && has_busy_rmid(d)) {
-		/*
-		 * When a package is going down, forcefully
-		 * decrement rmid->ebusy. There is no way to know
-		 * that the L3 was flushed and hence may lead to
-		 * incorrect counts in rare scenarios, but leaving
-		 * the RMID as busy creates RMID leaks if the
-		 * package never comes back.
-		 */
-		__check_limbo(d, true);
-		cancel_delayed_work(&d->cqm_limbo);
-	}
-
-	domain_destroy_mon_state(d);
-
-out_unlock:
-	mutex_unlock(&rdtgroup_mutex);
-}
-
-static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
-{
-	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
-	size_t tsize;
-
-	if (resctrl_arch_is_llc_occupancy_enabled()) {
-		d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL);
-		if (!d->rmid_busy_llc)
-			return -ENOMEM;
-	}
-	if (resctrl_arch_is_mbm_total_enabled()) {
-		tsize = sizeof(*d->mbm_total);
-		d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL);
-		if (!d->mbm_total) {
-			bitmap_free(d->rmid_busy_llc);
-			return -ENOMEM;
-		}
-	}
-	if (resctrl_arch_is_mbm_local_enabled()) {
-		tsize = sizeof(*d->mbm_local);
-		d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL);
-		if (!d->mbm_local) {
-			bitmap_free(d->rmid_busy_llc);
-			kfree(d->mbm_total);
-			return -ENOMEM;
-		}
-	}
-
-	return 0;
-}
-
-int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)
-{
-	int err = 0;
-
-	mutex_lock(&rdtgroup_mutex);
-
-	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA) {
-		/* RDT_RESOURCE_MBA is never mon_capable */
-		err = mba_sc_domain_allocate(r, d);
-		goto out_unlock;
-	}
-
-	if (!r->mon_capable)
-		goto out_unlock;
-
-	err = domain_setup_mon_state(r, d);
-	if (err)
-		goto out_unlock;
-
-	if (resctrl_is_mbm_enabled()) {
-		INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow);
-		mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL,
-					   RESCTRL_PICK_ANY_CPU);
-	}
-
-	if (resctrl_arch_is_llc_occupancy_enabled())
-		INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo);
-
-	/*
-	 * If the filesystem is not mounted then only the default resource group
-	 * exists. Creation of its directories is deferred until mount time
-	 * by rdt_get_tree() calling mkdir_mondata_all().
-	 * If resctrl is mounted, add per domain monitor data directories.
-	 */
-	if (resctrl_mounted && resctrl_arch_mon_capable())
-		mkdir_mondata_subdir_allrdtgrp(r, d);
-
-out_unlock:
-	mutex_unlock(&rdtgroup_mutex);
-
-	return err;
-}
-
-void resctrl_online_cpu(unsigned int cpu)
-{
-	mutex_lock(&rdtgroup_mutex);
-	/* The CPU is set in default rdtgroup after online. */
-	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
-	mutex_unlock(&rdtgroup_mutex);
-}
-
-static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)
-{
-	struct rdtgroup *cr;
-
-	list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) {
-		if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask))
-			break;
-	}
-}
-
-void resctrl_offline_cpu(unsigned int cpu)
-{
-	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
-	struct rdtgroup *rdtgrp;
-	struct rdt_domain *d;
-
-	mutex_lock(&rdtgroup_mutex);
-	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
-		if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) {
-			clear_childcpus(rdtgrp, cpu);
-			break;
-		}
-	}
-
-	if (!l3->mon_capable)
-		goto out_unlock;
-
-	d = resctrl_get_domain_from_cpu(cpu, l3);
-	if (d) {
-		if (resctrl_is_mbm_enabled() && cpu == d->mbm_work_cpu) {
-			cancel_delayed_work(&d->mbm_over);
-			mbm_setup_overflow_handler(d, 0, cpu);
-		}
-		if (resctrl_arch_is_llc_occupancy_enabled() &&
-		    cpu == d->cqm_work_cpu && has_busy_rmid(d)) {
-			cancel_delayed_work(&d->cqm_limbo);
-			cqm_setup_limbo_handler(d, 0, cpu);
-		}
-	}
-
-out_unlock:
-	mutex_unlock(&rdtgroup_mutex);
-}
-
-/*
- * resctrl_init - resctrl filesystem initialization
- *
- * Setup resctrl file system including set up root, create mount point,
- * register resctrl filesystem, and initialize files under root directory.
- *
- * Return: 0 on success or -errno
- */
-int resctrl_init(void)
-{
-	int ret = 0;
-
-	seq_buf_init(&last_cmd_status, last_cmd_status_buf,
-		     sizeof(last_cmd_status_buf));
-
-	rdtgroup_setup_default();
-
-	thread_throttle_mode_init();
-
-	ret = resctrl_mon_resource_init();
-	if (ret)
-		return ret;
-
-	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
-	if (ret)
-		return ret;
-
-	ret = register_filesystem(&rdt_fs_type);
-	if (ret)
-		goto cleanup_mountpoint;
-
-	/*
-	 * Adding the resctrl debugfs directory here may not be ideal since
-	 * it would let the resctrl debugfs directory appear on the debugfs
-	 * filesystem before the resctrl filesystem is mounted.
-	 * It may also be ok since that would enable debugging of RDT before
-	 * resctrl is mounted.
-	 * The reason why the debugfs directory is created here and not in
-	 * rdt_get_tree() is because rdt_get_tree() takes rdtgroup_mutex and
-	 * during the debugfs directory creation also &sb->s_type->i_mutex_key
-	 * (the lockdep class of inode->i_rwsem). Other filesystem
-	 * interactions (eg. SyS_getdents) have the lock ordering:
-	 * &sb->s_type->i_mutex_key --> &mm->mmap_lock
-	 * During mmap(), called with &mm->mmap_lock, the rdtgroup_mutex
-	 * is taken, thus creating dependency:
-	 * &mm->mmap_lock --> rdtgroup_mutex for the latter that can cause
-	 * issues considering the other two lock dependencies.
-	 * By creating the debugfs directory here we avoid a dependency
-	 * that may cause deadlock (even though file operations cannot
-	 * occur until the filesystem is mounted, but I do not know how to
-	 * tell lockdep that).
-	 */
-	debugfs_resctrl = debugfs_create_dir("resctrl", NULL);
-
-	return 0;
-
-cleanup_mountpoint:
-	sysfs_remove_mount_point(fs_kobj, "resctrl");
-
-	return ret;
-}
-
-void resctrl_exit(void)
-{
-	debugfs_remove_recursive(debugfs_resctrl);
-	unregister_filesystem(&rdt_fs_type);
-	sysfs_remove_mount_point(fs_kobj, "resctrl");
-
-	resctrl_mon_resource_exit();
-}
diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
index e69de29bb2d1..a8f2dd66ede3 100644
--- a/fs/resctrl/ctrlmondata.c
+++ b/fs/resctrl/ctrlmondata.c
@@ -0,0 +1,527 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Resource Director Technology(RDT)
+ * - Cache Allocation code.
+ *
+ * Copyright (C) 2016 Intel Corporation
+ *
+ * Authors:
+ *    Fenghua Yu <fenghua.yu@intel.com>
+ *    Tony Luck <tony.luck@intel.com>
+ *
+ * More information about RDT be found in the Intel (R) x86 Architecture
+ * Software Developer Manual June 2016, volume 3, section 17.17.
+ */
+
+#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
+
+#include <linux/cpu.h>
+#include <linux/kernfs.h>
+#include <linux/seq_file.h>
+#include <linux/slab.h>
+#include "internal.h"
+
+struct rdt_parse_data {
+	struct rdtgroup		*rdtgrp;
+	char			*buf;
+};
+
+typedef int (ctrlval_parser_t)(struct rdt_parse_data *data,
+			       struct resctrl_schema *s,
+			       struct rdt_domain *d);
+
+/*
+ * Check whether MBA bandwidth percentage value is correct. The value is
+ * checked against the minimum and max bandwidth values specified by the
+ * hardware. The allocated bandwidth percentage is rounded to the next
+ * control step available on the hardware.
+ */
+static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
+{
+	unsigned long bw;
+	int ret;
+
+	/*
+	 * Only linear delay values is supported for current Intel SKUs.
+	 */
+	if (!r->membw.delay_linear && r->membw.arch_needs_linear) {
+		rdt_last_cmd_puts("No support for non-linear MB domains\n");
+		return false;
+	}
+
+	ret = kstrtoul(buf, 10, &bw);
+	if (ret) {
+		rdt_last_cmd_printf("Non-decimal digit in MB value %s\n", buf);
+		return false;
+	}
+
+	if ((bw < r->membw.min_bw || bw > r->default_ctrl) &&
+	    !is_mba_sc(r)) {
+		rdt_last_cmd_printf("MB value %ld out of range [%d,%d]\n", bw,
+				    r->membw.min_bw, r->default_ctrl);
+		return false;
+	}
+
+	*data = roundup(bw, (unsigned long)r->membw.bw_gran);
+	return true;
+}
+
+static int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
+		    struct rdt_domain *d)
+{
+	struct resctrl_staged_config *cfg;
+	u32 closid = data->rdtgrp->closid;
+	struct rdt_resource *r = s->res;
+	unsigned long bw_val;
+
+	cfg = &d->staged_config[s->conf_type];
+	if (cfg->have_new_ctrl) {
+		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
+		return -EINVAL;
+	}
+
+	if (!bw_validate(data->buf, &bw_val, r))
+		return -EINVAL;
+
+	if (is_mba_sc(r)) {
+		d->mbps_val[closid] = bw_val;
+		return 0;
+	}
+
+	cfg->new_ctrl = bw_val;
+	cfg->have_new_ctrl = true;
+
+	return 0;
+}
+
+/*
+ * Check whether a cache bit mask is valid.
+ * On Intel CPUs, non-contiguous 1s value support is indicated by CPUID:
+ *   - CPUID.0x10.1:ECX[3]: L3 non-contiguous 1s value supported if 1
+ *   - CPUID.0x10.2:ECX[3]: L2 non-contiguous 1s value supported if 1
+ *
+ * Haswell does not support a non-contiguous 1s value and additionally
+ * requires at least two bits set.
+ * AMD allows non-contiguous bitmasks.
+ */
+static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
+{
+	unsigned long first_bit, zero_bit, val;
+	unsigned int cbm_len = r->cache.cbm_len;
+	int ret;
+
+	ret = kstrtoul(buf, 16, &val);
+	if (ret) {
+		rdt_last_cmd_printf("Non-hex character in the mask %s\n", buf);
+		return false;
+	}
+
+	if ((r->cache.min_cbm_bits > 0 && val == 0) || val > r->default_ctrl) {
+		rdt_last_cmd_puts("Mask out of range\n");
+		return false;
+	}
+
+	first_bit = find_first_bit(&val, cbm_len);
+	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
+
+	/* Are non-contiguous bitmasks allowed? */
+	if (!r->cache.arch_has_sparse_bitmasks &&
+	    (find_next_bit(&val, cbm_len, zero_bit) < cbm_len)) {
+		rdt_last_cmd_printf("The mask %lx has non-consecutive 1-bits\n", val);
+		return false;
+	}
+
+	if ((zero_bit - first_bit) < r->cache.min_cbm_bits) {
+		rdt_last_cmd_printf("Need at least %d bits in the mask\n",
+				    r->cache.min_cbm_bits);
+		return false;
+	}
+
+	*data = val;
+	return true;
+}
+
+/*
+ * Read one cache bit mask (hex). Check that it is valid for the current
+ * resource type.
+ */
+static int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
+		     struct rdt_domain *d)
+{
+	struct rdtgroup *rdtgrp = data->rdtgrp;
+	struct resctrl_staged_config *cfg;
+	struct rdt_resource *r = s->res;
+	u32 cbm_val;
+
+	cfg = &d->staged_config[s->conf_type];
+	if (cfg->have_new_ctrl) {
+		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
+		return -EINVAL;
+	}
+
+	/*
+	 * Cannot set up more than one pseudo-locked region in a cache
+	 * hierarchy.
+	 */
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
+	    rdtgroup_pseudo_locked_in_hierarchy(d)) {
+		rdt_last_cmd_puts("Pseudo-locked region in hierarchy\n");
+		return -EINVAL;
+	}
+
+	if (!cbm_validate(data->buf, &cbm_val, r))
+		return -EINVAL;
+
+	if ((rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
+	     rdtgrp->mode == RDT_MODE_SHAREABLE) &&
+	    rdtgroup_cbm_overlaps_pseudo_locked(d, cbm_val)) {
+		rdt_last_cmd_puts("CBM overlaps with pseudo-locked region\n");
+		return -EINVAL;
+	}
+
+	/*
+	 * The CBM may not overlap with the CBM of another closid if
+	 * either is exclusive.
+	 */
+	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, true)) {
+		rdt_last_cmd_puts("Overlaps with exclusive group\n");
+		return -EINVAL;
+	}
+
+	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, false)) {
+		if (rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
+		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+			rdt_last_cmd_puts("Overlaps with other group\n");
+			return -EINVAL;
+		}
+	}
+
+	cfg->new_ctrl = cbm_val;
+	cfg->have_new_ctrl = true;
+
+	return 0;
+}
+
+static ctrlval_parser_t *get_parser(struct rdt_resource *res)
+{
+	if (res->fflags & RFTYPE_RES_CACHE)
+		return &parse_cbm;
+	else
+		return &parse_bw;
+}
+
+/*
+ * For each domain in this resource we expect to find a series of:
+ *	id=mask
+ * separated by ";". The "id" is in decimal, and must match one of
+ * the "id"s for this resource.
+ */
+static int parse_line(char *line, struct resctrl_schema *s,
+		      struct rdtgroup *rdtgrp)
+{
+	ctrlval_parser_t *parse_ctrlval = get_parser(s->res);
+	enum resctrl_conf_type t = s->conf_type;
+	struct resctrl_staged_config *cfg;
+	struct rdt_resource *r = s->res;
+	struct rdt_parse_data data;
+	char *dom = NULL, *id;
+	struct rdt_domain *d;
+	unsigned long dom_id;
+
+	/* Walking r->domains, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
+	    (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)) {
+		rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
+		return -EINVAL;
+	}
+
+next:
+	if (!line || line[0] == '\0')
+		return 0;
+	dom = strsep(&line, ";");
+	id = strsep(&dom, "=");
+	if (!dom || kstrtoul(id, 10, &dom_id)) {
+		rdt_last_cmd_puts("Missing '=' or non-numeric domain\n");
+		return -EINVAL;
+	}
+	dom = strim(dom);
+	list_for_each_entry(d, &r->domains, list) {
+		if (d->id == dom_id) {
+			data.buf = dom;
+			data.rdtgrp = rdtgrp;
+			if (parse_ctrlval(&data, s, d))
+				return -EINVAL;
+			if (rdtgrp->mode ==  RDT_MODE_PSEUDO_LOCKSETUP) {
+				cfg = &d->staged_config[t];
+				/*
+				 * In pseudo-locking setup mode and just
+				 * parsed a valid CBM that should be
+				 * pseudo-locked. Only one locked region per
+				 * resource group and domain so just do
+				 * the required initialization for single
+				 * region and return.
+				 */
+				rdtgrp->plr->s = s;
+				rdtgrp->plr->d = d;
+				rdtgrp->plr->cbm = cfg->new_ctrl;
+				d->plr = rdtgrp->plr;
+				return 0;
+			}
+			goto next;
+		}
+	}
+	return -EINVAL;
+}
+
+static int rdtgroup_parse_resource(char *resname, char *tok,
+				   struct rdtgroup *rdtgrp)
+{
+	struct resctrl_schema *s;
+
+	list_for_each_entry(s, &resctrl_schema_all, list) {
+		if (!strcmp(resname, s->name) && rdtgrp->closid < s->num_closid)
+			return parse_line(tok, s, rdtgrp);
+	}
+	rdt_last_cmd_printf("Unknown or unsupported resource name '%s'\n", resname);
+	return -EINVAL;
+}
+
+ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
+				char *buf, size_t nbytes, loff_t off)
+{
+	struct resctrl_schema *s;
+	struct rdtgroup *rdtgrp;
+	struct rdt_resource *r;
+	char *tok, *resname;
+	int ret = 0;
+
+	/* Valid input requires a trailing newline */
+	if (nbytes == 0 || buf[nbytes - 1] != '\n')
+		return -EINVAL;
+	buf[nbytes - 1] = '\0';
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		rdtgroup_kn_unlock(of->kn);
+		return -ENOENT;
+	}
+	rdt_last_cmd_clear();
+
+	/*
+	 * No changes to pseudo-locked region allowed. It has to be removed
+	 * and re-created instead.
+	 */
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
+		ret = -EINVAL;
+		rdt_last_cmd_puts("Resource group is pseudo-locked\n");
+		goto out;
+	}
+
+	rdt_staged_configs_clear();
+
+	while ((tok = strsep(&buf, "\n")) != NULL) {
+		resname = strim(strsep(&tok, ":"));
+		if (!tok) {
+			rdt_last_cmd_puts("Missing ':'\n");
+			ret = -EINVAL;
+			goto out;
+		}
+		if (tok[0] == '\0') {
+			rdt_last_cmd_printf("Missing '%s' value\n", resname);
+			ret = -EINVAL;
+			goto out;
+		}
+		ret = rdtgroup_parse_resource(resname, tok, rdtgrp);
+		if (ret)
+			goto out;
+	}
+
+	list_for_each_entry(s, &resctrl_schema_all, list) {
+		r = s->res;
+
+		/*
+		 * Writes to mba_sc resources update the software controller,
+		 * not the control MSR.
+		 */
+		if (is_mba_sc(r))
+			continue;
+
+		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
+		if (ret)
+			goto out;
+	}
+
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+		/*
+		 * If pseudo-locking fails we keep the resource group in
+		 * mode RDT_MODE_PSEUDO_LOCKSETUP with its class of service
+		 * active and updated for just the domain the pseudo-locked
+		 * region was requested for.
+		 */
+		ret = rdtgroup_pseudo_lock_create(rdtgrp);
+	}
+
+out:
+	rdt_staged_configs_clear();
+	rdtgroup_kn_unlock(of->kn);
+	return ret ?: nbytes;
+}
+
+static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int closid)
+{
+	struct rdt_resource *r = schema->res;
+	struct rdt_domain *dom;
+	bool sep = false;
+	u32 ctrl_val;
+
+	/* Walking r->domains, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+	seq_printf(s, "%*s:", max_name_width, schema->name);
+	list_for_each_entry(dom, &r->domains, list) {
+		if (sep)
+			seq_puts(s, ";");
+
+		if (is_mba_sc(r))
+			ctrl_val = dom->mbps_val[closid];
+		else
+			ctrl_val = resctrl_arch_get_config(r, dom, closid,
+							   schema->conf_type);
+
+		seq_printf(s, r->format_str, dom->id, max_data_width,
+			   ctrl_val);
+		sep = true;
+	}
+	seq_puts(s, "\n");
+}
+
+int rdtgroup_schemata_show(struct kernfs_open_file *of,
+			   struct seq_file *s, void *v)
+{
+	struct resctrl_schema *schema;
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+	u32 closid;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (rdtgrp) {
+		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+			list_for_each_entry(schema, &resctrl_schema_all, list) {
+				seq_printf(s, "%s:uninitialized\n", schema->name);
+			}
+		} else if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
+			if (!rdtgrp->plr->d) {
+				rdt_last_cmd_clear();
+				rdt_last_cmd_puts("Cache domain offline\n");
+				ret = -ENODEV;
+			} else {
+				seq_printf(s, "%s:%d=%x\n",
+					   rdtgrp->plr->s->res->name,
+					   rdtgrp->plr->d->id,
+					   rdtgrp->plr->cbm);
+			}
+		} else {
+			closid = rdtgrp->closid;
+			list_for_each_entry(schema, &resctrl_schema_all, list) {
+				if (closid < schema->num_closid)
+					show_doms(s, schema, closid);
+			}
+		}
+	} else {
+		ret = -ENOENT;
+	}
+	rdtgroup_kn_unlock(of->kn);
+	return ret;
+}
+
+static int smp_mon_event_count(void *arg)
+{
+	mon_event_count(arg);
+
+	return 0;
+}
+
+void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
+		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
+		    int evtid, int first)
+{
+	int cpu;
+
+	/* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+	/*
+	 * Setup the parameters to pass to mon_event_count() to read the data.
+	 */
+	rr->rgrp = rdtgrp;
+	rr->evtid = evtid;
+	rr->r = r;
+	rr->d = d;
+	rr->val = 0;
+	rr->first = first;
+	rr->arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, evtid);
+	if (IS_ERR(rr->arch_mon_ctx)) {
+		rr->err = -EINVAL;
+		return;
+	}
+
+	cpu = cpumask_any_housekeeping(&d->cpu_mask, RESCTRL_PICK_ANY_CPU);
+
+	/*
+	 * cpumask_any_housekeeping() prefers housekeeping CPUs, but
+	 * are all the CPUs nohz_full? If yes, pick a CPU to IPI.
+	 * MPAM's resctrl_arch_rmid_read() is unable to read the
+	 * counters on some platforms if its called in IRQ context.
+	 */
+	if (tick_nohz_full_cpu(cpu))
+		smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1);
+	else
+		smp_call_on_cpu(cpu, smp_mon_event_count, rr, false);
+
+	resctrl_arch_mon_ctx_free(r, evtid, rr->arch_mon_ctx);
+}
+
+int rdtgroup_mondata_show(struct seq_file *m, void *arg)
+{
+	struct kernfs_open_file *of = m->private;
+	u32 resid, evtid, domid;
+	struct rdtgroup *rdtgrp;
+	struct rdt_resource *r;
+	union mon_data_bits md;
+	struct rdt_domain *d;
+	struct rmid_read rr;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	md.priv = of->kn->priv;
+	resid = md.u.rid;
+	domid = md.u.domid;
+	evtid = md.u.evtid;
+
+	r = resctrl_arch_get_resource(resid);
+	d = resctrl_arch_find_domain(r, domid);
+	if (IS_ERR_OR_NULL(d)) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	mon_event_read(&rr, r, d, rdtgrp, evtid, false);
+
+	if (rr.err == -EIO)
+		seq_puts(m, "Error\n");
+	else if (rr.err == -EINVAL)
+		seq_puts(m, "Unavailable\n");
+	else
+		seq_printf(m, "%llu\n", rr.val);
+
+out:
+	rdtgroup_kn_unlock(of->kn);
+	return ret;
+}
diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index e69de29bb2d1..f73267762a87 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -0,0 +1,340 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _FS_RESCTRL_INTERNAL_H
+#define _FS_RESCTRL_INTERNAL_H
+
+#include <linux/resctrl.h>
+#include <linux/sched.h>
+#include <linux/kernfs.h>
+#include <linux/fs_context.h>
+#include <linux/jump_label.h>
+#include <linux/tick.h>
+
+#include <asm/resctrl.h>
+
+/**
+ * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that
+ *			        aren't marked nohz_full
+ * @mask:	The mask to pick a CPU from.
+ * @exclude_cpu:The CPU to avoid picking.
+ *
+ * Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping
+ * CPUs that don't use nohz_full, these are preferred. Pass
+ * RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs.
+ *
+ * When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available.
+ */
+static inline unsigned int
+cpumask_any_housekeeping(const struct cpumask *mask, int exclude_cpu)
+{
+	unsigned int cpu, hk_cpu;
+
+	if (exclude_cpu == RESCTRL_PICK_ANY_CPU)
+		cpu = cpumask_any(mask);
+	else
+		cpu = cpumask_any_but(mask, exclude_cpu);
+
+	if (!IS_ENABLED(CONFIG_NO_HZ_FULL))
+		return cpu;
+
+	/* If the CPU picked isn't marked nohz_full nothing more needs doing. */
+	if (cpu < nr_cpu_ids && !tick_nohz_full_cpu(cpu))
+		return cpu;
+
+	/* Try to find a CPU that isn't nohz_full to use in preference */
+	hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask);
+	if (hk_cpu == exclude_cpu)
+		hk_cpu = cpumask_nth_andnot(1, mask, tick_nohz_full_mask);
+
+	if (hk_cpu < nr_cpu_ids)
+		cpu = hk_cpu;
+
+	return cpu;
+}
+
+struct rdt_fs_context {
+	struct kernfs_fs_context	kfc;
+	bool				enable_cdpl2;
+	bool				enable_cdpl3;
+	bool				enable_mba_mbps;
+	bool				enable_debug;
+};
+
+static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
+{
+	struct kernfs_fs_context *kfc = fc->fs_private;
+
+	return container_of(kfc, struct rdt_fs_context, kfc);
+}
+
+/**
+ * struct mon_evt - Entry in the event list of a resource
+ * @evtid:		event id
+ * @name:		name of the event
+ * @configurable:	true if the event is configurable
+ * @list:		entry in &rdt_resource->evt_list
+ */
+struct mon_evt {
+	enum resctrl_event_id	evtid;
+	char			*name;
+	bool			configurable;
+	struct list_head	list;
+};
+
+/**
+ * union mon_data_bits - Monitoring details for each event file
+ * @priv:              Used to store monitoring event data in @u
+ *                     as kernfs private data
+ * @rid:               Resource id associated with the event file
+ * @evtid:             Event id associated with the event file
+ * @domid:             The domain to which the event file belongs
+ * @u:                 Name of the bit fields struct
+ */
+union mon_data_bits {
+	void *priv;
+	struct {
+		unsigned int rid		: 10;
+		enum resctrl_event_id evtid	: 8;
+		unsigned int domid		: 14;
+	} u;
+};
+
+struct rmid_read {
+	struct rdtgroup		*rgrp;
+	struct rdt_resource	*r;
+	struct rdt_domain	*d;
+	enum resctrl_event_id	evtid;
+	bool			first;
+	int			err;
+	u64			val;
+	void			*arch_mon_ctx;
+};
+
+extern struct list_head resctrl_schema_all;
+extern bool resctrl_mounted;
+
+enum rdt_group_type {
+	RDTCTRL_GROUP = 0,
+	RDTMON_GROUP,
+	RDT_NUM_GROUP,
+};
+
+/**
+ * enum rdtgrp_mode - Mode of a RDT resource group
+ * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations
+ * @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed
+ * @RDT_MODE_PSEUDO_LOCKSETUP: Resource group will be used for Pseudo-Locking
+ * @RDT_MODE_PSEUDO_LOCKED: No sharing of this resource group's allocations
+ *                          allowed AND the allocations are Cache Pseudo-Locked
+ * @RDT_NUM_MODES: Total number of modes
+ *
+ * The mode of a resource group enables control over the allowed overlap
+ * between allocations associated with different resource groups (classes
+ * of service). User is able to modify the mode of a resource group by
+ * writing to the "mode" resctrl file associated with the resource group.
+ *
+ * The "shareable", "exclusive", and "pseudo-locksetup" modes are set by
+ * writing the appropriate text to the "mode" file. A resource group enters
+ * "pseudo-locked" mode after the schemata is written while the resource
+ * group is in "pseudo-locksetup" mode.
+ */
+enum rdtgrp_mode {
+	RDT_MODE_SHAREABLE = 0,
+	RDT_MODE_EXCLUSIVE,
+	RDT_MODE_PSEUDO_LOCKSETUP,
+	RDT_MODE_PSEUDO_LOCKED,
+
+	/* Must be last */
+	RDT_NUM_MODES,
+};
+
+/**
+ * struct mongroup - store mon group's data in resctrl fs.
+ * @mon_data_kn:		kernfs node for the mon_data directory
+ * @parent:			parent rdtgrp
+ * @crdtgrp_list:		child rdtgroup node list
+ * @rmid:			rmid for this rdtgroup
+ */
+struct mongroup {
+	struct kernfs_node	*mon_data_kn;
+	struct rdtgroup		*parent;
+	struct list_head	crdtgrp_list;
+	u32			rmid;
+};
+
+/**
+ * struct rdtgroup - store rdtgroup's data in resctrl file system.
+ * @kn:				kernfs node
+ * @rdtgroup_list:		linked list for all rdtgroups
+ * @closid:			closid for this rdtgroup
+ * @cpu_mask:			CPUs assigned to this rdtgroup
+ * @flags:			status bits
+ * @waitcount:			how many cpus expect to find this
+ *				group when they acquire rdtgroup_mutex
+ * @type:			indicates type of this rdtgroup - either
+ *				monitor only or ctrl_mon group
+ * @mon:			mongroup related data
+ * @mode:			mode of resource group
+ * @plr:			pseudo-locked region
+ */
+struct rdtgroup {
+	struct kernfs_node		*kn;
+	struct list_head		rdtgroup_list;
+	u32				closid;
+	struct cpumask			cpu_mask;
+	int				flags;
+	atomic_t			waitcount;
+	enum rdt_group_type		type;
+	struct mongroup			mon;
+	enum rdtgrp_mode		mode;
+	struct pseudo_lock_region	*plr;
+};
+
+/* List of all resource groups */
+extern struct list_head rdt_all_groups;
+
+extern int max_name_width, max_data_width;
+
+/**
+ * struct rftype - describe each file in the resctrl file system
+ * @name:	File name
+ * @mode:	Access mode
+ * @kf_ops:	File operations
+ * @flags:	File specific RFTYPE_FLAGS_* flags
+ * @fflags:	File specific RFTYPE_* flags
+ * @seq_show:	Show content of the file
+ * @write:	Write to the file
+ */
+struct rftype {
+	char			*name;
+	umode_t			mode;
+	const struct kernfs_ops	*kf_ops;
+	unsigned long		flags;
+	unsigned long		fflags;
+
+	int (*seq_show)(struct kernfs_open_file *of,
+			struct seq_file *sf, void *v);
+	/*
+	 * write() is the generic write callback which maps directly to
+	 * kernfs write operation and overrides all other operations.
+	 * Maximum write size is determined by ->max_write_len.
+	 */
+	ssize_t (*write)(struct kernfs_open_file *of,
+			 char *buf, size_t nbytes, loff_t off);
+};
+
+/**
+ * struct mbm_state - status for each MBM counter in each domain
+ * @prev_bw_bytes: Previous bytes value read for bandwidth calculation
+ * @prev_bw:	The most recent bandwidth in MBps
+ */
+struct mbm_state {
+	u64	prev_bw_bytes;
+	u32	prev_bw;
+};
+
+static inline  bool is_mba_sc(struct rdt_resource *r)
+{
+	if (!r)
+		r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
+
+	/*
+	 * The software controller support is only applicable to MBA resource.
+	 * Make sure to check for resource type.
+	 */
+	if (r->rid != RDT_RESOURCE_MBA)
+		return false;
+
+	return r->membw.mba_sc;
+}
+
+extern struct mutex rdtgroup_mutex;
+extern struct rdtgroup rdtgroup_default;
+extern struct dentry *debugfs_resctrl;
+
+void rdt_last_cmd_clear(void);
+void rdt_last_cmd_puts(const char *s);
+__printf(1, 2)
+void rdt_last_cmd_printf(const char *fmt, ...);
+
+struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
+void rdtgroup_kn_unlock(struct kernfs_node *kn);
+int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name);
+int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
+			     umode_t mask);
+ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
+				char *buf, size_t nbytes, loff_t off);
+int rdtgroup_schemata_show(struct kernfs_open_file *of,
+			   struct seq_file *s, void *v);
+bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
+			   unsigned long cbm, int closid, bool exclusive);
+unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d,
+				  unsigned long cbm);
+enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);
+int rdtgroup_tasks_assigned(struct rdtgroup *r);
+int closids_supported(void);
+void closid_free(int closid);
+int alloc_rmid(u32 closid);
+void free_rmid(u32 closid, u32 rmid);
+void resctrl_mon_resource_exit(void);
+void mon_event_count(void *info);
+int rdtgroup_mondata_show(struct seq_file *m, void *arg);
+void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
+		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
+		    int evtid, int first);
+int resctrl_mon_resource_init(void);
+void mbm_setup_overflow_handler(struct rdt_domain *dom,
+				unsigned long delay_ms,
+				int exclude_cpu);
+void mbm_handle_overflow(struct work_struct *work);
+bool is_mba_sc(struct rdt_resource *r);
+void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
+			     int exclude_cpu);
+void cqm_handle_limbo(struct work_struct *work);
+bool has_busy_rmid(struct rdt_domain *d);
+void __check_limbo(struct rdt_domain *d, bool force_free);
+void mbm_config_rftype_init(const char *config);
+void rdt_staged_configs_clear(void);
+bool closid_allocated(unsigned int closid);
+int resctrl_find_cleanest_closid(void);
+
+#ifdef CONFIG_RESCTRL_FS_PSEUDO_LOCK
+int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
+int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp);
+bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm);
+bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d);
+int rdt_pseudo_lock_init(void);
+void rdt_pseudo_lock_release(void);
+int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
+void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
+#else
+static inline int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
+{
+	return false;
+}
+
+static inline bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
+{
+	return false;
+}
+
+static inline int rdt_pseudo_lock_init(void) { return 0; }
+static inline void rdt_pseudo_lock_release(void) { }
+static inline int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp) { }
+#endif /* CONFIG_RESCTRL_FS_PSEUDO_LOCK */
+
+#endif /* _FS_RESCTRL_INTERNAL_H */
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index e69de29bb2d1..06f660dfd929 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -0,0 +1,843 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Resource Director Technology(RDT)
+ * - Monitoring code
+ *
+ * Copyright (C) 2017 Intel Corporation
+ *
+ * Author:
+ *    Vikas Shivappa <vikas.shivappa@intel.com>
+ *
+ * This replaces the cqm.c based on perf but we reuse a lot of
+ * code and datastructures originally from Peter Zijlstra and Matt Fleming.
+ *
+ * More information about RDT be found in the Intel (R) x86 Architecture
+ * Software Developer Manual June 2016, volume 3, section 17.17.
+ */
+
+#include <linux/cpu.h>
+#include <linux/module.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include "internal.h"
+
+/*
+ * struct rmid_entry - dirty tracking for all RMID.
+ * @closid:	The CLOSID for this entry.
+ * @rmid:	The RMID for this entry.
+ * @busy:	The number of domains with cached data using this RMID.
+ * @list:	Member of the rmid_free_lru list when busy == 0.
+ *
+ * Depending on the architecture the correct monitor is accessed using
+ * both @closid and @rmid, or @rmid only.
+ *
+ * Take the rdtgroup_mutex when accessing.
+ */
+struct rmid_entry {
+	u32				closid;
+	u32				rmid;
+	int				busy;
+	struct list_head		list;
+};
+
+/*
+ * @rmid_free_lru - A least recently used list of free RMIDs
+ *     These RMIDs are guaranteed to have an occupancy less than the
+ *     threshold occupancy
+ */
+static LIST_HEAD(rmid_free_lru);
+
+/*
+ * @closid_num_dirty_rmid    The number of dirty RMID each CLOSID has.
+ *     Only allocated when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined.
+ *     Indexed by CLOSID. Protected by rdtgroup_mutex.
+ */
+static u32 *closid_num_dirty_rmid;
+
+/*
+ * @rmid_limbo_count - count of currently unused but (potentially)
+ *     dirty RMIDs.
+ *     This counts RMIDs that no one is currently using but that
+ *     may have a occupancy value > resctrl_rmid_realloc_threshold. User can
+ *     change the threshold occupancy value.
+ */
+static unsigned int rmid_limbo_count;
+
+/*
+ * @rmid_entry - The entry in the limbo and free lists.
+ */
+static struct rmid_entry	*rmid_ptrs;
+
+/*
+ * This is the threshold cache occupancy in bytes at which we will consider an
+ * RMID available for re-allocation.
+ */
+unsigned int resctrl_rmid_realloc_threshold;
+
+/*
+ * This is the maximum value for the reallocation threshold, in bytes.
+ */
+unsigned int resctrl_rmid_realloc_limit;
+
+/*
+ * x86 and arm64 differ in their handling of monitoring.
+ * x86's RMID are independent numbers, there is only one source of traffic
+ * with an RMID value of '1'.
+ * arm64's PMG extends the PARTID/CLOSID space, there are multiple sources of
+ * traffic with a PMG value of '1', one for each CLOSID, meaning the RMID
+ * value is no longer unique.
+ * To account for this, resctrl uses an index. On x86 this is just the RMID,
+ * on arm64 it encodes the CLOSID and RMID. This gives a unique number.
+ *
+ * The domain's rmid_busy_llc and rmid_ptrs[] are sized by index. The arch code
+ * must accept an attempt to read every index.
+ */
+static inline struct rmid_entry *__rmid_entry(u32 idx)
+{
+	struct rmid_entry *entry;
+	u32 closid, rmid;
+
+	entry = &rmid_ptrs[idx];
+	resctrl_arch_rmid_idx_decode(idx, &closid, &rmid);
+
+	WARN_ON_ONCE(entry->closid != closid);
+	WARN_ON_ONCE(entry->rmid != rmid);
+
+	return entry;
+}
+
+static void limbo_release_entry(struct rmid_entry *entry)
+{
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	rmid_limbo_count--;
+	list_add_tail(&entry->list, &rmid_free_lru);
+
+	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
+		closid_num_dirty_rmid[entry->closid]--;
+}
+
+/*
+ * Check the RMIDs that are marked as busy for this domain. If the
+ * reported LLC occupancy is below the threshold clear the busy bit and
+ * decrement the count. If the busy count gets to zero on an RMID, we
+ * free the RMID
+ */
+void __check_limbo(struct rdt_domain *d, bool force_free)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
+	struct rmid_entry *entry;
+	u32 idx, cur_idx = 1;
+	void *arch_mon_ctx;
+	bool rmid_dirty;
+	u64 val = 0;
+
+	arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID);
+	if (IS_ERR(arch_mon_ctx)) {
+		pr_warn_ratelimited("Failed to allocate monitor context: %ld",
+				    PTR_ERR(arch_mon_ctx));
+		return;
+	}
+
+	/*
+	 * Skip RMID 0 and start from RMID 1 and check all the RMIDs that
+	 * are marked as busy for occupancy < threshold. If the occupancy
+	 * is less than the threshold decrement the busy counter of the
+	 * RMID and move it to the free list when the counter reaches 0.
+	 */
+	for (;;) {
+		idx = find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx);
+		if (idx >= idx_limit)
+			break;
+
+		entry = __rmid_entry(idx);
+		if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid,
+					   QOS_L3_OCCUP_EVENT_ID, &val,
+					   arch_mon_ctx)) {
+			rmid_dirty = true;
+		} else {
+			rmid_dirty = (val >= resctrl_rmid_realloc_threshold);
+		}
+
+		if (force_free || !rmid_dirty) {
+			clear_bit(idx, d->rmid_busy_llc);
+			if (!--entry->busy)
+				limbo_release_entry(entry);
+		}
+		cur_idx = idx + 1;
+	}
+
+	resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx);
+}
+
+bool has_busy_rmid(struct rdt_domain *d)
+{
+	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
+
+	return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit;
+}
+
+static struct rmid_entry *resctrl_find_free_rmid(u32 closid)
+{
+	struct rmid_entry *itr;
+	u32 itr_idx, cmp_idx;
+
+	if (list_empty(&rmid_free_lru))
+		return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC);
+
+	list_for_each_entry(itr, &rmid_free_lru, list) {
+		/*
+		 * Get the index of this free RMID, and the index it would need
+		 * to be if it were used with this CLOSID.
+		 * If the CLOSID is irrelevant on this architecture, the two
+		 * index values are always the same on every entry and thus the
+		 * very first entry will be returned.
+		 */
+		itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid);
+		cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid);
+
+		if (itr_idx == cmp_idx)
+			return itr;
+	}
+
+	return ERR_PTR(-ENOSPC);
+}
+
+/**
+ * resctrl_find_cleanest_closid() - Find a CLOSID where all the associated
+ *                                  RMID are clean, or the CLOSID that has
+ *                                  the most clean RMID.
+ *
+ * MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocated CLOSID
+ * may not be able to allocate clean RMID. To avoid this the allocator will
+ * choose the CLOSID with the most clean RMID.
+ *
+ * When the CLOSID and RMID are independent numbers, the first free CLOSID will
+ * be returned.
+ */
+int resctrl_find_cleanest_closid(void)
+{
+	u32 cleanest_closid = ~0;
+	int i = 0;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
+		return -EIO;
+
+	for (i = 0; i < closids_supported(); i++) {
+		int num_dirty;
+
+		if (closid_allocated(i))
+			continue;
+
+		num_dirty = closid_num_dirty_rmid[i];
+		if (num_dirty == 0)
+			return i;
+
+		if (cleanest_closid == ~0)
+			cleanest_closid = i;
+
+		if (num_dirty < closid_num_dirty_rmid[cleanest_closid])
+			cleanest_closid = i;
+	}
+
+	if (cleanest_closid == ~0)
+		return -ENOSPC;
+
+	return cleanest_closid;
+}
+
+/*
+ * For MPAM the RMID value is not unique, and has to be considered with
+ * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which
+ * allows all domains to be managed by a single free list.
+ * Each domain also has a rmid_busy_llc to reduce the work of the limbo handler.
+ */
+int alloc_rmid(u32 closid)
+{
+	struct rmid_entry *entry;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	entry = resctrl_find_free_rmid(closid);
+	if (IS_ERR(entry))
+		return PTR_ERR(entry);
+
+	list_del(&entry->list);
+	return entry->rmid;
+}
+
+static void add_rmid_to_limbo(struct rmid_entry *entry)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+	struct rdt_domain *d;
+	u32 idx;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	/* Walking r->domains, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+	idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid);
+
+	entry->busy = 0;
+	list_for_each_entry(d, &r->domains, list) {
+		/*
+		 * For the first limbo RMID in the domain,
+		 * setup up the limbo worker.
+		 */
+		if (!has_busy_rmid(d))
+			cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL,
+						RESCTRL_PICK_ANY_CPU);
+		set_bit(idx, d->rmid_busy_llc);
+		entry->busy++;
+	}
+
+	rmid_limbo_count++;
+	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
+		closid_num_dirty_rmid[entry->closid]++;
+}
+
+void free_rmid(u32 closid, u32 rmid)
+{
+	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+	struct rmid_entry *entry;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	/*
+	 * Do not allow the default rmid to be free'd. Comparing by index
+	 * allows architectures that ignore the closid parameter to avoid an
+	 * unnecessary check.
+	 */
+	if (idx == resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
+						RESCTRL_RESERVED_RMID))
+		return;
+
+	entry = __rmid_entry(idx);
+
+	if (resctrl_arch_is_llc_occupancy_enabled())
+		add_rmid_to_limbo(entry);
+	else
+		list_add_tail(&entry->list, &rmid_free_lru);
+}
+
+static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid,
+				       u32 rmid, enum resctrl_event_id evtid)
+{
+	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+
+	switch (evtid) {
+	case QOS_L3_MBM_TOTAL_EVENT_ID:
+		return &d->mbm_total[idx];
+	case QOS_L3_MBM_LOCAL_EVENT_ID:
+		return &d->mbm_local[idx];
+	default:
+		return NULL;
+	}
+}
+
+static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr)
+{
+	struct mbm_state *m;
+	u64 tval = 0;
+
+	if (rr->first) {
+		resctrl_arch_reset_rmid(rr->r, rr->d, closid, rmid, rr->evtid);
+		m = get_mbm_state(rr->d, closid, rmid, rr->evtid);
+		if (m)
+			memset(m, 0, sizeof(struct mbm_state));
+		return 0;
+	}
+
+	rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid,
+					 &tval, rr->arch_mon_ctx);
+	if (rr->err)
+		return rr->err;
+
+	rr->val += tval;
+
+	return 0;
+}
+
+/*
+ * mbm_bw_count() - Update bw count from values previously read by
+ *		    __mon_event_count().
+ * @closid:	The closid used to identify the cached mbm_state.
+ * @rmid:	The rmid used to identify the cached mbm_state.
+ * @rr:		The struct rmid_read populated by __mon_event_count().
+ *
+ * Supporting function to calculate the memory bandwidth
+ * and delta bandwidth in MBps. The chunks value previously read by
+ * __mon_event_count() is compared with the chunks value from the previous
+ * invocation. This must be called once per second to maintain values in MBps.
+ */
+static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr)
+{
+	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+	struct mbm_state *m = &rr->d->mbm_local[idx];
+	u64 cur_bw, bytes, cur_bytes;
+
+	cur_bytes = rr->val;
+	bytes = cur_bytes - m->prev_bw_bytes;
+	m->prev_bw_bytes = cur_bytes;
+
+	cur_bw = bytes / SZ_1M;
+
+	m->prev_bw = cur_bw;
+}
+
+/*
+ * This is scheduled by mon_event_read() to read the CQM/MBM counters
+ * on a domain.
+ */
+void mon_event_count(void *info)
+{
+	struct rdtgroup *rdtgrp, *entry;
+	struct rmid_read *rr = info;
+	struct list_head *head;
+	int ret;
+
+	rdtgrp = rr->rgrp;
+
+	ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr);
+
+	/*
+	 * For Ctrl groups read data from child monitor groups and
+	 * add them together. Count events which are read successfully.
+	 * Discard the rmid_read's reporting errors.
+	 */
+	head = &rdtgrp->mon.crdtgrp_list;
+
+	if (rdtgrp->type == RDTCTRL_GROUP) {
+		list_for_each_entry(entry, head, mon.crdtgrp_list) {
+			if (__mon_event_count(entry->closid, entry->mon.rmid,
+					      rr) == 0)
+				ret = 0;
+		}
+	}
+
+	/*
+	 * __mon_event_count() calls for newly created monitor groups may
+	 * report -EINVAL/Unavailable if the monitor hasn't seen any traffic.
+	 * Discard error if any of the monitor event reads succeeded.
+	 */
+	if (ret == 0)
+		rr->err = 0;
+}
+
+/*
+ * Feedback loop for MBA software controller (mba_sc)
+ *
+ * mba_sc is a feedback loop where we periodically read MBM counters and
+ * adjust the bandwidth percentage values via the IA32_MBA_THRTL_MSRs so
+ * that:
+ *
+ *   current bandwidth(cur_bw) < user specified bandwidth(user_bw)
+ *
+ * This uses the MBM counters to measure the bandwidth and MBA throttle
+ * MSRs to control the bandwidth for a particular rdtgrp. It builds on the
+ * fact that resctrl rdtgroups have both monitoring and control.
+ *
+ * The frequency of the checks is 1s and we just tag along the MBM overflow
+ * timer. Having 1s interval makes the calculation of bandwidth simpler.
+ *
+ * Although MBA's goal is to restrict the bandwidth to a maximum, there may
+ * be a need to increase the bandwidth to avoid unnecessarily restricting
+ * the L2 <-> L3 traffic.
+ *
+ * Since MBA controls the L2 external bandwidth where as MBM measures the
+ * L3 external bandwidth the following sequence could lead to such a
+ * situation.
+ *
+ * Consider an rdtgroup which had high L3 <-> memory traffic in initial
+ * phases -> mba_sc kicks in and reduced bandwidth percentage values -> but
+ * after some time rdtgroup has mostly L2 <-> L3 traffic.
+ *
+ * In this case we may restrict the rdtgroup's L2 <-> L3 traffic as its
+ * throttle MSRs already have low percentage values.  To avoid
+ * unnecessarily restricting such rdtgroups, we also increase the bandwidth.
+ */
+static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
+{
+	u32 closid, rmid, cur_msr_val, new_msr_val;
+	struct mbm_state *pmbm_data, *cmbm_data;
+	struct rdt_resource *r_mba;
+	struct rdt_domain *dom_mba;
+	u32 cur_bw, user_bw, idx;
+	struct list_head *head;
+	struct rdtgroup *entry;
+
+	if (!resctrl_arch_is_mbm_local_enabled())
+		return;
+
+	r_mba = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
+
+	closid = rgrp->closid;
+	rmid = rgrp->mon.rmid;
+	idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+	pmbm_data = &dom_mbm->mbm_local[idx];
+
+	dom_mba = resctrl_get_domain_from_cpu(smp_processor_id(), r_mba);
+	if (!dom_mba) {
+		pr_warn_once("Failure to get domain for MBA update\n");
+		return;
+	}
+
+	cur_bw = pmbm_data->prev_bw;
+	user_bw = dom_mba->mbps_val[closid];
+
+	/* MBA resource doesn't support CDP */
+	cur_msr_val = resctrl_arch_get_config(r_mba, dom_mba, closid, CDP_NONE);
+
+	/*
+	 * For Ctrl groups read data from child monitor groups.
+	 */
+	head = &rgrp->mon.crdtgrp_list;
+	list_for_each_entry(entry, head, mon.crdtgrp_list) {
+		cmbm_data = &dom_mbm->mbm_local[entry->mon.rmid];
+		cur_bw += cmbm_data->prev_bw;
+	}
+
+	/*
+	 * Scale up/down the bandwidth linearly for the ctrl group.  The
+	 * bandwidth step is the bandwidth granularity specified by the
+	 * hardware.
+	 * Always increase throttling if current bandwidth is above the
+	 * target set by user.
+	 * But avoid thrashing up and down on every poll by checking
+	 * whether a decrease in throttling is likely to push the group
+	 * back over target. E.g. if currently throttling to 30% of bandwidth
+	 * on a system with 10% granularity steps, check whether moving to
+	 * 40% would go past the limit by multiplying current bandwidth by
+	 * "(30 + 10) / 30".
+	 */
+	if (cur_msr_val > r_mba->membw.min_bw && user_bw < cur_bw) {
+		new_msr_val = cur_msr_val - r_mba->membw.bw_gran;
+	} else if (cur_msr_val < MAX_MBA_BW &&
+		   (user_bw > (cur_bw * (cur_msr_val + r_mba->membw.min_bw) / cur_msr_val))) {
+		new_msr_val = cur_msr_val + r_mba->membw.bw_gran;
+	} else {
+		return;
+	}
+
+	resctrl_arch_update_one(r_mba, dom_mba, closid, CDP_NONE, new_msr_val);
+}
+
+static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,
+		       u32 closid, u32 rmid)
+{
+	struct rmid_read rr;
+
+	rr.first = false;
+	rr.r = r;
+	rr.d = d;
+
+	/*
+	 * This is protected from concurrent reads from user
+	 * as both the user and we hold the global mutex.
+	 */
+	if (resctrl_arch_is_mbm_total_enabled()) {
+		rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID;
+		rr.val = 0;
+		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
+		if (IS_ERR(rr.arch_mon_ctx)) {
+			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
+					    PTR_ERR(rr.arch_mon_ctx));
+			return;
+		}
+
+		__mon_event_count(closid, rmid, &rr);
+
+		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
+	}
+	if (resctrl_arch_is_mbm_local_enabled()) {
+		rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID;
+		rr.val = 0;
+		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
+		if (IS_ERR(rr.arch_mon_ctx)) {
+			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
+					    PTR_ERR(rr.arch_mon_ctx));
+			return;
+		}
+
+		__mon_event_count(closid, rmid, &rr);
+
+		/*
+		 * Call the MBA software controller only for the
+		 * control groups and when user has enabled
+		 * the software controller explicitly.
+		 */
+		if (is_mba_sc(NULL))
+			mbm_bw_count(closid, rmid, &rr);
+
+		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
+	}
+}
+
+/*
+ * Handler to scan the limbo list and move the RMIDs
+ * to free list whose occupancy < threshold_occupancy.
+ */
+void cqm_handle_limbo(struct work_struct *work)
+{
+	unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL);
+	struct rdt_domain *d;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	d = container_of(work, struct rdt_domain, cqm_limbo.work);
+
+	__check_limbo(d, false);
+
+	if (has_busy_rmid(d)) {
+		d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
+							   RESCTRL_PICK_ANY_CPU);
+		schedule_delayed_work_on(d->cqm_work_cpu, &d->cqm_limbo,
+					 delay);
+	}
+
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+}
+
+/**
+ * cqm_setup_limbo_handler() - Schedule the limbo handler to run for this
+ *                             domain.
+ * @dom:           The domain the limbo handler should run for.
+ * @delay_ms:      How far in the future the handler should run.
+ * @exclude_cpu:   Which CPU the handler should not run on,
+ *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
+ */
+void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
+			     int exclude_cpu)
+{
+	unsigned long delay = msecs_to_jiffies(delay_ms);
+	int cpu;
+
+	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
+	dom->cqm_work_cpu = cpu;
+
+	if (cpu < nr_cpu_ids)
+		schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay);
+}
+
+void mbm_handle_overflow(struct work_struct *work)
+{
+	unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL);
+	struct rdtgroup *prgrp, *crgrp;
+	struct list_head *head;
+	struct rdt_resource *r;
+	struct rdt_domain *d;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	/*
+	 * If the filesystem has been unmounted this work no longer needs to
+	 * run.
+	 */
+	if (!resctrl_mounted || !resctrl_arch_mon_capable())
+		goto out_unlock;
+
+	r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+	d = container_of(work, struct rdt_domain, mbm_over.work);
+
+	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
+		mbm_update(r, d, prgrp->closid, prgrp->mon.rmid);
+
+		head = &prgrp->mon.crdtgrp_list;
+		list_for_each_entry(crgrp, head, mon.crdtgrp_list)
+			mbm_update(r, d, crgrp->closid, crgrp->mon.rmid);
+
+		if (is_mba_sc(NULL))
+			update_mba_bw(prgrp, d);
+	}
+
+	/*
+	 * Re-check for housekeeping CPUs. This allows the overflow handler to
+	 * move off a nohz_full CPU quickly.
+	 */
+	d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
+						   RESCTRL_PICK_ANY_CPU);
+	schedule_delayed_work_on(d->mbm_work_cpu, &d->mbm_over, delay);
+
+out_unlock:
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+}
+
+/**
+ * mbm_setup_overflow_handler() - Schedule the overflow handler to run for this
+ *                                domain.
+ * @dom:           The domain the overflow handler should run for.
+ * @delay_ms:      How far in the future the handler should run.
+ * @exclude_cpu:   Which CPU the handler should not run on,
+ *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
+ */
+void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms,
+				int exclude_cpu)
+{
+	unsigned long delay = msecs_to_jiffies(delay_ms);
+	int cpu;
+
+	/*
+	 * When a domain comes online there is no guarantee the filesystem is
+	 * mounted. If not, there is no need to catch counter overflow.
+	 */
+	if (!resctrl_mounted || !resctrl_arch_mon_capable())
+		return;
+	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
+	dom->mbm_work_cpu = cpu;
+
+	if (cpu < nr_cpu_ids)
+		schedule_delayed_work_on(cpu, &dom->mbm_over, delay);
+}
+
+static int dom_data_init(struct rdt_resource *r)
+{
+	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
+	u32 num_closid = resctrl_arch_get_num_closid(r);
+	struct rmid_entry *entry = NULL;
+	int err = 0, i;
+	u32 idx;
+
+	mutex_lock(&rdtgroup_mutex);
+	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
+		u32 *tmp;
+
+		/*
+		 * If the architecture hasn't provided a sanitised value here,
+		 * this may result in larger arrays than necessary. Resctrl will
+		 * use a smaller system wide value based on the resources in
+		 * use.
+		 */
+		tmp = kcalloc(num_closid, sizeof(*tmp), GFP_KERNEL);
+		if (!tmp) {
+			err = -ENOMEM;
+			goto out_unlock;
+		}
+
+		closid_num_dirty_rmid = tmp;
+	}
+
+	rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL);
+	if (!rmid_ptrs) {
+		if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
+			kfree(closid_num_dirty_rmid);
+			closid_num_dirty_rmid = NULL;
+		}
+		err = -ENOMEM;
+		goto out_unlock;
+	}
+
+	for (i = 0; i < idx_limit; i++) {
+		entry = &rmid_ptrs[i];
+		INIT_LIST_HEAD(&entry->list);
+
+		resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid);
+		list_add_tail(&entry->list, &rmid_free_lru);
+	}
+
+	/*
+	 * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and
+	 * are always allocated. These are used for the rdtgroup_default
+	 * control group, which will be setup later in rdtgroup_init().
+	 */
+	idx = resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
+					   RESCTRL_RESERVED_RMID);
+	entry = __rmid_entry(idx);
+	list_del(&entry->list);
+
+out_unlock:
+	mutex_unlock(&rdtgroup_mutex);
+
+	return err;
+}
+
+static void dom_data_exit(struct rdt_resource *r)
+{
+	if (!r->mon_capable)
+		return;
+
+	mutex_lock(&rdtgroup_mutex);
+	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
+		kfree(closid_num_dirty_rmid);
+		closid_num_dirty_rmid = NULL;
+	}
+
+	kfree(rmid_ptrs);
+	rmid_ptrs = NULL;
+
+	mutex_unlock(&rdtgroup_mutex);
+}
+
+static struct mon_evt llc_occupancy_event = {
+	.name		= "llc_occupancy",
+	.evtid		= QOS_L3_OCCUP_EVENT_ID,
+};
+
+static struct mon_evt mbm_total_event = {
+	.name		= "mbm_total_bytes",
+	.evtid		= QOS_L3_MBM_TOTAL_EVENT_ID,
+};
+
+static struct mon_evt mbm_local_event = {
+	.name		= "mbm_local_bytes",
+	.evtid		= QOS_L3_MBM_LOCAL_EVENT_ID,
+};
+
+/*
+ * Initialize the event list for the resource.
+ *
+ * Note that MBM events are also part of RDT_RESOURCE_L3 resource
+ * because as per the SDM the total and local memory bandwidth
+ * are enumerated as part of L3 monitoring.
+ */
+static void l3_mon_evt_init(struct rdt_resource *r)
+{
+	INIT_LIST_HEAD(&r->evt_list);
+
+	if (resctrl_arch_is_llc_occupancy_enabled())
+		list_add_tail(&llc_occupancy_event.list, &r->evt_list);
+	if (resctrl_arch_is_mbm_total_enabled())
+		list_add_tail(&mbm_total_event.list, &r->evt_list);
+	if (resctrl_arch_is_mbm_local_enabled())
+		list_add_tail(&mbm_local_event.list, &r->evt_list);
+}
+
+int resctrl_mon_resource_init(void)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+	int ret;
+
+	if (!r->mon_capable)
+		return 0;
+
+	ret = dom_data_init(r);
+	if (ret)
+		return ret;
+
+	l3_mon_evt_init(r);
+
+	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_TOTAL_EVENT_ID)) {
+		mbm_total_event.configurable = true;
+		mbm_config_rftype_init("mbm_total_bytes_config");
+	}
+	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_LOCAL_EVENT_ID)) {
+		mbm_local_event.configurable = true;
+		mbm_config_rftype_init("mbm_local_bytes_config");
+	}
+
+	return 0;
+}
+
+void resctrl_mon_resource_exit(void)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+
+	dom_data_exit(r);
+}
diff --git a/fs/resctrl/psuedo_lock.c b/fs/resctrl/psuedo_lock.c
index e69de29bb2d1..077c2abb6edd 100644
--- a/fs/resctrl/psuedo_lock.c
+++ b/fs/resctrl/psuedo_lock.c
@@ -0,0 +1,1122 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Resource Director Technology (RDT)
+ *
+ * Pseudo-locking support built on top of Cache Allocation Technology (CAT)
+ *
+ * Copyright (C) 2018 Intel Corporation
+ *
+ * Author: Reinette Chatre <reinette.chatre@intel.com>
+ */
+
+#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
+
+#include <linux/cacheinfo.h>
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/debugfs.h>
+#include <linux/kthread.h>
+#include <linux/mman.h>
+#include <linux/perf_event.h>
+#include <linux/pm_qos.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include <asm/cacheflush.h>
+#include <asm/resctrl.h>
+#include <asm/perf_event.h>
+
+#include "internal.h"
+
+/*
+ * Major number assigned to and shared by all devices exposing
+ * pseudo-locked regions.
+ */
+static unsigned int pseudo_lock_major;
+static unsigned long pseudo_lock_minor_avail = GENMASK(MINORBITS, 0);
+
+static char *pseudo_lock_devnode(const struct device *dev, umode_t *mode)
+{
+	const struct rdtgroup *rdtgrp;
+
+	rdtgrp = dev_get_drvdata(dev);
+	if (mode)
+		*mode = 0600;
+	return kasprintf(GFP_KERNEL, "pseudo_lock/%s", rdtgrp->kn->name);
+}
+
+static const struct class pseudo_lock_class = {
+	.name = "pseudo_lock",
+	.devnode = pseudo_lock_devnode,
+};
+
+/**
+ * pseudo_lock_minor_get - Obtain available minor number
+ * @minor: Pointer to where new minor number will be stored
+ *
+ * A bitmask is used to track available minor numbers. Here the next free
+ * minor number is marked as unavailable and returned.
+ *
+ * Return: 0 on success, <0 on failure.
+ */
+static int pseudo_lock_minor_get(unsigned int *minor)
+{
+	unsigned long first_bit;
+
+	first_bit = find_first_bit(&pseudo_lock_minor_avail, MINORBITS);
+
+	if (first_bit == MINORBITS)
+		return -ENOSPC;
+
+	__clear_bit(first_bit, &pseudo_lock_minor_avail);
+	*minor = first_bit;
+
+	return 0;
+}
+
+/**
+ * pseudo_lock_minor_release - Return minor number to available
+ * @minor: The minor number made available
+ */
+static void pseudo_lock_minor_release(unsigned int minor)
+{
+	__set_bit(minor, &pseudo_lock_minor_avail);
+}
+
+/**
+ * region_find_by_minor - Locate a pseudo-lock region by inode minor number
+ * @minor: The minor number of the device representing pseudo-locked region
+ *
+ * When the character device is accessed we need to determine which
+ * pseudo-locked region it belongs to. This is done by matching the minor
+ * number of the device to the pseudo-locked region it belongs.
+ *
+ * Minor numbers are assigned at the time a pseudo-locked region is associated
+ * with a cache instance.
+ *
+ * Return: On success return pointer to resource group owning the pseudo-locked
+ *         region, NULL on failure.
+ */
+static struct rdtgroup *region_find_by_minor(unsigned int minor)
+{
+	struct rdtgroup *rdtgrp, *rdtgrp_match = NULL;
+
+	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
+		if (rdtgrp->plr && rdtgrp->plr->minor == minor) {
+			rdtgrp_match = rdtgrp;
+			break;
+		}
+	}
+	return rdtgrp_match;
+}
+
+/**
+ * struct pseudo_lock_pm_req - A power management QoS request list entry
+ * @list:	Entry within the @pm_reqs list for a pseudo-locked region
+ * @req:	PM QoS request
+ */
+struct pseudo_lock_pm_req {
+	struct list_head list;
+	struct dev_pm_qos_request req;
+};
+
+static void pseudo_lock_cstates_relax(struct pseudo_lock_region *plr)
+{
+	struct pseudo_lock_pm_req *pm_req, *next;
+
+	list_for_each_entry_safe(pm_req, next, &plr->pm_reqs, list) {
+		dev_pm_qos_remove_request(&pm_req->req);
+		list_del(&pm_req->list);
+		kfree(pm_req);
+	}
+}
+
+/**
+ * pseudo_lock_cstates_constrain - Restrict cores from entering C6
+ * @plr: Pseudo-locked region
+ *
+ * To prevent the cache from being affected by power management entering
+ * C6 has to be avoided. This is accomplished by requesting a latency
+ * requirement lower than lowest C6 exit latency of all supported
+ * platforms as found in the cpuidle state tables in the intel_idle driver.
+ * At this time it is possible to do so with a single latency requirement
+ * for all supported platforms.
+ *
+ * Since Goldmont is supported, which is affected by X86_BUG_MONITOR,
+ * the ACPI latencies need to be considered while keeping in mind that C2
+ * may be set to map to deeper sleep states. In this case the latency
+ * requirement needs to prevent entering C2 also.
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static int pseudo_lock_cstates_constrain(struct pseudo_lock_region *plr)
+{
+	struct pseudo_lock_pm_req *pm_req;
+	int cpu;
+	int ret;
+
+	for_each_cpu(cpu, &plr->d->cpu_mask) {
+		pm_req = kzalloc(sizeof(*pm_req), GFP_KERNEL);
+		if (!pm_req) {
+			rdt_last_cmd_puts("Failure to allocate memory for PM QoS\n");
+			ret = -ENOMEM;
+			goto out_err;
+		}
+		ret = dev_pm_qos_add_request(get_cpu_device(cpu),
+					     &pm_req->req,
+					     DEV_PM_QOS_RESUME_LATENCY,
+					     30);
+		if (ret < 0) {
+			rdt_last_cmd_printf("Failed to add latency req CPU%d\n",
+					    cpu);
+			kfree(pm_req);
+			ret = -1;
+			goto out_err;
+		}
+		list_add(&pm_req->list, &plr->pm_reqs);
+	}
+
+	return 0;
+
+out_err:
+	pseudo_lock_cstates_relax(plr);
+	return ret;
+}
+
+/**
+ * pseudo_lock_region_clear - Reset pseudo-lock region data
+ * @plr: pseudo-lock region
+ *
+ * All content of the pseudo-locked region is reset - any memory allocated
+ * freed.
+ *
+ * Return: void
+ */
+static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)
+{
+	plr->size = 0;
+	plr->line_size = 0;
+	kfree(plr->kmem);
+	plr->kmem = NULL;
+	plr->s = NULL;
+	if (plr->d)
+		plr->d->plr = NULL;
+	plr->d = NULL;
+	plr->cbm = 0;
+	plr->debugfs_dir = NULL;
+}
+
+/**
+ * pseudo_lock_region_init - Initialize pseudo-lock region information
+ * @plr: pseudo-lock region
+ *
+ * Called after user provided a schemata to be pseudo-locked. From the
+ * schemata the &struct pseudo_lock_region is on entry already initialized
+ * with the resource, domain, and capacity bitmask. Here the information
+ * required for pseudo-locking is deduced from this data and &struct
+ * pseudo_lock_region initialized further. This information includes:
+ * - size in bytes of the region to be pseudo-locked
+ * - cache line size to know the stride with which data needs to be accessed
+ *   to be pseudo-locked
+ * - a cpu associated with the cache instance on which the pseudo-locking
+ *   flow can be executed
+ *
+ * Return: 0 on success, <0 on failure. Descriptive error will be written
+ * to last_cmd_status buffer.
+ */
+static int pseudo_lock_region_init(struct pseudo_lock_region *plr)
+{
+	struct cpu_cacheinfo *ci;
+	int ret;
+	int i;
+
+	/* Pick the first cpu we find that is associated with the cache. */
+	plr->cpu = cpumask_first(&plr->d->cpu_mask);
+
+	if (!cpu_online(plr->cpu)) {
+		rdt_last_cmd_printf("CPU %u associated with cache not online\n",
+				    plr->cpu);
+		ret = -ENODEV;
+		goto out_region;
+	}
+
+	ci = get_cpu_cacheinfo(plr->cpu);
+
+	plr->size = rdtgroup_cbm_to_size(plr->s->res, plr->d, plr->cbm);
+
+	for (i = 0; i < ci->num_leaves; i++) {
+		if (ci->info_list[i].level == plr->s->res->cache_level) {
+			plr->line_size = ci->info_list[i].coherency_line_size;
+			return 0;
+		}
+	}
+
+	ret = -1;
+	rdt_last_cmd_puts("Unable to determine cache line size\n");
+out_region:
+	pseudo_lock_region_clear(plr);
+	return ret;
+}
+
+/**
+ * pseudo_lock_init - Initialize a pseudo-lock region
+ * @rdtgrp: resource group to which new pseudo-locked region will belong
+ *
+ * A pseudo-locked region is associated with a resource group. When this
+ * association is created the pseudo-locked region is initialized. The
+ * details of the pseudo-locked region are not known at this time so only
+ * allocation is done and association established.
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static int pseudo_lock_init(struct rdtgroup *rdtgrp)
+{
+	struct pseudo_lock_region *plr;
+
+	plr = kzalloc(sizeof(*plr), GFP_KERNEL);
+	if (!plr)
+		return -ENOMEM;
+
+	init_waitqueue_head(&plr->lock_thread_wq);
+	INIT_LIST_HEAD(&plr->pm_reqs);
+	rdtgrp->plr = plr;
+	return 0;
+}
+
+/**
+ * pseudo_lock_region_alloc - Allocate kernel memory that will be pseudo-locked
+ * @plr: pseudo-lock region
+ *
+ * Initialize the details required to set up the pseudo-locked region and
+ * allocate the contiguous memory that will be pseudo-locked to the cache.
+ *
+ * Return: 0 on success, <0 on failure.  Descriptive error will be written
+ * to last_cmd_status buffer.
+ */
+static int pseudo_lock_region_alloc(struct pseudo_lock_region *plr)
+{
+	int ret;
+
+	ret = pseudo_lock_region_init(plr);
+	if (ret < 0)
+		return ret;
+
+	/*
+	 * We do not yet support contiguous regions larger than
+	 * KMALLOC_MAX_SIZE.
+	 */
+	if (plr->size > KMALLOC_MAX_SIZE) {
+		rdt_last_cmd_puts("Requested region exceeds maximum size\n");
+		ret = -E2BIG;
+		goto out_region;
+	}
+
+	plr->kmem = kzalloc(plr->size, GFP_KERNEL);
+	if (!plr->kmem) {
+		rdt_last_cmd_puts("Unable to allocate memory\n");
+		ret = -ENOMEM;
+		goto out_region;
+	}
+
+	ret = 0;
+	goto out;
+out_region:
+	pseudo_lock_region_clear(plr);
+out:
+	return ret;
+}
+
+/**
+ * pseudo_lock_free - Free a pseudo-locked region
+ * @rdtgrp: resource group to which pseudo-locked region belonged
+ *
+ * The pseudo-locked region's resources have already been released, or not
+ * yet created at this point. Now it can be freed and disassociated from the
+ * resource group.
+ *
+ * Return: void
+ */
+static void pseudo_lock_free(struct rdtgroup *rdtgrp)
+{
+	pseudo_lock_region_clear(rdtgrp->plr);
+	kfree(rdtgrp->plr);
+	rdtgrp->plr = NULL;
+}
+
+/**
+ * rdtgroup_monitor_in_progress - Test if monitoring in progress
+ * @rdtgrp: resource group being queried
+ *
+ * Return: 1 if monitor groups have been created for this resource
+ * group, 0 otherwise.
+ */
+static int rdtgroup_monitor_in_progress(struct rdtgroup *rdtgrp)
+{
+	return !list_empty(&rdtgrp->mon.crdtgrp_list);
+}
+
+/**
+ * rdtgroup_locksetup_user_restrict - Restrict user access to group
+ * @rdtgrp: resource group needing access restricted
+ *
+ * A resource group used for cache pseudo-locking cannot have cpus or tasks
+ * assigned to it. This is communicated to the user by restricting access
+ * to all the files that can be used to make such changes.
+ *
+ * Permissions restored with rdtgroup_locksetup_user_restore()
+ *
+ * Return: 0 on success, <0 on failure. If a failure occurs during the
+ * restriction of access an attempt will be made to restore permissions but
+ * the state of the mode of these files will be uncertain when a failure
+ * occurs.
+ */
+static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)
+{
+	int ret;
+
+	ret = rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
+	if (ret)
+		return ret;
+
+	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
+	if (ret)
+		goto err_tasks;
+
+	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
+	if (ret)
+		goto err_cpus;
+
+	if (resctrl_arch_mon_capable()) {
+		ret = rdtgroup_kn_mode_restrict(rdtgrp, "mon_groups");
+		if (ret)
+			goto err_cpus_list;
+	}
+
+	ret = 0;
+	goto out;
+
+err_cpus_list:
+	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
+err_cpus:
+	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
+err_tasks:
+	rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
+out:
+	return ret;
+}
+
+/**
+ * rdtgroup_locksetup_user_restore - Restore user access to group
+ * @rdtgrp: resource group needing access restored
+ *
+ * Restore all file access previously removed using
+ * rdtgroup_locksetup_user_restrict()
+ *
+ * Return: 0 on success, <0 on failure.  If a failure occurs during the
+ * restoration of access an attempt will be made to restrict permissions
+ * again but the state of the mode of these files will be uncertain when
+ * a failure occurs.
+ */
+static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)
+{
+	int ret;
+
+	ret = rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
+	if (ret)
+		return ret;
+
+	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
+	if (ret)
+		goto err_tasks;
+
+	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
+	if (ret)
+		goto err_cpus;
+
+	if (resctrl_arch_mon_capable()) {
+		ret = rdtgroup_kn_mode_restore(rdtgrp, "mon_groups", 0777);
+		if (ret)
+			goto err_cpus_list;
+	}
+
+	ret = 0;
+	goto out;
+
+err_cpus_list:
+	rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
+err_cpus:
+	rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
+err_tasks:
+	rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
+out:
+	return ret;
+}
+
+/**
+ * rdtgroup_locksetup_enter - Resource group enters locksetup mode
+ * @rdtgrp: resource group requested to enter locksetup mode
+ *
+ * A resource group enters locksetup mode to reflect that it would be used
+ * to represent a pseudo-locked region and is in the process of being set
+ * up to do so. A resource group used for a pseudo-locked region would
+ * lose the closid associated with it so we cannot allow it to have any
+ * tasks or cpus assigned nor permit tasks or cpus to be assigned in the
+ * future. Monitoring of a pseudo-locked region is not allowed either.
+ *
+ * The above and more restrictions on a pseudo-locked region are checked
+ * for and enforced before the resource group enters the locksetup mode.
+ *
+ * Returns: 0 if the resource group successfully entered locksetup mode, <0
+ * on failure. On failure the last_cmd_status buffer is updated with text to
+ * communicate details of failure to the user.
+ */
+int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
+{
+	int ret;
+
+	/*
+	 * The default resource group can neither be removed nor lose the
+	 * default closid associated with it.
+	 */
+	if (rdtgrp == &rdtgroup_default) {
+		rdt_last_cmd_puts("Cannot pseudo-lock default group\n");
+		return -EINVAL;
+	}
+
+	/*
+	 * Cache Pseudo-locking not supported when CDP is enabled.
+	 *
+	 * Some things to consider if you would like to enable this
+	 * support (using L3 CDP as example):
+	 * - When CDP is enabled two separate resources are exposed,
+	 *   L3DATA and L3CODE, but they are actually on the same cache.
+	 *   The implication for pseudo-locking is that if a
+	 *   pseudo-locked region is created on a domain of one
+	 *   resource (eg. L3CODE), then a pseudo-locked region cannot
+	 *   be created on that same domain of the other resource
+	 *   (eg. L3DATA). This is because the creation of a
+	 *   pseudo-locked region involves a call to wbinvd that will
+	 *   affect all cache allocations on particular domain.
+	 * - Considering the previous, it may be possible to only
+	 *   expose one of the CDP resources to pseudo-locking and
+	 *   hide the other. For example, we could consider to only
+	 *   expose L3DATA and since the L3 cache is unified it is
+	 *   still possible to place instructions there are execute it.
+	 * - If only one region is exposed to pseudo-locking we should
+	 *   still keep in mind that availability of a portion of cache
+	 *   for pseudo-locking should take into account both resources.
+	 *   Similarly, if a pseudo-locked region is created in one
+	 *   resource, the portion of cache used by it should be made
+	 *   unavailable to all future allocations from both resources.
+	 */
+	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3) ||
+	    resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2)) {
+		rdt_last_cmd_puts("CDP enabled\n");
+		return -EINVAL;
+	}
+
+	/*
+	 * Not knowing the bits to disable prefetching implies that this
+	 * platform does not support Cache Pseudo-Locking.
+	 */
+	if (resctrl_arch_get_prefetch_disable_bits() == 0) {
+		rdt_last_cmd_puts("Pseudo-locking not supported\n");
+		return -EINVAL;
+	}
+
+	if (rdtgroup_monitor_in_progress(rdtgrp)) {
+		rdt_last_cmd_puts("Monitoring in progress\n");
+		return -EINVAL;
+	}
+
+	if (rdtgroup_tasks_assigned(rdtgrp)) {
+		rdt_last_cmd_puts("Tasks assigned to resource group\n");
+		return -EINVAL;
+	}
+
+	if (!cpumask_empty(&rdtgrp->cpu_mask)) {
+		rdt_last_cmd_puts("CPUs assigned to resource group\n");
+		return -EINVAL;
+	}
+
+	if (rdtgroup_locksetup_user_restrict(rdtgrp)) {
+		rdt_last_cmd_puts("Unable to modify resctrl permissions\n");
+		return -EIO;
+	}
+
+	ret = pseudo_lock_init(rdtgrp);
+	if (ret) {
+		rdt_last_cmd_puts("Unable to init pseudo-lock region\n");
+		goto out_release;
+	}
+
+	/*
+	 * If this system is capable of monitoring a rmid would have been
+	 * allocated when the control group was created. This is not needed
+	 * anymore when this group would be used for pseudo-locking. This
+	 * is safe to call on platforms not capable of monitoring.
+	 */
+	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
+
+	ret = 0;
+	goto out;
+
+out_release:
+	rdtgroup_locksetup_user_restore(rdtgrp);
+out:
+	return ret;
+}
+
+/**
+ * rdtgroup_locksetup_exit - resource group exist locksetup mode
+ * @rdtgrp: resource group
+ *
+ * When a resource group exits locksetup mode the earlier restrictions are
+ * lifted.
+ *
+ * Return: 0 on success, <0 on failure
+ */
+int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
+{
+	int ret;
+
+	if (resctrl_arch_mon_capable()) {
+		ret = alloc_rmid(rdtgrp->closid);
+		if (ret < 0) {
+			rdt_last_cmd_puts("Out of RMIDs\n");
+			return ret;
+		}
+		rdtgrp->mon.rmid = ret;
+	}
+
+	ret = rdtgroup_locksetup_user_restore(rdtgrp);
+	if (ret) {
+		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
+		return ret;
+	}
+
+	pseudo_lock_free(rdtgrp);
+	return 0;
+}
+
+/**
+ * rdtgroup_cbm_overlaps_pseudo_locked - Test if CBM or portion is pseudo-locked
+ * @d: RDT domain
+ * @cbm: CBM to test
+ *
+ * @d represents a cache instance and @cbm a capacity bitmask that is
+ * considered for it. Determine if @cbm overlaps with any existing
+ * pseudo-locked region on @d.
+ *
+ * @cbm is unsigned long, even if only 32 bits are used, to make the
+ * bitmap functions work correctly.
+ *
+ * Return: true if @cbm overlaps with pseudo-locked region on @d, false
+ * otherwise.
+ */
+bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
+{
+	unsigned int cbm_len;
+	unsigned long cbm_b;
+
+	if (d->plr) {
+		cbm_len = d->plr->s->res->cache.cbm_len;
+		cbm_b = d->plr->cbm;
+		if (bitmap_intersects(&cbm, &cbm_b, cbm_len))
+			return true;
+	}
+	return false;
+}
+
+/**
+ * rdtgroup_pseudo_locked_in_hierarchy - Pseudo-locked region in cache hierarchy
+ * @d: RDT domain under test
+ *
+ * The setup of a pseudo-locked region affects all cache instances within
+ * the hierarchy of the region. It is thus essential to know if any
+ * pseudo-locked regions exist within a cache hierarchy to prevent any
+ * attempts to create new pseudo-locked regions in the same hierarchy.
+ *
+ * Return: true if a pseudo-locked region exists in the hierarchy of @d or
+ *         if it is not possible to test due to memory allocation issue,
+ *         false otherwise.
+ */
+bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
+{
+	cpumask_var_t cpu_with_psl;
+	enum resctrl_res_level i;
+	struct rdt_resource *r;
+	struct rdt_domain *d_i;
+	bool ret = false;
+
+	/* Walking r->domains, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+	if (!zalloc_cpumask_var(&cpu_with_psl, GFP_KERNEL))
+		return true;
+
+	/*
+	 * First determine which cpus have pseudo-locked regions
+	 * associated with them.
+	 */
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->alloc_capable)
+			continue;
+
+		list_for_each_entry(d_i, &r->domains, list) {
+			if (d_i->plr)
+				cpumask_or(cpu_with_psl, cpu_with_psl,
+					   &d_i->cpu_mask);
+		}
+	}
+
+	/*
+	 * Next test if new pseudo-locked region would intersect with
+	 * existing region.
+	 */
+	if (cpumask_intersects(&d->cpu_mask, cpu_with_psl))
+		ret = true;
+
+	free_cpumask_var(cpu_with_psl);
+	return ret;
+}
+
+/**
+ * pseudo_lock_measure_cycles - Trigger latency measure to pseudo-locked region
+ * @rdtgrp: Resource group to which the pseudo-locked region belongs.
+ * @sel: Selector of which measurement to perform on a pseudo-locked region.
+ *
+ * The measurement of latency to access a pseudo-locked region should be
+ * done from a cpu that is associated with that pseudo-locked region.
+ * Determine which cpu is associated with this region and start a thread on
+ * that cpu to perform the measurement, wait for that thread to complete.
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel)
+{
+	struct pseudo_lock_region *plr = rdtgrp->plr;
+	struct task_struct *thread;
+	unsigned int cpu;
+	int ret = -1;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	if (rdtgrp->flags & RDT_DELETED) {
+		ret = -ENODEV;
+		goto out;
+	}
+
+	if (!plr->d) {
+		ret = -ENODEV;
+		goto out;
+	}
+
+	plr->thread_done = 0;
+	cpu = cpumask_first(&plr->d->cpu_mask);
+	if (!cpu_online(cpu)) {
+		ret = -ENODEV;
+		goto out;
+	}
+
+	plr->cpu = cpu;
+
+	if (sel == 1)
+		thread = kthread_create_on_node(resctrl_arch_measure_cycles_lat_fn,
+						plr, cpu_to_node(cpu),
+						"pseudo_lock_measure/%u",
+						cpu);
+	else if (sel == 2)
+		thread = kthread_create_on_node(resctrl_arch_measure_l2_residency,
+						plr, cpu_to_node(cpu),
+						"pseudo_lock_measure/%u",
+						cpu);
+	else if (sel == 3)
+		thread = kthread_create_on_node(resctrl_arch_measure_l3_residency,
+						plr, cpu_to_node(cpu),
+						"pseudo_lock_measure/%u",
+						cpu);
+	else
+		goto out;
+
+	if (IS_ERR(thread)) {
+		ret = PTR_ERR(thread);
+		goto out;
+	}
+	kthread_bind(thread, cpu);
+	wake_up_process(thread);
+
+	ret = wait_event_interruptible(plr->lock_thread_wq,
+				       plr->thread_done == 1);
+	if (ret < 0)
+		goto out;
+
+	ret = 0;
+
+out:
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+	return ret;
+}
+
+static ssize_t pseudo_lock_measure_trigger(struct file *file,
+					   const char __user *user_buf,
+					   size_t count, loff_t *ppos)
+{
+	struct rdtgroup *rdtgrp = file->private_data;
+	size_t buf_size;
+	char buf[32];
+	int ret;
+	int sel;
+
+	buf_size = min(count, (sizeof(buf) - 1));
+	if (copy_from_user(buf, user_buf, buf_size))
+		return -EFAULT;
+
+	buf[buf_size] = '\0';
+	ret = kstrtoint(buf, 10, &sel);
+	if (ret == 0) {
+		if (sel != 1 && sel != 2 && sel != 3)
+			return -EINVAL;
+		ret = debugfs_file_get(file->f_path.dentry);
+		if (ret)
+			return ret;
+		ret = pseudo_lock_measure_cycles(rdtgrp, sel);
+		if (ret == 0)
+			ret = count;
+		debugfs_file_put(file->f_path.dentry);
+	}
+
+	return ret;
+}
+
+static const struct file_operations pseudo_measure_fops = {
+	.write = pseudo_lock_measure_trigger,
+	.open = simple_open,
+	.llseek = default_llseek,
+};
+
+/**
+ * rdtgroup_pseudo_lock_create - Create a pseudo-locked region
+ * @rdtgrp: resource group to which pseudo-lock region belongs
+ *
+ * Called when a resource group in the pseudo-locksetup mode receives a
+ * valid schemata that should be pseudo-locked. Since the resource group is
+ * in pseudo-locksetup mode the &struct pseudo_lock_region has already been
+ * allocated and initialized with the essential information. If a failure
+ * occurs the resource group remains in the pseudo-locksetup mode with the
+ * &struct pseudo_lock_region associated with it, but cleared from all
+ * information and ready for the user to re-attempt pseudo-locking by
+ * writing the schemata again.
+ *
+ * Return: 0 if the pseudo-locked region was successfully pseudo-locked, <0
+ * on failure. Descriptive error will be written to last_cmd_status buffer.
+ */
+int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
+{
+	struct pseudo_lock_region *plr = rdtgrp->plr;
+	struct task_struct *thread;
+	unsigned int new_minor;
+	struct device *dev;
+	int ret;
+
+	ret = pseudo_lock_region_alloc(plr);
+	if (ret < 0)
+		return ret;
+
+	ret = pseudo_lock_cstates_constrain(plr);
+	if (ret < 0) {
+		ret = -EINVAL;
+		goto out_region;
+	}
+
+	plr->thread_done = 0;
+
+	plr->closid = rdtgrp->closid;
+	thread = kthread_create_on_node(resctrl_arch_pseudo_lock_fn, plr,
+					cpu_to_node(plr->cpu),
+					"pseudo_lock/%u", plr->cpu);
+	if (IS_ERR(thread)) {
+		ret = PTR_ERR(thread);
+		rdt_last_cmd_printf("Locking thread returned error %d\n", ret);
+		goto out_cstates;
+	}
+
+	kthread_bind(thread, plr->cpu);
+	wake_up_process(thread);
+
+	ret = wait_event_interruptible(plr->lock_thread_wq,
+				       plr->thread_done == 1);
+	if (ret < 0) {
+		/*
+		 * If the thread does not get on the CPU for whatever
+		 * reason and the process which sets up the region is
+		 * interrupted then this will leave the thread in runnable
+		 * state and once it gets on the CPU it will dereference
+		 * the cleared, but not freed, plr struct resulting in an
+		 * empty pseudo-locking loop.
+		 */
+		rdt_last_cmd_puts("Locking thread interrupted\n");
+		goto out_cstates;
+	}
+
+	ret = pseudo_lock_minor_get(&new_minor);
+	if (ret < 0) {
+		rdt_last_cmd_puts("Unable to obtain a new minor number\n");
+		goto out_cstates;
+	}
+
+	/*
+	 * Unlock access but do not release the reference. The
+	 * pseudo-locked region will still be here on return.
+	 *
+	 * The mutex has to be released temporarily to avoid a potential
+	 * deadlock with the mm->mmap_lock which is obtained in the
+	 * device_create() and debugfs_create_dir() callpath below as well as
+	 * before the mmap() callback is called.
+	 */
+	mutex_unlock(&rdtgroup_mutex);
+
+	if (!IS_ERR_OR_NULL(debugfs_resctrl)) {
+		plr->debugfs_dir = debugfs_create_dir(rdtgrp->kn->name,
+						      debugfs_resctrl);
+		if (!IS_ERR_OR_NULL(plr->debugfs_dir))
+			debugfs_create_file("pseudo_lock_measure", 0200,
+					    plr->debugfs_dir, rdtgrp,
+					    &pseudo_measure_fops);
+	}
+
+	dev = device_create(&pseudo_lock_class, NULL,
+			    MKDEV(pseudo_lock_major, new_minor),
+			    rdtgrp, "%s", rdtgrp->kn->name);
+
+	mutex_lock(&rdtgroup_mutex);
+
+	if (IS_ERR(dev)) {
+		ret = PTR_ERR(dev);
+		rdt_last_cmd_printf("Failed to create character device: %d\n",
+				    ret);
+		goto out_debugfs;
+	}
+
+	/* We released the mutex - check if group was removed while we did so */
+	if (rdtgrp->flags & RDT_DELETED) {
+		ret = -ENODEV;
+		goto out_device;
+	}
+
+	plr->minor = new_minor;
+
+	rdtgrp->mode = RDT_MODE_PSEUDO_LOCKED;
+	closid_free(rdtgrp->closid);
+	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0444);
+	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0444);
+
+	ret = 0;
+	goto out;
+
+out_device:
+	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, new_minor));
+out_debugfs:
+	debugfs_remove_recursive(plr->debugfs_dir);
+	pseudo_lock_minor_release(new_minor);
+out_cstates:
+	pseudo_lock_cstates_relax(plr);
+out_region:
+	pseudo_lock_region_clear(plr);
+out:
+	return ret;
+}
+
+/**
+ * rdtgroup_pseudo_lock_remove - Remove a pseudo-locked region
+ * @rdtgrp: resource group to which the pseudo-locked region belongs
+ *
+ * The removal of a pseudo-locked region can be initiated when the resource
+ * group is removed from user space via a "rmdir" from userspace or the
+ * unmount of the resctrl filesystem. On removal the resource group does
+ * not go back to pseudo-locksetup mode before it is removed, instead it is
+ * removed directly. There is thus asymmetry with the creation where the
+ * &struct pseudo_lock_region is removed here while it was not created in
+ * rdtgroup_pseudo_lock_create().
+ *
+ * Return: void
+ */
+void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp)
+{
+	struct pseudo_lock_region *plr = rdtgrp->plr;
+
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+		/*
+		 * Default group cannot be a pseudo-locked region so we can
+		 * free closid here.
+		 */
+		closid_free(rdtgrp->closid);
+		goto free;
+	}
+
+	pseudo_lock_cstates_relax(plr);
+	debugfs_remove_recursive(rdtgrp->plr->debugfs_dir);
+	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, plr->minor));
+	pseudo_lock_minor_release(plr->minor);
+
+free:
+	pseudo_lock_free(rdtgrp);
+}
+
+static int pseudo_lock_dev_open(struct inode *inode, struct file *filp)
+{
+	struct rdtgroup *rdtgrp;
+
+	mutex_lock(&rdtgroup_mutex);
+
+	rdtgrp = region_find_by_minor(iminor(inode));
+	if (!rdtgrp) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -ENODEV;
+	}
+
+	filp->private_data = rdtgrp;
+	atomic_inc(&rdtgrp->waitcount);
+	/* Perform a non-seekable open - llseek is not supported */
+	filp->f_mode &= ~(FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE);
+
+	mutex_unlock(&rdtgroup_mutex);
+
+	return 0;
+}
+
+static int pseudo_lock_dev_release(struct inode *inode, struct file *filp)
+{
+	struct rdtgroup *rdtgrp;
+
+	mutex_lock(&rdtgroup_mutex);
+	rdtgrp = filp->private_data;
+	WARN_ON(!rdtgrp);
+	if (!rdtgrp) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -ENODEV;
+	}
+	filp->private_data = NULL;
+	atomic_dec(&rdtgrp->waitcount);
+	mutex_unlock(&rdtgroup_mutex);
+	return 0;
+}
+
+static int pseudo_lock_dev_mremap(struct vm_area_struct *area)
+{
+	/* Not supported */
+	return -EINVAL;
+}
+
+static const struct vm_operations_struct pseudo_mmap_ops = {
+	.mremap = pseudo_lock_dev_mremap,
+};
+
+static int pseudo_lock_dev_mmap(struct file *filp, struct vm_area_struct *vma)
+{
+	unsigned long vsize = vma->vm_end - vma->vm_start;
+	unsigned long off = vma->vm_pgoff << PAGE_SHIFT;
+	struct pseudo_lock_region *plr;
+	struct rdtgroup *rdtgrp;
+	unsigned long physical;
+	unsigned long psize;
+
+	mutex_lock(&rdtgroup_mutex);
+
+	rdtgrp = filp->private_data;
+	WARN_ON(!rdtgrp);
+	if (!rdtgrp) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -ENODEV;
+	}
+
+	plr = rdtgrp->plr;
+
+	if (!plr->d) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -ENODEV;
+	}
+
+	/*
+	 * Task is required to run with affinity to the cpus associated
+	 * with the pseudo-locked region. If this is not the case the task
+	 * may be scheduled elsewhere and invalidate entries in the
+	 * pseudo-locked region.
+	 */
+	if (!cpumask_subset(current->cpus_ptr, &plr->d->cpu_mask)) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -EINVAL;
+	}
+
+	physical = __pa(plr->kmem) >> PAGE_SHIFT;
+	psize = plr->size - off;
+
+	if (off > plr->size) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -ENOSPC;
+	}
+
+	/*
+	 * Ensure changes are carried directly to the memory being mapped,
+	 * do not allow copy-on-write mapping.
+	 */
+	if (!(vma->vm_flags & VM_SHARED)) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -EINVAL;
+	}
+
+	if (vsize > psize) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -ENOSPC;
+	}
+
+	memset(plr->kmem + off, 0, vsize);
+
+	if (remap_pfn_range(vma, vma->vm_start, physical + vma->vm_pgoff,
+			    vsize, vma->vm_page_prot)) {
+		mutex_unlock(&rdtgroup_mutex);
+		return -EAGAIN;
+	}
+	vma->vm_ops = &pseudo_mmap_ops;
+	mutex_unlock(&rdtgroup_mutex);
+	return 0;
+}
+
+static const struct file_operations pseudo_lock_dev_fops = {
+	.owner =	THIS_MODULE,
+	.llseek =	no_llseek,
+	.read =		NULL,
+	.write =	NULL,
+	.open =		pseudo_lock_dev_open,
+	.release =	pseudo_lock_dev_release,
+	.mmap =		pseudo_lock_dev_mmap,
+};
+
+int rdt_pseudo_lock_init(void)
+{
+	int ret;
+
+	ret = register_chrdev(0, "pseudo_lock", &pseudo_lock_dev_fops);
+	if (ret < 0)
+		return ret;
+
+	pseudo_lock_major = ret;
+
+	ret = class_register(&pseudo_lock_class);
+	if (ret) {
+		unregister_chrdev(pseudo_lock_major, "pseudo_lock");
+		return ret;
+	}
+
+	return 0;
+}
+
+void rdt_pseudo_lock_release(void)
+{
+	class_unregister(&pseudo_lock_class);
+	unregister_chrdev(pseudo_lock_major, "pseudo_lock");
+	pseudo_lock_major = 0;
+}
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index e69de29bb2d1..936fc6e47386 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -0,0 +1,4013 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * User interface for Resource Allocation in Resource Director Technology(RDT)
+ *
+ * Copyright (C) 2016 Intel Corporation
+ *
+ * Author: Fenghua Yu <fenghua.yu@intel.com>
+ *
+ * More information about RDT be found in the Intel (R) x86 Architecture
+ * Software Developer Manual.
+ */
+
+#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
+
+#include <linux/cacheinfo.h>
+#include <linux/cpu.h>
+#include <linux/debugfs.h>
+#include <linux/fs.h>
+#include <linux/fs_parser.h>
+#include <linux/sysfs.h>
+#include <linux/kernfs.h>
+#include <linux/seq_buf.h>
+#include <linux/seq_file.h>
+#include <linux/sched/signal.h>
+#include <linux/sched/task.h>
+#include <linux/slab.h>
+#include <linux/task_work.h>
+#include <linux/user_namespace.h>
+
+#include <uapi/linux/magic.h>
+
+#include <asm/resctrl.h>
+#include "internal.h"
+
+/* Mutex to protect rdtgroup access. */
+DEFINE_MUTEX(rdtgroup_mutex);
+
+static struct kernfs_root *rdt_root;
+struct rdtgroup rdtgroup_default;
+LIST_HEAD(rdt_all_groups);
+
+/* list of entries for the schemata file */
+LIST_HEAD(resctrl_schema_all);
+
+/* The filesystem can only be mounted once. */
+bool resctrl_mounted;
+
+/* Kernel fs node for "info" directory under root */
+static struct kernfs_node *kn_info;
+
+/* Kernel fs node for "mon_groups" directory under root */
+static struct kernfs_node *kn_mongrp;
+
+/* Kernel fs node for "mon_data" directory under root */
+static struct kernfs_node *kn_mondata;
+
+/*
+ * Used to store the max resource name width and max resource data width
+ * to display the schemata in a tabular format
+ */
+int max_name_width, max_data_width;
+
+static struct seq_buf last_cmd_status;
+static char last_cmd_status_buf[512];
+
+static int rdtgroup_setup_root(struct rdt_fs_context *ctx);
+static void rdtgroup_destroy_root(void);
+
+struct dentry *debugfs_resctrl;
+
+static bool resctrl_debug;
+
+void rdt_last_cmd_clear(void)
+{
+	lockdep_assert_held(&rdtgroup_mutex);
+	seq_buf_clear(&last_cmd_status);
+}
+
+void rdt_last_cmd_puts(const char *s)
+{
+	lockdep_assert_held(&rdtgroup_mutex);
+	seq_buf_puts(&last_cmd_status, s);
+}
+
+void rdt_last_cmd_printf(const char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	lockdep_assert_held(&rdtgroup_mutex);
+	seq_buf_vprintf(&last_cmd_status, fmt, ap);
+	va_end(ap);
+}
+
+void rdt_staged_configs_clear(void)
+{
+	enum resctrl_res_level i;
+	struct rdt_resource *r;
+	struct rdt_domain *dom;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->alloc_capable)
+			continue;
+
+		list_for_each_entry(dom, &r->domains, list)
+			memset(dom->staged_config, 0, sizeof(dom->staged_config));
+	}
+}
+
+static bool resctrl_is_mbm_enabled(void)
+{
+	return (resctrl_arch_is_mbm_total_enabled() ||
+		resctrl_arch_is_mbm_local_enabled());
+}
+
+static bool resctrl_is_mbm_event(int e)
+{
+	return (e >= QOS_L3_MBM_TOTAL_EVENT_ID &&
+		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
+}
+
+/*
+ * Trivial allocator for CLOSIDs. Since h/w only supports a small number,
+ * we can keep a bitmap of free CLOSIDs in a single integer.
+ *
+ * Using a global CLOSID across all resources has some advantages and
+ * some drawbacks:
+ * + We can simply set current's closid to assign a task to a resource
+ *   group.
+ * + Context switch code can avoid extra memory references deciding which
+ *   CLOSID to load into the PQR_ASSOC MSR
+ * - We give up some options in configuring resource groups across multi-socket
+ *   systems.
+ * - Our choices on how to configure each resource become progressively more
+ *   limited as the number of resources grows.
+ */
+static unsigned long closid_free_map;
+static int closid_free_map_len;
+
+int closids_supported(void)
+{
+	return closid_free_map_len;
+}
+
+static void closid_init(void)
+{
+	struct resctrl_schema *s;
+	u32 rdt_min_closid = 32;
+
+	/* Compute rdt_min_closid across all resources */
+	list_for_each_entry(s, &resctrl_schema_all, list)
+		rdt_min_closid = min(rdt_min_closid, s->num_closid);
+
+	closid_free_map = BIT_MASK(rdt_min_closid) - 1;
+
+	/* RESCTRL_RESERVED_CLOSID is always reserved for the default group */
+	__clear_bit(RESCTRL_RESERVED_CLOSID, &closid_free_map);
+	closid_free_map_len = rdt_min_closid;
+}
+
+static int closid_alloc(void)
+{
+	int cleanest_closid;
+	u32 closid;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
+	    resctrl_arch_is_llc_occupancy_enabled()) {
+		cleanest_closid = resctrl_find_cleanest_closid();
+		if (cleanest_closid < 0)
+			return cleanest_closid;
+		closid = cleanest_closid;
+	} else {
+		closid = ffs(closid_free_map);
+		if (closid == 0)
+			return -ENOSPC;
+		closid--;
+	}
+	__clear_bit(closid, &closid_free_map);
+
+	return closid;
+}
+
+void closid_free(int closid)
+{
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	__set_bit(closid, &closid_free_map);
+}
+
+/**
+ * closid_allocated - test if provided closid is in use
+ * @closid: closid to be tested
+ *
+ * Return: true if @closid is currently associated with a resource group,
+ * false if @closid is free
+ */
+bool closid_allocated(unsigned int closid)
+{
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	return !test_bit(closid, &closid_free_map);
+}
+
+/**
+ * rdtgroup_mode_by_closid - Return mode of resource group with closid
+ * @closid: closid if the resource group
+ *
+ * Each resource group is associated with a @closid. Here the mode
+ * of a resource group can be queried by searching for it using its closid.
+ *
+ * Return: mode as &enum rdtgrp_mode of resource group with closid @closid
+ */
+enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)
+{
+	struct rdtgroup *rdtgrp;
+
+	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
+		if (rdtgrp->closid == closid)
+			return rdtgrp->mode;
+	}
+
+	return RDT_NUM_MODES;
+}
+
+static const char * const rdt_mode_str[] = {
+	[RDT_MODE_SHAREABLE]		= "shareable",
+	[RDT_MODE_EXCLUSIVE]		= "exclusive",
+	[RDT_MODE_PSEUDO_LOCKSETUP]	= "pseudo-locksetup",
+	[RDT_MODE_PSEUDO_LOCKED]	= "pseudo-locked",
+};
+
+/**
+ * rdtgroup_mode_str - Return the string representation of mode
+ * @mode: the resource group mode as &enum rdtgroup_mode
+ *
+ * Return: string representation of valid mode, "unknown" otherwise
+ */
+static const char *rdtgroup_mode_str(enum rdtgrp_mode mode)
+{
+	if (mode < RDT_MODE_SHAREABLE || mode >= RDT_NUM_MODES)
+		return "unknown";
+
+	return rdt_mode_str[mode];
+}
+
+/* set uid and gid of rdtgroup dirs and files to that of the creator */
+static int rdtgroup_kn_set_ugid(struct kernfs_node *kn)
+{
+	struct iattr iattr = { .ia_valid = ATTR_UID | ATTR_GID,
+				.ia_uid = current_fsuid(),
+				.ia_gid = current_fsgid(), };
+
+	if (uid_eq(iattr.ia_uid, GLOBAL_ROOT_UID) &&
+	    gid_eq(iattr.ia_gid, GLOBAL_ROOT_GID))
+		return 0;
+
+	return kernfs_setattr(kn, &iattr);
+}
+
+static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
+{
+	struct kernfs_node *kn;
+	int ret;
+
+	kn = __kernfs_create_file(parent_kn, rft->name, rft->mode,
+				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID,
+				  0, rft->kf_ops, rft, NULL, NULL);
+	if (IS_ERR(kn))
+		return PTR_ERR(kn);
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret) {
+		kernfs_remove(kn);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int rdtgroup_seqfile_show(struct seq_file *m, void *arg)
+{
+	struct kernfs_open_file *of = m->private;
+	struct rftype *rft = of->kn->priv;
+
+	if (rft->seq_show)
+		return rft->seq_show(of, m, arg);
+	return 0;
+}
+
+static ssize_t rdtgroup_file_write(struct kernfs_open_file *of, char *buf,
+				   size_t nbytes, loff_t off)
+{
+	struct rftype *rft = of->kn->priv;
+
+	if (rft->write)
+		return rft->write(of, buf, nbytes, off);
+
+	return -EINVAL;
+}
+
+static const struct kernfs_ops rdtgroup_kf_single_ops = {
+	.atomic_write_len	= PAGE_SIZE,
+	.write			= rdtgroup_file_write,
+	.seq_show		= rdtgroup_seqfile_show,
+};
+
+static const struct kernfs_ops kf_mondata_ops = {
+	.atomic_write_len	= PAGE_SIZE,
+	.seq_show		= rdtgroup_mondata_show,
+};
+
+static bool is_cpu_list(struct kernfs_open_file *of)
+{
+	struct rftype *rft = of->kn->priv;
+
+	return rft->flags & RFTYPE_FLAGS_CPUS_LIST;
+}
+
+static int rdtgroup_cpus_show(struct kernfs_open_file *of,
+			      struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	struct cpumask *mask;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+
+	if (rdtgrp) {
+		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
+			if (!rdtgrp->plr->d) {
+				rdt_last_cmd_clear();
+				rdt_last_cmd_puts("Cache domain offline\n");
+				ret = -ENODEV;
+			} else {
+				mask = &rdtgrp->plr->d->cpu_mask;
+				seq_printf(s, is_cpu_list(of) ?
+					   "%*pbl\n" : "%*pb\n",
+					   cpumask_pr_args(mask));
+			}
+		} else {
+			seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
+				   cpumask_pr_args(&rdtgrp->cpu_mask));
+		}
+	} else {
+		ret = -ENOENT;
+	}
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
+/*
+ * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
+ *
+ * Per task closids/rmids must have been set up before calling this function.
+ * @r may be NULL.
+ */
+static void
+update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
+{
+	struct resctrl_cpu_sync defaults;
+	struct resctrl_cpu_sync *defaults_p = NULL;
+
+	if (r) {
+		defaults.closid = r->closid;
+		defaults.rmid = r->mon.rmid;
+		defaults_p = &defaults;
+	}
+
+	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
+			 1);
+}
+
+static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
+			  cpumask_var_t tmpmask)
+{
+	struct rdtgroup *prgrp = rdtgrp->mon.parent, *crgrp;
+	struct list_head *head;
+
+	/* Check whether cpus belong to parent ctrl group */
+	cpumask_andnot(tmpmask, newmask, &prgrp->cpu_mask);
+	if (!cpumask_empty(tmpmask)) {
+		rdt_last_cmd_puts("Can only add CPUs to mongroup that belong to parent\n");
+		return -EINVAL;
+	}
+
+	/* Check whether cpus are dropped from this group */
+	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
+	if (!cpumask_empty(tmpmask)) {
+		/* Give any dropped cpus to parent rdtgroup */
+		cpumask_or(&prgrp->cpu_mask, &prgrp->cpu_mask, tmpmask);
+		update_closid_rmid(tmpmask, prgrp);
+	}
+
+	/*
+	 * If we added cpus, remove them from previous group that owned them
+	 * and update per-cpu rmid
+	 */
+	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
+	if (!cpumask_empty(tmpmask)) {
+		head = &prgrp->mon.crdtgrp_list;
+		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
+			if (crgrp == rdtgrp)
+				continue;
+			cpumask_andnot(&crgrp->cpu_mask, &crgrp->cpu_mask,
+				       tmpmask);
+		}
+		update_closid_rmid(tmpmask, rdtgrp);
+	}
+
+	/* Done pushing/pulling - update this group with new mask */
+	cpumask_copy(&rdtgrp->cpu_mask, newmask);
+
+	return 0;
+}
+
+static void cpumask_rdtgrp_clear(struct rdtgroup *r, struct cpumask *m)
+{
+	struct rdtgroup *crgrp;
+
+	cpumask_andnot(&r->cpu_mask, &r->cpu_mask, m);
+	/* update the child mon group masks as well*/
+	list_for_each_entry(crgrp, &r->mon.crdtgrp_list, mon.crdtgrp_list)
+		cpumask_and(&crgrp->cpu_mask, &r->cpu_mask, &crgrp->cpu_mask);
+}
+
+static int cpus_ctrl_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
+			   cpumask_var_t tmpmask, cpumask_var_t tmpmask1)
+{
+	struct rdtgroup *r, *crgrp;
+	struct list_head *head;
+
+	/* Check whether cpus are dropped from this group */
+	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
+	if (!cpumask_empty(tmpmask)) {
+		/* Can't drop from default group */
+		if (rdtgrp == &rdtgroup_default) {
+			rdt_last_cmd_puts("Can't drop CPUs from default group\n");
+			return -EINVAL;
+		}
+
+		/* Give any dropped cpus to rdtgroup_default */
+		cpumask_or(&rdtgroup_default.cpu_mask,
+			   &rdtgroup_default.cpu_mask, tmpmask);
+		update_closid_rmid(tmpmask, &rdtgroup_default);
+	}
+
+	/*
+	 * If we added cpus, remove them from previous group and
+	 * the prev group's child groups that owned them
+	 * and update per-cpu closid/rmid.
+	 */
+	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
+	if (!cpumask_empty(tmpmask)) {
+		list_for_each_entry(r, &rdt_all_groups, rdtgroup_list) {
+			if (r == rdtgrp)
+				continue;
+			cpumask_and(tmpmask1, &r->cpu_mask, tmpmask);
+			if (!cpumask_empty(tmpmask1))
+				cpumask_rdtgrp_clear(r, tmpmask1);
+		}
+		update_closid_rmid(tmpmask, rdtgrp);
+	}
+
+	/* Done pushing/pulling - update this group with new mask */
+	cpumask_copy(&rdtgrp->cpu_mask, newmask);
+
+	/*
+	 * Clear child mon group masks since there is a new parent mask
+	 * now and update the rmid for the cpus the child lost.
+	 */
+	head = &rdtgrp->mon.crdtgrp_list;
+	list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
+		cpumask_and(tmpmask, &rdtgrp->cpu_mask, &crgrp->cpu_mask);
+		update_closid_rmid(tmpmask, rdtgrp);
+		cpumask_clear(&crgrp->cpu_mask);
+	}
+
+	return 0;
+}
+
+static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of,
+				   char *buf, size_t nbytes, loff_t off)
+{
+	cpumask_var_t tmpmask, newmask, tmpmask1;
+	struct rdtgroup *rdtgrp;
+	int ret;
+
+	if (!buf)
+		return -EINVAL;
+
+	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
+		return -ENOMEM;
+	if (!zalloc_cpumask_var(&newmask, GFP_KERNEL)) {
+		free_cpumask_var(tmpmask);
+		return -ENOMEM;
+	}
+	if (!zalloc_cpumask_var(&tmpmask1, GFP_KERNEL)) {
+		free_cpumask_var(tmpmask);
+		free_cpumask_var(newmask);
+		return -ENOMEM;
+	}
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		ret = -ENOENT;
+		goto unlock;
+	}
+
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
+	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+		ret = -EINVAL;
+		rdt_last_cmd_puts("Pseudo-locking in progress\n");
+		goto unlock;
+	}
+
+	if (is_cpu_list(of))
+		ret = cpulist_parse(buf, newmask);
+	else
+		ret = cpumask_parse(buf, newmask);
+
+	if (ret) {
+		rdt_last_cmd_puts("Bad CPU list/mask\n");
+		goto unlock;
+	}
+
+	/* check that user didn't specify any offline cpus */
+	cpumask_andnot(tmpmask, newmask, cpu_online_mask);
+	if (!cpumask_empty(tmpmask)) {
+		ret = -EINVAL;
+		rdt_last_cmd_puts("Can only assign online CPUs\n");
+		goto unlock;
+	}
+
+	if (rdtgrp->type == RDTCTRL_GROUP)
+		ret = cpus_ctrl_write(rdtgrp, newmask, tmpmask, tmpmask1);
+	else if (rdtgrp->type == RDTMON_GROUP)
+		ret = cpus_mon_write(rdtgrp, newmask, tmpmask);
+	else
+		ret = -EINVAL;
+
+unlock:
+	rdtgroup_kn_unlock(of->kn);
+	free_cpumask_var(tmpmask);
+	free_cpumask_var(newmask);
+	free_cpumask_var(tmpmask1);
+
+	return ret ?: nbytes;
+}
+
+/**
+ * rdtgroup_remove - the helper to remove resource group safely
+ * @rdtgrp: resource group to remove
+ *
+ * On resource group creation via a mkdir, an extra kernfs_node reference is
+ * taken to ensure that the rdtgroup structure remains accessible for the
+ * rdtgroup_kn_unlock() calls where it is removed.
+ *
+ * Drop the extra reference here, then free the rdtgroup structure.
+ *
+ * Return: void
+ */
+static void rdtgroup_remove(struct rdtgroup *rdtgrp)
+{
+	kernfs_put(rdtgrp->kn);
+	kfree(rdtgrp);
+}
+
+static void _update_task_closid_rmid(void *task)
+{
+	/*
+	 * If the task is still current on this CPU, update PQR_ASSOC MSR.
+	 * Otherwise, the MSR is updated when the task is scheduled in.
+	 */
+	if (task == current)
+		resctrl_arch_sched_in(task);
+}
+
+static void update_task_closid_rmid(struct task_struct *t)
+{
+	if (IS_ENABLED(CONFIG_SMP) && task_curr(t))
+		smp_call_function_single(task_cpu(t), _update_task_closid_rmid, t, 1);
+	else
+		_update_task_closid_rmid(t);
+}
+
+static bool task_in_rdtgroup(struct task_struct *tsk, struct rdtgroup *rdtgrp)
+{
+	u32 closid, rmid = rdtgrp->mon.rmid;
+
+	if (rdtgrp->type == RDTCTRL_GROUP)
+		closid = rdtgrp->closid;
+	else if (rdtgrp->type == RDTMON_GROUP)
+		closid = rdtgrp->mon.parent->closid;
+	else
+		return false;
+
+	return resctrl_arch_match_closid(tsk, closid) &&
+	       resctrl_arch_match_rmid(tsk, closid, rmid);
+}
+
+static int __rdtgroup_move_task(struct task_struct *tsk,
+				struct rdtgroup *rdtgrp)
+{
+	/* If the task is already in rdtgrp, no need to move the task. */
+	if (task_in_rdtgroup(tsk, rdtgrp))
+		return 0;
+
+	/*
+	 * Set the task's closid/rmid before the PQR_ASSOC MSR can be
+	 * updated by them.
+	 *
+	 * For ctrl_mon groups, move both closid and rmid.
+	 * For monitor groups, can move the tasks only from
+	 * their parent CTRL group.
+	 */
+	if (rdtgrp->type == RDTMON_GROUP &&
+	    !resctrl_arch_match_closid(tsk, rdtgrp->mon.parent->closid)) {
+		rdt_last_cmd_puts("Can't move task to different control group\n");
+		return -EINVAL;
+	}
+
+	if (rdtgrp->type == RDTMON_GROUP)
+		resctrl_arch_set_closid_rmid(tsk, rdtgrp->mon.parent->closid,
+					     rdtgrp->mon.rmid);
+	else
+		resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid,
+					     rdtgrp->mon.rmid);
+
+	/*
+	 * Ensure the task's closid and rmid are written before determining if
+	 * the task is current that will decide if it will be interrupted.
+	 * This pairs with the full barrier between the rq->curr update and
+	 * resctrl_arch_sched_in() during context switch.
+	 */
+	smp_mb();
+
+	/*
+	 * By now, the task's closid and rmid are set. If the task is current
+	 * on a CPU, the PQR_ASSOC MSR needs to be updated to make the resource
+	 * group go into effect. If the task is not current, the MSR will be
+	 * updated when the task is scheduled in.
+	 */
+	update_task_closid_rmid(tsk);
+
+	return 0;
+}
+
+static bool is_closid_match(struct task_struct *t, struct rdtgroup *r)
+{
+	return (resctrl_arch_alloc_capable() && (r->type == RDTCTRL_GROUP) &&
+		resctrl_arch_match_closid(t, r->closid));
+}
+
+static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r)
+{
+	return (resctrl_arch_mon_capable() && (r->type == RDTMON_GROUP) &&
+		resctrl_arch_match_rmid(t, r->mon.parent->closid,
+					r->mon.rmid));
+}
+
+/**
+ * rdtgroup_tasks_assigned - Test if tasks have been assigned to resource group
+ * @r: Resource group
+ *
+ * Return: 1 if tasks have been assigned to @r, 0 otherwise
+ */
+int rdtgroup_tasks_assigned(struct rdtgroup *r)
+{
+	struct task_struct *p, *t;
+	int ret = 0;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	rcu_read_lock();
+	for_each_process_thread(p, t) {
+		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
+			ret = 1;
+			break;
+		}
+	}
+	rcu_read_unlock();
+
+	return ret;
+}
+
+static int rdtgroup_task_write_permission(struct task_struct *task,
+					  struct kernfs_open_file *of)
+{
+	const struct cred *tcred = get_task_cred(task);
+	const struct cred *cred = current_cred();
+	int ret = 0;
+
+	/*
+	 * Even if we're attaching all tasks in the thread group, we only
+	 * need to check permissions on one of them.
+	 */
+	if (!uid_eq(cred->euid, GLOBAL_ROOT_UID) &&
+	    !uid_eq(cred->euid, tcred->uid) &&
+	    !uid_eq(cred->euid, tcred->suid)) {
+		rdt_last_cmd_printf("No permission to move task %d\n", task->pid);
+		ret = -EPERM;
+	}
+
+	put_cred(tcred);
+	return ret;
+}
+
+static int rdtgroup_move_task(pid_t pid, struct rdtgroup *rdtgrp,
+			      struct kernfs_open_file *of)
+{
+	struct task_struct *tsk;
+	int ret;
+
+	rcu_read_lock();
+	if (pid) {
+		tsk = find_task_by_vpid(pid);
+		if (!tsk) {
+			rcu_read_unlock();
+			rdt_last_cmd_printf("No task %d\n", pid);
+			return -ESRCH;
+		}
+	} else {
+		tsk = current;
+	}
+
+	get_task_struct(tsk);
+	rcu_read_unlock();
+
+	ret = rdtgroup_task_write_permission(tsk, of);
+	if (!ret)
+		ret = __rdtgroup_move_task(tsk, rdtgrp);
+
+	put_task_struct(tsk);
+	return ret;
+}
+
+static ssize_t rdtgroup_tasks_write(struct kernfs_open_file *of,
+				    char *buf, size_t nbytes, loff_t off)
+{
+	struct rdtgroup *rdtgrp;
+	char *pid_str;
+	int ret = 0;
+	pid_t pid;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		rdtgroup_kn_unlock(of->kn);
+		return -ENOENT;
+	}
+	rdt_last_cmd_clear();
+
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
+	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+		ret = -EINVAL;
+		rdt_last_cmd_puts("Pseudo-locking in progress\n");
+		goto unlock;
+	}
+
+	while (buf && buf[0] != '\0' && buf[0] != '\n') {
+		pid_str = strim(strsep(&buf, ","));
+
+		if (kstrtoint(pid_str, 0, &pid)) {
+			rdt_last_cmd_printf("Task list parsing error pid %s\n", pid_str);
+			ret = -EINVAL;
+			break;
+		}
+
+		if (pid < 0) {
+			rdt_last_cmd_printf("Invalid pid %d\n", pid);
+			ret = -EINVAL;
+			break;
+		}
+
+		ret = rdtgroup_move_task(pid, rdtgrp, of);
+		if (ret) {
+			rdt_last_cmd_printf("Error while processing task %d\n", pid);
+			break;
+		}
+	}
+
+unlock:
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret ?: nbytes;
+}
+
+static void show_rdt_tasks(struct rdtgroup *r, struct seq_file *s)
+{
+	struct task_struct *p, *t;
+	pid_t pid;
+
+	rcu_read_lock();
+	for_each_process_thread(p, t) {
+		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
+			pid = task_pid_vnr(t);
+			if (pid)
+				seq_printf(s, "%d\n", pid);
+		}
+	}
+	rcu_read_unlock();
+}
+
+static int rdtgroup_tasks_show(struct kernfs_open_file *of,
+			       struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (rdtgrp)
+		show_rdt_tasks(rdtgrp, s);
+	else
+		ret = -ENOENT;
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
+static int rdtgroup_closid_show(struct kernfs_open_file *of,
+				struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (rdtgrp)
+		seq_printf(s, "%u\n", rdtgrp->closid);
+	else
+		ret = -ENOENT;
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
+static int rdtgroup_rmid_show(struct kernfs_open_file *of,
+			      struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (rdtgrp)
+		seq_printf(s, "%u\n", rdtgrp->mon.rmid);
+	else
+		ret = -ENOENT;
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
+#ifdef CONFIG_PROC_CPU_RESCTRL
+
+/*
+ * A task can only be part of one resctrl control group and of one monitor
+ * group which is associated to that control group.
+ *
+ * 1)   res:
+ *      mon:
+ *
+ *    resctrl is not available.
+ *
+ * 2)   res:/
+ *      mon:
+ *
+ *    Task is part of the root resctrl control group, and it is not associated
+ *    to any monitor group.
+ *
+ * 3)  res:/
+ *     mon:mon0
+ *
+ *    Task is part of the root resctrl control group and monitor group mon0.
+ *
+ * 4)  res:group0
+ *     mon:
+ *
+ *    Task is part of resctrl control group group0, and it is not associated
+ *    to any monitor group.
+ *
+ * 5) res:group0
+ *    mon:mon1
+ *
+ *    Task is part of resctrl control group group0 and monitor group mon1.
+ */
+int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,
+		      struct pid *pid, struct task_struct *tsk)
+{
+	struct rdtgroup *rdtg;
+	int ret = 0;
+
+	mutex_lock(&rdtgroup_mutex);
+
+	/* Return empty if resctrl has not been mounted. */
+	if (!resctrl_mounted) {
+		seq_puts(s, "res:\nmon:\n");
+		goto unlock;
+	}
+
+	list_for_each_entry(rdtg, &rdt_all_groups, rdtgroup_list) {
+		struct rdtgroup *crg;
+
+		/*
+		 * Task information is only relevant for shareable
+		 * and exclusive groups.
+		 */
+		if (rdtg->mode != RDT_MODE_SHAREABLE &&
+		    rdtg->mode != RDT_MODE_EXCLUSIVE)
+			continue;
+
+		if (!resctrl_arch_match_closid(tsk, rdtg->closid))
+			continue;
+
+		seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "",
+			   rdtg->kn->name);
+		seq_puts(s, "mon:");
+		list_for_each_entry(crg, &rdtg->mon.crdtgrp_list,
+				    mon.crdtgrp_list) {
+			if (!resctrl_arch_match_rmid(tsk, crg->mon.parent->closid,
+						     crg->mon.rmid))
+				continue;
+			seq_printf(s, "%s", crg->kn->name);
+			break;
+		}
+		seq_putc(s, '\n');
+		goto unlock;
+	}
+	/*
+	 * The above search should succeed. Otherwise return
+	 * with an error.
+	 */
+	ret = -ENOENT;
+unlock:
+	mutex_unlock(&rdtgroup_mutex);
+
+	return ret;
+}
+#endif
+
+static int rdt_last_cmd_status_show(struct kernfs_open_file *of,
+				    struct seq_file *seq, void *v)
+{
+	int len;
+
+	mutex_lock(&rdtgroup_mutex);
+	len = seq_buf_used(&last_cmd_status);
+	if (len)
+		seq_printf(seq, "%.*s", len, last_cmd_status_buf);
+	else
+		seq_puts(seq, "ok\n");
+	mutex_unlock(&rdtgroup_mutex);
+	return 0;
+}
+
+static int rdt_num_closids_show(struct kernfs_open_file *of,
+				struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+
+	seq_printf(seq, "%u\n", s->num_closid);
+	return 0;
+}
+
+static int rdt_default_ctrl_show(struct kernfs_open_file *of,
+			     struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	seq_printf(seq, "%x\n", r->default_ctrl);
+	return 0;
+}
+
+static int rdt_min_cbm_bits_show(struct kernfs_open_file *of,
+			     struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	seq_printf(seq, "%u\n", r->cache.min_cbm_bits);
+	return 0;
+}
+
+static int rdt_shareable_bits_show(struct kernfs_open_file *of,
+				   struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	seq_printf(seq, "%x\n", r->cache.shareable_bits);
+	return 0;
+}
+
+/*
+ * rdt_bit_usage_show - Display current usage of resources
+ *
+ * A domain is a shared resource that can now be allocated differently. Here
+ * we display the current regions of the domain as an annotated bitmask.
+ * For each domain of this resource its allocation bitmask
+ * is annotated as below to indicate the current usage of the corresponding bit:
+ *   0 - currently unused
+ *   X - currently available for sharing and used by software and hardware
+ *   H - currently used by hardware only but available for software use
+ *   S - currently used and shareable by software only
+ *   E - currently used exclusively by one resource group
+ *   P - currently pseudo-locked by one resource group
+ */
+static int rdt_bit_usage_show(struct kernfs_open_file *of,
+			      struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	/*
+	 * Use unsigned long even though only 32 bits are used to ensure
+	 * test_bit() is used safely.
+	 */
+	unsigned long sw_shareable = 0, hw_shareable = 0;
+	unsigned long exclusive = 0, pseudo_locked = 0;
+	struct rdt_resource *r = s->res;
+	struct rdt_domain *dom;
+	int i, hwb, swb, excl, psl;
+	enum rdtgrp_mode mode;
+	bool sep = false;
+	u32 ctrl_val;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+	hw_shareable = r->cache.shareable_bits;
+	list_for_each_entry(dom, &r->domains, list) {
+		if (sep)
+			seq_putc(seq, ';');
+		sw_shareable = 0;
+		exclusive = 0;
+		seq_printf(seq, "%d=", dom->id);
+		for (i = 0; i < closids_supported(); i++) {
+			if (!closid_allocated(i))
+				continue;
+			ctrl_val = resctrl_arch_get_config(r, dom, i,
+							   s->conf_type);
+			mode = rdtgroup_mode_by_closid(i);
+			switch (mode) {
+			case RDT_MODE_SHAREABLE:
+				sw_shareable |= ctrl_val;
+				break;
+			case RDT_MODE_EXCLUSIVE:
+				exclusive |= ctrl_val;
+				break;
+			case RDT_MODE_PSEUDO_LOCKSETUP:
+			/*
+			 * RDT_MODE_PSEUDO_LOCKSETUP is possible
+			 * here but not included since the CBM
+			 * associated with this CLOSID in this mode
+			 * is not initialized and no task or cpu can be
+			 * assigned this CLOSID.
+			 */
+				break;
+			case RDT_MODE_PSEUDO_LOCKED:
+			case RDT_NUM_MODES:
+				WARN(1,
+				     "invalid mode for closid %d\n", i);
+				break;
+			}
+		}
+		for (i = r->cache.cbm_len - 1; i >= 0; i--) {
+			pseudo_locked = dom->plr ? dom->plr->cbm : 0;
+			hwb = test_bit(i, &hw_shareable);
+			swb = test_bit(i, &sw_shareable);
+			excl = test_bit(i, &exclusive);
+			psl = test_bit(i, &pseudo_locked);
+			if (hwb && swb)
+				seq_putc(seq, 'X');
+			else if (hwb && !swb)
+				seq_putc(seq, 'H');
+			else if (!hwb && swb)
+				seq_putc(seq, 'S');
+			else if (excl)
+				seq_putc(seq, 'E');
+			else if (psl)
+				seq_putc(seq, 'P');
+			else /* Unused bits remain */
+				seq_putc(seq, '0');
+		}
+		sep = true;
+	}
+	seq_putc(seq, '\n');
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+	return 0;
+}
+
+static int rdt_min_bw_show(struct kernfs_open_file *of,
+			     struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	seq_printf(seq, "%u\n", r->membw.min_bw);
+	return 0;
+}
+
+static int rdt_num_rmids_show(struct kernfs_open_file *of,
+			      struct seq_file *seq, void *v)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+
+	seq_printf(seq, "%d\n", r->num_rmid);
+
+	return 0;
+}
+
+static int rdt_mon_features_show(struct kernfs_open_file *of,
+				 struct seq_file *seq, void *v)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+	struct mon_evt *mevt;
+
+	list_for_each_entry(mevt, &r->evt_list, list) {
+		seq_printf(seq, "%s\n", mevt->name);
+		if (mevt->configurable)
+			seq_printf(seq, "%s_config\n", mevt->name);
+	}
+
+	return 0;
+}
+
+static int rdt_bw_gran_show(struct kernfs_open_file *of,
+			     struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	seq_printf(seq, "%u\n", r->membw.bw_gran);
+	return 0;
+}
+
+static int rdt_delay_linear_show(struct kernfs_open_file *of,
+			     struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	seq_printf(seq, "%u\n", r->membw.delay_linear);
+	return 0;
+}
+
+static int max_threshold_occ_show(struct kernfs_open_file *of,
+				  struct seq_file *seq, void *v)
+{
+	seq_printf(seq, "%u\n", resctrl_rmid_realloc_threshold);
+
+	return 0;
+}
+
+static int rdt_thread_throttle_mode_show(struct kernfs_open_file *of,
+					 struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	if (r->membw.throttle_mode == THREAD_THROTTLE_PER_THREAD)
+		seq_puts(seq, "per-thread\n");
+	else
+		seq_puts(seq, "max\n");
+
+	return 0;
+}
+
+static ssize_t max_threshold_occ_write(struct kernfs_open_file *of,
+				       char *buf, size_t nbytes, loff_t off)
+{
+	unsigned int bytes;
+	int ret;
+
+	ret = kstrtouint(buf, 0, &bytes);
+	if (ret)
+		return ret;
+
+	if (bytes > resctrl_rmid_realloc_limit)
+		return -EINVAL;
+
+	resctrl_rmid_realloc_threshold = resctrl_arch_round_mon_val(bytes);
+
+	return nbytes;
+}
+
+/*
+ * rdtgroup_mode_show - Display mode of this resource group
+ */
+static int rdtgroup_mode_show(struct kernfs_open_file *of,
+			      struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		rdtgroup_kn_unlock(of->kn);
+		return -ENOENT;
+	}
+
+	seq_printf(s, "%s\n", rdtgroup_mode_str(rdtgrp->mode));
+
+	rdtgroup_kn_unlock(of->kn);
+	return 0;
+}
+
+static enum resctrl_conf_type resctrl_peer_type(enum resctrl_conf_type my_type)
+{
+	switch (my_type) {
+	case CDP_CODE:
+		return CDP_DATA;
+	case CDP_DATA:
+		return CDP_CODE;
+	default:
+	case CDP_NONE:
+		return CDP_NONE;
+	}
+}
+
+static int rdt_has_sparse_bitmasks_show(struct kernfs_open_file *of,
+					struct seq_file *seq, void *v)
+{
+	struct resctrl_schema *s = of->kn->parent->priv;
+	struct rdt_resource *r = s->res;
+
+	seq_printf(seq, "%u\n", r->cache.arch_has_sparse_bitmasks);
+
+	return 0;
+}
+
+/**
+ * __rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other
+ * @r: Resource to which domain instance @d belongs.
+ * @d: The domain instance for which @closid is being tested.
+ * @cbm: Capacity bitmask being tested.
+ * @closid: Intended closid for @cbm.
+ * @type: CDP type of @r.
+ * @exclusive: Only check if overlaps with exclusive resource groups
+ *
+ * Checks if provided @cbm intended to be used for @closid on domain
+ * @d overlaps with any other closids or other hardware usage associated
+ * with this domain. If @exclusive is true then only overlaps with
+ * resource groups in exclusive mode will be considered. If @exclusive
+ * is false then overlaps with any resource group or hardware entities
+ * will be considered.
+ *
+ * @cbm is unsigned long, even if only 32 bits are used, to make the
+ * bitmap functions work correctly.
+ *
+ * Return: false if CBM does not overlap, true if it does.
+ */
+static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
+				    unsigned long cbm, int closid,
+				    enum resctrl_conf_type type, bool exclusive)
+{
+	enum rdtgrp_mode mode;
+	unsigned long ctrl_b;
+	int i;
+
+	/* Check for any overlap with regions used by hardware directly */
+	if (!exclusive) {
+		ctrl_b = r->cache.shareable_bits;
+		if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len))
+			return true;
+	}
+
+	/* Check for overlap with other resource groups */
+	for (i = 0; i < closids_supported(); i++) {
+		ctrl_b = resctrl_arch_get_config(r, d, i, type);
+		mode = rdtgroup_mode_by_closid(i);
+		if (closid_allocated(i) && i != closid &&
+		    mode != RDT_MODE_PSEUDO_LOCKSETUP) {
+			if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len)) {
+				if (exclusive) {
+					if (mode == RDT_MODE_EXCLUSIVE)
+						return true;
+					continue;
+				}
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/**
+ * rdtgroup_cbm_overlaps - Does CBM overlap with other use of hardware
+ * @s: Schema for the resource to which domain instance @d belongs.
+ * @d: The domain instance for which @closid is being tested.
+ * @cbm: Capacity bitmask being tested.
+ * @closid: Intended closid for @cbm.
+ * @exclusive: Only check if overlaps with exclusive resource groups
+ *
+ * Resources that can be allocated using a CBM can use the CBM to control
+ * the overlap of these allocations. rdtgroup_cmb_overlaps() is the test
+ * for overlap. Overlap test is not limited to the specific resource for
+ * which the CBM is intended though - when dealing with CDP resources that
+ * share the underlying hardware the overlap check should be performed on
+ * the CDP resource sharing the hardware also.
+ *
+ * Refer to description of __rdtgroup_cbm_overlaps() for the details of the
+ * overlap test.
+ *
+ * Return: true if CBM overlap detected, false if there is no overlap
+ */
+bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
+			   unsigned long cbm, int closid, bool exclusive)
+{
+	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
+	struct rdt_resource *r = s->res;
+
+	if (__rdtgroup_cbm_overlaps(r, d, cbm, closid, s->conf_type,
+				    exclusive))
+		return true;
+
+	if (!resctrl_arch_get_cdp_enabled(r->rid))
+		return false;
+	return  __rdtgroup_cbm_overlaps(r, d, cbm, closid, peer_type, exclusive);
+}
+
+/**
+ * rdtgroup_mode_test_exclusive - Test if this resource group can be exclusive
+ * @rdtgrp: Resource group identified through its closid.
+ *
+ * An exclusive resource group implies that there should be no sharing of
+ * its allocated resources. At the time this group is considered to be
+ * exclusive this test can determine if its current schemata supports this
+ * setting by testing for overlap with all other resource groups.
+ *
+ * Return: true if resource group can be exclusive, false if there is overlap
+ * with allocations of other resource groups and thus this resource group
+ * cannot be exclusive.
+ */
+static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
+{
+	int closid = rdtgrp->closid;
+	struct resctrl_schema *s;
+	struct rdt_resource *r;
+	bool has_cache = false;
+	struct rdt_domain *d;
+	u32 ctrl;
+
+	/* Walking r->domains, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+	list_for_each_entry(s, &resctrl_schema_all, list) {
+		r = s->res;
+		if (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)
+			continue;
+		has_cache = true;
+		list_for_each_entry(d, &r->domains, list) {
+			ctrl = resctrl_arch_get_config(r, d, closid,
+						       s->conf_type);
+			if (rdtgroup_cbm_overlaps(s, d, ctrl, closid, false)) {
+				rdt_last_cmd_puts("Schemata overlaps\n");
+				return false;
+			}
+		}
+	}
+
+	if (!has_cache) {
+		rdt_last_cmd_puts("Cannot be exclusive without CAT/CDP\n");
+		return false;
+	}
+
+	return true;
+}
+
+/*
+ * rdtgroup_mode_write - Modify the resource group's mode
+ */
+static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
+				   char *buf, size_t nbytes, loff_t off)
+{
+	struct rdtgroup *rdtgrp;
+	enum rdtgrp_mode mode;
+	int ret = 0;
+
+	/* Valid input requires a trailing newline */
+	if (nbytes == 0 || buf[nbytes - 1] != '\n')
+		return -EINVAL;
+	buf[nbytes - 1] = '\0';
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		rdtgroup_kn_unlock(of->kn);
+		return -ENOENT;
+	}
+
+	rdt_last_cmd_clear();
+
+	mode = rdtgrp->mode;
+
+	if ((!strcmp(buf, "shareable") && mode == RDT_MODE_SHAREABLE) ||
+	    (!strcmp(buf, "exclusive") && mode == RDT_MODE_EXCLUSIVE) ||
+	    (!strcmp(buf, "pseudo-locksetup") &&
+	     mode == RDT_MODE_PSEUDO_LOCKSETUP) ||
+	    (!strcmp(buf, "pseudo-locked") && mode == RDT_MODE_PSEUDO_LOCKED))
+		goto out;
+
+	if (mode == RDT_MODE_PSEUDO_LOCKED) {
+		rdt_last_cmd_puts("Cannot change pseudo-locked group\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (!strcmp(buf, "shareable")) {
+		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+			ret = rdtgroup_locksetup_exit(rdtgrp);
+			if (ret)
+				goto out;
+		}
+		rdtgrp->mode = RDT_MODE_SHAREABLE;
+	} else if (!strcmp(buf, "exclusive")) {
+		if (!rdtgroup_mode_test_exclusive(rdtgrp)) {
+			ret = -EINVAL;
+			goto out;
+		}
+		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+			ret = rdtgroup_locksetup_exit(rdtgrp);
+			if (ret)
+				goto out;
+		}
+		rdtgrp->mode = RDT_MODE_EXCLUSIVE;
+	} else if (IS_ENABLED(CONFIG_RESCTRL_FS_PSEUDO_LOCK) &&
+		   !strcmp(buf, "pseudo-locksetup")) {
+		ret = rdtgroup_locksetup_enter(rdtgrp);
+		if (ret)
+			goto out;
+		rdtgrp->mode = RDT_MODE_PSEUDO_LOCKSETUP;
+	} else {
+		rdt_last_cmd_puts("Unknown or unsupported mode\n");
+		ret = -EINVAL;
+	}
+
+out:
+	rdtgroup_kn_unlock(of->kn);
+	return ret ?: nbytes;
+}
+
+/**
+ * rdtgroup_cbm_to_size - Translate CBM to size in bytes
+ * @r: RDT resource to which @d belongs.
+ * @d: RDT domain instance.
+ * @cbm: bitmask for which the size should be computed.
+ *
+ * The bitmask provided associated with the RDT domain instance @d will be
+ * translated into how many bytes it represents. The size in bytes is
+ * computed by first dividing the total cache size by the CBM length to
+ * determine how many bytes each bit in the bitmask represents. The result
+ * is multiplied with the number of bits set in the bitmask.
+ *
+ * @cbm is unsigned long, even if only 32 bits are used to make the
+ * bitmap functions work correctly.
+ */
+unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,
+				  struct rdt_domain *d, unsigned long cbm)
+{
+	struct cpu_cacheinfo *ci;
+	unsigned int size = 0;
+	int num_b, i;
+
+	num_b = bitmap_weight(&cbm, r->cache.cbm_len);
+	ci = get_cpu_cacheinfo(cpumask_any(&d->cpu_mask));
+	for (i = 0; i < ci->num_leaves; i++) {
+		if (ci->info_list[i].level == r->cache_level) {
+			size = ci->info_list[i].size / r->cache.cbm_len * num_b;
+			break;
+		}
+	}
+
+	return size;
+}
+
+/*
+ * rdtgroup_size_show - Display size in bytes of allocated regions
+ *
+ * The "size" file mirrors the layout of the "schemata" file, printing the
+ * size in bytes of each region instead of the capacity bitmask.
+ */
+static int rdtgroup_size_show(struct kernfs_open_file *of,
+			      struct seq_file *s, void *v)
+{
+	struct resctrl_schema *schema;
+	enum resctrl_conf_type type;
+	struct rdtgroup *rdtgrp;
+	struct rdt_resource *r;
+	struct rdt_domain *d;
+	unsigned int size;
+	int ret = 0;
+	u32 closid;
+	bool sep;
+	u32 ctrl;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		rdtgroup_kn_unlock(of->kn);
+		return -ENOENT;
+	}
+
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
+		if (!rdtgrp->plr->d) {
+			rdt_last_cmd_clear();
+			rdt_last_cmd_puts("Cache domain offline\n");
+			ret = -ENODEV;
+		} else {
+			seq_printf(s, "%*s:", max_name_width,
+				   rdtgrp->plr->s->name);
+			size = rdtgroup_cbm_to_size(rdtgrp->plr->s->res,
+						    rdtgrp->plr->d,
+						    rdtgrp->plr->cbm);
+			seq_printf(s, "%d=%u\n", rdtgrp->plr->d->id, size);
+		}
+		goto out;
+	}
+
+	closid = rdtgrp->closid;
+
+	list_for_each_entry(schema, &resctrl_schema_all, list) {
+		r = schema->res;
+		type = schema->conf_type;
+		sep = false;
+		seq_printf(s, "%*s:", max_name_width, schema->name);
+		list_for_each_entry(d, &r->domains, list) {
+			if (sep)
+				seq_putc(s, ';');
+			if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
+				size = 0;
+			} else {
+				if (is_mba_sc(r))
+					ctrl = d->mbps_val[closid];
+				else
+					ctrl = resctrl_arch_get_config(r, d,
+								       closid,
+								       type);
+				if (r->rid == RDT_RESOURCE_MBA ||
+				    r->rid == RDT_RESOURCE_SMBA)
+					size = ctrl;
+				else
+					size = rdtgroup_cbm_to_size(r, d, ctrl);
+			}
+			seq_printf(s, "%d=%u", d->id, size);
+			sep = true;
+		}
+		seq_putc(s, '\n');
+	}
+
+out:
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
+static void mondata_config_read(struct resctrl_mon_config_info *mon_info)
+{
+	smp_call_function_any(&mon_info->d->cpu_mask,
+			      resctrl_arch_mon_event_config_read, mon_info, 1);
+}
+
+static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid)
+{
+	struct resctrl_mon_config_info mon_info = {0};
+	struct rdt_domain *dom;
+	bool sep = false;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	list_for_each_entry(dom, &r->domains, list) {
+		if (sep)
+			seq_puts(s, ";");
+
+		memset(&mon_info, 0, sizeof(struct resctrl_mon_config_info));
+		mon_info.r = r;
+		mon_info.d = dom;
+		mon_info.evtid = evtid;
+		mondata_config_read(&mon_info);
+
+		seq_printf(s, "%d=0x%02x", dom->id, mon_info.mon_config);
+		sep = true;
+	}
+	seq_puts(s, "\n");
+
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+
+	return 0;
+}
+
+static int mbm_total_bytes_config_show(struct kernfs_open_file *of,
+				       struct seq_file *seq, void *v)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+
+	mbm_config_show(seq, r, QOS_L3_MBM_TOTAL_EVENT_ID);
+
+	return 0;
+}
+
+static int mbm_local_bytes_config_show(struct kernfs_open_file *of,
+				       struct seq_file *seq, void *v)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+
+	mbm_config_show(seq, r, QOS_L3_MBM_LOCAL_EVENT_ID);
+
+	return 0;
+}
+
+static int mbm_config_write_domain(struct rdt_resource *r,
+				   struct rdt_domain *d, u32 evtid, u32 val)
+{
+	struct resctrl_mon_config_info mon_info = {0};
+
+	/*
+	 * Read the current config value first. If both are the same then
+	 * no need to write it again.
+	 */
+	mon_info.r = r;
+	mon_info.d = d;
+	mon_info.evtid = evtid;
+	mondata_config_read(&mon_info);
+	if (mon_info.mon_config == val)
+		return 0;
+
+	mon_info.mon_config = val;
+
+	/*
+	 * Update MSR_IA32_EVT_CFG_BASE MSR on one of the CPUs in the
+	 * domain. The MSRs offset from MSR MSR_IA32_EVT_CFG_BASE
+	 * are scoped at the domain level. Writing any of these MSRs
+	 * on one CPU is observed by all the CPUs in the domain.
+	 */
+	smp_call_function_any(&d->cpu_mask, resctrl_arch_mon_event_config_write,
+			      &mon_info, 1);
+	if (mon_info.err) {
+		rdt_last_cmd_puts("Invalid event configuration\n");
+		return mon_info.err;
+	}
+
+	/*
+	 * When an Event Configuration is changed, the bandwidth counters
+	 * for all RMIDs and Events will be cleared by the hardware. The
+	 * hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for
+	 * every RMID on the next read to any event for every RMID.
+	 * Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62)
+	 * cleared while it is tracked by the hardware. Clear the
+	 * mbm_local and mbm_total counts for all the RMIDs.
+	 */
+	resctrl_arch_reset_rmid_all(r, d);
+
+	return 0;
+}
+
+static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
+{
+	char *dom_str = NULL, *id_str;
+	unsigned long dom_id, val;
+	struct rdt_domain *d;
+	int err;
+
+	/* Walking r->domains, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+next:
+	if (!tok || tok[0] == '\0')
+		return 0;
+
+	/* Start processing the strings for each domain */
+	dom_str = strim(strsep(&tok, ";"));
+	id_str = strsep(&dom_str, "=");
+
+	if (!id_str || kstrtoul(id_str, 10, &dom_id)) {
+		rdt_last_cmd_puts("Missing '=' or non-numeric domain id\n");
+		return -EINVAL;
+	}
+
+	if (!dom_str || kstrtoul(dom_str, 16, &val)) {
+		rdt_last_cmd_puts("Non-numeric event configuration value\n");
+		return -EINVAL;
+	}
+
+	/* Value from user cannot be more than the supported set of events */
+	if ((val & r->mbm_cfg_mask) != val) {
+		rdt_last_cmd_printf("Invalid event configuration: max valid mask is 0x%02x\n",
+				    r->mbm_cfg_mask);
+		return -EINVAL;
+	}
+
+	list_for_each_entry(d, &r->domains, list) {
+		if (d->id == dom_id) {
+			err = mbm_config_write_domain(r, d, evtid, val);
+			if (err)
+				return err;
+			goto next;
+		}
+	}
+
+	return -EINVAL;
+}
+
+static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of,
+					    char *buf, size_t nbytes,
+					    loff_t off)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+	int ret;
+
+	/* Valid input requires a trailing newline */
+	if (nbytes == 0 || buf[nbytes - 1] != '\n')
+		return -EINVAL;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	rdt_last_cmd_clear();
+
+	buf[nbytes - 1] = '\0';
+
+	ret = mon_config_write(r, buf, QOS_L3_MBM_TOTAL_EVENT_ID);
+
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+
+	return ret ?: nbytes;
+}
+
+static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of,
+					    char *buf, size_t nbytes,
+					    loff_t off)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+	int ret;
+
+	/* Valid input requires a trailing newline */
+	if (nbytes == 0 || buf[nbytes - 1] != '\n')
+		return -EINVAL;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	rdt_last_cmd_clear();
+
+	buf[nbytes - 1] = '\0';
+
+	ret = mon_config_write(r, buf, QOS_L3_MBM_LOCAL_EVENT_ID);
+
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+
+	return ret ?: nbytes;
+}
+
+/* rdtgroup information files for one cache resource. */
+static struct rftype res_common_files[] = {
+	{
+		.name		= "last_cmd_status",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_last_cmd_status_show,
+		.fflags		= RFTYPE_TOP_INFO,
+	},
+	{
+		.name		= "num_closids",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_num_closids_show,
+		.fflags		= RFTYPE_CTRL_INFO,
+	},
+	{
+		.name		= "mon_features",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_mon_features_show,
+		.fflags		= RFTYPE_MON_INFO,
+	},
+	{
+		.name		= "num_rmids",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_num_rmids_show,
+		.fflags		= RFTYPE_MON_INFO,
+	},
+	{
+		.name		= "cbm_mask",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_default_ctrl_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
+	},
+	{
+		.name		= "min_cbm_bits",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_min_cbm_bits_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
+	},
+	{
+		.name		= "shareable_bits",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_shareable_bits_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
+	},
+	{
+		.name		= "bit_usage",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_bit_usage_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
+	},
+	{
+		.name		= "min_bandwidth",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_min_bw_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
+	},
+	{
+		.name		= "bandwidth_gran",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_bw_gran_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
+	},
+	{
+		.name		= "delay_linear",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_delay_linear_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
+	},
+	/*
+	 * Platform specific which (if any) capabilities are provided by
+	 * thread_throttle_mode. Defer "fflags" initialization to platform
+	 * discovery.
+	 */
+	{
+		.name		= "thread_throttle_mode",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_thread_throttle_mode_show,
+	},
+	{
+		.name		= "max_threshold_occupancy",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= max_threshold_occ_write,
+		.seq_show	= max_threshold_occ_show,
+		.fflags		= RFTYPE_MON_INFO | RFTYPE_RES_CACHE,
+	},
+	{
+		.name		= "mbm_total_bytes_config",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= mbm_total_bytes_config_show,
+		.write		= mbm_total_bytes_config_write,
+	},
+	{
+		.name		= "mbm_local_bytes_config",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= mbm_local_bytes_config_show,
+		.write		= mbm_local_bytes_config_write,
+	},
+	{
+		.name		= "cpus",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_cpus_write,
+		.seq_show	= rdtgroup_cpus_show,
+		.fflags		= RFTYPE_BASE,
+	},
+	{
+		.name		= "cpus_list",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_cpus_write,
+		.seq_show	= rdtgroup_cpus_show,
+		.flags		= RFTYPE_FLAGS_CPUS_LIST,
+		.fflags		= RFTYPE_BASE,
+	},
+	{
+		.name		= "tasks",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_tasks_write,
+		.seq_show	= rdtgroup_tasks_show,
+		.fflags		= RFTYPE_BASE,
+	},
+	{
+		.name		= "mon_hw_id",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_rmid_show,
+		.fflags		= RFTYPE_MON_BASE | RFTYPE_DEBUG,
+	},
+	{
+		.name		= "schemata",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_schemata_write,
+		.seq_show	= rdtgroup_schemata_show,
+		.fflags		= RFTYPE_CTRL_BASE,
+	},
+	{
+		.name		= "mode",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_mode_write,
+		.seq_show	= rdtgroup_mode_show,
+		.fflags		= RFTYPE_CTRL_BASE,
+	},
+	{
+		.name		= "size",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_size_show,
+		.fflags		= RFTYPE_CTRL_BASE,
+	},
+	{
+		.name		= "sparse_masks",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_has_sparse_bitmasks_show,
+		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
+	},
+	{
+		.name		= "ctrl_hw_id",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_closid_show,
+		.fflags		= RFTYPE_CTRL_BASE | RFTYPE_DEBUG,
+	},
+
+};
+
+static int rdtgroup_add_files(struct kernfs_node *kn, unsigned long fflags)
+{
+	struct rftype *rfts, *rft;
+	int ret, len;
+
+	rfts = res_common_files;
+	len = ARRAY_SIZE(res_common_files);
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	if (resctrl_debug)
+		fflags |= RFTYPE_DEBUG;
+
+	for (rft = rfts; rft < rfts + len; rft++) {
+		if (rft->fflags && ((fflags & rft->fflags) == rft->fflags)) {
+			ret = rdtgroup_add_file(kn, rft);
+			if (ret)
+				goto error;
+		}
+	}
+
+	return 0;
+error:
+	pr_warn("Failed to add %s, err=%d\n", rft->name, ret);
+	while (--rft >= rfts) {
+		if ((fflags & rft->fflags) == rft->fflags)
+			kernfs_remove_by_name(kn, rft->name);
+	}
+	return ret;
+}
+
+static struct rftype *rdtgroup_get_rftype_by_name(const char *name)
+{
+	struct rftype *rfts, *rft;
+	int len;
+
+	rfts = res_common_files;
+	len = ARRAY_SIZE(res_common_files);
+
+	for (rft = rfts; rft < rfts + len; rft++) {
+		if (!strcmp(rft->name, name))
+			return rft;
+	}
+
+	return NULL;
+}
+
+static void thread_throttle_mode_init(void)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
+	struct rftype *rft;
+
+	if (!r->alloc_capable ||
+	    r->membw.throttle_mode == THREAD_THROTTLE_UNDEFINED)
+		return;
+
+	rft = rdtgroup_get_rftype_by_name("thread_throttle_mode");
+	if (!rft)
+		return;
+
+	rft->fflags = RFTYPE_CTRL_INFO | RFTYPE_RES_MB;
+}
+
+void mbm_config_rftype_init(const char *config)
+{
+	struct rftype *rft;
+
+	rft = rdtgroup_get_rftype_by_name(config);
+	if (rft)
+		rft->fflags = RFTYPE_MON_INFO | RFTYPE_RES_CACHE;
+}
+
+/**
+ * rdtgroup_kn_mode_restrict - Restrict user access to named resctrl file
+ * @r: The resource group with which the file is associated.
+ * @name: Name of the file
+ *
+ * The permissions of named resctrl file, directory, or link are modified
+ * to not allow read, write, or execute by any user.
+ *
+ * WARNING: This function is intended to communicate to the user that the
+ * resctrl file has been locked down - that it is not relevant to the
+ * particular state the system finds itself in. It should not be relied
+ * on to protect from user access because after the file's permissions
+ * are restricted the user can still change the permissions using chmod
+ * from the command line.
+ *
+ * Return: 0 on success, <0 on failure.
+ */
+int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name)
+{
+	struct iattr iattr = {.ia_valid = ATTR_MODE,};
+	struct kernfs_node *kn;
+	int ret = 0;
+
+	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
+	if (!kn)
+		return -ENOENT;
+
+	switch (kernfs_type(kn)) {
+	case KERNFS_DIR:
+		iattr.ia_mode = S_IFDIR;
+		break;
+	case KERNFS_FILE:
+		iattr.ia_mode = S_IFREG;
+		break;
+	case KERNFS_LINK:
+		iattr.ia_mode = S_IFLNK;
+		break;
+	}
+
+	ret = kernfs_setattr(kn, &iattr);
+	kernfs_put(kn);
+	return ret;
+}
+
+/**
+ * rdtgroup_kn_mode_restore - Restore user access to named resctrl file
+ * @r: The resource group with which the file is associated.
+ * @name: Name of the file
+ * @mask: Mask of permissions that should be restored
+ *
+ * Restore the permissions of the named file. If @name is a directory the
+ * permissions of its parent will be used.
+ *
+ * Return: 0 on success, <0 on failure.
+ */
+int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
+			     umode_t mask)
+{
+	struct iattr iattr = {.ia_valid = ATTR_MODE,};
+	struct kernfs_node *kn, *parent;
+	struct rftype *rfts, *rft;
+	int ret, len;
+
+	rfts = res_common_files;
+	len = ARRAY_SIZE(res_common_files);
+
+	for (rft = rfts; rft < rfts + len; rft++) {
+		if (!strcmp(rft->name, name))
+			iattr.ia_mode = rft->mode & mask;
+	}
+
+	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
+	if (!kn)
+		return -ENOENT;
+
+	switch (kernfs_type(kn)) {
+	case KERNFS_DIR:
+		parent = kernfs_get_parent(kn);
+		if (parent) {
+			iattr.ia_mode |= parent->mode;
+			kernfs_put(parent);
+		}
+		iattr.ia_mode |= S_IFDIR;
+		break;
+	case KERNFS_FILE:
+		iattr.ia_mode |= S_IFREG;
+		break;
+	case KERNFS_LINK:
+		iattr.ia_mode |= S_IFLNK;
+		break;
+	}
+
+	ret = kernfs_setattr(kn, &iattr);
+	kernfs_put(kn);
+	return ret;
+}
+
+static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
+				      unsigned long fflags)
+{
+	struct kernfs_node *kn_subdir;
+	int ret;
+
+	kn_subdir = kernfs_create_dir(kn_info, name,
+				      kn_info->mode, priv);
+	if (IS_ERR(kn_subdir))
+		return PTR_ERR(kn_subdir);
+
+	ret = rdtgroup_kn_set_ugid(kn_subdir);
+	if (ret)
+		return ret;
+
+	ret = rdtgroup_add_files(kn_subdir, fflags);
+	if (!ret)
+		kernfs_activate(kn_subdir);
+
+	return ret;
+}
+
+static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
+{
+	enum resctrl_res_level i;
+	struct resctrl_schema *s;
+	struct rdt_resource *r;
+	unsigned long fflags;
+	char name[32];
+	int ret;
+
+	/* create the directory */
+	kn_info = kernfs_create_dir(parent_kn, "info", parent_kn->mode, NULL);
+	if (IS_ERR(kn_info))
+		return PTR_ERR(kn_info);
+
+	ret = rdtgroup_add_files(kn_info, RFTYPE_TOP_INFO);
+	if (ret)
+		goto out_destroy;
+
+	/* loop over enabled controls, these are all alloc_capable */
+	list_for_each_entry(s, &resctrl_schema_all, list) {
+		r = s->res;
+		fflags = r->fflags | RFTYPE_CTRL_INFO;
+		ret = rdtgroup_mkdir_info_resdir(s, s->name, fflags);
+		if (ret)
+			goto out_destroy;
+	}
+
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->mon_capable)
+			continue;
+
+		fflags =  r->fflags | RFTYPE_MON_INFO;
+		sprintf(name, "%s_MON", r->name);
+		ret = rdtgroup_mkdir_info_resdir(r, name, fflags);
+		if (ret)
+			goto out_destroy;
+	}
+
+	ret = rdtgroup_kn_set_ugid(kn_info);
+	if (ret)
+		goto out_destroy;
+
+	kernfs_activate(kn_info);
+
+	return 0;
+
+out_destroy:
+	kernfs_remove(kn_info);
+	return ret;
+}
+
+static int
+mongroup_create_dir(struct kernfs_node *parent_kn, struct rdtgroup *prgrp,
+		    char *name, struct kernfs_node **dest_kn)
+{
+	struct kernfs_node *kn;
+	int ret;
+
+	/* create the directory */
+	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
+	if (IS_ERR(kn))
+		return PTR_ERR(kn);
+
+	if (dest_kn)
+		*dest_kn = kn;
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret)
+		goto out_destroy;
+
+	kernfs_activate(kn);
+
+	return 0;
+
+out_destroy:
+	kernfs_remove(kn);
+	return ret;
+}
+
+static inline bool is_mba_linear(void)
+{
+	return resctrl_arch_get_resource(RDT_RESOURCE_MBA)->membw.delay_linear;
+}
+
+static int mba_sc_domain_allocate(struct rdt_resource *r, struct rdt_domain *d)
+{
+	u32 num_closid = resctrl_arch_get_num_closid(r);
+	int cpu = cpumask_any(&d->cpu_mask);
+	int i;
+
+	d->mbps_val = kcalloc_node(num_closid, sizeof(*d->mbps_val),
+				   GFP_KERNEL, cpu_to_node(cpu));
+	if (!d->mbps_val)
+		return -ENOMEM;
+
+	for (i = 0; i < num_closid; i++)
+		d->mbps_val[i] = MBA_MAX_MBPS;
+
+	return 0;
+}
+
+static void mba_sc_domain_destroy(struct rdt_resource *r,
+				  struct rdt_domain *d)
+{
+	kfree(d->mbps_val);
+	d->mbps_val = NULL;
+}
+
+/*
+ * MBA software controller is supported only if
+ * MBM is supported and MBA is in linear scale.
+ */
+static bool supports_mba_mbps(void)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
+
+	return (resctrl_arch_is_mbm_local_enabled() &&
+		r->alloc_capable && is_mba_linear());
+}
+
+/*
+ * Enable or disable the MBA software controller
+ * which helps user specify bandwidth in MBps.
+ */
+static int set_mba_sc(bool mba_sc)
+{
+	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
+	u32 num_closid = resctrl_arch_get_num_closid(r);
+	struct rdt_domain *d;
+	int i;
+
+	if (!supports_mba_mbps() || mba_sc == is_mba_sc(r))
+		return -EINVAL;
+
+	r->membw.mba_sc = mba_sc;
+
+	list_for_each_entry(d, &r->domains, list) {
+		for (i = 0; i < num_closid; i++)
+			d->mbps_val[i] = MBA_MAX_MBPS;
+	}
+
+	return 0;
+}
+
+/*
+ * We don't allow rdtgroup directories to be created anywhere
+ * except the root directory. Thus when looking for the rdtgroup
+ * structure for a kernfs node we are either looking at a directory,
+ * in which case the rdtgroup structure is pointed at by the "priv"
+ * field, otherwise we have a file, and need only look to the parent
+ * to find the rdtgroup.
+ */
+static struct rdtgroup *kernfs_to_rdtgroup(struct kernfs_node *kn)
+{
+	if (kernfs_type(kn) == KERNFS_DIR) {
+		/*
+		 * All the resource directories use "kn->priv"
+		 * to point to the "struct rdtgroup" for the
+		 * resource. "info" and its subdirectories don't
+		 * have rdtgroup structures, so return NULL here.
+		 */
+		if (kn == kn_info || kn->parent == kn_info)
+			return NULL;
+		else
+			return kn->priv;
+	} else {
+		return kn->parent->priv;
+	}
+}
+
+static void rdtgroup_kn_get(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
+{
+	atomic_inc(&rdtgrp->waitcount);
+	kernfs_break_active_protection(kn);
+}
+
+static void rdtgroup_kn_put(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
+{
+	if (atomic_dec_and_test(&rdtgrp->waitcount) &&
+	    (rdtgrp->flags & RDT_DELETED)) {
+		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
+		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
+			rdtgroup_pseudo_lock_remove(rdtgrp);
+		kernfs_unbreak_active_protection(kn);
+		rdtgroup_remove(rdtgrp);
+	} else {
+		kernfs_unbreak_active_protection(kn);
+	}
+}
+
+struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn)
+{
+	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
+
+	if (!rdtgrp)
+		return NULL;
+
+	rdtgroup_kn_get(rdtgrp, kn);
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	/* Was this group deleted while we waited? */
+	if (rdtgrp->flags & RDT_DELETED)
+		return NULL;
+
+	return rdtgrp;
+}
+
+void rdtgroup_kn_unlock(struct kernfs_node *kn)
+{
+	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
+
+	if (!rdtgrp)
+		return;
+
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+
+	rdtgroup_kn_put(rdtgrp, kn);
+}
+
+static int mkdir_mondata_all(struct kernfs_node *parent_kn,
+			     struct rdtgroup *prgrp,
+			     struct kernfs_node **mon_data_kn);
+
+static void rdt_disable_ctx(void)
+{
+	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
+	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
+	set_mba_sc(false);
+
+	resctrl_debug = false;
+}
+
+static int rdt_enable_ctx(struct rdt_fs_context *ctx)
+{
+	int ret = 0;
+
+	if (ctx->enable_cdpl2) {
+		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, true);
+		if (ret)
+			goto out_done;
+	}
+
+	if (ctx->enable_cdpl3) {
+		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, true);
+		if (ret)
+			goto out_cdpl2;
+	}
+
+	if (ctx->enable_mba_mbps) {
+		ret = set_mba_sc(true);
+		if (ret)
+			goto out_cdpl3;
+	}
+
+	if (ctx->enable_debug)
+		resctrl_debug = true;
+
+	return 0;
+
+out_cdpl3:
+	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
+out_cdpl2:
+	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
+out_done:
+	return ret;
+}
+
+static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type)
+{
+	struct resctrl_schema *s;
+	const char *suffix = "";
+	int ret, cl;
+
+	s = kzalloc(sizeof(*s), GFP_KERNEL);
+	if (!s)
+		return -ENOMEM;
+
+	s->res = r;
+	s->num_closid = resctrl_arch_get_num_closid(r);
+	if (resctrl_arch_get_cdp_enabled(r->rid))
+		s->num_closid /= 2;
+
+	s->conf_type = type;
+	switch (type) {
+	case CDP_CODE:
+		suffix = "CODE";
+		break;
+	case CDP_DATA:
+		suffix = "DATA";
+		break;
+	case CDP_NONE:
+		suffix = "";
+		break;
+	}
+
+	ret = snprintf(s->name, sizeof(s->name), "%s%s", r->name, suffix);
+	if (ret >= sizeof(s->name)) {
+		kfree(s);
+		return -EINVAL;
+	}
+
+	cl = strlen(s->name);
+
+	/*
+	 * If CDP is supported by this resource, but not enabled,
+	 * include the suffix. This ensures the tabular format of the
+	 * schemata file does not change between mounts of the filesystem.
+	 */
+	if (r->cdp_capable && !resctrl_arch_get_cdp_enabled(r->rid))
+		cl += 4;
+
+	if (cl > max_name_width)
+		max_name_width = cl;
+
+	/*
+	 * Choose a width for the resource data based on the resource that has
+	 * widest name and cbm.
+	 */
+	max_data_width = max(max_data_width, r->data_width);
+
+	INIT_LIST_HEAD(&s->list);
+	list_add(&s->list, &resctrl_schema_all);
+
+	return 0;
+}
+
+static int schemata_list_create(void)
+{
+	enum resctrl_res_level i;
+	struct rdt_resource *r;
+	int ret = 0;
+
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->alloc_capable)
+			continue;
+
+		if (resctrl_arch_get_cdp_enabled(r->rid)) {
+			ret = schemata_list_add(r, CDP_CODE);
+			if (ret)
+				break;
+
+			ret = schemata_list_add(r, CDP_DATA);
+		} else {
+			ret = schemata_list_add(r, CDP_NONE);
+		}
+
+		if (ret)
+			break;
+	}
+
+	return ret;
+}
+
+static void schemata_list_destroy(void)
+{
+	struct resctrl_schema *s, *tmp;
+
+	list_for_each_entry_safe(s, tmp, &resctrl_schema_all, list) {
+		list_del(&s->list);
+		kfree(s);
+	}
+}
+
+static int rdt_get_tree(struct fs_context *fc)
+{
+	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+	struct rdt_fs_context *ctx = rdt_fc2context(fc);
+	unsigned long flags = RFTYPE_CTRL_BASE;
+	struct rdt_domain *dom;
+	int ret;
+
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+	/*
+	 * resctrl file system can only be mounted once.
+	 */
+	if (resctrl_mounted) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	ret = rdtgroup_setup_root(ctx);
+	if (ret)
+		goto out;
+
+	ret = rdt_enable_ctx(ctx);
+	if (ret)
+		goto out_root;
+
+	ret = schemata_list_create();
+	if (ret) {
+		schemata_list_destroy();
+		goto out_ctx;
+	}
+
+	closid_init();
+
+	if (resctrl_arch_mon_capable())
+		flags |= RFTYPE_MON;
+
+	ret = rdtgroup_add_files(rdtgroup_default.kn, flags);
+	if (ret)
+		goto out_schemata_free;
+
+	kernfs_activate(rdtgroup_default.kn);
+
+	ret = rdtgroup_create_info_dir(rdtgroup_default.kn);
+	if (ret < 0)
+		goto out_schemata_free;
+
+	if (resctrl_arch_mon_capable()) {
+		ret = mongroup_create_dir(rdtgroup_default.kn,
+					  &rdtgroup_default, "mon_groups",
+					  &kn_mongrp);
+		if (ret < 0)
+			goto out_info;
+
+		ret = mkdir_mondata_all(rdtgroup_default.kn,
+					&rdtgroup_default, &kn_mondata);
+		if (ret < 0)
+			goto out_mongrp;
+		rdtgroup_default.mon.mon_data_kn = kn_mondata;
+	}
+
+	ret = rdt_pseudo_lock_init();
+	if (ret)
+		goto out_mondata;
+
+	ret = kernfs_get_tree(fc);
+	if (ret < 0)
+		goto out_psl;
+
+	if (resctrl_arch_alloc_capable())
+		resctrl_arch_enable_alloc();
+	if (resctrl_arch_mon_capable())
+		resctrl_arch_enable_mon();
+
+	if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable())
+		resctrl_mounted = true;
+
+	if (resctrl_is_mbm_enabled()) {
+		list_for_each_entry(dom, &l3->domains, list)
+			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
+						   RESCTRL_PICK_ANY_CPU);
+	}
+
+	goto out;
+
+out_psl:
+	rdt_pseudo_lock_release();
+out_mondata:
+	if (resctrl_arch_mon_capable())
+		kernfs_remove(kn_mondata);
+out_mongrp:
+	if (resctrl_arch_mon_capable())
+		kernfs_remove(kn_mongrp);
+out_info:
+	kernfs_remove(kn_info);
+out_schemata_free:
+	schemata_list_destroy();
+out_ctx:
+	rdt_disable_ctx();
+out_root:
+	rdtgroup_destroy_root();
+out:
+	rdt_last_cmd_clear();
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+	return ret;
+}
+
+enum rdt_param {
+	Opt_cdp,
+	Opt_cdpl2,
+	Opt_mba_mbps,
+	Opt_debug,
+	nr__rdt_params
+};
+
+static const struct fs_parameter_spec rdt_fs_parameters[] = {
+	fsparam_flag("cdp",		Opt_cdp),
+	fsparam_flag("cdpl2",		Opt_cdpl2),
+	fsparam_flag("mba_MBps",	Opt_mba_mbps),
+	fsparam_flag("debug",		Opt_debug),
+	{}
+};
+
+static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
+{
+	struct rdt_fs_context *ctx = rdt_fc2context(fc);
+	struct fs_parse_result result;
+	int opt;
+
+	opt = fs_parse(fc, rdt_fs_parameters, param, &result);
+	if (opt < 0)
+		return opt;
+
+	switch (opt) {
+	case Opt_cdp:
+		ctx->enable_cdpl3 = true;
+		return 0;
+	case Opt_cdpl2:
+		ctx->enable_cdpl2 = true;
+		return 0;
+	case Opt_mba_mbps:
+		if (!supports_mba_mbps())
+			return -EINVAL;
+		ctx->enable_mba_mbps = true;
+		return 0;
+	case Opt_debug:
+		ctx->enable_debug = true;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static void rdt_fs_context_free(struct fs_context *fc)
+{
+	struct rdt_fs_context *ctx = rdt_fc2context(fc);
+
+	kernfs_free_fs_context(fc);
+	kfree(ctx);
+}
+
+static const struct fs_context_operations rdt_fs_context_ops = {
+	.free		= rdt_fs_context_free,
+	.parse_param	= rdt_parse_param,
+	.get_tree	= rdt_get_tree,
+};
+
+static int rdt_init_fs_context(struct fs_context *fc)
+{
+	struct rdt_fs_context *ctx;
+
+	ctx = kzalloc(sizeof(struct rdt_fs_context), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	ctx->kfc.magic = RDTGROUP_SUPER_MAGIC;
+	fc->fs_private = &ctx->kfc;
+	fc->ops = &rdt_fs_context_ops;
+	put_user_ns(fc->user_ns);
+	fc->user_ns = get_user_ns(&init_user_ns);
+	fc->global = true;
+	return 0;
+}
+
+/*
+ * Move tasks from one to the other group. If @from is NULL, then all tasks
+ * in the systems are moved unconditionally (used for teardown).
+ *
+ * If @mask is not NULL the cpus on which moved tasks are running are set
+ * in that mask so the update smp function call is restricted to affected
+ * cpus.
+ */
+static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,
+				 struct cpumask *mask)
+{
+	struct task_struct *p, *t;
+
+	read_lock(&tasklist_lock);
+	for_each_process_thread(p, t) {
+		if (!from || is_closid_match(t, from) ||
+		    is_rmid_match(t, from)) {
+			resctrl_arch_set_closid_rmid(t, to->closid,
+						     to->mon.rmid);
+
+			/*
+			 * Order the closid/rmid stores above before the loads
+			 * in task_curr(). This pairs with the full barrier
+			 * between the rq->curr update and
+			 * resctrl_arch_sched_in() during context switch.
+			 */
+			smp_mb();
+
+			/*
+			 * If the task is on a CPU, set the CPU in the mask.
+			 * The detection is inaccurate as tasks might move or
+			 * schedule before the smp function call takes place.
+			 * In such a case the function call is pointless, but
+			 * there is no other side effect.
+			 */
+			if (IS_ENABLED(CONFIG_SMP) && mask && task_curr(t))
+				cpumask_set_cpu(task_cpu(t), mask);
+		}
+	}
+	read_unlock(&tasklist_lock);
+}
+
+static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp)
+{
+	struct rdtgroup *sentry, *stmp;
+	struct list_head *head;
+
+	head = &rdtgrp->mon.crdtgrp_list;
+	list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) {
+		free_rmid(sentry->closid, sentry->mon.rmid);
+		list_del(&sentry->mon.crdtgrp_list);
+
+		if (atomic_read(&sentry->waitcount) != 0)
+			sentry->flags = RDT_DELETED;
+		else
+			rdtgroup_remove(sentry);
+	}
+}
+
+/*
+ * Forcibly remove all of subdirectories under root.
+ */
+static void rmdir_all_sub(void)
+{
+	struct rdtgroup *rdtgrp, *tmp;
+
+	/* Move all tasks to the default resource group */
+	rdt_move_group_tasks(NULL, &rdtgroup_default, NULL);
+
+	list_for_each_entry_safe(rdtgrp, tmp, &rdt_all_groups, rdtgroup_list) {
+		/* Free any child rmids */
+		free_all_child_rdtgrp(rdtgrp);
+
+		/* Remove each rdtgroup other than root */
+		if (rdtgrp == &rdtgroup_default)
+			continue;
+
+		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
+		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
+			rdtgroup_pseudo_lock_remove(rdtgrp);
+
+		/*
+		 * Give any CPUs back to the default group. We cannot copy
+		 * cpu_online_mask because a CPU might have executed the
+		 * offline callback already, but is still marked online.
+		 */
+		cpumask_or(&rdtgroup_default.cpu_mask,
+			   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
+
+		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
+
+		kernfs_remove(rdtgrp->kn);
+		list_del(&rdtgrp->rdtgroup_list);
+
+		if (atomic_read(&rdtgrp->waitcount) != 0)
+			rdtgrp->flags = RDT_DELETED;
+		else
+			rdtgroup_remove(rdtgrp);
+	}
+	/* Notify online CPUs to update per cpu storage and PQR_ASSOC MSR */
+	update_closid_rmid(cpu_online_mask, &rdtgroup_default);
+
+	kernfs_remove(kn_info);
+	kernfs_remove(kn_mongrp);
+	kernfs_remove(kn_mondata);
+}
+
+static void rdt_kill_sb(struct super_block *sb)
+{
+	cpus_read_lock();
+	mutex_lock(&rdtgroup_mutex);
+
+	rdt_disable_ctx();
+
+	/* Put everything back to default values. */
+	resctrl_arch_reset_resources();
+
+	rmdir_all_sub();
+	rdt_pseudo_lock_release();
+	rdtgroup_default.mode = RDT_MODE_SHAREABLE;
+	schemata_list_destroy();
+	rdtgroup_destroy_root();
+	if (resctrl_arch_alloc_capable())
+		resctrl_arch_disable_alloc();
+	if (resctrl_arch_mon_capable())
+		resctrl_arch_disable_mon();
+	resctrl_mounted = false;
+	kernfs_kill_sb(sb);
+	mutex_unlock(&rdtgroup_mutex);
+	cpus_read_unlock();
+}
+
+static struct file_system_type rdt_fs_type = {
+	.name			= "resctrl",
+	.init_fs_context	= rdt_init_fs_context,
+	.parameters		= rdt_fs_parameters,
+	.kill_sb		= rdt_kill_sb,
+};
+
+static int mon_addfile(struct kernfs_node *parent_kn, const char *name,
+		       void *priv)
+{
+	struct kernfs_node *kn;
+	int ret = 0;
+
+	kn = __kernfs_create_file(parent_kn, name, 0444,
+				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, 0,
+				  &kf_mondata_ops, priv, NULL, NULL);
+	if (IS_ERR(kn))
+		return PTR_ERR(kn);
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret) {
+		kernfs_remove(kn);
+		return ret;
+	}
+
+	return ret;
+}
+
+/*
+ * Remove all subdirectories of mon_data of ctrl_mon groups
+ * and monitor groups with given domain id.
+ */
+static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
+					   unsigned int dom_id)
+{
+	struct rdtgroup *prgrp, *crgrp;
+	char name[32];
+
+	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
+		sprintf(name, "mon_%s_%02d", r->name, dom_id);
+		kernfs_remove_by_name(prgrp->mon.mon_data_kn, name);
+
+		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+			kernfs_remove_by_name(crgrp->mon.mon_data_kn, name);
+	}
+}
+
+static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,
+				struct rdt_domain *d,
+				struct rdt_resource *r, struct rdtgroup *prgrp)
+{
+	union mon_data_bits priv;
+	struct kernfs_node *kn;
+	struct mon_evt *mevt;
+	struct rmid_read rr;
+	char name[32];
+	int ret;
+
+	sprintf(name, "mon_%s_%02d", r->name, d->id);
+	/* create the directory */
+	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
+	if (IS_ERR(kn))
+		return PTR_ERR(kn);
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret)
+		goto out_destroy;
+
+	if (WARN_ON(list_empty(&r->evt_list))) {
+		ret = -EPERM;
+		goto out_destroy;
+	}
+
+	priv.u.rid = r->rid;
+	priv.u.domid = d->id;
+	list_for_each_entry(mevt, &r->evt_list, list) {
+		priv.u.evtid = mevt->evtid;
+		ret = mon_addfile(kn, mevt->name, priv.priv);
+		if (ret)
+			goto out_destroy;
+
+		if (resctrl_is_mbm_event(mevt->evtid))
+			mon_event_read(&rr, r, d, prgrp, mevt->evtid, true);
+	}
+	kernfs_activate(kn);
+	return 0;
+
+out_destroy:
+	kernfs_remove(kn);
+	return ret;
+}
+
+/*
+ * Add all subdirectories of mon_data for "ctrl_mon" groups
+ * and "monitor" groups with given domain id.
+ */
+static void mkdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
+					   struct rdt_domain *d)
+{
+	struct kernfs_node *parent_kn;
+	struct rdtgroup *prgrp, *crgrp;
+	struct list_head *head;
+
+	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
+		parent_kn = prgrp->mon.mon_data_kn;
+		mkdir_mondata_subdir(parent_kn, d, r, prgrp);
+
+		head = &prgrp->mon.crdtgrp_list;
+		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
+			parent_kn = crgrp->mon.mon_data_kn;
+			mkdir_mondata_subdir(parent_kn, d, r, crgrp);
+		}
+	}
+}
+
+static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn,
+				       struct rdt_resource *r,
+				       struct rdtgroup *prgrp)
+{
+	struct rdt_domain *dom;
+	int ret;
+
+	/* Walking r->domains, ensure it can't race with cpuhp */
+	lockdep_assert_cpus_held();
+
+	list_for_each_entry(dom, &r->domains, list) {
+		ret = mkdir_mondata_subdir(parent_kn, dom, r, prgrp);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+/*
+ * This creates a directory mon_data which contains the monitored data.
+ *
+ * mon_data has one directory for each domain which are named
+ * in the format mon_<domain_name>_<domain_id>. For ex: A mon_data
+ * with L3 domain looks as below:
+ * ./mon_data:
+ * mon_L3_00
+ * mon_L3_01
+ * mon_L3_02
+ * ...
+ *
+ * Each domain directory has one file per event:
+ * ./mon_L3_00/:
+ * llc_occupancy
+ *
+ */
+static int mkdir_mondata_all(struct kernfs_node *parent_kn,
+			     struct rdtgroup *prgrp,
+			     struct kernfs_node **dest_kn)
+{
+	enum resctrl_res_level i;
+	struct rdt_resource *r;
+	struct kernfs_node *kn;
+	int ret;
+
+	/*
+	 * Create the mon_data directory first.
+	 */
+	ret = mongroup_create_dir(parent_kn, prgrp, "mon_data", &kn);
+	if (ret)
+		return ret;
+
+	if (dest_kn)
+		*dest_kn = kn;
+
+	/*
+	 * Create the subdirectories for each domain. Note that all events
+	 * in a domain like L3 are grouped into a resource whose domain is L3
+	 */
+	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
+		r = resctrl_arch_get_resource(i);
+		if (!r->mon_capable)
+			continue;
+
+		ret = mkdir_mondata_subdir_alldom(kn, r, prgrp);
+		if (ret)
+			goto out_destroy;
+	}
+
+	return 0;
+
+out_destroy:
+	kernfs_remove(kn);
+	return ret;
+}
+
+/**
+ * cbm_ensure_valid - Enforce validity on provided CBM
+ * @_val:	Candidate CBM
+ * @r:		RDT resource to which the CBM belongs
+ *
+ * The provided CBM represents all cache portions available for use. This
+ * may be represented by a bitmap that does not consist of contiguous ones
+ * and thus be an invalid CBM.
+ * Here the provided CBM is forced to be a valid CBM by only considering
+ * the first set of contiguous bits as valid and clearing all bits.
+ * The intention here is to provide a valid default CBM with which a new
+ * resource group is initialized. The user can follow this with a
+ * modification to the CBM if the default does not satisfy the
+ * requirements.
+ */
+static u32 cbm_ensure_valid(u32 _val, struct rdt_resource *r)
+{
+	unsigned int cbm_len = r->cache.cbm_len;
+	unsigned long first_bit, zero_bit;
+	unsigned long val = _val;
+
+	if (!val)
+		return 0;
+
+	first_bit = find_first_bit(&val, cbm_len);
+	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
+
+	/* Clear any remaining bits to ensure contiguous region */
+	bitmap_clear(&val, zero_bit, cbm_len - zero_bit);
+	return (u32)val;
+}
+
+/*
+ * Initialize cache resources per RDT domain
+ *
+ * Set the RDT domain up to start off with all usable allocations. That is,
+ * all shareable and unused bits. All-zero CBM is invalid.
+ */
+static int __init_one_rdt_domain(struct rdt_domain *d, struct resctrl_schema *s,
+				 u32 closid)
+{
+	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
+	enum resctrl_conf_type t = s->conf_type;
+	struct resctrl_staged_config *cfg;
+	struct rdt_resource *r = s->res;
+	u32 used_b = 0, unused_b = 0;
+	unsigned long tmp_cbm;
+	enum rdtgrp_mode mode;
+	u32 peer_ctl, ctrl_val;
+	int i;
+
+	cfg = &d->staged_config[t];
+	cfg->have_new_ctrl = false;
+	cfg->new_ctrl = r->cache.shareable_bits;
+	used_b = r->cache.shareable_bits;
+	for (i = 0; i < closids_supported(); i++) {
+		if (closid_allocated(i) && i != closid) {
+			mode = rdtgroup_mode_by_closid(i);
+			if (mode == RDT_MODE_PSEUDO_LOCKSETUP)
+				/*
+				 * ctrl values for locksetup aren't relevant
+				 * until the schemata is written, and the mode
+				 * becomes RDT_MODE_PSEUDO_LOCKED.
+				 */
+				continue;
+			/*
+			 * If CDP is active include peer domain's
+			 * usage to ensure there is no overlap
+			 * with an exclusive group.
+			 */
+			if (resctrl_arch_get_cdp_enabled(r->rid))
+				peer_ctl = resctrl_arch_get_config(r, d, i,
+								   peer_type);
+			else
+				peer_ctl = 0;
+			ctrl_val = resctrl_arch_get_config(r, d, i,
+							   s->conf_type);
+			used_b |= ctrl_val | peer_ctl;
+			if (mode == RDT_MODE_SHAREABLE)
+				cfg->new_ctrl |= ctrl_val | peer_ctl;
+		}
+	}
+	if (d->plr && d->plr->cbm > 0)
+		used_b |= d->plr->cbm;
+	unused_b = used_b ^ (BIT_MASK(r->cache.cbm_len) - 1);
+	unused_b &= BIT_MASK(r->cache.cbm_len) - 1;
+	cfg->new_ctrl |= unused_b;
+	/*
+	 * Force the initial CBM to be valid, user can
+	 * modify the CBM based on system availability.
+	 */
+	cfg->new_ctrl = cbm_ensure_valid(cfg->new_ctrl, r);
+	/*
+	 * Assign the u32 CBM to an unsigned long to ensure that
+	 * bitmap_weight() does not access out-of-bound memory.
+	 */
+	tmp_cbm = cfg->new_ctrl;
+	if (bitmap_weight(&tmp_cbm, r->cache.cbm_len) < r->cache.min_cbm_bits) {
+		rdt_last_cmd_printf("No space on %s:%d\n", s->name, d->id);
+		return -ENOSPC;
+	}
+	cfg->have_new_ctrl = true;
+
+	return 0;
+}
+
+/*
+ * Initialize cache resources with default values.
+ *
+ * A new RDT group is being created on an allocation capable (CAT)
+ * supporting system. Set this group up to start off with all usable
+ * allocations.
+ *
+ * If there are no more shareable bits available on any domain then
+ * the entire allocation will fail.
+ */
+static int rdtgroup_init_cat(struct resctrl_schema *s, u32 closid)
+{
+	struct rdt_domain *d;
+	int ret;
+
+	list_for_each_entry(d, &s->res->domains, list) {
+		ret = __init_one_rdt_domain(d, s, closid);
+		if (ret < 0)
+			return ret;
+	}
+
+	return 0;
+}
+
+/* Initialize MBA resource with default values. */
+static void rdtgroup_init_mba(struct rdt_resource *r, u32 closid)
+{
+	struct resctrl_staged_config *cfg;
+	struct rdt_domain *d;
+
+	list_for_each_entry(d, &r->domains, list) {
+		if (is_mba_sc(r)) {
+			d->mbps_val[closid] = MBA_MAX_MBPS;
+			continue;
+		}
+
+		cfg = &d->staged_config[CDP_NONE];
+		cfg->new_ctrl = r->default_ctrl;
+		cfg->have_new_ctrl = true;
+	}
+}
+
+/* Initialize the RDT group's allocations. */
+static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
+{
+	struct resctrl_schema *s;
+	struct rdt_resource *r;
+	int ret = 0;
+
+	rdt_staged_configs_clear();
+
+	list_for_each_entry(s, &resctrl_schema_all, list) {
+		r = s->res;
+		if (r->rid == RDT_RESOURCE_MBA ||
+		    r->rid == RDT_RESOURCE_SMBA) {
+			rdtgroup_init_mba(r, rdtgrp->closid);
+			if (is_mba_sc(r))
+				continue;
+		} else {
+			ret = rdtgroup_init_cat(s, rdtgrp->closid);
+			if (ret < 0)
+				goto out;
+		}
+
+		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
+		if (ret < 0) {
+			rdt_last_cmd_puts("Failed to initialize allocations\n");
+			goto out;
+		}
+
+	}
+
+	rdtgrp->mode = RDT_MODE_SHAREABLE;
+
+out:
+	rdt_staged_configs_clear();
+	return ret;
+}
+
+static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)
+{
+	int ret;
+
+	if (!resctrl_arch_mon_capable())
+		return 0;
+
+	ret = alloc_rmid(rdtgrp->closid);
+	if (ret < 0) {
+		rdt_last_cmd_puts("Out of RMIDs\n");
+		return ret;
+	}
+	rdtgrp->mon.rmid = ret;
+
+	ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn);
+	if (ret) {
+		rdt_last_cmd_puts("kernfs subdir error\n");
+		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp)
+{
+	if (resctrl_arch_mon_capable())
+		free_rmid(rgrp->closid, rgrp->mon.rmid);
+}
+
+static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,
+			     const char *name, umode_t mode,
+			     enum rdt_group_type rtype, struct rdtgroup **r)
+{
+	struct rdtgroup *prdtgrp, *rdtgrp;
+	unsigned long files = 0;
+	struct kernfs_node *kn;
+	int ret;
+
+	prdtgrp = rdtgroup_kn_lock_live(parent_kn);
+	if (!prdtgrp) {
+		ret = -ENODEV;
+		goto out_unlock;
+	}
+
+	if (rtype == RDTMON_GROUP &&
+	    (prdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
+	     prdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)) {
+		ret = -EINVAL;
+		rdt_last_cmd_puts("Pseudo-locking in progress\n");
+		goto out_unlock;
+	}
+
+	/* allocate the rdtgroup. */
+	rdtgrp = kzalloc(sizeof(*rdtgrp), GFP_KERNEL);
+	if (!rdtgrp) {
+		ret = -ENOSPC;
+		rdt_last_cmd_puts("Kernel out of memory\n");
+		goto out_unlock;
+	}
+	*r = rdtgrp;
+	rdtgrp->mon.parent = prdtgrp;
+	rdtgrp->type = rtype;
+	INIT_LIST_HEAD(&rdtgrp->mon.crdtgrp_list);
+
+	/* kernfs creates the directory for rdtgrp */
+	kn = kernfs_create_dir(parent_kn, name, mode, rdtgrp);
+	if (IS_ERR(kn)) {
+		ret = PTR_ERR(kn);
+		rdt_last_cmd_puts("kernfs create error\n");
+		goto out_free_rgrp;
+	}
+	rdtgrp->kn = kn;
+
+	/*
+	 * kernfs_remove() will drop the reference count on "kn" which
+	 * will free it. But we still need it to stick around for the
+	 * rdtgroup_kn_unlock(kn) call. Take one extra reference here,
+	 * which will be dropped by kernfs_put() in rdtgroup_remove().
+	 */
+	kernfs_get(kn);
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret) {
+		rdt_last_cmd_puts("kernfs perm error\n");
+		goto out_destroy;
+	}
+
+	if (rtype == RDTCTRL_GROUP) {
+		files = RFTYPE_BASE | RFTYPE_CTRL;
+		if (resctrl_arch_mon_capable())
+			files |= RFTYPE_MON;
+	} else {
+		files = RFTYPE_BASE | RFTYPE_MON;
+	}
+
+	ret = rdtgroup_add_files(kn, files);
+	if (ret) {
+		rdt_last_cmd_puts("kernfs fill error\n");
+		goto out_destroy;
+	}
+
+	/*
+	 * The caller unlocks the parent_kn upon success.
+	 */
+	return 0;
+
+out_destroy:
+	kernfs_put(rdtgrp->kn);
+	kernfs_remove(rdtgrp->kn);
+out_free_rgrp:
+	kfree(rdtgrp);
+out_unlock:
+	rdtgroup_kn_unlock(parent_kn);
+	return ret;
+}
+
+static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp)
+{
+	kernfs_remove(rgrp->kn);
+	rdtgroup_remove(rgrp);
+}
+
+/*
+ * Create a monitor group under "mon_groups" directory of a control
+ * and monitor group(ctrl_mon). This is a resource group
+ * to monitor a subset of tasks and cpus in its parent ctrl_mon group.
+ */
+static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn,
+			      const char *name, umode_t mode)
+{
+	struct rdtgroup *rdtgrp, *prgrp;
+	int ret;
+
+	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTMON_GROUP, &rdtgrp);
+	if (ret)
+		return ret;
+
+	prgrp = rdtgrp->mon.parent;
+	rdtgrp->closid = prgrp->closid;
+
+	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
+	if (ret) {
+		mkdir_rdt_prepare_clean(rdtgrp);
+		goto out_unlock;
+	}
+
+	kernfs_activate(rdtgrp->kn);
+
+	/*
+	 * Add the rdtgrp to the list of rdtgrps the parent
+	 * ctrl_mon group has to track.
+	 */
+	list_add_tail(&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list);
+
+out_unlock:
+	rdtgroup_kn_unlock(parent_kn);
+	return ret;
+}
+
+/*
+ * These are rdtgroups created under the root directory. Can be used
+ * to allocate and monitor resources.
+ */
+static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,
+				   const char *name, umode_t mode)
+{
+	struct rdtgroup *rdtgrp;
+	struct kernfs_node *kn;
+	u32 closid;
+	int ret;
+
+	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTCTRL_GROUP, &rdtgrp);
+	if (ret)
+		return ret;
+
+	kn = rdtgrp->kn;
+	ret = closid_alloc();
+	if (ret < 0) {
+		rdt_last_cmd_puts("Out of CLOSIDs\n");
+		goto out_common_fail;
+	}
+	closid = ret;
+	ret = 0;
+
+	rdtgrp->closid = closid;
+
+	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
+	if (ret)
+		goto out_closid_free;
+
+	kernfs_activate(rdtgrp->kn);
+
+	ret = rdtgroup_init_alloc(rdtgrp);
+	if (ret < 0)
+		goto out_rmid_free;
+
+	list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups);
+
+	if (resctrl_arch_mon_capable()) {
+		/*
+		 * Create an empty mon_groups directory to hold the subset
+		 * of tasks and cpus to monitor.
+		 */
+		ret = mongroup_create_dir(kn, rdtgrp, "mon_groups", NULL);
+		if (ret) {
+			rdt_last_cmd_puts("kernfs subdir error\n");
+			goto out_del_list;
+		}
+	}
+
+	goto out_unlock;
+
+out_del_list:
+	list_del(&rdtgrp->rdtgroup_list);
+out_rmid_free:
+	mkdir_rdt_prepare_rmid_free(rdtgrp);
+out_closid_free:
+	closid_free(closid);
+out_common_fail:
+	mkdir_rdt_prepare_clean(rdtgrp);
+out_unlock:
+	rdtgroup_kn_unlock(parent_kn);
+	return ret;
+}
+
+/*
+ * We allow creating mon groups only with in a directory called "mon_groups"
+ * which is present in every ctrl_mon group. Check if this is a valid
+ * "mon_groups" directory.
+ *
+ * 1. The directory should be named "mon_groups".
+ * 2. The mon group itself should "not" be named "mon_groups".
+ *   This makes sure "mon_groups" directory always has a ctrl_mon group
+ *   as parent.
+ */
+static bool is_mon_groups(struct kernfs_node *kn, const char *name)
+{
+	return (!strcmp(kn->name, "mon_groups") &&
+		strcmp(name, "mon_groups"));
+}
+
+static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
+			  umode_t mode)
+{
+	/* Do not accept '\n' to avoid unparsable situation. */
+	if (strchr(name, '\n'))
+		return -EINVAL;
+
+	/*
+	 * If the parent directory is the root directory and RDT
+	 * allocation is supported, add a control and monitoring
+	 * subdirectory
+	 */
+	if (resctrl_arch_alloc_capable() && parent_kn == rdtgroup_default.kn)
+		return rdtgroup_mkdir_ctrl_mon(parent_kn, name, mode);
+
+	/*
+	 * If RDT monitoring is supported and the parent directory is a valid
+	 * "mon_groups" directory, add a monitoring subdirectory.
+	 */
+	if (resctrl_arch_mon_capable() && is_mon_groups(parent_kn, name))
+		return rdtgroup_mkdir_mon(parent_kn, name, mode);
+
+	return -EPERM;
+}
+
+static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
+{
+	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
+	u32 closid, rmid;
+	int cpu;
+
+	/* Give any tasks back to the parent group */
+	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
+
+	/* Update per cpu rmid of the moved CPUs first */
+	closid = rdtgrp->closid;
+	rmid = prdtgrp->mon.rmid;
+	for_each_cpu(cpu, &rdtgrp->cpu_mask)
+		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
+
+	/*
+	 * Update the MSR on moved CPUs and CPUs which have moved
+	 * task running on them.
+	 */
+	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
+	update_closid_rmid(tmpmask, NULL);
+
+	rdtgrp->flags = RDT_DELETED;
+	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
+
+	/*
+	 * Remove the rdtgrp from the parent ctrl_mon group's list
+	 */
+	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
+	list_del(&rdtgrp->mon.crdtgrp_list);
+
+	kernfs_remove(rdtgrp->kn);
+
+	return 0;
+}
+
+static int rdtgroup_ctrl_remove(struct rdtgroup *rdtgrp)
+{
+	rdtgrp->flags = RDT_DELETED;
+	list_del(&rdtgrp->rdtgroup_list);
+
+	kernfs_remove(rdtgrp->kn);
+	return 0;
+}
+
+static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
+{
+	u32 closid, rmid;
+	int cpu;
+
+	/* Give any tasks back to the default group */
+	rdt_move_group_tasks(rdtgrp, &rdtgroup_default, tmpmask);
+
+	/* Give any CPUs back to the default group */
+	cpumask_or(&rdtgroup_default.cpu_mask,
+		   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
+
+	/* Update per cpu closid and rmid of the moved CPUs first */
+	closid = rdtgroup_default.closid;
+	rmid = rdtgroup_default.mon.rmid;
+	for_each_cpu(cpu, &rdtgrp->cpu_mask)
+		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
+
+	/*
+	 * Update the MSR on moved CPUs and CPUs which have moved
+	 * task running on them.
+	 */
+	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
+	update_closid_rmid(tmpmask, NULL);
+
+	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
+	closid_free(rdtgrp->closid);
+
+	rdtgroup_ctrl_remove(rdtgrp);
+
+	/*
+	 * Free all the child monitor group rmids.
+	 */
+	free_all_child_rdtgrp(rdtgrp);
+
+	return 0;
+}
+
+static int rdtgroup_rmdir(struct kernfs_node *kn)
+{
+	struct kernfs_node *parent_kn = kn->parent;
+	struct rdtgroup *rdtgrp;
+	cpumask_var_t tmpmask;
+	int ret = 0;
+
+	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
+		return -ENOMEM;
+
+	rdtgrp = rdtgroup_kn_lock_live(kn);
+	if (!rdtgrp) {
+		ret = -EPERM;
+		goto out;
+	}
+
+	/*
+	 * If the rdtgroup is a ctrl_mon group and parent directory
+	 * is the root directory, remove the ctrl_mon group.
+	 *
+	 * If the rdtgroup is a mon group and parent directory
+	 * is a valid "mon_groups" directory, remove the mon group.
+	 */
+	if (rdtgrp->type == RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn &&
+	    rdtgrp != &rdtgroup_default) {
+		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
+		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
+			ret = rdtgroup_ctrl_remove(rdtgrp);
+		} else {
+			ret = rdtgroup_rmdir_ctrl(rdtgrp, tmpmask);
+		}
+	} else if (rdtgrp->type == RDTMON_GROUP &&
+		 is_mon_groups(parent_kn, kn->name)) {
+		ret = rdtgroup_rmdir_mon(rdtgrp, tmpmask);
+	} else {
+		ret = -EPERM;
+	}
+
+out:
+	rdtgroup_kn_unlock(kn);
+	free_cpumask_var(tmpmask);
+	return ret;
+}
+
+/**
+ * mongrp_reparent() - replace parent CTRL_MON group of a MON group
+ * @rdtgrp:		the MON group whose parent should be replaced
+ * @new_prdtgrp:	replacement parent CTRL_MON group for @rdtgrp
+ * @cpus:		cpumask provided by the caller for use during this call
+ *
+ * Replaces the parent CTRL_MON group for a MON group, resulting in all member
+ * tasks' CLOSID immediately changing to that of the new parent group.
+ * Monitoring data for the group is unaffected by this operation.
+ */
+static void mongrp_reparent(struct rdtgroup *rdtgrp,
+			    struct rdtgroup *new_prdtgrp,
+			    cpumask_var_t cpus)
+{
+	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
+
+	WARN_ON(rdtgrp->type != RDTMON_GROUP);
+	WARN_ON(new_prdtgrp->type != RDTCTRL_GROUP);
+
+	/* Nothing to do when simply renaming a MON group. */
+	if (prdtgrp == new_prdtgrp)
+		return;
+
+	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
+	list_move_tail(&rdtgrp->mon.crdtgrp_list,
+		       &new_prdtgrp->mon.crdtgrp_list);
+
+	rdtgrp->mon.parent = new_prdtgrp;
+	rdtgrp->closid = new_prdtgrp->closid;
+
+	/* Propagate updated closid to all tasks in this group. */
+	rdt_move_group_tasks(rdtgrp, rdtgrp, cpus);
+
+	update_closid_rmid(cpus, NULL);
+}
+
+static int rdtgroup_rename(struct kernfs_node *kn,
+			   struct kernfs_node *new_parent, const char *new_name)
+{
+	struct rdtgroup *new_prdtgrp;
+	struct rdtgroup *rdtgrp;
+	cpumask_var_t tmpmask;
+	int ret;
+
+	rdtgrp = kernfs_to_rdtgroup(kn);
+	new_prdtgrp = kernfs_to_rdtgroup(new_parent);
+	if (!rdtgrp || !new_prdtgrp)
+		return -ENOENT;
+
+	/* Release both kernfs active_refs before obtaining rdtgroup mutex. */
+	rdtgroup_kn_get(rdtgrp, kn);
+	rdtgroup_kn_get(new_prdtgrp, new_parent);
+
+	mutex_lock(&rdtgroup_mutex);
+
+	rdt_last_cmd_clear();
+
+	/*
+	 * Don't allow kernfs_to_rdtgroup() to return a parent rdtgroup if
+	 * either kernfs_node is a file.
+	 */
+	if (kernfs_type(kn) != KERNFS_DIR ||
+	    kernfs_type(new_parent) != KERNFS_DIR) {
+		rdt_last_cmd_puts("Source and destination must be directories");
+		ret = -EPERM;
+		goto out;
+	}
+
+	if ((rdtgrp->flags & RDT_DELETED) || (new_prdtgrp->flags & RDT_DELETED)) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	if (rdtgrp->type != RDTMON_GROUP || !kn->parent ||
+	    !is_mon_groups(kn->parent, kn->name)) {
+		rdt_last_cmd_puts("Source must be a MON group\n");
+		ret = -EPERM;
+		goto out;
+	}
+
+	if (!is_mon_groups(new_parent, new_name)) {
+		rdt_last_cmd_puts("Destination must be a mon_groups subdirectory\n");
+		ret = -EPERM;
+		goto out;
+	}
+
+	/*
+	 * If the MON group is monitoring CPUs, the CPUs must be assigned to the
+	 * current parent CTRL_MON group and therefore cannot be assigned to
+	 * the new parent, making the move illegal.
+	 */
+	if (!cpumask_empty(&rdtgrp->cpu_mask) &&
+	    rdtgrp->mon.parent != new_prdtgrp) {
+		rdt_last_cmd_puts("Cannot move a MON group that monitors CPUs\n");
+		ret = -EPERM;
+		goto out;
+	}
+
+	/*
+	 * Allocate the cpumask for use in mongrp_reparent() to avoid the
+	 * possibility of failing to allocate it after kernfs_rename() has
+	 * succeeded.
+	 */
+	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL)) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	/*
+	 * Perform all input validation and allocations needed to ensure
+	 * mongrp_reparent() will succeed before calling kernfs_rename(),
+	 * otherwise it would be necessary to revert this call if
+	 * mongrp_reparent() failed.
+	 */
+	ret = kernfs_rename(kn, new_parent, new_name);
+	if (!ret)
+		mongrp_reparent(rdtgrp, new_prdtgrp, tmpmask);
+
+	free_cpumask_var(tmpmask);
+
+out:
+	mutex_unlock(&rdtgroup_mutex);
+	rdtgroup_kn_put(rdtgrp, kn);
+	rdtgroup_kn_put(new_prdtgrp, new_parent);
+	return ret;
+}
+
+static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
+{
+	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3))
+		seq_puts(seq, ",cdp");
+
+	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2))
+		seq_puts(seq, ",cdpl2");
+
+	if (is_mba_sc(resctrl_arch_get_resource(RDT_RESOURCE_MBA)))
+		seq_puts(seq, ",mba_MBps");
+
+	if (resctrl_debug)
+		seq_puts(seq, ",debug");
+
+	return 0;
+}
+
+static struct kernfs_syscall_ops rdtgroup_kf_syscall_ops = {
+	.mkdir		= rdtgroup_mkdir,
+	.rmdir		= rdtgroup_rmdir,
+	.rename		= rdtgroup_rename,
+	.show_options	= rdtgroup_show_options,
+};
+
+static int rdtgroup_setup_root(struct rdt_fs_context *ctx)
+{
+	rdt_root = kernfs_create_root(&rdtgroup_kf_syscall_ops,
+				      KERNFS_ROOT_CREATE_DEACTIVATED |
+				      KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK,
+				      &rdtgroup_default);
+	if (IS_ERR(rdt_root))
+		return PTR_ERR(rdt_root);
+
+	ctx->kfc.root = rdt_root;
+	rdtgroup_default.kn = kernfs_root_to_node(rdt_root);
+
+	return 0;
+}
+
+static void rdtgroup_destroy_root(void)
+{
+	kernfs_destroy_root(rdt_root);
+	rdtgroup_default.kn = NULL;
+}
+
+static void rdtgroup_setup_default(void)
+{
+	mutex_lock(&rdtgroup_mutex);
+
+	rdtgroup_default.closid = RESCTRL_RESERVED_CLOSID;
+	rdtgroup_default.mon.rmid = RESCTRL_RESERVED_RMID;
+	rdtgroup_default.type = RDTCTRL_GROUP;
+	INIT_LIST_HEAD(&rdtgroup_default.mon.crdtgrp_list);
+
+	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
+
+	mutex_unlock(&rdtgroup_mutex);
+}
+
+static void domain_destroy_mon_state(struct rdt_domain *d)
+{
+	bitmap_free(d->rmid_busy_llc);
+	kfree(d->mbm_total);
+	kfree(d->mbm_local);
+}
+
+void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)
+{
+	mutex_lock(&rdtgroup_mutex);
+
+	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA)
+		mba_sc_domain_destroy(r, d);
+
+	if (!r->mon_capable)
+		goto out_unlock;
+
+	/*
+	 * If resctrl is mounted, remove all the
+	 * per domain monitor data directories.
+	 */
+	if (resctrl_mounted && resctrl_arch_mon_capable())
+		rmdir_mondata_subdir_allrdtgrp(r, d->id);
+
+	if (resctrl_is_mbm_enabled())
+		cancel_delayed_work(&d->mbm_over);
+	if (resctrl_arch_is_llc_occupancy_enabled() && has_busy_rmid(d)) {
+		/*
+		 * When a package is going down, forcefully
+		 * decrement rmid->ebusy. There is no way to know
+		 * that the L3 was flushed and hence may lead to
+		 * incorrect counts in rare scenarios, but leaving
+		 * the RMID as busy creates RMID leaks if the
+		 * package never comes back.
+		 */
+		__check_limbo(d, true);
+		cancel_delayed_work(&d->cqm_limbo);
+	}
+
+	domain_destroy_mon_state(d);
+
+out_unlock:
+	mutex_unlock(&rdtgroup_mutex);
+}
+
+static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
+{
+	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
+	size_t tsize;
+
+	if (resctrl_arch_is_llc_occupancy_enabled()) {
+		d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL);
+		if (!d->rmid_busy_llc)
+			return -ENOMEM;
+	}
+	if (resctrl_arch_is_mbm_total_enabled()) {
+		tsize = sizeof(*d->mbm_total);
+		d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL);
+		if (!d->mbm_total) {
+			bitmap_free(d->rmid_busy_llc);
+			return -ENOMEM;
+		}
+	}
+	if (resctrl_arch_is_mbm_local_enabled()) {
+		tsize = sizeof(*d->mbm_local);
+		d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL);
+		if (!d->mbm_local) {
+			bitmap_free(d->rmid_busy_llc);
+			kfree(d->mbm_total);
+			return -ENOMEM;
+		}
+	}
+
+	return 0;
+}
+
+int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)
+{
+	int err = 0;
+
+	mutex_lock(&rdtgroup_mutex);
+
+	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA) {
+		/* RDT_RESOURCE_MBA is never mon_capable */
+		err = mba_sc_domain_allocate(r, d);
+		goto out_unlock;
+	}
+
+	if (!r->mon_capable)
+		goto out_unlock;
+
+	err = domain_setup_mon_state(r, d);
+	if (err)
+		goto out_unlock;
+
+	if (resctrl_is_mbm_enabled()) {
+		INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow);
+		mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL,
+					   RESCTRL_PICK_ANY_CPU);
+	}
+
+	if (resctrl_arch_is_llc_occupancy_enabled())
+		INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo);
+
+	/*
+	 * If the filesystem is not mounted then only the default resource group
+	 * exists. Creation of its directories is deferred until mount time
+	 * by rdt_get_tree() calling mkdir_mondata_all().
+	 * If resctrl is mounted, add per domain monitor data directories.
+	 */
+	if (resctrl_mounted && resctrl_arch_mon_capable())
+		mkdir_mondata_subdir_allrdtgrp(r, d);
+
+out_unlock:
+	mutex_unlock(&rdtgroup_mutex);
+
+	return err;
+}
+
+void resctrl_online_cpu(unsigned int cpu)
+{
+	mutex_lock(&rdtgroup_mutex);
+	/* The CPU is set in default rdtgroup after online. */
+	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
+	mutex_unlock(&rdtgroup_mutex);
+}
+
+static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)
+{
+	struct rdtgroup *cr;
+
+	list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) {
+		if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask))
+			break;
+	}
+}
+
+void resctrl_offline_cpu(unsigned int cpu)
+{
+	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+	struct rdtgroup *rdtgrp;
+	struct rdt_domain *d;
+
+	mutex_lock(&rdtgroup_mutex);
+	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
+		if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) {
+			clear_childcpus(rdtgrp, cpu);
+			break;
+		}
+	}
+
+	if (!l3->mon_capable)
+		goto out_unlock;
+
+	d = resctrl_get_domain_from_cpu(cpu, l3);
+	if (d) {
+		if (resctrl_is_mbm_enabled() && cpu == d->mbm_work_cpu) {
+			cancel_delayed_work(&d->mbm_over);
+			mbm_setup_overflow_handler(d, 0, cpu);
+		}
+		if (resctrl_arch_is_llc_occupancy_enabled() &&
+		    cpu == d->cqm_work_cpu && has_busy_rmid(d)) {
+			cancel_delayed_work(&d->cqm_limbo);
+			cqm_setup_limbo_handler(d, 0, cpu);
+		}
+	}
+
+out_unlock:
+	mutex_unlock(&rdtgroup_mutex);
+}
+
+/*
+ * resctrl_init - resctrl filesystem initialization
+ *
+ * Setup resctrl file system including set up root, create mount point,
+ * register resctrl filesystem, and initialize files under root directory.
+ *
+ * Return: 0 on success or -errno
+ */
+int resctrl_init(void)
+{
+	int ret = 0;
+
+	seq_buf_init(&last_cmd_status, last_cmd_status_buf,
+		     sizeof(last_cmd_status_buf));
+
+	rdtgroup_setup_default();
+
+	thread_throttle_mode_init();
+
+	ret = resctrl_mon_resource_init();
+	if (ret)
+		return ret;
+
+	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
+	if (ret)
+		return ret;
+
+	ret = register_filesystem(&rdt_fs_type);
+	if (ret)
+		goto cleanup_mountpoint;
+
+	/*
+	 * Adding the resctrl debugfs directory here may not be ideal since
+	 * it would let the resctrl debugfs directory appear on the debugfs
+	 * filesystem before the resctrl filesystem is mounted.
+	 * It may also be ok since that would enable debugging of RDT before
+	 * resctrl is mounted.
+	 * The reason why the debugfs directory is created here and not in
+	 * rdt_get_tree() is because rdt_get_tree() takes rdtgroup_mutex and
+	 * during the debugfs directory creation also &sb->s_type->i_mutex_key
+	 * (the lockdep class of inode->i_rwsem). Other filesystem
+	 * interactions (eg. SyS_getdents) have the lock ordering:
+	 * &sb->s_type->i_mutex_key --> &mm->mmap_lock
+	 * During mmap(), called with &mm->mmap_lock, the rdtgroup_mutex
+	 * is taken, thus creating dependency:
+	 * &mm->mmap_lock --> rdtgroup_mutex for the latter that can cause
+	 * issues considering the other two lock dependencies.
+	 * By creating the debugfs directory here we avoid a dependency
+	 * that may cause deadlock (even though file operations cannot
+	 * occur until the filesystem is mounted, but I do not know how to
+	 * tell lockdep that).
+	 */
+	debugfs_resctrl = debugfs_create_dir("resctrl", NULL);
+
+	return 0;
+
+cleanup_mountpoint:
+	sysfs_remove_mount_point(fs_kobj, "resctrl");
+
+	return ret;
+}
+
+void resctrl_exit(void)
+{
+	debugfs_remove_recursive(debugfs_resctrl);
+	unregister_filesystem(&rdt_fs_type);
+	sysfs_remove_mount_point(fs_kobj, "resctrl");
+
+	resctrl_mon_resource_exit();
+}
-- 
2.39.2

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Fenghua Yu]

Hi, James,

On 3/21/24 09:51, James Morse wrote:
> resctrl is linux's defacto interface for managing cache and bandwidth
> policies for groups of tasks.
> 
> To allow other architectures to make use of this pseudo filesystem,
> move it live in /fs/resctrl instead of /arch/x86.
> 
> This move leaves behind the parts of resctrl that form the architecture
> interface for x86.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> Discussion needed on how/when to merge this, as it would conflict with
> all outstanding series. It's probably worth deferring to some opportune
> time, but is included here for illustration.
> ---
>   arch/x86/kernel/cpu/resctrl/core.c        |   15 -
>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  505 ---
>   arch/x86/kernel/cpu/resctrl/internal.h    |  310 --
>   arch/x86/kernel/cpu/resctrl/monitor.c     |  821 -----
>   arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1093 ------
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3994 --------------------
>   fs/resctrl/ctrlmondata.c                  |  527 +++
>   fs/resctrl/internal.h                     |  340 ++
>   fs/resctrl/monitor.c                      |  843 +++++
>   fs/resctrl/psuedo_lock.c                  | 1122 ++++++
>   fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
>   11 files changed, 6845 insertions(+), 6738 deletions(-)
> 

checkpatch reports warnings and checks on this patch. Please fix them. e.g.

CHECK: Blank lines aren't necessary before a close brace '}'
#13340: FILE: fs/resctrl/rdtgroup.c:3184:
+
+	}

Thanks.

-Fenghua

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

On Tue, Mar 26, 2024 at 12:44:26PM -0700, Fenghua Yu wrote:
> Hi, James,
> 
> On 3/21/24 09:51, James Morse wrote:
> > resctrl is linux's defacto interface for managing cache and bandwidth
> > policies for groups of tasks.
> > 
> > To allow other architectures to make use of this pseudo filesystem,
> > move it live in /fs/resctrl instead of /arch/x86.
> > 
> > This move leaves behind the parts of resctrl that form the architecture
> > interface for x86.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> > Discussion needed on how/when to merge this, as it would conflict with
> > all outstanding series. It's probably worth deferring to some opportune
> > time, but is included here for illustration.
> > ---
> >   arch/x86/kernel/cpu/resctrl/core.c        |   15 -
> >   arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  505 ---
> >   arch/x86/kernel/cpu/resctrl/internal.h    |  310 --
> >   arch/x86/kernel/cpu/resctrl/monitor.c     |  821 -----
> >   arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1093 ------
> >   arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3994 --------------------
> >   fs/resctrl/ctrlmondata.c                  |  527 +++
> >   fs/resctrl/internal.h                     |  340 ++
> >   fs/resctrl/monitor.c                      |  843 +++++
> >   fs/resctrl/psuedo_lock.c                  | 1122 ++++++
> >   fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
> >   11 files changed, 6845 insertions(+), 6738 deletions(-)
> > 
> 
> checkpatch reports warnings and checks on this patch. Please fix them. e.g.
> 
> CHECK: Blank lines aren't necessary before a close brace '}'
> #13340: FILE: fs/resctrl/rdtgroup.c:3184:
> +
> +	}

Thanks for spotting these...

However, this is a "move code around with no functional change" patch,
so I think that it should paste the original code across verbatim
without trying to address style violations.  (Otherwise, there is no
hope of checking whether this patch is correct or not...)

For the above example, see:
47820e73f5b3 ("x86/resctrl: Initialize a new resource group with default MBA values")

Other than code that is moved or cloned from previously existing code,
do you see any new style problems actually introduced by this patch?


Notwithstanding the above, this series will conflict with a lot of the
in-flight changes pending for resctrl, so it could be a good opportunity
to fix some legacy style nits.

Reinette, do you have a view on this?  If legacy style problems get
addressed in the moved code, are they essential for this series or could
that be done in a follow-up?

Cheers
---Dave

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Reinette Chatre]

Hi Dave,

On 4/11/2024 7:30 AM, Dave Martin wrote:
> On Tue, Mar 26, 2024 at 12:44:26PM -0700, Fenghua Yu wrote:
>> Hi, James,
>>
>> On 3/21/24 09:51, James Morse wrote:
>>> resctrl is linux's defacto interface for managing cache and bandwidth
>>> policies for groups of tasks.
>>>
>>> To allow other architectures to make use of this pseudo filesystem,
>>> move it live in /fs/resctrl instead of /arch/x86.
>>>
>>> This move leaves behind the parts of resctrl that form the architecture
>>> interface for x86.
>>>
>>> Signed-off-by: James Morse <james.morse@arm.com>
>>> ---
>>> Discussion needed on how/when to merge this, as it would conflict with
>>> all outstanding series. It's probably worth deferring to some opportune
>>> time, but is included here for illustration.
>>> ---
>>>   arch/x86/kernel/cpu/resctrl/core.c        |   15 -
>>>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  505 ---
>>>   arch/x86/kernel/cpu/resctrl/internal.h    |  310 --
>>>   arch/x86/kernel/cpu/resctrl/monitor.c     |  821 -----
>>>   arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1093 ------
>>>   arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3994 --------------------
>>>   fs/resctrl/ctrlmondata.c                  |  527 +++
>>>   fs/resctrl/internal.h                     |  340 ++
>>>   fs/resctrl/monitor.c                      |  843 +++++
>>>   fs/resctrl/psuedo_lock.c                  | 1122 ++++++
>>>   fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
>>>   11 files changed, 6845 insertions(+), 6738 deletions(-)
>>>
>>
>> checkpatch reports warnings and checks on this patch. Please fix them. e.g.
>>
>> CHECK: Blank lines aren't necessary before a close brace '}'
>> #13340: FILE: fs/resctrl/rdtgroup.c:3184:
>> +
>> +	}
> 
> Thanks for spotting these...
> 
> However, this is a "move code around with no functional change" patch,
> so I think that it should paste the original code across verbatim
> without trying to address style violations.  (Otherwise, there is no
> hope of checking whether this patch is correct or not...)

I agree that this patch is too big for it to do more than just move
code (please see next comments though).

> 
> For the above example, see:
> 47820e73f5b3 ("x86/resctrl: Initialize a new resource group with default MBA values")
> 
> Other than code that is moved or cloned from previously existing code,
> do you see any new style problems actually introduced by this patch?
> 
> 
> Notwithstanding the above, this series will conflict with a lot of the
> in-flight changes pending for resctrl, so it could be a good opportunity
> to fix some legacy style nits.
> 
> Reinette, do you have a view on this?  If legacy style problems get
> addressed in the moved code, are they essential for this series or could
> that be done in a follow-up?

On its path upstream this series will be scrutinized by various checkers and 
to ensure a smooth merge I would like to recommend that this series aim to
get as clean slate as possible from the basic checkers.

Could a patch addressing these legacy issues precede this patch instead?

I do not think all need to be addressed though. Some of the spelling warnings
are false positives and the camel case appears to be the custom for filesystem
parameter code.

It is not obvious to me that all are legacy issues though ... could you
take a second look at the "WARNING: Use #include <linux/resctrl.h>
instead of <asm/resctrl.h>" ones?

Reinette

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

On Thu, Apr 11, 2024 at 10:45:01AM -0700, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/11/2024 7:30 AM, Dave Martin wrote:
> > On Tue, Mar 26, 2024 at 12:44:26PM -0700, Fenghua Yu wrote:
> >> Hi, James,
> >>
> >> On 3/21/24 09:51, James Morse wrote:
> >>> resctrl is linux's defacto interface for managing cache and bandwidth
> >>> policies for groups of tasks.
> >>>
> >>> To allow other architectures to make use of this pseudo filesystem,
> >>> move it live in /fs/resctrl instead of /arch/x86.
> >>>
> >>> This move leaves behind the parts of resctrl that form the architecture
> >>> interface for x86.
> >>>
> >>> Signed-off-by: James Morse <james.morse@arm.com>
> >>> ---
> >>> Discussion needed on how/when to merge this, as it would conflict with
> >>> all outstanding series. It's probably worth deferring to some opportune
> >>> time, but is included here for illustration.
> >>> ---
> >>>   arch/x86/kernel/cpu/resctrl/core.c        |   15 -
> >>>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  505 ---
> >>>   arch/x86/kernel/cpu/resctrl/internal.h    |  310 --
> >>>   arch/x86/kernel/cpu/resctrl/monitor.c     |  821 -----
> >>>   arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1093 ------
> >>>   arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3994 --------------------
> >>>   fs/resctrl/ctrlmondata.c                  |  527 +++
> >>>   fs/resctrl/internal.h                     |  340 ++
> >>>   fs/resctrl/monitor.c                      |  843 +++++
> >>>   fs/resctrl/psuedo_lock.c                  | 1122 ++++++
> >>>   fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
> >>>   11 files changed, 6845 insertions(+), 6738 deletions(-)
> >>>
> >>
> >> checkpatch reports warnings and checks on this patch. Please fix them. e.g.
> >>
> >> CHECK: Blank lines aren't necessary before a close brace '}'
> >> #13340: FILE: fs/resctrl/rdtgroup.c:3184:
> >> +
> >> +	}
> > 
> > Thanks for spotting these...
> > 
> > However, this is a "move code around with no functional change" patch,
> > so I think that it should paste the original code across verbatim
> > without trying to address style violations.  (Otherwise, there is no
> > hope of checking whether this patch is correct or not...)
> 
> I agree that this patch is too big for it to do more than just move
> code (please see next comments though).
> 
> > 
> > For the above example, see:
> > 47820e73f5b3 ("x86/resctrl: Initialize a new resource group with default MBA values")
> > 
> > Other than code that is moved or cloned from previously existing code,
> > do you see any new style problems actually introduced by this patch?
> > 
> > 
> > Notwithstanding the above, this series will conflict with a lot of the
> > in-flight changes pending for resctrl, so it could be a good opportunity
> > to fix some legacy style nits.
> > 
> > Reinette, do you have a view on this?  If legacy style problems get
> > addressed in the moved code, are they essential for this series or could
> > that be done in a follow-up?
> 
> On its path upstream this series will be scrutinized by various checkers and 
> to ensure a smooth merge I would like to recommend that this series aim to
> get as clean slate as possible from the basic checkers.
> 
> Could a patch addressing these legacy issues precede this patch instead?
> 
> I do not think all need to be addressed though. Some of the spelling warnings
> are false positives and the camel case appears to be the custom for filesystem
> parameter code.
> 
> It is not obvious to me that all are legacy issues though ... could you
> take a second look at the "WARNING: Use #include <linux/resctrl.h>
> instead of <asm/resctrl.h>" ones?
> 
> Reinette

Ack, that does make sense, and it's probably better than letting the dust
settle on this series before applying further cleanups.

I'll make a note to review.

Some of the #include <asm/*> look to have been inherited from the
previous x86 arch code where they would have been more appropriate, but
perhaps some can change to <linux/*> now that more of the definitions
are in the common headers.

Cheers
---Dave

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Reinette Chatre]

Hi James,

On 3/21/2024 9:51 AM, James Morse wrote:
> resctrl is linux's defacto interface for managing cache and bandwidth
> policies for groups of tasks.
> 
> To allow other architectures to make use of this pseudo filesystem,
> move it live in /fs/resctrl instead of /arch/x86.

"move it live" -> "move it to live"?

> 
> This move leaves behind the parts of resctrl that form the architecture
> interface for x86.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> Discussion needed on how/when to merge this, as it would conflict with
> all outstanding series. It's probably worth deferring to some opportune
> time, but is included here for illustration.
> ---

For reference, there are currently two series that are ready to be
considered for inclusion:

https://lore.kernel.org/lkml/20240308213846.77075-1-tony.luck@intel.com/
https://lore.kernel.org/lkml/20240408092303.26413-1-haifeng.xu@shopee.com/

Reinette

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

On Mon, Apr 08, 2024 at 08:43:30PM -0700, Reinette Chatre wrote:
> Hi James,
> 
> On 3/21/2024 9:51 AM, James Morse wrote:
> > resctrl is linux's defacto interface for managing cache and bandwidth
> > policies for groups of tasks.
> > 
> > To allow other architectures to make use of this pseudo filesystem,
> > move it live in /fs/resctrl instead of /arch/x86.
> 
> "move it live" -> "move it to live"?

Noted, thanks.

> > 
> > This move leaves behind the parts of resctrl that form the architecture
> > interface for x86.
> > 
> > Signed-off-by: James Morse <james.morse@arm.com>
> > ---
> > Discussion needed on how/when to merge this, as it would conflict with
> > all outstanding series. It's probably worth deferring to some opportune
> > time, but is included here for illustration.
> > ---
> 
> For reference, there are currently two series that are ready to be
> considered for inclusion:
> 
> https://lore.kernel.org/lkml/20240308213846.77075-1-tony.luck@intel.com/
> https://lore.kernel.org/lkml/20240408092303.26413-1-haifeng.xu@shopee.com/
> 
> Reinette

Understood.

Cheers
---Dave

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Moger, Babu]

James/Dave,
This is a huge change. Can this be broken into multiple patches?
It will be a major task in case we need to bisect to pinpoint any issues
later.
Thanks
Babu

On 3/21/24 11:51, James Morse wrote:
> resctrl is linux's defacto interface for managing cache and bandwidth
> policies for groups of tasks.
> 
> To allow other architectures to make use of this pseudo filesystem,
> move it live in /fs/resctrl instead of /arch/x86.
> 
> This move leaves behind the parts of resctrl that form the architecture
> interface for x86.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> Discussion needed on how/when to merge this, as it would conflict with
> all outstanding series. It's probably worth deferring to some opportune
> time, but is included here for illustration.
> ---
>  arch/x86/kernel/cpu/resctrl/core.c        |   15 -
>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  505 ---
>  arch/x86/kernel/cpu/resctrl/internal.h    |  310 --
>  arch/x86/kernel/cpu/resctrl/monitor.c     |  821 -----
>  arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1093 ------
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3994 --------------------
>  fs/resctrl/ctrlmondata.c                  |  527 +++
>  fs/resctrl/internal.h                     |  340 ++
>  fs/resctrl/monitor.c                      |  843 +++++
>  fs/resctrl/psuedo_lock.c                  | 1122 ++++++
>  fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
>  11 files changed, 6845 insertions(+), 6738 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index c0fb2e22e110..8ddfebd5f008 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -164,21 +164,6 @@ static inline void cache_alloc_hsw_probe(void)
>  	rdt_alloc_capable = true;
>  }
>  
> -bool is_mba_sc(struct rdt_resource *r)
> -{
> -	if (!r)
> -		r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> -
> -	/*
> -	 * The software controller support is only applicable to MBA resource.
> -	 * Make sure to check for resource type.
> -	 */
> -	if (r->rid != RDT_RESOURCE_MBA)
> -		return false;
> -
> -	return r->membw.mba_sc;
> -}
> -
>  /*
>   * rdt_get_mb_table() - get a mapping of bandwidth(b/w) percentage values
>   * exposed to user interface and the h/w understandable delay values.
> diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> index b4627ae19291..c5c3eaea27b6 100644
> --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> @@ -23,260 +23,6 @@
>  
>  #include "internal.h"
>  
> -struct rdt_parse_data {
> -	struct rdtgroup		*rdtgrp;
> -	char			*buf;
> -};
> -
> -typedef int (ctrlval_parser_t)(struct rdt_parse_data *data,
> -			       struct resctrl_schema *s,
> -			       struct rdt_domain *d);
> -
> -/*
> - * Check whether MBA bandwidth percentage value is correct. The value is
> - * checked against the minimum and max bandwidth values specified by the
> - * hardware. The allocated bandwidth percentage is rounded to the next
> - * control step available on the hardware.
> - */
> -static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
> -{
> -	unsigned long bw;
> -	int ret;
> -
> -	/*
> -	 * Only linear delay values is supported for current Intel SKUs.
> -	 */
> -	if (!r->membw.delay_linear && r->membw.arch_needs_linear) {
> -		rdt_last_cmd_puts("No support for non-linear MB domains\n");
> -		return false;
> -	}
> -
> -	ret = kstrtoul(buf, 10, &bw);
> -	if (ret) {
> -		rdt_last_cmd_printf("Non-decimal digit in MB value %s\n", buf);
> -		return false;
> -	}
> -
> -	if ((bw < r->membw.min_bw || bw > r->default_ctrl) &&
> -	    !is_mba_sc(r)) {
> -		rdt_last_cmd_printf("MB value %ld out of range [%d,%d]\n", bw,
> -				    r->membw.min_bw, r->default_ctrl);
> -		return false;
> -	}
> -
> -	*data = roundup(bw, (unsigned long)r->membw.bw_gran);
> -	return true;
> -}
> -
> -static int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
> -		    struct rdt_domain *d)
> -{
> -	struct resctrl_staged_config *cfg;
> -	u32 closid = data->rdtgrp->closid;
> -	struct rdt_resource *r = s->res;
> -	unsigned long bw_val;
> -
> -	cfg = &d->staged_config[s->conf_type];
> -	if (cfg->have_new_ctrl) {
> -		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
> -		return -EINVAL;
> -	}
> -
> -	if (!bw_validate(data->buf, &bw_val, r))
> -		return -EINVAL;
> -
> -	if (is_mba_sc(r)) {
> -		d->mbps_val[closid] = bw_val;
> -		return 0;
> -	}
> -
> -	cfg->new_ctrl = bw_val;
> -	cfg->have_new_ctrl = true;
> -
> -	return 0;
> -}
> -
> -/*
> - * Check whether a cache bit mask is valid.
> - * On Intel CPUs, non-contiguous 1s value support is indicated by CPUID:
> - *   - CPUID.0x10.1:ECX[3]: L3 non-contiguous 1s value supported if 1
> - *   - CPUID.0x10.2:ECX[3]: L2 non-contiguous 1s value supported if 1
> - *
> - * Haswell does not support a non-contiguous 1s value and additionally
> - * requires at least two bits set.
> - * AMD allows non-contiguous bitmasks.
> - */
> -static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
> -{
> -	unsigned long first_bit, zero_bit, val;
> -	unsigned int cbm_len = r->cache.cbm_len;
> -	int ret;
> -
> -	ret = kstrtoul(buf, 16, &val);
> -	if (ret) {
> -		rdt_last_cmd_printf("Non-hex character in the mask %s\n", buf);
> -		return false;
> -	}
> -
> -	if ((r->cache.min_cbm_bits > 0 && val == 0) || val > r->default_ctrl) {
> -		rdt_last_cmd_puts("Mask out of range\n");
> -		return false;
> -	}
> -
> -	first_bit = find_first_bit(&val, cbm_len);
> -	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
> -
> -	/* Are non-contiguous bitmasks allowed? */
> -	if (!r->cache.arch_has_sparse_bitmasks &&
> -	    (find_next_bit(&val, cbm_len, zero_bit) < cbm_len)) {
> -		rdt_last_cmd_printf("The mask %lx has non-consecutive 1-bits\n", val);
> -		return false;
> -	}
> -
> -	if ((zero_bit - first_bit) < r->cache.min_cbm_bits) {
> -		rdt_last_cmd_printf("Need at least %d bits in the mask\n",
> -				    r->cache.min_cbm_bits);
> -		return false;
> -	}
> -
> -	*data = val;
> -	return true;
> -}
> -
> -/*
> - * Read one cache bit mask (hex). Check that it is valid for the current
> - * resource type.
> - */
> -static int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
> -		     struct rdt_domain *d)
> -{
> -	struct rdtgroup *rdtgrp = data->rdtgrp;
> -	struct resctrl_staged_config *cfg;
> -	struct rdt_resource *r = s->res;
> -	u32 cbm_val;
> -
> -	cfg = &d->staged_config[s->conf_type];
> -	if (cfg->have_new_ctrl) {
> -		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
> -		return -EINVAL;
> -	}
> -
> -	/*
> -	 * Cannot set up more than one pseudo-locked region in a cache
> -	 * hierarchy.
> -	 */
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
> -	    rdtgroup_pseudo_locked_in_hierarchy(d)) {
> -		rdt_last_cmd_puts("Pseudo-locked region in hierarchy\n");
> -		return -EINVAL;
> -	}
> -
> -	if (!cbm_validate(data->buf, &cbm_val, r))
> -		return -EINVAL;
> -
> -	if ((rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
> -	     rdtgrp->mode == RDT_MODE_SHAREABLE) &&
> -	    rdtgroup_cbm_overlaps_pseudo_locked(d, cbm_val)) {
> -		rdt_last_cmd_puts("CBM overlaps with pseudo-locked region\n");
> -		return -EINVAL;
> -	}
> -
> -	/*
> -	 * The CBM may not overlap with the CBM of another closid if
> -	 * either is exclusive.
> -	 */
> -	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, true)) {
> -		rdt_last_cmd_puts("Overlaps with exclusive group\n");
> -		return -EINVAL;
> -	}
> -
> -	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, false)) {
> -		if (rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
> -		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -			rdt_last_cmd_puts("Overlaps with other group\n");
> -			return -EINVAL;
> -		}
> -	}
> -
> -	cfg->new_ctrl = cbm_val;
> -	cfg->have_new_ctrl = true;
> -
> -	return 0;
> -}
> -
> -static ctrlval_parser_t *get_parser(struct rdt_resource *res)
> -{
> -	if (res->fflags & RFTYPE_RES_CACHE)
> -		return &parse_cbm;
> -	else
> -		return &parse_bw;
> -}
> -
> -/*
> - * For each domain in this resource we expect to find a series of:
> - *	id=mask
> - * separated by ";". The "id" is in decimal, and must match one of
> - * the "id"s for this resource.
> - */
> -static int parse_line(char *line, struct resctrl_schema *s,
> -		      struct rdtgroup *rdtgrp)
> -{
> -	ctrlval_parser_t *parse_ctrlval = get_parser(s->res);
> -	enum resctrl_conf_type t = s->conf_type;
> -	struct resctrl_staged_config *cfg;
> -	struct rdt_resource *r = s->res;
> -	struct rdt_parse_data data;
> -	char *dom = NULL, *id;
> -	struct rdt_domain *d;
> -	unsigned long dom_id;
> -
> -	/* Walking r->domains, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
> -	    (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)) {
> -		rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
> -		return -EINVAL;
> -	}
> -
> -next:
> -	if (!line || line[0] == '\0')
> -		return 0;
> -	dom = strsep(&line, ";");
> -	id = strsep(&dom, "=");
> -	if (!dom || kstrtoul(id, 10, &dom_id)) {
> -		rdt_last_cmd_puts("Missing '=' or non-numeric domain\n");
> -		return -EINVAL;
> -	}
> -	dom = strim(dom);
> -	list_for_each_entry(d, &r->domains, list) {
> -		if (d->id == dom_id) {
> -			data.buf = dom;
> -			data.rdtgrp = rdtgrp;
> -			if (parse_ctrlval(&data, s, d))
> -				return -EINVAL;
> -			if (rdtgrp->mode ==  RDT_MODE_PSEUDO_LOCKSETUP) {
> -				cfg = &d->staged_config[t];
> -				/*
> -				 * In pseudo-locking setup mode and just
> -				 * parsed a valid CBM that should be
> -				 * pseudo-locked. Only one locked region per
> -				 * resource group and domain so just do
> -				 * the required initialization for single
> -				 * region and return.
> -				 */
> -				rdtgrp->plr->s = s;
> -				rdtgrp->plr->d = d;
> -				rdtgrp->plr->cbm = cfg->new_ctrl;
> -				d->plr = rdtgrp->plr;
> -				return 0;
> -			}
> -			goto next;
> -		}
> -	}
> -	return -EINVAL;
> -}
> -
>  static bool apply_config(struct rdt_hw_domain *hw_dom,
>  			 struct resctrl_staged_config *cfg, u32 idx,
>  			 cpumask_var_t cpu_mask)
> @@ -365,100 +111,6 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
>  	return 0;
>  }
>  
> -static int rdtgroup_parse_resource(char *resname, char *tok,
> -				   struct rdtgroup *rdtgrp)
> -{
> -	struct resctrl_schema *s;
> -
> -	list_for_each_entry(s, &resctrl_schema_all, list) {
> -		if (!strcmp(resname, s->name) && rdtgrp->closid < s->num_closid)
> -			return parse_line(tok, s, rdtgrp);
> -	}
> -	rdt_last_cmd_printf("Unknown or unsupported resource name '%s'\n", resname);
> -	return -EINVAL;
> -}
> -
> -ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
> -				char *buf, size_t nbytes, loff_t off)
> -{
> -	struct resctrl_schema *s;
> -	struct rdtgroup *rdtgrp;
> -	struct rdt_resource *r;
> -	char *tok, *resname;
> -	int ret = 0;
> -
> -	/* Valid input requires a trailing newline */
> -	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> -		return -EINVAL;
> -	buf[nbytes - 1] = '\0';
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (!rdtgrp) {
> -		rdtgroup_kn_unlock(of->kn);
> -		return -ENOENT;
> -	}
> -	rdt_last_cmd_clear();
> -
> -	/*
> -	 * No changes to pseudo-locked region allowed. It has to be removed
> -	 * and re-created instead.
> -	 */
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> -		ret = -EINVAL;
> -		rdt_last_cmd_puts("Resource group is pseudo-locked\n");
> -		goto out;
> -	}
> -
> -	rdt_staged_configs_clear();
> -
> -	while ((tok = strsep(&buf, "\n")) != NULL) {
> -		resname = strim(strsep(&tok, ":"));
> -		if (!tok) {
> -			rdt_last_cmd_puts("Missing ':'\n");
> -			ret = -EINVAL;
> -			goto out;
> -		}
> -		if (tok[0] == '\0') {
> -			rdt_last_cmd_printf("Missing '%s' value\n", resname);
> -			ret = -EINVAL;
> -			goto out;
> -		}
> -		ret = rdtgroup_parse_resource(resname, tok, rdtgrp);
> -		if (ret)
> -			goto out;
> -	}
> -
> -	list_for_each_entry(s, &resctrl_schema_all, list) {
> -		r = s->res;
> -
> -		/*
> -		 * Writes to mba_sc resources update the software controller,
> -		 * not the control MSR.
> -		 */
> -		if (is_mba_sc(r))
> -			continue;
> -
> -		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
> -		if (ret)
> -			goto out;
> -	}
> -
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -		/*
> -		 * If pseudo-locking fails we keep the resource group in
> -		 * mode RDT_MODE_PSEUDO_LOCKSETUP with its class of service
> -		 * active and updated for just the domain the pseudo-locked
> -		 * region was requested for.
> -		 */
> -		ret = rdtgroup_pseudo_lock_create(rdtgrp);
> -	}
> -
> -out:
> -	rdt_staged_configs_clear();
> -	rdtgroup_kn_unlock(of->kn);
> -	return ret ?: nbytes;
> -}
> -
>  u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
>  			    u32 closid, enum resctrl_conf_type type)
>  {
> @@ -467,160 +119,3 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
>  
>  	return hw_dom->ctrl_val[idx];
>  }
> -
> -static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int closid)
> -{
> -	struct rdt_resource *r = schema->res;
> -	struct rdt_domain *dom;
> -	bool sep = false;
> -	u32 ctrl_val;
> -
> -	/* Walking r->domains, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -	seq_printf(s, "%*s:", max_name_width, schema->name);
> -	list_for_each_entry(dom, &r->domains, list) {
> -		if (sep)
> -			seq_puts(s, ";");
> -
> -		if (is_mba_sc(r))
> -			ctrl_val = dom->mbps_val[closid];
> -		else
> -			ctrl_val = resctrl_arch_get_config(r, dom, closid,
> -							   schema->conf_type);
> -
> -		seq_printf(s, r->format_str, dom->id, max_data_width,
> -			   ctrl_val);
> -		sep = true;
> -	}
> -	seq_puts(s, "\n");
> -}
> -
> -int rdtgroup_schemata_show(struct kernfs_open_file *of,
> -			   struct seq_file *s, void *v)
> -{
> -	struct resctrl_schema *schema;
> -	struct rdtgroup *rdtgrp;
> -	int ret = 0;
> -	u32 closid;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (rdtgrp) {
> -		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -			list_for_each_entry(schema, &resctrl_schema_all, list) {
> -				seq_printf(s, "%s:uninitialized\n", schema->name);
> -			}
> -		} else if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> -			if (!rdtgrp->plr->d) {
> -				rdt_last_cmd_clear();
> -				rdt_last_cmd_puts("Cache domain offline\n");
> -				ret = -ENODEV;
> -			} else {
> -				seq_printf(s, "%s:%d=%x\n",
> -					   rdtgrp->plr->s->res->name,
> -					   rdtgrp->plr->d->id,
> -					   rdtgrp->plr->cbm);
> -			}
> -		} else {
> -			closid = rdtgrp->closid;
> -			list_for_each_entry(schema, &resctrl_schema_all, list) {
> -				if (closid < schema->num_closid)
> -					show_doms(s, schema, closid);
> -			}
> -		}
> -	} else {
> -		ret = -ENOENT;
> -	}
> -	rdtgroup_kn_unlock(of->kn);
> -	return ret;
> -}
> -
> -static int smp_mon_event_count(void *arg)
> -{
> -	mon_event_count(arg);
> -
> -	return 0;
> -}
> -
> -void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
> -		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
> -		    int evtid, int first)
> -{
> -	int cpu;
> -
> -	/* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -	/*
> -	 * Setup the parameters to pass to mon_event_count() to read the data.
> -	 */
> -	rr->rgrp = rdtgrp;
> -	rr->evtid = evtid;
> -	rr->r = r;
> -	rr->d = d;
> -	rr->val = 0;
> -	rr->first = first;
> -	rr->arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, evtid);
> -	if (IS_ERR(rr->arch_mon_ctx)) {
> -		rr->err = -EINVAL;
> -		return;
> -	}
> -
> -	cpu = cpumask_any_housekeeping(&d->cpu_mask, RESCTRL_PICK_ANY_CPU);
> -
> -	/*
> -	 * cpumask_any_housekeeping() prefers housekeeping CPUs, but
> -	 * are all the CPUs nohz_full? If yes, pick a CPU to IPI.
> -	 * MPAM's resctrl_arch_rmid_read() is unable to read the
> -	 * counters on some platforms if its called in IRQ context.
> -	 */
> -	if (tick_nohz_full_cpu(cpu))
> -		smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1);
> -	else
> -		smp_call_on_cpu(cpu, smp_mon_event_count, rr, false);
> -
> -	resctrl_arch_mon_ctx_free(r, evtid, rr->arch_mon_ctx);
> -}
> -
> -int rdtgroup_mondata_show(struct seq_file *m, void *arg)
> -{
> -	struct kernfs_open_file *of = m->private;
> -	u32 resid, evtid, domid;
> -	struct rdtgroup *rdtgrp;
> -	struct rdt_resource *r;
> -	union mon_data_bits md;
> -	struct rdt_domain *d;
> -	struct rmid_read rr;
> -	int ret = 0;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (!rdtgrp) {
> -		ret = -ENOENT;
> -		goto out;
> -	}
> -
> -	md.priv = of->kn->priv;
> -	resid = md.u.rid;
> -	domid = md.u.domid;
> -	evtid = md.u.evtid;
> -
> -	r = resctrl_arch_get_resource(resid);
> -	d = resctrl_arch_find_domain(r, domid);
> -	if (IS_ERR_OR_NULL(d)) {
> -		ret = -ENOENT;
> -		goto out;
> -	}
> -
> -	mon_event_read(&rr, r, d, rdtgrp, evtid, false);
> -
> -	if (rr.err == -EIO)
> -		seq_puts(m, "Error\n");
> -	else if (rr.err == -EINVAL)
> -		seq_puts(m, "Unavailable\n");
> -	else
> -		seq_printf(m, "%llu\n", rr.val);
> -
> -out:
> -	rdtgroup_kn_unlock(of->kn);
> -	return ret;
> -}
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index 0f7e3f10941b..bf3538992667 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -26,227 +26,6 @@
>   */
>  #define MBM_CNTR_WIDTH_OFFSET_MAX (62 - MBM_CNTR_WIDTH_BASE)
>  
> -/**
> - * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that
> - *			        aren't marked nohz_full
> - * @mask:	The mask to pick a CPU from.
> - * @exclude_cpu:The CPU to avoid picking.
> - *
> - * Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping
> - * CPUs that don't use nohz_full, these are preferred. Pass
> - * RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs.
> - *
> - * When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available.
> - */
> -static inline unsigned int
> -cpumask_any_housekeeping(const struct cpumask *mask, int exclude_cpu)
> -{
> -	unsigned int cpu, hk_cpu;
> -
> -	if (exclude_cpu == RESCTRL_PICK_ANY_CPU)
> -		cpu = cpumask_any(mask);
> -	else
> -		cpu = cpumask_any_but(mask, exclude_cpu);
> -
> -	if (!IS_ENABLED(CONFIG_NO_HZ_FULL))
> -		return cpu;
> -
> -	/* If the CPU picked isn't marked nohz_full nothing more needs doing. */
> -	if (cpu < nr_cpu_ids && !tick_nohz_full_cpu(cpu))
> -		return cpu;
> -
> -	/* Try to find a CPU that isn't nohz_full to use in preference */
> -	hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask);
> -	if (hk_cpu == exclude_cpu)
> -		hk_cpu = cpumask_nth_andnot(1, mask, tick_nohz_full_mask);
> -
> -	if (hk_cpu < nr_cpu_ids)
> -		cpu = hk_cpu;
> -
> -	return cpu;
> -}
> -
> -struct rdt_fs_context {
> -	struct kernfs_fs_context	kfc;
> -	bool				enable_cdpl2;
> -	bool				enable_cdpl3;
> -	bool				enable_mba_mbps;
> -	bool				enable_debug;
> -};
> -
> -static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
> -{
> -	struct kernfs_fs_context *kfc = fc->fs_private;
> -
> -	return container_of(kfc, struct rdt_fs_context, kfc);
> -}
> -
> -/**
> - * struct mon_evt - Entry in the event list of a resource
> - * @evtid:		event id
> - * @name:		name of the event
> - * @configurable:	true if the event is configurable
> - * @list:		entry in &rdt_resource->evt_list
> - */
> -struct mon_evt {
> -	enum resctrl_event_id	evtid;
> -	char			*name;
> -	bool			configurable;
> -	struct list_head	list;
> -};
> -
> -/**
> - * union mon_data_bits - Monitoring details for each event file
> - * @priv:              Used to store monitoring event data in @u
> - *                     as kernfs private data
> - * @rid:               Resource id associated with the event file
> - * @evtid:             Event id associated with the event file
> - * @domid:             The domain to which the event file belongs
> - * @u:                 Name of the bit fields struct
> - */
> -union mon_data_bits {
> -	void *priv;
> -	struct {
> -		unsigned int rid		: 10;
> -		enum resctrl_event_id evtid	: 8;
> -		unsigned int domid		: 14;
> -	} u;
> -};
> -
> -struct rmid_read {
> -	struct rdtgroup		*rgrp;
> -	struct rdt_resource	*r;
> -	struct rdt_domain	*d;
> -	enum resctrl_event_id	evtid;
> -	bool			first;
> -	int			err;
> -	u64			val;
> -	void			*arch_mon_ctx;
> -};
> -
> -extern struct list_head resctrl_schema_all;
> -extern bool resctrl_mounted;
> -
> -enum rdt_group_type {
> -	RDTCTRL_GROUP = 0,
> -	RDTMON_GROUP,
> -	RDT_NUM_GROUP,
> -};
> -
> -/**
> - * enum rdtgrp_mode - Mode of a RDT resource group
> - * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations
> - * @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed
> - * @RDT_MODE_PSEUDO_LOCKSETUP: Resource group will be used for Pseudo-Locking
> - * @RDT_MODE_PSEUDO_LOCKED: No sharing of this resource group's allocations
> - *                          allowed AND the allocations are Cache Pseudo-Locked
> - * @RDT_NUM_MODES: Total number of modes
> - *
> - * The mode of a resource group enables control over the allowed overlap
> - * between allocations associated with different resource groups (classes
> - * of service). User is able to modify the mode of a resource group by
> - * writing to the "mode" resctrl file associated with the resource group.
> - *
> - * The "shareable", "exclusive", and "pseudo-locksetup" modes are set by
> - * writing the appropriate text to the "mode" file. A resource group enters
> - * "pseudo-locked" mode after the schemata is written while the resource
> - * group is in "pseudo-locksetup" mode.
> - */
> -enum rdtgrp_mode {
> -	RDT_MODE_SHAREABLE = 0,
> -	RDT_MODE_EXCLUSIVE,
> -	RDT_MODE_PSEUDO_LOCKSETUP,
> -	RDT_MODE_PSEUDO_LOCKED,
> -
> -	/* Must be last */
> -	RDT_NUM_MODES,
> -};
> -
> -/**
> - * struct mongroup - store mon group's data in resctrl fs.
> - * @mon_data_kn:		kernfs node for the mon_data directory
> - * @parent:			parent rdtgrp
> - * @crdtgrp_list:		child rdtgroup node list
> - * @rmid:			rmid for this rdtgroup
> - */
> -struct mongroup {
> -	struct kernfs_node	*mon_data_kn;
> -	struct rdtgroup		*parent;
> -	struct list_head	crdtgrp_list;
> -	u32			rmid;
> -};
> -
> -/**
> - * struct rdtgroup - store rdtgroup's data in resctrl file system.
> - * @kn:				kernfs node
> - * @rdtgroup_list:		linked list for all rdtgroups
> - * @closid:			closid for this rdtgroup
> - * @cpu_mask:			CPUs assigned to this rdtgroup
> - * @flags:			status bits
> - * @waitcount:			how many cpus expect to find this
> - *				group when they acquire rdtgroup_mutex
> - * @type:			indicates type of this rdtgroup - either
> - *				monitor only or ctrl_mon group
> - * @mon:			mongroup related data
> - * @mode:			mode of resource group
> - * @plr:			pseudo-locked region
> - */
> -struct rdtgroup {
> -	struct kernfs_node		*kn;
> -	struct list_head		rdtgroup_list;
> -	u32				closid;
> -	struct cpumask			cpu_mask;
> -	int				flags;
> -	atomic_t			waitcount;
> -	enum rdt_group_type		type;
> -	struct mongroup			mon;
> -	enum rdtgrp_mode		mode;
> -	struct pseudo_lock_region	*plr;
> -};
> -
> -/* List of all resource groups */
> -extern struct list_head rdt_all_groups;
> -
> -extern int max_name_width, max_data_width;
> -
> -/**
> - * struct rftype - describe each file in the resctrl file system
> - * @name:	File name
> - * @mode:	Access mode
> - * @kf_ops:	File operations
> - * @flags:	File specific RFTYPE_FLAGS_* flags
> - * @fflags:	File specific RFTYPE_* flags
> - * @seq_show:	Show content of the file
> - * @write:	Write to the file
> - */
> -struct rftype {
> -	char			*name;
> -	umode_t			mode;
> -	const struct kernfs_ops	*kf_ops;
> -	unsigned long		flags;
> -	unsigned long		fflags;
> -
> -	int (*seq_show)(struct kernfs_open_file *of,
> -			struct seq_file *sf, void *v);
> -	/*
> -	 * write() is the generic write callback which maps directly to
> -	 * kernfs write operation and overrides all other operations.
> -	 * Maximum write size is determined by ->max_write_len.
> -	 */
> -	ssize_t (*write)(struct kernfs_open_file *of,
> -			 char *buf, size_t nbytes, loff_t off);
> -};
> -
> -/**
> - * struct mbm_state - status for each MBM counter in each domain
> - * @prev_bw_bytes: Previous bytes value read for bandwidth calculation
> - * @prev_bw:	The most recent bandwidth in MBps
> - */
> -struct mbm_state {
> -	u64	prev_bw_bytes;
> -	u32	prev_bw;
> -};
> -
>  /**
>   * struct arch_mbm_state - values used to compute resctrl_arch_rmid_read()s
>   *			   return value.
> @@ -327,11 +106,7 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *r
>  	return container_of(r, struct rdt_hw_resource, r_resctrl);
>  }
>  
> -extern struct mutex rdtgroup_mutex;
> -
>  extern struct rdt_hw_resource rdt_resources_all[];
> -extern struct rdtgroup rdtgroup_default;
> -extern struct dentry *debugfs_resctrl;
>  
>  static inline struct rdt_resource *resctrl_inc(struct rdt_resource *res)
>  {
> @@ -395,95 +170,10 @@ union cpuid_0x10_x_edx {
>  	unsigned int full;
>  };
>  
> -void rdt_last_cmd_clear(void);
> -void rdt_last_cmd_puts(const char *s);
> -__printf(1, 2)
> -void rdt_last_cmd_printf(const char *fmt, ...);
> -
>  void rdt_ctrl_update(void *arg);
> -struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
> -void rdtgroup_kn_unlock(struct kernfs_node *kn);
> -int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name);
> -int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
> -			     umode_t mask);
> -ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
> -				char *buf, size_t nbytes, loff_t off);
> -int rdtgroup_schemata_show(struct kernfs_open_file *of,
> -			   struct seq_file *s, void *v);
> -bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
> -			   unsigned long cbm, int closid, bool exclusive);
> -unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d,
> -				  unsigned long cbm);
> -enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);
> -int rdtgroup_tasks_assigned(struct rdtgroup *r);
> -int closids_supported(void);
> -void closid_free(int closid);
> -int alloc_rmid(u32 closid);
> -void free_rmid(u32 closid, u32 rmid);
>  int rdt_get_mon_l3_config(struct rdt_resource *r);
> -void resctrl_mon_resource_exit(void);
>  bool rdt_cpu_has(int flag);
> -void mon_event_count(void *info);
> -int rdtgroup_mondata_show(struct seq_file *m, void *arg);
> -void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
> -		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
> -		    int evtid, int first);
> -int resctrl_mon_resource_init(void);
> -void mbm_setup_overflow_handler(struct rdt_domain *dom,
> -				unsigned long delay_ms,
> -				int exclude_cpu);
> -void mbm_handle_overflow(struct work_struct *work);
>  void __init intel_rdt_mbm_apply_quirk(void);
> -bool is_mba_sc(struct rdt_resource *r);
> -void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
> -			     int exclude_cpu);
> -void cqm_handle_limbo(struct work_struct *work);
> -bool has_busy_rmid(struct rdt_domain *d);
> -void __check_limbo(struct rdt_domain *d, bool force_free);
>  void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
> -void mbm_config_rftype_init(const char *config);
> -void rdt_staged_configs_clear(void);
> -bool closid_allocated(unsigned int closid);
> -int resctrl_find_cleanest_closid(void);
> -
> -#ifdef CONFIG_RESCTRL_FS_PSEUDO_LOCK
> -int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
> -int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp);
> -bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm);
> -bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d);
> -int rdt_pseudo_lock_init(void);
> -void rdt_pseudo_lock_release(void);
> -int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
> -void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
> -#else
> -static inline int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
> -{
> -	return -EOPNOTSUPP;
> -}
> -
> -static inline int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
> -{
> -	return -EOPNOTSUPP;
> -}
> -
> -static inline bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
> -{
> -	return false;
> -}
> -
> -static inline bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
> -{
> -	return false;
> -}
> -
> -static inline int rdt_pseudo_lock_init(void) { return 0; }
> -static inline void rdt_pseudo_lock_release(void) { }
> -static inline int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
> -{
> -	return -EOPNOTSUPP;
> -}
> -
> -static inline void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp) { }
> -#endif /* CONFIG_RESCTRL_FS_PSEUDO_LOCK */
>  
>  #endif /* _ASM_X86_RESCTRL_INTERNAL_H */
> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index 7e6fca138cb7..02fb9d87479a 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -25,53 +25,6 @@
>  
>  #include "internal.h"
>  
> -/**
> - * struct rmid_entry - dirty tracking for all RMID.
> - * @closid:	The CLOSID for this entry.
> - * @rmid:	The RMID for this entry.
> - * @busy:	The number of domains with cached data using this RMID.
> - * @list:	Member of the rmid_free_lru list when busy == 0.
> - *
> - * Depending on the architecture the correct monitor is accessed using
> - * both @closid and @rmid, or @rmid only.
> - *
> - * Take the rdtgroup_mutex when accessing.
> - */
> -struct rmid_entry {
> -	u32				closid;
> -	u32				rmid;
> -	int				busy;
> -	struct list_head		list;
> -};
> -
> -/*
> - * @rmid_free_lru - A least recently used list of free RMIDs
> - *     These RMIDs are guaranteed to have an occupancy less than the
> - *     threshold occupancy
> - */
> -static LIST_HEAD(rmid_free_lru);
> -
> -/*
> - * @closid_num_dirty_rmid    The number of dirty RMID each CLOSID has.
> - *     Only allocated when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined.
> - *     Indexed by CLOSID. Protected by rdtgroup_mutex.
> - */
> -static u32 *closid_num_dirty_rmid;
> -
> -/*
> - * @rmid_limbo_count - count of currently unused but (potentially)
> - *     dirty RMIDs.
> - *     This counts RMIDs that no one is currently using but that
> - *     may have a occupancy value > resctrl_rmid_realloc_threshold. User can
> - *     change the threshold occupancy value.
> - */
> -static unsigned int rmid_limbo_count;
> -
> -/*
> - * @rmid_entry - The entry in the limbo and free lists.
> - */
> -static struct rmid_entry	*rmid_ptrs;
> -
>  /*
>   * Global boolean for rdt_monitor which is true if any
>   * resource monitoring is enabled.
> @@ -83,17 +36,6 @@ bool rdt_mon_capable;
>   */
>  unsigned int rdt_mon_features;
>  
> -/*
> - * This is the threshold cache occupancy in bytes at which we will consider an
> - * RMID available for re-allocation.
> - */
> -unsigned int resctrl_rmid_realloc_threshold;
> -
> -/*
> - * This is the maximum value for the reallocation threshold, in bytes.
> - */
> -unsigned int resctrl_rmid_realloc_limit;
> -
>  #define CF(cf)	((unsigned long)(1048576 * (cf) + 0.5))
>  
>  /*
> @@ -157,33 +99,6 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)
>  	return val;
>  }
>  
> -/*
> - * x86 and arm64 differ in their handling of monitoring.
> - * x86's RMID are independent numbers, there is only one source of traffic
> - * with an RMID value of '1'.
> - * arm64's PMG extends the PARTID/CLOSID space, there are multiple sources of
> - * traffic with a PMG value of '1', one for each CLOSID, meaning the RMID
> - * value is no longer unique.
> - * To account for this, resctrl uses an index. On x86 this is just the RMID,
> - * on arm64 it encodes the CLOSID and RMID. This gives a unique number.
> - *
> - * The domain's rmid_busy_llc and rmid_ptrs[] are sized by index. The arch code
> - * must accept an attempt to read every index.
> - */
> -static inline struct rmid_entry *__rmid_entry(u32 idx)
> -{
> -	struct rmid_entry *entry;
> -	u32 closid, rmid;
> -
> -	entry = &rmid_ptrs[idx];
> -	resctrl_arch_rmid_idx_decode(idx, &closid, &rmid);
> -
> -	WARN_ON_ONCE(entry->closid != closid);
> -	WARN_ON_ONCE(entry->rmid != rmid);
> -
> -	return entry;
> -}
> -
>  static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)
>  {
>  	u64 msr_val;
> @@ -302,735 +217,6 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,
>  	return 0;
>  }
>  
> -static void limbo_release_entry(struct rmid_entry *entry)
> -{
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	rmid_limbo_count--;
> -	list_add_tail(&entry->list, &rmid_free_lru);
> -
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
> -		closid_num_dirty_rmid[entry->closid]--;
> -}
> -
> -/*
> - * Check the RMIDs that are marked as busy for this domain. If the
> - * reported LLC occupancy is below the threshold clear the busy bit and
> - * decrement the count. If the busy count gets to zero on an RMID, we
> - * free the RMID
> - */
> -void __check_limbo(struct rdt_domain *d, bool force_free)
> -{
> -	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> -	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> -	struct rmid_entry *entry;
> -	u32 idx, cur_idx = 1;
> -	void *arch_mon_ctx;
> -	bool rmid_dirty;
> -	u64 val = 0;
> -
> -	arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID);
> -	if (IS_ERR(arch_mon_ctx)) {
> -		pr_warn_ratelimited("Failed to allocate monitor context: %ld",
> -				    PTR_ERR(arch_mon_ctx));
> -		return;
> -	}
> -
> -	/*
> -	 * Skip RMID 0 and start from RMID 1 and check all the RMIDs that
> -	 * are marked as busy for occupancy < threshold. If the occupancy
> -	 * is less than the threshold decrement the busy counter of the
> -	 * RMID and move it to the free list when the counter reaches 0.
> -	 */
> -	for (;;) {
> -		idx = find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx);
> -		if (idx >= idx_limit)
> -			break;
> -
> -		entry = __rmid_entry(idx);
> -		if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid,
> -					   QOS_L3_OCCUP_EVENT_ID, &val,
> -					   arch_mon_ctx)) {
> -			rmid_dirty = true;
> -		} else {
> -			rmid_dirty = (val >= resctrl_rmid_realloc_threshold);
> -		}
> -
> -		if (force_free || !rmid_dirty) {
> -			clear_bit(idx, d->rmid_busy_llc);
> -			if (!--entry->busy)
> -				limbo_release_entry(entry);
> -		}
> -		cur_idx = idx + 1;
> -	}
> -
> -	resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx);
> -}
> -
> -bool has_busy_rmid(struct rdt_domain *d)
> -{
> -	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> -
> -	return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit;
> -}
> -
> -static struct rmid_entry *resctrl_find_free_rmid(u32 closid)
> -{
> -	struct rmid_entry *itr;
> -	u32 itr_idx, cmp_idx;
> -
> -	if (list_empty(&rmid_free_lru))
> -		return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC);
> -
> -	list_for_each_entry(itr, &rmid_free_lru, list) {
> -		/*
> -		 * Get the index of this free RMID, and the index it would need
> -		 * to be if it were used with this CLOSID.
> -		 * If the CLOSID is irrelevant on this architecture, the two
> -		 * index values are always the same on every entry and thus the
> -		 * very first entry will be returned.
> -		 */
> -		itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid);
> -		cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid);
> -
> -		if (itr_idx == cmp_idx)
> -			return itr;
> -	}
> -
> -	return ERR_PTR(-ENOSPC);
> -}
> -
> -/**
> - * resctrl_find_cleanest_closid() - Find a CLOSID where all the associated
> - *                                  RMID are clean, or the CLOSID that has
> - *                                  the most clean RMID.
> - *
> - * MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocated CLOSID
> - * may not be able to allocate clean RMID. To avoid this the allocator will
> - * choose the CLOSID with the most clean RMID.
> - *
> - * When the CLOSID and RMID are independent numbers, the first free CLOSID will
> - * be returned.
> - */
> -int resctrl_find_cleanest_closid(void)
> -{
> -	u32 cleanest_closid = ~0;
> -	int i = 0;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
> -		return -EIO;
> -
> -	for (i = 0; i < closids_supported(); i++) {
> -		int num_dirty;
> -
> -		if (closid_allocated(i))
> -			continue;
> -
> -		num_dirty = closid_num_dirty_rmid[i];
> -		if (num_dirty == 0)
> -			return i;
> -
> -		if (cleanest_closid == ~0)
> -			cleanest_closid = i;
> -
> -		if (num_dirty < closid_num_dirty_rmid[cleanest_closid])
> -			cleanest_closid = i;
> -	}
> -
> -	if (cleanest_closid == ~0)
> -		return -ENOSPC;
> -
> -	return cleanest_closid;
> -}
> -
> -/*
> - * For MPAM the RMID value is not unique, and has to be considered with
> - * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which
> - * allows all domains to be managed by a single free list.
> - * Each domain also has a rmid_busy_llc to reduce the work of the limbo handler.
> - */
> -int alloc_rmid(u32 closid)
> -{
> -	struct rmid_entry *entry;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	entry = resctrl_find_free_rmid(closid);
> -	if (IS_ERR(entry))
> -		return PTR_ERR(entry);
> -
> -	list_del(&entry->list);
> -	return entry->rmid;
> -}
> -
> -static void add_rmid_to_limbo(struct rmid_entry *entry)
> -{
> -	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> -	struct rdt_domain *d;
> -	u32 idx;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	/* Walking r->domains, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -	idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid);
> -
> -	entry->busy = 0;
> -	list_for_each_entry(d, &r->domains, list) {
> -		/*
> -		 * For the first limbo RMID in the domain,
> -		 * setup up the limbo worker.
> -		 */
> -		if (!has_busy_rmid(d))
> -			cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL,
> -						RESCTRL_PICK_ANY_CPU);
> -		set_bit(idx, d->rmid_busy_llc);
> -		entry->busy++;
> -	}
> -
> -	rmid_limbo_count++;
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
> -		closid_num_dirty_rmid[entry->closid]++;
> -}
> -
> -void free_rmid(u32 closid, u32 rmid)
> -{
> -	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> -	struct rmid_entry *entry;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	/*
> -	 * Do not allow the default rmid to be free'd. Comparing by index
> -	 * allows architectures that ignore the closid parameter to avoid an
> -	 * unnecessary check.
> -	 */
> -	if (idx == resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
> -						RESCTRL_RESERVED_RMID))
> -		return;
> -
> -	entry = __rmid_entry(idx);
> -
> -	if (resctrl_arch_is_llc_occupancy_enabled())
> -		add_rmid_to_limbo(entry);
> -	else
> -		list_add_tail(&entry->list, &rmid_free_lru);
> -}
> -
> -static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid,
> -				       u32 rmid, enum resctrl_event_id evtid)
> -{
> -	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> -
> -	switch (evtid) {
> -	case QOS_L3_MBM_TOTAL_EVENT_ID:
> -		return &d->mbm_total[idx];
> -	case QOS_L3_MBM_LOCAL_EVENT_ID:
> -		return &d->mbm_local[idx];
> -	default:
> -		return NULL;
> -	}
> -}
> -
> -static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr)
> -{
> -	struct mbm_state *m;
> -	u64 tval = 0;
> -
> -	if (rr->first) {
> -		resctrl_arch_reset_rmid(rr->r, rr->d, closid, rmid, rr->evtid);
> -		m = get_mbm_state(rr->d, closid, rmid, rr->evtid);
> -		if (m)
> -			memset(m, 0, sizeof(struct mbm_state));
> -		return 0;
> -	}
> -
> -	rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid,
> -					 &tval, rr->arch_mon_ctx);
> -	if (rr->err)
> -		return rr->err;
> -
> -	rr->val += tval;
> -
> -	return 0;
> -}
> -
> -/*
> - * mbm_bw_count() - Update bw count from values previously read by
> - *		    __mon_event_count().
> - * @closid:	The closid used to identify the cached mbm_state.
> - * @rmid:	The rmid used to identify the cached mbm_state.
> - * @rr:		The struct rmid_read populated by __mon_event_count().
> - *
> - * Supporting function to calculate the memory bandwidth
> - * and delta bandwidth in MBps. The chunks value previously read by
> - * __mon_event_count() is compared with the chunks value from the previous
> - * invocation. This must be called once per second to maintain values in MBps.
> - */
> -static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr)
> -{
> -	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> -	struct mbm_state *m = &rr->d->mbm_local[idx];
> -	u64 cur_bw, bytes, cur_bytes;
> -
> -	cur_bytes = rr->val;
> -	bytes = cur_bytes - m->prev_bw_bytes;
> -	m->prev_bw_bytes = cur_bytes;
> -
> -	cur_bw = bytes / SZ_1M;
> -
> -	m->prev_bw = cur_bw;
> -}
> -
> -/*
> - * This is scheduled by mon_event_read() to read the CQM/MBM counters
> - * on a domain.
> - */
> -void mon_event_count(void *info)
> -{
> -	struct rdtgroup *rdtgrp, *entry;
> -	struct rmid_read *rr = info;
> -	struct list_head *head;
> -	int ret;
> -
> -	rdtgrp = rr->rgrp;
> -
> -	ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr);
> -
> -	/*
> -	 * For Ctrl groups read data from child monitor groups and
> -	 * add them together. Count events which are read successfully.
> -	 * Discard the rmid_read's reporting errors.
> -	 */
> -	head = &rdtgrp->mon.crdtgrp_list;
> -
> -	if (rdtgrp->type == RDTCTRL_GROUP) {
> -		list_for_each_entry(entry, head, mon.crdtgrp_list) {
> -			if (__mon_event_count(entry->closid, entry->mon.rmid,
> -					      rr) == 0)
> -				ret = 0;
> -		}
> -	}
> -
> -	/*
> -	 * __mon_event_count() calls for newly created monitor groups may
> -	 * report -EINVAL/Unavailable if the monitor hasn't seen any traffic.
> -	 * Discard error if any of the monitor event reads succeeded.
> -	 */
> -	if (ret == 0)
> -		rr->err = 0;
> -}
> -
> -/*
> - * Feedback loop for MBA software controller (mba_sc)
> - *
> - * mba_sc is a feedback loop where we periodically read MBM counters and
> - * adjust the bandwidth percentage values via the IA32_MBA_THRTL_MSRs so
> - * that:
> - *
> - *   current bandwidth(cur_bw) < user specified bandwidth(user_bw)
> - *
> - * This uses the MBM counters to measure the bandwidth and MBA throttle
> - * MSRs to control the bandwidth for a particular rdtgrp. It builds on the
> - * fact that resctrl rdtgroups have both monitoring and control.
> - *
> - * The frequency of the checks is 1s and we just tag along the MBM overflow
> - * timer. Having 1s interval makes the calculation of bandwidth simpler.
> - *
> - * Although MBA's goal is to restrict the bandwidth to a maximum, there may
> - * be a need to increase the bandwidth to avoid unnecessarily restricting
> - * the L2 <-> L3 traffic.
> - *
> - * Since MBA controls the L2 external bandwidth where as MBM measures the
> - * L3 external bandwidth the following sequence could lead to such a
> - * situation.
> - *
> - * Consider an rdtgroup which had high L3 <-> memory traffic in initial
> - * phases -> mba_sc kicks in and reduced bandwidth percentage values -> but
> - * after some time rdtgroup has mostly L2 <-> L3 traffic.
> - *
> - * In this case we may restrict the rdtgroup's L2 <-> L3 traffic as its
> - * throttle MSRs already have low percentage values.  To avoid
> - * unnecessarily restricting such rdtgroups, we also increase the bandwidth.
> - */
> -static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
> -{
> -	u32 closid, rmid, cur_msr_val, new_msr_val;
> -	struct mbm_state *pmbm_data, *cmbm_data;
> -	struct rdt_resource *r_mba;
> -	struct rdt_domain *dom_mba;
> -	u32 cur_bw, user_bw, idx;
> -	struct list_head *head;
> -	struct rdtgroup *entry;
> -
> -	if (!resctrl_arch_is_mbm_local_enabled())
> -		return;
> -
> -	r_mba = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> -
> -	closid = rgrp->closid;
> -	rmid = rgrp->mon.rmid;
> -	idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> -	pmbm_data = &dom_mbm->mbm_local[idx];
> -
> -	dom_mba = resctrl_get_domain_from_cpu(smp_processor_id(), r_mba);
> -	if (!dom_mba) {
> -		pr_warn_once("Failure to get domain for MBA update\n");
> -		return;
> -	}
> -
> -	cur_bw = pmbm_data->prev_bw;
> -	user_bw = dom_mba->mbps_val[closid];
> -
> -	/* MBA resource doesn't support CDP */
> -	cur_msr_val = resctrl_arch_get_config(r_mba, dom_mba, closid, CDP_NONE);
> -
> -	/*
> -	 * For Ctrl groups read data from child monitor groups.
> -	 */
> -	head = &rgrp->mon.crdtgrp_list;
> -	list_for_each_entry(entry, head, mon.crdtgrp_list) {
> -		cmbm_data = &dom_mbm->mbm_local[entry->mon.rmid];
> -		cur_bw += cmbm_data->prev_bw;
> -	}
> -
> -	/*
> -	 * Scale up/down the bandwidth linearly for the ctrl group.  The
> -	 * bandwidth step is the bandwidth granularity specified by the
> -	 * hardware.
> -	 * Always increase throttling if current bandwidth is above the
> -	 * target set by user.
> -	 * But avoid thrashing up and down on every poll by checking
> -	 * whether a decrease in throttling is likely to push the group
> -	 * back over target. E.g. if currently throttling to 30% of bandwidth
> -	 * on a system with 10% granularity steps, check whether moving to
> -	 * 40% would go past the limit by multiplying current bandwidth by
> -	 * "(30 + 10) / 30".
> -	 */
> -	if (cur_msr_val > r_mba->membw.min_bw && user_bw < cur_bw) {
> -		new_msr_val = cur_msr_val - r_mba->membw.bw_gran;
> -	} else if (cur_msr_val < MAX_MBA_BW &&
> -		   (user_bw > (cur_bw * (cur_msr_val + r_mba->membw.min_bw) / cur_msr_val))) {
> -		new_msr_val = cur_msr_val + r_mba->membw.bw_gran;
> -	} else {
> -		return;
> -	}
> -
> -	resctrl_arch_update_one(r_mba, dom_mba, closid, CDP_NONE, new_msr_val);
> -}
> -
> -static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,
> -		       u32 closid, u32 rmid)
> -{
> -	struct rmid_read rr;
> -
> -	rr.first = false;
> -	rr.r = r;
> -	rr.d = d;
> -
> -	/*
> -	 * This is protected from concurrent reads from user
> -	 * as both the user and we hold the global mutex.
> -	 */
> -	if (resctrl_arch_is_mbm_total_enabled()) {
> -		rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID;
> -		rr.val = 0;
> -		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
> -		if (IS_ERR(rr.arch_mon_ctx)) {
> -			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
> -					    PTR_ERR(rr.arch_mon_ctx));
> -			return;
> -		}
> -
> -		__mon_event_count(closid, rmid, &rr);
> -
> -		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
> -	}
> -	if (resctrl_arch_is_mbm_local_enabled()) {
> -		rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID;
> -		rr.val = 0;
> -		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
> -		if (IS_ERR(rr.arch_mon_ctx)) {
> -			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
> -					    PTR_ERR(rr.arch_mon_ctx));
> -			return;
> -		}
> -
> -		__mon_event_count(closid, rmid, &rr);
> -
> -		/*
> -		 * Call the MBA software controller only for the
> -		 * control groups and when user has enabled
> -		 * the software controller explicitly.
> -		 */
> -		if (is_mba_sc(NULL))
> -			mbm_bw_count(closid, rmid, &rr);
> -
> -		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
> -	}
> -}
> -
> -/*
> - * Handler to scan the limbo list and move the RMIDs
> - * to free list whose occupancy < threshold_occupancy.
> - */
> -void cqm_handle_limbo(struct work_struct *work)
> -{
> -	unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL);
> -	struct rdt_domain *d;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	d = container_of(work, struct rdt_domain, cqm_limbo.work);
> -
> -	__check_limbo(d, false);
> -
> -	if (has_busy_rmid(d)) {
> -		d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
> -							   RESCTRL_PICK_ANY_CPU);
> -		schedule_delayed_work_on(d->cqm_work_cpu, &d->cqm_limbo,
> -					 delay);
> -	}
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -}
> -
> -/**
> - * cqm_setup_limbo_handler() - Schedule the limbo handler to run for this
> - *                             domain.
> - * @dom:           The domain the limbo handler should run for.
> - * @delay_ms:      How far in the future the handler should run.
> - * @exclude_cpu:   Which CPU the handler should not run on,
> - *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
> - */
> -void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
> -			     int exclude_cpu)
> -{
> -	unsigned long delay = msecs_to_jiffies(delay_ms);
> -	int cpu;
> -
> -	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
> -	dom->cqm_work_cpu = cpu;
> -
> -	if (cpu < nr_cpu_ids)
> -		schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay);
> -}
> -
> -void mbm_handle_overflow(struct work_struct *work)
> -{
> -	unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL);
> -	struct rdtgroup *prgrp, *crgrp;
> -	struct list_head *head;
> -	struct rdt_resource *r;
> -	struct rdt_domain *d;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	/*
> -	 * If the filesystem has been unmounted this work no longer needs to
> -	 * run.
> -	 */
> -	if (!resctrl_mounted || !resctrl_arch_mon_capable())
> -		goto out_unlock;
> -
> -	r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> -	d = container_of(work, struct rdt_domain, mbm_over.work);
> -
> -	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> -		mbm_update(r, d, prgrp->closid, prgrp->mon.rmid);
> -
> -		head = &prgrp->mon.crdtgrp_list;
> -		list_for_each_entry(crgrp, head, mon.crdtgrp_list)
> -			mbm_update(r, d, crgrp->closid, crgrp->mon.rmid);
> -
> -		if (is_mba_sc(NULL))
> -			update_mba_bw(prgrp, d);
> -	}
> -
> -	/*
> -	 * Re-check for housekeeping CPUs. This allows the overflow handler to
> -	 * move off a nohz_full CPU quickly.
> -	 */
> -	d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
> -						   RESCTRL_PICK_ANY_CPU);
> -	schedule_delayed_work_on(d->mbm_work_cpu, &d->mbm_over, delay);
> -
> -out_unlock:
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -}
> -
> -/**
> - * mbm_setup_overflow_handler() - Schedule the overflow handler to run for this
> - *                                domain.
> - * @dom:           The domain the overflow handler should run for.
> - * @delay_ms:      How far in the future the handler should run.
> - * @exclude_cpu:   Which CPU the handler should not run on,
> - *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
> - */
> -void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms,
> -				int exclude_cpu)
> -{
> -	unsigned long delay = msecs_to_jiffies(delay_ms);
> -	int cpu;
> -
> -	/*
> -	 * When a domain comes online there is no guarantee the filesystem is
> -	 * mounted. If not, there is no need to catch counter overflow.
> -	 */
> -	if (!resctrl_mounted || !resctrl_arch_mon_capable())
> -		return;
> -	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
> -	dom->mbm_work_cpu = cpu;
> -
> -	if (cpu < nr_cpu_ids)
> -		schedule_delayed_work_on(cpu, &dom->mbm_over, delay);
> -}
> -
> -static int dom_data_init(struct rdt_resource *r)
> -{
> -	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> -	u32 num_closid = resctrl_arch_get_num_closid(r);
> -	struct rmid_entry *entry = NULL;
> -	int err = 0, i;
> -	u32 idx;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> -		u32 *tmp;
> -
> -		/*
> -		 * If the architecture hasn't provided a sanitised value here,
> -		 * this may result in larger arrays than necessary. Resctrl will
> -		 * use a smaller system wide value based on the resources in
> -		 * use.
> -		 */
> -		tmp = kcalloc(num_closid, sizeof(*tmp), GFP_KERNEL);
> -		if (!tmp) {
> -			err = -ENOMEM;
> -			goto out_unlock;
> -		}
> -
> -		closid_num_dirty_rmid = tmp;
> -	}
> -
> -	rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL);
> -	if (!rmid_ptrs) {
> -		if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> -			kfree(closid_num_dirty_rmid);
> -			closid_num_dirty_rmid = NULL;
> -		}
> -		err = -ENOMEM;
> -		goto out_unlock;
> -	}
> -
> -	for (i = 0; i < idx_limit; i++) {
> -		entry = &rmid_ptrs[i];
> -		INIT_LIST_HEAD(&entry->list);
> -
> -		resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid);
> -		list_add_tail(&entry->list, &rmid_free_lru);
> -	}
> -
> -	/*
> -	 * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and
> -	 * are always allocated. These are used for the rdtgroup_default
> -	 * control group, which will be setup later in rdtgroup_init().
> -	 */
> -	idx = resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
> -					   RESCTRL_RESERVED_RMID);
> -	entry = __rmid_entry(idx);
> -	list_del(&entry->list);
> -
> -out_unlock:
> -	mutex_unlock(&rdtgroup_mutex);
> -
> -	return err;
> -}
> -
> -static void dom_data_exit(struct rdt_resource *r)
> -{
> -	if (!r->mon_capable)
> -		return;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> -		kfree(closid_num_dirty_rmid);
> -		closid_num_dirty_rmid = NULL;
> -	}
> -
> -	kfree(rmid_ptrs);
> -	rmid_ptrs = NULL;
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -}
> -
> -static struct mon_evt llc_occupancy_event = {
> -	.name		= "llc_occupancy",
> -	.evtid		= QOS_L3_OCCUP_EVENT_ID,
> -};
> -
> -static struct mon_evt mbm_total_event = {
> -	.name		= "mbm_total_bytes",
> -	.evtid		= QOS_L3_MBM_TOTAL_EVENT_ID,
> -};
> -
> -static struct mon_evt mbm_local_event = {
> -	.name		= "mbm_local_bytes",
> -	.evtid		= QOS_L3_MBM_LOCAL_EVENT_ID,
> -};
> -
> -/*
> - * Initialize the event list for the resource.
> - *
> - * Note that MBM events are also part of RDT_RESOURCE_L3 resource
> - * because as per the SDM the total and local memory bandwidth
> - * are enumerated as part of L3 monitoring.
> - */
> -static void l3_mon_evt_init(struct rdt_resource *r)
> -{
> -	INIT_LIST_HEAD(&r->evt_list);
> -
> -	if (resctrl_arch_is_llc_occupancy_enabled())
> -		list_add_tail(&llc_occupancy_event.list, &r->evt_list);
> -	if (resctrl_arch_is_mbm_total_enabled())
> -		list_add_tail(&mbm_total_event.list, &r->evt_list);
> -	if (resctrl_arch_is_mbm_local_enabled())
> -		list_add_tail(&mbm_local_event.list, &r->evt_list);
> -}
> -
> -int resctrl_mon_resource_init(void)
> -{
> -	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> -	int ret;
> -
> -	if (!r->mon_capable)
> -		return 0;
> -
> -	ret = dom_data_init(r);
> -	if (ret)
> -		return ret;
> -
> -	l3_mon_evt_init(r);
> -
> -	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_TOTAL_EVENT_ID)) {
> -		mbm_total_event.configurable = true;
> -		mbm_config_rftype_init("mbm_total_bytes_config");
> -	}
> -	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_LOCAL_EVENT_ID)) {
> -		mbm_local_event.configurable = true;
> -		mbm_config_rftype_init("mbm_local_bytes_config");
> -	}
> -
> -	return 0;
> -}
> -
>  int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>  {
>  	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
> @@ -1076,13 +262,6 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>  	return 0;
>  }
>  
> -void resctrl_mon_resource_exit(void)
> -{
> -	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> -
> -	dom_data_exit(r);
> -}
> -
>  void __init intel_rdt_mbm_apply_quirk(void)
>  {
>  	int cf_index;
> diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> index ba51ab1f70e6..ba1596afee10 100644
> --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> @@ -39,28 +39,6 @@
>   */
>  static u64 prefetch_disable_bits;
>  
> -/*
> - * Major number assigned to and shared by all devices exposing
> - * pseudo-locked regions.
> - */
> -static unsigned int pseudo_lock_major;
> -static unsigned long pseudo_lock_minor_avail = GENMASK(MINORBITS, 0);
> -
> -static char *pseudo_lock_devnode(const struct device *dev, umode_t *mode)
> -{
> -	const struct rdtgroup *rdtgrp;
> -
> -	rdtgrp = dev_get_drvdata(dev);
> -	if (mode)
> -		*mode = 0600;
> -	return kasprintf(GFP_KERNEL, "pseudo_lock/%s", rdtgrp->kn->name);
> -}
> -
> -static const struct class pseudo_lock_class = {
> -	.name = "pseudo_lock",
> -	.devnode = pseudo_lock_devnode,
> -};
> -
>  /**
>   * resctrl_arch_get_prefetch_disable_bits - prefetch disable bits of supported
>   *                                          platforms
> @@ -121,299 +99,6 @@ u64 resctrl_arch_get_prefetch_disable_bits(void)
>  	return prefetch_disable_bits;
>  }
>  
> -/**
> - * pseudo_lock_minor_get - Obtain available minor number
> - * @minor: Pointer to where new minor number will be stored
> - *
> - * A bitmask is used to track available minor numbers. Here the next free
> - * minor number is marked as unavailable and returned.
> - *
> - * Return: 0 on success, <0 on failure.
> - */
> -static int pseudo_lock_minor_get(unsigned int *minor)
> -{
> -	unsigned long first_bit;
> -
> -	first_bit = find_first_bit(&pseudo_lock_minor_avail, MINORBITS);
> -
> -	if (first_bit == MINORBITS)
> -		return -ENOSPC;
> -
> -	__clear_bit(first_bit, &pseudo_lock_minor_avail);
> -	*minor = first_bit;
> -
> -	return 0;
> -}
> -
> -/**
> - * pseudo_lock_minor_release - Return minor number to available
> - * @minor: The minor number made available
> - */
> -static void pseudo_lock_minor_release(unsigned int minor)
> -{
> -	__set_bit(minor, &pseudo_lock_minor_avail);
> -}
> -
> -/**
> - * region_find_by_minor - Locate a pseudo-lock region by inode minor number
> - * @minor: The minor number of the device representing pseudo-locked region
> - *
> - * When the character device is accessed we need to determine which
> - * pseudo-locked region it belongs to. This is done by matching the minor
> - * number of the device to the pseudo-locked region it belongs.
> - *
> - * Minor numbers are assigned at the time a pseudo-locked region is associated
> - * with a cache instance.
> - *
> - * Return: On success return pointer to resource group owning the pseudo-locked
> - *         region, NULL on failure.
> - */
> -static struct rdtgroup *region_find_by_minor(unsigned int minor)
> -{
> -	struct rdtgroup *rdtgrp, *rdtgrp_match = NULL;
> -
> -	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
> -		if (rdtgrp->plr && rdtgrp->plr->minor == minor) {
> -			rdtgrp_match = rdtgrp;
> -			break;
> -		}
> -	}
> -	return rdtgrp_match;
> -}
> -
> -/**
> - * struct pseudo_lock_pm_req - A power management QoS request list entry
> - * @list:	Entry within the @pm_reqs list for a pseudo-locked region
> - * @req:	PM QoS request
> - */
> -struct pseudo_lock_pm_req {
> -	struct list_head list;
> -	struct dev_pm_qos_request req;
> -};
> -
> -static void pseudo_lock_cstates_relax(struct pseudo_lock_region *plr)
> -{
> -	struct pseudo_lock_pm_req *pm_req, *next;
> -
> -	list_for_each_entry_safe(pm_req, next, &plr->pm_reqs, list) {
> -		dev_pm_qos_remove_request(&pm_req->req);
> -		list_del(&pm_req->list);
> -		kfree(pm_req);
> -	}
> -}
> -
> -/**
> - * pseudo_lock_cstates_constrain - Restrict cores from entering C6
> - * @plr: Pseudo-locked region
> - *
> - * To prevent the cache from being affected by power management entering
> - * C6 has to be avoided. This is accomplished by requesting a latency
> - * requirement lower than lowest C6 exit latency of all supported
> - * platforms as found in the cpuidle state tables in the intel_idle driver.
> - * At this time it is possible to do so with a single latency requirement
> - * for all supported platforms.
> - *
> - * Since Goldmont is supported, which is affected by X86_BUG_MONITOR,
> - * the ACPI latencies need to be considered while keeping in mind that C2
> - * may be set to map to deeper sleep states. In this case the latency
> - * requirement needs to prevent entering C2 also.
> - *
> - * Return: 0 on success, <0 on failure
> - */
> -static int pseudo_lock_cstates_constrain(struct pseudo_lock_region *plr)
> -{
> -	struct pseudo_lock_pm_req *pm_req;
> -	int cpu;
> -	int ret;
> -
> -	for_each_cpu(cpu, &plr->d->cpu_mask) {
> -		pm_req = kzalloc(sizeof(*pm_req), GFP_KERNEL);
> -		if (!pm_req) {
> -			rdt_last_cmd_puts("Failure to allocate memory for PM QoS\n");
> -			ret = -ENOMEM;
> -			goto out_err;
> -		}
> -		ret = dev_pm_qos_add_request(get_cpu_device(cpu),
> -					     &pm_req->req,
> -					     DEV_PM_QOS_RESUME_LATENCY,
> -					     30);
> -		if (ret < 0) {
> -			rdt_last_cmd_printf("Failed to add latency req CPU%d\n",
> -					    cpu);
> -			kfree(pm_req);
> -			ret = -1;
> -			goto out_err;
> -		}
> -		list_add(&pm_req->list, &plr->pm_reqs);
> -	}
> -
> -	return 0;
> -
> -out_err:
> -	pseudo_lock_cstates_relax(plr);
> -	return ret;
> -}
> -
> -/**
> - * pseudo_lock_region_clear - Reset pseudo-lock region data
> - * @plr: pseudo-lock region
> - *
> - * All content of the pseudo-locked region is reset - any memory allocated
> - * freed.
> - *
> - * Return: void
> - */
> -static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)
> -{
> -	plr->size = 0;
> -	plr->line_size = 0;
> -	kfree(plr->kmem);
> -	plr->kmem = NULL;
> -	plr->s = NULL;
> -	if (plr->d)
> -		plr->d->plr = NULL;
> -	plr->d = NULL;
> -	plr->cbm = 0;
> -	plr->debugfs_dir = NULL;
> -}
> -
> -/**
> - * pseudo_lock_region_init - Initialize pseudo-lock region information
> - * @plr: pseudo-lock region
> - *
> - * Called after user provided a schemata to be pseudo-locked. From the
> - * schemata the &struct pseudo_lock_region is on entry already initialized
> - * with the resource, domain, and capacity bitmask. Here the information
> - * required for pseudo-locking is deduced from this data and &struct
> - * pseudo_lock_region initialized further. This information includes:
> - * - size in bytes of the region to be pseudo-locked
> - * - cache line size to know the stride with which data needs to be accessed
> - *   to be pseudo-locked
> - * - a cpu associated with the cache instance on which the pseudo-locking
> - *   flow can be executed
> - *
> - * Return: 0 on success, <0 on failure. Descriptive error will be written
> - * to last_cmd_status buffer.
> - */
> -static int pseudo_lock_region_init(struct pseudo_lock_region *plr)
> -{
> -	struct cpu_cacheinfo *ci;
> -	int ret;
> -	int i;
> -
> -	/* Pick the first cpu we find that is associated with the cache. */
> -	plr->cpu = cpumask_first(&plr->d->cpu_mask);
> -
> -	if (!cpu_online(plr->cpu)) {
> -		rdt_last_cmd_printf("CPU %u associated with cache not online\n",
> -				    plr->cpu);
> -		ret = -ENODEV;
> -		goto out_region;
> -	}
> -
> -	ci = get_cpu_cacheinfo(plr->cpu);
> -
> -	plr->size = rdtgroup_cbm_to_size(plr->s->res, plr->d, plr->cbm);
> -
> -	for (i = 0; i < ci->num_leaves; i++) {
> -		if (ci->info_list[i].level == plr->s->res->cache_level) {
> -			plr->line_size = ci->info_list[i].coherency_line_size;
> -			return 0;
> -		}
> -	}
> -
> -	ret = -1;
> -	rdt_last_cmd_puts("Unable to determine cache line size\n");
> -out_region:
> -	pseudo_lock_region_clear(plr);
> -	return ret;
> -}
> -
> -/**
> - * pseudo_lock_init - Initialize a pseudo-lock region
> - * @rdtgrp: resource group to which new pseudo-locked region will belong
> - *
> - * A pseudo-locked region is associated with a resource group. When this
> - * association is created the pseudo-locked region is initialized. The
> - * details of the pseudo-locked region are not known at this time so only
> - * allocation is done and association established.
> - *
> - * Return: 0 on success, <0 on failure
> - */
> -static int pseudo_lock_init(struct rdtgroup *rdtgrp)
> -{
> -	struct pseudo_lock_region *plr;
> -
> -	plr = kzalloc(sizeof(*plr), GFP_KERNEL);
> -	if (!plr)
> -		return -ENOMEM;
> -
> -	init_waitqueue_head(&plr->lock_thread_wq);
> -	INIT_LIST_HEAD(&plr->pm_reqs);
> -	rdtgrp->plr = plr;
> -	return 0;
> -}
> -
> -/**
> - * pseudo_lock_region_alloc - Allocate kernel memory that will be pseudo-locked
> - * @plr: pseudo-lock region
> - *
> - * Initialize the details required to set up the pseudo-locked region and
> - * allocate the contiguous memory that will be pseudo-locked to the cache.
> - *
> - * Return: 0 on success, <0 on failure.  Descriptive error will be written
> - * to last_cmd_status buffer.
> - */
> -static int pseudo_lock_region_alloc(struct pseudo_lock_region *plr)
> -{
> -	int ret;
> -
> -	ret = pseudo_lock_region_init(plr);
> -	if (ret < 0)
> -		return ret;
> -
> -	/*
> -	 * We do not yet support contiguous regions larger than
> -	 * KMALLOC_MAX_SIZE.
> -	 */
> -	if (plr->size > KMALLOC_MAX_SIZE) {
> -		rdt_last_cmd_puts("Requested region exceeds maximum size\n");
> -		ret = -E2BIG;
> -		goto out_region;
> -	}
> -
> -	plr->kmem = kzalloc(plr->size, GFP_KERNEL);
> -	if (!plr->kmem) {
> -		rdt_last_cmd_puts("Unable to allocate memory\n");
> -		ret = -ENOMEM;
> -		goto out_region;
> -	}
> -
> -	ret = 0;
> -	goto out;
> -out_region:
> -	pseudo_lock_region_clear(plr);
> -out:
> -	return ret;
> -}
> -
> -/**
> - * pseudo_lock_free - Free a pseudo-locked region
> - * @rdtgrp: resource group to which pseudo-locked region belonged
> - *
> - * The pseudo-locked region's resources have already been released, or not
> - * yet created at this point. Now it can be freed and disassociated from the
> - * resource group.
> - *
> - * Return: void
> - */
> -static void pseudo_lock_free(struct rdtgroup *rdtgrp)
> -{
> -	pseudo_lock_region_clear(rdtgrp->plr);
> -	kfree(rdtgrp->plr);
> -	rdtgrp->plr = NULL;
> -}
> -
>  /**
>   * resctrl_arch_pseudo_lock_fn - Load kernel memory into cache
>   * @_plr: the pseudo-lock region descriptor
> @@ -543,345 +228,6 @@ int resctrl_arch_pseudo_lock_fn(void *_plr)
>  	return 0;
>  }
>  
> -/**
> - * rdtgroup_monitor_in_progress - Test if monitoring in progress
> - * @rdtgrp: resource group being queried
> - *
> - * Return: 1 if monitor groups have been created for this resource
> - * group, 0 otherwise.
> - */
> -static int rdtgroup_monitor_in_progress(struct rdtgroup *rdtgrp)
> -{
> -	return !list_empty(&rdtgrp->mon.crdtgrp_list);
> -}
> -
> -/**
> - * rdtgroup_locksetup_user_restrict - Restrict user access to group
> - * @rdtgrp: resource group needing access restricted
> - *
> - * A resource group used for cache pseudo-locking cannot have cpus or tasks
> - * assigned to it. This is communicated to the user by restricting access
> - * to all the files that can be used to make such changes.
> - *
> - * Permissions restored with rdtgroup_locksetup_user_restore()
> - *
> - * Return: 0 on success, <0 on failure. If a failure occurs during the
> - * restriction of access an attempt will be made to restore permissions but
> - * the state of the mode of these files will be uncertain when a failure
> - * occurs.
> - */
> -static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)
> -{
> -	int ret;
> -
> -	ret = rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
> -	if (ret)
> -		return ret;
> -
> -	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
> -	if (ret)
> -		goto err_tasks;
> -
> -	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
> -	if (ret)
> -		goto err_cpus;
> -
> -	if (resctrl_arch_mon_capable()) {
> -		ret = rdtgroup_kn_mode_restrict(rdtgrp, "mon_groups");
> -		if (ret)
> -			goto err_cpus_list;
> -	}
> -
> -	ret = 0;
> -	goto out;
> -
> -err_cpus_list:
> -	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
> -err_cpus:
> -	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
> -err_tasks:
> -	rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
> -out:
> -	return ret;
> -}
> -
> -/**
> - * rdtgroup_locksetup_user_restore - Restore user access to group
> - * @rdtgrp: resource group needing access restored
> - *
> - * Restore all file access previously removed using
> - * rdtgroup_locksetup_user_restrict()
> - *
> - * Return: 0 on success, <0 on failure.  If a failure occurs during the
> - * restoration of access an attempt will be made to restrict permissions
> - * again but the state of the mode of these files will be uncertain when
> - * a failure occurs.
> - */
> -static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)
> -{
> -	int ret;
> -
> -	ret = rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
> -	if (ret)
> -		return ret;
> -
> -	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
> -	if (ret)
> -		goto err_tasks;
> -
> -	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
> -	if (ret)
> -		goto err_cpus;
> -
> -	if (resctrl_arch_mon_capable()) {
> -		ret = rdtgroup_kn_mode_restore(rdtgrp, "mon_groups", 0777);
> -		if (ret)
> -			goto err_cpus_list;
> -	}
> -
> -	ret = 0;
> -	goto out;
> -
> -err_cpus_list:
> -	rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
> -err_cpus:
> -	rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
> -err_tasks:
> -	rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
> -out:
> -	return ret;
> -}
> -
> -/**
> - * rdtgroup_locksetup_enter - Resource group enters locksetup mode
> - * @rdtgrp: resource group requested to enter locksetup mode
> - *
> - * A resource group enters locksetup mode to reflect that it would be used
> - * to represent a pseudo-locked region and is in the process of being set
> - * up to do so. A resource group used for a pseudo-locked region would
> - * lose the closid associated with it so we cannot allow it to have any
> - * tasks or cpus assigned nor permit tasks or cpus to be assigned in the
> - * future. Monitoring of a pseudo-locked region is not allowed either.
> - *
> - * The above and more restrictions on a pseudo-locked region are checked
> - * for and enforced before the resource group enters the locksetup mode.
> - *
> - * Returns: 0 if the resource group successfully entered locksetup mode, <0
> - * on failure. On failure the last_cmd_status buffer is updated with text to
> - * communicate details of failure to the user.
> - */
> -int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
> -{
> -	int ret;
> -
> -	/*
> -	 * The default resource group can neither be removed nor lose the
> -	 * default closid associated with it.
> -	 */
> -	if (rdtgrp == &rdtgroup_default) {
> -		rdt_last_cmd_puts("Cannot pseudo-lock default group\n");
> -		return -EINVAL;
> -	}
> -
> -	/*
> -	 * Cache Pseudo-locking not supported when CDP is enabled.
> -	 *
> -	 * Some things to consider if you would like to enable this
> -	 * support (using L3 CDP as example):
> -	 * - When CDP is enabled two separate resources are exposed,
> -	 *   L3DATA and L3CODE, but they are actually on the same cache.
> -	 *   The implication for pseudo-locking is that if a
> -	 *   pseudo-locked region is created on a domain of one
> -	 *   resource (eg. L3CODE), then a pseudo-locked region cannot
> -	 *   be created on that same domain of the other resource
> -	 *   (eg. L3DATA). This is because the creation of a
> -	 *   pseudo-locked region involves a call to wbinvd that will
> -	 *   affect all cache allocations on particular domain.
> -	 * - Considering the previous, it may be possible to only
> -	 *   expose one of the CDP resources to pseudo-locking and
> -	 *   hide the other. For example, we could consider to only
> -	 *   expose L3DATA and since the L3 cache is unified it is
> -	 *   still possible to place instructions there are execute it.
> -	 * - If only one region is exposed to pseudo-locking we should
> -	 *   still keep in mind that availability of a portion of cache
> -	 *   for pseudo-locking should take into account both resources.
> -	 *   Similarly, if a pseudo-locked region is created in one
> -	 *   resource, the portion of cache used by it should be made
> -	 *   unavailable to all future allocations from both resources.
> -	 */
> -	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3) ||
> -	    resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2)) {
> -		rdt_last_cmd_puts("CDP enabled\n");
> -		return -EINVAL;
> -	}
> -
> -	/*
> -	 * Not knowing the bits to disable prefetching implies that this
> -	 * platform does not support Cache Pseudo-Locking.
> -	 */
> -	if (resctrl_arch_get_prefetch_disable_bits() == 0) {
> -		rdt_last_cmd_puts("Pseudo-locking not supported\n");
> -		return -EINVAL;
> -	}
> -
> -	if (rdtgroup_monitor_in_progress(rdtgrp)) {
> -		rdt_last_cmd_puts("Monitoring in progress\n");
> -		return -EINVAL;
> -	}
> -
> -	if (rdtgroup_tasks_assigned(rdtgrp)) {
> -		rdt_last_cmd_puts("Tasks assigned to resource group\n");
> -		return -EINVAL;
> -	}
> -
> -	if (!cpumask_empty(&rdtgrp->cpu_mask)) {
> -		rdt_last_cmd_puts("CPUs assigned to resource group\n");
> -		return -EINVAL;
> -	}
> -
> -	if (rdtgroup_locksetup_user_restrict(rdtgrp)) {
> -		rdt_last_cmd_puts("Unable to modify resctrl permissions\n");
> -		return -EIO;
> -	}
> -
> -	ret = pseudo_lock_init(rdtgrp);
> -	if (ret) {
> -		rdt_last_cmd_puts("Unable to init pseudo-lock region\n");
> -		goto out_release;
> -	}
> -
> -	/*
> -	 * If this system is capable of monitoring a rmid would have been
> -	 * allocated when the control group was created. This is not needed
> -	 * anymore when this group would be used for pseudo-locking. This
> -	 * is safe to call on platforms not capable of monitoring.
> -	 */
> -	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> -
> -	ret = 0;
> -	goto out;
> -
> -out_release:
> -	rdtgroup_locksetup_user_restore(rdtgrp);
> -out:
> -	return ret;
> -}
> -
> -/**
> - * rdtgroup_locksetup_exit - resource group exist locksetup mode
> - * @rdtgrp: resource group
> - *
> - * When a resource group exits locksetup mode the earlier restrictions are
> - * lifted.
> - *
> - * Return: 0 on success, <0 on failure
> - */
> -int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
> -{
> -	int ret;
> -
> -	if (resctrl_arch_mon_capable()) {
> -		ret = alloc_rmid(rdtgrp->closid);
> -		if (ret < 0) {
> -			rdt_last_cmd_puts("Out of RMIDs\n");
> -			return ret;
> -		}
> -		rdtgrp->mon.rmid = ret;
> -	}
> -
> -	ret = rdtgroup_locksetup_user_restore(rdtgrp);
> -	if (ret) {
> -		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> -		return ret;
> -	}
> -
> -	pseudo_lock_free(rdtgrp);
> -	return 0;
> -}
> -
> -/**
> - * rdtgroup_cbm_overlaps_pseudo_locked - Test if CBM or portion is pseudo-locked
> - * @d: RDT domain
> - * @cbm: CBM to test
> - *
> - * @d represents a cache instance and @cbm a capacity bitmask that is
> - * considered for it. Determine if @cbm overlaps with any existing
> - * pseudo-locked region on @d.
> - *
> - * @cbm is unsigned long, even if only 32 bits are used, to make the
> - * bitmap functions work correctly.
> - *
> - * Return: true if @cbm overlaps with pseudo-locked region on @d, false
> - * otherwise.
> - */
> -bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
> -{
> -	unsigned int cbm_len;
> -	unsigned long cbm_b;
> -
> -	if (d->plr) {
> -		cbm_len = d->plr->s->res->cache.cbm_len;
> -		cbm_b = d->plr->cbm;
> -		if (bitmap_intersects(&cbm, &cbm_b, cbm_len))
> -			return true;
> -	}
> -	return false;
> -}
> -
> -/**
> - * rdtgroup_pseudo_locked_in_hierarchy - Pseudo-locked region in cache hierarchy
> - * @d: RDT domain under test
> - *
> - * The setup of a pseudo-locked region affects all cache instances within
> - * the hierarchy of the region. It is thus essential to know if any
> - * pseudo-locked regions exist within a cache hierarchy to prevent any
> - * attempts to create new pseudo-locked regions in the same hierarchy.
> - *
> - * Return: true if a pseudo-locked region exists in the hierarchy of @d or
> - *         if it is not possible to test due to memory allocation issue,
> - *         false otherwise.
> - */
> -bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
> -{
> -	cpumask_var_t cpu_with_psl;
> -	enum resctrl_res_level i;
> -	struct rdt_resource *r;
> -	struct rdt_domain *d_i;
> -	bool ret = false;
> -
> -	/* Walking r->domains, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -	if (!zalloc_cpumask_var(&cpu_with_psl, GFP_KERNEL))
> -		return true;
> -
> -	/*
> -	 * First determine which cpus have pseudo-locked regions
> -	 * associated with them.
> -	 */
> -	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> -		r = resctrl_arch_get_resource(i);
> -		if (!r->alloc_capable)
> -			continue;
> -
> -		list_for_each_entry(d_i, &r->domains, list) {
> -			if (d_i->plr)
> -				cpumask_or(cpu_with_psl, cpu_with_psl,
> -					   &d_i->cpu_mask);
> -		}
> -	}
> -
> -	/*
> -	 * Next test if new pseudo-locked region would intersect with
> -	 * existing region.
> -	 */
> -	if (cpumask_intersects(&d->cpu_mask, cpu_with_psl))
> -		ret = true;
> -
> -	free_cpumask_var(cpu_with_psl);
> -	return ret;
> -}
> -
>  /**
>   * resctrl_arch_measure_cycles_lat_fn - Measure cycle latency to read
>   *                                      pseudo-locked memory
> @@ -1174,442 +520,3 @@ int resctrl_arch_measure_l3_residency(void *_plr)
>  	wake_up_interruptible(&plr->lock_thread_wq);
>  	return 0;
>  }
> -
> -/**
> - * pseudo_lock_measure_cycles - Trigger latency measure to pseudo-locked region
> - * @rdtgrp: Resource group to which the pseudo-locked region belongs.
> - * @sel: Selector of which measurement to perform on a pseudo-locked region.
> - *
> - * The measurement of latency to access a pseudo-locked region should be
> - * done from a cpu that is associated with that pseudo-locked region.
> - * Determine which cpu is associated with this region and start a thread on
> - * that cpu to perform the measurement, wait for that thread to complete.
> - *
> - * Return: 0 on success, <0 on failure
> - */
> -static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel)
> -{
> -	struct pseudo_lock_region *plr = rdtgrp->plr;
> -	struct task_struct *thread;
> -	unsigned int cpu;
> -	int ret = -1;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	if (rdtgrp->flags & RDT_DELETED) {
> -		ret = -ENODEV;
> -		goto out;
> -	}
> -
> -	if (!plr->d) {
> -		ret = -ENODEV;
> -		goto out;
> -	}
> -
> -	plr->thread_done = 0;
> -	cpu = cpumask_first(&plr->d->cpu_mask);
> -	if (!cpu_online(cpu)) {
> -		ret = -ENODEV;
> -		goto out;
> -	}
> -
> -	plr->cpu = cpu;
> -
> -	if (sel == 1)
> -		thread = kthread_create_on_node(resctrl_arch_measure_cycles_lat_fn,
> -						plr, cpu_to_node(cpu),
> -						"pseudo_lock_measure/%u",
> -						cpu);
> -	else if (sel == 2)
> -		thread = kthread_create_on_node(resctrl_arch_measure_l2_residency,
> -						plr, cpu_to_node(cpu),
> -						"pseudo_lock_measure/%u",
> -						cpu);
> -	else if (sel == 3)
> -		thread = kthread_create_on_node(resctrl_arch_measure_l3_residency,
> -						plr, cpu_to_node(cpu),
> -						"pseudo_lock_measure/%u",
> -						cpu);
> -	else
> -		goto out;
> -
> -	if (IS_ERR(thread)) {
> -		ret = PTR_ERR(thread);
> -		goto out;
> -	}
> -	kthread_bind(thread, cpu);
> -	wake_up_process(thread);
> -
> -	ret = wait_event_interruptible(plr->lock_thread_wq,
> -				       plr->thread_done == 1);
> -	if (ret < 0)
> -		goto out;
> -
> -	ret = 0;
> -
> -out:
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -	return ret;
> -}
> -
> -static ssize_t pseudo_lock_measure_trigger(struct file *file,
> -					   const char __user *user_buf,
> -					   size_t count, loff_t *ppos)
> -{
> -	struct rdtgroup *rdtgrp = file->private_data;
> -	size_t buf_size;
> -	char buf[32];
> -	int ret;
> -	int sel;
> -
> -	buf_size = min(count, (sizeof(buf) - 1));
> -	if (copy_from_user(buf, user_buf, buf_size))
> -		return -EFAULT;
> -
> -	buf[buf_size] = '\0';
> -	ret = kstrtoint(buf, 10, &sel);
> -	if (ret == 0) {
> -		if (sel != 1 && sel != 2 && sel != 3)
> -			return -EINVAL;
> -		ret = debugfs_file_get(file->f_path.dentry);
> -		if (ret)
> -			return ret;
> -		ret = pseudo_lock_measure_cycles(rdtgrp, sel);
> -		if (ret == 0)
> -			ret = count;
> -		debugfs_file_put(file->f_path.dentry);
> -	}
> -
> -	return ret;
> -}
> -
> -static const struct file_operations pseudo_measure_fops = {
> -	.write = pseudo_lock_measure_trigger,
> -	.open = simple_open,
> -	.llseek = default_llseek,
> -};
> -
> -/**
> - * rdtgroup_pseudo_lock_create - Create a pseudo-locked region
> - * @rdtgrp: resource group to which pseudo-lock region belongs
> - *
> - * Called when a resource group in the pseudo-locksetup mode receives a
> - * valid schemata that should be pseudo-locked. Since the resource group is
> - * in pseudo-locksetup mode the &struct pseudo_lock_region has already been
> - * allocated and initialized with the essential information. If a failure
> - * occurs the resource group remains in the pseudo-locksetup mode with the
> - * &struct pseudo_lock_region associated with it, but cleared from all
> - * information and ready for the user to re-attempt pseudo-locking by
> - * writing the schemata again.
> - *
> - * Return: 0 if the pseudo-locked region was successfully pseudo-locked, <0
> - * on failure. Descriptive error will be written to last_cmd_status buffer.
> - */
> -int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
> -{
> -	struct pseudo_lock_region *plr = rdtgrp->plr;
> -	struct task_struct *thread;
> -	unsigned int new_minor;
> -	struct device *dev;
> -	int ret;
> -
> -	ret = pseudo_lock_region_alloc(plr);
> -	if (ret < 0)
> -		return ret;
> -
> -	ret = pseudo_lock_cstates_constrain(plr);
> -	if (ret < 0) {
> -		ret = -EINVAL;
> -		goto out_region;
> -	}
> -
> -	plr->thread_done = 0;
> -
> -	plr->closid = rdtgrp->closid;
> -	thread = kthread_create_on_node(resctrl_arch_pseudo_lock_fn, plr,
> -					cpu_to_node(plr->cpu),
> -					"pseudo_lock/%u", plr->cpu);
> -	if (IS_ERR(thread)) {
> -		ret = PTR_ERR(thread);
> -		rdt_last_cmd_printf("Locking thread returned error %d\n", ret);
> -		goto out_cstates;
> -	}
> -
> -	kthread_bind(thread, plr->cpu);
> -	wake_up_process(thread);
> -
> -	ret = wait_event_interruptible(plr->lock_thread_wq,
> -				       plr->thread_done == 1);
> -	if (ret < 0) {
> -		/*
> -		 * If the thread does not get on the CPU for whatever
> -		 * reason and the process which sets up the region is
> -		 * interrupted then this will leave the thread in runnable
> -		 * state and once it gets on the CPU it will dereference
> -		 * the cleared, but not freed, plr struct resulting in an
> -		 * empty pseudo-locking loop.
> -		 */
> -		rdt_last_cmd_puts("Locking thread interrupted\n");
> -		goto out_cstates;
> -	}
> -
> -	ret = pseudo_lock_minor_get(&new_minor);
> -	if (ret < 0) {
> -		rdt_last_cmd_puts("Unable to obtain a new minor number\n");
> -		goto out_cstates;
> -	}
> -
> -	/*
> -	 * Unlock access but do not release the reference. The
> -	 * pseudo-locked region will still be here on return.
> -	 *
> -	 * The mutex has to be released temporarily to avoid a potential
> -	 * deadlock with the mm->mmap_lock which is obtained in the
> -	 * device_create() and debugfs_create_dir() callpath below as well as
> -	 * before the mmap() callback is called.
> -	 */
> -	mutex_unlock(&rdtgroup_mutex);
> -
> -	if (!IS_ERR_OR_NULL(debugfs_resctrl)) {
> -		plr->debugfs_dir = debugfs_create_dir(rdtgrp->kn->name,
> -						      debugfs_resctrl);
> -		if (!IS_ERR_OR_NULL(plr->debugfs_dir))
> -			debugfs_create_file("pseudo_lock_measure", 0200,
> -					    plr->debugfs_dir, rdtgrp,
> -					    &pseudo_measure_fops);
> -	}
> -
> -	dev = device_create(&pseudo_lock_class, NULL,
> -			    MKDEV(pseudo_lock_major, new_minor),
> -			    rdtgrp, "%s", rdtgrp->kn->name);
> -
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	if (IS_ERR(dev)) {
> -		ret = PTR_ERR(dev);
> -		rdt_last_cmd_printf("Failed to create character device: %d\n",
> -				    ret);
> -		goto out_debugfs;
> -	}
> -
> -	/* We released the mutex - check if group was removed while we did so */
> -	if (rdtgrp->flags & RDT_DELETED) {
> -		ret = -ENODEV;
> -		goto out_device;
> -	}
> -
> -	plr->minor = new_minor;
> -
> -	rdtgrp->mode = RDT_MODE_PSEUDO_LOCKED;
> -	closid_free(rdtgrp->closid);
> -	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0444);
> -	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0444);
> -
> -	ret = 0;
> -	goto out;
> -
> -out_device:
> -	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, new_minor));
> -out_debugfs:
> -	debugfs_remove_recursive(plr->debugfs_dir);
> -	pseudo_lock_minor_release(new_minor);
> -out_cstates:
> -	pseudo_lock_cstates_relax(plr);
> -out_region:
> -	pseudo_lock_region_clear(plr);
> -out:
> -	return ret;
> -}
> -
> -/**
> - * rdtgroup_pseudo_lock_remove - Remove a pseudo-locked region
> - * @rdtgrp: resource group to which the pseudo-locked region belongs
> - *
> - * The removal of a pseudo-locked region can be initiated when the resource
> - * group is removed from user space via a "rmdir" from userspace or the
> - * unmount of the resctrl filesystem. On removal the resource group does
> - * not go back to pseudo-locksetup mode before it is removed, instead it is
> - * removed directly. There is thus asymmetry with the creation where the
> - * &struct pseudo_lock_region is removed here while it was not created in
> - * rdtgroup_pseudo_lock_create().
> - *
> - * Return: void
> - */
> -void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp)
> -{
> -	struct pseudo_lock_region *plr = rdtgrp->plr;
> -
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -		/*
> -		 * Default group cannot be a pseudo-locked region so we can
> -		 * free closid here.
> -		 */
> -		closid_free(rdtgrp->closid);
> -		goto free;
> -	}
> -
> -	pseudo_lock_cstates_relax(plr);
> -	debugfs_remove_recursive(rdtgrp->plr->debugfs_dir);
> -	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, plr->minor));
> -	pseudo_lock_minor_release(plr->minor);
> -
> -free:
> -	pseudo_lock_free(rdtgrp);
> -}
> -
> -static int pseudo_lock_dev_open(struct inode *inode, struct file *filp)
> -{
> -	struct rdtgroup *rdtgrp;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	rdtgrp = region_find_by_minor(iminor(inode));
> -	if (!rdtgrp) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -ENODEV;
> -	}
> -
> -	filp->private_data = rdtgrp;
> -	atomic_inc(&rdtgrp->waitcount);
> -	/* Perform a non-seekable open - llseek is not supported */
> -	filp->f_mode &= ~(FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE);
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -
> -	return 0;
> -}
> -
> -static int pseudo_lock_dev_release(struct inode *inode, struct file *filp)
> -{
> -	struct rdtgroup *rdtgrp;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -	rdtgrp = filp->private_data;
> -	WARN_ON(!rdtgrp);
> -	if (!rdtgrp) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -ENODEV;
> -	}
> -	filp->private_data = NULL;
> -	atomic_dec(&rdtgrp->waitcount);
> -	mutex_unlock(&rdtgroup_mutex);
> -	return 0;
> -}
> -
> -static int pseudo_lock_dev_mremap(struct vm_area_struct *area)
> -{
> -	/* Not supported */
> -	return -EINVAL;
> -}
> -
> -static const struct vm_operations_struct pseudo_mmap_ops = {
> -	.mremap = pseudo_lock_dev_mremap,
> -};
> -
> -static int pseudo_lock_dev_mmap(struct file *filp, struct vm_area_struct *vma)
> -{
> -	unsigned long vsize = vma->vm_end - vma->vm_start;
> -	unsigned long off = vma->vm_pgoff << PAGE_SHIFT;
> -	struct pseudo_lock_region *plr;
> -	struct rdtgroup *rdtgrp;
> -	unsigned long physical;
> -	unsigned long psize;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	rdtgrp = filp->private_data;
> -	WARN_ON(!rdtgrp);
> -	if (!rdtgrp) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -ENODEV;
> -	}
> -
> -	plr = rdtgrp->plr;
> -
> -	if (!plr->d) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -ENODEV;
> -	}
> -
> -	/*
> -	 * Task is required to run with affinity to the cpus associated
> -	 * with the pseudo-locked region. If this is not the case the task
> -	 * may be scheduled elsewhere and invalidate entries in the
> -	 * pseudo-locked region.
> -	 */
> -	if (!cpumask_subset(current->cpus_ptr, &plr->d->cpu_mask)) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -EINVAL;
> -	}
> -
> -	physical = __pa(plr->kmem) >> PAGE_SHIFT;
> -	psize = plr->size - off;
> -
> -	if (off > plr->size) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -ENOSPC;
> -	}
> -
> -	/*
> -	 * Ensure changes are carried directly to the memory being mapped,
> -	 * do not allow copy-on-write mapping.
> -	 */
> -	if (!(vma->vm_flags & VM_SHARED)) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -EINVAL;
> -	}
> -
> -	if (vsize > psize) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -ENOSPC;
> -	}
> -
> -	memset(plr->kmem + off, 0, vsize);
> -
> -	if (remap_pfn_range(vma, vma->vm_start, physical + vma->vm_pgoff,
> -			    vsize, vma->vm_page_prot)) {
> -		mutex_unlock(&rdtgroup_mutex);
> -		return -EAGAIN;
> -	}
> -	vma->vm_ops = &pseudo_mmap_ops;
> -	mutex_unlock(&rdtgroup_mutex);
> -	return 0;
> -}
> -
> -static const struct file_operations pseudo_lock_dev_fops = {
> -	.owner =	THIS_MODULE,
> -	.llseek =	no_llseek,
> -	.read =		NULL,
> -	.write =	NULL,
> -	.open =		pseudo_lock_dev_open,
> -	.release =	pseudo_lock_dev_release,
> -	.mmap =		pseudo_lock_dev_mmap,
> -};
> -
> -int rdt_pseudo_lock_init(void)
> -{
> -	int ret;
> -
> -	ret = register_chrdev(0, "pseudo_lock", &pseudo_lock_dev_fops);
> -	if (ret < 0)
> -		return ret;
> -
> -	pseudo_lock_major = ret;
> -
> -	ret = class_register(&pseudo_lock_class);
> -	if (ret) {
> -		unregister_chrdev(pseudo_lock_major, "pseudo_lock");
> -		return ret;
> -	}
> -
> -	return 0;
> -}
> -
> -void rdt_pseudo_lock_release(void)
> -{
> -	class_unregister(&pseudo_lock_class);
> -	unregister_chrdev(pseudo_lock_major, "pseudo_lock");
> -	pseudo_lock_major = 0;
> -}
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 1425a33d201d..fe3952514add 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -12,22 +12,8 @@
>  
>  #define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
>  
> -#include <linux/cacheinfo.h>
>  #include <linux/cpu.h>
> -#include <linux/debugfs.h>
> -#include <linux/fs.h>
> -#include <linux/fs_parser.h>
> -#include <linux/sysfs.h>
> -#include <linux/kernfs.h>
> -#include <linux/seq_buf.h>
> -#include <linux/seq_file.h>
> -#include <linux/sched/signal.h>
> -#include <linux/sched/task.h>
>  #include <linux/slab.h>
> -#include <linux/task_work.h>
> -#include <linux/user_namespace.h>
> -
> -#include <uapi/linux/magic.h>
>  
>  #include <asm/resctrl.h>
>  #include "internal.h"
> @@ -36,328 +22,6 @@ DEFINE_STATIC_KEY_FALSE(rdt_enable_key);
>  DEFINE_STATIC_KEY_FALSE(rdt_mon_enable_key);
>  DEFINE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
>  
> -/* Mutex to protect rdtgroup access. */
> -DEFINE_MUTEX(rdtgroup_mutex);
> -
> -static struct kernfs_root *rdt_root;
> -struct rdtgroup rdtgroup_default;
> -LIST_HEAD(rdt_all_groups);
> -
> -/* list of entries for the schemata file */
> -LIST_HEAD(resctrl_schema_all);
> -
> -/* The filesystem can only be mounted once. */
> -bool resctrl_mounted;
> -
> -/* Kernel fs node for "info" directory under root */
> -static struct kernfs_node *kn_info;
> -
> -/* Kernel fs node for "mon_groups" directory under root */
> -static struct kernfs_node *kn_mongrp;
> -
> -/* Kernel fs node for "mon_data" directory under root */
> -static struct kernfs_node *kn_mondata;
> -
> -/*
> - * Used to store the max resource name width and max resource data width
> - * to display the schemata in a tabular format
> - */
> -int max_name_width, max_data_width;
> -
> -static struct seq_buf last_cmd_status;
> -static char last_cmd_status_buf[512];
> -
> -static int rdtgroup_setup_root(struct rdt_fs_context *ctx);
> -static void rdtgroup_destroy_root(void);
> -
> -struct dentry *debugfs_resctrl;
> -
> -static bool resctrl_debug;
> -
> -void rdt_last_cmd_clear(void)
> -{
> -	lockdep_assert_held(&rdtgroup_mutex);
> -	seq_buf_clear(&last_cmd_status);
> -}
> -
> -void rdt_last_cmd_puts(const char *s)
> -{
> -	lockdep_assert_held(&rdtgroup_mutex);
> -	seq_buf_puts(&last_cmd_status, s);
> -}
> -
> -void rdt_last_cmd_printf(const char *fmt, ...)
> -{
> -	va_list ap;
> -
> -	va_start(ap, fmt);
> -	lockdep_assert_held(&rdtgroup_mutex);
> -	seq_buf_vprintf(&last_cmd_status, fmt, ap);
> -	va_end(ap);
> -}
> -
> -void rdt_staged_configs_clear(void)
> -{
> -	enum resctrl_res_level i;
> -	struct rdt_resource *r;
> -	struct rdt_domain *dom;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> -		r = resctrl_arch_get_resource(i);
> -		if (!r->alloc_capable)
> -			continue;
> -
> -		list_for_each_entry(dom, &r->domains, list)
> -			memset(dom->staged_config, 0, sizeof(dom->staged_config));
> -	}
> -}
> -
> -static bool resctrl_is_mbm_enabled(void)
> -{
> -	return (resctrl_arch_is_mbm_total_enabled() ||
> -		resctrl_arch_is_mbm_local_enabled());
> -}
> -
> -static bool resctrl_is_mbm_event(int e)
> -{
> -	return (e >= QOS_L3_MBM_TOTAL_EVENT_ID &&
> -		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
> -}
> -
> -/*
> - * Trivial allocator for CLOSIDs. Since h/w only supports a small number,
> - * we can keep a bitmap of free CLOSIDs in a single integer.
> - *
> - * Using a global CLOSID across all resources has some advantages and
> - * some drawbacks:
> - * + We can simply set current's closid to assign a task to a resource
> - *   group.
> - * + Context switch code can avoid extra memory references deciding which
> - *   CLOSID to load into the PQR_ASSOC MSR
> - * - We give up some options in configuring resource groups across multi-socket
> - *   systems.
> - * - Our choices on how to configure each resource become progressively more
> - *   limited as the number of resources grows.
> - */
> -static unsigned long closid_free_map;
> -static int closid_free_map_len;
> -
> -int closids_supported(void)
> -{
> -	return closid_free_map_len;
> -}
> -
> -static void closid_init(void)
> -{
> -	struct resctrl_schema *s;
> -	u32 rdt_min_closid = 32;
> -
> -	/* Compute rdt_min_closid across all resources */
> -	list_for_each_entry(s, &resctrl_schema_all, list)
> -		rdt_min_closid = min(rdt_min_closid, s->num_closid);
> -
> -	closid_free_map = BIT_MASK(rdt_min_closid) - 1;
> -
> -	/* RESCTRL_RESERVED_CLOSID is always reserved for the default group */
> -	__clear_bit(RESCTRL_RESERVED_CLOSID, &closid_free_map);
> -	closid_free_map_len = rdt_min_closid;
> -}
> -
> -static int closid_alloc(void)
> -{
> -	int cleanest_closid;
> -	u32 closid;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> -	    resctrl_arch_is_llc_occupancy_enabled()) {
> -		cleanest_closid = resctrl_find_cleanest_closid();
> -		if (cleanest_closid < 0)
> -			return cleanest_closid;
> -		closid = cleanest_closid;
> -	} else {
> -		closid = ffs(closid_free_map);
> -		if (closid == 0)
> -			return -ENOSPC;
> -		closid--;
> -	}
> -	__clear_bit(closid, &closid_free_map);
> -
> -	return closid;
> -}
> -
> -void closid_free(int closid)
> -{
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	__set_bit(closid, &closid_free_map);
> -}
> -
> -/**
> - * closid_allocated - test if provided closid is in use
> - * @closid: closid to be tested
> - *
> - * Return: true if @closid is currently associated with a resource group,
> - * false if @closid is free
> - */
> -bool closid_allocated(unsigned int closid)
> -{
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	return !test_bit(closid, &closid_free_map);
> -}
> -
> -/**
> - * rdtgroup_mode_by_closid - Return mode of resource group with closid
> - * @closid: closid if the resource group
> - *
> - * Each resource group is associated with a @closid. Here the mode
> - * of a resource group can be queried by searching for it using its closid.
> - *
> - * Return: mode as &enum rdtgrp_mode of resource group with closid @closid
> - */
> -enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)
> -{
> -	struct rdtgroup *rdtgrp;
> -
> -	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
> -		if (rdtgrp->closid == closid)
> -			return rdtgrp->mode;
> -	}
> -
> -	return RDT_NUM_MODES;
> -}
> -
> -static const char * const rdt_mode_str[] = {
> -	[RDT_MODE_SHAREABLE]		= "shareable",
> -	[RDT_MODE_EXCLUSIVE]		= "exclusive",
> -	[RDT_MODE_PSEUDO_LOCKSETUP]	= "pseudo-locksetup",
> -	[RDT_MODE_PSEUDO_LOCKED]	= "pseudo-locked",
> -};
> -
> -/**
> - * rdtgroup_mode_str - Return the string representation of mode
> - * @mode: the resource group mode as &enum rdtgroup_mode
> - *
> - * Return: string representation of valid mode, "unknown" otherwise
> - */
> -static const char *rdtgroup_mode_str(enum rdtgrp_mode mode)
> -{
> -	if (mode < RDT_MODE_SHAREABLE || mode >= RDT_NUM_MODES)
> -		return "unknown";
> -
> -	return rdt_mode_str[mode];
> -}
> -
> -/* set uid and gid of rdtgroup dirs and files to that of the creator */
> -static int rdtgroup_kn_set_ugid(struct kernfs_node *kn)
> -{
> -	struct iattr iattr = { .ia_valid = ATTR_UID | ATTR_GID,
> -				.ia_uid = current_fsuid(),
> -				.ia_gid = current_fsgid(), };
> -
> -	if (uid_eq(iattr.ia_uid, GLOBAL_ROOT_UID) &&
> -	    gid_eq(iattr.ia_gid, GLOBAL_ROOT_GID))
> -		return 0;
> -
> -	return kernfs_setattr(kn, &iattr);
> -}
> -
> -static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
> -{
> -	struct kernfs_node *kn;
> -	int ret;
> -
> -	kn = __kernfs_create_file(parent_kn, rft->name, rft->mode,
> -				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID,
> -				  0, rft->kf_ops, rft, NULL, NULL);
> -	if (IS_ERR(kn))
> -		return PTR_ERR(kn);
> -
> -	ret = rdtgroup_kn_set_ugid(kn);
> -	if (ret) {
> -		kernfs_remove(kn);
> -		return ret;
> -	}
> -
> -	return 0;
> -}
> -
> -static int rdtgroup_seqfile_show(struct seq_file *m, void *arg)
> -{
> -	struct kernfs_open_file *of = m->private;
> -	struct rftype *rft = of->kn->priv;
> -
> -	if (rft->seq_show)
> -		return rft->seq_show(of, m, arg);
> -	return 0;
> -}
> -
> -static ssize_t rdtgroup_file_write(struct kernfs_open_file *of, char *buf,
> -				   size_t nbytes, loff_t off)
> -{
> -	struct rftype *rft = of->kn->priv;
> -
> -	if (rft->write)
> -		return rft->write(of, buf, nbytes, off);
> -
> -	return -EINVAL;
> -}
> -
> -static const struct kernfs_ops rdtgroup_kf_single_ops = {
> -	.atomic_write_len	= PAGE_SIZE,
> -	.write			= rdtgroup_file_write,
> -	.seq_show		= rdtgroup_seqfile_show,
> -};
> -
> -static const struct kernfs_ops kf_mondata_ops = {
> -	.atomic_write_len	= PAGE_SIZE,
> -	.seq_show		= rdtgroup_mondata_show,
> -};
> -
> -static bool is_cpu_list(struct kernfs_open_file *of)
> -{
> -	struct rftype *rft = of->kn->priv;
> -
> -	return rft->flags & RFTYPE_FLAGS_CPUS_LIST;
> -}
> -
> -static int rdtgroup_cpus_show(struct kernfs_open_file *of,
> -			      struct seq_file *s, void *v)
> -{
> -	struct rdtgroup *rdtgrp;
> -	struct cpumask *mask;
> -	int ret = 0;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -
> -	if (rdtgrp) {
> -		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> -			if (!rdtgrp->plr->d) {
> -				rdt_last_cmd_clear();
> -				rdt_last_cmd_puts("Cache domain offline\n");
> -				ret = -ENODEV;
> -			} else {
> -				mask = &rdtgrp->plr->d->cpu_mask;
> -				seq_printf(s, is_cpu_list(of) ?
> -					   "%*pbl\n" : "%*pb\n",
> -					   cpumask_pr_args(mask));
> -			}
> -		} else {
> -			seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
> -				   cpumask_pr_args(&rdtgrp->cpu_mask));
> -		}
> -	} else {
> -		ret = -ENOENT;
> -	}
> -	rdtgroup_kn_unlock(of->kn);
> -
> -	return ret;
> -}
> -
>  /*
>   * This is safe against resctrl_arch_sched_in() called from __switch_to()
>   * because __switch_to() is executed with interrupts disabled. A local call
> @@ -381,1206 +45,6 @@ void resctrl_arch_sync_cpu_defaults(void *info)
>  	resctrl_arch_sched_in(current);
>  }
>  
> -/*
> - * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
> - *
> - * Per task closids/rmids must have been set up before calling this function.
> - * @r may be NULL.
> - */
> -static void
> -update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
> -{
> -	struct resctrl_cpu_sync defaults;
> -	struct resctrl_cpu_sync *defaults_p = NULL;
> -
> -	if (r) {
> -		defaults.closid = r->closid;
> -		defaults.rmid = r->mon.rmid;
> -		defaults_p = &defaults;
> -	}
> -
> -	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
> -			 1);
> -}
> -
> -static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
> -			  cpumask_var_t tmpmask)
> -{
> -	struct rdtgroup *prgrp = rdtgrp->mon.parent, *crgrp;
> -	struct list_head *head;
> -
> -	/* Check whether cpus belong to parent ctrl group */
> -	cpumask_andnot(tmpmask, newmask, &prgrp->cpu_mask);
> -	if (!cpumask_empty(tmpmask)) {
> -		rdt_last_cmd_puts("Can only add CPUs to mongroup that belong to parent\n");
> -		return -EINVAL;
> -	}
> -
> -	/* Check whether cpus are dropped from this group */
> -	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
> -	if (!cpumask_empty(tmpmask)) {
> -		/* Give any dropped cpus to parent rdtgroup */
> -		cpumask_or(&prgrp->cpu_mask, &prgrp->cpu_mask, tmpmask);
> -		update_closid_rmid(tmpmask, prgrp);
> -	}
> -
> -	/*
> -	 * If we added cpus, remove them from previous group that owned them
> -	 * and update per-cpu rmid
> -	 */
> -	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
> -	if (!cpumask_empty(tmpmask)) {
> -		head = &prgrp->mon.crdtgrp_list;
> -		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
> -			if (crgrp == rdtgrp)
> -				continue;
> -			cpumask_andnot(&crgrp->cpu_mask, &crgrp->cpu_mask,
> -				       tmpmask);
> -		}
> -		update_closid_rmid(tmpmask, rdtgrp);
> -	}
> -
> -	/* Done pushing/pulling - update this group with new mask */
> -	cpumask_copy(&rdtgrp->cpu_mask, newmask);
> -
> -	return 0;
> -}
> -
> -static void cpumask_rdtgrp_clear(struct rdtgroup *r, struct cpumask *m)
> -{
> -	struct rdtgroup *crgrp;
> -
> -	cpumask_andnot(&r->cpu_mask, &r->cpu_mask, m);
> -	/* update the child mon group masks as well*/
> -	list_for_each_entry(crgrp, &r->mon.crdtgrp_list, mon.crdtgrp_list)
> -		cpumask_and(&crgrp->cpu_mask, &r->cpu_mask, &crgrp->cpu_mask);
> -}
> -
> -static int cpus_ctrl_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
> -			   cpumask_var_t tmpmask, cpumask_var_t tmpmask1)
> -{
> -	struct rdtgroup *r, *crgrp;
> -	struct list_head *head;
> -
> -	/* Check whether cpus are dropped from this group */
> -	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
> -	if (!cpumask_empty(tmpmask)) {
> -		/* Can't drop from default group */
> -		if (rdtgrp == &rdtgroup_default) {
> -			rdt_last_cmd_puts("Can't drop CPUs from default group\n");
> -			return -EINVAL;
> -		}
> -
> -		/* Give any dropped cpus to rdtgroup_default */
> -		cpumask_or(&rdtgroup_default.cpu_mask,
> -			   &rdtgroup_default.cpu_mask, tmpmask);
> -		update_closid_rmid(tmpmask, &rdtgroup_default);
> -	}
> -
> -	/*
> -	 * If we added cpus, remove them from previous group and
> -	 * the prev group's child groups that owned them
> -	 * and update per-cpu closid/rmid.
> -	 */
> -	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
> -	if (!cpumask_empty(tmpmask)) {
> -		list_for_each_entry(r, &rdt_all_groups, rdtgroup_list) {
> -			if (r == rdtgrp)
> -				continue;
> -			cpumask_and(tmpmask1, &r->cpu_mask, tmpmask);
> -			if (!cpumask_empty(tmpmask1))
> -				cpumask_rdtgrp_clear(r, tmpmask1);
> -		}
> -		update_closid_rmid(tmpmask, rdtgrp);
> -	}
> -
> -	/* Done pushing/pulling - update this group with new mask */
> -	cpumask_copy(&rdtgrp->cpu_mask, newmask);
> -
> -	/*
> -	 * Clear child mon group masks since there is a new parent mask
> -	 * now and update the rmid for the cpus the child lost.
> -	 */
> -	head = &rdtgrp->mon.crdtgrp_list;
> -	list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
> -		cpumask_and(tmpmask, &rdtgrp->cpu_mask, &crgrp->cpu_mask);
> -		update_closid_rmid(tmpmask, rdtgrp);
> -		cpumask_clear(&crgrp->cpu_mask);
> -	}
> -
> -	return 0;
> -}
> -
> -static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of,
> -				   char *buf, size_t nbytes, loff_t off)
> -{
> -	cpumask_var_t tmpmask, newmask, tmpmask1;
> -	struct rdtgroup *rdtgrp;
> -	int ret;
> -
> -	if (!buf)
> -		return -EINVAL;
> -
> -	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
> -		return -ENOMEM;
> -	if (!zalloc_cpumask_var(&newmask, GFP_KERNEL)) {
> -		free_cpumask_var(tmpmask);
> -		return -ENOMEM;
> -	}
> -	if (!zalloc_cpumask_var(&tmpmask1, GFP_KERNEL)) {
> -		free_cpumask_var(tmpmask);
> -		free_cpumask_var(newmask);
> -		return -ENOMEM;
> -	}
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (!rdtgrp) {
> -		ret = -ENOENT;
> -		goto unlock;
> -	}
> -
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
> -	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -		ret = -EINVAL;
> -		rdt_last_cmd_puts("Pseudo-locking in progress\n");
> -		goto unlock;
> -	}
> -
> -	if (is_cpu_list(of))
> -		ret = cpulist_parse(buf, newmask);
> -	else
> -		ret = cpumask_parse(buf, newmask);
> -
> -	if (ret) {
> -		rdt_last_cmd_puts("Bad CPU list/mask\n");
> -		goto unlock;
> -	}
> -
> -	/* check that user didn't specify any offline cpus */
> -	cpumask_andnot(tmpmask, newmask, cpu_online_mask);
> -	if (!cpumask_empty(tmpmask)) {
> -		ret = -EINVAL;
> -		rdt_last_cmd_puts("Can only assign online CPUs\n");
> -		goto unlock;
> -	}
> -
> -	if (rdtgrp->type == RDTCTRL_GROUP)
> -		ret = cpus_ctrl_write(rdtgrp, newmask, tmpmask, tmpmask1);
> -	else if (rdtgrp->type == RDTMON_GROUP)
> -		ret = cpus_mon_write(rdtgrp, newmask, tmpmask);
> -	else
> -		ret = -EINVAL;
> -
> -unlock:
> -	rdtgroup_kn_unlock(of->kn);
> -	free_cpumask_var(tmpmask);
> -	free_cpumask_var(newmask);
> -	free_cpumask_var(tmpmask1);
> -
> -	return ret ?: nbytes;
> -}
> -
> -/**
> - * rdtgroup_remove - the helper to remove resource group safely
> - * @rdtgrp: resource group to remove
> - *
> - * On resource group creation via a mkdir, an extra kernfs_node reference is
> - * taken to ensure that the rdtgroup structure remains accessible for the
> - * rdtgroup_kn_unlock() calls where it is removed.
> - *
> - * Drop the extra reference here, then free the rdtgroup structure.
> - *
> - * Return: void
> - */
> -static void rdtgroup_remove(struct rdtgroup *rdtgrp)
> -{
> -	kernfs_put(rdtgrp->kn);
> -	kfree(rdtgrp);
> -}
> -
> -static void _update_task_closid_rmid(void *task)
> -{
> -	/*
> -	 * If the task is still current on this CPU, update PQR_ASSOC MSR.
> -	 * Otherwise, the MSR is updated when the task is scheduled in.
> -	 */
> -	if (task == current)
> -		resctrl_arch_sched_in(task);
> -}
> -
> -static void update_task_closid_rmid(struct task_struct *t)
> -{
> -	if (IS_ENABLED(CONFIG_SMP) && task_curr(t))
> -		smp_call_function_single(task_cpu(t), _update_task_closid_rmid, t, 1);
> -	else
> -		_update_task_closid_rmid(t);
> -}
> -
> -static bool task_in_rdtgroup(struct task_struct *tsk, struct rdtgroup *rdtgrp)
> -{
> -	u32 closid, rmid = rdtgrp->mon.rmid;
> -
> -	if (rdtgrp->type == RDTCTRL_GROUP)
> -		closid = rdtgrp->closid;
> -	else if (rdtgrp->type == RDTMON_GROUP)
> -		closid = rdtgrp->mon.parent->closid;
> -	else
> -		return false;
> -
> -	return resctrl_arch_match_closid(tsk, closid) &&
> -	       resctrl_arch_match_rmid(tsk, closid, rmid);
> -}
> -
> -static int __rdtgroup_move_task(struct task_struct *tsk,
> -				struct rdtgroup *rdtgrp)
> -{
> -	/* If the task is already in rdtgrp, no need to move the task. */
> -	if (task_in_rdtgroup(tsk, rdtgrp))
> -		return 0;
> -
> -	/*
> -	 * Set the task's closid/rmid before the PQR_ASSOC MSR can be
> -	 * updated by them.
> -	 *
> -	 * For ctrl_mon groups, move both closid and rmid.
> -	 * For monitor groups, can move the tasks only from
> -	 * their parent CTRL group.
> -	 */
> -	if (rdtgrp->type == RDTMON_GROUP &&
> -	    !resctrl_arch_match_closid(tsk, rdtgrp->mon.parent->closid)) {
> -		rdt_last_cmd_puts("Can't move task to different control group\n");
> -		return -EINVAL;
> -	}
> -
> -	if (rdtgrp->type == RDTMON_GROUP)
> -		resctrl_arch_set_closid_rmid(tsk, rdtgrp->mon.parent->closid,
> -					     rdtgrp->mon.rmid);
> -	else
> -		resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid,
> -					     rdtgrp->mon.rmid);
> -
> -	/*
> -	 * Ensure the task's closid and rmid are written before determining if
> -	 * the task is current that will decide if it will be interrupted.
> -	 * This pairs with the full barrier between the rq->curr update and
> -	 * resctrl_arch_sched_in() during context switch.
> -	 */
> -	smp_mb();
> -
> -	/*
> -	 * By now, the task's closid and rmid are set. If the task is current
> -	 * on a CPU, the PQR_ASSOC MSR needs to be updated to make the resource
> -	 * group go into effect. If the task is not current, the MSR will be
> -	 * updated when the task is scheduled in.
> -	 */
> -	update_task_closid_rmid(tsk);
> -
> -	return 0;
> -}
> -
> -static bool is_closid_match(struct task_struct *t, struct rdtgroup *r)
> -{
> -	return (resctrl_arch_alloc_capable() && (r->type == RDTCTRL_GROUP) &&
> -		resctrl_arch_match_closid(t, r->closid));
> -}
> -
> -static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r)
> -{
> -	return (resctrl_arch_mon_capable() && (r->type == RDTMON_GROUP) &&
> -		resctrl_arch_match_rmid(t, r->mon.parent->closid,
> -					r->mon.rmid));
> -}
> -
> -/**
> - * rdtgroup_tasks_assigned - Test if tasks have been assigned to resource group
> - * @r: Resource group
> - *
> - * Return: 1 if tasks have been assigned to @r, 0 otherwise
> - */
> -int rdtgroup_tasks_assigned(struct rdtgroup *r)
> -{
> -	struct task_struct *p, *t;
> -	int ret = 0;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	rcu_read_lock();
> -	for_each_process_thread(p, t) {
> -		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
> -			ret = 1;
> -			break;
> -		}
> -	}
> -	rcu_read_unlock();
> -
> -	return ret;
> -}
> -
> -static int rdtgroup_task_write_permission(struct task_struct *task,
> -					  struct kernfs_open_file *of)
> -{
> -	const struct cred *tcred = get_task_cred(task);
> -	const struct cred *cred = current_cred();
> -	int ret = 0;
> -
> -	/*
> -	 * Even if we're attaching all tasks in the thread group, we only
> -	 * need to check permissions on one of them.
> -	 */
> -	if (!uid_eq(cred->euid, GLOBAL_ROOT_UID) &&
> -	    !uid_eq(cred->euid, tcred->uid) &&
> -	    !uid_eq(cred->euid, tcred->suid)) {
> -		rdt_last_cmd_printf("No permission to move task %d\n", task->pid);
> -		ret = -EPERM;
> -	}
> -
> -	put_cred(tcred);
> -	return ret;
> -}
> -
> -static int rdtgroup_move_task(pid_t pid, struct rdtgroup *rdtgrp,
> -			      struct kernfs_open_file *of)
> -{
> -	struct task_struct *tsk;
> -	int ret;
> -
> -	rcu_read_lock();
> -	if (pid) {
> -		tsk = find_task_by_vpid(pid);
> -		if (!tsk) {
> -			rcu_read_unlock();
> -			rdt_last_cmd_printf("No task %d\n", pid);
> -			return -ESRCH;
> -		}
> -	} else {
> -		tsk = current;
> -	}
> -
> -	get_task_struct(tsk);
> -	rcu_read_unlock();
> -
> -	ret = rdtgroup_task_write_permission(tsk, of);
> -	if (!ret)
> -		ret = __rdtgroup_move_task(tsk, rdtgrp);
> -
> -	put_task_struct(tsk);
> -	return ret;
> -}
> -
> -static ssize_t rdtgroup_tasks_write(struct kernfs_open_file *of,
> -				    char *buf, size_t nbytes, loff_t off)
> -{
> -	struct rdtgroup *rdtgrp;
> -	char *pid_str;
> -	int ret = 0;
> -	pid_t pid;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (!rdtgrp) {
> -		rdtgroup_kn_unlock(of->kn);
> -		return -ENOENT;
> -	}
> -	rdt_last_cmd_clear();
> -
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
> -	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -		ret = -EINVAL;
> -		rdt_last_cmd_puts("Pseudo-locking in progress\n");
> -		goto unlock;
> -	}
> -
> -	while (buf && buf[0] != '\0' && buf[0] != '\n') {
> -		pid_str = strim(strsep(&buf, ","));
> -
> -		if (kstrtoint(pid_str, 0, &pid)) {
> -			rdt_last_cmd_printf("Task list parsing error pid %s\n", pid_str);
> -			ret = -EINVAL;
> -			break;
> -		}
> -
> -		if (pid < 0) {
> -			rdt_last_cmd_printf("Invalid pid %d\n", pid);
> -			ret = -EINVAL;
> -			break;
> -		}
> -
> -		ret = rdtgroup_move_task(pid, rdtgrp, of);
> -		if (ret) {
> -			rdt_last_cmd_printf("Error while processing task %d\n", pid);
> -			break;
> -		}
> -	}
> -
> -unlock:
> -	rdtgroup_kn_unlock(of->kn);
> -
> -	return ret ?: nbytes;
> -}
> -
> -static void show_rdt_tasks(struct rdtgroup *r, struct seq_file *s)
> -{
> -	struct task_struct *p, *t;
> -	pid_t pid;
> -
> -	rcu_read_lock();
> -	for_each_process_thread(p, t) {
> -		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
> -			pid = task_pid_vnr(t);
> -			if (pid)
> -				seq_printf(s, "%d\n", pid);
> -		}
> -	}
> -	rcu_read_unlock();
> -}
> -
> -static int rdtgroup_tasks_show(struct kernfs_open_file *of,
> -			       struct seq_file *s, void *v)
> -{
> -	struct rdtgroup *rdtgrp;
> -	int ret = 0;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (rdtgrp)
> -		show_rdt_tasks(rdtgrp, s);
> -	else
> -		ret = -ENOENT;
> -	rdtgroup_kn_unlock(of->kn);
> -
> -	return ret;
> -}
> -
> -static int rdtgroup_closid_show(struct kernfs_open_file *of,
> -				struct seq_file *s, void *v)
> -{
> -	struct rdtgroup *rdtgrp;
> -	int ret = 0;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (rdtgrp)
> -		seq_printf(s, "%u\n", rdtgrp->closid);
> -	else
> -		ret = -ENOENT;
> -	rdtgroup_kn_unlock(of->kn);
> -
> -	return ret;
> -}
> -
> -static int rdtgroup_rmid_show(struct kernfs_open_file *of,
> -			      struct seq_file *s, void *v)
> -{
> -	struct rdtgroup *rdtgrp;
> -	int ret = 0;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (rdtgrp)
> -		seq_printf(s, "%u\n", rdtgrp->mon.rmid);
> -	else
> -		ret = -ENOENT;
> -	rdtgroup_kn_unlock(of->kn);
> -
> -	return ret;
> -}
> -
> -#ifdef CONFIG_PROC_CPU_RESCTRL
> -
> -/*
> - * A task can only be part of one resctrl control group and of one monitor
> - * group which is associated to that control group.
> - *
> - * 1)   res:
> - *      mon:
> - *
> - *    resctrl is not available.
> - *
> - * 2)   res:/
> - *      mon:
> - *
> - *    Task is part of the root resctrl control group, and it is not associated
> - *    to any monitor group.
> - *
> - * 3)  res:/
> - *     mon:mon0
> - *
> - *    Task is part of the root resctrl control group and monitor group mon0.
> - *
> - * 4)  res:group0
> - *     mon:
> - *
> - *    Task is part of resctrl control group group0, and it is not associated
> - *    to any monitor group.
> - *
> - * 5) res:group0
> - *    mon:mon1
> - *
> - *    Task is part of resctrl control group group0 and monitor group mon1.
> - */
> -int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,
> -		      struct pid *pid, struct task_struct *tsk)
> -{
> -	struct rdtgroup *rdtg;
> -	int ret = 0;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	/* Return empty if resctrl has not been mounted. */
> -	if (!resctrl_mounted) {
> -		seq_puts(s, "res:\nmon:\n");
> -		goto unlock;
> -	}
> -
> -	list_for_each_entry(rdtg, &rdt_all_groups, rdtgroup_list) {
> -		struct rdtgroup *crg;
> -
> -		/*
> -		 * Task information is only relevant for shareable
> -		 * and exclusive groups.
> -		 */
> -		if (rdtg->mode != RDT_MODE_SHAREABLE &&
> -		    rdtg->mode != RDT_MODE_EXCLUSIVE)
> -			continue;
> -
> -		if (!resctrl_arch_match_closid(tsk, rdtg->closid))
> -			continue;
> -
> -		seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "",
> -			   rdtg->kn->name);
> -		seq_puts(s, "mon:");
> -		list_for_each_entry(crg, &rdtg->mon.crdtgrp_list,
> -				    mon.crdtgrp_list) {
> -			if (!resctrl_arch_match_rmid(tsk, crg->mon.parent->closid,
> -						     crg->mon.rmid))
> -				continue;
> -			seq_printf(s, "%s", crg->kn->name);
> -			break;
> -		}
> -		seq_putc(s, '\n');
> -		goto unlock;
> -	}
> -	/*
> -	 * The above search should succeed. Otherwise return
> -	 * with an error.
> -	 */
> -	ret = -ENOENT;
> -unlock:
> -	mutex_unlock(&rdtgroup_mutex);
> -
> -	return ret;
> -}
> -#endif
> -
> -static int rdt_last_cmd_status_show(struct kernfs_open_file *of,
> -				    struct seq_file *seq, void *v)
> -{
> -	int len;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -	len = seq_buf_used(&last_cmd_status);
> -	if (len)
> -		seq_printf(seq, "%.*s", len, last_cmd_status_buf);
> -	else
> -		seq_puts(seq, "ok\n");
> -	mutex_unlock(&rdtgroup_mutex);
> -	return 0;
> -}
> -
> -static int rdt_num_closids_show(struct kernfs_open_file *of,
> -				struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -
> -	seq_printf(seq, "%u\n", s->num_closid);
> -	return 0;
> -}
> -
> -static int rdt_default_ctrl_show(struct kernfs_open_file *of,
> -			     struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	seq_printf(seq, "%x\n", r->default_ctrl);
> -	return 0;
> -}
> -
> -static int rdt_min_cbm_bits_show(struct kernfs_open_file *of,
> -			     struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	seq_printf(seq, "%u\n", r->cache.min_cbm_bits);
> -	return 0;
> -}
> -
> -static int rdt_shareable_bits_show(struct kernfs_open_file *of,
> -				   struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	seq_printf(seq, "%x\n", r->cache.shareable_bits);
> -	return 0;
> -}
> -
> -/*
> - * rdt_bit_usage_show - Display current usage of resources
> - *
> - * A domain is a shared resource that can now be allocated differently. Here
> - * we display the current regions of the domain as an annotated bitmask.
> - * For each domain of this resource its allocation bitmask
> - * is annotated as below to indicate the current usage of the corresponding bit:
> - *   0 - currently unused
> - *   X - currently available for sharing and used by software and hardware
> - *   H - currently used by hardware only but available for software use
> - *   S - currently used and shareable by software only
> - *   E - currently used exclusively by one resource group
> - *   P - currently pseudo-locked by one resource group
> - */
> -static int rdt_bit_usage_show(struct kernfs_open_file *of,
> -			      struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	/*
> -	 * Use unsigned long even though only 32 bits are used to ensure
> -	 * test_bit() is used safely.
> -	 */
> -	unsigned long sw_shareable = 0, hw_shareable = 0;
> -	unsigned long exclusive = 0, pseudo_locked = 0;
> -	struct rdt_resource *r = s->res;
> -	struct rdt_domain *dom;
> -	int i, hwb, swb, excl, psl;
> -	enum rdtgrp_mode mode;
> -	bool sep = false;
> -	u32 ctrl_val;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -	hw_shareable = r->cache.shareable_bits;
> -	list_for_each_entry(dom, &r->domains, list) {
> -		if (sep)
> -			seq_putc(seq, ';');
> -		sw_shareable = 0;
> -		exclusive = 0;
> -		seq_printf(seq, "%d=", dom->id);
> -		for (i = 0; i < closids_supported(); i++) {
> -			if (!closid_allocated(i))
> -				continue;
> -			ctrl_val = resctrl_arch_get_config(r, dom, i,
> -							   s->conf_type);
> -			mode = rdtgroup_mode_by_closid(i);
> -			switch (mode) {
> -			case RDT_MODE_SHAREABLE:
> -				sw_shareable |= ctrl_val;
> -				break;
> -			case RDT_MODE_EXCLUSIVE:
> -				exclusive |= ctrl_val;
> -				break;
> -			case RDT_MODE_PSEUDO_LOCKSETUP:
> -			/*
> -			 * RDT_MODE_PSEUDO_LOCKSETUP is possible
> -			 * here but not included since the CBM
> -			 * associated with this CLOSID in this mode
> -			 * is not initialized and no task or cpu can be
> -			 * assigned this CLOSID.
> -			 */
> -				break;
> -			case RDT_MODE_PSEUDO_LOCKED:
> -			case RDT_NUM_MODES:
> -				WARN(1,
> -				     "invalid mode for closid %d\n", i);
> -				break;
> -			}
> -		}
> -		for (i = r->cache.cbm_len - 1; i >= 0; i--) {
> -			pseudo_locked = dom->plr ? dom->plr->cbm : 0;
> -			hwb = test_bit(i, &hw_shareable);
> -			swb = test_bit(i, &sw_shareable);
> -			excl = test_bit(i, &exclusive);
> -			psl = test_bit(i, &pseudo_locked);
> -			if (hwb && swb)
> -				seq_putc(seq, 'X');
> -			else if (hwb && !swb)
> -				seq_putc(seq, 'H');
> -			else if (!hwb && swb)
> -				seq_putc(seq, 'S');
> -			else if (excl)
> -				seq_putc(seq, 'E');
> -			else if (psl)
> -				seq_putc(seq, 'P');
> -			else /* Unused bits remain */
> -				seq_putc(seq, '0');
> -		}
> -		sep = true;
> -	}
> -	seq_putc(seq, '\n');
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -	return 0;
> -}
> -
> -static int rdt_min_bw_show(struct kernfs_open_file *of,
> -			     struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	seq_printf(seq, "%u\n", r->membw.min_bw);
> -	return 0;
> -}
> -
> -static int rdt_num_rmids_show(struct kernfs_open_file *of,
> -			      struct seq_file *seq, void *v)
> -{
> -	struct rdt_resource *r = of->kn->parent->priv;
> -
> -	seq_printf(seq, "%d\n", r->num_rmid);
> -
> -	return 0;
> -}
> -
> -static int rdt_mon_features_show(struct kernfs_open_file *of,
> -				 struct seq_file *seq, void *v)
> -{
> -	struct rdt_resource *r = of->kn->parent->priv;
> -	struct mon_evt *mevt;
> -
> -	list_for_each_entry(mevt, &r->evt_list, list) {
> -		seq_printf(seq, "%s\n", mevt->name);
> -		if (mevt->configurable)
> -			seq_printf(seq, "%s_config\n", mevt->name);
> -	}
> -
> -	return 0;
> -}
> -
> -static int rdt_bw_gran_show(struct kernfs_open_file *of,
> -			     struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	seq_printf(seq, "%u\n", r->membw.bw_gran);
> -	return 0;
> -}
> -
> -static int rdt_delay_linear_show(struct kernfs_open_file *of,
> -			     struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	seq_printf(seq, "%u\n", r->membw.delay_linear);
> -	return 0;
> -}
> -
> -static int max_threshold_occ_show(struct kernfs_open_file *of,
> -				  struct seq_file *seq, void *v)
> -{
> -	seq_printf(seq, "%u\n", resctrl_rmid_realloc_threshold);
> -
> -	return 0;
> -}
> -
> -static int rdt_thread_throttle_mode_show(struct kernfs_open_file *of,
> -					 struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	if (r->membw.throttle_mode == THREAD_THROTTLE_PER_THREAD)
> -		seq_puts(seq, "per-thread\n");
> -	else
> -		seq_puts(seq, "max\n");
> -
> -	return 0;
> -}
> -
> -static ssize_t max_threshold_occ_write(struct kernfs_open_file *of,
> -				       char *buf, size_t nbytes, loff_t off)
> -{
> -	unsigned int bytes;
> -	int ret;
> -
> -	ret = kstrtouint(buf, 0, &bytes);
> -	if (ret)
> -		return ret;
> -
> -	if (bytes > resctrl_rmid_realloc_limit)
> -		return -EINVAL;
> -
> -	resctrl_rmid_realloc_threshold = resctrl_arch_round_mon_val(bytes);
> -
> -	return nbytes;
> -}
> -
> -/*
> - * rdtgroup_mode_show - Display mode of this resource group
> - */
> -static int rdtgroup_mode_show(struct kernfs_open_file *of,
> -			      struct seq_file *s, void *v)
> -{
> -	struct rdtgroup *rdtgrp;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (!rdtgrp) {
> -		rdtgroup_kn_unlock(of->kn);
> -		return -ENOENT;
> -	}
> -
> -	seq_printf(s, "%s\n", rdtgroup_mode_str(rdtgrp->mode));
> -
> -	rdtgroup_kn_unlock(of->kn);
> -	return 0;
> -}
> -
> -static enum resctrl_conf_type resctrl_peer_type(enum resctrl_conf_type my_type)
> -{
> -	switch (my_type) {
> -	case CDP_CODE:
> -		return CDP_DATA;
> -	case CDP_DATA:
> -		return CDP_CODE;
> -	default:
> -	case CDP_NONE:
> -		return CDP_NONE;
> -	}
> -}
> -
> -static int rdt_has_sparse_bitmasks_show(struct kernfs_open_file *of,
> -					struct seq_file *seq, void *v)
> -{
> -	struct resctrl_schema *s = of->kn->parent->priv;
> -	struct rdt_resource *r = s->res;
> -
> -	seq_printf(seq, "%u\n", r->cache.arch_has_sparse_bitmasks);
> -
> -	return 0;
> -}
> -
> -/**
> - * __rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other
> - * @r: Resource to which domain instance @d belongs.
> - * @d: The domain instance for which @closid is being tested.
> - * @cbm: Capacity bitmask being tested.
> - * @closid: Intended closid for @cbm.
> - * @type: CDP type of @r.
> - * @exclusive: Only check if overlaps with exclusive resource groups
> - *
> - * Checks if provided @cbm intended to be used for @closid on domain
> - * @d overlaps with any other closids or other hardware usage associated
> - * with this domain. If @exclusive is true then only overlaps with
> - * resource groups in exclusive mode will be considered. If @exclusive
> - * is false then overlaps with any resource group or hardware entities
> - * will be considered.
> - *
> - * @cbm is unsigned long, even if only 32 bits are used, to make the
> - * bitmap functions work correctly.
> - *
> - * Return: false if CBM does not overlap, true if it does.
> - */
> -static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
> -				    unsigned long cbm, int closid,
> -				    enum resctrl_conf_type type, bool exclusive)
> -{
> -	enum rdtgrp_mode mode;
> -	unsigned long ctrl_b;
> -	int i;
> -
> -	/* Check for any overlap with regions used by hardware directly */
> -	if (!exclusive) {
> -		ctrl_b = r->cache.shareable_bits;
> -		if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len))
> -			return true;
> -	}
> -
> -	/* Check for overlap with other resource groups */
> -	for (i = 0; i < closids_supported(); i++) {
> -		ctrl_b = resctrl_arch_get_config(r, d, i, type);
> -		mode = rdtgroup_mode_by_closid(i);
> -		if (closid_allocated(i) && i != closid &&
> -		    mode != RDT_MODE_PSEUDO_LOCKSETUP) {
> -			if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len)) {
> -				if (exclusive) {
> -					if (mode == RDT_MODE_EXCLUSIVE)
> -						return true;
> -					continue;
> -				}
> -				return true;
> -			}
> -		}
> -	}
> -
> -	return false;
> -}
> -
> -/**
> - * rdtgroup_cbm_overlaps - Does CBM overlap with other use of hardware
> - * @s: Schema for the resource to which domain instance @d belongs.
> - * @d: The domain instance for which @closid is being tested.
> - * @cbm: Capacity bitmask being tested.
> - * @closid: Intended closid for @cbm.
> - * @exclusive: Only check if overlaps with exclusive resource groups
> - *
> - * Resources that can be allocated using a CBM can use the CBM to control
> - * the overlap of these allocations. rdtgroup_cmb_overlaps() is the test
> - * for overlap. Overlap test is not limited to the specific resource for
> - * which the CBM is intended though - when dealing with CDP resources that
> - * share the underlying hardware the overlap check should be performed on
> - * the CDP resource sharing the hardware also.
> - *
> - * Refer to description of __rdtgroup_cbm_overlaps() for the details of the
> - * overlap test.
> - *
> - * Return: true if CBM overlap detected, false if there is no overlap
> - */
> -bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
> -			   unsigned long cbm, int closid, bool exclusive)
> -{
> -	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
> -	struct rdt_resource *r = s->res;
> -
> -	if (__rdtgroup_cbm_overlaps(r, d, cbm, closid, s->conf_type,
> -				    exclusive))
> -		return true;
> -
> -	if (!resctrl_arch_get_cdp_enabled(r->rid))
> -		return false;
> -	return  __rdtgroup_cbm_overlaps(r, d, cbm, closid, peer_type, exclusive);
> -}
> -
> -/**
> - * rdtgroup_mode_test_exclusive - Test if this resource group can be exclusive
> - * @rdtgrp: Resource group identified through its closid.
> - *
> - * An exclusive resource group implies that there should be no sharing of
> - * its allocated resources. At the time this group is considered to be
> - * exclusive this test can determine if its current schemata supports this
> - * setting by testing for overlap with all other resource groups.
> - *
> - * Return: true if resource group can be exclusive, false if there is overlap
> - * with allocations of other resource groups and thus this resource group
> - * cannot be exclusive.
> - */
> -static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
> -{
> -	int closid = rdtgrp->closid;
> -	struct resctrl_schema *s;
> -	struct rdt_resource *r;
> -	bool has_cache = false;
> -	struct rdt_domain *d;
> -	u32 ctrl;
> -
> -	/* Walking r->domains, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -	list_for_each_entry(s, &resctrl_schema_all, list) {
> -		r = s->res;
> -		if (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)
> -			continue;
> -		has_cache = true;
> -		list_for_each_entry(d, &r->domains, list) {
> -			ctrl = resctrl_arch_get_config(r, d, closid,
> -						       s->conf_type);
> -			if (rdtgroup_cbm_overlaps(s, d, ctrl, closid, false)) {
> -				rdt_last_cmd_puts("Schemata overlaps\n");
> -				return false;
> -			}
> -		}
> -	}
> -
> -	if (!has_cache) {
> -		rdt_last_cmd_puts("Cannot be exclusive without CAT/CDP\n");
> -		return false;
> -	}
> -
> -	return true;
> -}
> -
> -/*
> - * rdtgroup_mode_write - Modify the resource group's mode
> - */
> -static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
> -				   char *buf, size_t nbytes, loff_t off)
> -{
> -	struct rdtgroup *rdtgrp;
> -	enum rdtgrp_mode mode;
> -	int ret = 0;
> -
> -	/* Valid input requires a trailing newline */
> -	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> -		return -EINVAL;
> -	buf[nbytes - 1] = '\0';
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (!rdtgrp) {
> -		rdtgroup_kn_unlock(of->kn);
> -		return -ENOENT;
> -	}
> -
> -	rdt_last_cmd_clear();
> -
> -	mode = rdtgrp->mode;
> -
> -	if ((!strcmp(buf, "shareable") && mode == RDT_MODE_SHAREABLE) ||
> -	    (!strcmp(buf, "exclusive") && mode == RDT_MODE_EXCLUSIVE) ||
> -	    (!strcmp(buf, "pseudo-locksetup") &&
> -	     mode == RDT_MODE_PSEUDO_LOCKSETUP) ||
> -	    (!strcmp(buf, "pseudo-locked") && mode == RDT_MODE_PSEUDO_LOCKED))
> -		goto out;
> -
> -	if (mode == RDT_MODE_PSEUDO_LOCKED) {
> -		rdt_last_cmd_puts("Cannot change pseudo-locked group\n");
> -		ret = -EINVAL;
> -		goto out;
> -	}
> -
> -	if (!strcmp(buf, "shareable")) {
> -		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -			ret = rdtgroup_locksetup_exit(rdtgrp);
> -			if (ret)
> -				goto out;
> -		}
> -		rdtgrp->mode = RDT_MODE_SHAREABLE;
> -	} else if (!strcmp(buf, "exclusive")) {
> -		if (!rdtgroup_mode_test_exclusive(rdtgrp)) {
> -			ret = -EINVAL;
> -			goto out;
> -		}
> -		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -			ret = rdtgroup_locksetup_exit(rdtgrp);
> -			if (ret)
> -				goto out;
> -		}
> -		rdtgrp->mode = RDT_MODE_EXCLUSIVE;
> -	} else if (IS_ENABLED(CONFIG_RESCTRL_FS_PSEUDO_LOCK) &&
> -		   !strcmp(buf, "pseudo-locksetup")) {
> -		ret = rdtgroup_locksetup_enter(rdtgrp);
> -		if (ret)
> -			goto out;
> -		rdtgrp->mode = RDT_MODE_PSEUDO_LOCKSETUP;
> -	} else {
> -		rdt_last_cmd_puts("Unknown or unsupported mode\n");
> -		ret = -EINVAL;
> -	}
> -
> -out:
> -	rdtgroup_kn_unlock(of->kn);
> -	return ret ?: nbytes;
> -}
> -
> -/**
> - * rdtgroup_cbm_to_size - Translate CBM to size in bytes
> - * @r: RDT resource to which @d belongs.
> - * @d: RDT domain instance.
> - * @cbm: bitmask for which the size should be computed.
> - *
> - * The bitmask provided associated with the RDT domain instance @d will be
> - * translated into how many bytes it represents. The size in bytes is
> - * computed by first dividing the total cache size by the CBM length to
> - * determine how many bytes each bit in the bitmask represents. The result
> - * is multiplied with the number of bits set in the bitmask.
> - *
> - * @cbm is unsigned long, even if only 32 bits are used to make the
> - * bitmap functions work correctly.
> - */
> -unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,
> -				  struct rdt_domain *d, unsigned long cbm)
> -{
> -	struct cpu_cacheinfo *ci;
> -	unsigned int size = 0;
> -	int num_b, i;
> -
> -	num_b = bitmap_weight(&cbm, r->cache.cbm_len);
> -	ci = get_cpu_cacheinfo(cpumask_any(&d->cpu_mask));
> -	for (i = 0; i < ci->num_leaves; i++) {
> -		if (ci->info_list[i].level == r->cache_level) {
> -			size = ci->info_list[i].size / r->cache.cbm_len * num_b;
> -			break;
> -		}
> -	}
> -
> -	return size;
> -}
> -
> -/*
> - * rdtgroup_size_show - Display size in bytes of allocated regions
> - *
> - * The "size" file mirrors the layout of the "schemata" file, printing the
> - * size in bytes of each region instead of the capacity bitmask.
> - */
> -static int rdtgroup_size_show(struct kernfs_open_file *of,
> -			      struct seq_file *s, void *v)
> -{
> -	struct resctrl_schema *schema;
> -	enum resctrl_conf_type type;
> -	struct rdtgroup *rdtgrp;
> -	struct rdt_resource *r;
> -	struct rdt_domain *d;
> -	unsigned int size;
> -	int ret = 0;
> -	u32 closid;
> -	bool sep;
> -	u32 ctrl;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> -	if (!rdtgrp) {
> -		rdtgroup_kn_unlock(of->kn);
> -		return -ENOENT;
> -	}
> -
> -	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> -		if (!rdtgrp->plr->d) {
> -			rdt_last_cmd_clear();
> -			rdt_last_cmd_puts("Cache domain offline\n");
> -			ret = -ENODEV;
> -		} else {
> -			seq_printf(s, "%*s:", max_name_width,
> -				   rdtgrp->plr->s->name);
> -			size = rdtgroup_cbm_to_size(rdtgrp->plr->s->res,
> -						    rdtgrp->plr->d,
> -						    rdtgrp->plr->cbm);
> -			seq_printf(s, "%d=%u\n", rdtgrp->plr->d->id, size);
> -		}
> -		goto out;
> -	}
> -
> -	closid = rdtgrp->closid;
> -
> -	list_for_each_entry(schema, &resctrl_schema_all, list) {
> -		r = schema->res;
> -		type = schema->conf_type;
> -		sep = false;
> -		seq_printf(s, "%*s:", max_name_width, schema->name);
> -		list_for_each_entry(d, &r->domains, list) {
> -			if (sep)
> -				seq_putc(s, ';');
> -			if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> -				size = 0;
> -			} else {
> -				if (is_mba_sc(r))
> -					ctrl = d->mbps_val[closid];
> -				else
> -					ctrl = resctrl_arch_get_config(r, d,
> -								       closid,
> -								       type);
> -				if (r->rid == RDT_RESOURCE_MBA ||
> -				    r->rid == RDT_RESOURCE_SMBA)
> -					size = ctrl;
> -				else
> -					size = rdtgroup_cbm_to_size(r, d, ctrl);
> -			}
> -			seq_printf(s, "%d=%u", d->id, size);
> -			sep = true;
> -		}
> -		seq_putc(s, '\n');
> -	}
> -
> -out:
> -	rdtgroup_kn_unlock(of->kn);
> -
> -	return ret;
> -}
> -
>  #define INVALID_CONFIG_INDEX   UINT_MAX
>  
>  /**
> @@ -1622,62 +86,6 @@ void resctrl_arch_mon_event_config_read(void *info)
>  	mon_info->mon_config = msrval & MAX_EVT_CONFIG_BITS;
>  }
>  
> -static void mondata_config_read(struct resctrl_mon_config_info *mon_info)
> -{
> -	smp_call_function_any(&mon_info->d->cpu_mask,
> -			      resctrl_arch_mon_event_config_read, mon_info, 1);
> -}
> -
> -static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid)
> -{
> -	struct resctrl_mon_config_info mon_info = {0};
> -	struct rdt_domain *dom;
> -	bool sep = false;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	list_for_each_entry(dom, &r->domains, list) {
> -		if (sep)
> -			seq_puts(s, ";");
> -
> -		memset(&mon_info, 0, sizeof(struct resctrl_mon_config_info));
> -		mon_info.r = r;
> -		mon_info.d = dom;
> -		mon_info.evtid = evtid;
> -		mondata_config_read(&mon_info);
> -
> -		seq_printf(s, "%d=0x%02x", dom->id, mon_info.mon_config);
> -		sep = true;
> -	}
> -	seq_puts(s, "\n");
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -
> -	return 0;
> -}
> -
> -static int mbm_total_bytes_config_show(struct kernfs_open_file *of,
> -				       struct seq_file *seq, void *v)
> -{
> -	struct rdt_resource *r = of->kn->parent->priv;
> -
> -	mbm_config_show(seq, r, QOS_L3_MBM_TOTAL_EVENT_ID);
> -
> -	return 0;
> -}
> -
> -static int mbm_local_bytes_config_show(struct kernfs_open_file *of,
> -				       struct seq_file *seq, void *v)
> -{
> -	struct rdt_resource *r = of->kn->parent->priv;
> -
> -	mbm_config_show(seq, r, QOS_L3_MBM_LOCAL_EVENT_ID);
> -
> -	return 0;
> -}
> -
>  void resctrl_arch_mon_event_config_write(void *info)
>  {
>  	struct resctrl_mon_config_info *mon_info = info;
> @@ -1694,605 +102,6 @@ void resctrl_arch_mon_event_config_write(void *info)
>  	mon_info->err = 0;
>  }
>  
> -static int mbm_config_write_domain(struct rdt_resource *r,
> -				   struct rdt_domain *d, u32 evtid, u32 val)
> -{
> -	struct resctrl_mon_config_info mon_info = {0};
> -
> -	/*
> -	 * Read the current config value first. If both are the same then
> -	 * no need to write it again.
> -	 */
> -	mon_info.r = r;
> -	mon_info.d = d;
> -	mon_info.evtid = evtid;
> -	mondata_config_read(&mon_info);
> -	if (mon_info.mon_config == val)
> -		return 0;
> -
> -	mon_info.mon_config = val;
> -
> -	/*
> -	 * Update MSR_IA32_EVT_CFG_BASE MSR on one of the CPUs in the
> -	 * domain. The MSRs offset from MSR MSR_IA32_EVT_CFG_BASE
> -	 * are scoped at the domain level. Writing any of these MSRs
> -	 * on one CPU is observed by all the CPUs in the domain.
> -	 */
> -	smp_call_function_any(&d->cpu_mask, resctrl_arch_mon_event_config_write,
> -			      &mon_info, 1);
> -	if (mon_info.err) {
> -		rdt_last_cmd_puts("Invalid event configuration\n");
> -		return mon_info.err;
> -	}
> -
> -	/*
> -	 * When an Event Configuration is changed, the bandwidth counters
> -	 * for all RMIDs and Events will be cleared by the hardware. The
> -	 * hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for
> -	 * every RMID on the next read to any event for every RMID.
> -	 * Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62)
> -	 * cleared while it is tracked by the hardware. Clear the
> -	 * mbm_local and mbm_total counts for all the RMIDs.
> -	 */
> -	resctrl_arch_reset_rmid_all(r, d);
> -
> -	return 0;
> -}
> -
> -static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
> -{
> -	char *dom_str = NULL, *id_str;
> -	unsigned long dom_id, val;
> -	struct rdt_domain *d;
> -	int err;
> -
> -	/* Walking r->domains, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -next:
> -	if (!tok || tok[0] == '\0')
> -		return 0;
> -
> -	/* Start processing the strings for each domain */
> -	dom_str = strim(strsep(&tok, ";"));
> -	id_str = strsep(&dom_str, "=");
> -
> -	if (!id_str || kstrtoul(id_str, 10, &dom_id)) {
> -		rdt_last_cmd_puts("Missing '=' or non-numeric domain id\n");
> -		return -EINVAL;
> -	}
> -
> -	if (!dom_str || kstrtoul(dom_str, 16, &val)) {
> -		rdt_last_cmd_puts("Non-numeric event configuration value\n");
> -		return -EINVAL;
> -	}
> -
> -	/* Value from user cannot be more than the supported set of events */
> -	if ((val & r->mbm_cfg_mask) != val) {
> -		rdt_last_cmd_printf("Invalid event configuration: max valid mask is 0x%02x\n",
> -				    r->mbm_cfg_mask);
> -		return -EINVAL;
> -	}
> -
> -	list_for_each_entry(d, &r->domains, list) {
> -		if (d->id == dom_id) {
> -			err = mbm_config_write_domain(r, d, evtid, val);
> -			if (err)
> -				return err;
> -			goto next;
> -		}
> -	}
> -
> -	return -EINVAL;
> -}
> -
> -static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of,
> -					    char *buf, size_t nbytes,
> -					    loff_t off)
> -{
> -	struct rdt_resource *r = of->kn->parent->priv;
> -	int ret;
> -
> -	/* Valid input requires a trailing newline */
> -	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> -		return -EINVAL;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	rdt_last_cmd_clear();
> -
> -	buf[nbytes - 1] = '\0';
> -
> -	ret = mon_config_write(r, buf, QOS_L3_MBM_TOTAL_EVENT_ID);
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -
> -	return ret ?: nbytes;
> -}
> -
> -static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of,
> -					    char *buf, size_t nbytes,
> -					    loff_t off)
> -{
> -	struct rdt_resource *r = of->kn->parent->priv;
> -	int ret;
> -
> -	/* Valid input requires a trailing newline */
> -	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> -		return -EINVAL;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	rdt_last_cmd_clear();
> -
> -	buf[nbytes - 1] = '\0';
> -
> -	ret = mon_config_write(r, buf, QOS_L3_MBM_LOCAL_EVENT_ID);
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -
> -	return ret ?: nbytes;
> -}
> -
> -/* rdtgroup information files for one cache resource. */
> -static struct rftype res_common_files[] = {
> -	{
> -		.name		= "last_cmd_status",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_last_cmd_status_show,
> -		.fflags		= RFTYPE_TOP_INFO,
> -	},
> -	{
> -		.name		= "num_closids",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_num_closids_show,
> -		.fflags		= RFTYPE_CTRL_INFO,
> -	},
> -	{
> -		.name		= "mon_features",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_mon_features_show,
> -		.fflags		= RFTYPE_MON_INFO,
> -	},
> -	{
> -		.name		= "num_rmids",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_num_rmids_show,
> -		.fflags		= RFTYPE_MON_INFO,
> -	},
> -	{
> -		.name		= "cbm_mask",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_default_ctrl_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> -	},
> -	{
> -		.name		= "min_cbm_bits",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_min_cbm_bits_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> -	},
> -	{
> -		.name		= "shareable_bits",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_shareable_bits_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> -	},
> -	{
> -		.name		= "bit_usage",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_bit_usage_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> -	},
> -	{
> -		.name		= "min_bandwidth",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_min_bw_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
> -	},
> -	{
> -		.name		= "bandwidth_gran",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_bw_gran_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
> -	},
> -	{
> -		.name		= "delay_linear",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_delay_linear_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
> -	},
> -	/*
> -	 * Platform specific which (if any) capabilities are provided by
> -	 * thread_throttle_mode. Defer "fflags" initialization to platform
> -	 * discovery.
> -	 */
> -	{
> -		.name		= "thread_throttle_mode",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_thread_throttle_mode_show,
> -	},
> -	{
> -		.name		= "max_threshold_occupancy",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.write		= max_threshold_occ_write,
> -		.seq_show	= max_threshold_occ_show,
> -		.fflags		= RFTYPE_MON_INFO | RFTYPE_RES_CACHE,
> -	},
> -	{
> -		.name		= "mbm_total_bytes_config",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= mbm_total_bytes_config_show,
> -		.write		= mbm_total_bytes_config_write,
> -	},
> -	{
> -		.name		= "mbm_local_bytes_config",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= mbm_local_bytes_config_show,
> -		.write		= mbm_local_bytes_config_write,
> -	},
> -	{
> -		.name		= "cpus",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.write		= rdtgroup_cpus_write,
> -		.seq_show	= rdtgroup_cpus_show,
> -		.fflags		= RFTYPE_BASE,
> -	},
> -	{
> -		.name		= "cpus_list",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.write		= rdtgroup_cpus_write,
> -		.seq_show	= rdtgroup_cpus_show,
> -		.flags		= RFTYPE_FLAGS_CPUS_LIST,
> -		.fflags		= RFTYPE_BASE,
> -	},
> -	{
> -		.name		= "tasks",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.write		= rdtgroup_tasks_write,
> -		.seq_show	= rdtgroup_tasks_show,
> -		.fflags		= RFTYPE_BASE,
> -	},
> -	{
> -		.name		= "mon_hw_id",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdtgroup_rmid_show,
> -		.fflags		= RFTYPE_MON_BASE | RFTYPE_DEBUG,
> -	},
> -	{
> -		.name		= "schemata",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.write		= rdtgroup_schemata_write,
> -		.seq_show	= rdtgroup_schemata_show,
> -		.fflags		= RFTYPE_CTRL_BASE,
> -	},
> -	{
> -		.name		= "mode",
> -		.mode		= 0644,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.write		= rdtgroup_mode_write,
> -		.seq_show	= rdtgroup_mode_show,
> -		.fflags		= RFTYPE_CTRL_BASE,
> -	},
> -	{
> -		.name		= "size",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdtgroup_size_show,
> -		.fflags		= RFTYPE_CTRL_BASE,
> -	},
> -	{
> -		.name		= "sparse_masks",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdt_has_sparse_bitmasks_show,
> -		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> -	},
> -	{
> -		.name		= "ctrl_hw_id",
> -		.mode		= 0444,
> -		.kf_ops		= &rdtgroup_kf_single_ops,
> -		.seq_show	= rdtgroup_closid_show,
> -		.fflags		= RFTYPE_CTRL_BASE | RFTYPE_DEBUG,
> -	},
> -
> -};
> -
> -static int rdtgroup_add_files(struct kernfs_node *kn, unsigned long fflags)
> -{
> -	struct rftype *rfts, *rft;
> -	int ret, len;
> -
> -	rfts = res_common_files;
> -	len = ARRAY_SIZE(res_common_files);
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> -
> -	if (resctrl_debug)
> -		fflags |= RFTYPE_DEBUG;
> -
> -	for (rft = rfts; rft < rfts + len; rft++) {
> -		if (rft->fflags && ((fflags & rft->fflags) == rft->fflags)) {
> -			ret = rdtgroup_add_file(kn, rft);
> -			if (ret)
> -				goto error;
> -		}
> -	}
> -
> -	return 0;
> -error:
> -	pr_warn("Failed to add %s, err=%d\n", rft->name, ret);
> -	while (--rft >= rfts) {
> -		if ((fflags & rft->fflags) == rft->fflags)
> -			kernfs_remove_by_name(kn, rft->name);
> -	}
> -	return ret;
> -}
> -
> -static struct rftype *rdtgroup_get_rftype_by_name(const char *name)
> -{
> -	struct rftype *rfts, *rft;
> -	int len;
> -
> -	rfts = res_common_files;
> -	len = ARRAY_SIZE(res_common_files);
> -
> -	for (rft = rfts; rft < rfts + len; rft++) {
> -		if (!strcmp(rft->name, name))
> -			return rft;
> -	}
> -
> -	return NULL;
> -}
> -
> -static void thread_throttle_mode_init(void)
> -{
> -	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> -	struct rftype *rft;
> -
> -	if (!r->alloc_capable ||
> -	    r->membw.throttle_mode == THREAD_THROTTLE_UNDEFINED)
> -		return;
> -
> -	rft = rdtgroup_get_rftype_by_name("thread_throttle_mode");
> -	if (!rft)
> -		return;
> -
> -	rft->fflags = RFTYPE_CTRL_INFO | RFTYPE_RES_MB;
> -}
> -
> -void mbm_config_rftype_init(const char *config)
> -{
> -	struct rftype *rft;
> -
> -	rft = rdtgroup_get_rftype_by_name(config);
> -	if (rft)
> -		rft->fflags = RFTYPE_MON_INFO | RFTYPE_RES_CACHE;
> -}
> -
> -/**
> - * rdtgroup_kn_mode_restrict - Restrict user access to named resctrl file
> - * @r: The resource group with which the file is associated.
> - * @name: Name of the file
> - *
> - * The permissions of named resctrl file, directory, or link are modified
> - * to not allow read, write, or execute by any user.
> - *
> - * WARNING: This function is intended to communicate to the user that the
> - * resctrl file has been locked down - that it is not relevant to the
> - * particular state the system finds itself in. It should not be relied
> - * on to protect from user access because after the file's permissions
> - * are restricted the user can still change the permissions using chmod
> - * from the command line.
> - *
> - * Return: 0 on success, <0 on failure.
> - */
> -int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name)
> -{
> -	struct iattr iattr = {.ia_valid = ATTR_MODE,};
> -	struct kernfs_node *kn;
> -	int ret = 0;
> -
> -	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
> -	if (!kn)
> -		return -ENOENT;
> -
> -	switch (kernfs_type(kn)) {
> -	case KERNFS_DIR:
> -		iattr.ia_mode = S_IFDIR;
> -		break;
> -	case KERNFS_FILE:
> -		iattr.ia_mode = S_IFREG;
> -		break;
> -	case KERNFS_LINK:
> -		iattr.ia_mode = S_IFLNK;
> -		break;
> -	}
> -
> -	ret = kernfs_setattr(kn, &iattr);
> -	kernfs_put(kn);
> -	return ret;
> -}
> -
> -/**
> - * rdtgroup_kn_mode_restore - Restore user access to named resctrl file
> - * @r: The resource group with which the file is associated.
> - * @name: Name of the file
> - * @mask: Mask of permissions that should be restored
> - *
> - * Restore the permissions of the named file. If @name is a directory the
> - * permissions of its parent will be used.
> - *
> - * Return: 0 on success, <0 on failure.
> - */
> -int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
> -			     umode_t mask)
> -{
> -	struct iattr iattr = {.ia_valid = ATTR_MODE,};
> -	struct kernfs_node *kn, *parent;
> -	struct rftype *rfts, *rft;
> -	int ret, len;
> -
> -	rfts = res_common_files;
> -	len = ARRAY_SIZE(res_common_files);
> -
> -	for (rft = rfts; rft < rfts + len; rft++) {
> -		if (!strcmp(rft->name, name))
> -			iattr.ia_mode = rft->mode & mask;
> -	}
> -
> -	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
> -	if (!kn)
> -		return -ENOENT;
> -
> -	switch (kernfs_type(kn)) {
> -	case KERNFS_DIR:
> -		parent = kernfs_get_parent(kn);
> -		if (parent) {
> -			iattr.ia_mode |= parent->mode;
> -			kernfs_put(parent);
> -		}
> -		iattr.ia_mode |= S_IFDIR;
> -		break;
> -	case KERNFS_FILE:
> -		iattr.ia_mode |= S_IFREG;
> -		break;
> -	case KERNFS_LINK:
> -		iattr.ia_mode |= S_IFLNK;
> -		break;
> -	}
> -
> -	ret = kernfs_setattr(kn, &iattr);
> -	kernfs_put(kn);
> -	return ret;
> -}
> -
> -static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
> -				      unsigned long fflags)
> -{
> -	struct kernfs_node *kn_subdir;
> -	int ret;
> -
> -	kn_subdir = kernfs_create_dir(kn_info, name,
> -				      kn_info->mode, priv);
> -	if (IS_ERR(kn_subdir))
> -		return PTR_ERR(kn_subdir);
> -
> -	ret = rdtgroup_kn_set_ugid(kn_subdir);
> -	if (ret)
> -		return ret;
> -
> -	ret = rdtgroup_add_files(kn_subdir, fflags);
> -	if (!ret)
> -		kernfs_activate(kn_subdir);
> -
> -	return ret;
> -}
> -
> -static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
> -{
> -	enum resctrl_res_level i;
> -	struct resctrl_schema *s;
> -	struct rdt_resource *r;
> -	unsigned long fflags;
> -	char name[32];
> -	int ret;
> -
> -	/* create the directory */
> -	kn_info = kernfs_create_dir(parent_kn, "info", parent_kn->mode, NULL);
> -	if (IS_ERR(kn_info))
> -		return PTR_ERR(kn_info);
> -
> -	ret = rdtgroup_add_files(kn_info, RFTYPE_TOP_INFO);
> -	if (ret)
> -		goto out_destroy;
> -
> -	/* loop over enabled controls, these are all alloc_capable */
> -	list_for_each_entry(s, &resctrl_schema_all, list) {
> -		r = s->res;
> -		fflags = r->fflags | RFTYPE_CTRL_INFO;
> -		ret = rdtgroup_mkdir_info_resdir(s, s->name, fflags);
> -		if (ret)
> -			goto out_destroy;
> -	}
> -
> -	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> -		r = resctrl_arch_get_resource(i);
> -		if (!r->mon_capable)
> -			continue;
> -
> -		fflags =  r->fflags | RFTYPE_MON_INFO;
> -		sprintf(name, "%s_MON", r->name);
> -		ret = rdtgroup_mkdir_info_resdir(r, name, fflags);
> -		if (ret)
> -			goto out_destroy;
> -	}
> -
> -	ret = rdtgroup_kn_set_ugid(kn_info);
> -	if (ret)
> -		goto out_destroy;
> -
> -	kernfs_activate(kn_info);
> -
> -	return 0;
> -
> -out_destroy:
> -	kernfs_remove(kn_info);
> -	return ret;
> -}
> -
> -static int
> -mongroup_create_dir(struct kernfs_node *parent_kn, struct rdtgroup *prgrp,
> -		    char *name, struct kernfs_node **dest_kn)
> -{
> -	struct kernfs_node *kn;
> -	int ret;
> -
> -	/* create the directory */
> -	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
> -	if (IS_ERR(kn))
> -		return PTR_ERR(kn);
> -
> -	if (dest_kn)
> -		*dest_kn = kn;
> -
> -	ret = rdtgroup_kn_set_ugid(kn);
> -	if (ret)
> -		goto out_destroy;
> -
> -	kernfs_activate(kn);
> -
> -	return 0;
> -
> -out_destroy:
> -	kernfs_remove(kn);
> -	return ret;
> -}
> -
>  static void l3_qos_cfg_update(void *arg)
>  {
>  	bool *enable = arg;
> @@ -2307,11 +116,6 @@ static void l2_qos_cfg_update(void *arg)
>  	wrmsrl(MSR_IA32_L2_QOS_CFG, *enable ? L2_QOS_CDP_ENABLE : 0ULL);
>  }
>  
> -static inline bool is_mba_linear(void)
> -{
> -	return resctrl_arch_get_resource(RDT_RESOURCE_MBA)->membw.delay_linear;
> -}
> -
>  static int set_cache_qos_cfg(int level, bool enable)
>  {
>  	void (*update)(void *arg);
> @@ -2367,66 +171,6 @@ void rdt_domain_reconfigure_cdp(struct rdt_resource *r)
>  		l3_qos_cfg_update(&hw_res->cdp_enabled);
>  }
>  
> -static int mba_sc_domain_allocate(struct rdt_resource *r, struct rdt_domain *d)
> -{
> -	u32 num_closid = resctrl_arch_get_num_closid(r);
> -	int cpu = cpumask_any(&d->cpu_mask);
> -	int i;
> -
> -	d->mbps_val = kcalloc_node(num_closid, sizeof(*d->mbps_val),
> -				   GFP_KERNEL, cpu_to_node(cpu));
> -	if (!d->mbps_val)
> -		return -ENOMEM;
> -
> -	for (i = 0; i < num_closid; i++)
> -		d->mbps_val[i] = MBA_MAX_MBPS;
> -
> -	return 0;
> -}
> -
> -static void mba_sc_domain_destroy(struct rdt_resource *r,
> -				  struct rdt_domain *d)
> -{
> -	kfree(d->mbps_val);
> -	d->mbps_val = NULL;
> -}
> -
> -/*
> - * MBA software controller is supported only if
> - * MBM is supported and MBA is in linear scale.
> - */
> -static bool supports_mba_mbps(void)
> -{
> -	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> -
> -	return (resctrl_arch_is_mbm_local_enabled() &&
> -		r->alloc_capable && is_mba_linear());
> -}
> -
> -/*
> - * Enable or disable the MBA software controller
> - * which helps user specify bandwidth in MBps.
> - */
> -static int set_mba_sc(bool mba_sc)
> -{
> -	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> -	u32 num_closid = resctrl_arch_get_num_closid(r);
> -	struct rdt_domain *d;
> -	int i;
> -
> -	if (!supports_mba_mbps() || mba_sc == is_mba_sc(r))
> -		return -EINVAL;
> -
> -	r->membw.mba_sc = mba_sc;
> -
> -	list_for_each_entry(d, &r->domains, list) {
> -		for (i = 0; i < num_closid; i++)
> -			d->mbps_val[i] = MBA_MAX_MBPS;
> -	}
> -
> -	return 0;
> -}
> -
>  static int cdp_enable(int level)
>  {
>  	struct rdt_resource *r_l = &rdt_resources_all[level].r_resctrl;
> @@ -2467,414 +211,6 @@ int resctrl_arch_set_cdp_enabled(enum resctrl_res_level l, bool enable)
>  	return 0;
>  }
>  
> -/*
> - * We don't allow rdtgroup directories to be created anywhere
> - * except the root directory. Thus when looking for the rdtgroup
> - * structure for a kernfs node we are either looking at a directory,
> - * in which case the rdtgroup structure is pointed at by the "priv"
> - * field, otherwise we have a file, and need only look to the parent
> - * to find the rdtgroup.
> - */
> -static struct rdtgroup *kernfs_to_rdtgroup(struct kernfs_node *kn)
> -{
> -	if (kernfs_type(kn) == KERNFS_DIR) {
> -		/*
> -		 * All the resource directories use "kn->priv"
> -		 * to point to the "struct rdtgroup" for the
> -		 * resource. "info" and its subdirectories don't
> -		 * have rdtgroup structures, so return NULL here.
> -		 */
> -		if (kn == kn_info || kn->parent == kn_info)
> -			return NULL;
> -		else
> -			return kn->priv;
> -	} else {
> -		return kn->parent->priv;
> -	}
> -}
> -
> -static void rdtgroup_kn_get(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
> -{
> -	atomic_inc(&rdtgrp->waitcount);
> -	kernfs_break_active_protection(kn);
> -}
> -
> -static void rdtgroup_kn_put(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
> -{
> -	if (atomic_dec_and_test(&rdtgrp->waitcount) &&
> -	    (rdtgrp->flags & RDT_DELETED)) {
> -		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> -		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
> -			rdtgroup_pseudo_lock_remove(rdtgrp);
> -		kernfs_unbreak_active_protection(kn);
> -		rdtgroup_remove(rdtgrp);
> -	} else {
> -		kernfs_unbreak_active_protection(kn);
> -	}
> -}
> -
> -struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn)
> -{
> -	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
> -
> -	if (!rdtgrp)
> -		return NULL;
> -
> -	rdtgroup_kn_get(rdtgrp, kn);
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	/* Was this group deleted while we waited? */
> -	if (rdtgrp->flags & RDT_DELETED)
> -		return NULL;
> -
> -	return rdtgrp;
> -}
> -
> -void rdtgroup_kn_unlock(struct kernfs_node *kn)
> -{
> -	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
> -
> -	if (!rdtgrp)
> -		return;
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -
> -	rdtgroup_kn_put(rdtgrp, kn);
> -}
> -
> -static int mkdir_mondata_all(struct kernfs_node *parent_kn,
> -			     struct rdtgroup *prgrp,
> -			     struct kernfs_node **mon_data_kn);
> -
> -static void rdt_disable_ctx(void)
> -{
> -	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
> -	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
> -	set_mba_sc(false);
> -
> -	resctrl_debug = false;
> -}
> -
> -static int rdt_enable_ctx(struct rdt_fs_context *ctx)
> -{
> -	int ret = 0;
> -
> -	if (ctx->enable_cdpl2) {
> -		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, true);
> -		if (ret)
> -			goto out_done;
> -	}
> -
> -	if (ctx->enable_cdpl3) {
> -		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, true);
> -		if (ret)
> -			goto out_cdpl2;
> -	}
> -
> -	if (ctx->enable_mba_mbps) {
> -		ret = set_mba_sc(true);
> -		if (ret)
> -			goto out_cdpl3;
> -	}
> -
> -	if (ctx->enable_debug)
> -		resctrl_debug = true;
> -
> -	return 0;
> -
> -out_cdpl3:
> -	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
> -out_cdpl2:
> -	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
> -out_done:
> -	return ret;
> -}
> -
> -static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type)
> -{
> -	struct resctrl_schema *s;
> -	const char *suffix = "";
> -	int ret, cl;
> -
> -	s = kzalloc(sizeof(*s), GFP_KERNEL);
> -	if (!s)
> -		return -ENOMEM;
> -
> -	s->res = r;
> -	s->num_closid = resctrl_arch_get_num_closid(r);
> -	if (resctrl_arch_get_cdp_enabled(r->rid))
> -		s->num_closid /= 2;
> -
> -	s->conf_type = type;
> -	switch (type) {
> -	case CDP_CODE:
> -		suffix = "CODE";
> -		break;
> -	case CDP_DATA:
> -		suffix = "DATA";
> -		break;
> -	case CDP_NONE:
> -		suffix = "";
> -		break;
> -	}
> -
> -	ret = snprintf(s->name, sizeof(s->name), "%s%s", r->name, suffix);
> -	if (ret >= sizeof(s->name)) {
> -		kfree(s);
> -		return -EINVAL;
> -	}
> -
> -	cl = strlen(s->name);
> -
> -	/*
> -	 * If CDP is supported by this resource, but not enabled,
> -	 * include the suffix. This ensures the tabular format of the
> -	 * schemata file does not change between mounts of the filesystem.
> -	 */
> -	if (r->cdp_capable && !resctrl_arch_get_cdp_enabled(r->rid))
> -		cl += 4;
> -
> -	if (cl > max_name_width)
> -		max_name_width = cl;
> -
> -	/*
> -	 * Choose a width for the resource data based on the resource that has
> -	 * widest name and cbm.
> -	 */
> -	max_data_width = max(max_data_width, r->data_width);
> -
> -	INIT_LIST_HEAD(&s->list);
> -	list_add(&s->list, &resctrl_schema_all);
> -
> -	return 0;
> -}
> -
> -static int schemata_list_create(void)
> -{
> -	enum resctrl_res_level i;
> -	struct rdt_resource *r;
> -	int ret = 0;
> -
> -	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> -		r = resctrl_arch_get_resource(i);
> -		if (!r->alloc_capable)
> -			continue;
> -
> -		if (resctrl_arch_get_cdp_enabled(r->rid)) {
> -			ret = schemata_list_add(r, CDP_CODE);
> -			if (ret)
> -				break;
> -
> -			ret = schemata_list_add(r, CDP_DATA);
> -		} else {
> -			ret = schemata_list_add(r, CDP_NONE);
> -		}
> -
> -		if (ret)
> -			break;
> -	}
> -
> -	return ret;
> -}
> -
> -static void schemata_list_destroy(void)
> -{
> -	struct resctrl_schema *s, *tmp;
> -
> -	list_for_each_entry_safe(s, tmp, &resctrl_schema_all, list) {
> -		list_del(&s->list);
> -		kfree(s);
> -	}
> -}
> -
> -static int rdt_get_tree(struct fs_context *fc)
> -{
> -	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> -	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> -	unsigned long flags = RFTYPE_CTRL_BASE;
> -	struct rdt_domain *dom;
> -	int ret;
> -
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -	/*
> -	 * resctrl file system can only be mounted once.
> -	 */
> -	if (resctrl_mounted) {
> -		ret = -EBUSY;
> -		goto out;
> -	}
> -
> -	ret = rdtgroup_setup_root(ctx);
> -	if (ret)
> -		goto out;
> -
> -	ret = rdt_enable_ctx(ctx);
> -	if (ret)
> -		goto out_root;
> -
> -	ret = schemata_list_create();
> -	if (ret) {
> -		schemata_list_destroy();
> -		goto out_ctx;
> -	}
> -
> -	closid_init();
> -
> -	if (resctrl_arch_mon_capable())
> -		flags |= RFTYPE_MON;
> -
> -	ret = rdtgroup_add_files(rdtgroup_default.kn, flags);
> -	if (ret)
> -		goto out_schemata_free;
> -
> -	kernfs_activate(rdtgroup_default.kn);
> -
> -	ret = rdtgroup_create_info_dir(rdtgroup_default.kn);
> -	if (ret < 0)
> -		goto out_schemata_free;
> -
> -	if (resctrl_arch_mon_capable()) {
> -		ret = mongroup_create_dir(rdtgroup_default.kn,
> -					  &rdtgroup_default, "mon_groups",
> -					  &kn_mongrp);
> -		if (ret < 0)
> -			goto out_info;
> -
> -		ret = mkdir_mondata_all(rdtgroup_default.kn,
> -					&rdtgroup_default, &kn_mondata);
> -		if (ret < 0)
> -			goto out_mongrp;
> -		rdtgroup_default.mon.mon_data_kn = kn_mondata;
> -	}
> -
> -	ret = rdt_pseudo_lock_init();
> -	if (ret)
> -		goto out_mondata;
> -
> -	ret = kernfs_get_tree(fc);
> -	if (ret < 0)
> -		goto out_psl;
> -
> -	if (resctrl_arch_alloc_capable())
> -		resctrl_arch_enable_alloc();
> -	if (resctrl_arch_mon_capable())
> -		resctrl_arch_enable_mon();
> -
> -	if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable())
> -		resctrl_mounted = true;
> -
> -	if (resctrl_is_mbm_enabled()) {
> -		list_for_each_entry(dom, &l3->domains, list)
> -			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
> -						   RESCTRL_PICK_ANY_CPU);
> -	}
> -
> -	goto out;
> -
> -out_psl:
> -	rdt_pseudo_lock_release();
> -out_mondata:
> -	if (resctrl_arch_mon_capable())
> -		kernfs_remove(kn_mondata);
> -out_mongrp:
> -	if (resctrl_arch_mon_capable())
> -		kernfs_remove(kn_mongrp);
> -out_info:
> -	kernfs_remove(kn_info);
> -out_schemata_free:
> -	schemata_list_destroy();
> -out_ctx:
> -	rdt_disable_ctx();
> -out_root:
> -	rdtgroup_destroy_root();
> -out:
> -	rdt_last_cmd_clear();
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -	return ret;
> -}
> -
> -enum rdt_param {
> -	Opt_cdp,
> -	Opt_cdpl2,
> -	Opt_mba_mbps,
> -	Opt_debug,
> -	nr__rdt_params
> -};
> -
> -static const struct fs_parameter_spec rdt_fs_parameters[] = {
> -	fsparam_flag("cdp",		Opt_cdp),
> -	fsparam_flag("cdpl2",		Opt_cdpl2),
> -	fsparam_flag("mba_MBps",	Opt_mba_mbps),
> -	fsparam_flag("debug",		Opt_debug),
> -	{}
> -};
> -
> -static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
> -{
> -	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> -	struct fs_parse_result result;
> -	int opt;
> -
> -	opt = fs_parse(fc, rdt_fs_parameters, param, &result);
> -	if (opt < 0)
> -		return opt;
> -
> -	switch (opt) {
> -	case Opt_cdp:
> -		ctx->enable_cdpl3 = true;
> -		return 0;
> -	case Opt_cdpl2:
> -		ctx->enable_cdpl2 = true;
> -		return 0;
> -	case Opt_mba_mbps:
> -		if (!supports_mba_mbps())
> -			return -EINVAL;
> -		ctx->enable_mba_mbps = true;
> -		return 0;
> -	case Opt_debug:
> -		ctx->enable_debug = true;
> -		return 0;
> -	}
> -
> -	return -EINVAL;
> -}
> -
> -static void rdt_fs_context_free(struct fs_context *fc)
> -{
> -	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> -
> -	kernfs_free_fs_context(fc);
> -	kfree(ctx);
> -}
> -
> -static const struct fs_context_operations rdt_fs_context_ops = {
> -	.free		= rdt_fs_context_free,
> -	.parse_param	= rdt_parse_param,
> -	.get_tree	= rdt_get_tree,
> -};
> -
> -static int rdt_init_fs_context(struct fs_context *fc)
> -{
> -	struct rdt_fs_context *ctx;
> -
> -	ctx = kzalloc(sizeof(struct rdt_fs_context), GFP_KERNEL);
> -	if (!ctx)
> -		return -ENOMEM;
> -
> -	ctx->kfc.magic = RDTGROUP_SUPER_MAGIC;
> -	fc->fs_private = &ctx->kfc;
> -	fc->ops = &rdt_fs_context_ops;
> -	put_user_ns(fc->user_ns);
> -	fc->user_ns = get_user_ns(&init_user_ns);
> -	fc->global = true;
> -	return 0;
> -}
> -
>  static int reset_all_ctrls(struct rdt_resource *r)
>  {
>  	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
> @@ -2922,1333 +258,3 @@ void resctrl_arch_reset_resources(void)
>  	for_each_capable_rdt_resource(r)
>  		reset_all_ctrls(r);
>  }
> -
> -/*
> - * Move tasks from one to the other group. If @from is NULL, then all tasks
> - * in the systems are moved unconditionally (used for teardown).
> - *
> - * If @mask is not NULL the cpus on which moved tasks are running are set
> - * in that mask so the update smp function call is restricted to affected
> - * cpus.
> - */
> -static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,
> -				 struct cpumask *mask)
> -{
> -	struct task_struct *p, *t;
> -
> -	read_lock(&tasklist_lock);
> -	for_each_process_thread(p, t) {
> -		if (!from || is_closid_match(t, from) ||
> -		    is_rmid_match(t, from)) {
> -			resctrl_arch_set_closid_rmid(t, to->closid,
> -						     to->mon.rmid);
> -
> -			/*
> -			 * Order the closid/rmid stores above before the loads
> -			 * in task_curr(). This pairs with the full barrier
> -			 * between the rq->curr update and
> -			 * resctrl_arch_sched_in() during context switch.
> -			 */
> -			smp_mb();
> -
> -			/*
> -			 * If the task is on a CPU, set the CPU in the mask.
> -			 * The detection is inaccurate as tasks might move or
> -			 * schedule before the smp function call takes place.
> -			 * In such a case the function call is pointless, but
> -			 * there is no other side effect.
> -			 */
> -			if (IS_ENABLED(CONFIG_SMP) && mask && task_curr(t))
> -				cpumask_set_cpu(task_cpu(t), mask);
> -		}
> -	}
> -	read_unlock(&tasklist_lock);
> -}
> -
> -static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp)
> -{
> -	struct rdtgroup *sentry, *stmp;
> -	struct list_head *head;
> -
> -	head = &rdtgrp->mon.crdtgrp_list;
> -	list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) {
> -		free_rmid(sentry->closid, sentry->mon.rmid);
> -		list_del(&sentry->mon.crdtgrp_list);
> -
> -		if (atomic_read(&sentry->waitcount) != 0)
> -			sentry->flags = RDT_DELETED;
> -		else
> -			rdtgroup_remove(sentry);
> -	}
> -}
> -
> -/*
> - * Forcibly remove all of subdirectories under root.
> - */
> -static void rmdir_all_sub(void)
> -{
> -	struct rdtgroup *rdtgrp, *tmp;
> -
> -	/* Move all tasks to the default resource group */
> -	rdt_move_group_tasks(NULL, &rdtgroup_default, NULL);
> -
> -	list_for_each_entry_safe(rdtgrp, tmp, &rdt_all_groups, rdtgroup_list) {
> -		/* Free any child rmids */
> -		free_all_child_rdtgrp(rdtgrp);
> -
> -		/* Remove each rdtgroup other than root */
> -		if (rdtgrp == &rdtgroup_default)
> -			continue;
> -
> -		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> -		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
> -			rdtgroup_pseudo_lock_remove(rdtgrp);
> -
> -		/*
> -		 * Give any CPUs back to the default group. We cannot copy
> -		 * cpu_online_mask because a CPU might have executed the
> -		 * offline callback already, but is still marked online.
> -		 */
> -		cpumask_or(&rdtgroup_default.cpu_mask,
> -			   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
> -
> -		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> -
> -		kernfs_remove(rdtgrp->kn);
> -		list_del(&rdtgrp->rdtgroup_list);
> -
> -		if (atomic_read(&rdtgrp->waitcount) != 0)
> -			rdtgrp->flags = RDT_DELETED;
> -		else
> -			rdtgroup_remove(rdtgrp);
> -	}
> -	/* Notify online CPUs to update per cpu storage and PQR_ASSOC MSR */
> -	update_closid_rmid(cpu_online_mask, &rdtgroup_default);
> -
> -	kernfs_remove(kn_info);
> -	kernfs_remove(kn_mongrp);
> -	kernfs_remove(kn_mondata);
> -}
> -
> -static void rdt_kill_sb(struct super_block *sb)
> -{
> -	cpus_read_lock();
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	rdt_disable_ctx();
> -
> -	/* Put everything back to default values. */
> -	resctrl_arch_reset_resources();
> -
> -	rmdir_all_sub();
> -	rdt_pseudo_lock_release();
> -	rdtgroup_default.mode = RDT_MODE_SHAREABLE;
> -	schemata_list_destroy();
> -	rdtgroup_destroy_root();
> -	if (resctrl_arch_alloc_capable())
> -		resctrl_arch_disable_alloc();
> -	if (resctrl_arch_mon_capable())
> -		resctrl_arch_disable_mon();
> -	resctrl_mounted = false;
> -	kernfs_kill_sb(sb);
> -	mutex_unlock(&rdtgroup_mutex);
> -	cpus_read_unlock();
> -}
> -
> -static struct file_system_type rdt_fs_type = {
> -	.name			= "resctrl",
> -	.init_fs_context	= rdt_init_fs_context,
> -	.parameters		= rdt_fs_parameters,
> -	.kill_sb		= rdt_kill_sb,
> -};
> -
> -static int mon_addfile(struct kernfs_node *parent_kn, const char *name,
> -		       void *priv)
> -{
> -	struct kernfs_node *kn;
> -	int ret = 0;
> -
> -	kn = __kernfs_create_file(parent_kn, name, 0444,
> -				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, 0,
> -				  &kf_mondata_ops, priv, NULL, NULL);
> -	if (IS_ERR(kn))
> -		return PTR_ERR(kn);
> -
> -	ret = rdtgroup_kn_set_ugid(kn);
> -	if (ret) {
> -		kernfs_remove(kn);
> -		return ret;
> -	}
> -
> -	return ret;
> -}
> -
> -/*
> - * Remove all subdirectories of mon_data of ctrl_mon groups
> - * and monitor groups with given domain id.
> - */
> -static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
> -					   unsigned int dom_id)
> -{
> -	struct rdtgroup *prgrp, *crgrp;
> -	char name[32];
> -
> -	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> -		sprintf(name, "mon_%s_%02d", r->name, dom_id);
> -		kernfs_remove_by_name(prgrp->mon.mon_data_kn, name);
> -
> -		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
> -			kernfs_remove_by_name(crgrp->mon.mon_data_kn, name);
> -	}
> -}
> -
> -static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,
> -				struct rdt_domain *d,
> -				struct rdt_resource *r, struct rdtgroup *prgrp)
> -{
> -	union mon_data_bits priv;
> -	struct kernfs_node *kn;
> -	struct mon_evt *mevt;
> -	struct rmid_read rr;
> -	char name[32];
> -	int ret;
> -
> -	sprintf(name, "mon_%s_%02d", r->name, d->id);
> -	/* create the directory */
> -	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
> -	if (IS_ERR(kn))
> -		return PTR_ERR(kn);
> -
> -	ret = rdtgroup_kn_set_ugid(kn);
> -	if (ret)
> -		goto out_destroy;
> -
> -	if (WARN_ON(list_empty(&r->evt_list))) {
> -		ret = -EPERM;
> -		goto out_destroy;
> -	}
> -
> -	priv.u.rid = r->rid;
> -	priv.u.domid = d->id;
> -	list_for_each_entry(mevt, &r->evt_list, list) {
> -		priv.u.evtid = mevt->evtid;
> -		ret = mon_addfile(kn, mevt->name, priv.priv);
> -		if (ret)
> -			goto out_destroy;
> -
> -		if (resctrl_is_mbm_event(mevt->evtid))
> -			mon_event_read(&rr, r, d, prgrp, mevt->evtid, true);
> -	}
> -	kernfs_activate(kn);
> -	return 0;
> -
> -out_destroy:
> -	kernfs_remove(kn);
> -	return ret;
> -}
> -
> -/*
> - * Add all subdirectories of mon_data for "ctrl_mon" groups
> - * and "monitor" groups with given domain id.
> - */
> -static void mkdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
> -					   struct rdt_domain *d)
> -{
> -	struct kernfs_node *parent_kn;
> -	struct rdtgroup *prgrp, *crgrp;
> -	struct list_head *head;
> -
> -	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> -		parent_kn = prgrp->mon.mon_data_kn;
> -		mkdir_mondata_subdir(parent_kn, d, r, prgrp);
> -
> -		head = &prgrp->mon.crdtgrp_list;
> -		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
> -			parent_kn = crgrp->mon.mon_data_kn;
> -			mkdir_mondata_subdir(parent_kn, d, r, crgrp);
> -		}
> -	}
> -}
> -
> -static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn,
> -				       struct rdt_resource *r,
> -				       struct rdtgroup *prgrp)
> -{
> -	struct rdt_domain *dom;
> -	int ret;
> -
> -	/* Walking r->domains, ensure it can't race with cpuhp */
> -	lockdep_assert_cpus_held();
> -
> -	list_for_each_entry(dom, &r->domains, list) {
> -		ret = mkdir_mondata_subdir(parent_kn, dom, r, prgrp);
> -		if (ret)
> -			return ret;
> -	}
> -
> -	return 0;
> -}
> -
> -/*
> - * This creates a directory mon_data which contains the monitored data.
> - *
> - * mon_data has one directory for each domain which are named
> - * in the format mon_<domain_name>_<domain_id>. For ex: A mon_data
> - * with L3 domain looks as below:
> - * ./mon_data:
> - * mon_L3_00
> - * mon_L3_01
> - * mon_L3_02
> - * ...
> - *
> - * Each domain directory has one file per event:
> - * ./mon_L3_00/:
> - * llc_occupancy
> - *
> - */
> -static int mkdir_mondata_all(struct kernfs_node *parent_kn,
> -			     struct rdtgroup *prgrp,
> -			     struct kernfs_node **dest_kn)
> -{
> -	enum resctrl_res_level i;
> -	struct rdt_resource *r;
> -	struct kernfs_node *kn;
> -	int ret;
> -
> -	/*
> -	 * Create the mon_data directory first.
> -	 */
> -	ret = mongroup_create_dir(parent_kn, prgrp, "mon_data", &kn);
> -	if (ret)
> -		return ret;
> -
> -	if (dest_kn)
> -		*dest_kn = kn;
> -
> -	/*
> -	 * Create the subdirectories for each domain. Note that all events
> -	 * in a domain like L3 are grouped into a resource whose domain is L3
> -	 */
> -	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> -		r = resctrl_arch_get_resource(i);
> -		if (!r->mon_capable)
> -			continue;
> -
> -		ret = mkdir_mondata_subdir_alldom(kn, r, prgrp);
> -		if (ret)
> -			goto out_destroy;
> -	}
> -
> -	return 0;
> -
> -out_destroy:
> -	kernfs_remove(kn);
> -	return ret;
> -}
> -
> -/**
> - * cbm_ensure_valid - Enforce validity on provided CBM
> - * @_val:	Candidate CBM
> - * @r:		RDT resource to which the CBM belongs
> - *
> - * The provided CBM represents all cache portions available for use. This
> - * may be represented by a bitmap that does not consist of contiguous ones
> - * and thus be an invalid CBM.
> - * Here the provided CBM is forced to be a valid CBM by only considering
> - * the first set of contiguous bits as valid and clearing all bits.
> - * The intention here is to provide a valid default CBM with which a new
> - * resource group is initialized. The user can follow this with a
> - * modification to the CBM if the default does not satisfy the
> - * requirements.
> - */
> -static u32 cbm_ensure_valid(u32 _val, struct rdt_resource *r)
> -{
> -	unsigned int cbm_len = r->cache.cbm_len;
> -	unsigned long first_bit, zero_bit;
> -	unsigned long val = _val;
> -
> -	if (!val)
> -		return 0;
> -
> -	first_bit = find_first_bit(&val, cbm_len);
> -	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
> -
> -	/* Clear any remaining bits to ensure contiguous region */
> -	bitmap_clear(&val, zero_bit, cbm_len - zero_bit);
> -	return (u32)val;
> -}
> -
> -/*
> - * Initialize cache resources per RDT domain
> - *
> - * Set the RDT domain up to start off with all usable allocations. That is,
> - * all shareable and unused bits. All-zero CBM is invalid.
> - */
> -static int __init_one_rdt_domain(struct rdt_domain *d, struct resctrl_schema *s,
> -				 u32 closid)
> -{
> -	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
> -	enum resctrl_conf_type t = s->conf_type;
> -	struct resctrl_staged_config *cfg;
> -	struct rdt_resource *r = s->res;
> -	u32 used_b = 0, unused_b = 0;
> -	unsigned long tmp_cbm;
> -	enum rdtgrp_mode mode;
> -	u32 peer_ctl, ctrl_val;
> -	int i;
> -
> -	cfg = &d->staged_config[t];
> -	cfg->have_new_ctrl = false;
> -	cfg->new_ctrl = r->cache.shareable_bits;
> -	used_b = r->cache.shareable_bits;
> -	for (i = 0; i < closids_supported(); i++) {
> -		if (closid_allocated(i) && i != closid) {
> -			mode = rdtgroup_mode_by_closid(i);
> -			if (mode == RDT_MODE_PSEUDO_LOCKSETUP)
> -				/*
> -				 * ctrl values for locksetup aren't relevant
> -				 * until the schemata is written, and the mode
> -				 * becomes RDT_MODE_PSEUDO_LOCKED.
> -				 */
> -				continue;
> -			/*
> -			 * If CDP is active include peer domain's
> -			 * usage to ensure there is no overlap
> -			 * with an exclusive group.
> -			 */
> -			if (resctrl_arch_get_cdp_enabled(r->rid))
> -				peer_ctl = resctrl_arch_get_config(r, d, i,
> -								   peer_type);
> -			else
> -				peer_ctl = 0;
> -			ctrl_val = resctrl_arch_get_config(r, d, i,
> -							   s->conf_type);
> -			used_b |= ctrl_val | peer_ctl;
> -			if (mode == RDT_MODE_SHAREABLE)
> -				cfg->new_ctrl |= ctrl_val | peer_ctl;
> -		}
> -	}
> -	if (d->plr && d->plr->cbm > 0)
> -		used_b |= d->plr->cbm;
> -	unused_b = used_b ^ (BIT_MASK(r->cache.cbm_len) - 1);
> -	unused_b &= BIT_MASK(r->cache.cbm_len) - 1;
> -	cfg->new_ctrl |= unused_b;
> -	/*
> -	 * Force the initial CBM to be valid, user can
> -	 * modify the CBM based on system availability.
> -	 */
> -	cfg->new_ctrl = cbm_ensure_valid(cfg->new_ctrl, r);
> -	/*
> -	 * Assign the u32 CBM to an unsigned long to ensure that
> -	 * bitmap_weight() does not access out-of-bound memory.
> -	 */
> -	tmp_cbm = cfg->new_ctrl;
> -	if (bitmap_weight(&tmp_cbm, r->cache.cbm_len) < r->cache.min_cbm_bits) {
> -		rdt_last_cmd_printf("No space on %s:%d\n", s->name, d->id);
> -		return -ENOSPC;
> -	}
> -	cfg->have_new_ctrl = true;
> -
> -	return 0;
> -}
> -
> -/*
> - * Initialize cache resources with default values.
> - *
> - * A new RDT group is being created on an allocation capable (CAT)
> - * supporting system. Set this group up to start off with all usable
> - * allocations.
> - *
> - * If there are no more shareable bits available on any domain then
> - * the entire allocation will fail.
> - */
> -static int rdtgroup_init_cat(struct resctrl_schema *s, u32 closid)
> -{
> -	struct rdt_domain *d;
> -	int ret;
> -
> -	list_for_each_entry(d, &s->res->domains, list) {
> -		ret = __init_one_rdt_domain(d, s, closid);
> -		if (ret < 0)
> -			return ret;
> -	}
> -
> -	return 0;
> -}
> -
> -/* Initialize MBA resource with default values. */
> -static void rdtgroup_init_mba(struct rdt_resource *r, u32 closid)
> -{
> -	struct resctrl_staged_config *cfg;
> -	struct rdt_domain *d;
> -
> -	list_for_each_entry(d, &r->domains, list) {
> -		if (is_mba_sc(r)) {
> -			d->mbps_val[closid] = MBA_MAX_MBPS;
> -			continue;
> -		}
> -
> -		cfg = &d->staged_config[CDP_NONE];
> -		cfg->new_ctrl = r->default_ctrl;
> -		cfg->have_new_ctrl = true;
> -	}
> -}
> -
> -/* Initialize the RDT group's allocations. */
> -static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
> -{
> -	struct resctrl_schema *s;
> -	struct rdt_resource *r;
> -	int ret = 0;
> -
> -	rdt_staged_configs_clear();
> -
> -	list_for_each_entry(s, &resctrl_schema_all, list) {
> -		r = s->res;
> -		if (r->rid == RDT_RESOURCE_MBA ||
> -		    r->rid == RDT_RESOURCE_SMBA) {
> -			rdtgroup_init_mba(r, rdtgrp->closid);
> -			if (is_mba_sc(r))
> -				continue;
> -		} else {
> -			ret = rdtgroup_init_cat(s, rdtgrp->closid);
> -			if (ret < 0)
> -				goto out;
> -		}
> -
> -		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
> -		if (ret < 0) {
> -			rdt_last_cmd_puts("Failed to initialize allocations\n");
> -			goto out;
> -		}
> -
> -	}
> -
> -	rdtgrp->mode = RDT_MODE_SHAREABLE;
> -
> -out:
> -	rdt_staged_configs_clear();
> -	return ret;
> -}
> -
> -static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)
> -{
> -	int ret;
> -
> -	if (!resctrl_arch_mon_capable())
> -		return 0;
> -
> -	ret = alloc_rmid(rdtgrp->closid);
> -	if (ret < 0) {
> -		rdt_last_cmd_puts("Out of RMIDs\n");
> -		return ret;
> -	}
> -	rdtgrp->mon.rmid = ret;
> -
> -	ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn);
> -	if (ret) {
> -		rdt_last_cmd_puts("kernfs subdir error\n");
> -		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> -		return ret;
> -	}
> -
> -	return 0;
> -}
> -
> -static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp)
> -{
> -	if (resctrl_arch_mon_capable())
> -		free_rmid(rgrp->closid, rgrp->mon.rmid);
> -}
> -
> -static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,
> -			     const char *name, umode_t mode,
> -			     enum rdt_group_type rtype, struct rdtgroup **r)
> -{
> -	struct rdtgroup *prdtgrp, *rdtgrp;
> -	unsigned long files = 0;
> -	struct kernfs_node *kn;
> -	int ret;
> -
> -	prdtgrp = rdtgroup_kn_lock_live(parent_kn);
> -	if (!prdtgrp) {
> -		ret = -ENODEV;
> -		goto out_unlock;
> -	}
> -
> -	if (rtype == RDTMON_GROUP &&
> -	    (prdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> -	     prdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)) {
> -		ret = -EINVAL;
> -		rdt_last_cmd_puts("Pseudo-locking in progress\n");
> -		goto out_unlock;
> -	}
> -
> -	/* allocate the rdtgroup. */
> -	rdtgrp = kzalloc(sizeof(*rdtgrp), GFP_KERNEL);
> -	if (!rdtgrp) {
> -		ret = -ENOSPC;
> -		rdt_last_cmd_puts("Kernel out of memory\n");
> -		goto out_unlock;
> -	}
> -	*r = rdtgrp;
> -	rdtgrp->mon.parent = prdtgrp;
> -	rdtgrp->type = rtype;
> -	INIT_LIST_HEAD(&rdtgrp->mon.crdtgrp_list);
> -
> -	/* kernfs creates the directory for rdtgrp */
> -	kn = kernfs_create_dir(parent_kn, name, mode, rdtgrp);
> -	if (IS_ERR(kn)) {
> -		ret = PTR_ERR(kn);
> -		rdt_last_cmd_puts("kernfs create error\n");
> -		goto out_free_rgrp;
> -	}
> -	rdtgrp->kn = kn;
> -
> -	/*
> -	 * kernfs_remove() will drop the reference count on "kn" which
> -	 * will free it. But we still need it to stick around for the
> -	 * rdtgroup_kn_unlock(kn) call. Take one extra reference here,
> -	 * which will be dropped by kernfs_put() in rdtgroup_remove().
> -	 */
> -	kernfs_get(kn);
> -
> -	ret = rdtgroup_kn_set_ugid(kn);
> -	if (ret) {
> -		rdt_last_cmd_puts("kernfs perm error\n");
> -		goto out_destroy;
> -	}
> -
> -	if (rtype == RDTCTRL_GROUP) {
> -		files = RFTYPE_BASE | RFTYPE_CTRL;
> -		if (resctrl_arch_mon_capable())
> -			files |= RFTYPE_MON;
> -	} else {
> -		files = RFTYPE_BASE | RFTYPE_MON;
> -	}
> -
> -	ret = rdtgroup_add_files(kn, files);
> -	if (ret) {
> -		rdt_last_cmd_puts("kernfs fill error\n");
> -		goto out_destroy;
> -	}
> -
> -	/*
> -	 * The caller unlocks the parent_kn upon success.
> -	 */
> -	return 0;
> -
> -out_destroy:
> -	kernfs_put(rdtgrp->kn);
> -	kernfs_remove(rdtgrp->kn);
> -out_free_rgrp:
> -	kfree(rdtgrp);
> -out_unlock:
> -	rdtgroup_kn_unlock(parent_kn);
> -	return ret;
> -}
> -
> -static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp)
> -{
> -	kernfs_remove(rgrp->kn);
> -	rdtgroup_remove(rgrp);
> -}
> -
> -/*
> - * Create a monitor group under "mon_groups" directory of a control
> - * and monitor group(ctrl_mon). This is a resource group
> - * to monitor a subset of tasks and cpus in its parent ctrl_mon group.
> - */
> -static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn,
> -			      const char *name, umode_t mode)
> -{
> -	struct rdtgroup *rdtgrp, *prgrp;
> -	int ret;
> -
> -	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTMON_GROUP, &rdtgrp);
> -	if (ret)
> -		return ret;
> -
> -	prgrp = rdtgrp->mon.parent;
> -	rdtgrp->closid = prgrp->closid;
> -
> -	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
> -	if (ret) {
> -		mkdir_rdt_prepare_clean(rdtgrp);
> -		goto out_unlock;
> -	}
> -
> -	kernfs_activate(rdtgrp->kn);
> -
> -	/*
> -	 * Add the rdtgrp to the list of rdtgrps the parent
> -	 * ctrl_mon group has to track.
> -	 */
> -	list_add_tail(&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list);
> -
> -out_unlock:
> -	rdtgroup_kn_unlock(parent_kn);
> -	return ret;
> -}
> -
> -/*
> - * These are rdtgroups created under the root directory. Can be used
> - * to allocate and monitor resources.
> - */
> -static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,
> -				   const char *name, umode_t mode)
> -{
> -	struct rdtgroup *rdtgrp;
> -	struct kernfs_node *kn;
> -	u32 closid;
> -	int ret;
> -
> -	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTCTRL_GROUP, &rdtgrp);
> -	if (ret)
> -		return ret;
> -
> -	kn = rdtgrp->kn;
> -	ret = closid_alloc();
> -	if (ret < 0) {
> -		rdt_last_cmd_puts("Out of CLOSIDs\n");
> -		goto out_common_fail;
> -	}
> -	closid = ret;
> -	ret = 0;
> -
> -	rdtgrp->closid = closid;
> -
> -	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
> -	if (ret)
> -		goto out_closid_free;
> -
> -	kernfs_activate(rdtgrp->kn);
> -
> -	ret = rdtgroup_init_alloc(rdtgrp);
> -	if (ret < 0)
> -		goto out_rmid_free;
> -
> -	list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups);
> -
> -	if (resctrl_arch_mon_capable()) {
> -		/*
> -		 * Create an empty mon_groups directory to hold the subset
> -		 * of tasks and cpus to monitor.
> -		 */
> -		ret = mongroup_create_dir(kn, rdtgrp, "mon_groups", NULL);
> -		if (ret) {
> -			rdt_last_cmd_puts("kernfs subdir error\n");
> -			goto out_del_list;
> -		}
> -	}
> -
> -	goto out_unlock;
> -
> -out_del_list:
> -	list_del(&rdtgrp->rdtgroup_list);
> -out_rmid_free:
> -	mkdir_rdt_prepare_rmid_free(rdtgrp);
> -out_closid_free:
> -	closid_free(closid);
> -out_common_fail:
> -	mkdir_rdt_prepare_clean(rdtgrp);
> -out_unlock:
> -	rdtgroup_kn_unlock(parent_kn);
> -	return ret;
> -}
> -
> -/*
> - * We allow creating mon groups only with in a directory called "mon_groups"
> - * which is present in every ctrl_mon group. Check if this is a valid
> - * "mon_groups" directory.
> - *
> - * 1. The directory should be named "mon_groups".
> - * 2. The mon group itself should "not" be named "mon_groups".
> - *   This makes sure "mon_groups" directory always has a ctrl_mon group
> - *   as parent.
> - */
> -static bool is_mon_groups(struct kernfs_node *kn, const char *name)
> -{
> -	return (!strcmp(kn->name, "mon_groups") &&
> -		strcmp(name, "mon_groups"));
> -}
> -
> -static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
> -			  umode_t mode)
> -{
> -	/* Do not accept '\n' to avoid unparsable situation. */
> -	if (strchr(name, '\n'))
> -		return -EINVAL;
> -
> -	/*
> -	 * If the parent directory is the root directory and RDT
> -	 * allocation is supported, add a control and monitoring
> -	 * subdirectory
> -	 */
> -	if (resctrl_arch_alloc_capable() && parent_kn == rdtgroup_default.kn)
> -		return rdtgroup_mkdir_ctrl_mon(parent_kn, name, mode);
> -
> -	/*
> -	 * If RDT monitoring is supported and the parent directory is a valid
> -	 * "mon_groups" directory, add a monitoring subdirectory.
> -	 */
> -	if (resctrl_arch_mon_capable() && is_mon_groups(parent_kn, name))
> -		return rdtgroup_mkdir_mon(parent_kn, name, mode);
> -
> -	return -EPERM;
> -}
> -
> -static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
> -{
> -	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
> -	u32 closid, rmid;
> -	int cpu;
> -
> -	/* Give any tasks back to the parent group */
> -	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
> -
> -	/* Update per cpu rmid of the moved CPUs first */
> -	closid = rdtgrp->closid;
> -	rmid = prdtgrp->mon.rmid;
> -	for_each_cpu(cpu, &rdtgrp->cpu_mask)
> -		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
> -
> -	/*
> -	 * Update the MSR on moved CPUs and CPUs which have moved
> -	 * task running on them.
> -	 */
> -	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
> -	update_closid_rmid(tmpmask, NULL);
> -
> -	rdtgrp->flags = RDT_DELETED;
> -	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> -
> -	/*
> -	 * Remove the rdtgrp from the parent ctrl_mon group's list
> -	 */
> -	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
> -	list_del(&rdtgrp->mon.crdtgrp_list);
> -
> -	kernfs_remove(rdtgrp->kn);
> -
> -	return 0;
> -}
> -
> -static int rdtgroup_ctrl_remove(struct rdtgroup *rdtgrp)
> -{
> -	rdtgrp->flags = RDT_DELETED;
> -	list_del(&rdtgrp->rdtgroup_list);
> -
> -	kernfs_remove(rdtgrp->kn);
> -	return 0;
> -}
> -
> -static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
> -{
> -	u32 closid, rmid;
> -	int cpu;
> -
> -	/* Give any tasks back to the default group */
> -	rdt_move_group_tasks(rdtgrp, &rdtgroup_default, tmpmask);
> -
> -	/* Give any CPUs back to the default group */
> -	cpumask_or(&rdtgroup_default.cpu_mask,
> -		   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
> -
> -	/* Update per cpu closid and rmid of the moved CPUs first */
> -	closid = rdtgroup_default.closid;
> -	rmid = rdtgroup_default.mon.rmid;
> -	for_each_cpu(cpu, &rdtgrp->cpu_mask)
> -		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
> -
> -	/*
> -	 * Update the MSR on moved CPUs and CPUs which have moved
> -	 * task running on them.
> -	 */
> -	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
> -	update_closid_rmid(tmpmask, NULL);
> -
> -	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> -	closid_free(rdtgrp->closid);
> -
> -	rdtgroup_ctrl_remove(rdtgrp);
> -
> -	/*
> -	 * Free all the child monitor group rmids.
> -	 */
> -	free_all_child_rdtgrp(rdtgrp);
> -
> -	return 0;
> -}
> -
> -static int rdtgroup_rmdir(struct kernfs_node *kn)
> -{
> -	struct kernfs_node *parent_kn = kn->parent;
> -	struct rdtgroup *rdtgrp;
> -	cpumask_var_t tmpmask;
> -	int ret = 0;
> -
> -	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
> -		return -ENOMEM;
> -
> -	rdtgrp = rdtgroup_kn_lock_live(kn);
> -	if (!rdtgrp) {
> -		ret = -EPERM;
> -		goto out;
> -	}
> -
> -	/*
> -	 * If the rdtgroup is a ctrl_mon group and parent directory
> -	 * is the root directory, remove the ctrl_mon group.
> -	 *
> -	 * If the rdtgroup is a mon group and parent directory
> -	 * is a valid "mon_groups" directory, remove the mon group.
> -	 */
> -	if (rdtgrp->type == RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn &&
> -	    rdtgrp != &rdtgroup_default) {
> -		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> -		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> -			ret = rdtgroup_ctrl_remove(rdtgrp);
> -		} else {
> -			ret = rdtgroup_rmdir_ctrl(rdtgrp, tmpmask);
> -		}
> -	} else if (rdtgrp->type == RDTMON_GROUP &&
> -		 is_mon_groups(parent_kn, kn->name)) {
> -		ret = rdtgroup_rmdir_mon(rdtgrp, tmpmask);
> -	} else {
> -		ret = -EPERM;
> -	}
> -
> -out:
> -	rdtgroup_kn_unlock(kn);
> -	free_cpumask_var(tmpmask);
> -	return ret;
> -}
> -
> -/**
> - * mongrp_reparent() - replace parent CTRL_MON group of a MON group
> - * @rdtgrp:		the MON group whose parent should be replaced
> - * @new_prdtgrp:	replacement parent CTRL_MON group for @rdtgrp
> - * @cpus:		cpumask provided by the caller for use during this call
> - *
> - * Replaces the parent CTRL_MON group for a MON group, resulting in all member
> - * tasks' CLOSID immediately changing to that of the new parent group.
> - * Monitoring data for the group is unaffected by this operation.
> - */
> -static void mongrp_reparent(struct rdtgroup *rdtgrp,
> -			    struct rdtgroup *new_prdtgrp,
> -			    cpumask_var_t cpus)
> -{
> -	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
> -
> -	WARN_ON(rdtgrp->type != RDTMON_GROUP);
> -	WARN_ON(new_prdtgrp->type != RDTCTRL_GROUP);
> -
> -	/* Nothing to do when simply renaming a MON group. */
> -	if (prdtgrp == new_prdtgrp)
> -		return;
> -
> -	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
> -	list_move_tail(&rdtgrp->mon.crdtgrp_list,
> -		       &new_prdtgrp->mon.crdtgrp_list);
> -
> -	rdtgrp->mon.parent = new_prdtgrp;
> -	rdtgrp->closid = new_prdtgrp->closid;
> -
> -	/* Propagate updated closid to all tasks in this group. */
> -	rdt_move_group_tasks(rdtgrp, rdtgrp, cpus);
> -
> -	update_closid_rmid(cpus, NULL);
> -}
> -
> -static int rdtgroup_rename(struct kernfs_node *kn,
> -			   struct kernfs_node *new_parent, const char *new_name)
> -{
> -	struct rdtgroup *new_prdtgrp;
> -	struct rdtgroup *rdtgrp;
> -	cpumask_var_t tmpmask;
> -	int ret;
> -
> -	rdtgrp = kernfs_to_rdtgroup(kn);
> -	new_prdtgrp = kernfs_to_rdtgroup(new_parent);
> -	if (!rdtgrp || !new_prdtgrp)
> -		return -ENOENT;
> -
> -	/* Release both kernfs active_refs before obtaining rdtgroup mutex. */
> -	rdtgroup_kn_get(rdtgrp, kn);
> -	rdtgroup_kn_get(new_prdtgrp, new_parent);
> -
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	rdt_last_cmd_clear();
> -
> -	/*
> -	 * Don't allow kernfs_to_rdtgroup() to return a parent rdtgroup if
> -	 * either kernfs_node is a file.
> -	 */
> -	if (kernfs_type(kn) != KERNFS_DIR ||
> -	    kernfs_type(new_parent) != KERNFS_DIR) {
> -		rdt_last_cmd_puts("Source and destination must be directories");
> -		ret = -EPERM;
> -		goto out;
> -	}
> -
> -	if ((rdtgrp->flags & RDT_DELETED) || (new_prdtgrp->flags & RDT_DELETED)) {
> -		ret = -ENOENT;
> -		goto out;
> -	}
> -
> -	if (rdtgrp->type != RDTMON_GROUP || !kn->parent ||
> -	    !is_mon_groups(kn->parent, kn->name)) {
> -		rdt_last_cmd_puts("Source must be a MON group\n");
> -		ret = -EPERM;
> -		goto out;
> -	}
> -
> -	if (!is_mon_groups(new_parent, new_name)) {
> -		rdt_last_cmd_puts("Destination must be a mon_groups subdirectory\n");
> -		ret = -EPERM;
> -		goto out;
> -	}
> -
> -	/*
> -	 * If the MON group is monitoring CPUs, the CPUs must be assigned to the
> -	 * current parent CTRL_MON group and therefore cannot be assigned to
> -	 * the new parent, making the move illegal.
> -	 */
> -	if (!cpumask_empty(&rdtgrp->cpu_mask) &&
> -	    rdtgrp->mon.parent != new_prdtgrp) {
> -		rdt_last_cmd_puts("Cannot move a MON group that monitors CPUs\n");
> -		ret = -EPERM;
> -		goto out;
> -	}
> -
> -	/*
> -	 * Allocate the cpumask for use in mongrp_reparent() to avoid the
> -	 * possibility of failing to allocate it after kernfs_rename() has
> -	 * succeeded.
> -	 */
> -	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL)) {
> -		ret = -ENOMEM;
> -		goto out;
> -	}
> -
> -	/*
> -	 * Perform all input validation and allocations needed to ensure
> -	 * mongrp_reparent() will succeed before calling kernfs_rename(),
> -	 * otherwise it would be necessary to revert this call if
> -	 * mongrp_reparent() failed.
> -	 */
> -	ret = kernfs_rename(kn, new_parent, new_name);
> -	if (!ret)
> -		mongrp_reparent(rdtgrp, new_prdtgrp, tmpmask);
> -
> -	free_cpumask_var(tmpmask);
> -
> -out:
> -	mutex_unlock(&rdtgroup_mutex);
> -	rdtgroup_kn_put(rdtgrp, kn);
> -	rdtgroup_kn_put(new_prdtgrp, new_parent);
> -	return ret;
> -}
> -
> -static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
> -{
> -	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3))
> -		seq_puts(seq, ",cdp");
> -
> -	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2))
> -		seq_puts(seq, ",cdpl2");
> -
> -	if (is_mba_sc(resctrl_arch_get_resource(RDT_RESOURCE_MBA)))
> -		seq_puts(seq, ",mba_MBps");
> -
> -	if (resctrl_debug)
> -		seq_puts(seq, ",debug");
> -
> -	return 0;
> -}
> -
> -static struct kernfs_syscall_ops rdtgroup_kf_syscall_ops = {
> -	.mkdir		= rdtgroup_mkdir,
> -	.rmdir		= rdtgroup_rmdir,
> -	.rename		= rdtgroup_rename,
> -	.show_options	= rdtgroup_show_options,
> -};
> -
> -static int rdtgroup_setup_root(struct rdt_fs_context *ctx)
> -{
> -	rdt_root = kernfs_create_root(&rdtgroup_kf_syscall_ops,
> -				      KERNFS_ROOT_CREATE_DEACTIVATED |
> -				      KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK,
> -				      &rdtgroup_default);
> -	if (IS_ERR(rdt_root))
> -		return PTR_ERR(rdt_root);
> -
> -	ctx->kfc.root = rdt_root;
> -	rdtgroup_default.kn = kernfs_root_to_node(rdt_root);
> -
> -	return 0;
> -}
> -
> -static void rdtgroup_destroy_root(void)
> -{
> -	kernfs_destroy_root(rdt_root);
> -	rdtgroup_default.kn = NULL;
> -}
> -
> -static void rdtgroup_setup_default(void)
> -{
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	rdtgroup_default.closid = RESCTRL_RESERVED_CLOSID;
> -	rdtgroup_default.mon.rmid = RESCTRL_RESERVED_RMID;
> -	rdtgroup_default.type = RDTCTRL_GROUP;
> -	INIT_LIST_HEAD(&rdtgroup_default.mon.crdtgrp_list);
> -
> -	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
> -
> -	mutex_unlock(&rdtgroup_mutex);
> -}
> -
> -static void domain_destroy_mon_state(struct rdt_domain *d)
> -{
> -	bitmap_free(d->rmid_busy_llc);
> -	kfree(d->mbm_total);
> -	kfree(d->mbm_local);
> -}
> -
> -void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)
> -{
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA)
> -		mba_sc_domain_destroy(r, d);
> -
> -	if (!r->mon_capable)
> -		goto out_unlock;
> -
> -	/*
> -	 * If resctrl is mounted, remove all the
> -	 * per domain monitor data directories.
> -	 */
> -	if (resctrl_mounted && resctrl_arch_mon_capable())
> -		rmdir_mondata_subdir_allrdtgrp(r, d->id);
> -
> -	if (resctrl_is_mbm_enabled())
> -		cancel_delayed_work(&d->mbm_over);
> -	if (resctrl_arch_is_llc_occupancy_enabled() && has_busy_rmid(d)) {
> -		/*
> -		 * When a package is going down, forcefully
> -		 * decrement rmid->ebusy. There is no way to know
> -		 * that the L3 was flushed and hence may lead to
> -		 * incorrect counts in rare scenarios, but leaving
> -		 * the RMID as busy creates RMID leaks if the
> -		 * package never comes back.
> -		 */
> -		__check_limbo(d, true);
> -		cancel_delayed_work(&d->cqm_limbo);
> -	}
> -
> -	domain_destroy_mon_state(d);
> -
> -out_unlock:
> -	mutex_unlock(&rdtgroup_mutex);
> -}
> -
> -static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
> -{
> -	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> -	size_t tsize;
> -
> -	if (resctrl_arch_is_llc_occupancy_enabled()) {
> -		d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL);
> -		if (!d->rmid_busy_llc)
> -			return -ENOMEM;
> -	}
> -	if (resctrl_arch_is_mbm_total_enabled()) {
> -		tsize = sizeof(*d->mbm_total);
> -		d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL);
> -		if (!d->mbm_total) {
> -			bitmap_free(d->rmid_busy_llc);
> -			return -ENOMEM;
> -		}
> -	}
> -	if (resctrl_arch_is_mbm_local_enabled()) {
> -		tsize = sizeof(*d->mbm_local);
> -		d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL);
> -		if (!d->mbm_local) {
> -			bitmap_free(d->rmid_busy_llc);
> -			kfree(d->mbm_total);
> -			return -ENOMEM;
> -		}
> -	}
> -
> -	return 0;
> -}
> -
> -int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)
> -{
> -	int err = 0;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -
> -	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA) {
> -		/* RDT_RESOURCE_MBA is never mon_capable */
> -		err = mba_sc_domain_allocate(r, d);
> -		goto out_unlock;
> -	}
> -
> -	if (!r->mon_capable)
> -		goto out_unlock;
> -
> -	err = domain_setup_mon_state(r, d);
> -	if (err)
> -		goto out_unlock;
> -
> -	if (resctrl_is_mbm_enabled()) {
> -		INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow);
> -		mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL,
> -					   RESCTRL_PICK_ANY_CPU);
> -	}
> -
> -	if (resctrl_arch_is_llc_occupancy_enabled())
> -		INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo);
> -
> -	/*
> -	 * If the filesystem is not mounted then only the default resource group
> -	 * exists. Creation of its directories is deferred until mount time
> -	 * by rdt_get_tree() calling mkdir_mondata_all().
> -	 * If resctrl is mounted, add per domain monitor data directories.
> -	 */
> -	if (resctrl_mounted && resctrl_arch_mon_capable())
> -		mkdir_mondata_subdir_allrdtgrp(r, d);
> -
> -out_unlock:
> -	mutex_unlock(&rdtgroup_mutex);
> -
> -	return err;
> -}
> -
> -void resctrl_online_cpu(unsigned int cpu)
> -{
> -	mutex_lock(&rdtgroup_mutex);
> -	/* The CPU is set in default rdtgroup after online. */
> -	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
> -	mutex_unlock(&rdtgroup_mutex);
> -}
> -
> -static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)
> -{
> -	struct rdtgroup *cr;
> -
> -	list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) {
> -		if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask))
> -			break;
> -	}
> -}
> -
> -void resctrl_offline_cpu(unsigned int cpu)
> -{
> -	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> -	struct rdtgroup *rdtgrp;
> -	struct rdt_domain *d;
> -
> -	mutex_lock(&rdtgroup_mutex);
> -	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
> -		if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) {
> -			clear_childcpus(rdtgrp, cpu);
> -			break;
> -		}
> -	}
> -
> -	if (!l3->mon_capable)
> -		goto out_unlock;
> -
> -	d = resctrl_get_domain_from_cpu(cpu, l3);
> -	if (d) {
> -		if (resctrl_is_mbm_enabled() && cpu == d->mbm_work_cpu) {
> -			cancel_delayed_work(&d->mbm_over);
> -			mbm_setup_overflow_handler(d, 0, cpu);
> -		}
> -		if (resctrl_arch_is_llc_occupancy_enabled() &&
> -		    cpu == d->cqm_work_cpu && has_busy_rmid(d)) {
> -			cancel_delayed_work(&d->cqm_limbo);
> -			cqm_setup_limbo_handler(d, 0, cpu);
> -		}
> -	}
> -
> -out_unlock:
> -	mutex_unlock(&rdtgroup_mutex);
> -}
> -
> -/*
> - * resctrl_init - resctrl filesystem initialization
> - *
> - * Setup resctrl file system including set up root, create mount point,
> - * register resctrl filesystem, and initialize files under root directory.
> - *
> - * Return: 0 on success or -errno
> - */
> -int resctrl_init(void)
> -{
> -	int ret = 0;
> -
> -	seq_buf_init(&last_cmd_status, last_cmd_status_buf,
> -		     sizeof(last_cmd_status_buf));
> -
> -	rdtgroup_setup_default();
> -
> -	thread_throttle_mode_init();
> -
> -	ret = resctrl_mon_resource_init();
> -	if (ret)
> -		return ret;
> -
> -	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
> -	if (ret)
> -		return ret;
> -
> -	ret = register_filesystem(&rdt_fs_type);
> -	if (ret)
> -		goto cleanup_mountpoint;
> -
> -	/*
> -	 * Adding the resctrl debugfs directory here may not be ideal since
> -	 * it would let the resctrl debugfs directory appear on the debugfs
> -	 * filesystem before the resctrl filesystem is mounted.
> -	 * It may also be ok since that would enable debugging of RDT before
> -	 * resctrl is mounted.
> -	 * The reason why the debugfs directory is created here and not in
> -	 * rdt_get_tree() is because rdt_get_tree() takes rdtgroup_mutex and
> -	 * during the debugfs directory creation also &sb->s_type->i_mutex_key
> -	 * (the lockdep class of inode->i_rwsem). Other filesystem
> -	 * interactions (eg. SyS_getdents) have the lock ordering:
> -	 * &sb->s_type->i_mutex_key --> &mm->mmap_lock
> -	 * During mmap(), called with &mm->mmap_lock, the rdtgroup_mutex
> -	 * is taken, thus creating dependency:
> -	 * &mm->mmap_lock --> rdtgroup_mutex for the latter that can cause
> -	 * issues considering the other two lock dependencies.
> -	 * By creating the debugfs directory here we avoid a dependency
> -	 * that may cause deadlock (even though file operations cannot
> -	 * occur until the filesystem is mounted, but I do not know how to
> -	 * tell lockdep that).
> -	 */
> -	debugfs_resctrl = debugfs_create_dir("resctrl", NULL);
> -
> -	return 0;
> -
> -cleanup_mountpoint:
> -	sysfs_remove_mount_point(fs_kobj, "resctrl");
> -
> -	return ret;
> -}
> -
> -void resctrl_exit(void)
> -{
> -	debugfs_remove_recursive(debugfs_resctrl);
> -	unregister_filesystem(&rdt_fs_type);
> -	sysfs_remove_mount_point(fs_kobj, "resctrl");
> -
> -	resctrl_mon_resource_exit();
> -}
> diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
> index e69de29bb2d1..a8f2dd66ede3 100644
> --- a/fs/resctrl/ctrlmondata.c
> +++ b/fs/resctrl/ctrlmondata.c
> @@ -0,0 +1,527 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Resource Director Technology(RDT)
> + * - Cache Allocation code.
> + *
> + * Copyright (C) 2016 Intel Corporation
> + *
> + * Authors:
> + *    Fenghua Yu <fenghua.yu@intel.com>
> + *    Tony Luck <tony.luck@intel.com>
> + *
> + * More information about RDT be found in the Intel (R) x86 Architecture
> + * Software Developer Manual June 2016, volume 3, section 17.17.
> + */
> +
> +#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
> +
> +#include <linux/cpu.h>
> +#include <linux/kernfs.h>
> +#include <linux/seq_file.h>
> +#include <linux/slab.h>
> +#include "internal.h"
> +
> +struct rdt_parse_data {
> +	struct rdtgroup		*rdtgrp;
> +	char			*buf;
> +};
> +
> +typedef int (ctrlval_parser_t)(struct rdt_parse_data *data,
> +			       struct resctrl_schema *s,
> +			       struct rdt_domain *d);
> +
> +/*
> + * Check whether MBA bandwidth percentage value is correct. The value is
> + * checked against the minimum and max bandwidth values specified by the
> + * hardware. The allocated bandwidth percentage is rounded to the next
> + * control step available on the hardware.
> + */
> +static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
> +{
> +	unsigned long bw;
> +	int ret;
> +
> +	/*
> +	 * Only linear delay values is supported for current Intel SKUs.
> +	 */
> +	if (!r->membw.delay_linear && r->membw.arch_needs_linear) {
> +		rdt_last_cmd_puts("No support for non-linear MB domains\n");
> +		return false;
> +	}
> +
> +	ret = kstrtoul(buf, 10, &bw);
> +	if (ret) {
> +		rdt_last_cmd_printf("Non-decimal digit in MB value %s\n", buf);
> +		return false;
> +	}
> +
> +	if ((bw < r->membw.min_bw || bw > r->default_ctrl) &&
> +	    !is_mba_sc(r)) {
> +		rdt_last_cmd_printf("MB value %ld out of range [%d,%d]\n", bw,
> +				    r->membw.min_bw, r->default_ctrl);
> +		return false;
> +	}
> +
> +	*data = roundup(bw, (unsigned long)r->membw.bw_gran);
> +	return true;
> +}
> +
> +static int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s,
> +		    struct rdt_domain *d)
> +{
> +	struct resctrl_staged_config *cfg;
> +	u32 closid = data->rdtgrp->closid;
> +	struct rdt_resource *r = s->res;
> +	unsigned long bw_val;
> +
> +	cfg = &d->staged_config[s->conf_type];
> +	if (cfg->have_new_ctrl) {
> +		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
> +		return -EINVAL;
> +	}
> +
> +	if (!bw_validate(data->buf, &bw_val, r))
> +		return -EINVAL;
> +
> +	if (is_mba_sc(r)) {
> +		d->mbps_val[closid] = bw_val;
> +		return 0;
> +	}
> +
> +	cfg->new_ctrl = bw_val;
> +	cfg->have_new_ctrl = true;
> +
> +	return 0;
> +}
> +
> +/*
> + * Check whether a cache bit mask is valid.
> + * On Intel CPUs, non-contiguous 1s value support is indicated by CPUID:
> + *   - CPUID.0x10.1:ECX[3]: L3 non-contiguous 1s value supported if 1
> + *   - CPUID.0x10.2:ECX[3]: L2 non-contiguous 1s value supported if 1
> + *
> + * Haswell does not support a non-contiguous 1s value and additionally
> + * requires at least two bits set.
> + * AMD allows non-contiguous bitmasks.
> + */
> +static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
> +{
> +	unsigned long first_bit, zero_bit, val;
> +	unsigned int cbm_len = r->cache.cbm_len;
> +	int ret;
> +
> +	ret = kstrtoul(buf, 16, &val);
> +	if (ret) {
> +		rdt_last_cmd_printf("Non-hex character in the mask %s\n", buf);
> +		return false;
> +	}
> +
> +	if ((r->cache.min_cbm_bits > 0 && val == 0) || val > r->default_ctrl) {
> +		rdt_last_cmd_puts("Mask out of range\n");
> +		return false;
> +	}
> +
> +	first_bit = find_first_bit(&val, cbm_len);
> +	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
> +
> +	/* Are non-contiguous bitmasks allowed? */
> +	if (!r->cache.arch_has_sparse_bitmasks &&
> +	    (find_next_bit(&val, cbm_len, zero_bit) < cbm_len)) {
> +		rdt_last_cmd_printf("The mask %lx has non-consecutive 1-bits\n", val);
> +		return false;
> +	}
> +
> +	if ((zero_bit - first_bit) < r->cache.min_cbm_bits) {
> +		rdt_last_cmd_printf("Need at least %d bits in the mask\n",
> +				    r->cache.min_cbm_bits);
> +		return false;
> +	}
> +
> +	*data = val;
> +	return true;
> +}
> +
> +/*
> + * Read one cache bit mask (hex). Check that it is valid for the current
> + * resource type.
> + */
> +static int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s,
> +		     struct rdt_domain *d)
> +{
> +	struct rdtgroup *rdtgrp = data->rdtgrp;
> +	struct resctrl_staged_config *cfg;
> +	struct rdt_resource *r = s->res;
> +	u32 cbm_val;
> +
> +	cfg = &d->staged_config[s->conf_type];
> +	if (cfg->have_new_ctrl) {
> +		rdt_last_cmd_printf("Duplicate domain %d\n", d->id);
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * Cannot set up more than one pseudo-locked region in a cache
> +	 * hierarchy.
> +	 */
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
> +	    rdtgroup_pseudo_locked_in_hierarchy(d)) {
> +		rdt_last_cmd_puts("Pseudo-locked region in hierarchy\n");
> +		return -EINVAL;
> +	}
> +
> +	if (!cbm_validate(data->buf, &cbm_val, r))
> +		return -EINVAL;
> +
> +	if ((rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
> +	     rdtgrp->mode == RDT_MODE_SHAREABLE) &&
> +	    rdtgroup_cbm_overlaps_pseudo_locked(d, cbm_val)) {
> +		rdt_last_cmd_puts("CBM overlaps with pseudo-locked region\n");
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * The CBM may not overlap with the CBM of another closid if
> +	 * either is exclusive.
> +	 */
> +	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, true)) {
> +		rdt_last_cmd_puts("Overlaps with exclusive group\n");
> +		return -EINVAL;
> +	}
> +
> +	if (rdtgroup_cbm_overlaps(s, d, cbm_val, rdtgrp->closid, false)) {
> +		if (rdtgrp->mode == RDT_MODE_EXCLUSIVE ||
> +		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +			rdt_last_cmd_puts("Overlaps with other group\n");
> +			return -EINVAL;
> +		}
> +	}
> +
> +	cfg->new_ctrl = cbm_val;
> +	cfg->have_new_ctrl = true;
> +
> +	return 0;
> +}
> +
> +static ctrlval_parser_t *get_parser(struct rdt_resource *res)
> +{
> +	if (res->fflags & RFTYPE_RES_CACHE)
> +		return &parse_cbm;
> +	else
> +		return &parse_bw;
> +}
> +
> +/*
> + * For each domain in this resource we expect to find a series of:
> + *	id=mask
> + * separated by ";". The "id" is in decimal, and must match one of
> + * the "id"s for this resource.
> + */
> +static int parse_line(char *line, struct resctrl_schema *s,
> +		      struct rdtgroup *rdtgrp)
> +{
> +	ctrlval_parser_t *parse_ctrlval = get_parser(s->res);
> +	enum resctrl_conf_type t = s->conf_type;
> +	struct resctrl_staged_config *cfg;
> +	struct rdt_resource *r = s->res;
> +	struct rdt_parse_data data;
> +	char *dom = NULL, *id;
> +	struct rdt_domain *d;
> +	unsigned long dom_id;
> +
> +	/* Walking r->domains, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
> +	    (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)) {
> +		rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
> +		return -EINVAL;
> +	}
> +
> +next:
> +	if (!line || line[0] == '\0')
> +		return 0;
> +	dom = strsep(&line, ";");
> +	id = strsep(&dom, "=");
> +	if (!dom || kstrtoul(id, 10, &dom_id)) {
> +		rdt_last_cmd_puts("Missing '=' or non-numeric domain\n");
> +		return -EINVAL;
> +	}
> +	dom = strim(dom);
> +	list_for_each_entry(d, &r->domains, list) {
> +		if (d->id == dom_id) {
> +			data.buf = dom;
> +			data.rdtgrp = rdtgrp;
> +			if (parse_ctrlval(&data, s, d))
> +				return -EINVAL;
> +			if (rdtgrp->mode ==  RDT_MODE_PSEUDO_LOCKSETUP) {
> +				cfg = &d->staged_config[t];
> +				/*
> +				 * In pseudo-locking setup mode and just
> +				 * parsed a valid CBM that should be
> +				 * pseudo-locked. Only one locked region per
> +				 * resource group and domain so just do
> +				 * the required initialization for single
> +				 * region and return.
> +				 */
> +				rdtgrp->plr->s = s;
> +				rdtgrp->plr->d = d;
> +				rdtgrp->plr->cbm = cfg->new_ctrl;
> +				d->plr = rdtgrp->plr;
> +				return 0;
> +			}
> +			goto next;
> +		}
> +	}
> +	return -EINVAL;
> +}
> +
> +static int rdtgroup_parse_resource(char *resname, char *tok,
> +				   struct rdtgroup *rdtgrp)
> +{
> +	struct resctrl_schema *s;
> +
> +	list_for_each_entry(s, &resctrl_schema_all, list) {
> +		if (!strcmp(resname, s->name) && rdtgrp->closid < s->num_closid)
> +			return parse_line(tok, s, rdtgrp);
> +	}
> +	rdt_last_cmd_printf("Unknown or unsupported resource name '%s'\n", resname);
> +	return -EINVAL;
> +}
> +
> +ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
> +				char *buf, size_t nbytes, loff_t off)
> +{
> +	struct resctrl_schema *s;
> +	struct rdtgroup *rdtgrp;
> +	struct rdt_resource *r;
> +	char *tok, *resname;
> +	int ret = 0;
> +
> +	/* Valid input requires a trailing newline */
> +	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> +		return -EINVAL;
> +	buf[nbytes - 1] = '\0';
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (!rdtgrp) {
> +		rdtgroup_kn_unlock(of->kn);
> +		return -ENOENT;
> +	}
> +	rdt_last_cmd_clear();
> +
> +	/*
> +	 * No changes to pseudo-locked region allowed. It has to be removed
> +	 * and re-created instead.
> +	 */
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> +		ret = -EINVAL;
> +		rdt_last_cmd_puts("Resource group is pseudo-locked\n");
> +		goto out;
> +	}
> +
> +	rdt_staged_configs_clear();
> +
> +	while ((tok = strsep(&buf, "\n")) != NULL) {
> +		resname = strim(strsep(&tok, ":"));
> +		if (!tok) {
> +			rdt_last_cmd_puts("Missing ':'\n");
> +			ret = -EINVAL;
> +			goto out;
> +		}
> +		if (tok[0] == '\0') {
> +			rdt_last_cmd_printf("Missing '%s' value\n", resname);
> +			ret = -EINVAL;
> +			goto out;
> +		}
> +		ret = rdtgroup_parse_resource(resname, tok, rdtgrp);
> +		if (ret)
> +			goto out;
> +	}
> +
> +	list_for_each_entry(s, &resctrl_schema_all, list) {
> +		r = s->res;
> +
> +		/*
> +		 * Writes to mba_sc resources update the software controller,
> +		 * not the control MSR.
> +		 */
> +		if (is_mba_sc(r))
> +			continue;
> +
> +		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
> +		if (ret)
> +			goto out;
> +	}
> +
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +		/*
> +		 * If pseudo-locking fails we keep the resource group in
> +		 * mode RDT_MODE_PSEUDO_LOCKSETUP with its class of service
> +		 * active and updated for just the domain the pseudo-locked
> +		 * region was requested for.
> +		 */
> +		ret = rdtgroup_pseudo_lock_create(rdtgrp);
> +	}
> +
> +out:
> +	rdt_staged_configs_clear();
> +	rdtgroup_kn_unlock(of->kn);
> +	return ret ?: nbytes;
> +}
> +
> +static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int closid)
> +{
> +	struct rdt_resource *r = schema->res;
> +	struct rdt_domain *dom;
> +	bool sep = false;
> +	u32 ctrl_val;
> +
> +	/* Walking r->domains, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +	seq_printf(s, "%*s:", max_name_width, schema->name);
> +	list_for_each_entry(dom, &r->domains, list) {
> +		if (sep)
> +			seq_puts(s, ";");
> +
> +		if (is_mba_sc(r))
> +			ctrl_val = dom->mbps_val[closid];
> +		else
> +			ctrl_val = resctrl_arch_get_config(r, dom, closid,
> +							   schema->conf_type);
> +
> +		seq_printf(s, r->format_str, dom->id, max_data_width,
> +			   ctrl_val);
> +		sep = true;
> +	}
> +	seq_puts(s, "\n");
> +}
> +
> +int rdtgroup_schemata_show(struct kernfs_open_file *of,
> +			   struct seq_file *s, void *v)
> +{
> +	struct resctrl_schema *schema;
> +	struct rdtgroup *rdtgrp;
> +	int ret = 0;
> +	u32 closid;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (rdtgrp) {
> +		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +			list_for_each_entry(schema, &resctrl_schema_all, list) {
> +				seq_printf(s, "%s:uninitialized\n", schema->name);
> +			}
> +		} else if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> +			if (!rdtgrp->plr->d) {
> +				rdt_last_cmd_clear();
> +				rdt_last_cmd_puts("Cache domain offline\n");
> +				ret = -ENODEV;
> +			} else {
> +				seq_printf(s, "%s:%d=%x\n",
> +					   rdtgrp->plr->s->res->name,
> +					   rdtgrp->plr->d->id,
> +					   rdtgrp->plr->cbm);
> +			}
> +		} else {
> +			closid = rdtgrp->closid;
> +			list_for_each_entry(schema, &resctrl_schema_all, list) {
> +				if (closid < schema->num_closid)
> +					show_doms(s, schema, closid);
> +			}
> +		}
> +	} else {
> +		ret = -ENOENT;
> +	}
> +	rdtgroup_kn_unlock(of->kn);
> +	return ret;
> +}
> +
> +static int smp_mon_event_count(void *arg)
> +{
> +	mon_event_count(arg);
> +
> +	return 0;
> +}
> +
> +void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
> +		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
> +		    int evtid, int first)
> +{
> +	int cpu;
> +
> +	/* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +	/*
> +	 * Setup the parameters to pass to mon_event_count() to read the data.
> +	 */
> +	rr->rgrp = rdtgrp;
> +	rr->evtid = evtid;
> +	rr->r = r;
> +	rr->d = d;
> +	rr->val = 0;
> +	rr->first = first;
> +	rr->arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, evtid);
> +	if (IS_ERR(rr->arch_mon_ctx)) {
> +		rr->err = -EINVAL;
> +		return;
> +	}
> +
> +	cpu = cpumask_any_housekeeping(&d->cpu_mask, RESCTRL_PICK_ANY_CPU);
> +
> +	/*
> +	 * cpumask_any_housekeeping() prefers housekeeping CPUs, but
> +	 * are all the CPUs nohz_full? If yes, pick a CPU to IPI.
> +	 * MPAM's resctrl_arch_rmid_read() is unable to read the
> +	 * counters on some platforms if its called in IRQ context.
> +	 */
> +	if (tick_nohz_full_cpu(cpu))
> +		smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1);
> +	else
> +		smp_call_on_cpu(cpu, smp_mon_event_count, rr, false);
> +
> +	resctrl_arch_mon_ctx_free(r, evtid, rr->arch_mon_ctx);
> +}
> +
> +int rdtgroup_mondata_show(struct seq_file *m, void *arg)
> +{
> +	struct kernfs_open_file *of = m->private;
> +	u32 resid, evtid, domid;
> +	struct rdtgroup *rdtgrp;
> +	struct rdt_resource *r;
> +	union mon_data_bits md;
> +	struct rdt_domain *d;
> +	struct rmid_read rr;
> +	int ret = 0;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (!rdtgrp) {
> +		ret = -ENOENT;
> +		goto out;
> +	}
> +
> +	md.priv = of->kn->priv;
> +	resid = md.u.rid;
> +	domid = md.u.domid;
> +	evtid = md.u.evtid;
> +
> +	r = resctrl_arch_get_resource(resid);
> +	d = resctrl_arch_find_domain(r, domid);
> +	if (IS_ERR_OR_NULL(d)) {
> +		ret = -ENOENT;
> +		goto out;
> +	}
> +
> +	mon_event_read(&rr, r, d, rdtgrp, evtid, false);
> +
> +	if (rr.err == -EIO)
> +		seq_puts(m, "Error\n");
> +	else if (rr.err == -EINVAL)
> +		seq_puts(m, "Unavailable\n");
> +	else
> +		seq_printf(m, "%llu\n", rr.val);
> +
> +out:
> +	rdtgroup_kn_unlock(of->kn);
> +	return ret;
> +}
> diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
> index e69de29bb2d1..f73267762a87 100644
> --- a/fs/resctrl/internal.h
> +++ b/fs/resctrl/internal.h
> @@ -0,0 +1,340 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _FS_RESCTRL_INTERNAL_H
> +#define _FS_RESCTRL_INTERNAL_H
> +
> +#include <linux/resctrl.h>
> +#include <linux/sched.h>
> +#include <linux/kernfs.h>
> +#include <linux/fs_context.h>
> +#include <linux/jump_label.h>
> +#include <linux/tick.h>
> +
> +#include <asm/resctrl.h>
> +
> +/**
> + * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that
> + *			        aren't marked nohz_full
> + * @mask:	The mask to pick a CPU from.
> + * @exclude_cpu:The CPU to avoid picking.
> + *
> + * Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping
> + * CPUs that don't use nohz_full, these are preferred. Pass
> + * RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs.
> + *
> + * When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available.
> + */
> +static inline unsigned int
> +cpumask_any_housekeeping(const struct cpumask *mask, int exclude_cpu)
> +{
> +	unsigned int cpu, hk_cpu;
> +
> +	if (exclude_cpu == RESCTRL_PICK_ANY_CPU)
> +		cpu = cpumask_any(mask);
> +	else
> +		cpu = cpumask_any_but(mask, exclude_cpu);
> +
> +	if (!IS_ENABLED(CONFIG_NO_HZ_FULL))
> +		return cpu;
> +
> +	/* If the CPU picked isn't marked nohz_full nothing more needs doing. */
> +	if (cpu < nr_cpu_ids && !tick_nohz_full_cpu(cpu))
> +		return cpu;
> +
> +	/* Try to find a CPU that isn't nohz_full to use in preference */
> +	hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask);
> +	if (hk_cpu == exclude_cpu)
> +		hk_cpu = cpumask_nth_andnot(1, mask, tick_nohz_full_mask);
> +
> +	if (hk_cpu < nr_cpu_ids)
> +		cpu = hk_cpu;
> +
> +	return cpu;
> +}
> +
> +struct rdt_fs_context {
> +	struct kernfs_fs_context	kfc;
> +	bool				enable_cdpl2;
> +	bool				enable_cdpl3;
> +	bool				enable_mba_mbps;
> +	bool				enable_debug;
> +};
> +
> +static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
> +{
> +	struct kernfs_fs_context *kfc = fc->fs_private;
> +
> +	return container_of(kfc, struct rdt_fs_context, kfc);
> +}
> +
> +/**
> + * struct mon_evt - Entry in the event list of a resource
> + * @evtid:		event id
> + * @name:		name of the event
> + * @configurable:	true if the event is configurable
> + * @list:		entry in &rdt_resource->evt_list
> + */
> +struct mon_evt {
> +	enum resctrl_event_id	evtid;
> +	char			*name;
> +	bool			configurable;
> +	struct list_head	list;
> +};
> +
> +/**
> + * union mon_data_bits - Monitoring details for each event file
> + * @priv:              Used to store monitoring event data in @u
> + *                     as kernfs private data
> + * @rid:               Resource id associated with the event file
> + * @evtid:             Event id associated with the event file
> + * @domid:             The domain to which the event file belongs
> + * @u:                 Name of the bit fields struct
> + */
> +union mon_data_bits {
> +	void *priv;
> +	struct {
> +		unsigned int rid		: 10;
> +		enum resctrl_event_id evtid	: 8;
> +		unsigned int domid		: 14;
> +	} u;
> +};
> +
> +struct rmid_read {
> +	struct rdtgroup		*rgrp;
> +	struct rdt_resource	*r;
> +	struct rdt_domain	*d;
> +	enum resctrl_event_id	evtid;
> +	bool			first;
> +	int			err;
> +	u64			val;
> +	void			*arch_mon_ctx;
> +};
> +
> +extern struct list_head resctrl_schema_all;
> +extern bool resctrl_mounted;
> +
> +enum rdt_group_type {
> +	RDTCTRL_GROUP = 0,
> +	RDTMON_GROUP,
> +	RDT_NUM_GROUP,
> +};
> +
> +/**
> + * enum rdtgrp_mode - Mode of a RDT resource group
> + * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations
> + * @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed
> + * @RDT_MODE_PSEUDO_LOCKSETUP: Resource group will be used for Pseudo-Locking
> + * @RDT_MODE_PSEUDO_LOCKED: No sharing of this resource group's allocations
> + *                          allowed AND the allocations are Cache Pseudo-Locked
> + * @RDT_NUM_MODES: Total number of modes
> + *
> + * The mode of a resource group enables control over the allowed overlap
> + * between allocations associated with different resource groups (classes
> + * of service). User is able to modify the mode of a resource group by
> + * writing to the "mode" resctrl file associated with the resource group.
> + *
> + * The "shareable", "exclusive", and "pseudo-locksetup" modes are set by
> + * writing the appropriate text to the "mode" file. A resource group enters
> + * "pseudo-locked" mode after the schemata is written while the resource
> + * group is in "pseudo-locksetup" mode.
> + */
> +enum rdtgrp_mode {
> +	RDT_MODE_SHAREABLE = 0,
> +	RDT_MODE_EXCLUSIVE,
> +	RDT_MODE_PSEUDO_LOCKSETUP,
> +	RDT_MODE_PSEUDO_LOCKED,
> +
> +	/* Must be last */
> +	RDT_NUM_MODES,
> +};
> +
> +/**
> + * struct mongroup - store mon group's data in resctrl fs.
> + * @mon_data_kn:		kernfs node for the mon_data directory
> + * @parent:			parent rdtgrp
> + * @crdtgrp_list:		child rdtgroup node list
> + * @rmid:			rmid for this rdtgroup
> + */
> +struct mongroup {
> +	struct kernfs_node	*mon_data_kn;
> +	struct rdtgroup		*parent;
> +	struct list_head	crdtgrp_list;
> +	u32			rmid;
> +};
> +
> +/**
> + * struct rdtgroup - store rdtgroup's data in resctrl file system.
> + * @kn:				kernfs node
> + * @rdtgroup_list:		linked list for all rdtgroups
> + * @closid:			closid for this rdtgroup
> + * @cpu_mask:			CPUs assigned to this rdtgroup
> + * @flags:			status bits
> + * @waitcount:			how many cpus expect to find this
> + *				group when they acquire rdtgroup_mutex
> + * @type:			indicates type of this rdtgroup - either
> + *				monitor only or ctrl_mon group
> + * @mon:			mongroup related data
> + * @mode:			mode of resource group
> + * @plr:			pseudo-locked region
> + */
> +struct rdtgroup {
> +	struct kernfs_node		*kn;
> +	struct list_head		rdtgroup_list;
> +	u32				closid;
> +	struct cpumask			cpu_mask;
> +	int				flags;
> +	atomic_t			waitcount;
> +	enum rdt_group_type		type;
> +	struct mongroup			mon;
> +	enum rdtgrp_mode		mode;
> +	struct pseudo_lock_region	*plr;
> +};
> +
> +/* List of all resource groups */
> +extern struct list_head rdt_all_groups;
> +
> +extern int max_name_width, max_data_width;
> +
> +/**
> + * struct rftype - describe each file in the resctrl file system
> + * @name:	File name
> + * @mode:	Access mode
> + * @kf_ops:	File operations
> + * @flags:	File specific RFTYPE_FLAGS_* flags
> + * @fflags:	File specific RFTYPE_* flags
> + * @seq_show:	Show content of the file
> + * @write:	Write to the file
> + */
> +struct rftype {
> +	char			*name;
> +	umode_t			mode;
> +	const struct kernfs_ops	*kf_ops;
> +	unsigned long		flags;
> +	unsigned long		fflags;
> +
> +	int (*seq_show)(struct kernfs_open_file *of,
> +			struct seq_file *sf, void *v);
> +	/*
> +	 * write() is the generic write callback which maps directly to
> +	 * kernfs write operation and overrides all other operations.
> +	 * Maximum write size is determined by ->max_write_len.
> +	 */
> +	ssize_t (*write)(struct kernfs_open_file *of,
> +			 char *buf, size_t nbytes, loff_t off);
> +};
> +
> +/**
> + * struct mbm_state - status for each MBM counter in each domain
> + * @prev_bw_bytes: Previous bytes value read for bandwidth calculation
> + * @prev_bw:	The most recent bandwidth in MBps
> + */
> +struct mbm_state {
> +	u64	prev_bw_bytes;
> +	u32	prev_bw;
> +};
> +
> +static inline  bool is_mba_sc(struct rdt_resource *r)
> +{
> +	if (!r)
> +		r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> +
> +	/*
> +	 * The software controller support is only applicable to MBA resource.
> +	 * Make sure to check for resource type.
> +	 */
> +	if (r->rid != RDT_RESOURCE_MBA)
> +		return false;
> +
> +	return r->membw.mba_sc;
> +}
> +
> +extern struct mutex rdtgroup_mutex;
> +extern struct rdtgroup rdtgroup_default;
> +extern struct dentry *debugfs_resctrl;
> +
> +void rdt_last_cmd_clear(void);
> +void rdt_last_cmd_puts(const char *s);
> +__printf(1, 2)
> +void rdt_last_cmd_printf(const char *fmt, ...);
> +
> +struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
> +void rdtgroup_kn_unlock(struct kernfs_node *kn);
> +int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name);
> +int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
> +			     umode_t mask);
> +ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
> +				char *buf, size_t nbytes, loff_t off);
> +int rdtgroup_schemata_show(struct kernfs_open_file *of,
> +			   struct seq_file *s, void *v);
> +bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
> +			   unsigned long cbm, int closid, bool exclusive);
> +unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d,
> +				  unsigned long cbm);
> +enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);
> +int rdtgroup_tasks_assigned(struct rdtgroup *r);
> +int closids_supported(void);
> +void closid_free(int closid);
> +int alloc_rmid(u32 closid);
> +void free_rmid(u32 closid, u32 rmid);
> +void resctrl_mon_resource_exit(void);
> +void mon_event_count(void *info);
> +int rdtgroup_mondata_show(struct seq_file *m, void *arg);
> +void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
> +		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
> +		    int evtid, int first);
> +int resctrl_mon_resource_init(void);
> +void mbm_setup_overflow_handler(struct rdt_domain *dom,
> +				unsigned long delay_ms,
> +				int exclude_cpu);
> +void mbm_handle_overflow(struct work_struct *work);
> +bool is_mba_sc(struct rdt_resource *r);
> +void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
> +			     int exclude_cpu);
> +void cqm_handle_limbo(struct work_struct *work);
> +bool has_busy_rmid(struct rdt_domain *d);
> +void __check_limbo(struct rdt_domain *d, bool force_free);
> +void mbm_config_rftype_init(const char *config);
> +void rdt_staged_configs_clear(void);
> +bool closid_allocated(unsigned int closid);
> +int resctrl_find_cleanest_closid(void);
> +
> +#ifdef CONFIG_RESCTRL_FS_PSEUDO_LOCK
> +int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
> +int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp);
> +bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm);
> +bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d);
> +int rdt_pseudo_lock_init(void);
> +void rdt_pseudo_lock_release(void);
> +int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
> +void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
> +#else
> +static inline int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +static inline int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +static inline bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
> +{
> +	return false;
> +}
> +
> +static inline bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
> +{
> +	return false;
> +}
> +
> +static inline int rdt_pseudo_lock_init(void) { return 0; }
> +static inline void rdt_pseudo_lock_release(void) { }
> +static inline int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +static inline void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp) { }
> +#endif /* CONFIG_RESCTRL_FS_PSEUDO_LOCK */
> +
> +#endif /* _FS_RESCTRL_INTERNAL_H */
> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> index e69de29bb2d1..06f660dfd929 100644
> --- a/fs/resctrl/monitor.c
> +++ b/fs/resctrl/monitor.c
> @@ -0,0 +1,843 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Resource Director Technology(RDT)
> + * - Monitoring code
> + *
> + * Copyright (C) 2017 Intel Corporation
> + *
> + * Author:
> + *    Vikas Shivappa <vikas.shivappa@intel.com>
> + *
> + * This replaces the cqm.c based on perf but we reuse a lot of
> + * code and datastructures originally from Peter Zijlstra and Matt Fleming.
> + *
> + * More information about RDT be found in the Intel (R) x86 Architecture
> + * Software Developer Manual June 2016, volume 3, section 17.17.
> + */
> +
> +#include <linux/cpu.h>
> +#include <linux/module.h>
> +#include <linux/sizes.h>
> +#include <linux/slab.h>
> +#include "internal.h"
> +
> +/*
> + * struct rmid_entry - dirty tracking for all RMID.
> + * @closid:	The CLOSID for this entry.
> + * @rmid:	The RMID for this entry.
> + * @busy:	The number of domains with cached data using this RMID.
> + * @list:	Member of the rmid_free_lru list when busy == 0.
> + *
> + * Depending on the architecture the correct monitor is accessed using
> + * both @closid and @rmid, or @rmid only.
> + *
> + * Take the rdtgroup_mutex when accessing.
> + */
> +struct rmid_entry {
> +	u32				closid;
> +	u32				rmid;
> +	int				busy;
> +	struct list_head		list;
> +};
> +
> +/*
> + * @rmid_free_lru - A least recently used list of free RMIDs
> + *     These RMIDs are guaranteed to have an occupancy less than the
> + *     threshold occupancy
> + */
> +static LIST_HEAD(rmid_free_lru);
> +
> +/*
> + * @closid_num_dirty_rmid    The number of dirty RMID each CLOSID has.
> + *     Only allocated when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined.
> + *     Indexed by CLOSID. Protected by rdtgroup_mutex.
> + */
> +static u32 *closid_num_dirty_rmid;
> +
> +/*
> + * @rmid_limbo_count - count of currently unused but (potentially)
> + *     dirty RMIDs.
> + *     This counts RMIDs that no one is currently using but that
> + *     may have a occupancy value > resctrl_rmid_realloc_threshold. User can
> + *     change the threshold occupancy value.
> + */
> +static unsigned int rmid_limbo_count;
> +
> +/*
> + * @rmid_entry - The entry in the limbo and free lists.
> + */
> +static struct rmid_entry	*rmid_ptrs;
> +
> +/*
> + * This is the threshold cache occupancy in bytes at which we will consider an
> + * RMID available for re-allocation.
> + */
> +unsigned int resctrl_rmid_realloc_threshold;
> +
> +/*
> + * This is the maximum value for the reallocation threshold, in bytes.
> + */
> +unsigned int resctrl_rmid_realloc_limit;
> +
> +/*
> + * x86 and arm64 differ in their handling of monitoring.
> + * x86's RMID are independent numbers, there is only one source of traffic
> + * with an RMID value of '1'.
> + * arm64's PMG extends the PARTID/CLOSID space, there are multiple sources of
> + * traffic with a PMG value of '1', one for each CLOSID, meaning the RMID
> + * value is no longer unique.
> + * To account for this, resctrl uses an index. On x86 this is just the RMID,
> + * on arm64 it encodes the CLOSID and RMID. This gives a unique number.
> + *
> + * The domain's rmid_busy_llc and rmid_ptrs[] are sized by index. The arch code
> + * must accept an attempt to read every index.
> + */
> +static inline struct rmid_entry *__rmid_entry(u32 idx)
> +{
> +	struct rmid_entry *entry;
> +	u32 closid, rmid;
> +
> +	entry = &rmid_ptrs[idx];
> +	resctrl_arch_rmid_idx_decode(idx, &closid, &rmid);
> +
> +	WARN_ON_ONCE(entry->closid != closid);
> +	WARN_ON_ONCE(entry->rmid != rmid);
> +
> +	return entry;
> +}
> +
> +static void limbo_release_entry(struct rmid_entry *entry)
> +{
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	rmid_limbo_count--;
> +	list_add_tail(&entry->list, &rmid_free_lru);
> +
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
> +		closid_num_dirty_rmid[entry->closid]--;
> +}
> +
> +/*
> + * Check the RMIDs that are marked as busy for this domain. If the
> + * reported LLC occupancy is below the threshold clear the busy bit and
> + * decrement the count. If the busy count gets to zero on an RMID, we
> + * free the RMID
> + */
> +void __check_limbo(struct rdt_domain *d, bool force_free)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> +	struct rmid_entry *entry;
> +	u32 idx, cur_idx = 1;
> +	void *arch_mon_ctx;
> +	bool rmid_dirty;
> +	u64 val = 0;
> +
> +	arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID);
> +	if (IS_ERR(arch_mon_ctx)) {
> +		pr_warn_ratelimited("Failed to allocate monitor context: %ld",
> +				    PTR_ERR(arch_mon_ctx));
> +		return;
> +	}
> +
> +	/*
> +	 * Skip RMID 0 and start from RMID 1 and check all the RMIDs that
> +	 * are marked as busy for occupancy < threshold. If the occupancy
> +	 * is less than the threshold decrement the busy counter of the
> +	 * RMID and move it to the free list when the counter reaches 0.
> +	 */
> +	for (;;) {
> +		idx = find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx);
> +		if (idx >= idx_limit)
> +			break;
> +
> +		entry = __rmid_entry(idx);
> +		if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid,
> +					   QOS_L3_OCCUP_EVENT_ID, &val,
> +					   arch_mon_ctx)) {
> +			rmid_dirty = true;
> +		} else {
> +			rmid_dirty = (val >= resctrl_rmid_realloc_threshold);
> +		}
> +
> +		if (force_free || !rmid_dirty) {
> +			clear_bit(idx, d->rmid_busy_llc);
> +			if (!--entry->busy)
> +				limbo_release_entry(entry);
> +		}
> +		cur_idx = idx + 1;
> +	}
> +
> +	resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx);
> +}
> +
> +bool has_busy_rmid(struct rdt_domain *d)
> +{
> +	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> +
> +	return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit;
> +}
> +
> +static struct rmid_entry *resctrl_find_free_rmid(u32 closid)
> +{
> +	struct rmid_entry *itr;
> +	u32 itr_idx, cmp_idx;
> +
> +	if (list_empty(&rmid_free_lru))
> +		return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC);
> +
> +	list_for_each_entry(itr, &rmid_free_lru, list) {
> +		/*
> +		 * Get the index of this free RMID, and the index it would need
> +		 * to be if it were used with this CLOSID.
> +		 * If the CLOSID is irrelevant on this architecture, the two
> +		 * index values are always the same on every entry and thus the
> +		 * very first entry will be returned.
> +		 */
> +		itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid);
> +		cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid);
> +
> +		if (itr_idx == cmp_idx)
> +			return itr;
> +	}
> +
> +	return ERR_PTR(-ENOSPC);
> +}
> +
> +/**
> + * resctrl_find_cleanest_closid() - Find a CLOSID where all the associated
> + *                                  RMID are clean, or the CLOSID that has
> + *                                  the most clean RMID.
> + *
> + * MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocated CLOSID
> + * may not be able to allocate clean RMID. To avoid this the allocator will
> + * choose the CLOSID with the most clean RMID.
> + *
> + * When the CLOSID and RMID are independent numbers, the first free CLOSID will
> + * be returned.
> + */
> +int resctrl_find_cleanest_closid(void)
> +{
> +	u32 cleanest_closid = ~0;
> +	int i = 0;
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
> +		return -EIO;
> +
> +	for (i = 0; i < closids_supported(); i++) {
> +		int num_dirty;
> +
> +		if (closid_allocated(i))
> +			continue;
> +
> +		num_dirty = closid_num_dirty_rmid[i];
> +		if (num_dirty == 0)
> +			return i;
> +
> +		if (cleanest_closid == ~0)
> +			cleanest_closid = i;
> +
> +		if (num_dirty < closid_num_dirty_rmid[cleanest_closid])
> +			cleanest_closid = i;
> +	}
> +
> +	if (cleanest_closid == ~0)
> +		return -ENOSPC;
> +
> +	return cleanest_closid;
> +}
> +
> +/*
> + * For MPAM the RMID value is not unique, and has to be considered with
> + * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which
> + * allows all domains to be managed by a single free list.
> + * Each domain also has a rmid_busy_llc to reduce the work of the limbo handler.
> + */
> +int alloc_rmid(u32 closid)
> +{
> +	struct rmid_entry *entry;
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	entry = resctrl_find_free_rmid(closid);
> +	if (IS_ERR(entry))
> +		return PTR_ERR(entry);
> +
> +	list_del(&entry->list);
> +	return entry->rmid;
> +}
> +
> +static void add_rmid_to_limbo(struct rmid_entry *entry)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	struct rdt_domain *d;
> +	u32 idx;
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	/* Walking r->domains, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +	idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid);
> +
> +	entry->busy = 0;
> +	list_for_each_entry(d, &r->domains, list) {
> +		/*
> +		 * For the first limbo RMID in the domain,
> +		 * setup up the limbo worker.
> +		 */
> +		if (!has_busy_rmid(d))
> +			cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL,
> +						RESCTRL_PICK_ANY_CPU);
> +		set_bit(idx, d->rmid_busy_llc);
> +		entry->busy++;
> +	}
> +
> +	rmid_limbo_count++;
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
> +		closid_num_dirty_rmid[entry->closid]++;
> +}
> +
> +void free_rmid(u32 closid, u32 rmid)
> +{
> +	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> +	struct rmid_entry *entry;
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	/*
> +	 * Do not allow the default rmid to be free'd. Comparing by index
> +	 * allows architectures that ignore the closid parameter to avoid an
> +	 * unnecessary check.
> +	 */
> +	if (idx == resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
> +						RESCTRL_RESERVED_RMID))
> +		return;
> +
> +	entry = __rmid_entry(idx);
> +
> +	if (resctrl_arch_is_llc_occupancy_enabled())
> +		add_rmid_to_limbo(entry);
> +	else
> +		list_add_tail(&entry->list, &rmid_free_lru);
> +}
> +
> +static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid,
> +				       u32 rmid, enum resctrl_event_id evtid)
> +{
> +	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> +
> +	switch (evtid) {
> +	case QOS_L3_MBM_TOTAL_EVENT_ID:
> +		return &d->mbm_total[idx];
> +	case QOS_L3_MBM_LOCAL_EVENT_ID:
> +		return &d->mbm_local[idx];
> +	default:
> +		return NULL;
> +	}
> +}
> +
> +static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr)
> +{
> +	struct mbm_state *m;
> +	u64 tval = 0;
> +
> +	if (rr->first) {
> +		resctrl_arch_reset_rmid(rr->r, rr->d, closid, rmid, rr->evtid);
> +		m = get_mbm_state(rr->d, closid, rmid, rr->evtid);
> +		if (m)
> +			memset(m, 0, sizeof(struct mbm_state));
> +		return 0;
> +	}
> +
> +	rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid,
> +					 &tval, rr->arch_mon_ctx);
> +	if (rr->err)
> +		return rr->err;
> +
> +	rr->val += tval;
> +
> +	return 0;
> +}
> +
> +/*
> + * mbm_bw_count() - Update bw count from values previously read by
> + *		    __mon_event_count().
> + * @closid:	The closid used to identify the cached mbm_state.
> + * @rmid:	The rmid used to identify the cached mbm_state.
> + * @rr:		The struct rmid_read populated by __mon_event_count().
> + *
> + * Supporting function to calculate the memory bandwidth
> + * and delta bandwidth in MBps. The chunks value previously read by
> + * __mon_event_count() is compared with the chunks value from the previous
> + * invocation. This must be called once per second to maintain values in MBps.
> + */
> +static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr)
> +{
> +	u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> +	struct mbm_state *m = &rr->d->mbm_local[idx];
> +	u64 cur_bw, bytes, cur_bytes;
> +
> +	cur_bytes = rr->val;
> +	bytes = cur_bytes - m->prev_bw_bytes;
> +	m->prev_bw_bytes = cur_bytes;
> +
> +	cur_bw = bytes / SZ_1M;
> +
> +	m->prev_bw = cur_bw;
> +}
> +
> +/*
> + * This is scheduled by mon_event_read() to read the CQM/MBM counters
> + * on a domain.
> + */
> +void mon_event_count(void *info)
> +{
> +	struct rdtgroup *rdtgrp, *entry;
> +	struct rmid_read *rr = info;
> +	struct list_head *head;
> +	int ret;
> +
> +	rdtgrp = rr->rgrp;
> +
> +	ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr);
> +
> +	/*
> +	 * For Ctrl groups read data from child monitor groups and
> +	 * add them together. Count events which are read successfully.
> +	 * Discard the rmid_read's reporting errors.
> +	 */
> +	head = &rdtgrp->mon.crdtgrp_list;
> +
> +	if (rdtgrp->type == RDTCTRL_GROUP) {
> +		list_for_each_entry(entry, head, mon.crdtgrp_list) {
> +			if (__mon_event_count(entry->closid, entry->mon.rmid,
> +					      rr) == 0)
> +				ret = 0;
> +		}
> +	}
> +
> +	/*
> +	 * __mon_event_count() calls for newly created monitor groups may
> +	 * report -EINVAL/Unavailable if the monitor hasn't seen any traffic.
> +	 * Discard error if any of the monitor event reads succeeded.
> +	 */
> +	if (ret == 0)
> +		rr->err = 0;
> +}
> +
> +/*
> + * Feedback loop for MBA software controller (mba_sc)
> + *
> + * mba_sc is a feedback loop where we periodically read MBM counters and
> + * adjust the bandwidth percentage values via the IA32_MBA_THRTL_MSRs so
> + * that:
> + *
> + *   current bandwidth(cur_bw) < user specified bandwidth(user_bw)
> + *
> + * This uses the MBM counters to measure the bandwidth and MBA throttle
> + * MSRs to control the bandwidth for a particular rdtgrp. It builds on the
> + * fact that resctrl rdtgroups have both monitoring and control.
> + *
> + * The frequency of the checks is 1s and we just tag along the MBM overflow
> + * timer. Having 1s interval makes the calculation of bandwidth simpler.
> + *
> + * Although MBA's goal is to restrict the bandwidth to a maximum, there may
> + * be a need to increase the bandwidth to avoid unnecessarily restricting
> + * the L2 <-> L3 traffic.
> + *
> + * Since MBA controls the L2 external bandwidth where as MBM measures the
> + * L3 external bandwidth the following sequence could lead to such a
> + * situation.
> + *
> + * Consider an rdtgroup which had high L3 <-> memory traffic in initial
> + * phases -> mba_sc kicks in and reduced bandwidth percentage values -> but
> + * after some time rdtgroup has mostly L2 <-> L3 traffic.
> + *
> + * In this case we may restrict the rdtgroup's L2 <-> L3 traffic as its
> + * throttle MSRs already have low percentage values.  To avoid
> + * unnecessarily restricting such rdtgroups, we also increase the bandwidth.
> + */
> +static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
> +{
> +	u32 closid, rmid, cur_msr_val, new_msr_val;
> +	struct mbm_state *pmbm_data, *cmbm_data;
> +	struct rdt_resource *r_mba;
> +	struct rdt_domain *dom_mba;
> +	u32 cur_bw, user_bw, idx;
> +	struct list_head *head;
> +	struct rdtgroup *entry;
> +
> +	if (!resctrl_arch_is_mbm_local_enabled())
> +		return;
> +
> +	r_mba = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> +
> +	closid = rgrp->closid;
> +	rmid = rgrp->mon.rmid;
> +	idx = resctrl_arch_rmid_idx_encode(closid, rmid);
> +	pmbm_data = &dom_mbm->mbm_local[idx];
> +
> +	dom_mba = resctrl_get_domain_from_cpu(smp_processor_id(), r_mba);
> +	if (!dom_mba) {
> +		pr_warn_once("Failure to get domain for MBA update\n");
> +		return;
> +	}
> +
> +	cur_bw = pmbm_data->prev_bw;
> +	user_bw = dom_mba->mbps_val[closid];
> +
> +	/* MBA resource doesn't support CDP */
> +	cur_msr_val = resctrl_arch_get_config(r_mba, dom_mba, closid, CDP_NONE);
> +
> +	/*
> +	 * For Ctrl groups read data from child monitor groups.
> +	 */
> +	head = &rgrp->mon.crdtgrp_list;
> +	list_for_each_entry(entry, head, mon.crdtgrp_list) {
> +		cmbm_data = &dom_mbm->mbm_local[entry->mon.rmid];
> +		cur_bw += cmbm_data->prev_bw;
> +	}
> +
> +	/*
> +	 * Scale up/down the bandwidth linearly for the ctrl group.  The
> +	 * bandwidth step is the bandwidth granularity specified by the
> +	 * hardware.
> +	 * Always increase throttling if current bandwidth is above the
> +	 * target set by user.
> +	 * But avoid thrashing up and down on every poll by checking
> +	 * whether a decrease in throttling is likely to push the group
> +	 * back over target. E.g. if currently throttling to 30% of bandwidth
> +	 * on a system with 10% granularity steps, check whether moving to
> +	 * 40% would go past the limit by multiplying current bandwidth by
> +	 * "(30 + 10) / 30".
> +	 */
> +	if (cur_msr_val > r_mba->membw.min_bw && user_bw < cur_bw) {
> +		new_msr_val = cur_msr_val - r_mba->membw.bw_gran;
> +	} else if (cur_msr_val < MAX_MBA_BW &&
> +		   (user_bw > (cur_bw * (cur_msr_val + r_mba->membw.min_bw) / cur_msr_val))) {
> +		new_msr_val = cur_msr_val + r_mba->membw.bw_gran;
> +	} else {
> +		return;
> +	}
> +
> +	resctrl_arch_update_one(r_mba, dom_mba, closid, CDP_NONE, new_msr_val);
> +}
> +
> +static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,
> +		       u32 closid, u32 rmid)
> +{
> +	struct rmid_read rr;
> +
> +	rr.first = false;
> +	rr.r = r;
> +	rr.d = d;
> +
> +	/*
> +	 * This is protected from concurrent reads from user
> +	 * as both the user and we hold the global mutex.
> +	 */
> +	if (resctrl_arch_is_mbm_total_enabled()) {
> +		rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID;
> +		rr.val = 0;
> +		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
> +		if (IS_ERR(rr.arch_mon_ctx)) {
> +			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
> +					    PTR_ERR(rr.arch_mon_ctx));
> +			return;
> +		}
> +
> +		__mon_event_count(closid, rmid, &rr);
> +
> +		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
> +	}
> +	if (resctrl_arch_is_mbm_local_enabled()) {
> +		rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID;
> +		rr.val = 0;
> +		rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid);
> +		if (IS_ERR(rr.arch_mon_ctx)) {
> +			pr_warn_ratelimited("Failed to allocate monitor context: %ld",
> +					    PTR_ERR(rr.arch_mon_ctx));
> +			return;
> +		}
> +
> +		__mon_event_count(closid, rmid, &rr);
> +
> +		/*
> +		 * Call the MBA software controller only for the
> +		 * control groups and when user has enabled
> +		 * the software controller explicitly.
> +		 */
> +		if (is_mba_sc(NULL))
> +			mbm_bw_count(closid, rmid, &rr);
> +
> +		resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx);
> +	}
> +}
> +
> +/*
> + * Handler to scan the limbo list and move the RMIDs
> + * to free list whose occupancy < threshold_occupancy.
> + */
> +void cqm_handle_limbo(struct work_struct *work)
> +{
> +	unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL);
> +	struct rdt_domain *d;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	d = container_of(work, struct rdt_domain, cqm_limbo.work);
> +
> +	__check_limbo(d, false);
> +
> +	if (has_busy_rmid(d)) {
> +		d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
> +							   RESCTRL_PICK_ANY_CPU);
> +		schedule_delayed_work_on(d->cqm_work_cpu, &d->cqm_limbo,
> +					 delay);
> +	}
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +}
> +
> +/**
> + * cqm_setup_limbo_handler() - Schedule the limbo handler to run for this
> + *                             domain.
> + * @dom:           The domain the limbo handler should run for.
> + * @delay_ms:      How far in the future the handler should run.
> + * @exclude_cpu:   Which CPU the handler should not run on,
> + *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
> + */
> +void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
> +			     int exclude_cpu)
> +{
> +	unsigned long delay = msecs_to_jiffies(delay_ms);
> +	int cpu;
> +
> +	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
> +	dom->cqm_work_cpu = cpu;
> +
> +	if (cpu < nr_cpu_ids)
> +		schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay);
> +}
> +
> +void mbm_handle_overflow(struct work_struct *work)
> +{
> +	unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL);
> +	struct rdtgroup *prgrp, *crgrp;
> +	struct list_head *head;
> +	struct rdt_resource *r;
> +	struct rdt_domain *d;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	/*
> +	 * If the filesystem has been unmounted this work no longer needs to
> +	 * run.
> +	 */
> +	if (!resctrl_mounted || !resctrl_arch_mon_capable())
> +		goto out_unlock;
> +
> +	r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	d = container_of(work, struct rdt_domain, mbm_over.work);
> +
> +	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> +		mbm_update(r, d, prgrp->closid, prgrp->mon.rmid);
> +
> +		head = &prgrp->mon.crdtgrp_list;
> +		list_for_each_entry(crgrp, head, mon.crdtgrp_list)
> +			mbm_update(r, d, crgrp->closid, crgrp->mon.rmid);
> +
> +		if (is_mba_sc(NULL))
> +			update_mba_bw(prgrp, d);
> +	}
> +
> +	/*
> +	 * Re-check for housekeeping CPUs. This allows the overflow handler to
> +	 * move off a nohz_full CPU quickly.
> +	 */
> +	d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask,
> +						   RESCTRL_PICK_ANY_CPU);
> +	schedule_delayed_work_on(d->mbm_work_cpu, &d->mbm_over, delay);
> +
> +out_unlock:
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +}
> +
> +/**
> + * mbm_setup_overflow_handler() - Schedule the overflow handler to run for this
> + *                                domain.
> + * @dom:           The domain the overflow handler should run for.
> + * @delay_ms:      How far in the future the handler should run.
> + * @exclude_cpu:   Which CPU the handler should not run on,
> + *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
> + */
> +void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms,
> +				int exclude_cpu)
> +{
> +	unsigned long delay = msecs_to_jiffies(delay_ms);
> +	int cpu;
> +
> +	/*
> +	 * When a domain comes online there is no guarantee the filesystem is
> +	 * mounted. If not, there is no need to catch counter overflow.
> +	 */
> +	if (!resctrl_mounted || !resctrl_arch_mon_capable())
> +		return;
> +	cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu);
> +	dom->mbm_work_cpu = cpu;
> +
> +	if (cpu < nr_cpu_ids)
> +		schedule_delayed_work_on(cpu, &dom->mbm_over, delay);
> +}
> +
> +static int dom_data_init(struct rdt_resource *r)
> +{
> +	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> +	u32 num_closid = resctrl_arch_get_num_closid(r);
> +	struct rmid_entry *entry = NULL;
> +	int err = 0, i;
> +	u32 idx;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> +		u32 *tmp;
> +
> +		/*
> +		 * If the architecture hasn't provided a sanitised value here,
> +		 * this may result in larger arrays than necessary. Resctrl will
> +		 * use a smaller system wide value based on the resources in
> +		 * use.
> +		 */
> +		tmp = kcalloc(num_closid, sizeof(*tmp), GFP_KERNEL);
> +		if (!tmp) {
> +			err = -ENOMEM;
> +			goto out_unlock;
> +		}
> +
> +		closid_num_dirty_rmid = tmp;
> +	}
> +
> +	rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL);
> +	if (!rmid_ptrs) {
> +		if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> +			kfree(closid_num_dirty_rmid);
> +			closid_num_dirty_rmid = NULL;
> +		}
> +		err = -ENOMEM;
> +		goto out_unlock;
> +	}
> +
> +	for (i = 0; i < idx_limit; i++) {
> +		entry = &rmid_ptrs[i];
> +		INIT_LIST_HEAD(&entry->list);
> +
> +		resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid);
> +		list_add_tail(&entry->list, &rmid_free_lru);
> +	}
> +
> +	/*
> +	 * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and
> +	 * are always allocated. These are used for the rdtgroup_default
> +	 * control group, which will be setup later in rdtgroup_init().
> +	 */
> +	idx = resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
> +					   RESCTRL_RESERVED_RMID);
> +	entry = __rmid_entry(idx);
> +	list_del(&entry->list);
> +
> +out_unlock:
> +	mutex_unlock(&rdtgroup_mutex);
> +
> +	return err;
> +}
> +
> +static void dom_data_exit(struct rdt_resource *r)
> +{
> +	if (!r->mon_capable)
> +		return;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) {
> +		kfree(closid_num_dirty_rmid);
> +		closid_num_dirty_rmid = NULL;
> +	}
> +
> +	kfree(rmid_ptrs);
> +	rmid_ptrs = NULL;
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +}
> +
> +static struct mon_evt llc_occupancy_event = {
> +	.name		= "llc_occupancy",
> +	.evtid		= QOS_L3_OCCUP_EVENT_ID,
> +};
> +
> +static struct mon_evt mbm_total_event = {
> +	.name		= "mbm_total_bytes",
> +	.evtid		= QOS_L3_MBM_TOTAL_EVENT_ID,
> +};
> +
> +static struct mon_evt mbm_local_event = {
> +	.name		= "mbm_local_bytes",
> +	.evtid		= QOS_L3_MBM_LOCAL_EVENT_ID,
> +};
> +
> +/*
> + * Initialize the event list for the resource.
> + *
> + * Note that MBM events are also part of RDT_RESOURCE_L3 resource
> + * because as per the SDM the total and local memory bandwidth
> + * are enumerated as part of L3 monitoring.
> + */
> +static void l3_mon_evt_init(struct rdt_resource *r)
> +{
> +	INIT_LIST_HEAD(&r->evt_list);
> +
> +	if (resctrl_arch_is_llc_occupancy_enabled())
> +		list_add_tail(&llc_occupancy_event.list, &r->evt_list);
> +	if (resctrl_arch_is_mbm_total_enabled())
> +		list_add_tail(&mbm_total_event.list, &r->evt_list);
> +	if (resctrl_arch_is_mbm_local_enabled())
> +		list_add_tail(&mbm_local_event.list, &r->evt_list);
> +}
> +
> +int resctrl_mon_resource_init(void)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	int ret;
> +
> +	if (!r->mon_capable)
> +		return 0;
> +
> +	ret = dom_data_init(r);
> +	if (ret)
> +		return ret;
> +
> +	l3_mon_evt_init(r);
> +
> +	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_TOTAL_EVENT_ID)) {
> +		mbm_total_event.configurable = true;
> +		mbm_config_rftype_init("mbm_total_bytes_config");
> +	}
> +	if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_LOCAL_EVENT_ID)) {
> +		mbm_local_event.configurable = true;
> +		mbm_config_rftype_init("mbm_local_bytes_config");
> +	}
> +
> +	return 0;
> +}
> +
> +void resctrl_mon_resource_exit(void)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +
> +	dom_data_exit(r);
> +}
> diff --git a/fs/resctrl/psuedo_lock.c b/fs/resctrl/psuedo_lock.c
> index e69de29bb2d1..077c2abb6edd 100644
> --- a/fs/resctrl/psuedo_lock.c
> +++ b/fs/resctrl/psuedo_lock.c
> @@ -0,0 +1,1122 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Resource Director Technology (RDT)
> + *
> + * Pseudo-locking support built on top of Cache Allocation Technology (CAT)
> + *
> + * Copyright (C) 2018 Intel Corporation
> + *
> + * Author: Reinette Chatre <reinette.chatre@intel.com>
> + */
> +
> +#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
> +
> +#include <linux/cacheinfo.h>
> +#include <linux/cpu.h>
> +#include <linux/cpumask.h>
> +#include <linux/debugfs.h>
> +#include <linux/kthread.h>
> +#include <linux/mman.h>
> +#include <linux/perf_event.h>
> +#include <linux/pm_qos.h>
> +#include <linux/slab.h>
> +#include <linux/uaccess.h>
> +
> +#include <asm/cacheflush.h>
> +#include <asm/resctrl.h>
> +#include <asm/perf_event.h>
> +
> +#include "internal.h"
> +
> +/*
> + * Major number assigned to and shared by all devices exposing
> + * pseudo-locked regions.
> + */
> +static unsigned int pseudo_lock_major;
> +static unsigned long pseudo_lock_minor_avail = GENMASK(MINORBITS, 0);
> +
> +static char *pseudo_lock_devnode(const struct device *dev, umode_t *mode)
> +{
> +	const struct rdtgroup *rdtgrp;
> +
> +	rdtgrp = dev_get_drvdata(dev);
> +	if (mode)
> +		*mode = 0600;
> +	return kasprintf(GFP_KERNEL, "pseudo_lock/%s", rdtgrp->kn->name);
> +}
> +
> +static const struct class pseudo_lock_class = {
> +	.name = "pseudo_lock",
> +	.devnode = pseudo_lock_devnode,
> +};
> +
> +/**
> + * pseudo_lock_minor_get - Obtain available minor number
> + * @minor: Pointer to where new minor number will be stored
> + *
> + * A bitmask is used to track available minor numbers. Here the next free
> + * minor number is marked as unavailable and returned.
> + *
> + * Return: 0 on success, <0 on failure.
> + */
> +static int pseudo_lock_minor_get(unsigned int *minor)
> +{
> +	unsigned long first_bit;
> +
> +	first_bit = find_first_bit(&pseudo_lock_minor_avail, MINORBITS);
> +
> +	if (first_bit == MINORBITS)
> +		return -ENOSPC;
> +
> +	__clear_bit(first_bit, &pseudo_lock_minor_avail);
> +	*minor = first_bit;
> +
> +	return 0;
> +}
> +
> +/**
> + * pseudo_lock_minor_release - Return minor number to available
> + * @minor: The minor number made available
> + */
> +static void pseudo_lock_minor_release(unsigned int minor)
> +{
> +	__set_bit(minor, &pseudo_lock_minor_avail);
> +}
> +
> +/**
> + * region_find_by_minor - Locate a pseudo-lock region by inode minor number
> + * @minor: The minor number of the device representing pseudo-locked region
> + *
> + * When the character device is accessed we need to determine which
> + * pseudo-locked region it belongs to. This is done by matching the minor
> + * number of the device to the pseudo-locked region it belongs.
> + *
> + * Minor numbers are assigned at the time a pseudo-locked region is associated
> + * with a cache instance.
> + *
> + * Return: On success return pointer to resource group owning the pseudo-locked
> + *         region, NULL on failure.
> + */
> +static struct rdtgroup *region_find_by_minor(unsigned int minor)
> +{
> +	struct rdtgroup *rdtgrp, *rdtgrp_match = NULL;
> +
> +	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
> +		if (rdtgrp->plr && rdtgrp->plr->minor == minor) {
> +			rdtgrp_match = rdtgrp;
> +			break;
> +		}
> +	}
> +	return rdtgrp_match;
> +}
> +
> +/**
> + * struct pseudo_lock_pm_req - A power management QoS request list entry
> + * @list:	Entry within the @pm_reqs list for a pseudo-locked region
> + * @req:	PM QoS request
> + */
> +struct pseudo_lock_pm_req {
> +	struct list_head list;
> +	struct dev_pm_qos_request req;
> +};
> +
> +static void pseudo_lock_cstates_relax(struct pseudo_lock_region *plr)
> +{
> +	struct pseudo_lock_pm_req *pm_req, *next;
> +
> +	list_for_each_entry_safe(pm_req, next, &plr->pm_reqs, list) {
> +		dev_pm_qos_remove_request(&pm_req->req);
> +		list_del(&pm_req->list);
> +		kfree(pm_req);
> +	}
> +}
> +
> +/**
> + * pseudo_lock_cstates_constrain - Restrict cores from entering C6
> + * @plr: Pseudo-locked region
> + *
> + * To prevent the cache from being affected by power management entering
> + * C6 has to be avoided. This is accomplished by requesting a latency
> + * requirement lower than lowest C6 exit latency of all supported
> + * platforms as found in the cpuidle state tables in the intel_idle driver.
> + * At this time it is possible to do so with a single latency requirement
> + * for all supported platforms.
> + *
> + * Since Goldmont is supported, which is affected by X86_BUG_MONITOR,
> + * the ACPI latencies need to be considered while keeping in mind that C2
> + * may be set to map to deeper sleep states. In this case the latency
> + * requirement needs to prevent entering C2 also.
> + *
> + * Return: 0 on success, <0 on failure
> + */
> +static int pseudo_lock_cstates_constrain(struct pseudo_lock_region *plr)
> +{
> +	struct pseudo_lock_pm_req *pm_req;
> +	int cpu;
> +	int ret;
> +
> +	for_each_cpu(cpu, &plr->d->cpu_mask) {
> +		pm_req = kzalloc(sizeof(*pm_req), GFP_KERNEL);
> +		if (!pm_req) {
> +			rdt_last_cmd_puts("Failure to allocate memory for PM QoS\n");
> +			ret = -ENOMEM;
> +			goto out_err;
> +		}
> +		ret = dev_pm_qos_add_request(get_cpu_device(cpu),
> +					     &pm_req->req,
> +					     DEV_PM_QOS_RESUME_LATENCY,
> +					     30);
> +		if (ret < 0) {
> +			rdt_last_cmd_printf("Failed to add latency req CPU%d\n",
> +					    cpu);
> +			kfree(pm_req);
> +			ret = -1;
> +			goto out_err;
> +		}
> +		list_add(&pm_req->list, &plr->pm_reqs);
> +	}
> +
> +	return 0;
> +
> +out_err:
> +	pseudo_lock_cstates_relax(plr);
> +	return ret;
> +}
> +
> +/**
> + * pseudo_lock_region_clear - Reset pseudo-lock region data
> + * @plr: pseudo-lock region
> + *
> + * All content of the pseudo-locked region is reset - any memory allocated
> + * freed.
> + *
> + * Return: void
> + */
> +static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)
> +{
> +	plr->size = 0;
> +	plr->line_size = 0;
> +	kfree(plr->kmem);
> +	plr->kmem = NULL;
> +	plr->s = NULL;
> +	if (plr->d)
> +		plr->d->plr = NULL;
> +	plr->d = NULL;
> +	plr->cbm = 0;
> +	plr->debugfs_dir = NULL;
> +}
> +
> +/**
> + * pseudo_lock_region_init - Initialize pseudo-lock region information
> + * @plr: pseudo-lock region
> + *
> + * Called after user provided a schemata to be pseudo-locked. From the
> + * schemata the &struct pseudo_lock_region is on entry already initialized
> + * with the resource, domain, and capacity bitmask. Here the information
> + * required for pseudo-locking is deduced from this data and &struct
> + * pseudo_lock_region initialized further. This information includes:
> + * - size in bytes of the region to be pseudo-locked
> + * - cache line size to know the stride with which data needs to be accessed
> + *   to be pseudo-locked
> + * - a cpu associated with the cache instance on which the pseudo-locking
> + *   flow can be executed
> + *
> + * Return: 0 on success, <0 on failure. Descriptive error will be written
> + * to last_cmd_status buffer.
> + */
> +static int pseudo_lock_region_init(struct pseudo_lock_region *plr)
> +{
> +	struct cpu_cacheinfo *ci;
> +	int ret;
> +	int i;
> +
> +	/* Pick the first cpu we find that is associated with the cache. */
> +	plr->cpu = cpumask_first(&plr->d->cpu_mask);
> +
> +	if (!cpu_online(plr->cpu)) {
> +		rdt_last_cmd_printf("CPU %u associated with cache not online\n",
> +				    plr->cpu);
> +		ret = -ENODEV;
> +		goto out_region;
> +	}
> +
> +	ci = get_cpu_cacheinfo(plr->cpu);
> +
> +	plr->size = rdtgroup_cbm_to_size(plr->s->res, plr->d, plr->cbm);
> +
> +	for (i = 0; i < ci->num_leaves; i++) {
> +		if (ci->info_list[i].level == plr->s->res->cache_level) {
> +			plr->line_size = ci->info_list[i].coherency_line_size;
> +			return 0;
> +		}
> +	}
> +
> +	ret = -1;
> +	rdt_last_cmd_puts("Unable to determine cache line size\n");
> +out_region:
> +	pseudo_lock_region_clear(plr);
> +	return ret;
> +}
> +
> +/**
> + * pseudo_lock_init - Initialize a pseudo-lock region
> + * @rdtgrp: resource group to which new pseudo-locked region will belong
> + *
> + * A pseudo-locked region is associated with a resource group. When this
> + * association is created the pseudo-locked region is initialized. The
> + * details of the pseudo-locked region are not known at this time so only
> + * allocation is done and association established.
> + *
> + * Return: 0 on success, <0 on failure
> + */
> +static int pseudo_lock_init(struct rdtgroup *rdtgrp)
> +{
> +	struct pseudo_lock_region *plr;
> +
> +	plr = kzalloc(sizeof(*plr), GFP_KERNEL);
> +	if (!plr)
> +		return -ENOMEM;
> +
> +	init_waitqueue_head(&plr->lock_thread_wq);
> +	INIT_LIST_HEAD(&plr->pm_reqs);
> +	rdtgrp->plr = plr;
> +	return 0;
> +}
> +
> +/**
> + * pseudo_lock_region_alloc - Allocate kernel memory that will be pseudo-locked
> + * @plr: pseudo-lock region
> + *
> + * Initialize the details required to set up the pseudo-locked region and
> + * allocate the contiguous memory that will be pseudo-locked to the cache.
> + *
> + * Return: 0 on success, <0 on failure.  Descriptive error will be written
> + * to last_cmd_status buffer.
> + */
> +static int pseudo_lock_region_alloc(struct pseudo_lock_region *plr)
> +{
> +	int ret;
> +
> +	ret = pseudo_lock_region_init(plr);
> +	if (ret < 0)
> +		return ret;
> +
> +	/*
> +	 * We do not yet support contiguous regions larger than
> +	 * KMALLOC_MAX_SIZE.
> +	 */
> +	if (plr->size > KMALLOC_MAX_SIZE) {
> +		rdt_last_cmd_puts("Requested region exceeds maximum size\n");
> +		ret = -E2BIG;
> +		goto out_region;
> +	}
> +
> +	plr->kmem = kzalloc(plr->size, GFP_KERNEL);
> +	if (!plr->kmem) {
> +		rdt_last_cmd_puts("Unable to allocate memory\n");
> +		ret = -ENOMEM;
> +		goto out_region;
> +	}
> +
> +	ret = 0;
> +	goto out;
> +out_region:
> +	pseudo_lock_region_clear(plr);
> +out:
> +	return ret;
> +}
> +
> +/**
> + * pseudo_lock_free - Free a pseudo-locked region
> + * @rdtgrp: resource group to which pseudo-locked region belonged
> + *
> + * The pseudo-locked region's resources have already been released, or not
> + * yet created at this point. Now it can be freed and disassociated from the
> + * resource group.
> + *
> + * Return: void
> + */
> +static void pseudo_lock_free(struct rdtgroup *rdtgrp)
> +{
> +	pseudo_lock_region_clear(rdtgrp->plr);
> +	kfree(rdtgrp->plr);
> +	rdtgrp->plr = NULL;
> +}
> +
> +/**
> + * rdtgroup_monitor_in_progress - Test if monitoring in progress
> + * @rdtgrp: resource group being queried
> + *
> + * Return: 1 if monitor groups have been created for this resource
> + * group, 0 otherwise.
> + */
> +static int rdtgroup_monitor_in_progress(struct rdtgroup *rdtgrp)
> +{
> +	return !list_empty(&rdtgrp->mon.crdtgrp_list);
> +}
> +
> +/**
> + * rdtgroup_locksetup_user_restrict - Restrict user access to group
> + * @rdtgrp: resource group needing access restricted
> + *
> + * A resource group used for cache pseudo-locking cannot have cpus or tasks
> + * assigned to it. This is communicated to the user by restricting access
> + * to all the files that can be used to make such changes.
> + *
> + * Permissions restored with rdtgroup_locksetup_user_restore()
> + *
> + * Return: 0 on success, <0 on failure. If a failure occurs during the
> + * restriction of access an attempt will be made to restore permissions but
> + * the state of the mode of these files will be uncertain when a failure
> + * occurs.
> + */
> +static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)
> +{
> +	int ret;
> +
> +	ret = rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
> +	if (ret)
> +		return ret;
> +
> +	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
> +	if (ret)
> +		goto err_tasks;
> +
> +	ret = rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
> +	if (ret)
> +		goto err_cpus;
> +
> +	if (resctrl_arch_mon_capable()) {
> +		ret = rdtgroup_kn_mode_restrict(rdtgrp, "mon_groups");
> +		if (ret)
> +			goto err_cpus_list;
> +	}
> +
> +	ret = 0;
> +	goto out;
> +
> +err_cpus_list:
> +	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
> +err_cpus:
> +	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
> +err_tasks:
> +	rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
> +out:
> +	return ret;
> +}
> +
> +/**
> + * rdtgroup_locksetup_user_restore - Restore user access to group
> + * @rdtgrp: resource group needing access restored
> + *
> + * Restore all file access previously removed using
> + * rdtgroup_locksetup_user_restrict()
> + *
> + * Return: 0 on success, <0 on failure.  If a failure occurs during the
> + * restoration of access an attempt will be made to restrict permissions
> + * again but the state of the mode of these files will be uncertain when
> + * a failure occurs.
> + */
> +static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)
> +{
> +	int ret;
> +
> +	ret = rdtgroup_kn_mode_restore(rdtgrp, "tasks", 0777);
> +	if (ret)
> +		return ret;
> +
> +	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0777);
> +	if (ret)
> +		goto err_tasks;
> +
> +	ret = rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0777);
> +	if (ret)
> +		goto err_cpus;
> +
> +	if (resctrl_arch_mon_capable()) {
> +		ret = rdtgroup_kn_mode_restore(rdtgrp, "mon_groups", 0777);
> +		if (ret)
> +			goto err_cpus_list;
> +	}
> +
> +	ret = 0;
> +	goto out;
> +
> +err_cpus_list:
> +	rdtgroup_kn_mode_restrict(rdtgrp, "cpus_list");
> +err_cpus:
> +	rdtgroup_kn_mode_restrict(rdtgrp, "cpus");
> +err_tasks:
> +	rdtgroup_kn_mode_restrict(rdtgrp, "tasks");
> +out:
> +	return ret;
> +}
> +
> +/**
> + * rdtgroup_locksetup_enter - Resource group enters locksetup mode
> + * @rdtgrp: resource group requested to enter locksetup mode
> + *
> + * A resource group enters locksetup mode to reflect that it would be used
> + * to represent a pseudo-locked region and is in the process of being set
> + * up to do so. A resource group used for a pseudo-locked region would
> + * lose the closid associated with it so we cannot allow it to have any
> + * tasks or cpus assigned nor permit tasks or cpus to be assigned in the
> + * future. Monitoring of a pseudo-locked region is not allowed either.
> + *
> + * The above and more restrictions on a pseudo-locked region are checked
> + * for and enforced before the resource group enters the locksetup mode.
> + *
> + * Returns: 0 if the resource group successfully entered locksetup mode, <0
> + * on failure. On failure the last_cmd_status buffer is updated with text to
> + * communicate details of failure to the user.
> + */
> +int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)
> +{
> +	int ret;
> +
> +	/*
> +	 * The default resource group can neither be removed nor lose the
> +	 * default closid associated with it.
> +	 */
> +	if (rdtgrp == &rdtgroup_default) {
> +		rdt_last_cmd_puts("Cannot pseudo-lock default group\n");
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * Cache Pseudo-locking not supported when CDP is enabled.
> +	 *
> +	 * Some things to consider if you would like to enable this
> +	 * support (using L3 CDP as example):
> +	 * - When CDP is enabled two separate resources are exposed,
> +	 *   L3DATA and L3CODE, but they are actually on the same cache.
> +	 *   The implication for pseudo-locking is that if a
> +	 *   pseudo-locked region is created on a domain of one
> +	 *   resource (eg. L3CODE), then a pseudo-locked region cannot
> +	 *   be created on that same domain of the other resource
> +	 *   (eg. L3DATA). This is because the creation of a
> +	 *   pseudo-locked region involves a call to wbinvd that will
> +	 *   affect all cache allocations on particular domain.
> +	 * - Considering the previous, it may be possible to only
> +	 *   expose one of the CDP resources to pseudo-locking and
> +	 *   hide the other. For example, we could consider to only
> +	 *   expose L3DATA and since the L3 cache is unified it is
> +	 *   still possible to place instructions there are execute it.
> +	 * - If only one region is exposed to pseudo-locking we should
> +	 *   still keep in mind that availability of a portion of cache
> +	 *   for pseudo-locking should take into account both resources.
> +	 *   Similarly, if a pseudo-locked region is created in one
> +	 *   resource, the portion of cache used by it should be made
> +	 *   unavailable to all future allocations from both resources.
> +	 */
> +	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3) ||
> +	    resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2)) {
> +		rdt_last_cmd_puts("CDP enabled\n");
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * Not knowing the bits to disable prefetching implies that this
> +	 * platform does not support Cache Pseudo-Locking.
> +	 */
> +	if (resctrl_arch_get_prefetch_disable_bits() == 0) {
> +		rdt_last_cmd_puts("Pseudo-locking not supported\n");
> +		return -EINVAL;
> +	}
> +
> +	if (rdtgroup_monitor_in_progress(rdtgrp)) {
> +		rdt_last_cmd_puts("Monitoring in progress\n");
> +		return -EINVAL;
> +	}
> +
> +	if (rdtgroup_tasks_assigned(rdtgrp)) {
> +		rdt_last_cmd_puts("Tasks assigned to resource group\n");
> +		return -EINVAL;
> +	}
> +
> +	if (!cpumask_empty(&rdtgrp->cpu_mask)) {
> +		rdt_last_cmd_puts("CPUs assigned to resource group\n");
> +		return -EINVAL;
> +	}
> +
> +	if (rdtgroup_locksetup_user_restrict(rdtgrp)) {
> +		rdt_last_cmd_puts("Unable to modify resctrl permissions\n");
> +		return -EIO;
> +	}
> +
> +	ret = pseudo_lock_init(rdtgrp);
> +	if (ret) {
> +		rdt_last_cmd_puts("Unable to init pseudo-lock region\n");
> +		goto out_release;
> +	}
> +
> +	/*
> +	 * If this system is capable of monitoring a rmid would have been
> +	 * allocated when the control group was created. This is not needed
> +	 * anymore when this group would be used for pseudo-locking. This
> +	 * is safe to call on platforms not capable of monitoring.
> +	 */
> +	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> +
> +	ret = 0;
> +	goto out;
> +
> +out_release:
> +	rdtgroup_locksetup_user_restore(rdtgrp);
> +out:
> +	return ret;
> +}
> +
> +/**
> + * rdtgroup_locksetup_exit - resource group exist locksetup mode
> + * @rdtgrp: resource group
> + *
> + * When a resource group exits locksetup mode the earlier restrictions are
> + * lifted.
> + *
> + * Return: 0 on success, <0 on failure
> + */
> +int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)
> +{
> +	int ret;
> +
> +	if (resctrl_arch_mon_capable()) {
> +		ret = alloc_rmid(rdtgrp->closid);
> +		if (ret < 0) {
> +			rdt_last_cmd_puts("Out of RMIDs\n");
> +			return ret;
> +		}
> +		rdtgrp->mon.rmid = ret;
> +	}
> +
> +	ret = rdtgroup_locksetup_user_restore(rdtgrp);
> +	if (ret) {
> +		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> +		return ret;
> +	}
> +
> +	pseudo_lock_free(rdtgrp);
> +	return 0;
> +}
> +
> +/**
> + * rdtgroup_cbm_overlaps_pseudo_locked - Test if CBM or portion is pseudo-locked
> + * @d: RDT domain
> + * @cbm: CBM to test
> + *
> + * @d represents a cache instance and @cbm a capacity bitmask that is
> + * considered for it. Determine if @cbm overlaps with any existing
> + * pseudo-locked region on @d.
> + *
> + * @cbm is unsigned long, even if only 32 bits are used, to make the
> + * bitmap functions work correctly.
> + *
> + * Return: true if @cbm overlaps with pseudo-locked region on @d, false
> + * otherwise.
> + */
> +bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
> +{
> +	unsigned int cbm_len;
> +	unsigned long cbm_b;
> +
> +	if (d->plr) {
> +		cbm_len = d->plr->s->res->cache.cbm_len;
> +		cbm_b = d->plr->cbm;
> +		if (bitmap_intersects(&cbm, &cbm_b, cbm_len))
> +			return true;
> +	}
> +	return false;
> +}
> +
> +/**
> + * rdtgroup_pseudo_locked_in_hierarchy - Pseudo-locked region in cache hierarchy
> + * @d: RDT domain under test
> + *
> + * The setup of a pseudo-locked region affects all cache instances within
> + * the hierarchy of the region. It is thus essential to know if any
> + * pseudo-locked regions exist within a cache hierarchy to prevent any
> + * attempts to create new pseudo-locked regions in the same hierarchy.
> + *
> + * Return: true if a pseudo-locked region exists in the hierarchy of @d or
> + *         if it is not possible to test due to memory allocation issue,
> + *         false otherwise.
> + */
> +bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
> +{
> +	cpumask_var_t cpu_with_psl;
> +	enum resctrl_res_level i;
> +	struct rdt_resource *r;
> +	struct rdt_domain *d_i;
> +	bool ret = false;
> +
> +	/* Walking r->domains, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +	if (!zalloc_cpumask_var(&cpu_with_psl, GFP_KERNEL))
> +		return true;
> +
> +	/*
> +	 * First determine which cpus have pseudo-locked regions
> +	 * associated with them.
> +	 */
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
> +		list_for_each_entry(d_i, &r->domains, list) {
> +			if (d_i->plr)
> +				cpumask_or(cpu_with_psl, cpu_with_psl,
> +					   &d_i->cpu_mask);
> +		}
> +	}
> +
> +	/*
> +	 * Next test if new pseudo-locked region would intersect with
> +	 * existing region.
> +	 */
> +	if (cpumask_intersects(&d->cpu_mask, cpu_with_psl))
> +		ret = true;
> +
> +	free_cpumask_var(cpu_with_psl);
> +	return ret;
> +}
> +
> +/**
> + * pseudo_lock_measure_cycles - Trigger latency measure to pseudo-locked region
> + * @rdtgrp: Resource group to which the pseudo-locked region belongs.
> + * @sel: Selector of which measurement to perform on a pseudo-locked region.
> + *
> + * The measurement of latency to access a pseudo-locked region should be
> + * done from a cpu that is associated with that pseudo-locked region.
> + * Determine which cpu is associated with this region and start a thread on
> + * that cpu to perform the measurement, wait for that thread to complete.
> + *
> + * Return: 0 on success, <0 on failure
> + */
> +static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel)
> +{
> +	struct pseudo_lock_region *plr = rdtgrp->plr;
> +	struct task_struct *thread;
> +	unsigned int cpu;
> +	int ret = -1;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	if (rdtgrp->flags & RDT_DELETED) {
> +		ret = -ENODEV;
> +		goto out;
> +	}
> +
> +	if (!plr->d) {
> +		ret = -ENODEV;
> +		goto out;
> +	}
> +
> +	plr->thread_done = 0;
> +	cpu = cpumask_first(&plr->d->cpu_mask);
> +	if (!cpu_online(cpu)) {
> +		ret = -ENODEV;
> +		goto out;
> +	}
> +
> +	plr->cpu = cpu;
> +
> +	if (sel == 1)
> +		thread = kthread_create_on_node(resctrl_arch_measure_cycles_lat_fn,
> +						plr, cpu_to_node(cpu),
> +						"pseudo_lock_measure/%u",
> +						cpu);
> +	else if (sel == 2)
> +		thread = kthread_create_on_node(resctrl_arch_measure_l2_residency,
> +						plr, cpu_to_node(cpu),
> +						"pseudo_lock_measure/%u",
> +						cpu);
> +	else if (sel == 3)
> +		thread = kthread_create_on_node(resctrl_arch_measure_l3_residency,
> +						plr, cpu_to_node(cpu),
> +						"pseudo_lock_measure/%u",
> +						cpu);
> +	else
> +		goto out;
> +
> +	if (IS_ERR(thread)) {
> +		ret = PTR_ERR(thread);
> +		goto out;
> +	}
> +	kthread_bind(thread, cpu);
> +	wake_up_process(thread);
> +
> +	ret = wait_event_interruptible(plr->lock_thread_wq,
> +				       plr->thread_done == 1);
> +	if (ret < 0)
> +		goto out;
> +
> +	ret = 0;
> +
> +out:
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +	return ret;
> +}
> +
> +static ssize_t pseudo_lock_measure_trigger(struct file *file,
> +					   const char __user *user_buf,
> +					   size_t count, loff_t *ppos)
> +{
> +	struct rdtgroup *rdtgrp = file->private_data;
> +	size_t buf_size;
> +	char buf[32];
> +	int ret;
> +	int sel;
> +
> +	buf_size = min(count, (sizeof(buf) - 1));
> +	if (copy_from_user(buf, user_buf, buf_size))
> +		return -EFAULT;
> +
> +	buf[buf_size] = '\0';
> +	ret = kstrtoint(buf, 10, &sel);
> +	if (ret == 0) {
> +		if (sel != 1 && sel != 2 && sel != 3)
> +			return -EINVAL;
> +		ret = debugfs_file_get(file->f_path.dentry);
> +		if (ret)
> +			return ret;
> +		ret = pseudo_lock_measure_cycles(rdtgrp, sel);
> +		if (ret == 0)
> +			ret = count;
> +		debugfs_file_put(file->f_path.dentry);
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct file_operations pseudo_measure_fops = {
> +	.write = pseudo_lock_measure_trigger,
> +	.open = simple_open,
> +	.llseek = default_llseek,
> +};
> +
> +/**
> + * rdtgroup_pseudo_lock_create - Create a pseudo-locked region
> + * @rdtgrp: resource group to which pseudo-lock region belongs
> + *
> + * Called when a resource group in the pseudo-locksetup mode receives a
> + * valid schemata that should be pseudo-locked. Since the resource group is
> + * in pseudo-locksetup mode the &struct pseudo_lock_region has already been
> + * allocated and initialized with the essential information. If a failure
> + * occurs the resource group remains in the pseudo-locksetup mode with the
> + * &struct pseudo_lock_region associated with it, but cleared from all
> + * information and ready for the user to re-attempt pseudo-locking by
> + * writing the schemata again.
> + *
> + * Return: 0 if the pseudo-locked region was successfully pseudo-locked, <0
> + * on failure. Descriptive error will be written to last_cmd_status buffer.
> + */
> +int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp)
> +{
> +	struct pseudo_lock_region *plr = rdtgrp->plr;
> +	struct task_struct *thread;
> +	unsigned int new_minor;
> +	struct device *dev;
> +	int ret;
> +
> +	ret = pseudo_lock_region_alloc(plr);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = pseudo_lock_cstates_constrain(plr);
> +	if (ret < 0) {
> +		ret = -EINVAL;
> +		goto out_region;
> +	}
> +
> +	plr->thread_done = 0;
> +
> +	plr->closid = rdtgrp->closid;
> +	thread = kthread_create_on_node(resctrl_arch_pseudo_lock_fn, plr,
> +					cpu_to_node(plr->cpu),
> +					"pseudo_lock/%u", plr->cpu);
> +	if (IS_ERR(thread)) {
> +		ret = PTR_ERR(thread);
> +		rdt_last_cmd_printf("Locking thread returned error %d\n", ret);
> +		goto out_cstates;
> +	}
> +
> +	kthread_bind(thread, plr->cpu);
> +	wake_up_process(thread);
> +
> +	ret = wait_event_interruptible(plr->lock_thread_wq,
> +				       plr->thread_done == 1);
> +	if (ret < 0) {
> +		/*
> +		 * If the thread does not get on the CPU for whatever
> +		 * reason and the process which sets up the region is
> +		 * interrupted then this will leave the thread in runnable
> +		 * state and once it gets on the CPU it will dereference
> +		 * the cleared, but not freed, plr struct resulting in an
> +		 * empty pseudo-locking loop.
> +		 */
> +		rdt_last_cmd_puts("Locking thread interrupted\n");
> +		goto out_cstates;
> +	}
> +
> +	ret = pseudo_lock_minor_get(&new_minor);
> +	if (ret < 0) {
> +		rdt_last_cmd_puts("Unable to obtain a new minor number\n");
> +		goto out_cstates;
> +	}
> +
> +	/*
> +	 * Unlock access but do not release the reference. The
> +	 * pseudo-locked region will still be here on return.
> +	 *
> +	 * The mutex has to be released temporarily to avoid a potential
> +	 * deadlock with the mm->mmap_lock which is obtained in the
> +	 * device_create() and debugfs_create_dir() callpath below as well as
> +	 * before the mmap() callback is called.
> +	 */
> +	mutex_unlock(&rdtgroup_mutex);
> +
> +	if (!IS_ERR_OR_NULL(debugfs_resctrl)) {
> +		plr->debugfs_dir = debugfs_create_dir(rdtgrp->kn->name,
> +						      debugfs_resctrl);
> +		if (!IS_ERR_OR_NULL(plr->debugfs_dir))
> +			debugfs_create_file("pseudo_lock_measure", 0200,
> +					    plr->debugfs_dir, rdtgrp,
> +					    &pseudo_measure_fops);
> +	}
> +
> +	dev = device_create(&pseudo_lock_class, NULL,
> +			    MKDEV(pseudo_lock_major, new_minor),
> +			    rdtgrp, "%s", rdtgrp->kn->name);
> +
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	if (IS_ERR(dev)) {
> +		ret = PTR_ERR(dev);
> +		rdt_last_cmd_printf("Failed to create character device: %d\n",
> +				    ret);
> +		goto out_debugfs;
> +	}
> +
> +	/* We released the mutex - check if group was removed while we did so */
> +	if (rdtgrp->flags & RDT_DELETED) {
> +		ret = -ENODEV;
> +		goto out_device;
> +	}
> +
> +	plr->minor = new_minor;
> +
> +	rdtgrp->mode = RDT_MODE_PSEUDO_LOCKED;
> +	closid_free(rdtgrp->closid);
> +	rdtgroup_kn_mode_restore(rdtgrp, "cpus", 0444);
> +	rdtgroup_kn_mode_restore(rdtgrp, "cpus_list", 0444);
> +
> +	ret = 0;
> +	goto out;
> +
> +out_device:
> +	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, new_minor));
> +out_debugfs:
> +	debugfs_remove_recursive(plr->debugfs_dir);
> +	pseudo_lock_minor_release(new_minor);
> +out_cstates:
> +	pseudo_lock_cstates_relax(plr);
> +out_region:
> +	pseudo_lock_region_clear(plr);
> +out:
> +	return ret;
> +}
> +
> +/**
> + * rdtgroup_pseudo_lock_remove - Remove a pseudo-locked region
> + * @rdtgrp: resource group to which the pseudo-locked region belongs
> + *
> + * The removal of a pseudo-locked region can be initiated when the resource
> + * group is removed from user space via a "rmdir" from userspace or the
> + * unmount of the resctrl filesystem. On removal the resource group does
> + * not go back to pseudo-locksetup mode before it is removed, instead it is
> + * removed directly. There is thus asymmetry with the creation where the
> + * &struct pseudo_lock_region is removed here while it was not created in
> + * rdtgroup_pseudo_lock_create().
> + *
> + * Return: void
> + */
> +void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp)
> +{
> +	struct pseudo_lock_region *plr = rdtgrp->plr;
> +
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +		/*
> +		 * Default group cannot be a pseudo-locked region so we can
> +		 * free closid here.
> +		 */
> +		closid_free(rdtgrp->closid);
> +		goto free;
> +	}
> +
> +	pseudo_lock_cstates_relax(plr);
> +	debugfs_remove_recursive(rdtgrp->plr->debugfs_dir);
> +	device_destroy(&pseudo_lock_class, MKDEV(pseudo_lock_major, plr->minor));
> +	pseudo_lock_minor_release(plr->minor);
> +
> +free:
> +	pseudo_lock_free(rdtgrp);
> +}
> +
> +static int pseudo_lock_dev_open(struct inode *inode, struct file *filp)
> +{
> +	struct rdtgroup *rdtgrp;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	rdtgrp = region_find_by_minor(iminor(inode));
> +	if (!rdtgrp) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -ENODEV;
> +	}
> +
> +	filp->private_data = rdtgrp;
> +	atomic_inc(&rdtgrp->waitcount);
> +	/* Perform a non-seekable open - llseek is not supported */
> +	filp->f_mode &= ~(FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE);
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +
> +	return 0;
> +}
> +
> +static int pseudo_lock_dev_release(struct inode *inode, struct file *filp)
> +{
> +	struct rdtgroup *rdtgrp;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +	rdtgrp = filp->private_data;
> +	WARN_ON(!rdtgrp);
> +	if (!rdtgrp) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -ENODEV;
> +	}
> +	filp->private_data = NULL;
> +	atomic_dec(&rdtgrp->waitcount);
> +	mutex_unlock(&rdtgroup_mutex);
> +	return 0;
> +}
> +
> +static int pseudo_lock_dev_mremap(struct vm_area_struct *area)
> +{
> +	/* Not supported */
> +	return -EINVAL;
> +}
> +
> +static const struct vm_operations_struct pseudo_mmap_ops = {
> +	.mremap = pseudo_lock_dev_mremap,
> +};
> +
> +static int pseudo_lock_dev_mmap(struct file *filp, struct vm_area_struct *vma)
> +{
> +	unsigned long vsize = vma->vm_end - vma->vm_start;
> +	unsigned long off = vma->vm_pgoff << PAGE_SHIFT;
> +	struct pseudo_lock_region *plr;
> +	struct rdtgroup *rdtgrp;
> +	unsigned long physical;
> +	unsigned long psize;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	rdtgrp = filp->private_data;
> +	WARN_ON(!rdtgrp);
> +	if (!rdtgrp) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -ENODEV;
> +	}
> +
> +	plr = rdtgrp->plr;
> +
> +	if (!plr->d) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -ENODEV;
> +	}
> +
> +	/*
> +	 * Task is required to run with affinity to the cpus associated
> +	 * with the pseudo-locked region. If this is not the case the task
> +	 * may be scheduled elsewhere and invalidate entries in the
> +	 * pseudo-locked region.
> +	 */
> +	if (!cpumask_subset(current->cpus_ptr, &plr->d->cpu_mask)) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -EINVAL;
> +	}
> +
> +	physical = __pa(plr->kmem) >> PAGE_SHIFT;
> +	psize = plr->size - off;
> +
> +	if (off > plr->size) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -ENOSPC;
> +	}
> +
> +	/*
> +	 * Ensure changes are carried directly to the memory being mapped,
> +	 * do not allow copy-on-write mapping.
> +	 */
> +	if (!(vma->vm_flags & VM_SHARED)) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -EINVAL;
> +	}
> +
> +	if (vsize > psize) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -ENOSPC;
> +	}
> +
> +	memset(plr->kmem + off, 0, vsize);
> +
> +	if (remap_pfn_range(vma, vma->vm_start, physical + vma->vm_pgoff,
> +			    vsize, vma->vm_page_prot)) {
> +		mutex_unlock(&rdtgroup_mutex);
> +		return -EAGAIN;
> +	}
> +	vma->vm_ops = &pseudo_mmap_ops;
> +	mutex_unlock(&rdtgroup_mutex);
> +	return 0;
> +}
> +
> +static const struct file_operations pseudo_lock_dev_fops = {
> +	.owner =	THIS_MODULE,
> +	.llseek =	no_llseek,
> +	.read =		NULL,
> +	.write =	NULL,
> +	.open =		pseudo_lock_dev_open,
> +	.release =	pseudo_lock_dev_release,
> +	.mmap =		pseudo_lock_dev_mmap,
> +};
> +
> +int rdt_pseudo_lock_init(void)
> +{
> +	int ret;
> +
> +	ret = register_chrdev(0, "pseudo_lock", &pseudo_lock_dev_fops);
> +	if (ret < 0)
> +		return ret;
> +
> +	pseudo_lock_major = ret;
> +
> +	ret = class_register(&pseudo_lock_class);
> +	if (ret) {
> +		unregister_chrdev(pseudo_lock_major, "pseudo_lock");
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +void rdt_pseudo_lock_release(void)
> +{
> +	class_unregister(&pseudo_lock_class);
> +	unregister_chrdev(pseudo_lock_major, "pseudo_lock");
> +	pseudo_lock_major = 0;
> +}
> diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
> index e69de29bb2d1..936fc6e47386 100644
> --- a/fs/resctrl/rdtgroup.c
> +++ b/fs/resctrl/rdtgroup.c
> @@ -0,0 +1,4013 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * User interface for Resource Allocation in Resource Director Technology(RDT)
> + *
> + * Copyright (C) 2016 Intel Corporation
> + *
> + * Author: Fenghua Yu <fenghua.yu@intel.com>
> + *
> + * More information about RDT be found in the Intel (R) x86 Architecture
> + * Software Developer Manual.
> + */
> +
> +#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
> +
> +#include <linux/cacheinfo.h>
> +#include <linux/cpu.h>
> +#include <linux/debugfs.h>
> +#include <linux/fs.h>
> +#include <linux/fs_parser.h>
> +#include <linux/sysfs.h>
> +#include <linux/kernfs.h>
> +#include <linux/seq_buf.h>
> +#include <linux/seq_file.h>
> +#include <linux/sched/signal.h>
> +#include <linux/sched/task.h>
> +#include <linux/slab.h>
> +#include <linux/task_work.h>
> +#include <linux/user_namespace.h>
> +
> +#include <uapi/linux/magic.h>
> +
> +#include <asm/resctrl.h>
> +#include "internal.h"
> +
> +/* Mutex to protect rdtgroup access. */
> +DEFINE_MUTEX(rdtgroup_mutex);
> +
> +static struct kernfs_root *rdt_root;
> +struct rdtgroup rdtgroup_default;
> +LIST_HEAD(rdt_all_groups);
> +
> +/* list of entries for the schemata file */
> +LIST_HEAD(resctrl_schema_all);
> +
> +/* The filesystem can only be mounted once. */
> +bool resctrl_mounted;
> +
> +/* Kernel fs node for "info" directory under root */
> +static struct kernfs_node *kn_info;
> +
> +/* Kernel fs node for "mon_groups" directory under root */
> +static struct kernfs_node *kn_mongrp;
> +
> +/* Kernel fs node for "mon_data" directory under root */
> +static struct kernfs_node *kn_mondata;
> +
> +/*
> + * Used to store the max resource name width and max resource data width
> + * to display the schemata in a tabular format
> + */
> +int max_name_width, max_data_width;
> +
> +static struct seq_buf last_cmd_status;
> +static char last_cmd_status_buf[512];
> +
> +static int rdtgroup_setup_root(struct rdt_fs_context *ctx);
> +static void rdtgroup_destroy_root(void);
> +
> +struct dentry *debugfs_resctrl;
> +
> +static bool resctrl_debug;
> +
> +void rdt_last_cmd_clear(void)
> +{
> +	lockdep_assert_held(&rdtgroup_mutex);
> +	seq_buf_clear(&last_cmd_status);
> +}
> +
> +void rdt_last_cmd_puts(const char *s)
> +{
> +	lockdep_assert_held(&rdtgroup_mutex);
> +	seq_buf_puts(&last_cmd_status, s);
> +}
> +
> +void rdt_last_cmd_printf(const char *fmt, ...)
> +{
> +	va_list ap;
> +
> +	va_start(ap, fmt);
> +	lockdep_assert_held(&rdtgroup_mutex);
> +	seq_buf_vprintf(&last_cmd_status, fmt, ap);
> +	va_end(ap);
> +}
> +
> +void rdt_staged_configs_clear(void)
> +{
> +	enum resctrl_res_level i;
> +	struct rdt_resource *r;
> +	struct rdt_domain *dom;
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
> +		list_for_each_entry(dom, &r->domains, list)
> +			memset(dom->staged_config, 0, sizeof(dom->staged_config));
> +	}
> +}
> +
> +static bool resctrl_is_mbm_enabled(void)
> +{
> +	return (resctrl_arch_is_mbm_total_enabled() ||
> +		resctrl_arch_is_mbm_local_enabled());
> +}
> +
> +static bool resctrl_is_mbm_event(int e)
> +{
> +	return (e >= QOS_L3_MBM_TOTAL_EVENT_ID &&
> +		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
> +}
> +
> +/*
> + * Trivial allocator for CLOSIDs. Since h/w only supports a small number,
> + * we can keep a bitmap of free CLOSIDs in a single integer.
> + *
> + * Using a global CLOSID across all resources has some advantages and
> + * some drawbacks:
> + * + We can simply set current's closid to assign a task to a resource
> + *   group.
> + * + Context switch code can avoid extra memory references deciding which
> + *   CLOSID to load into the PQR_ASSOC MSR
> + * - We give up some options in configuring resource groups across multi-socket
> + *   systems.
> + * - Our choices on how to configure each resource become progressively more
> + *   limited as the number of resources grows.
> + */
> +static unsigned long closid_free_map;
> +static int closid_free_map_len;
> +
> +int closids_supported(void)
> +{
> +	return closid_free_map_len;
> +}
> +
> +static void closid_init(void)
> +{
> +	struct resctrl_schema *s;
> +	u32 rdt_min_closid = 32;
> +
> +	/* Compute rdt_min_closid across all resources */
> +	list_for_each_entry(s, &resctrl_schema_all, list)
> +		rdt_min_closid = min(rdt_min_closid, s->num_closid);
> +
> +	closid_free_map = BIT_MASK(rdt_min_closid) - 1;
> +
> +	/* RESCTRL_RESERVED_CLOSID is always reserved for the default group */
> +	__clear_bit(RESCTRL_RESERVED_CLOSID, &closid_free_map);
> +	closid_free_map_len = rdt_min_closid;
> +}
> +
> +static int closid_alloc(void)
> +{
> +	int cleanest_closid;
> +	u32 closid;
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) &&
> +	    resctrl_arch_is_llc_occupancy_enabled()) {
> +		cleanest_closid = resctrl_find_cleanest_closid();
> +		if (cleanest_closid < 0)
> +			return cleanest_closid;
> +		closid = cleanest_closid;
> +	} else {
> +		closid = ffs(closid_free_map);
> +		if (closid == 0)
> +			return -ENOSPC;
> +		closid--;
> +	}
> +	__clear_bit(closid, &closid_free_map);
> +
> +	return closid;
> +}
> +
> +void closid_free(int closid)
> +{
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	__set_bit(closid, &closid_free_map);
> +}
> +
> +/**
> + * closid_allocated - test if provided closid is in use
> + * @closid: closid to be tested
> + *
> + * Return: true if @closid is currently associated with a resource group,
> + * false if @closid is free
> + */
> +bool closid_allocated(unsigned int closid)
> +{
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	return !test_bit(closid, &closid_free_map);
> +}
> +
> +/**
> + * rdtgroup_mode_by_closid - Return mode of resource group with closid
> + * @closid: closid if the resource group
> + *
> + * Each resource group is associated with a @closid. Here the mode
> + * of a resource group can be queried by searching for it using its closid.
> + *
> + * Return: mode as &enum rdtgrp_mode of resource group with closid @closid
> + */
> +enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)
> +{
> +	struct rdtgroup *rdtgrp;
> +
> +	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
> +		if (rdtgrp->closid == closid)
> +			return rdtgrp->mode;
> +	}
> +
> +	return RDT_NUM_MODES;
> +}
> +
> +static const char * const rdt_mode_str[] = {
> +	[RDT_MODE_SHAREABLE]		= "shareable",
> +	[RDT_MODE_EXCLUSIVE]		= "exclusive",
> +	[RDT_MODE_PSEUDO_LOCKSETUP]	= "pseudo-locksetup",
> +	[RDT_MODE_PSEUDO_LOCKED]	= "pseudo-locked",
> +};
> +
> +/**
> + * rdtgroup_mode_str - Return the string representation of mode
> + * @mode: the resource group mode as &enum rdtgroup_mode
> + *
> + * Return: string representation of valid mode, "unknown" otherwise
> + */
> +static const char *rdtgroup_mode_str(enum rdtgrp_mode mode)
> +{
> +	if (mode < RDT_MODE_SHAREABLE || mode >= RDT_NUM_MODES)
> +		return "unknown";
> +
> +	return rdt_mode_str[mode];
> +}
> +
> +/* set uid and gid of rdtgroup dirs and files to that of the creator */
> +static int rdtgroup_kn_set_ugid(struct kernfs_node *kn)
> +{
> +	struct iattr iattr = { .ia_valid = ATTR_UID | ATTR_GID,
> +				.ia_uid = current_fsuid(),
> +				.ia_gid = current_fsgid(), };
> +
> +	if (uid_eq(iattr.ia_uid, GLOBAL_ROOT_UID) &&
> +	    gid_eq(iattr.ia_gid, GLOBAL_ROOT_GID))
> +		return 0;
> +
> +	return kernfs_setattr(kn, &iattr);
> +}
> +
> +static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
> +{
> +	struct kernfs_node *kn;
> +	int ret;
> +
> +	kn = __kernfs_create_file(parent_kn, rft->name, rft->mode,
> +				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID,
> +				  0, rft->kf_ops, rft, NULL, NULL);
> +	if (IS_ERR(kn))
> +		return PTR_ERR(kn);
> +
> +	ret = rdtgroup_kn_set_ugid(kn);
> +	if (ret) {
> +		kernfs_remove(kn);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int rdtgroup_seqfile_show(struct seq_file *m, void *arg)
> +{
> +	struct kernfs_open_file *of = m->private;
> +	struct rftype *rft = of->kn->priv;
> +
> +	if (rft->seq_show)
> +		return rft->seq_show(of, m, arg);
> +	return 0;
> +}
> +
> +static ssize_t rdtgroup_file_write(struct kernfs_open_file *of, char *buf,
> +				   size_t nbytes, loff_t off)
> +{
> +	struct rftype *rft = of->kn->priv;
> +
> +	if (rft->write)
> +		return rft->write(of, buf, nbytes, off);
> +
> +	return -EINVAL;
> +}
> +
> +static const struct kernfs_ops rdtgroup_kf_single_ops = {
> +	.atomic_write_len	= PAGE_SIZE,
> +	.write			= rdtgroup_file_write,
> +	.seq_show		= rdtgroup_seqfile_show,
> +};
> +
> +static const struct kernfs_ops kf_mondata_ops = {
> +	.atomic_write_len	= PAGE_SIZE,
> +	.seq_show		= rdtgroup_mondata_show,
> +};
> +
> +static bool is_cpu_list(struct kernfs_open_file *of)
> +{
> +	struct rftype *rft = of->kn->priv;
> +
> +	return rft->flags & RFTYPE_FLAGS_CPUS_LIST;
> +}
> +
> +static int rdtgroup_cpus_show(struct kernfs_open_file *of,
> +			      struct seq_file *s, void *v)
> +{
> +	struct rdtgroup *rdtgrp;
> +	struct cpumask *mask;
> +	int ret = 0;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +
> +	if (rdtgrp) {
> +		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> +			if (!rdtgrp->plr->d) {
> +				rdt_last_cmd_clear();
> +				rdt_last_cmd_puts("Cache domain offline\n");
> +				ret = -ENODEV;
> +			} else {
> +				mask = &rdtgrp->plr->d->cpu_mask;
> +				seq_printf(s, is_cpu_list(of) ?
> +					   "%*pbl\n" : "%*pb\n",
> +					   cpumask_pr_args(mask));
> +			}
> +		} else {
> +			seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
> +				   cpumask_pr_args(&rdtgrp->cpu_mask));
> +		}
> +	} else {
> +		ret = -ENOENT;
> +	}
> +	rdtgroup_kn_unlock(of->kn);
> +
> +	return ret;
> +}
> +
> +/*
> + * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
> + *
> + * Per task closids/rmids must have been set up before calling this function.
> + * @r may be NULL.
> + */
> +static void
> +update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
> +{
> +	struct resctrl_cpu_sync defaults;
> +	struct resctrl_cpu_sync *defaults_p = NULL;
> +
> +	if (r) {
> +		defaults.closid = r->closid;
> +		defaults.rmid = r->mon.rmid;
> +		defaults_p = &defaults;
> +	}
> +
> +	on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_defaults, defaults_p,
> +			 1);
> +}
> +
> +static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
> +			  cpumask_var_t tmpmask)
> +{
> +	struct rdtgroup *prgrp = rdtgrp->mon.parent, *crgrp;
> +	struct list_head *head;
> +
> +	/* Check whether cpus belong to parent ctrl group */
> +	cpumask_andnot(tmpmask, newmask, &prgrp->cpu_mask);
> +	if (!cpumask_empty(tmpmask)) {
> +		rdt_last_cmd_puts("Can only add CPUs to mongroup that belong to parent\n");
> +		return -EINVAL;
> +	}
> +
> +	/* Check whether cpus are dropped from this group */
> +	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
> +	if (!cpumask_empty(tmpmask)) {
> +		/* Give any dropped cpus to parent rdtgroup */
> +		cpumask_or(&prgrp->cpu_mask, &prgrp->cpu_mask, tmpmask);
> +		update_closid_rmid(tmpmask, prgrp);
> +	}
> +
> +	/*
> +	 * If we added cpus, remove them from previous group that owned them
> +	 * and update per-cpu rmid
> +	 */
> +	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
> +	if (!cpumask_empty(tmpmask)) {
> +		head = &prgrp->mon.crdtgrp_list;
> +		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
> +			if (crgrp == rdtgrp)
> +				continue;
> +			cpumask_andnot(&crgrp->cpu_mask, &crgrp->cpu_mask,
> +				       tmpmask);
> +		}
> +		update_closid_rmid(tmpmask, rdtgrp);
> +	}
> +
> +	/* Done pushing/pulling - update this group with new mask */
> +	cpumask_copy(&rdtgrp->cpu_mask, newmask);
> +
> +	return 0;
> +}
> +
> +static void cpumask_rdtgrp_clear(struct rdtgroup *r, struct cpumask *m)
> +{
> +	struct rdtgroup *crgrp;
> +
> +	cpumask_andnot(&r->cpu_mask, &r->cpu_mask, m);
> +	/* update the child mon group masks as well*/
> +	list_for_each_entry(crgrp, &r->mon.crdtgrp_list, mon.crdtgrp_list)
> +		cpumask_and(&crgrp->cpu_mask, &r->cpu_mask, &crgrp->cpu_mask);
> +}
> +
> +static int cpus_ctrl_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
> +			   cpumask_var_t tmpmask, cpumask_var_t tmpmask1)
> +{
> +	struct rdtgroup *r, *crgrp;
> +	struct list_head *head;
> +
> +	/* Check whether cpus are dropped from this group */
> +	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
> +	if (!cpumask_empty(tmpmask)) {
> +		/* Can't drop from default group */
> +		if (rdtgrp == &rdtgroup_default) {
> +			rdt_last_cmd_puts("Can't drop CPUs from default group\n");
> +			return -EINVAL;
> +		}
> +
> +		/* Give any dropped cpus to rdtgroup_default */
> +		cpumask_or(&rdtgroup_default.cpu_mask,
> +			   &rdtgroup_default.cpu_mask, tmpmask);
> +		update_closid_rmid(tmpmask, &rdtgroup_default);
> +	}
> +
> +	/*
> +	 * If we added cpus, remove them from previous group and
> +	 * the prev group's child groups that owned them
> +	 * and update per-cpu closid/rmid.
> +	 */
> +	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
> +	if (!cpumask_empty(tmpmask)) {
> +		list_for_each_entry(r, &rdt_all_groups, rdtgroup_list) {
> +			if (r == rdtgrp)
> +				continue;
> +			cpumask_and(tmpmask1, &r->cpu_mask, tmpmask);
> +			if (!cpumask_empty(tmpmask1))
> +				cpumask_rdtgrp_clear(r, tmpmask1);
> +		}
> +		update_closid_rmid(tmpmask, rdtgrp);
> +	}
> +
> +	/* Done pushing/pulling - update this group with new mask */
> +	cpumask_copy(&rdtgrp->cpu_mask, newmask);
> +
> +	/*
> +	 * Clear child mon group masks since there is a new parent mask
> +	 * now and update the rmid for the cpus the child lost.
> +	 */
> +	head = &rdtgrp->mon.crdtgrp_list;
> +	list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
> +		cpumask_and(tmpmask, &rdtgrp->cpu_mask, &crgrp->cpu_mask);
> +		update_closid_rmid(tmpmask, rdtgrp);
> +		cpumask_clear(&crgrp->cpu_mask);
> +	}
> +
> +	return 0;
> +}
> +
> +static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of,
> +				   char *buf, size_t nbytes, loff_t off)
> +{
> +	cpumask_var_t tmpmask, newmask, tmpmask1;
> +	struct rdtgroup *rdtgrp;
> +	int ret;
> +
> +	if (!buf)
> +		return -EINVAL;
> +
> +	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
> +		return -ENOMEM;
> +	if (!zalloc_cpumask_var(&newmask, GFP_KERNEL)) {
> +		free_cpumask_var(tmpmask);
> +		return -ENOMEM;
> +	}
> +	if (!zalloc_cpumask_var(&tmpmask1, GFP_KERNEL)) {
> +		free_cpumask_var(tmpmask);
> +		free_cpumask_var(newmask);
> +		return -ENOMEM;
> +	}
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (!rdtgrp) {
> +		ret = -ENOENT;
> +		goto unlock;
> +	}
> +
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
> +	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +		ret = -EINVAL;
> +		rdt_last_cmd_puts("Pseudo-locking in progress\n");
> +		goto unlock;
> +	}
> +
> +	if (is_cpu_list(of))
> +		ret = cpulist_parse(buf, newmask);
> +	else
> +		ret = cpumask_parse(buf, newmask);
> +
> +	if (ret) {
> +		rdt_last_cmd_puts("Bad CPU list/mask\n");
> +		goto unlock;
> +	}
> +
> +	/* check that user didn't specify any offline cpus */
> +	cpumask_andnot(tmpmask, newmask, cpu_online_mask);
> +	if (!cpumask_empty(tmpmask)) {
> +		ret = -EINVAL;
> +		rdt_last_cmd_puts("Can only assign online CPUs\n");
> +		goto unlock;
> +	}
> +
> +	if (rdtgrp->type == RDTCTRL_GROUP)
> +		ret = cpus_ctrl_write(rdtgrp, newmask, tmpmask, tmpmask1);
> +	else if (rdtgrp->type == RDTMON_GROUP)
> +		ret = cpus_mon_write(rdtgrp, newmask, tmpmask);
> +	else
> +		ret = -EINVAL;
> +
> +unlock:
> +	rdtgroup_kn_unlock(of->kn);
> +	free_cpumask_var(tmpmask);
> +	free_cpumask_var(newmask);
> +	free_cpumask_var(tmpmask1);
> +
> +	return ret ?: nbytes;
> +}
> +
> +/**
> + * rdtgroup_remove - the helper to remove resource group safely
> + * @rdtgrp: resource group to remove
> + *
> + * On resource group creation via a mkdir, an extra kernfs_node reference is
> + * taken to ensure that the rdtgroup structure remains accessible for the
> + * rdtgroup_kn_unlock() calls where it is removed.
> + *
> + * Drop the extra reference here, then free the rdtgroup structure.
> + *
> + * Return: void
> + */
> +static void rdtgroup_remove(struct rdtgroup *rdtgrp)
> +{
> +	kernfs_put(rdtgrp->kn);
> +	kfree(rdtgrp);
> +}
> +
> +static void _update_task_closid_rmid(void *task)
> +{
> +	/*
> +	 * If the task is still current on this CPU, update PQR_ASSOC MSR.
> +	 * Otherwise, the MSR is updated when the task is scheduled in.
> +	 */
> +	if (task == current)
> +		resctrl_arch_sched_in(task);
> +}
> +
> +static void update_task_closid_rmid(struct task_struct *t)
> +{
> +	if (IS_ENABLED(CONFIG_SMP) && task_curr(t))
> +		smp_call_function_single(task_cpu(t), _update_task_closid_rmid, t, 1);
> +	else
> +		_update_task_closid_rmid(t);
> +}
> +
> +static bool task_in_rdtgroup(struct task_struct *tsk, struct rdtgroup *rdtgrp)
> +{
> +	u32 closid, rmid = rdtgrp->mon.rmid;
> +
> +	if (rdtgrp->type == RDTCTRL_GROUP)
> +		closid = rdtgrp->closid;
> +	else if (rdtgrp->type == RDTMON_GROUP)
> +		closid = rdtgrp->mon.parent->closid;
> +	else
> +		return false;
> +
> +	return resctrl_arch_match_closid(tsk, closid) &&
> +	       resctrl_arch_match_rmid(tsk, closid, rmid);
> +}
> +
> +static int __rdtgroup_move_task(struct task_struct *tsk,
> +				struct rdtgroup *rdtgrp)
> +{
> +	/* If the task is already in rdtgrp, no need to move the task. */
> +	if (task_in_rdtgroup(tsk, rdtgrp))
> +		return 0;
> +
> +	/*
> +	 * Set the task's closid/rmid before the PQR_ASSOC MSR can be
> +	 * updated by them.
> +	 *
> +	 * For ctrl_mon groups, move both closid and rmid.
> +	 * For monitor groups, can move the tasks only from
> +	 * their parent CTRL group.
> +	 */
> +	if (rdtgrp->type == RDTMON_GROUP &&
> +	    !resctrl_arch_match_closid(tsk, rdtgrp->mon.parent->closid)) {
> +		rdt_last_cmd_puts("Can't move task to different control group\n");
> +		return -EINVAL;
> +	}
> +
> +	if (rdtgrp->type == RDTMON_GROUP)
> +		resctrl_arch_set_closid_rmid(tsk, rdtgrp->mon.parent->closid,
> +					     rdtgrp->mon.rmid);
> +	else
> +		resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid,
> +					     rdtgrp->mon.rmid);
> +
> +	/*
> +	 * Ensure the task's closid and rmid are written before determining if
> +	 * the task is current that will decide if it will be interrupted.
> +	 * This pairs with the full barrier between the rq->curr update and
> +	 * resctrl_arch_sched_in() during context switch.
> +	 */
> +	smp_mb();
> +
> +	/*
> +	 * By now, the task's closid and rmid are set. If the task is current
> +	 * on a CPU, the PQR_ASSOC MSR needs to be updated to make the resource
> +	 * group go into effect. If the task is not current, the MSR will be
> +	 * updated when the task is scheduled in.
> +	 */
> +	update_task_closid_rmid(tsk);
> +
> +	return 0;
> +}
> +
> +static bool is_closid_match(struct task_struct *t, struct rdtgroup *r)
> +{
> +	return (resctrl_arch_alloc_capable() && (r->type == RDTCTRL_GROUP) &&
> +		resctrl_arch_match_closid(t, r->closid));
> +}
> +
> +static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r)
> +{
> +	return (resctrl_arch_mon_capable() && (r->type == RDTMON_GROUP) &&
> +		resctrl_arch_match_rmid(t, r->mon.parent->closid,
> +					r->mon.rmid));
> +}
> +
> +/**
> + * rdtgroup_tasks_assigned - Test if tasks have been assigned to resource group
> + * @r: Resource group
> + *
> + * Return: 1 if tasks have been assigned to @r, 0 otherwise
> + */
> +int rdtgroup_tasks_assigned(struct rdtgroup *r)
> +{
> +	struct task_struct *p, *t;
> +	int ret = 0;
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	rcu_read_lock();
> +	for_each_process_thread(p, t) {
> +		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
> +			ret = 1;
> +			break;
> +		}
> +	}
> +	rcu_read_unlock();
> +
> +	return ret;
> +}
> +
> +static int rdtgroup_task_write_permission(struct task_struct *task,
> +					  struct kernfs_open_file *of)
> +{
> +	const struct cred *tcred = get_task_cred(task);
> +	const struct cred *cred = current_cred();
> +	int ret = 0;
> +
> +	/*
> +	 * Even if we're attaching all tasks in the thread group, we only
> +	 * need to check permissions on one of them.
> +	 */
> +	if (!uid_eq(cred->euid, GLOBAL_ROOT_UID) &&
> +	    !uid_eq(cred->euid, tcred->uid) &&
> +	    !uid_eq(cred->euid, tcred->suid)) {
> +		rdt_last_cmd_printf("No permission to move task %d\n", task->pid);
> +		ret = -EPERM;
> +	}
> +
> +	put_cred(tcred);
> +	return ret;
> +}
> +
> +static int rdtgroup_move_task(pid_t pid, struct rdtgroup *rdtgrp,
> +			      struct kernfs_open_file *of)
> +{
> +	struct task_struct *tsk;
> +	int ret;
> +
> +	rcu_read_lock();
> +	if (pid) {
> +		tsk = find_task_by_vpid(pid);
> +		if (!tsk) {
> +			rcu_read_unlock();
> +			rdt_last_cmd_printf("No task %d\n", pid);
> +			return -ESRCH;
> +		}
> +	} else {
> +		tsk = current;
> +	}
> +
> +	get_task_struct(tsk);
> +	rcu_read_unlock();
> +
> +	ret = rdtgroup_task_write_permission(tsk, of);
> +	if (!ret)
> +		ret = __rdtgroup_move_task(tsk, rdtgrp);
> +
> +	put_task_struct(tsk);
> +	return ret;
> +}
> +
> +static ssize_t rdtgroup_tasks_write(struct kernfs_open_file *of,
> +				    char *buf, size_t nbytes, loff_t off)
> +{
> +	struct rdtgroup *rdtgrp;
> +	char *pid_str;
> +	int ret = 0;
> +	pid_t pid;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (!rdtgrp) {
> +		rdtgroup_kn_unlock(of->kn);
> +		return -ENOENT;
> +	}
> +	rdt_last_cmd_clear();
> +
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED ||
> +	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +		ret = -EINVAL;
> +		rdt_last_cmd_puts("Pseudo-locking in progress\n");
> +		goto unlock;
> +	}
> +
> +	while (buf && buf[0] != '\0' && buf[0] != '\n') {
> +		pid_str = strim(strsep(&buf, ","));
> +
> +		if (kstrtoint(pid_str, 0, &pid)) {
> +			rdt_last_cmd_printf("Task list parsing error pid %s\n", pid_str);
> +			ret = -EINVAL;
> +			break;
> +		}
> +
> +		if (pid < 0) {
> +			rdt_last_cmd_printf("Invalid pid %d\n", pid);
> +			ret = -EINVAL;
> +			break;
> +		}
> +
> +		ret = rdtgroup_move_task(pid, rdtgrp, of);
> +		if (ret) {
> +			rdt_last_cmd_printf("Error while processing task %d\n", pid);
> +			break;
> +		}
> +	}
> +
> +unlock:
> +	rdtgroup_kn_unlock(of->kn);
> +
> +	return ret ?: nbytes;
> +}
> +
> +static void show_rdt_tasks(struct rdtgroup *r, struct seq_file *s)
> +{
> +	struct task_struct *p, *t;
> +	pid_t pid;
> +
> +	rcu_read_lock();
> +	for_each_process_thread(p, t) {
> +		if (is_closid_match(t, r) || is_rmid_match(t, r)) {
> +			pid = task_pid_vnr(t);
> +			if (pid)
> +				seq_printf(s, "%d\n", pid);
> +		}
> +	}
> +	rcu_read_unlock();
> +}
> +
> +static int rdtgroup_tasks_show(struct kernfs_open_file *of,
> +			       struct seq_file *s, void *v)
> +{
> +	struct rdtgroup *rdtgrp;
> +	int ret = 0;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (rdtgrp)
> +		show_rdt_tasks(rdtgrp, s);
> +	else
> +		ret = -ENOENT;
> +	rdtgroup_kn_unlock(of->kn);
> +
> +	return ret;
> +}
> +
> +static int rdtgroup_closid_show(struct kernfs_open_file *of,
> +				struct seq_file *s, void *v)
> +{
> +	struct rdtgroup *rdtgrp;
> +	int ret = 0;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (rdtgrp)
> +		seq_printf(s, "%u\n", rdtgrp->closid);
> +	else
> +		ret = -ENOENT;
> +	rdtgroup_kn_unlock(of->kn);
> +
> +	return ret;
> +}
> +
> +static int rdtgroup_rmid_show(struct kernfs_open_file *of,
> +			      struct seq_file *s, void *v)
> +{
> +	struct rdtgroup *rdtgrp;
> +	int ret = 0;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (rdtgrp)
> +		seq_printf(s, "%u\n", rdtgrp->mon.rmid);
> +	else
> +		ret = -ENOENT;
> +	rdtgroup_kn_unlock(of->kn);
> +
> +	return ret;
> +}
> +
> +#ifdef CONFIG_PROC_CPU_RESCTRL
> +
> +/*
> + * A task can only be part of one resctrl control group and of one monitor
> + * group which is associated to that control group.
> + *
> + * 1)   res:
> + *      mon:
> + *
> + *    resctrl is not available.
> + *
> + * 2)   res:/
> + *      mon:
> + *
> + *    Task is part of the root resctrl control group, and it is not associated
> + *    to any monitor group.
> + *
> + * 3)  res:/
> + *     mon:mon0
> + *
> + *    Task is part of the root resctrl control group and monitor group mon0.
> + *
> + * 4)  res:group0
> + *     mon:
> + *
> + *    Task is part of resctrl control group group0, and it is not associated
> + *    to any monitor group.
> + *
> + * 5) res:group0
> + *    mon:mon1
> + *
> + *    Task is part of resctrl control group group0 and monitor group mon1.
> + */
> +int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,
> +		      struct pid *pid, struct task_struct *tsk)
> +{
> +	struct rdtgroup *rdtg;
> +	int ret = 0;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	/* Return empty if resctrl has not been mounted. */
> +	if (!resctrl_mounted) {
> +		seq_puts(s, "res:\nmon:\n");
> +		goto unlock;
> +	}
> +
> +	list_for_each_entry(rdtg, &rdt_all_groups, rdtgroup_list) {
> +		struct rdtgroup *crg;
> +
> +		/*
> +		 * Task information is only relevant for shareable
> +		 * and exclusive groups.
> +		 */
> +		if (rdtg->mode != RDT_MODE_SHAREABLE &&
> +		    rdtg->mode != RDT_MODE_EXCLUSIVE)
> +			continue;
> +
> +		if (!resctrl_arch_match_closid(tsk, rdtg->closid))
> +			continue;
> +
> +		seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "",
> +			   rdtg->kn->name);
> +		seq_puts(s, "mon:");
> +		list_for_each_entry(crg, &rdtg->mon.crdtgrp_list,
> +				    mon.crdtgrp_list) {
> +			if (!resctrl_arch_match_rmid(tsk, crg->mon.parent->closid,
> +						     crg->mon.rmid))
> +				continue;
> +			seq_printf(s, "%s", crg->kn->name);
> +			break;
> +		}
> +		seq_putc(s, '\n');
> +		goto unlock;
> +	}
> +	/*
> +	 * The above search should succeed. Otherwise return
> +	 * with an error.
> +	 */
> +	ret = -ENOENT;
> +unlock:
> +	mutex_unlock(&rdtgroup_mutex);
> +
> +	return ret;
> +}
> +#endif
> +
> +static int rdt_last_cmd_status_show(struct kernfs_open_file *of,
> +				    struct seq_file *seq, void *v)
> +{
> +	int len;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +	len = seq_buf_used(&last_cmd_status);
> +	if (len)
> +		seq_printf(seq, "%.*s", len, last_cmd_status_buf);
> +	else
> +		seq_puts(seq, "ok\n");
> +	mutex_unlock(&rdtgroup_mutex);
> +	return 0;
> +}
> +
> +static int rdt_num_closids_show(struct kernfs_open_file *of,
> +				struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +
> +	seq_printf(seq, "%u\n", s->num_closid);
> +	return 0;
> +}
> +
> +static int rdt_default_ctrl_show(struct kernfs_open_file *of,
> +			     struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	seq_printf(seq, "%x\n", r->default_ctrl);
> +	return 0;
> +}
> +
> +static int rdt_min_cbm_bits_show(struct kernfs_open_file *of,
> +			     struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	seq_printf(seq, "%u\n", r->cache.min_cbm_bits);
> +	return 0;
> +}
> +
> +static int rdt_shareable_bits_show(struct kernfs_open_file *of,
> +				   struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	seq_printf(seq, "%x\n", r->cache.shareable_bits);
> +	return 0;
> +}
> +
> +/*
> + * rdt_bit_usage_show - Display current usage of resources
> + *
> + * A domain is a shared resource that can now be allocated differently. Here
> + * we display the current regions of the domain as an annotated bitmask.
> + * For each domain of this resource its allocation bitmask
> + * is annotated as below to indicate the current usage of the corresponding bit:
> + *   0 - currently unused
> + *   X - currently available for sharing and used by software and hardware
> + *   H - currently used by hardware only but available for software use
> + *   S - currently used and shareable by software only
> + *   E - currently used exclusively by one resource group
> + *   P - currently pseudo-locked by one resource group
> + */
> +static int rdt_bit_usage_show(struct kernfs_open_file *of,
> +			      struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	/*
> +	 * Use unsigned long even though only 32 bits are used to ensure
> +	 * test_bit() is used safely.
> +	 */
> +	unsigned long sw_shareable = 0, hw_shareable = 0;
> +	unsigned long exclusive = 0, pseudo_locked = 0;
> +	struct rdt_resource *r = s->res;
> +	struct rdt_domain *dom;
> +	int i, hwb, swb, excl, psl;
> +	enum rdtgrp_mode mode;
> +	bool sep = false;
> +	u32 ctrl_val;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +	hw_shareable = r->cache.shareable_bits;
> +	list_for_each_entry(dom, &r->domains, list) {
> +		if (sep)
> +			seq_putc(seq, ';');
> +		sw_shareable = 0;
> +		exclusive = 0;
> +		seq_printf(seq, "%d=", dom->id);
> +		for (i = 0; i < closids_supported(); i++) {
> +			if (!closid_allocated(i))
> +				continue;
> +			ctrl_val = resctrl_arch_get_config(r, dom, i,
> +							   s->conf_type);
> +			mode = rdtgroup_mode_by_closid(i);
> +			switch (mode) {
> +			case RDT_MODE_SHAREABLE:
> +				sw_shareable |= ctrl_val;
> +				break;
> +			case RDT_MODE_EXCLUSIVE:
> +				exclusive |= ctrl_val;
> +				break;
> +			case RDT_MODE_PSEUDO_LOCKSETUP:
> +			/*
> +			 * RDT_MODE_PSEUDO_LOCKSETUP is possible
> +			 * here but not included since the CBM
> +			 * associated with this CLOSID in this mode
> +			 * is not initialized and no task or cpu can be
> +			 * assigned this CLOSID.
> +			 */
> +				break;
> +			case RDT_MODE_PSEUDO_LOCKED:
> +			case RDT_NUM_MODES:
> +				WARN(1,
> +				     "invalid mode for closid %d\n", i);
> +				break;
> +			}
> +		}
> +		for (i = r->cache.cbm_len - 1; i >= 0; i--) {
> +			pseudo_locked = dom->plr ? dom->plr->cbm : 0;
> +			hwb = test_bit(i, &hw_shareable);
> +			swb = test_bit(i, &sw_shareable);
> +			excl = test_bit(i, &exclusive);
> +			psl = test_bit(i, &pseudo_locked);
> +			if (hwb && swb)
> +				seq_putc(seq, 'X');
> +			else if (hwb && !swb)
> +				seq_putc(seq, 'H');
> +			else if (!hwb && swb)
> +				seq_putc(seq, 'S');
> +			else if (excl)
> +				seq_putc(seq, 'E');
> +			else if (psl)
> +				seq_putc(seq, 'P');
> +			else /* Unused bits remain */
> +				seq_putc(seq, '0');
> +		}
> +		sep = true;
> +	}
> +	seq_putc(seq, '\n');
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +	return 0;
> +}
> +
> +static int rdt_min_bw_show(struct kernfs_open_file *of,
> +			     struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	seq_printf(seq, "%u\n", r->membw.min_bw);
> +	return 0;
> +}
> +
> +static int rdt_num_rmids_show(struct kernfs_open_file *of,
> +			      struct seq_file *seq, void *v)
> +{
> +	struct rdt_resource *r = of->kn->parent->priv;
> +
> +	seq_printf(seq, "%d\n", r->num_rmid);
> +
> +	return 0;
> +}
> +
> +static int rdt_mon_features_show(struct kernfs_open_file *of,
> +				 struct seq_file *seq, void *v)
> +{
> +	struct rdt_resource *r = of->kn->parent->priv;
> +	struct mon_evt *mevt;
> +
> +	list_for_each_entry(mevt, &r->evt_list, list) {
> +		seq_printf(seq, "%s\n", mevt->name);
> +		if (mevt->configurable)
> +			seq_printf(seq, "%s_config\n", mevt->name);
> +	}
> +
> +	return 0;
> +}
> +
> +static int rdt_bw_gran_show(struct kernfs_open_file *of,
> +			     struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	seq_printf(seq, "%u\n", r->membw.bw_gran);
> +	return 0;
> +}
> +
> +static int rdt_delay_linear_show(struct kernfs_open_file *of,
> +			     struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	seq_printf(seq, "%u\n", r->membw.delay_linear);
> +	return 0;
> +}
> +
> +static int max_threshold_occ_show(struct kernfs_open_file *of,
> +				  struct seq_file *seq, void *v)
> +{
> +	seq_printf(seq, "%u\n", resctrl_rmid_realloc_threshold);
> +
> +	return 0;
> +}
> +
> +static int rdt_thread_throttle_mode_show(struct kernfs_open_file *of,
> +					 struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	if (r->membw.throttle_mode == THREAD_THROTTLE_PER_THREAD)
> +		seq_puts(seq, "per-thread\n");
> +	else
> +		seq_puts(seq, "max\n");
> +
> +	return 0;
> +}
> +
> +static ssize_t max_threshold_occ_write(struct kernfs_open_file *of,
> +				       char *buf, size_t nbytes, loff_t off)
> +{
> +	unsigned int bytes;
> +	int ret;
> +
> +	ret = kstrtouint(buf, 0, &bytes);
> +	if (ret)
> +		return ret;
> +
> +	if (bytes > resctrl_rmid_realloc_limit)
> +		return -EINVAL;
> +
> +	resctrl_rmid_realloc_threshold = resctrl_arch_round_mon_val(bytes);
> +
> +	return nbytes;
> +}
> +
> +/*
> + * rdtgroup_mode_show - Display mode of this resource group
> + */
> +static int rdtgroup_mode_show(struct kernfs_open_file *of,
> +			      struct seq_file *s, void *v)
> +{
> +	struct rdtgroup *rdtgrp;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (!rdtgrp) {
> +		rdtgroup_kn_unlock(of->kn);
> +		return -ENOENT;
> +	}
> +
> +	seq_printf(s, "%s\n", rdtgroup_mode_str(rdtgrp->mode));
> +
> +	rdtgroup_kn_unlock(of->kn);
> +	return 0;
> +}
> +
> +static enum resctrl_conf_type resctrl_peer_type(enum resctrl_conf_type my_type)
> +{
> +	switch (my_type) {
> +	case CDP_CODE:
> +		return CDP_DATA;
> +	case CDP_DATA:
> +		return CDP_CODE;
> +	default:
> +	case CDP_NONE:
> +		return CDP_NONE;
> +	}
> +}
> +
> +static int rdt_has_sparse_bitmasks_show(struct kernfs_open_file *of,
> +					struct seq_file *seq, void *v)
> +{
> +	struct resctrl_schema *s = of->kn->parent->priv;
> +	struct rdt_resource *r = s->res;
> +
> +	seq_printf(seq, "%u\n", r->cache.arch_has_sparse_bitmasks);
> +
> +	return 0;
> +}
> +
> +/**
> + * __rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other
> + * @r: Resource to which domain instance @d belongs.
> + * @d: The domain instance for which @closid is being tested.
> + * @cbm: Capacity bitmask being tested.
> + * @closid: Intended closid for @cbm.
> + * @type: CDP type of @r.
> + * @exclusive: Only check if overlaps with exclusive resource groups
> + *
> + * Checks if provided @cbm intended to be used for @closid on domain
> + * @d overlaps with any other closids or other hardware usage associated
> + * with this domain. If @exclusive is true then only overlaps with
> + * resource groups in exclusive mode will be considered. If @exclusive
> + * is false then overlaps with any resource group or hardware entities
> + * will be considered.
> + *
> + * @cbm is unsigned long, even if only 32 bits are used, to make the
> + * bitmap functions work correctly.
> + *
> + * Return: false if CBM does not overlap, true if it does.
> + */
> +static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
> +				    unsigned long cbm, int closid,
> +				    enum resctrl_conf_type type, bool exclusive)
> +{
> +	enum rdtgrp_mode mode;
> +	unsigned long ctrl_b;
> +	int i;
> +
> +	/* Check for any overlap with regions used by hardware directly */
> +	if (!exclusive) {
> +		ctrl_b = r->cache.shareable_bits;
> +		if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len))
> +			return true;
> +	}
> +
> +	/* Check for overlap with other resource groups */
> +	for (i = 0; i < closids_supported(); i++) {
> +		ctrl_b = resctrl_arch_get_config(r, d, i, type);
> +		mode = rdtgroup_mode_by_closid(i);
> +		if (closid_allocated(i) && i != closid &&
> +		    mode != RDT_MODE_PSEUDO_LOCKSETUP) {
> +			if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len)) {
> +				if (exclusive) {
> +					if (mode == RDT_MODE_EXCLUSIVE)
> +						return true;
> +					continue;
> +				}
> +				return true;
> +			}
> +		}
> +	}
> +
> +	return false;
> +}
> +
> +/**
> + * rdtgroup_cbm_overlaps - Does CBM overlap with other use of hardware
> + * @s: Schema for the resource to which domain instance @d belongs.
> + * @d: The domain instance for which @closid is being tested.
> + * @cbm: Capacity bitmask being tested.
> + * @closid: Intended closid for @cbm.
> + * @exclusive: Only check if overlaps with exclusive resource groups
> + *
> + * Resources that can be allocated using a CBM can use the CBM to control
> + * the overlap of these allocations. rdtgroup_cmb_overlaps() is the test
> + * for overlap. Overlap test is not limited to the specific resource for
> + * which the CBM is intended though - when dealing with CDP resources that
> + * share the underlying hardware the overlap check should be performed on
> + * the CDP resource sharing the hardware also.
> + *
> + * Refer to description of __rdtgroup_cbm_overlaps() for the details of the
> + * overlap test.
> + *
> + * Return: true if CBM overlap detected, false if there is no overlap
> + */
> +bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_domain *d,
> +			   unsigned long cbm, int closid, bool exclusive)
> +{
> +	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
> +	struct rdt_resource *r = s->res;
> +
> +	if (__rdtgroup_cbm_overlaps(r, d, cbm, closid, s->conf_type,
> +				    exclusive))
> +		return true;
> +
> +	if (!resctrl_arch_get_cdp_enabled(r->rid))
> +		return false;
> +	return  __rdtgroup_cbm_overlaps(r, d, cbm, closid, peer_type, exclusive);
> +}
> +
> +/**
> + * rdtgroup_mode_test_exclusive - Test if this resource group can be exclusive
> + * @rdtgrp: Resource group identified through its closid.
> + *
> + * An exclusive resource group implies that there should be no sharing of
> + * its allocated resources. At the time this group is considered to be
> + * exclusive this test can determine if its current schemata supports this
> + * setting by testing for overlap with all other resource groups.
> + *
> + * Return: true if resource group can be exclusive, false if there is overlap
> + * with allocations of other resource groups and thus this resource group
> + * cannot be exclusive.
> + */
> +static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
> +{
> +	int closid = rdtgrp->closid;
> +	struct resctrl_schema *s;
> +	struct rdt_resource *r;
> +	bool has_cache = false;
> +	struct rdt_domain *d;
> +	u32 ctrl;
> +
> +	/* Walking r->domains, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +	list_for_each_entry(s, &resctrl_schema_all, list) {
> +		r = s->res;
> +		if (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)
> +			continue;
> +		has_cache = true;
> +		list_for_each_entry(d, &r->domains, list) {
> +			ctrl = resctrl_arch_get_config(r, d, closid,
> +						       s->conf_type);
> +			if (rdtgroup_cbm_overlaps(s, d, ctrl, closid, false)) {
> +				rdt_last_cmd_puts("Schemata overlaps\n");
> +				return false;
> +			}
> +		}
> +	}
> +
> +	if (!has_cache) {
> +		rdt_last_cmd_puts("Cannot be exclusive without CAT/CDP\n");
> +		return false;
> +	}
> +
> +	return true;
> +}
> +
> +/*
> + * rdtgroup_mode_write - Modify the resource group's mode
> + */
> +static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
> +				   char *buf, size_t nbytes, loff_t off)
> +{
> +	struct rdtgroup *rdtgrp;
> +	enum rdtgrp_mode mode;
> +	int ret = 0;
> +
> +	/* Valid input requires a trailing newline */
> +	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> +		return -EINVAL;
> +	buf[nbytes - 1] = '\0';
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (!rdtgrp) {
> +		rdtgroup_kn_unlock(of->kn);
> +		return -ENOENT;
> +	}
> +
> +	rdt_last_cmd_clear();
> +
> +	mode = rdtgrp->mode;
> +
> +	if ((!strcmp(buf, "shareable") && mode == RDT_MODE_SHAREABLE) ||
> +	    (!strcmp(buf, "exclusive") && mode == RDT_MODE_EXCLUSIVE) ||
> +	    (!strcmp(buf, "pseudo-locksetup") &&
> +	     mode == RDT_MODE_PSEUDO_LOCKSETUP) ||
> +	    (!strcmp(buf, "pseudo-locked") && mode == RDT_MODE_PSEUDO_LOCKED))
> +		goto out;
> +
> +	if (mode == RDT_MODE_PSEUDO_LOCKED) {
> +		rdt_last_cmd_puts("Cannot change pseudo-locked group\n");
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (!strcmp(buf, "shareable")) {
> +		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +			ret = rdtgroup_locksetup_exit(rdtgrp);
> +			if (ret)
> +				goto out;
> +		}
> +		rdtgrp->mode = RDT_MODE_SHAREABLE;
> +	} else if (!strcmp(buf, "exclusive")) {
> +		if (!rdtgroup_mode_test_exclusive(rdtgrp)) {
> +			ret = -EINVAL;
> +			goto out;
> +		}
> +		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +			ret = rdtgroup_locksetup_exit(rdtgrp);
> +			if (ret)
> +				goto out;
> +		}
> +		rdtgrp->mode = RDT_MODE_EXCLUSIVE;
> +	} else if (IS_ENABLED(CONFIG_RESCTRL_FS_PSEUDO_LOCK) &&
> +		   !strcmp(buf, "pseudo-locksetup")) {
> +		ret = rdtgroup_locksetup_enter(rdtgrp);
> +		if (ret)
> +			goto out;
> +		rdtgrp->mode = RDT_MODE_PSEUDO_LOCKSETUP;
> +	} else {
> +		rdt_last_cmd_puts("Unknown or unsupported mode\n");
> +		ret = -EINVAL;
> +	}
> +
> +out:
> +	rdtgroup_kn_unlock(of->kn);
> +	return ret ?: nbytes;
> +}
> +
> +/**
> + * rdtgroup_cbm_to_size - Translate CBM to size in bytes
> + * @r: RDT resource to which @d belongs.
> + * @d: RDT domain instance.
> + * @cbm: bitmask for which the size should be computed.
> + *
> + * The bitmask provided associated with the RDT domain instance @d will be
> + * translated into how many bytes it represents. The size in bytes is
> + * computed by first dividing the total cache size by the CBM length to
> + * determine how many bytes each bit in the bitmask represents. The result
> + * is multiplied with the number of bits set in the bitmask.
> + *
> + * @cbm is unsigned long, even if only 32 bits are used to make the
> + * bitmap functions work correctly.
> + */
> +unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,
> +				  struct rdt_domain *d, unsigned long cbm)
> +{
> +	struct cpu_cacheinfo *ci;
> +	unsigned int size = 0;
> +	int num_b, i;
> +
> +	num_b = bitmap_weight(&cbm, r->cache.cbm_len);
> +	ci = get_cpu_cacheinfo(cpumask_any(&d->cpu_mask));
> +	for (i = 0; i < ci->num_leaves; i++) {
> +		if (ci->info_list[i].level == r->cache_level) {
> +			size = ci->info_list[i].size / r->cache.cbm_len * num_b;
> +			break;
> +		}
> +	}
> +
> +	return size;
> +}
> +
> +/*
> + * rdtgroup_size_show - Display size in bytes of allocated regions
> + *
> + * The "size" file mirrors the layout of the "schemata" file, printing the
> + * size in bytes of each region instead of the capacity bitmask.
> + */
> +static int rdtgroup_size_show(struct kernfs_open_file *of,
> +			      struct seq_file *s, void *v)
> +{
> +	struct resctrl_schema *schema;
> +	enum resctrl_conf_type type;
> +	struct rdtgroup *rdtgrp;
> +	struct rdt_resource *r;
> +	struct rdt_domain *d;
> +	unsigned int size;
> +	int ret = 0;
> +	u32 closid;
> +	bool sep;
> +	u32 ctrl;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(of->kn);
> +	if (!rdtgrp) {
> +		rdtgroup_kn_unlock(of->kn);
> +		return -ENOENT;
> +	}
> +
> +	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> +		if (!rdtgrp->plr->d) {
> +			rdt_last_cmd_clear();
> +			rdt_last_cmd_puts("Cache domain offline\n");
> +			ret = -ENODEV;
> +		} else {
> +			seq_printf(s, "%*s:", max_name_width,
> +				   rdtgrp->plr->s->name);
> +			size = rdtgroup_cbm_to_size(rdtgrp->plr->s->res,
> +						    rdtgrp->plr->d,
> +						    rdtgrp->plr->cbm);
> +			seq_printf(s, "%d=%u\n", rdtgrp->plr->d->id, size);
> +		}
> +		goto out;
> +	}
> +
> +	closid = rdtgrp->closid;
> +
> +	list_for_each_entry(schema, &resctrl_schema_all, list) {
> +		r = schema->res;
> +		type = schema->conf_type;
> +		sep = false;
> +		seq_printf(s, "%*s:", max_name_width, schema->name);
> +		list_for_each_entry(d, &r->domains, list) {
> +			if (sep)
> +				seq_putc(s, ';');
> +			if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
> +				size = 0;
> +			} else {
> +				if (is_mba_sc(r))
> +					ctrl = d->mbps_val[closid];
> +				else
> +					ctrl = resctrl_arch_get_config(r, d,
> +								       closid,
> +								       type);
> +				if (r->rid == RDT_RESOURCE_MBA ||
> +				    r->rid == RDT_RESOURCE_SMBA)
> +					size = ctrl;
> +				else
> +					size = rdtgroup_cbm_to_size(r, d, ctrl);
> +			}
> +			seq_printf(s, "%d=%u", d->id, size);
> +			sep = true;
> +		}
> +		seq_putc(s, '\n');
> +	}
> +
> +out:
> +	rdtgroup_kn_unlock(of->kn);
> +
> +	return ret;
> +}
> +
> +static void mondata_config_read(struct resctrl_mon_config_info *mon_info)
> +{
> +	smp_call_function_any(&mon_info->d->cpu_mask,
> +			      resctrl_arch_mon_event_config_read, mon_info, 1);
> +}
> +
> +static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid)
> +{
> +	struct resctrl_mon_config_info mon_info = {0};
> +	struct rdt_domain *dom;
> +	bool sep = false;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	list_for_each_entry(dom, &r->domains, list) {
> +		if (sep)
> +			seq_puts(s, ";");
> +
> +		memset(&mon_info, 0, sizeof(struct resctrl_mon_config_info));
> +		mon_info.r = r;
> +		mon_info.d = dom;
> +		mon_info.evtid = evtid;
> +		mondata_config_read(&mon_info);
> +
> +		seq_printf(s, "%d=0x%02x", dom->id, mon_info.mon_config);
> +		sep = true;
> +	}
> +	seq_puts(s, "\n");
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +
> +	return 0;
> +}
> +
> +static int mbm_total_bytes_config_show(struct kernfs_open_file *of,
> +				       struct seq_file *seq, void *v)
> +{
> +	struct rdt_resource *r = of->kn->parent->priv;
> +
> +	mbm_config_show(seq, r, QOS_L3_MBM_TOTAL_EVENT_ID);
> +
> +	return 0;
> +}
> +
> +static int mbm_local_bytes_config_show(struct kernfs_open_file *of,
> +				       struct seq_file *seq, void *v)
> +{
> +	struct rdt_resource *r = of->kn->parent->priv;
> +
> +	mbm_config_show(seq, r, QOS_L3_MBM_LOCAL_EVENT_ID);
> +
> +	return 0;
> +}
> +
> +static int mbm_config_write_domain(struct rdt_resource *r,
> +				   struct rdt_domain *d, u32 evtid, u32 val)
> +{
> +	struct resctrl_mon_config_info mon_info = {0};
> +
> +	/*
> +	 * Read the current config value first. If both are the same then
> +	 * no need to write it again.
> +	 */
> +	mon_info.r = r;
> +	mon_info.d = d;
> +	mon_info.evtid = evtid;
> +	mondata_config_read(&mon_info);
> +	if (mon_info.mon_config == val)
> +		return 0;
> +
> +	mon_info.mon_config = val;
> +
> +	/*
> +	 * Update MSR_IA32_EVT_CFG_BASE MSR on one of the CPUs in the
> +	 * domain. The MSRs offset from MSR MSR_IA32_EVT_CFG_BASE
> +	 * are scoped at the domain level. Writing any of these MSRs
> +	 * on one CPU is observed by all the CPUs in the domain.
> +	 */
> +	smp_call_function_any(&d->cpu_mask, resctrl_arch_mon_event_config_write,
> +			      &mon_info, 1);
> +	if (mon_info.err) {
> +		rdt_last_cmd_puts("Invalid event configuration\n");
> +		return mon_info.err;
> +	}
> +
> +	/*
> +	 * When an Event Configuration is changed, the bandwidth counters
> +	 * for all RMIDs and Events will be cleared by the hardware. The
> +	 * hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for
> +	 * every RMID on the next read to any event for every RMID.
> +	 * Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62)
> +	 * cleared while it is tracked by the hardware. Clear the
> +	 * mbm_local and mbm_total counts for all the RMIDs.
> +	 */
> +	resctrl_arch_reset_rmid_all(r, d);
> +
> +	return 0;
> +}
> +
> +static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
> +{
> +	char *dom_str = NULL, *id_str;
> +	unsigned long dom_id, val;
> +	struct rdt_domain *d;
> +	int err;
> +
> +	/* Walking r->domains, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +next:
> +	if (!tok || tok[0] == '\0')
> +		return 0;
> +
> +	/* Start processing the strings for each domain */
> +	dom_str = strim(strsep(&tok, ";"));
> +	id_str = strsep(&dom_str, "=");
> +
> +	if (!id_str || kstrtoul(id_str, 10, &dom_id)) {
> +		rdt_last_cmd_puts("Missing '=' or non-numeric domain id\n");
> +		return -EINVAL;
> +	}
> +
> +	if (!dom_str || kstrtoul(dom_str, 16, &val)) {
> +		rdt_last_cmd_puts("Non-numeric event configuration value\n");
> +		return -EINVAL;
> +	}
> +
> +	/* Value from user cannot be more than the supported set of events */
> +	if ((val & r->mbm_cfg_mask) != val) {
> +		rdt_last_cmd_printf("Invalid event configuration: max valid mask is 0x%02x\n",
> +				    r->mbm_cfg_mask);
> +		return -EINVAL;
> +	}
> +
> +	list_for_each_entry(d, &r->domains, list) {
> +		if (d->id == dom_id) {
> +			err = mbm_config_write_domain(r, d, evtid, val);
> +			if (err)
> +				return err;
> +			goto next;
> +		}
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of,
> +					    char *buf, size_t nbytes,
> +					    loff_t off)
> +{
> +	struct rdt_resource *r = of->kn->parent->priv;
> +	int ret;
> +
> +	/* Valid input requires a trailing newline */
> +	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> +		return -EINVAL;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	rdt_last_cmd_clear();
> +
> +	buf[nbytes - 1] = '\0';
> +
> +	ret = mon_config_write(r, buf, QOS_L3_MBM_TOTAL_EVENT_ID);
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +
> +	return ret ?: nbytes;
> +}
> +
> +static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of,
> +					    char *buf, size_t nbytes,
> +					    loff_t off)
> +{
> +	struct rdt_resource *r = of->kn->parent->priv;
> +	int ret;
> +
> +	/* Valid input requires a trailing newline */
> +	if (nbytes == 0 || buf[nbytes - 1] != '\n')
> +		return -EINVAL;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	rdt_last_cmd_clear();
> +
> +	buf[nbytes - 1] = '\0';
> +
> +	ret = mon_config_write(r, buf, QOS_L3_MBM_LOCAL_EVENT_ID);
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +
> +	return ret ?: nbytes;
> +}
> +
> +/* rdtgroup information files for one cache resource. */
> +static struct rftype res_common_files[] = {
> +	{
> +		.name		= "last_cmd_status",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_last_cmd_status_show,
> +		.fflags		= RFTYPE_TOP_INFO,
> +	},
> +	{
> +		.name		= "num_closids",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_num_closids_show,
> +		.fflags		= RFTYPE_CTRL_INFO,
> +	},
> +	{
> +		.name		= "mon_features",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_mon_features_show,
> +		.fflags		= RFTYPE_MON_INFO,
> +	},
> +	{
> +		.name		= "num_rmids",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_num_rmids_show,
> +		.fflags		= RFTYPE_MON_INFO,
> +	},
> +	{
> +		.name		= "cbm_mask",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_default_ctrl_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> +	},
> +	{
> +		.name		= "min_cbm_bits",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_min_cbm_bits_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> +	},
> +	{
> +		.name		= "shareable_bits",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_shareable_bits_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> +	},
> +	{
> +		.name		= "bit_usage",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_bit_usage_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> +	},
> +	{
> +		.name		= "min_bandwidth",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_min_bw_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
> +	},
> +	{
> +		.name		= "bandwidth_gran",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_bw_gran_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
> +	},
> +	{
> +		.name		= "delay_linear",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_delay_linear_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_MB,
> +	},
> +	/*
> +	 * Platform specific which (if any) capabilities are provided by
> +	 * thread_throttle_mode. Defer "fflags" initialization to platform
> +	 * discovery.
> +	 */
> +	{
> +		.name		= "thread_throttle_mode",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_thread_throttle_mode_show,
> +	},
> +	{
> +		.name		= "max_threshold_occupancy",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.write		= max_threshold_occ_write,
> +		.seq_show	= max_threshold_occ_show,
> +		.fflags		= RFTYPE_MON_INFO | RFTYPE_RES_CACHE,
> +	},
> +	{
> +		.name		= "mbm_total_bytes_config",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= mbm_total_bytes_config_show,
> +		.write		= mbm_total_bytes_config_write,
> +	},
> +	{
> +		.name		= "mbm_local_bytes_config",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= mbm_local_bytes_config_show,
> +		.write		= mbm_local_bytes_config_write,
> +	},
> +	{
> +		.name		= "cpus",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.write		= rdtgroup_cpus_write,
> +		.seq_show	= rdtgroup_cpus_show,
> +		.fflags		= RFTYPE_BASE,
> +	},
> +	{
> +		.name		= "cpus_list",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.write		= rdtgroup_cpus_write,
> +		.seq_show	= rdtgroup_cpus_show,
> +		.flags		= RFTYPE_FLAGS_CPUS_LIST,
> +		.fflags		= RFTYPE_BASE,
> +	},
> +	{
> +		.name		= "tasks",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.write		= rdtgroup_tasks_write,
> +		.seq_show	= rdtgroup_tasks_show,
> +		.fflags		= RFTYPE_BASE,
> +	},
> +	{
> +		.name		= "mon_hw_id",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdtgroup_rmid_show,
> +		.fflags		= RFTYPE_MON_BASE | RFTYPE_DEBUG,
> +	},
> +	{
> +		.name		= "schemata",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.write		= rdtgroup_schemata_write,
> +		.seq_show	= rdtgroup_schemata_show,
> +		.fflags		= RFTYPE_CTRL_BASE,
> +	},
> +	{
> +		.name		= "mode",
> +		.mode		= 0644,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.write		= rdtgroup_mode_write,
> +		.seq_show	= rdtgroup_mode_show,
> +		.fflags		= RFTYPE_CTRL_BASE,
> +	},
> +	{
> +		.name		= "size",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdtgroup_size_show,
> +		.fflags		= RFTYPE_CTRL_BASE,
> +	},
> +	{
> +		.name		= "sparse_masks",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdt_has_sparse_bitmasks_show,
> +		.fflags		= RFTYPE_CTRL_INFO | RFTYPE_RES_CACHE,
> +	},
> +	{
> +		.name		= "ctrl_hw_id",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdtgroup_closid_show,
> +		.fflags		= RFTYPE_CTRL_BASE | RFTYPE_DEBUG,
> +	},
> +
> +};
> +
> +static int rdtgroup_add_files(struct kernfs_node *kn, unsigned long fflags)
> +{
> +	struct rftype *rfts, *rft;
> +	int ret, len;
> +
> +	rfts = res_common_files;
> +	len = ARRAY_SIZE(res_common_files);
> +
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	if (resctrl_debug)
> +		fflags |= RFTYPE_DEBUG;
> +
> +	for (rft = rfts; rft < rfts + len; rft++) {
> +		if (rft->fflags && ((fflags & rft->fflags) == rft->fflags)) {
> +			ret = rdtgroup_add_file(kn, rft);
> +			if (ret)
> +				goto error;
> +		}
> +	}
> +
> +	return 0;
> +error:
> +	pr_warn("Failed to add %s, err=%d\n", rft->name, ret);
> +	while (--rft >= rfts) {
> +		if ((fflags & rft->fflags) == rft->fflags)
> +			kernfs_remove_by_name(kn, rft->name);
> +	}
> +	return ret;
> +}
> +
> +static struct rftype *rdtgroup_get_rftype_by_name(const char *name)
> +{
> +	struct rftype *rfts, *rft;
> +	int len;
> +
> +	rfts = res_common_files;
> +	len = ARRAY_SIZE(res_common_files);
> +
> +	for (rft = rfts; rft < rfts + len; rft++) {
> +		if (!strcmp(rft->name, name))
> +			return rft;
> +	}
> +
> +	return NULL;
> +}
> +
> +static void thread_throttle_mode_init(void)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> +	struct rftype *rft;
> +
> +	if (!r->alloc_capable ||
> +	    r->membw.throttle_mode == THREAD_THROTTLE_UNDEFINED)
> +		return;
> +
> +	rft = rdtgroup_get_rftype_by_name("thread_throttle_mode");
> +	if (!rft)
> +		return;
> +
> +	rft->fflags = RFTYPE_CTRL_INFO | RFTYPE_RES_MB;
> +}
> +
> +void mbm_config_rftype_init(const char *config)
> +{
> +	struct rftype *rft;
> +
> +	rft = rdtgroup_get_rftype_by_name(config);
> +	if (rft)
> +		rft->fflags = RFTYPE_MON_INFO | RFTYPE_RES_CACHE;
> +}
> +
> +/**
> + * rdtgroup_kn_mode_restrict - Restrict user access to named resctrl file
> + * @r: The resource group with which the file is associated.
> + * @name: Name of the file
> + *
> + * The permissions of named resctrl file, directory, or link are modified
> + * to not allow read, write, or execute by any user.
> + *
> + * WARNING: This function is intended to communicate to the user that the
> + * resctrl file has been locked down - that it is not relevant to the
> + * particular state the system finds itself in. It should not be relied
> + * on to protect from user access because after the file's permissions
> + * are restricted the user can still change the permissions using chmod
> + * from the command line.
> + *
> + * Return: 0 on success, <0 on failure.
> + */
> +int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name)
> +{
> +	struct iattr iattr = {.ia_valid = ATTR_MODE,};
> +	struct kernfs_node *kn;
> +	int ret = 0;
> +
> +	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
> +	if (!kn)
> +		return -ENOENT;
> +
> +	switch (kernfs_type(kn)) {
> +	case KERNFS_DIR:
> +		iattr.ia_mode = S_IFDIR;
> +		break;
> +	case KERNFS_FILE:
> +		iattr.ia_mode = S_IFREG;
> +		break;
> +	case KERNFS_LINK:
> +		iattr.ia_mode = S_IFLNK;
> +		break;
> +	}
> +
> +	ret = kernfs_setattr(kn, &iattr);
> +	kernfs_put(kn);
> +	return ret;
> +}
> +
> +/**
> + * rdtgroup_kn_mode_restore - Restore user access to named resctrl file
> + * @r: The resource group with which the file is associated.
> + * @name: Name of the file
> + * @mask: Mask of permissions that should be restored
> + *
> + * Restore the permissions of the named file. If @name is a directory the
> + * permissions of its parent will be used.
> + *
> + * Return: 0 on success, <0 on failure.
> + */
> +int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name,
> +			     umode_t mask)
> +{
> +	struct iattr iattr = {.ia_valid = ATTR_MODE,};
> +	struct kernfs_node *kn, *parent;
> +	struct rftype *rfts, *rft;
> +	int ret, len;
> +
> +	rfts = res_common_files;
> +	len = ARRAY_SIZE(res_common_files);
> +
> +	for (rft = rfts; rft < rfts + len; rft++) {
> +		if (!strcmp(rft->name, name))
> +			iattr.ia_mode = rft->mode & mask;
> +	}
> +
> +	kn = kernfs_find_and_get_ns(r->kn, name, NULL);
> +	if (!kn)
> +		return -ENOENT;
> +
> +	switch (kernfs_type(kn)) {
> +	case KERNFS_DIR:
> +		parent = kernfs_get_parent(kn);
> +		if (parent) {
> +			iattr.ia_mode |= parent->mode;
> +			kernfs_put(parent);
> +		}
> +		iattr.ia_mode |= S_IFDIR;
> +		break;
> +	case KERNFS_FILE:
> +		iattr.ia_mode |= S_IFREG;
> +		break;
> +	case KERNFS_LINK:
> +		iattr.ia_mode |= S_IFLNK;
> +		break;
> +	}
> +
> +	ret = kernfs_setattr(kn, &iattr);
> +	kernfs_put(kn);
> +	return ret;
> +}
> +
> +static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
> +				      unsigned long fflags)
> +{
> +	struct kernfs_node *kn_subdir;
> +	int ret;
> +
> +	kn_subdir = kernfs_create_dir(kn_info, name,
> +				      kn_info->mode, priv);
> +	if (IS_ERR(kn_subdir))
> +		return PTR_ERR(kn_subdir);
> +
> +	ret = rdtgroup_kn_set_ugid(kn_subdir);
> +	if (ret)
> +		return ret;
> +
> +	ret = rdtgroup_add_files(kn_subdir, fflags);
> +	if (!ret)
> +		kernfs_activate(kn_subdir);
> +
> +	return ret;
> +}
> +
> +static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
> +{
> +	enum resctrl_res_level i;
> +	struct resctrl_schema *s;
> +	struct rdt_resource *r;
> +	unsigned long fflags;
> +	char name[32];
> +	int ret;
> +
> +	/* create the directory */
> +	kn_info = kernfs_create_dir(parent_kn, "info", parent_kn->mode, NULL);
> +	if (IS_ERR(kn_info))
> +		return PTR_ERR(kn_info);
> +
> +	ret = rdtgroup_add_files(kn_info, RFTYPE_TOP_INFO);
> +	if (ret)
> +		goto out_destroy;
> +
> +	/* loop over enabled controls, these are all alloc_capable */
> +	list_for_each_entry(s, &resctrl_schema_all, list) {
> +		r = s->res;
> +		fflags = r->fflags | RFTYPE_CTRL_INFO;
> +		ret = rdtgroup_mkdir_info_resdir(s, s->name, fflags);
> +		if (ret)
> +			goto out_destroy;
> +	}
> +
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->mon_capable)
> +			continue;
> +
> +		fflags =  r->fflags | RFTYPE_MON_INFO;
> +		sprintf(name, "%s_MON", r->name);
> +		ret = rdtgroup_mkdir_info_resdir(r, name, fflags);
> +		if (ret)
> +			goto out_destroy;
> +	}
> +
> +	ret = rdtgroup_kn_set_ugid(kn_info);
> +	if (ret)
> +		goto out_destroy;
> +
> +	kernfs_activate(kn_info);
> +
> +	return 0;
> +
> +out_destroy:
> +	kernfs_remove(kn_info);
> +	return ret;
> +}
> +
> +static int
> +mongroup_create_dir(struct kernfs_node *parent_kn, struct rdtgroup *prgrp,
> +		    char *name, struct kernfs_node **dest_kn)
> +{
> +	struct kernfs_node *kn;
> +	int ret;
> +
> +	/* create the directory */
> +	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
> +	if (IS_ERR(kn))
> +		return PTR_ERR(kn);
> +
> +	if (dest_kn)
> +		*dest_kn = kn;
> +
> +	ret = rdtgroup_kn_set_ugid(kn);
> +	if (ret)
> +		goto out_destroy;
> +
> +	kernfs_activate(kn);
> +
> +	return 0;
> +
> +out_destroy:
> +	kernfs_remove(kn);
> +	return ret;
> +}
> +
> +static inline bool is_mba_linear(void)
> +{
> +	return resctrl_arch_get_resource(RDT_RESOURCE_MBA)->membw.delay_linear;
> +}
> +
> +static int mba_sc_domain_allocate(struct rdt_resource *r, struct rdt_domain *d)
> +{
> +	u32 num_closid = resctrl_arch_get_num_closid(r);
> +	int cpu = cpumask_any(&d->cpu_mask);
> +	int i;
> +
> +	d->mbps_val = kcalloc_node(num_closid, sizeof(*d->mbps_val),
> +				   GFP_KERNEL, cpu_to_node(cpu));
> +	if (!d->mbps_val)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < num_closid; i++)
> +		d->mbps_val[i] = MBA_MAX_MBPS;
> +
> +	return 0;
> +}
> +
> +static void mba_sc_domain_destroy(struct rdt_resource *r,
> +				  struct rdt_domain *d)
> +{
> +	kfree(d->mbps_val);
> +	d->mbps_val = NULL;
> +}
> +
> +/*
> + * MBA software controller is supported only if
> + * MBM is supported and MBA is in linear scale.
> + */
> +static bool supports_mba_mbps(void)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> +
> +	return (resctrl_arch_is_mbm_local_enabled() &&
> +		r->alloc_capable && is_mba_linear());
> +}
> +
> +/*
> + * Enable or disable the MBA software controller
> + * which helps user specify bandwidth in MBps.
> + */
> +static int set_mba_sc(bool mba_sc)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> +	u32 num_closid = resctrl_arch_get_num_closid(r);
> +	struct rdt_domain *d;
> +	int i;
> +
> +	if (!supports_mba_mbps() || mba_sc == is_mba_sc(r))
> +		return -EINVAL;
> +
> +	r->membw.mba_sc = mba_sc;
> +
> +	list_for_each_entry(d, &r->domains, list) {
> +		for (i = 0; i < num_closid; i++)
> +			d->mbps_val[i] = MBA_MAX_MBPS;
> +	}
> +
> +	return 0;
> +}
> +
> +/*
> + * We don't allow rdtgroup directories to be created anywhere
> + * except the root directory. Thus when looking for the rdtgroup
> + * structure for a kernfs node we are either looking at a directory,
> + * in which case the rdtgroup structure is pointed at by the "priv"
> + * field, otherwise we have a file, and need only look to the parent
> + * to find the rdtgroup.
> + */
> +static struct rdtgroup *kernfs_to_rdtgroup(struct kernfs_node *kn)
> +{
> +	if (kernfs_type(kn) == KERNFS_DIR) {
> +		/*
> +		 * All the resource directories use "kn->priv"
> +		 * to point to the "struct rdtgroup" for the
> +		 * resource. "info" and its subdirectories don't
> +		 * have rdtgroup structures, so return NULL here.
> +		 */
> +		if (kn == kn_info || kn->parent == kn_info)
> +			return NULL;
> +		else
> +			return kn->priv;
> +	} else {
> +		return kn->parent->priv;
> +	}
> +}
> +
> +static void rdtgroup_kn_get(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
> +{
> +	atomic_inc(&rdtgrp->waitcount);
> +	kernfs_break_active_protection(kn);
> +}
> +
> +static void rdtgroup_kn_put(struct rdtgroup *rdtgrp, struct kernfs_node *kn)
> +{
> +	if (atomic_dec_and_test(&rdtgrp->waitcount) &&
> +	    (rdtgrp->flags & RDT_DELETED)) {
> +		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> +		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
> +			rdtgroup_pseudo_lock_remove(rdtgrp);
> +		kernfs_unbreak_active_protection(kn);
> +		rdtgroup_remove(rdtgrp);
> +	} else {
> +		kernfs_unbreak_active_protection(kn);
> +	}
> +}
> +
> +struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn)
> +{
> +	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
> +
> +	if (!rdtgrp)
> +		return NULL;
> +
> +	rdtgroup_kn_get(rdtgrp, kn);
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	/* Was this group deleted while we waited? */
> +	if (rdtgrp->flags & RDT_DELETED)
> +		return NULL;
> +
> +	return rdtgrp;
> +}
> +
> +void rdtgroup_kn_unlock(struct kernfs_node *kn)
> +{
> +	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
> +
> +	if (!rdtgrp)
> +		return;
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +
> +	rdtgroup_kn_put(rdtgrp, kn);
> +}
> +
> +static int mkdir_mondata_all(struct kernfs_node *parent_kn,
> +			     struct rdtgroup *prgrp,
> +			     struct kernfs_node **mon_data_kn);
> +
> +static void rdt_disable_ctx(void)
> +{
> +	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
> +	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
> +	set_mba_sc(false);
> +
> +	resctrl_debug = false;
> +}
> +
> +static int rdt_enable_ctx(struct rdt_fs_context *ctx)
> +{
> +	int ret = 0;
> +
> +	if (ctx->enable_cdpl2) {
> +		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, true);
> +		if (ret)
> +			goto out_done;
> +	}
> +
> +	if (ctx->enable_cdpl3) {
> +		ret = resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, true);
> +		if (ret)
> +			goto out_cdpl2;
> +	}
> +
> +	if (ctx->enable_mba_mbps) {
> +		ret = set_mba_sc(true);
> +		if (ret)
> +			goto out_cdpl3;
> +	}
> +
> +	if (ctx->enable_debug)
> +		resctrl_debug = true;
> +
> +	return 0;
> +
> +out_cdpl3:
> +	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
> +out_cdpl2:
> +	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
> +out_done:
> +	return ret;
> +}
> +
> +static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type)
> +{
> +	struct resctrl_schema *s;
> +	const char *suffix = "";
> +	int ret, cl;
> +
> +	s = kzalloc(sizeof(*s), GFP_KERNEL);
> +	if (!s)
> +		return -ENOMEM;
> +
> +	s->res = r;
> +	s->num_closid = resctrl_arch_get_num_closid(r);
> +	if (resctrl_arch_get_cdp_enabled(r->rid))
> +		s->num_closid /= 2;
> +
> +	s->conf_type = type;
> +	switch (type) {
> +	case CDP_CODE:
> +		suffix = "CODE";
> +		break;
> +	case CDP_DATA:
> +		suffix = "DATA";
> +		break;
> +	case CDP_NONE:
> +		suffix = "";
> +		break;
> +	}
> +
> +	ret = snprintf(s->name, sizeof(s->name), "%s%s", r->name, suffix);
> +	if (ret >= sizeof(s->name)) {
> +		kfree(s);
> +		return -EINVAL;
> +	}
> +
> +	cl = strlen(s->name);
> +
> +	/*
> +	 * If CDP is supported by this resource, but not enabled,
> +	 * include the suffix. This ensures the tabular format of the
> +	 * schemata file does not change between mounts of the filesystem.
> +	 */
> +	if (r->cdp_capable && !resctrl_arch_get_cdp_enabled(r->rid))
> +		cl += 4;
> +
> +	if (cl > max_name_width)
> +		max_name_width = cl;
> +
> +	/*
> +	 * Choose a width for the resource data based on the resource that has
> +	 * widest name and cbm.
> +	 */
> +	max_data_width = max(max_data_width, r->data_width);
> +
> +	INIT_LIST_HEAD(&s->list);
> +	list_add(&s->list, &resctrl_schema_all);
> +
> +	return 0;
> +}
> +
> +static int schemata_list_create(void)
> +{
> +	enum resctrl_res_level i;
> +	struct rdt_resource *r;
> +	int ret = 0;
> +
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
> +		if (resctrl_arch_get_cdp_enabled(r->rid)) {
> +			ret = schemata_list_add(r, CDP_CODE);
> +			if (ret)
> +				break;
> +
> +			ret = schemata_list_add(r, CDP_DATA);
> +		} else {
> +			ret = schemata_list_add(r, CDP_NONE);
> +		}
> +
> +		if (ret)
> +			break;
> +	}
> +
> +	return ret;
> +}
> +
> +static void schemata_list_destroy(void)
> +{
> +	struct resctrl_schema *s, *tmp;
> +
> +	list_for_each_entry_safe(s, tmp, &resctrl_schema_all, list) {
> +		list_del(&s->list);
> +		kfree(s);
> +	}
> +}
> +
> +static int rdt_get_tree(struct fs_context *fc)
> +{
> +	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> +	unsigned long flags = RFTYPE_CTRL_BASE;
> +	struct rdt_domain *dom;
> +	int ret;
> +
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +	/*
> +	 * resctrl file system can only be mounted once.
> +	 */
> +	if (resctrl_mounted) {
> +		ret = -EBUSY;
> +		goto out;
> +	}
> +
> +	ret = rdtgroup_setup_root(ctx);
> +	if (ret)
> +		goto out;
> +
> +	ret = rdt_enable_ctx(ctx);
> +	if (ret)
> +		goto out_root;
> +
> +	ret = schemata_list_create();
> +	if (ret) {
> +		schemata_list_destroy();
> +		goto out_ctx;
> +	}
> +
> +	closid_init();
> +
> +	if (resctrl_arch_mon_capable())
> +		flags |= RFTYPE_MON;
> +
> +	ret = rdtgroup_add_files(rdtgroup_default.kn, flags);
> +	if (ret)
> +		goto out_schemata_free;
> +
> +	kernfs_activate(rdtgroup_default.kn);
> +
> +	ret = rdtgroup_create_info_dir(rdtgroup_default.kn);
> +	if (ret < 0)
> +		goto out_schemata_free;
> +
> +	if (resctrl_arch_mon_capable()) {
> +		ret = mongroup_create_dir(rdtgroup_default.kn,
> +					  &rdtgroup_default, "mon_groups",
> +					  &kn_mongrp);
> +		if (ret < 0)
> +			goto out_info;
> +
> +		ret = mkdir_mondata_all(rdtgroup_default.kn,
> +					&rdtgroup_default, &kn_mondata);
> +		if (ret < 0)
> +			goto out_mongrp;
> +		rdtgroup_default.mon.mon_data_kn = kn_mondata;
> +	}
> +
> +	ret = rdt_pseudo_lock_init();
> +	if (ret)
> +		goto out_mondata;
> +
> +	ret = kernfs_get_tree(fc);
> +	if (ret < 0)
> +		goto out_psl;
> +
> +	if (resctrl_arch_alloc_capable())
> +		resctrl_arch_enable_alloc();
> +	if (resctrl_arch_mon_capable())
> +		resctrl_arch_enable_mon();
> +
> +	if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable())
> +		resctrl_mounted = true;
> +
> +	if (resctrl_is_mbm_enabled()) {
> +		list_for_each_entry(dom, &l3->domains, list)
> +			mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL,
> +						   RESCTRL_PICK_ANY_CPU);
> +	}
> +
> +	goto out;
> +
> +out_psl:
> +	rdt_pseudo_lock_release();
> +out_mondata:
> +	if (resctrl_arch_mon_capable())
> +		kernfs_remove(kn_mondata);
> +out_mongrp:
> +	if (resctrl_arch_mon_capable())
> +		kernfs_remove(kn_mongrp);
> +out_info:
> +	kernfs_remove(kn_info);
> +out_schemata_free:
> +	schemata_list_destroy();
> +out_ctx:
> +	rdt_disable_ctx();
> +out_root:
> +	rdtgroup_destroy_root();
> +out:
> +	rdt_last_cmd_clear();
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +	return ret;
> +}
> +
> +enum rdt_param {
> +	Opt_cdp,
> +	Opt_cdpl2,
> +	Opt_mba_mbps,
> +	Opt_debug,
> +	nr__rdt_params
> +};
> +
> +static const struct fs_parameter_spec rdt_fs_parameters[] = {
> +	fsparam_flag("cdp",		Opt_cdp),
> +	fsparam_flag("cdpl2",		Opt_cdpl2),
> +	fsparam_flag("mba_MBps",	Opt_mba_mbps),
> +	fsparam_flag("debug",		Opt_debug),
> +	{}
> +};
> +
> +static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
> +{
> +	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> +	struct fs_parse_result result;
> +	int opt;
> +
> +	opt = fs_parse(fc, rdt_fs_parameters, param, &result);
> +	if (opt < 0)
> +		return opt;
> +
> +	switch (opt) {
> +	case Opt_cdp:
> +		ctx->enable_cdpl3 = true;
> +		return 0;
> +	case Opt_cdpl2:
> +		ctx->enable_cdpl2 = true;
> +		return 0;
> +	case Opt_mba_mbps:
> +		if (!supports_mba_mbps())
> +			return -EINVAL;
> +		ctx->enable_mba_mbps = true;
> +		return 0;
> +	case Opt_debug:
> +		ctx->enable_debug = true;
> +		return 0;
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static void rdt_fs_context_free(struct fs_context *fc)
> +{
> +	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> +
> +	kernfs_free_fs_context(fc);
> +	kfree(ctx);
> +}
> +
> +static const struct fs_context_operations rdt_fs_context_ops = {
> +	.free		= rdt_fs_context_free,
> +	.parse_param	= rdt_parse_param,
> +	.get_tree	= rdt_get_tree,
> +};
> +
> +static int rdt_init_fs_context(struct fs_context *fc)
> +{
> +	struct rdt_fs_context *ctx;
> +
> +	ctx = kzalloc(sizeof(struct rdt_fs_context), GFP_KERNEL);
> +	if (!ctx)
> +		return -ENOMEM;
> +
> +	ctx->kfc.magic = RDTGROUP_SUPER_MAGIC;
> +	fc->fs_private = &ctx->kfc;
> +	fc->ops = &rdt_fs_context_ops;
> +	put_user_ns(fc->user_ns);
> +	fc->user_ns = get_user_ns(&init_user_ns);
> +	fc->global = true;
> +	return 0;
> +}
> +
> +/*
> + * Move tasks from one to the other group. If @from is NULL, then all tasks
> + * in the systems are moved unconditionally (used for teardown).
> + *
> + * If @mask is not NULL the cpus on which moved tasks are running are set
> + * in that mask so the update smp function call is restricted to affected
> + * cpus.
> + */
> +static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,
> +				 struct cpumask *mask)
> +{
> +	struct task_struct *p, *t;
> +
> +	read_lock(&tasklist_lock);
> +	for_each_process_thread(p, t) {
> +		if (!from || is_closid_match(t, from) ||
> +		    is_rmid_match(t, from)) {
> +			resctrl_arch_set_closid_rmid(t, to->closid,
> +						     to->mon.rmid);
> +
> +			/*
> +			 * Order the closid/rmid stores above before the loads
> +			 * in task_curr(). This pairs with the full barrier
> +			 * between the rq->curr update and
> +			 * resctrl_arch_sched_in() during context switch.
> +			 */
> +			smp_mb();
> +
> +			/*
> +			 * If the task is on a CPU, set the CPU in the mask.
> +			 * The detection is inaccurate as tasks might move or
> +			 * schedule before the smp function call takes place.
> +			 * In such a case the function call is pointless, but
> +			 * there is no other side effect.
> +			 */
> +			if (IS_ENABLED(CONFIG_SMP) && mask && task_curr(t))
> +				cpumask_set_cpu(task_cpu(t), mask);
> +		}
> +	}
> +	read_unlock(&tasklist_lock);
> +}
> +
> +static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp)
> +{
> +	struct rdtgroup *sentry, *stmp;
> +	struct list_head *head;
> +
> +	head = &rdtgrp->mon.crdtgrp_list;
> +	list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) {
> +		free_rmid(sentry->closid, sentry->mon.rmid);
> +		list_del(&sentry->mon.crdtgrp_list);
> +
> +		if (atomic_read(&sentry->waitcount) != 0)
> +			sentry->flags = RDT_DELETED;
> +		else
> +			rdtgroup_remove(sentry);
> +	}
> +}
> +
> +/*
> + * Forcibly remove all of subdirectories under root.
> + */
> +static void rmdir_all_sub(void)
> +{
> +	struct rdtgroup *rdtgrp, *tmp;
> +
> +	/* Move all tasks to the default resource group */
> +	rdt_move_group_tasks(NULL, &rdtgroup_default, NULL);
> +
> +	list_for_each_entry_safe(rdtgrp, tmp, &rdt_all_groups, rdtgroup_list) {
> +		/* Free any child rmids */
> +		free_all_child_rdtgrp(rdtgrp);
> +
> +		/* Remove each rdtgroup other than root */
> +		if (rdtgrp == &rdtgroup_default)
> +			continue;
> +
> +		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> +		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
> +			rdtgroup_pseudo_lock_remove(rdtgrp);
> +
> +		/*
> +		 * Give any CPUs back to the default group. We cannot copy
> +		 * cpu_online_mask because a CPU might have executed the
> +		 * offline callback already, but is still marked online.
> +		 */
> +		cpumask_or(&rdtgroup_default.cpu_mask,
> +			   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
> +
> +		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> +
> +		kernfs_remove(rdtgrp->kn);
> +		list_del(&rdtgrp->rdtgroup_list);
> +
> +		if (atomic_read(&rdtgrp->waitcount) != 0)
> +			rdtgrp->flags = RDT_DELETED;
> +		else
> +			rdtgroup_remove(rdtgrp);
> +	}
> +	/* Notify online CPUs to update per cpu storage and PQR_ASSOC MSR */
> +	update_closid_rmid(cpu_online_mask, &rdtgroup_default);
> +
> +	kernfs_remove(kn_info);
> +	kernfs_remove(kn_mongrp);
> +	kernfs_remove(kn_mondata);
> +}
> +
> +static void rdt_kill_sb(struct super_block *sb)
> +{
> +	cpus_read_lock();
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	rdt_disable_ctx();
> +
> +	/* Put everything back to default values. */
> +	resctrl_arch_reset_resources();
> +
> +	rmdir_all_sub();
> +	rdt_pseudo_lock_release();
> +	rdtgroup_default.mode = RDT_MODE_SHAREABLE;
> +	schemata_list_destroy();
> +	rdtgroup_destroy_root();
> +	if (resctrl_arch_alloc_capable())
> +		resctrl_arch_disable_alloc();
> +	if (resctrl_arch_mon_capable())
> +		resctrl_arch_disable_mon();
> +	resctrl_mounted = false;
> +	kernfs_kill_sb(sb);
> +	mutex_unlock(&rdtgroup_mutex);
> +	cpus_read_unlock();
> +}
> +
> +static struct file_system_type rdt_fs_type = {
> +	.name			= "resctrl",
> +	.init_fs_context	= rdt_init_fs_context,
> +	.parameters		= rdt_fs_parameters,
> +	.kill_sb		= rdt_kill_sb,
> +};
> +
> +static int mon_addfile(struct kernfs_node *parent_kn, const char *name,
> +		       void *priv)
> +{
> +	struct kernfs_node *kn;
> +	int ret = 0;
> +
> +	kn = __kernfs_create_file(parent_kn, name, 0444,
> +				  GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, 0,
> +				  &kf_mondata_ops, priv, NULL, NULL);
> +	if (IS_ERR(kn))
> +		return PTR_ERR(kn);
> +
> +	ret = rdtgroup_kn_set_ugid(kn);
> +	if (ret) {
> +		kernfs_remove(kn);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +/*
> + * Remove all subdirectories of mon_data of ctrl_mon groups
> + * and monitor groups with given domain id.
> + */
> +static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
> +					   unsigned int dom_id)
> +{
> +	struct rdtgroup *prgrp, *crgrp;
> +	char name[32];
> +
> +	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> +		sprintf(name, "mon_%s_%02d", r->name, dom_id);
> +		kernfs_remove_by_name(prgrp->mon.mon_data_kn, name);
> +
> +		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
> +			kernfs_remove_by_name(crgrp->mon.mon_data_kn, name);
> +	}
> +}
> +
> +static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,
> +				struct rdt_domain *d,
> +				struct rdt_resource *r, struct rdtgroup *prgrp)
> +{
> +	union mon_data_bits priv;
> +	struct kernfs_node *kn;
> +	struct mon_evt *mevt;
> +	struct rmid_read rr;
> +	char name[32];
> +	int ret;
> +
> +	sprintf(name, "mon_%s_%02d", r->name, d->id);
> +	/* create the directory */
> +	kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
> +	if (IS_ERR(kn))
> +		return PTR_ERR(kn);
> +
> +	ret = rdtgroup_kn_set_ugid(kn);
> +	if (ret)
> +		goto out_destroy;
> +
> +	if (WARN_ON(list_empty(&r->evt_list))) {
> +		ret = -EPERM;
> +		goto out_destroy;
> +	}
> +
> +	priv.u.rid = r->rid;
> +	priv.u.domid = d->id;
> +	list_for_each_entry(mevt, &r->evt_list, list) {
> +		priv.u.evtid = mevt->evtid;
> +		ret = mon_addfile(kn, mevt->name, priv.priv);
> +		if (ret)
> +			goto out_destroy;
> +
> +		if (resctrl_is_mbm_event(mevt->evtid))
> +			mon_event_read(&rr, r, d, prgrp, mevt->evtid, true);
> +	}
> +	kernfs_activate(kn);
> +	return 0;
> +
> +out_destroy:
> +	kernfs_remove(kn);
> +	return ret;
> +}
> +
> +/*
> + * Add all subdirectories of mon_data for "ctrl_mon" groups
> + * and "monitor" groups with given domain id.
> + */
> +static void mkdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
> +					   struct rdt_domain *d)
> +{
> +	struct kernfs_node *parent_kn;
> +	struct rdtgroup *prgrp, *crgrp;
> +	struct list_head *head;
> +
> +	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> +		parent_kn = prgrp->mon.mon_data_kn;
> +		mkdir_mondata_subdir(parent_kn, d, r, prgrp);
> +
> +		head = &prgrp->mon.crdtgrp_list;
> +		list_for_each_entry(crgrp, head, mon.crdtgrp_list) {
> +			parent_kn = crgrp->mon.mon_data_kn;
> +			mkdir_mondata_subdir(parent_kn, d, r, crgrp);
> +		}
> +	}
> +}
> +
> +static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn,
> +				       struct rdt_resource *r,
> +				       struct rdtgroup *prgrp)
> +{
> +	struct rdt_domain *dom;
> +	int ret;
> +
> +	/* Walking r->domains, ensure it can't race with cpuhp */
> +	lockdep_assert_cpus_held();
> +
> +	list_for_each_entry(dom, &r->domains, list) {
> +		ret = mkdir_mondata_subdir(parent_kn, dom, r, prgrp);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +/*
> + * This creates a directory mon_data which contains the monitored data.
> + *
> + * mon_data has one directory for each domain which are named
> + * in the format mon_<domain_name>_<domain_id>. For ex: A mon_data
> + * with L3 domain looks as below:
> + * ./mon_data:
> + * mon_L3_00
> + * mon_L3_01
> + * mon_L3_02
> + * ...
> + *
> + * Each domain directory has one file per event:
> + * ./mon_L3_00/:
> + * llc_occupancy
> + *
> + */
> +static int mkdir_mondata_all(struct kernfs_node *parent_kn,
> +			     struct rdtgroup *prgrp,
> +			     struct kernfs_node **dest_kn)
> +{
> +	enum resctrl_res_level i;
> +	struct rdt_resource *r;
> +	struct kernfs_node *kn;
> +	int ret;
> +
> +	/*
> +	 * Create the mon_data directory first.
> +	 */
> +	ret = mongroup_create_dir(parent_kn, prgrp, "mon_data", &kn);
> +	if (ret)
> +		return ret;
> +
> +	if (dest_kn)
> +		*dest_kn = kn;
> +
> +	/*
> +	 * Create the subdirectories for each domain. Note that all events
> +	 * in a domain like L3 are grouped into a resource whose domain is L3
> +	 */
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->mon_capable)
> +			continue;
> +
> +		ret = mkdir_mondata_subdir_alldom(kn, r, prgrp);
> +		if (ret)
> +			goto out_destroy;
> +	}
> +
> +	return 0;
> +
> +out_destroy:
> +	kernfs_remove(kn);
> +	return ret;
> +}
> +
> +/**
> + * cbm_ensure_valid - Enforce validity on provided CBM
> + * @_val:	Candidate CBM
> + * @r:		RDT resource to which the CBM belongs
> + *
> + * The provided CBM represents all cache portions available for use. This
> + * may be represented by a bitmap that does not consist of contiguous ones
> + * and thus be an invalid CBM.
> + * Here the provided CBM is forced to be a valid CBM by only considering
> + * the first set of contiguous bits as valid and clearing all bits.
> + * The intention here is to provide a valid default CBM with which a new
> + * resource group is initialized. The user can follow this with a
> + * modification to the CBM if the default does not satisfy the
> + * requirements.
> + */
> +static u32 cbm_ensure_valid(u32 _val, struct rdt_resource *r)
> +{
> +	unsigned int cbm_len = r->cache.cbm_len;
> +	unsigned long first_bit, zero_bit;
> +	unsigned long val = _val;
> +
> +	if (!val)
> +		return 0;
> +
> +	first_bit = find_first_bit(&val, cbm_len);
> +	zero_bit = find_next_zero_bit(&val, cbm_len, first_bit);
> +
> +	/* Clear any remaining bits to ensure contiguous region */
> +	bitmap_clear(&val, zero_bit, cbm_len - zero_bit);
> +	return (u32)val;
> +}
> +
> +/*
> + * Initialize cache resources per RDT domain
> + *
> + * Set the RDT domain up to start off with all usable allocations. That is,
> + * all shareable and unused bits. All-zero CBM is invalid.
> + */
> +static int __init_one_rdt_domain(struct rdt_domain *d, struct resctrl_schema *s,
> +				 u32 closid)
> +{
> +	enum resctrl_conf_type peer_type = resctrl_peer_type(s->conf_type);
> +	enum resctrl_conf_type t = s->conf_type;
> +	struct resctrl_staged_config *cfg;
> +	struct rdt_resource *r = s->res;
> +	u32 used_b = 0, unused_b = 0;
> +	unsigned long tmp_cbm;
> +	enum rdtgrp_mode mode;
> +	u32 peer_ctl, ctrl_val;
> +	int i;
> +
> +	cfg = &d->staged_config[t];
> +	cfg->have_new_ctrl = false;
> +	cfg->new_ctrl = r->cache.shareable_bits;
> +	used_b = r->cache.shareable_bits;
> +	for (i = 0; i < closids_supported(); i++) {
> +		if (closid_allocated(i) && i != closid) {
> +			mode = rdtgroup_mode_by_closid(i);
> +			if (mode == RDT_MODE_PSEUDO_LOCKSETUP)
> +				/*
> +				 * ctrl values for locksetup aren't relevant
> +				 * until the schemata is written, and the mode
> +				 * becomes RDT_MODE_PSEUDO_LOCKED.
> +				 */
> +				continue;
> +			/*
> +			 * If CDP is active include peer domain's
> +			 * usage to ensure there is no overlap
> +			 * with an exclusive group.
> +			 */
> +			if (resctrl_arch_get_cdp_enabled(r->rid))
> +				peer_ctl = resctrl_arch_get_config(r, d, i,
> +								   peer_type);
> +			else
> +				peer_ctl = 0;
> +			ctrl_val = resctrl_arch_get_config(r, d, i,
> +							   s->conf_type);
> +			used_b |= ctrl_val | peer_ctl;
> +			if (mode == RDT_MODE_SHAREABLE)
> +				cfg->new_ctrl |= ctrl_val | peer_ctl;
> +		}
> +	}
> +	if (d->plr && d->plr->cbm > 0)
> +		used_b |= d->plr->cbm;
> +	unused_b = used_b ^ (BIT_MASK(r->cache.cbm_len) - 1);
> +	unused_b &= BIT_MASK(r->cache.cbm_len) - 1;
> +	cfg->new_ctrl |= unused_b;
> +	/*
> +	 * Force the initial CBM to be valid, user can
> +	 * modify the CBM based on system availability.
> +	 */
> +	cfg->new_ctrl = cbm_ensure_valid(cfg->new_ctrl, r);
> +	/*
> +	 * Assign the u32 CBM to an unsigned long to ensure that
> +	 * bitmap_weight() does not access out-of-bound memory.
> +	 */
> +	tmp_cbm = cfg->new_ctrl;
> +	if (bitmap_weight(&tmp_cbm, r->cache.cbm_len) < r->cache.min_cbm_bits) {
> +		rdt_last_cmd_printf("No space on %s:%d\n", s->name, d->id);
> +		return -ENOSPC;
> +	}
> +	cfg->have_new_ctrl = true;
> +
> +	return 0;
> +}
> +
> +/*
> + * Initialize cache resources with default values.
> + *
> + * A new RDT group is being created on an allocation capable (CAT)
> + * supporting system. Set this group up to start off with all usable
> + * allocations.
> + *
> + * If there are no more shareable bits available on any domain then
> + * the entire allocation will fail.
> + */
> +static int rdtgroup_init_cat(struct resctrl_schema *s, u32 closid)
> +{
> +	struct rdt_domain *d;
> +	int ret;
> +
> +	list_for_each_entry(d, &s->res->domains, list) {
> +		ret = __init_one_rdt_domain(d, s, closid);
> +		if (ret < 0)
> +			return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +/* Initialize MBA resource with default values. */
> +static void rdtgroup_init_mba(struct rdt_resource *r, u32 closid)
> +{
> +	struct resctrl_staged_config *cfg;
> +	struct rdt_domain *d;
> +
> +	list_for_each_entry(d, &r->domains, list) {
> +		if (is_mba_sc(r)) {
> +			d->mbps_val[closid] = MBA_MAX_MBPS;
> +			continue;
> +		}
> +
> +		cfg = &d->staged_config[CDP_NONE];
> +		cfg->new_ctrl = r->default_ctrl;
> +		cfg->have_new_ctrl = true;
> +	}
> +}
> +
> +/* Initialize the RDT group's allocations. */
> +static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
> +{
> +	struct resctrl_schema *s;
> +	struct rdt_resource *r;
> +	int ret = 0;
> +
> +	rdt_staged_configs_clear();
> +
> +	list_for_each_entry(s, &resctrl_schema_all, list) {
> +		r = s->res;
> +		if (r->rid == RDT_RESOURCE_MBA ||
> +		    r->rid == RDT_RESOURCE_SMBA) {
> +			rdtgroup_init_mba(r, rdtgrp->closid);
> +			if (is_mba_sc(r))
> +				continue;
> +		} else {
> +			ret = rdtgroup_init_cat(s, rdtgrp->closid);
> +			if (ret < 0)
> +				goto out;
> +		}
> +
> +		ret = resctrl_arch_update_domains(r, rdtgrp->closid);
> +		if (ret < 0) {
> +			rdt_last_cmd_puts("Failed to initialize allocations\n");
> +			goto out;
> +		}
> +
> +	}
> +
> +	rdtgrp->mode = RDT_MODE_SHAREABLE;
> +
> +out:
> +	rdt_staged_configs_clear();
> +	return ret;
> +}
> +
> +static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)
> +{
> +	int ret;
> +
> +	if (!resctrl_arch_mon_capable())
> +		return 0;
> +
> +	ret = alloc_rmid(rdtgrp->closid);
> +	if (ret < 0) {
> +		rdt_last_cmd_puts("Out of RMIDs\n");
> +		return ret;
> +	}
> +	rdtgrp->mon.rmid = ret;
> +
> +	ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn);
> +	if (ret) {
> +		rdt_last_cmd_puts("kernfs subdir error\n");
> +		free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp)
> +{
> +	if (resctrl_arch_mon_capable())
> +		free_rmid(rgrp->closid, rgrp->mon.rmid);
> +}
> +
> +static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,
> +			     const char *name, umode_t mode,
> +			     enum rdt_group_type rtype, struct rdtgroup **r)
> +{
> +	struct rdtgroup *prdtgrp, *rdtgrp;
> +	unsigned long files = 0;
> +	struct kernfs_node *kn;
> +	int ret;
> +
> +	prdtgrp = rdtgroup_kn_lock_live(parent_kn);
> +	if (!prdtgrp) {
> +		ret = -ENODEV;
> +		goto out_unlock;
> +	}
> +
> +	if (rtype == RDTMON_GROUP &&
> +	    (prdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> +	     prdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)) {
> +		ret = -EINVAL;
> +		rdt_last_cmd_puts("Pseudo-locking in progress\n");
> +		goto out_unlock;
> +	}
> +
> +	/* allocate the rdtgroup. */
> +	rdtgrp = kzalloc(sizeof(*rdtgrp), GFP_KERNEL);
> +	if (!rdtgrp) {
> +		ret = -ENOSPC;
> +		rdt_last_cmd_puts("Kernel out of memory\n");
> +		goto out_unlock;
> +	}
> +	*r = rdtgrp;
> +	rdtgrp->mon.parent = prdtgrp;
> +	rdtgrp->type = rtype;
> +	INIT_LIST_HEAD(&rdtgrp->mon.crdtgrp_list);
> +
> +	/* kernfs creates the directory for rdtgrp */
> +	kn = kernfs_create_dir(parent_kn, name, mode, rdtgrp);
> +	if (IS_ERR(kn)) {
> +		ret = PTR_ERR(kn);
> +		rdt_last_cmd_puts("kernfs create error\n");
> +		goto out_free_rgrp;
> +	}
> +	rdtgrp->kn = kn;
> +
> +	/*
> +	 * kernfs_remove() will drop the reference count on "kn" which
> +	 * will free it. But we still need it to stick around for the
> +	 * rdtgroup_kn_unlock(kn) call. Take one extra reference here,
> +	 * which will be dropped by kernfs_put() in rdtgroup_remove().
> +	 */
> +	kernfs_get(kn);
> +
> +	ret = rdtgroup_kn_set_ugid(kn);
> +	if (ret) {
> +		rdt_last_cmd_puts("kernfs perm error\n");
> +		goto out_destroy;
> +	}
> +
> +	if (rtype == RDTCTRL_GROUP) {
> +		files = RFTYPE_BASE | RFTYPE_CTRL;
> +		if (resctrl_arch_mon_capable())
> +			files |= RFTYPE_MON;
> +	} else {
> +		files = RFTYPE_BASE | RFTYPE_MON;
> +	}
> +
> +	ret = rdtgroup_add_files(kn, files);
> +	if (ret) {
> +		rdt_last_cmd_puts("kernfs fill error\n");
> +		goto out_destroy;
> +	}
> +
> +	/*
> +	 * The caller unlocks the parent_kn upon success.
> +	 */
> +	return 0;
> +
> +out_destroy:
> +	kernfs_put(rdtgrp->kn);
> +	kernfs_remove(rdtgrp->kn);
> +out_free_rgrp:
> +	kfree(rdtgrp);
> +out_unlock:
> +	rdtgroup_kn_unlock(parent_kn);
> +	return ret;
> +}
> +
> +static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp)
> +{
> +	kernfs_remove(rgrp->kn);
> +	rdtgroup_remove(rgrp);
> +}
> +
> +/*
> + * Create a monitor group under "mon_groups" directory of a control
> + * and monitor group(ctrl_mon). This is a resource group
> + * to monitor a subset of tasks and cpus in its parent ctrl_mon group.
> + */
> +static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn,
> +			      const char *name, umode_t mode)
> +{
> +	struct rdtgroup *rdtgrp, *prgrp;
> +	int ret;
> +
> +	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTMON_GROUP, &rdtgrp);
> +	if (ret)
> +		return ret;
> +
> +	prgrp = rdtgrp->mon.parent;
> +	rdtgrp->closid = prgrp->closid;
> +
> +	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
> +	if (ret) {
> +		mkdir_rdt_prepare_clean(rdtgrp);
> +		goto out_unlock;
> +	}
> +
> +	kernfs_activate(rdtgrp->kn);
> +
> +	/*
> +	 * Add the rdtgrp to the list of rdtgrps the parent
> +	 * ctrl_mon group has to track.
> +	 */
> +	list_add_tail(&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list);
> +
> +out_unlock:
> +	rdtgroup_kn_unlock(parent_kn);
> +	return ret;
> +}
> +
> +/*
> + * These are rdtgroups created under the root directory. Can be used
> + * to allocate and monitor resources.
> + */
> +static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,
> +				   const char *name, umode_t mode)
> +{
> +	struct rdtgroup *rdtgrp;
> +	struct kernfs_node *kn;
> +	u32 closid;
> +	int ret;
> +
> +	ret = mkdir_rdt_prepare(parent_kn, name, mode, RDTCTRL_GROUP, &rdtgrp);
> +	if (ret)
> +		return ret;
> +
> +	kn = rdtgrp->kn;
> +	ret = closid_alloc();
> +	if (ret < 0) {
> +		rdt_last_cmd_puts("Out of CLOSIDs\n");
> +		goto out_common_fail;
> +	}
> +	closid = ret;
> +	ret = 0;
> +
> +	rdtgrp->closid = closid;
> +
> +	ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp);
> +	if (ret)
> +		goto out_closid_free;
> +
> +	kernfs_activate(rdtgrp->kn);
> +
> +	ret = rdtgroup_init_alloc(rdtgrp);
> +	if (ret < 0)
> +		goto out_rmid_free;
> +
> +	list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups);
> +
> +	if (resctrl_arch_mon_capable()) {
> +		/*
> +		 * Create an empty mon_groups directory to hold the subset
> +		 * of tasks and cpus to monitor.
> +		 */
> +		ret = mongroup_create_dir(kn, rdtgrp, "mon_groups", NULL);
> +		if (ret) {
> +			rdt_last_cmd_puts("kernfs subdir error\n");
> +			goto out_del_list;
> +		}
> +	}
> +
> +	goto out_unlock;
> +
> +out_del_list:
> +	list_del(&rdtgrp->rdtgroup_list);
> +out_rmid_free:
> +	mkdir_rdt_prepare_rmid_free(rdtgrp);
> +out_closid_free:
> +	closid_free(closid);
> +out_common_fail:
> +	mkdir_rdt_prepare_clean(rdtgrp);
> +out_unlock:
> +	rdtgroup_kn_unlock(parent_kn);
> +	return ret;
> +}
> +
> +/*
> + * We allow creating mon groups only with in a directory called "mon_groups"
> + * which is present in every ctrl_mon group. Check if this is a valid
> + * "mon_groups" directory.
> + *
> + * 1. The directory should be named "mon_groups".
> + * 2. The mon group itself should "not" be named "mon_groups".
> + *   This makes sure "mon_groups" directory always has a ctrl_mon group
> + *   as parent.
> + */
> +static bool is_mon_groups(struct kernfs_node *kn, const char *name)
> +{
> +	return (!strcmp(kn->name, "mon_groups") &&
> +		strcmp(name, "mon_groups"));
> +}
> +
> +static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
> +			  umode_t mode)
> +{
> +	/* Do not accept '\n' to avoid unparsable situation. */
> +	if (strchr(name, '\n'))
> +		return -EINVAL;
> +
> +	/*
> +	 * If the parent directory is the root directory and RDT
> +	 * allocation is supported, add a control and monitoring
> +	 * subdirectory
> +	 */
> +	if (resctrl_arch_alloc_capable() && parent_kn == rdtgroup_default.kn)
> +		return rdtgroup_mkdir_ctrl_mon(parent_kn, name, mode);
> +
> +	/*
> +	 * If RDT monitoring is supported and the parent directory is a valid
> +	 * "mon_groups" directory, add a monitoring subdirectory.
> +	 */
> +	if (resctrl_arch_mon_capable() && is_mon_groups(parent_kn, name))
> +		return rdtgroup_mkdir_mon(parent_kn, name, mode);
> +
> +	return -EPERM;
> +}
> +
> +static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
> +{
> +	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
> +	u32 closid, rmid;
> +	int cpu;
> +
> +	/* Give any tasks back to the parent group */
> +	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
> +
> +	/* Update per cpu rmid of the moved CPUs first */
> +	closid = rdtgrp->closid;
> +	rmid = prdtgrp->mon.rmid;
> +	for_each_cpu(cpu, &rdtgrp->cpu_mask)
> +		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
> +
> +	/*
> +	 * Update the MSR on moved CPUs and CPUs which have moved
> +	 * task running on them.
> +	 */
> +	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
> +	update_closid_rmid(tmpmask, NULL);
> +
> +	rdtgrp->flags = RDT_DELETED;
> +	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> +
> +	/*
> +	 * Remove the rdtgrp from the parent ctrl_mon group's list
> +	 */
> +	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
> +	list_del(&rdtgrp->mon.crdtgrp_list);
> +
> +	kernfs_remove(rdtgrp->kn);
> +
> +	return 0;
> +}
> +
> +static int rdtgroup_ctrl_remove(struct rdtgroup *rdtgrp)
> +{
> +	rdtgrp->flags = RDT_DELETED;
> +	list_del(&rdtgrp->rdtgroup_list);
> +
> +	kernfs_remove(rdtgrp->kn);
> +	return 0;
> +}
> +
> +static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
> +{
> +	u32 closid, rmid;
> +	int cpu;
> +
> +	/* Give any tasks back to the default group */
> +	rdt_move_group_tasks(rdtgrp, &rdtgroup_default, tmpmask);
> +
> +	/* Give any CPUs back to the default group */
> +	cpumask_or(&rdtgroup_default.cpu_mask,
> +		   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
> +
> +	/* Update per cpu closid and rmid of the moved CPUs first */
> +	closid = rdtgroup_default.closid;
> +	rmid = rdtgroup_default.mon.rmid;
> +	for_each_cpu(cpu, &rdtgrp->cpu_mask)
> +		resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid);
> +
> +	/*
> +	 * Update the MSR on moved CPUs and CPUs which have moved
> +	 * task running on them.
> +	 */
> +	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
> +	update_closid_rmid(tmpmask, NULL);
> +
> +	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
> +	closid_free(rdtgrp->closid);
> +
> +	rdtgroup_ctrl_remove(rdtgrp);
> +
> +	/*
> +	 * Free all the child monitor group rmids.
> +	 */
> +	free_all_child_rdtgrp(rdtgrp);
> +
> +	return 0;
> +}
> +
> +static int rdtgroup_rmdir(struct kernfs_node *kn)
> +{
> +	struct kernfs_node *parent_kn = kn->parent;
> +	struct rdtgroup *rdtgrp;
> +	cpumask_var_t tmpmask;
> +	int ret = 0;
> +
> +	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL))
> +		return -ENOMEM;
> +
> +	rdtgrp = rdtgroup_kn_lock_live(kn);
> +	if (!rdtgrp) {
> +		ret = -EPERM;
> +		goto out;
> +	}
> +
> +	/*
> +	 * If the rdtgroup is a ctrl_mon group and parent directory
> +	 * is the root directory, remove the ctrl_mon group.
> +	 *
> +	 * If the rdtgroup is a mon group and parent directory
> +	 * is a valid "mon_groups" directory, remove the mon group.
> +	 */
> +	if (rdtgrp->type == RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn &&
> +	    rdtgrp != &rdtgroup_default) {
> +		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
> +		    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
> +			ret = rdtgroup_ctrl_remove(rdtgrp);
> +		} else {
> +			ret = rdtgroup_rmdir_ctrl(rdtgrp, tmpmask);
> +		}
> +	} else if (rdtgrp->type == RDTMON_GROUP &&
> +		 is_mon_groups(parent_kn, kn->name)) {
> +		ret = rdtgroup_rmdir_mon(rdtgrp, tmpmask);
> +	} else {
> +		ret = -EPERM;
> +	}
> +
> +out:
> +	rdtgroup_kn_unlock(kn);
> +	free_cpumask_var(tmpmask);
> +	return ret;
> +}
> +
> +/**
> + * mongrp_reparent() - replace parent CTRL_MON group of a MON group
> + * @rdtgrp:		the MON group whose parent should be replaced
> + * @new_prdtgrp:	replacement parent CTRL_MON group for @rdtgrp
> + * @cpus:		cpumask provided by the caller for use during this call
> + *
> + * Replaces the parent CTRL_MON group for a MON group, resulting in all member
> + * tasks' CLOSID immediately changing to that of the new parent group.
> + * Monitoring data for the group is unaffected by this operation.
> + */
> +static void mongrp_reparent(struct rdtgroup *rdtgrp,
> +			    struct rdtgroup *new_prdtgrp,
> +			    cpumask_var_t cpus)
> +{
> +	struct rdtgroup *prdtgrp = rdtgrp->mon.parent;
> +
> +	WARN_ON(rdtgrp->type != RDTMON_GROUP);
> +	WARN_ON(new_prdtgrp->type != RDTCTRL_GROUP);
> +
> +	/* Nothing to do when simply renaming a MON group. */
> +	if (prdtgrp == new_prdtgrp)
> +		return;
> +
> +	WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
> +	list_move_tail(&rdtgrp->mon.crdtgrp_list,
> +		       &new_prdtgrp->mon.crdtgrp_list);
> +
> +	rdtgrp->mon.parent = new_prdtgrp;
> +	rdtgrp->closid = new_prdtgrp->closid;
> +
> +	/* Propagate updated closid to all tasks in this group. */
> +	rdt_move_group_tasks(rdtgrp, rdtgrp, cpus);
> +
> +	update_closid_rmid(cpus, NULL);
> +}
> +
> +static int rdtgroup_rename(struct kernfs_node *kn,
> +			   struct kernfs_node *new_parent, const char *new_name)
> +{
> +	struct rdtgroup *new_prdtgrp;
> +	struct rdtgroup *rdtgrp;
> +	cpumask_var_t tmpmask;
> +	int ret;
> +
> +	rdtgrp = kernfs_to_rdtgroup(kn);
> +	new_prdtgrp = kernfs_to_rdtgroup(new_parent);
> +	if (!rdtgrp || !new_prdtgrp)
> +		return -ENOENT;
> +
> +	/* Release both kernfs active_refs before obtaining rdtgroup mutex. */
> +	rdtgroup_kn_get(rdtgrp, kn);
> +	rdtgroup_kn_get(new_prdtgrp, new_parent);
> +
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	rdt_last_cmd_clear();
> +
> +	/*
> +	 * Don't allow kernfs_to_rdtgroup() to return a parent rdtgroup if
> +	 * either kernfs_node is a file.
> +	 */
> +	if (kernfs_type(kn) != KERNFS_DIR ||
> +	    kernfs_type(new_parent) != KERNFS_DIR) {
> +		rdt_last_cmd_puts("Source and destination must be directories");
> +		ret = -EPERM;
> +		goto out;
> +	}
> +
> +	if ((rdtgrp->flags & RDT_DELETED) || (new_prdtgrp->flags & RDT_DELETED)) {
> +		ret = -ENOENT;
> +		goto out;
> +	}
> +
> +	if (rdtgrp->type != RDTMON_GROUP || !kn->parent ||
> +	    !is_mon_groups(kn->parent, kn->name)) {
> +		rdt_last_cmd_puts("Source must be a MON group\n");
> +		ret = -EPERM;
> +		goto out;
> +	}
> +
> +	if (!is_mon_groups(new_parent, new_name)) {
> +		rdt_last_cmd_puts("Destination must be a mon_groups subdirectory\n");
> +		ret = -EPERM;
> +		goto out;
> +	}
> +
> +	/*
> +	 * If the MON group is monitoring CPUs, the CPUs must be assigned to the
> +	 * current parent CTRL_MON group and therefore cannot be assigned to
> +	 * the new parent, making the move illegal.
> +	 */
> +	if (!cpumask_empty(&rdtgrp->cpu_mask) &&
> +	    rdtgrp->mon.parent != new_prdtgrp) {
> +		rdt_last_cmd_puts("Cannot move a MON group that monitors CPUs\n");
> +		ret = -EPERM;
> +		goto out;
> +	}
> +
> +	/*
> +	 * Allocate the cpumask for use in mongrp_reparent() to avoid the
> +	 * possibility of failing to allocate it after kernfs_rename() has
> +	 * succeeded.
> +	 */
> +	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL)) {
> +		ret = -ENOMEM;
> +		goto out;
> +	}
> +
> +	/*
> +	 * Perform all input validation and allocations needed to ensure
> +	 * mongrp_reparent() will succeed before calling kernfs_rename(),
> +	 * otherwise it would be necessary to revert this call if
> +	 * mongrp_reparent() failed.
> +	 */
> +	ret = kernfs_rename(kn, new_parent, new_name);
> +	if (!ret)
> +		mongrp_reparent(rdtgrp, new_prdtgrp, tmpmask);
> +
> +	free_cpumask_var(tmpmask);
> +
> +out:
> +	mutex_unlock(&rdtgroup_mutex);
> +	rdtgroup_kn_put(rdtgrp, kn);
> +	rdtgroup_kn_put(new_prdtgrp, new_parent);
> +	return ret;
> +}
> +
> +static int rdtgroup_show_options(struct seq_file *seq, struct kernfs_root *kf)
> +{
> +	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L3))
> +		seq_puts(seq, ",cdp");
> +
> +	if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2))
> +		seq_puts(seq, ",cdpl2");
> +
> +	if (is_mba_sc(resctrl_arch_get_resource(RDT_RESOURCE_MBA)))
> +		seq_puts(seq, ",mba_MBps");
> +
> +	if (resctrl_debug)
> +		seq_puts(seq, ",debug");
> +
> +	return 0;
> +}
> +
> +static struct kernfs_syscall_ops rdtgroup_kf_syscall_ops = {
> +	.mkdir		= rdtgroup_mkdir,
> +	.rmdir		= rdtgroup_rmdir,
> +	.rename		= rdtgroup_rename,
> +	.show_options	= rdtgroup_show_options,
> +};
> +
> +static int rdtgroup_setup_root(struct rdt_fs_context *ctx)
> +{
> +	rdt_root = kernfs_create_root(&rdtgroup_kf_syscall_ops,
> +				      KERNFS_ROOT_CREATE_DEACTIVATED |
> +				      KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK,
> +				      &rdtgroup_default);
> +	if (IS_ERR(rdt_root))
> +		return PTR_ERR(rdt_root);
> +
> +	ctx->kfc.root = rdt_root;
> +	rdtgroup_default.kn = kernfs_root_to_node(rdt_root);
> +
> +	return 0;
> +}
> +
> +static void rdtgroup_destroy_root(void)
> +{
> +	kernfs_destroy_root(rdt_root);
> +	rdtgroup_default.kn = NULL;
> +}
> +
> +static void rdtgroup_setup_default(void)
> +{
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	rdtgroup_default.closid = RESCTRL_RESERVED_CLOSID;
> +	rdtgroup_default.mon.rmid = RESCTRL_RESERVED_RMID;
> +	rdtgroup_default.type = RDTCTRL_GROUP;
> +	INIT_LIST_HEAD(&rdtgroup_default.mon.crdtgrp_list);
> +
> +	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
> +
> +	mutex_unlock(&rdtgroup_mutex);
> +}
> +
> +static void domain_destroy_mon_state(struct rdt_domain *d)
> +{
> +	bitmap_free(d->rmid_busy_llc);
> +	kfree(d->mbm_total);
> +	kfree(d->mbm_local);
> +}
> +
> +void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)
> +{
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA)
> +		mba_sc_domain_destroy(r, d);
> +
> +	if (!r->mon_capable)
> +		goto out_unlock;
> +
> +	/*
> +	 * If resctrl is mounted, remove all the
> +	 * per domain monitor data directories.
> +	 */
> +	if (resctrl_mounted && resctrl_arch_mon_capable())
> +		rmdir_mondata_subdir_allrdtgrp(r, d->id);
> +
> +	if (resctrl_is_mbm_enabled())
> +		cancel_delayed_work(&d->mbm_over);
> +	if (resctrl_arch_is_llc_occupancy_enabled() && has_busy_rmid(d)) {
> +		/*
> +		 * When a package is going down, forcefully
> +		 * decrement rmid->ebusy. There is no way to know
> +		 * that the L3 was flushed and hence may lead to
> +		 * incorrect counts in rare scenarios, but leaving
> +		 * the RMID as busy creates RMID leaks if the
> +		 * package never comes back.
> +		 */
> +		__check_limbo(d, true);
> +		cancel_delayed_work(&d->cqm_limbo);
> +	}
> +
> +	domain_destroy_mon_state(d);
> +
> +out_unlock:
> +	mutex_unlock(&rdtgroup_mutex);
> +}
> +
> +static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
> +{
> +	u32 idx_limit = resctrl_arch_system_num_rmid_idx();
> +	size_t tsize;
> +
> +	if (resctrl_arch_is_llc_occupancy_enabled()) {
> +		d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL);
> +		if (!d->rmid_busy_llc)
> +			return -ENOMEM;
> +	}
> +	if (resctrl_arch_is_mbm_total_enabled()) {
> +		tsize = sizeof(*d->mbm_total);
> +		d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL);
> +		if (!d->mbm_total) {
> +			bitmap_free(d->rmid_busy_llc);
> +			return -ENOMEM;
> +		}
> +	}
> +	if (resctrl_arch_is_mbm_local_enabled()) {
> +		tsize = sizeof(*d->mbm_local);
> +		d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL);
> +		if (!d->mbm_local) {
> +			bitmap_free(d->rmid_busy_llc);
> +			kfree(d->mbm_total);
> +			return -ENOMEM;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)
> +{
> +	int err = 0;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +
> +	if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA) {
> +		/* RDT_RESOURCE_MBA is never mon_capable */
> +		err = mba_sc_domain_allocate(r, d);
> +		goto out_unlock;
> +	}
> +
> +	if (!r->mon_capable)
> +		goto out_unlock;
> +
> +	err = domain_setup_mon_state(r, d);
> +	if (err)
> +		goto out_unlock;
> +
> +	if (resctrl_is_mbm_enabled()) {
> +		INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow);
> +		mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL,
> +					   RESCTRL_PICK_ANY_CPU);
> +	}
> +
> +	if (resctrl_arch_is_llc_occupancy_enabled())
> +		INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo);
> +
> +	/*
> +	 * If the filesystem is not mounted then only the default resource group
> +	 * exists. Creation of its directories is deferred until mount time
> +	 * by rdt_get_tree() calling mkdir_mondata_all().
> +	 * If resctrl is mounted, add per domain monitor data directories.
> +	 */
> +	if (resctrl_mounted && resctrl_arch_mon_capable())
> +		mkdir_mondata_subdir_allrdtgrp(r, d);
> +
> +out_unlock:
> +	mutex_unlock(&rdtgroup_mutex);
> +
> +	return err;
> +}
> +
> +void resctrl_online_cpu(unsigned int cpu)
> +{
> +	mutex_lock(&rdtgroup_mutex);
> +	/* The CPU is set in default rdtgroup after online. */
> +	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
> +	mutex_unlock(&rdtgroup_mutex);
> +}
> +
> +static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)
> +{
> +	struct rdtgroup *cr;
> +
> +	list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) {
> +		if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask))
> +			break;
> +	}
> +}
> +
> +void resctrl_offline_cpu(unsigned int cpu)
> +{
> +	struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	struct rdtgroup *rdtgrp;
> +	struct rdt_domain *d;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +	list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) {
> +		if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) {
> +			clear_childcpus(rdtgrp, cpu);
> +			break;
> +		}
> +	}
> +
> +	if (!l3->mon_capable)
> +		goto out_unlock;
> +
> +	d = resctrl_get_domain_from_cpu(cpu, l3);
> +	if (d) {
> +		if (resctrl_is_mbm_enabled() && cpu == d->mbm_work_cpu) {
> +			cancel_delayed_work(&d->mbm_over);
> +			mbm_setup_overflow_handler(d, 0, cpu);
> +		}
> +		if (resctrl_arch_is_llc_occupancy_enabled() &&
> +		    cpu == d->cqm_work_cpu && has_busy_rmid(d)) {
> +			cancel_delayed_work(&d->cqm_limbo);
> +			cqm_setup_limbo_handler(d, 0, cpu);
> +		}
> +	}
> +
> +out_unlock:
> +	mutex_unlock(&rdtgroup_mutex);
> +}
> +
> +/*
> + * resctrl_init - resctrl filesystem initialization
> + *
> + * Setup resctrl file system including set up root, create mount point,
> + * register resctrl filesystem, and initialize files under root directory.
> + *
> + * Return: 0 on success or -errno
> + */
> +int resctrl_init(void)
> +{
> +	int ret = 0;
> +
> +	seq_buf_init(&last_cmd_status, last_cmd_status_buf,
> +		     sizeof(last_cmd_status_buf));
> +
> +	rdtgroup_setup_default();
> +
> +	thread_throttle_mode_init();
> +
> +	ret = resctrl_mon_resource_init();
> +	if (ret)
> +		return ret;
> +
> +	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
> +	if (ret)
> +		return ret;
> +
> +	ret = register_filesystem(&rdt_fs_type);
> +	if (ret)
> +		goto cleanup_mountpoint;
> +
> +	/*
> +	 * Adding the resctrl debugfs directory here may not be ideal since
> +	 * it would let the resctrl debugfs directory appear on the debugfs
> +	 * filesystem before the resctrl filesystem is mounted.
> +	 * It may also be ok since that would enable debugging of RDT before
> +	 * resctrl is mounted.
> +	 * The reason why the debugfs directory is created here and not in
> +	 * rdt_get_tree() is because rdt_get_tree() takes rdtgroup_mutex and
> +	 * during the debugfs directory creation also &sb->s_type->i_mutex_key
> +	 * (the lockdep class of inode->i_rwsem). Other filesystem
> +	 * interactions (eg. SyS_getdents) have the lock ordering:
> +	 * &sb->s_type->i_mutex_key --> &mm->mmap_lock
> +	 * During mmap(), called with &mm->mmap_lock, the rdtgroup_mutex
> +	 * is taken, thus creating dependency:
> +	 * &mm->mmap_lock --> rdtgroup_mutex for the latter that can cause
> +	 * issues considering the other two lock dependencies.
> +	 * By creating the debugfs directory here we avoid a dependency
> +	 * that may cause deadlock (even though file operations cannot
> +	 * occur until the filesystem is mounted, but I do not know how to
> +	 * tell lockdep that).
> +	 */
> +	debugfs_resctrl = debugfs_create_dir("resctrl", NULL);
> +
> +	return 0;
> +
> +cleanup_mountpoint:
> +	sysfs_remove_mount_point(fs_kobj, "resctrl");
> +
> +	return ret;
> +}
> +
> +void resctrl_exit(void)
> +{
> +	debugfs_remove_recursive(debugfs_resctrl);
> +	unregister_filesystem(&rdt_fs_type);
> +	sysfs_remove_mount_point(fs_kobj, "resctrl");
> +
> +	resctrl_mon_resource_exit();
> +}

-- 
Thanks
Babu Moger

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

Hi,

On Mon, Apr 15, 2024 at 03:44:48PM -0500, Moger, Babu wrote:
> James/Dave,
> This is a huge change. Can this be broken into multiple patches?
> It will be a major task in case we need to bisect to pinpoint any issues
> later.
> Thanks
> Babu

[...]

I guess this might be possible, though when moving groups of static
functions around that refer to each other it might be more trouble than
it is worth.

I'll need to discuss with James.

Either way, I think this patch will need to rebuilt when spinning v2 of
this series.  I've been rebasing it by hand, which is not exactly
foolproof!

It ought to be possible to break this up into one patch per affected .c
file at least, if people feel that it is worthwhile.

Cheers
---Dave

Re: [PATCH v1 31/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[James Morse]

Hi Babu,

On 17/04/2024 15:43, Dave Martin wrote:
> On Mon, Apr 15, 2024 at 03:44:48PM -0500, Moger, Babu wrote:
>> James/Dave,
>> This is a huge change. Can this be broken into multiple patches?
>> It will be a major task in case we need to bisect to pinpoint any issues
>> later.

> I guess this might be possible, though when moving groups of static
> functions around that refer to each other it might be more trouble than
> it is worth.
> 
> I'll need to discuss with James.
> 
> Either way, I think this patch will need to rebuilt when spinning v2 of
> this series.  I've been rebasing it by hand, which is not exactly
> foolproof!
> 
> It ought to be possible to break this up into one patch per affected .c
> file at least, if people feel that it is worthwhile.

That is how I keep it in my tree - but the intermediate entries don't build, so its only
useful for keeping my sanity.

I don't think the intermediate points are useful, it would be better to spend the time
generating confidence that this thing doesn't introduce any functional changes - if it
does, it should be reverted.

To that end:
So far I've done this by hand, but recently cooked a python script to generate this. I'll
include this in the next version instead. Hopefully this is a better way do demonstrate
'nothing up my sleeve' - it certainly makes it easier to regenerate this after other
series have been merged.

Obviously I'm not expecting this to be merged as is - as suggested in the cover letter we
need to find a least disruptive moment.


Thanks,

James

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Reinette Chatre]

Hi James and x86 Maintainers,

Please consider the file movements as captured in the diffstat below:

On 3/21/2024 9:50 AM, James Morse wrote:
>  MAINTAINERS                               |    2 +
>  arch/Kconfig                              |    8 +
>  arch/x86/Kconfig                          |    5 +-
>  arch/x86/include/asm/resctrl.h            |   45 +-
>  arch/x86/kernel/cpu/resctrl/Makefile      |    5 +-
>  arch/x86/kernel/cpu/resctrl/core.c        |  119 +-
>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  506 +--
>  arch/x86/kernel/cpu/resctrl/internal.h    |  433 +--
>  arch/x86/kernel/cpu/resctrl/monitor.c     |  813 +----
>  arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1126 +-----
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3929 +-------------------
>  arch/x86/kernel/process_32.c              |    2 +-
>  arch/x86/kernel/process_64.c              |    2 +-
>  fs/Kconfig                                |    1 +
>  fs/Makefile                               |    1 +
>  fs/resctrl/Kconfig                        |   23 +
>  fs/resctrl/Makefile                       |    3 +
>  fs/resctrl/ctrlmondata.c                  |  527 +++
>  fs/resctrl/internal.h                     |  340 ++
>  fs/resctrl/monitor.c                      |  843 +++++
>  fs/resctrl/psuedo_lock.c                  | 1122 ++++++

(sidenote: James, please note typo in psuedo_lock.c)

>  fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
>  include/linux/resctrl.h                   |  153 +-
>  include/linux/resctrl_types.h             |   98 +
>  24 files changed, 7244 insertions(+), 6875 deletions(-)
>  create mode 100644 fs/resctrl/Kconfig
>  create mode 100644 fs/resctrl/Makefile
>  create mode 100644 fs/resctrl/ctrlmondata.c
>  create mode 100644 fs/resctrl/internal.h
>  create mode 100644 fs/resctrl/monitor.c
>  create mode 100644 fs/resctrl/psuedo_lock.c
>  create mode 100644 fs/resctrl/rdtgroup.c
>  create mode 100644 include/linux/resctrl_types.h

I would like to check in on the sentiments regarding maintaining the resctrl
contributions after this work is merged. Considering that resctrl will 
be split between fs/resctrl and arch/x86, would it still be acceptable for
resctrl code to both areas (filesystem as well as arch) to flow via the tip tree with
help from x86 maintainers?

How will Arm patches flow?

James, are you planning a separate MAINTAINERS entry for the Arm specific code?
I would like to propose that you add yourself as a reviewer to the existing resctrl
MAINTAINERS entry to learn when any changes are submitted that may impact Arm. 

Reinette

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

On Mon, Apr 08, 2024 at 08:13:19PM -0700, Reinette Chatre wrote:
> Hi James and x86 Maintainers,
> 
> Please consider the file movements as captured in the diffstat below:
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> >  MAINTAINERS                               |    2 +
> >  arch/Kconfig                              |    8 +
> >  arch/x86/Kconfig                          |    5 +-
> >  arch/x86/include/asm/resctrl.h            |   45 +-
> >  arch/x86/kernel/cpu/resctrl/Makefile      |    5 +-
> >  arch/x86/kernel/cpu/resctrl/core.c        |  119 +-
> >  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  506 +--
> >  arch/x86/kernel/cpu/resctrl/internal.h    |  433 +--
> >  arch/x86/kernel/cpu/resctrl/monitor.c     |  813 +----
> >  arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1126 +-----
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3929 +-------------------
> >  arch/x86/kernel/process_32.c              |    2 +-
> >  arch/x86/kernel/process_64.c              |    2 +-
> >  fs/Kconfig                                |    1 +
> >  fs/Makefile                               |    1 +
> >  fs/resctrl/Kconfig                        |   23 +
> >  fs/resctrl/Makefile                       |    3 +
> >  fs/resctrl/ctrlmondata.c                  |  527 +++
> >  fs/resctrl/internal.h                     |  340 ++
> >  fs/resctrl/monitor.c                      |  843 +++++
> >  fs/resctrl/psuedo_lock.c                  | 1122 ++++++
> 
> (sidenote: James, please note typo in psuedo_lock.c)

Noted.

(So that's what the Psuedo Lock key on the keyboard does...)

> 
> >  fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
> >  include/linux/resctrl.h                   |  153 +-
> >  include/linux/resctrl_types.h             |   98 +
> >  24 files changed, 7244 insertions(+), 6875 deletions(-)
> >  create mode 100644 fs/resctrl/Kconfig
> >  create mode 100644 fs/resctrl/Makefile
> >  create mode 100644 fs/resctrl/ctrlmondata.c
> >  create mode 100644 fs/resctrl/internal.h
> >  create mode 100644 fs/resctrl/monitor.c
> >  create mode 100644 fs/resctrl/psuedo_lock.c
> >  create mode 100644 fs/resctrl/rdtgroup.c
> >  create mode 100644 include/linux/resctrl_types.h
> 
> I would like to check in on the sentiments regarding maintaining the resctrl
> contributions after this work is merged. Considering that resctrl will 
> be split between fs/resctrl and arch/x86, would it still be acceptable for
> resctrl code to both areas (filesystem as well as arch) to flow via the tip tree with
> help from x86 maintainers?
> 
> How will Arm patches flow?
> 
> James, are you planning a separate MAINTAINERS entry for the Arm specific code?
> I would like to propose that you add yourself as a reviewer to the existing resctrl
> MAINTAINERS entry to learn when any changes are submitted that may impact Arm. 
> 
> Reinette

I'll leave this for James to respond.

Cheers
---Dave

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

Hi Reinette,

Since we seem to be getting to the end of the outstanding discussion on
this series, what's your preference for moving this forward?

James will be back towards the end of next week AIUI, so it should be
possible to apply most of the changes that have been discussed and post
a v2 for review fairly quickly after that.

I'm happy to apply my fixups branch and post an untested "v1.1" for
review if you think it would be useful to see the de-noised output of
the review so far, but I don't want to fork the discussion or create
unnecessary work.  This interim respin would be subject to what James
wants to pick up.

Alternatively, I can try to follow up on the string parsing discussion
from patch #3 with some real code if you'd prefer to get a clean
interface in place for that.

Cheers
---Dave


On Mon, Apr 08, 2024 at 08:13:19PM -0700, Reinette Chatre wrote:
> Hi James and x86 Maintainers,
> 
> Please consider the file movements as captured in the diffstat below:
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
> >  MAINTAINERS                               |    2 +
> >  arch/Kconfig                              |    8 +
> >  arch/x86/Kconfig                          |    5 +-
> >  arch/x86/include/asm/resctrl.h            |   45 +-
> >  arch/x86/kernel/cpu/resctrl/Makefile      |    5 +-
> >  arch/x86/kernel/cpu/resctrl/core.c        |  119 +-
> >  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  506 +--
> >  arch/x86/kernel/cpu/resctrl/internal.h    |  433 +--
> >  arch/x86/kernel/cpu/resctrl/monitor.c     |  813 +----
> >  arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1126 +-----
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3929 +-------------------
> >  arch/x86/kernel/process_32.c              |    2 +-
> >  arch/x86/kernel/process_64.c              |    2 +-
> >  fs/Kconfig                                |    1 +
> >  fs/Makefile                               |    1 +
> >  fs/resctrl/Kconfig                        |   23 +
> >  fs/resctrl/Makefile                       |    3 +
> >  fs/resctrl/ctrlmondata.c                  |  527 +++
> >  fs/resctrl/internal.h                     |  340 ++
> >  fs/resctrl/monitor.c                      |  843 +++++
> >  fs/resctrl/psuedo_lock.c                  | 1122 ++++++
> 
> (sidenote: James, please note typo in psuedo_lock.c)
> 
> >  fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
> >  include/linux/resctrl.h                   |  153 +-
> >  include/linux/resctrl_types.h             |   98 +
> >  24 files changed, 7244 insertions(+), 6875 deletions(-)
> >  create mode 100644 fs/resctrl/Kconfig
> >  create mode 100644 fs/resctrl/Makefile
> >  create mode 100644 fs/resctrl/ctrlmondata.c
> >  create mode 100644 fs/resctrl/internal.h
> >  create mode 100644 fs/resctrl/monitor.c
> >  create mode 100644 fs/resctrl/psuedo_lock.c
> >  create mode 100644 fs/resctrl/rdtgroup.c
> >  create mode 100644 include/linux/resctrl_types.h
> 
> I would like to check in on the sentiments regarding maintaining the resctrl
> contributions after this work is merged. Considering that resctrl will 
> be split between fs/resctrl and arch/x86, would it still be acceptable for
> resctrl code to both areas (filesystem as well as arch) to flow via the tip tree with
> help from x86 maintainers?
> 
> How will Arm patches flow?
> 
> James, are you planning a separate MAINTAINERS entry for the Arm specific code?
> I would like to propose that you add yourself as a reviewer to the existing resctrl
> MAINTAINERS entry to learn when any changes are submitted that may impact Arm. 
> 
> Reinette
> 
> 

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Reinette Chatre]

Hi Dave,

On 4/18/2024 8:32 AM, Dave Martin wrote:
> Since we seem to be getting to the end of the outstanding discussion on
> this series, what's your preference for moving this forward?
> 
> James will be back towards the end of next week AIUI, so it should be
> possible to apply most of the changes that have been discussed and post
> a v2 for review fairly quickly after that.
> 
> I'm happy to apply my fixups branch and post an untested "v1.1" for
> review if you think it would be useful to see the de-noised output of
> the review so far, but I don't want to fork the discussion or create
> unnecessary work.  This interim respin would be subject to what James
> wants to pick up.

Please post v2 with discussions complete and feedback addressed
(unless there are some discussions that need to see how other changes turn out
before they can complete but I do not think this applies to this series).

To create some expectations, when you submit a new series it will go to the back of
my review queue and at this time there are three other resctrl pieces of work
waiting for review (btw ... one of them, [1], is waiting for feedback from Arm). 
 
> Alternatively, I can try to follow up on the string parsing discussion
> from patch #3 with some real code if you'd prefer to get a clean
> interface in place for that.

We can keep discussing that, sure.

Reinette

[1] https://lore.kernel.org/lkml/cover.1711674410.git.babu.moger@amd.com/

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Moger, Babu]

Hi Reinette,

On 4/19/24 23:06, Reinette Chatre wrote:
> Hi Dave,
> 
> On 4/18/2024 8:32 AM, Dave Martin wrote:
>> Since we seem to be getting to the end of the outstanding discussion on
>> this series, what's your preference for moving this forward?
>>
>> James will be back towards the end of next week AIUI, so it should be
>> possible to apply most of the changes that have been discussed and post
>> a v2 for review fairly quickly after that.
>>
>> I'm happy to apply my fixups branch and post an untested "v1.1" for
>> review if you think it would be useful to see the de-noised output of
>> the review so far, but I don't want to fork the discussion or create
>> unnecessary work.  This interim respin would be subject to what James
>> wants to pick up.
> 
> Please post v2 with discussions complete and feedback addressed
> (unless there are some discussions that need to see how other changes turn out
> before they can complete but I do not think this applies to this series).
> 
> To create some expectations, when you submit a new series it will go to the back of
> my review queue and at this time there are three other resctrl pieces of work
> waiting for review (btw ... one of them, [1], is waiting for feedback from Arm). 
>  
>> Alternatively, I can try to follow up on the string parsing discussion
>> from patch #3 with some real code if you'd prefer to get a clean
>> interface in place for that.
> 
> We can keep discussing that, sure.
> 
> Reinette
> 
> [1] https://lore.kernel.org/lkml/cover.1711674410.git.babu.moger@amd.com/

Do you have any more feedback on this series. I have few feedbacks from
Peter. I was planning to work on v4 of this series.

-- 
Thanks
Babu Moger

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Reinette Chatre]

Hi Babu and Dave,

On 4/22/2024 6:51 AM, Moger, Babu wrote:
> On 4/19/24 23:06, Reinette Chatre wrote:
>>
>> [1] https://lore.kernel.org/lkml/cover.1711674410.git.babu.moger@amd.com/
> 
> Do you have any more feedback on this series. I have few feedbacks from
> Peter. I was planning to work on v4 of this series.
> 

Babu: It is difficult to start drilling into the implementation before there
is agreement on the interface. One reason you went through the effort of
the first few iterations was to accommodate Arm's use cases as we understand
it, but we need to hear from Arm if we are on the right track here.
I do hope that we will hear something in the next couple of weeks. 

Dave: Could you please check in if the interface introduced [1] is something
of interest to Arm? If it is not, we can proceed with the implementation without
trying to consider how Arm may use/need such an interface. If it is, could you
please let us know when we can expect feedback from Arm? 

Dave: fyi ... there is a similar scenario with [2] that aims to address some Arm
feedback ([3]). Do not be deceived by version number of [2], which is categorized
as a "new approach" aiming to accommodate feedback. It has a long history [4].

Reinette  

[2] https://lore.kernel.org/lkml/20240327200352.236835-1-tony.luck@intel.com/
[3] https://lore.kernel.org/lkml/88430722-67b3-4f7d-8db2-95ee52b6f0b0@arm.com/
[4] https://lore.kernel.org/lkml/20240312214247.91772-1-tony.luck@intel.com/

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Moger, Babu]

On 4/22/24 11:01, Reinette Chatre wrote:
> Hi Babu and Dave,
> 
> On 4/22/2024 6:51 AM, Moger, Babu wrote:
>> On 4/19/24 23:06, Reinette Chatre wrote:
>>>
>>> [1] https://lore.kernel.org/lkml/cover.1711674410.git.babu.moger@amd.com/
>>
>> Do you have any more feedback on this series. I have few feedbacks from
>> Peter. I was planning to work on v4 of this series.
>>
> 
> Babu: It is difficult to start drilling into the implementation before there
> is agreement on the interface. One reason you went through the effort of
> the first few iterations was to accommodate Arm's use cases as we understand
> it, but we need to hear from Arm if we are on the right track here.
> I do hope that we will hear something in the next couple of weeks. 

Sure. Sounds good.
-- 
Thanks
Babu Moger

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Peter Newman]

Hi Dave,

On Mon, Apr 22, 2024 at 9:01 AM Reinette Chatre
<reinette.chatre@intel.com> wrote:
>
> Hi Babu and Dave,
>
> On 4/22/2024 6:51 AM, Moger, Babu wrote:
> > On 4/19/24 23:06, Reinette Chatre wrote:
> >>
> >> [1] https://lore.kernel.org/lkml/cover.1711674410.git.babu.moger@amd.com/
> >
> > Do you have any more feedback on this series. I have few feedbacks from
> > Peter. I was planning to work on v4 of this series.
> >
>
> Babu: It is difficult to start drilling into the implementation before there
> is agreement on the interface. One reason you went through the effort of
> the first few iterations was to accommodate Arm's use cases as we understand
> it, but we need to hear from Arm if we are on the right track here.
> I do hope that we will hear something in the next couple of weeks.
>
> Dave: Could you please check in if the interface introduced [1] is something
> of interest to Arm? If it is not, we can proceed with the implementation without
> trying to consider how Arm may use/need such an interface. If it is, could you
> please let us know when we can expect feedback from Arm?

Because MPAM implementations typically expose an MSC for each DRAM
channel, there is an alternate strategy we can use for the monitor
scalability problem:

When a single DRAM MSC does not provide enough monitors to track all
of the supported PARTID x PMG combinations simultaneously, the DRAM
MSCs collectively may provide a sufficient number of monitors.
Therefore, as long as the distribution of traffic among the DRAM
channels is uniform (or predictably non-uniform), it's possible to
estimate the total bandwidth with sufficient accuracy.

-Peter

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

On Mon, Apr 22, 2024 at 11:39:00AM -0700, Peter Newman wrote:
> Hi Dave,
> 
> On Mon, Apr 22, 2024 at 9:01 AM Reinette Chatre
> <reinette.chatre@intel.com> wrote:
> >
> > Hi Babu and Dave,
> >
> > On 4/22/2024 6:51 AM, Moger, Babu wrote:
> > > On 4/19/24 23:06, Reinette Chatre wrote:
> > >>
> > >> [1] https://lore.kernel.org/lkml/cover.1711674410.git.babu.moger@amd.com/
> > >
> > > Do you have any more feedback on this series. I have few feedbacks from
> > > Peter. I was planning to work on v4 of this series.
> > >
> >
> > Babu: It is difficult to start drilling into the implementation before there
> > is agreement on the interface. One reason you went through the effort of
> > the first few iterations was to accommodate Arm's use cases as we understand
> > it, but we need to hear from Arm if we are on the right track here.
> > I do hope that we will hear something in the next couple of weeks.
> >
> > Dave: Could you please check in if the interface introduced [1] is something
> > of interest to Arm? If it is not, we can proceed with the implementation without
> > trying to consider how Arm may use/need such an interface. If it is, could you
> > please let us know when we can expect feedback from Arm?
> 
> Because MPAM implementations typically expose an MSC for each DRAM
> channel, there is an alternate strategy we can use for the monitor
> scalability problem:
> 
> When a single DRAM MSC does not provide enough monitors to track all
> of the supported PARTID x PMG combinations simultaneously, the DRAM
> MSCs collectively may provide a sufficient number of monitors.
> Therefore, as long as the distribution of traffic among the DRAM
> channels is uniform (or predictably non-uniform), it's possible to
> estimate the total bandwidth with sufficient accuracy.
> 
> -Peter
> 

So you're suggesting that if (say) DRAM traffic is striped symmetrically
across N channels, and each has counters, then a counter matching
PARTID:PMG on just one channel should given an unbiased estimate of the
traffic from that group (with some sacrifice of precision, and assuming
the workload is non-pathological)?

I guess that could work, though this might work badly for some workloads
and might give a malicious workload a way to hide transactions from
monitoring if the placement of the counter is too fixed and/or
predictable.

Cheers
---Dave

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[James Morse]

Hi Reinette,

On 09/04/2024 04:13, Reinette Chatre wrote:
> Please consider the file movements as captured in the diffstat below:
> 
> On 3/21/2024 9:50 AM, James Morse wrote:
>>  MAINTAINERS                               |    2 +
>>  arch/Kconfig                              |    8 +
>>  arch/x86/Kconfig                          |    5 +-
>>  arch/x86/include/asm/resctrl.h            |   45 +-
>>  arch/x86/kernel/cpu/resctrl/Makefile      |    5 +-
>>  arch/x86/kernel/cpu/resctrl/core.c        |  119 +-
>>  arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  506 +--
>>  arch/x86/kernel/cpu/resctrl/internal.h    |  433 +--
>>  arch/x86/kernel/cpu/resctrl/monitor.c     |  813 +----
>>  arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 1126 +-----
>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 3929 +-------------------
>>  arch/x86/kernel/process_32.c              |    2 +-
>>  arch/x86/kernel/process_64.c              |    2 +-
>>  fs/Kconfig                                |    1 +
>>  fs/Makefile                               |    1 +
>>  fs/resctrl/Kconfig                        |   23 +
>>  fs/resctrl/Makefile                       |    3 +
>>  fs/resctrl/ctrlmondata.c                  |  527 +++
>>  fs/resctrl/internal.h                     |  340 ++
>>  fs/resctrl/monitor.c                      |  843 +++++
>>  fs/resctrl/psuedo_lock.c                  | 1122 ++++++
> 
> (sidenote: James, please note typo in psuedo_lock.c)

Yeah - I still can't spell that...


>>  fs/resctrl/rdtgroup.c                     | 4013 +++++++++++++++++++++
>>  include/linux/resctrl.h                   |  153 +-
>>  include/linux/resctrl_types.h             |   98 +
>>  24 files changed, 7244 insertions(+), 6875 deletions(-)
>>  create mode 100644 fs/resctrl/Kconfig
>>  create mode 100644 fs/resctrl/Makefile
>>  create mode 100644 fs/resctrl/ctrlmondata.c
>>  create mode 100644 fs/resctrl/internal.h
>>  create mode 100644 fs/resctrl/monitor.c
>>  create mode 100644 fs/resctrl/psuedo_lock.c
>>  create mode 100644 fs/resctrl/rdtgroup.c
>>  create mode 100644 include/linux/resctrl_types.h

> I would like to check in on the sentiments regarding maintaining the resctrl
> contributions after this work is merged. Considering that resctrl will 
> be split between fs/resctrl and arch/x86, would it still be acceptable for
> resctrl code to both areas (filesystem as well as arch) to flow via the tip tree with
> help from x86 maintainers?

I think it makes sense for all that to be funnelled via tip.


> How will Arm patches flow?

No preference. If it makes the most sense for them to go via tip, then lets do that.

There will be the occasional dependency on arm64, but that should be manageable.
e.g. The first merge of the MPAM driver comes with some arm64 code for context-switch.
That will need co-ordinating with Will+Catalin.


> James, are you planning a separate MAINTAINERS entry for the Arm specific code?
> I would like to propose that you add yourself as a reviewer to the existing resctrl
> MAINTAINERS entry to learn when any changes are submitted that may impact Arm. 

I'll add a patch doing that to the end of the MPAM tree.


Thanks,

James

Re: [PATCH v1 00/31] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

[Dave Martin]

On Thu, Mar 21, 2024 at 04:50:35PM +0000, James Morse wrote:
> Hello!
> 
> This is the final series that allows other architectures to implement resctrl.
> The last patch just moves the code, and its a bit of a monster. I don't expect
> that to get merged as part of this series - we should wait for it to make
> less impact on other series. It's included here to show what gets moved, and
> that structures/function-prototypes have the right visibility.
> 
> Otherwise this series renames functions and moves code around. With the
> exception of invalid configurations for the configurable-events, there should
> be no changes in behaviour caused by this series.
> 
> The driving pattern is to make things like struct rdtgroup private to resctrl.
> Features like pseudo-lock aren't going to work on arm64, the ability to disable
> it at compile time is added.
> 
> After this, I can start posting the MPAM driver to make use of resctrl on arm64.
> (What's MPAM? See the cover letter of the first series. [1])
> 
> This series is based on Linus' commit 23956900041d and can be retrieved from:
> https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git
> mpam/move_to_fs/v1
> 
> Sorry for the mid-merge window base, I'm away for a few weeks - this should
> rebase trivially onto rc1.
> 
> As ever - bugs welcome,
> Thanks,
> 
> James
> 
> [1] https://lore.kernel.org/lkml/20201030161120.227225-1-james.morse@arm.com/

[...]

Hi all,

I'm going to try to give a preliminary response to review feedback while
James is away, to help move this series forward.

Many thanks to those who have reviewed / commented so far!


Apologies for the slow-ish response from me: I'm still not that familiar
with this code, so I wanted to make sure I had broadly covered most of
the simpler comments before I started sending replies.

N.B.: where I say "Noted", I mean that I've stashed a fixup for James
to review when he gets back.  Most of these seem straightforward and
uncontroversial, but it's his code, so it will be up to him what
changes actually get incorporated in the respin.

Cheers
---Dave