Re: [PATCH v3 16/38] x86/resctrl: Move monitor init work to a resctrl init call

[Reinette Chatre <reinette.chatre@intel.com>]

Hi James,

On 7/1/24 11:17 AM, James Morse wrote:
> Hi Reinette,
> 
> On 28/06/2024 17:47, Reinette Chatre wrote:
>> On 6/14/24 8:00 AM, James Morse wrote:
>>> rdt_get_mon_l3_config() is called from the architecture's
>>> resctrl_arch_late_init(), and initialises both architecture specific
>>> fields, such as hw_res->mon_scale and resctrl filesystem fields
>>> by calling dom_data_init().
>>>
>>> To separate the filesystem and architecture parts of resctrl, this
>>> function needs splitting up.
>>>
>>> Add resctrl_mon_resource_init() to do the filesystem specific work,
>>> and call it from resctrl_init(). This runs later, but is still before
>>> the filesystem is mounted and the rmid_ptrs[] array can be used.
> 
>>> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
>>> index 7d6aebce75c1..527c0e9d7b2e 100644
>>> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
>>> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
>>> @@ -1016,12 +1016,28 @@ static void l3_mon_evt_init(struct rdt_resource *r)
>>>            list_add_tail(&mbm_local_event.list, &r->evt_list);
>>>    }
>>>    +int resctrl_mon_resource_init(void)
>>
>> (Lack of an __init is unexpected but I assume it was done since that will be removed
>> in later patch anyway?)
> 
> Yup - I'll add and remove that if you find it surprising.
> 
> 
>> This function needs a big warning to deter anybody from considering this to
>> be the place where any and all monitor related allocations happen. It needs
>> to warn developers that only resources that can only be touched after fs mount
>> may be allocated here.
> 
> I'm afraid I don't follow. Can you give an example of the scenario you are worried about?

My concern is not a scenario with current code flow but a request for informational
comments to prevent future mistakes. Specifically, as I understand the CPU online/offline
handlers can run before this function is called. Those handlers do a lot of setup, getting
resctrl and the system ready. It can be reasonable that some future action may need to touch
a new monitoring structure and with a name like resctrl_mon_resource_init() it seems appropriate
to allocate this new monitoring structure there. I am hoping that resctrl_mon_resource_init()
will have sufficient comments to deter that.

> This is called from resctrl_init(), which is called once the architecture code has done
> its setup, and reckons resctrl is something that can be supported on this platform. It
> would be safe for the limbo/overflow callbacks to start ticking after this point - but
> there is no point if the filesystem isn't mounted yet.
> Filesystem mount is triggered through rdt_get_tree(). The filesystem can't be mounted
> until resctrl_init() goes on to call register_filesystem().
> These allocations could be made later (at mount time), but they're allocated once up-front
> today.
> 
> 
> I've added:
> /**
>   * resctrl_mon_resource_init() - Initialise monitoring structures.

How about a more specific "Initialise monitoring structures used after filesystem mount"?

>   *
>   * Allocate and initialise the rmid_ptrs[] used for the limbo and free lists.
>   * Called once during boot after the struct rdt_resource's have been configured
>   * but before the filesystem is mounted.

Can there be a warning (please feel free to improve):
	"Only for resources used after filesystem mount. For example, do not allocate resources
	 needed by the CPU online/offline handlers since these handlers may run before this
	 function."

>   */
> 
> 
> Thanks,
> 
> James

Reinette