Re: [PATCH-next v6 1/6] cgroup/cpuset: Fix node inconsistencies between cpuset_update_tasks_nodemask() and cpuset_attach()

[Ridong Chen <ridong.chen@linux.dev>]

On 6/6/2026 1:09 AM, Waiman Long wrote:
> On 6/5/26 3:48 AM, Ridong Chen wrote:
>>
>> On 6/4/2026 11:02 PM, Waiman Long wrote:
>>> Whenever memory node mask is changed, there are 4 places where the node
>>> mask has to be updated or used.
>>>   1) task's node mask via cpuset_change_task_nodemask()
>>>   2) memory policy binding via mpol_rebind_mm()
>>>   3) if memory migration is enabled, migrate from old_mems_allowed to
>>>      the new node mask via cpuset_migrate_mm().
>>>   4) setting old_mems_allowed
>>>
>>> These memory actions are done in cpuset_update_tasks_nodemask() and
>>> cpuset_attach(). However there are inconsistencies in what node masks
>>> are being used in these 2 functions.
>>>
>>> In cpuset_update_tasks_nodemask(),
>>>   - cpuset_change_task_nodemask(): guarantee_online_mems()
>>>   - mpol_rebind_mm(): mems_allowed
>>>   - cpuset_migrate_mm(): guarantee_online_mems()
>>>   - old_mems_allowed: guarantee_online_mems()
>>>
>>> In cpuset_attach(),
>>>   - cpuset_change_task_nodemask(): guarantee_online_mems()
>>>   - mpol_rebind_mm(): effective_mems
>>>   - cpuset_migrate_mm(): effective_mems
>>>   - old_mems_allowed: effective_mems
>>>
>>> These inconsistencies dates back to quite a long time ago and it is
>>> hard to say what should be the correct values.
>>>
>>> The guarantee_online_mems() function returns a node mask from current or
>>> an ancestor cpuset that is a subset of node_states[N_MEMORY]. Nodes in
>>> node_states[N_MEMORY] are all online, i.e. in node_states[N_ONLINE].
>>> However, node in node_states[N_ONLINE] may not have memory. So
>>> node_states[N_MEMORY] should be a subset of node_states[N_ONLINE].
>>>
>>> The guarantee_online_mems() function should only be useful for v1 where
>>> mems_allowed is the same as effective_mems. With v2, the memory nodes
>>> in effective_mems should always be a subset of node_states[N_MEMORY].
>>> The only time that may not be true is when a memory hot-unplug operation
>>> is in progress and a memory node is removed from node_states[N_MEMORY]
>>> but not yet reflected in effective_mems as cpuset_handle_hotplug()
>>> has not yet been called from cpuset_track_online_nodes(). When
>>> cpuset_handle_hotplug() is called later, the memory node setting
>>> of the relevant cpusets and tasks will be updated. So replacing the
>>> guarantee_online_mems() call by just using cs->effective_mems should
>>> be fine.
>>>
>> The message should be updated.
> Could you be more specific about what message are you referring to?

After this patch, guarantee_online_mems() for v2 simply returns
cs->effective_mems directly. So the paragraph:

  "The guarantee_online_mems() function should only be useful for v1
   where mems_allowed is the same as effective_mems."

is no longer accurate — guarantee_online_mems() is now useful for
both v1 and v2, just with different behavior. For v2 it returns
effective_mems directly; for v1 it walks up the hierarchy to find
an ancestor with online memory nodes.

Similarly, "So replacing the guarantee_online_mems() call by just
using cs->effective_mems should be fine" describes the old reasoning,
but the patch actually keeps guarantee_online_mems() calls in place
and makes the function itself return effective_mems for v2. The commit
message should reflect what the patch actually does.

>>
>>> Let use the following setup for both of them and make them consistent.
>>>   - cpuset_change_task_nodemask(): guarantee_online_mems()
>>>   - mpol_rebind_mm(): effective_mems
>>>   - cpuset_migrate_mm(): guarantee_online_mems()
>>>   - old_mems_allowed: guarantee_online_mems()
>>>
>>> So for v2, it is effectively all effective_mems. For v1,
>>> mpol_rebind_mm()
>>> uses mems_allowed which may differ from what guarantee_online_mems()
>>> returns.
>>>
>>> Signed-off-by: Waiman Long <longman@redhat.com>
>>> ---
>>>   kernel/cgroup/cpuset.c | 37 +++++++++++++++++++++++++------------
>>>   1 file changed, 25 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
>>> index 6bdb68689c24..8305b5830c3c 100644
>>> --- a/kernel/cgroup/cpuset.c
>>> +++ b/kernel/cgroup/cpuset.c
>>> @@ -489,7 +489,10 @@ static void guarantee_active_cpus(struct
>>> task_struct *tsk,
>>>    * Return in *pmask the portion of a cpusets's mems_allowed that
>>>    * are online, with memory.  If none are online with memory, walk
>>>    * up the cpuset hierarchy until we find one that does have some
>>> - * online mems.  The top cpuset always has some mems online.
>>> + * online mems.  The top cpuset always has some mems online. With v2,
>>> + * effective_mems should always contain online memory nodes except
>>> + * during the transition period where a memory node hotunplug operation
>>> + * is in progress.
>>>    *
>>>    * One way or another, we guarantee to return some non-empty subset
>>>    * of node_states[N_MEMORY].
>>> @@ -498,6 +501,10 @@ static void guarantee_active_cpus(struct
>>> task_struct *tsk,
>>>    */
>>>   static void guarantee_online_mems(struct cpuset *cs, nodemask_t
>>> *pmask)
>>>   {
>>> +    if (cpuset_v2()) {
>>> +        *pmask = cs->effective_mems;
>>> +        return;
>>> +    }
>>>       while (!nodes_and(*pmask, cs->effective_mems,
>>> node_states[N_MEMORY]))
>>>           cs = parent_cs(cs);
>>>   }
>>> @@ -2616,6 +2623,13 @@ static void *cpuset_being_rebound;
>>>    * Iterate through each task of @cs updating its mems_allowed to the
>>>    * effective cpuset's.  As this function is called with
>>> cpuset_mutex held,
>>>    * cpuset membership stays stable.
>>> + *
>>> + * - cpuset_change_task_nodemask(): guarantee_online_mems()
>>> + * - mpol_rebind_mm(): effective_mems
>>> + * - cpuset_migrate_mm(): guarantee_online_mems()
>>> + * - old_mems_allowed: guarantee_online_mems()
>>> + *
>>> + * For v2, guarantee_online_mems() should just return effective_mems.
>>>    */
>>>   void cpuset_update_tasks_nodemask(struct cpuset *cs)
>>>   {
>>> @@ -2624,7 +2638,6 @@ void cpuset_update_tasks_nodemask(struct cpuset
>>> *cs)
>>>       struct task_struct *task;
>>>         cpuset_being_rebound = cs;        /* causes mpol_dup() rebind */
>>> -
>>>       guarantee_online_mems(cs, &newmems);
>>>         /*
>>> @@ -2650,7 +2663,7 @@ void cpuset_update_tasks_nodemask(struct cpuset
>>> *cs)
>>>             migrate = is_memory_migrate(cs);
>>>   -        mpol_rebind_mm(mm, &cs->mems_allowed);
>>> +        mpol_rebind_mm(mm, &cs->effective_mems);
>>>           if (migrate)
>>>               cpuset_migrate_mm(mm, &cs->old_mems_allowed, &newmems);
>>>           else
>>> @@ -3148,17 +3161,18 @@ static void cpuset_attach(struct
>>> cgroup_taskset *tset)
>>>         /*
>>>        * In the default hierarchy, enabling cpuset in the child cgroups
>>> -     * will trigger a number of cpuset_attach() calls with no change
>>> -     * in effective cpus and mems. In that case, we can optimize out
>>> -     * by skipping the task iteration and update.
>>> +     * will trigger a cpuset_attach() call with no change in
>>> effective cpus
>>> +     * and mems. In that case, we can optimize out by skipping the task
>>> +     * iteration and update.
>>>        */
>>> -    if (cpuset_v2() && !cpus_updated && !mems_updated) {
>>> +    if (cpuset_v2()) {
>>>           cpuset_attach_nodemask_to = cs->effective_mems;
>>> -        goto out;
>>> +        if (!cpus_updated && !mems_updated)
>>> +            goto out;
>>> +    } else {
>>> +        guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
>>>       }
>>>   -    guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
>>> -
>> Nit.
>>
>> I prefer:
>>
>>     guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
>>     if (cpuset_v2() && !cpus_updated && !mems_updated)
>>         goto out;
> 
> This is fixed in patch 7. I may have to send a new version of the patch
> 7 integrated back into individual patches.
> 
> Cheers,
> Longman
> 
>>>       cgroup_taskset_for_each(task, css, tset)
>>>           cpuset_attach_task(cs, task);
>>>   @@ -3168,7 +3182,6 @@ static void cpuset_attach(struct
>>> cgroup_taskset *tset)
>>>        * if there is no change in effective_mems and
>>> CS_MEMORY_MIGRATE is
>>>        * not set.
>>>        */
>>> -    cpuset_attach_nodemask_to = cs->effective_mems;
>>>       if (!is_memory_migrate(cs) && !mems_updated)
>>>           goto out;
>>>   @@ -3176,7 +3189,7 @@ static void cpuset_attach(struct
>>> cgroup_taskset *tset)
>>>           struct mm_struct *mm = get_task_mm(leader);
>>>             if (mm) {
>>> -            mpol_rebind_mm(mm, &cpuset_attach_nodemask_to);
>>> +            mpol_rebind_mm(mm, &cs->effective_mems);
>>>                 /*
>>>                * old_mems_allowed is the same with mems_allowed
>> Other than that, looks good to me.
>>
>> Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
>>
> 

-- 
Best regards,
Ridong