From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [91.218.175.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E80436B071 for ; Sun, 7 Jun 2026 03:30:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.179 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780803014; cv=none; b=VXYAPYAeAOh9mpw8PFLbWUG6d+blYoGHNsZYI6pOQpyHbkzybrccQkM0tWk1TttqBdOHjAUjX+Z/6Vswm++SVM2yHLP2Dnzq+WzcXtTqKAZVs6TzS/mYUE79gWD8y47ApF5wWacLWkTcakFELL7i/8lC/RtVE1cPg1QA226IFGc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780803014; c=relaxed/simple; bh=KnNaR6okz+2A7mcHPvtjC4xYzyKl6MgB9eXV/4NyuEY=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=hkpu0VH63DvW5hbBQZInVaixL7Eh45TGwkH17PwjQQJHgoFxTQQCZne722xRP+vPqBcXG0fFjg5bmGCpKhqKuYDQT4Hbia02Pe+9dpVJ35WFnpmrkgNqbzabTDkjiA6qBEyq0GUi79oC4yeVHtKpXbR8vrUnhiYfSy4dxP1j1fA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=j+adVxbI; arc=none smtp.client-ip=91.218.175.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="j+adVxbI" Message-ID: <2e50dea5-df07-4f89-9dbb-ad75f14cf9bf@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1780802999; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LOfNhxoJCV/m4rawUudY1tjBBTZEhXZEyPB1+8Fx27Q=; b=j+adVxbI7pa/SE2RAM/vqR88omaYN/rfCs1bL/Or7Ww9zAcAON507yVW+EuFERSd75qw7u +PpSsryJun5VH08nbE3ru+Ctklj38F7R9v7rSevFJcLxJA3TFhwnPLErBcOz8zOrgaapBu dUoR17aEpekXbbsxX7XJcaWgRJRX4ew= Date: Sun, 7 Jun 2026 11:29:53 +0800 Precedence: bulk X-Mailing-List: cgroups@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Subject: Re: [PATCH-next v6 1/6] cgroup/cpuset: Fix node inconsistencies between cpuset_update_tasks_nodemask() and cpuset_attach() To: Waiman Long , Tejun Heo , Johannes Weiner , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Peter Zijlstra Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Aaron Tomlin , Guopeng Zhang References: <20260604150229.414135-1-longman@redhat.com> <20260604150229.414135-2-longman@redhat.com> Content-Language: en-US X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Ridong Chen In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT On 6/6/2026 1:09 AM, Waiman Long wrote: > On 6/5/26 3:48 AM, Ridong Chen wrote: >> >> On 6/4/2026 11:02 PM, Waiman Long wrote: >>> Whenever memory node mask is changed, there are 4 places where the node >>> mask has to be updated or used. >>>   1) task's node mask via cpuset_change_task_nodemask() >>>   2) memory policy binding via mpol_rebind_mm() >>>   3) if memory migration is enabled, migrate from old_mems_allowed to >>>      the new node mask via cpuset_migrate_mm(). >>>   4) setting old_mems_allowed >>> >>> These memory actions are done in cpuset_update_tasks_nodemask() and >>> cpuset_attach(). However there are inconsistencies in what node masks >>> are being used in these 2 functions. >>> >>> In cpuset_update_tasks_nodemask(), >>>   - cpuset_change_task_nodemask(): guarantee_online_mems() >>>   - mpol_rebind_mm(): mems_allowed >>>   - cpuset_migrate_mm(): guarantee_online_mems() >>>   - old_mems_allowed: guarantee_online_mems() >>> >>> In cpuset_attach(), >>>   - cpuset_change_task_nodemask(): guarantee_online_mems() >>>   - mpol_rebind_mm(): effective_mems >>>   - cpuset_migrate_mm(): effective_mems >>>   - old_mems_allowed: effective_mems >>> >>> These inconsistencies dates back to quite a long time ago and it is >>> hard to say what should be the correct values. >>> >>> The guarantee_online_mems() function returns a node mask from current or >>> an ancestor cpuset that is a subset of node_states[N_MEMORY]. Nodes in >>> node_states[N_MEMORY] are all online, i.e. in node_states[N_ONLINE]. >>> However, node in node_states[N_ONLINE] may not have memory. So >>> node_states[N_MEMORY] should be a subset of node_states[N_ONLINE]. >>> >>> The guarantee_online_mems() function should only be useful for v1 where >>> mems_allowed is the same as effective_mems. With v2, the memory nodes >>> in effective_mems should always be a subset of node_states[N_MEMORY]. >>> The only time that may not be true is when a memory hot-unplug operation >>> is in progress and a memory node is removed from node_states[N_MEMORY] >>> but not yet reflected in effective_mems as cpuset_handle_hotplug() >>> has not yet been called from cpuset_track_online_nodes(). When >>> cpuset_handle_hotplug() is called later, the memory node setting >>> of the relevant cpusets and tasks will be updated. So replacing the >>> guarantee_online_mems() call by just using cs->effective_mems should >>> be fine. >>> >> The message should be updated. > Could you be more specific about what message are you referring to? After this patch, guarantee_online_mems() for v2 simply returns cs->effective_mems directly. So the paragraph: "The guarantee_online_mems() function should only be useful for v1 where mems_allowed is the same as effective_mems." is no longer accurate — guarantee_online_mems() is now useful for both v1 and v2, just with different behavior. For v2 it returns effective_mems directly; for v1 it walks up the hierarchy to find an ancestor with online memory nodes. Similarly, "So replacing the guarantee_online_mems() call by just using cs->effective_mems should be fine" describes the old reasoning, but the patch actually keeps guarantee_online_mems() calls in place and makes the function itself return effective_mems for v2. The commit message should reflect what the patch actually does. >> >>> Let use the following setup for both of them and make them consistent. >>>   - cpuset_change_task_nodemask(): guarantee_online_mems() >>>   - mpol_rebind_mm(): effective_mems >>>   - cpuset_migrate_mm(): guarantee_online_mems() >>>   - old_mems_allowed: guarantee_online_mems() >>> >>> So for v2, it is effectively all effective_mems. For v1, >>> mpol_rebind_mm() >>> uses mems_allowed which may differ from what guarantee_online_mems() >>> returns. >>> >>> Signed-off-by: Waiman Long >>> --- >>>   kernel/cgroup/cpuset.c | 37 +++++++++++++++++++++++++------------ >>>   1 file changed, 25 insertions(+), 12 deletions(-) >>> >>> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c >>> index 6bdb68689c24..8305b5830c3c 100644 >>> --- a/kernel/cgroup/cpuset.c >>> +++ b/kernel/cgroup/cpuset.c >>> @@ -489,7 +489,10 @@ static void guarantee_active_cpus(struct >>> task_struct *tsk, >>>    * Return in *pmask the portion of a cpusets's mems_allowed that >>>    * are online, with memory.  If none are online with memory, walk >>>    * up the cpuset hierarchy until we find one that does have some >>> - * online mems.  The top cpuset always has some mems online. >>> + * online mems.  The top cpuset always has some mems online. With v2, >>> + * effective_mems should always contain online memory nodes except >>> + * during the transition period where a memory node hotunplug operation >>> + * is in progress. >>>    * >>>    * One way or another, we guarantee to return some non-empty subset >>>    * of node_states[N_MEMORY]. >>> @@ -498,6 +501,10 @@ static void guarantee_active_cpus(struct >>> task_struct *tsk, >>>    */ >>>   static void guarantee_online_mems(struct cpuset *cs, nodemask_t >>> *pmask) >>>   { >>> +    if (cpuset_v2()) { >>> +        *pmask = cs->effective_mems; >>> +        return; >>> +    } >>>       while (!nodes_and(*pmask, cs->effective_mems, >>> node_states[N_MEMORY])) >>>           cs = parent_cs(cs); >>>   } >>> @@ -2616,6 +2623,13 @@ static void *cpuset_being_rebound; >>>    * Iterate through each task of @cs updating its mems_allowed to the >>>    * effective cpuset's.  As this function is called with >>> cpuset_mutex held, >>>    * cpuset membership stays stable. >>> + * >>> + * - cpuset_change_task_nodemask(): guarantee_online_mems() >>> + * - mpol_rebind_mm(): effective_mems >>> + * - cpuset_migrate_mm(): guarantee_online_mems() >>> + * - old_mems_allowed: guarantee_online_mems() >>> + * >>> + * For v2, guarantee_online_mems() should just return effective_mems. >>>    */ >>>   void cpuset_update_tasks_nodemask(struct cpuset *cs) >>>   { >>> @@ -2624,7 +2638,6 @@ void cpuset_update_tasks_nodemask(struct cpuset >>> *cs) >>>       struct task_struct *task; >>>         cpuset_being_rebound = cs;        /* causes mpol_dup() rebind */ >>> - >>>       guarantee_online_mems(cs, &newmems); >>>         /* >>> @@ -2650,7 +2663,7 @@ void cpuset_update_tasks_nodemask(struct cpuset >>> *cs) >>>             migrate = is_memory_migrate(cs); >>>   -        mpol_rebind_mm(mm, &cs->mems_allowed); >>> +        mpol_rebind_mm(mm, &cs->effective_mems); >>>           if (migrate) >>>               cpuset_migrate_mm(mm, &cs->old_mems_allowed, &newmems); >>>           else >>> @@ -3148,17 +3161,18 @@ static void cpuset_attach(struct >>> cgroup_taskset *tset) >>>         /* >>>        * In the default hierarchy, enabling cpuset in the child cgroups >>> -     * will trigger a number of cpuset_attach() calls with no change >>> -     * in effective cpus and mems. In that case, we can optimize out >>> -     * by skipping the task iteration and update. >>> +     * will trigger a cpuset_attach() call with no change in >>> effective cpus >>> +     * and mems. In that case, we can optimize out by skipping the task >>> +     * iteration and update. >>>        */ >>> -    if (cpuset_v2() && !cpus_updated && !mems_updated) { >>> +    if (cpuset_v2()) { >>>           cpuset_attach_nodemask_to = cs->effective_mems; >>> -        goto out; >>> +        if (!cpus_updated && !mems_updated) >>> +            goto out; >>> +    } else { >>> +        guarantee_online_mems(cs, &cpuset_attach_nodemask_to); >>>       } >>>   -    guarantee_online_mems(cs, &cpuset_attach_nodemask_to); >>> - >> Nit. >> >> I prefer: >> >>     guarantee_online_mems(cs, &cpuset_attach_nodemask_to); >>     if (cpuset_v2() && !cpus_updated && !mems_updated) >>         goto out; > > This is fixed in patch 7. I may have to send a new version of the patch > 7 integrated back into individual patches. > > Cheers, > Longman > >>>       cgroup_taskset_for_each(task, css, tset) >>>           cpuset_attach_task(cs, task); >>>   @@ -3168,7 +3182,6 @@ static void cpuset_attach(struct >>> cgroup_taskset *tset) >>>        * if there is no change in effective_mems and >>> CS_MEMORY_MIGRATE is >>>        * not set. >>>        */ >>> -    cpuset_attach_nodemask_to = cs->effective_mems; >>>       if (!is_memory_migrate(cs) && !mems_updated) >>>           goto out; >>>   @@ -3176,7 +3189,7 @@ static void cpuset_attach(struct >>> cgroup_taskset *tset) >>>           struct mm_struct *mm = get_task_mm(leader); >>>             if (mm) { >>> -            mpol_rebind_mm(mm, &cpuset_attach_nodemask_to); >>> +            mpol_rebind_mm(mm, &cs->effective_mems); >>>                 /* >>>                * old_mems_allowed is the same with mems_allowed >> Other than that, looks good to me. >> >> Reviewed-by: Ridong Chen >> > -- Best regards, Ridong