[PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Waiman Long]

The nr_deadline_tasks field in cpuset structure was introduced by
commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
cpuset_mutex, nr_deadline_tasks can be updated under two different
locks - cpuset_mutex in most cases or css_set_lock in cgroup_exit(). As
a result, data races can happen leading to incorrect nr_deadline_tasks
value.

Since it is not practical to somehow take cpuset_mutex in cgroup_exit(),
the easy way out to avoid this possible race condition is by making
nr_deadline_tasks an atomic_t value.

Fixes: 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task in cpusets")
Reported-by: Xia Fukun <xiafukun@huawei.com>
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 58ec88efa4f8..3f3da468f058 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -174,7 +174,7 @@ struct cpuset {
 	 * number of SCHED_DEADLINE tasks attached to this cpuset, so that we
 	 * know when to rebuild associated root domain bandwidth information.
 	 */
-	int nr_deadline_tasks;
+	atomic_t nr_deadline_tasks;
 	int nr_migrate_dl_tasks;
 	u64 sum_migrate_dl_bw;
 
@@ -234,14 +234,14 @@ void inc_dl_tasks_cs(struct task_struct *p)
 {
 	struct cpuset *cs = task_cs(p);
 
-	cs->nr_deadline_tasks++;
+	atomic_inc(&cs->nr_deadline_tasks);
 }
 
 void dec_dl_tasks_cs(struct task_struct *p)
 {
 	struct cpuset *cs = task_cs(p);
 
-	cs->nr_deadline_tasks--;
+	atomic_dec(&cs->nr_deadline_tasks);
 }
 
 /* bits in struct cpuset flags field */
@@ -1071,7 +1071,7 @@ static void dl_update_tasks_root_domain(struct cpuset *cs)
 	struct css_task_iter it;
 	struct task_struct *task;
 
-	if (cs->nr_deadline_tasks == 0)
+	if (atomic_read(&cs->nr_deadline_tasks) == 0)
 		return;
 
 	css_task_iter_start(&cs->css, 0, &it);
@@ -2721,8 +2721,8 @@ static void cpuset_attach(struct cgroup_taskset *tset)
 	cs->old_mems_allowed = cpuset_attach_nodemask_to;
 
 	if (cs->nr_migrate_dl_tasks) {
-		cs->nr_deadline_tasks += cs->nr_migrate_dl_tasks;
-		oldcs->nr_deadline_tasks -= cs->nr_migrate_dl_tasks;
+		atomic_add(cs->nr_migrate_dl_tasks, &cs->nr_deadline_tasks);
+		atomic_sub(cs->nr_migrate_dl_tasks, &oldcs->nr_deadline_tasks);
 		reset_migrate_dl_data(cs);
 	}
 
-- 
2.39.3

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Michal Koutný]

[-- Attachment #1: Type: text/plain, Size: 1125 bytes --]

On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long <longman@redhat.com> wrote:
> The nr_deadline_tasks field in cpuset structure was introduced by
> commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
> in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
> cpuset_mutex, nr_deadline_tasks can be updated under two different
> locks - cpuset_mutex in most cases or css_set_lock in cgroup_exit(). As
> a result, data races can happen leading to incorrect nr_deadline_tasks
> value.

The effect is that dl_update_tasks_root_domain() processes tasks
unnecessarily or that it incorrectly skips dl_add_task_root_domain()?

> Since it is not practical to somehow take cpuset_mutex in cgroup_exit(),
> the easy way out to avoid this possible race condition is by making
> nr_deadline_tasks an atomic_t value.

If css_set_lock is useless for this fields and it's going to be atomic,
could you please add (presumably) a cleanup that moves dec_dl_tasks_cs()
from under css_set_lock in cgroup_exit() to a (new but specific)
cpuset_cgrp_subsys.exit() handler?

Thanks,
Michal

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Waiman Long]

On 11/1/23 12:34, Michal Koutný wrote:
> On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long <longman@redhat.com> wrote:
>> The nr_deadline_tasks field in cpuset structure was introduced by
>> commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
>> in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
>> cpuset_mutex, nr_deadline_tasks can be updated under two different
>> locks - cpuset_mutex in most cases or css_set_lock in cgroup_exit(). As
>> a result, data races can happen leading to incorrect nr_deadline_tasks
>> value.
> The effect is that dl_update_tasks_root_domain() processes tasks
> unnecessarily or that it incorrectly skips dl_add_task_root_domain()?
The effect is that dl_update_tasks_root_domain() may return incorrectly 
or it is doing unnecessary work. Will update the commit log to reflect that.
>
>> Since it is not practical to somehow take cpuset_mutex in cgroup_exit(),
>> the easy way out to avoid this possible race condition is by making
>> nr_deadline_tasks an atomic_t value.
> If css_set_lock is useless for this fields and it's going to be atomic,
> could you please add (presumably) a cleanup that moves dec_dl_tasks_cs()
> from under css_set_lock in cgroup_exit() to a (new but specific)
> cpuset_cgrp_subsys.exit() handler?

But css_set_lock is needed for updating other css data. It is true that 
we can move dec_dl_tasks_cs() outside of the lock. I can do that in the 
next version.

Cheers,
Longman

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Juri Lelli]

Hi Waiman,

On 01/11/23 13:59, Waiman Long wrote:
> On 11/1/23 12:34, Michal Koutný wrote:
> > On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long <longman@redhat.com> wrote:
> > > The nr_deadline_tasks field in cpuset structure was introduced by
> > > commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
> > > in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
> > > cpuset_mutex, nr_deadline_tasks can be updated under two different
> > > locks - cpuset_mutex in most cases or css_set_lock in cgroup_exit(). As
> > > a result, data races can happen leading to incorrect nr_deadline_tasks
> > > value.
> > The effect is that dl_update_tasks_root_domain() processes tasks
> > unnecessarily or that it incorrectly skips dl_add_task_root_domain()?
> The effect is that dl_update_tasks_root_domain() may return incorrectly or
> it is doing unnecessary work. Will update the commit log to reflect that.
> > 
> > > Since it is not practical to somehow take cpuset_mutex in cgroup_exit(),
> > > the easy way out to avoid this possible race condition is by making
> > > nr_deadline_tasks an atomic_t value.
> > If css_set_lock is useless for this fields and it's going to be atomic,
> > could you please add (presumably) a cleanup that moves dec_dl_tasks_cs()
> > from under css_set_lock in cgroup_exit() to a (new but specific)
> > cpuset_cgrp_subsys.exit() handler?
> 
> But css_set_lock is needed for updating other css data. It is true that we
> can move dec_dl_tasks_cs() outside of the lock. I can do that in the next
> version.

Not sure if you had a chance to check my last question/comment on your
previous posting?

https://lore.kernel.org/lkml/ZSjfBWgZf15TchA5@localhost.localdomain/

Thanks,
Juri

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Waiman Long]

On 11/2/23 06:26, Juri Lelli wrote:
> Hi Waiman,
>
> On 01/11/23 13:59, Waiman Long wrote:
>> On 11/1/23 12:34, Michal Koutný wrote:
>>> On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long <longman@redhat.com> wrote:
>>>> The nr_deadline_tasks field in cpuset structure was introduced by
>>>> commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
>>>> in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
>>>> cpuset_mutex, nr_deadline_tasks can be updated under two different
>>>> locks - cpuset_mutex in most cases or css_set_lock in cgroup_exit(). As
>>>> a result, data races can happen leading to incorrect nr_deadline_tasks
>>>> value.
>>> The effect is that dl_update_tasks_root_domain() processes tasks
>>> unnecessarily or that it incorrectly skips dl_add_task_root_domain()?
>> The effect is that dl_update_tasks_root_domain() may return incorrectly or
>> it is doing unnecessary work. Will update the commit log to reflect that.
>>>> Since it is not practical to somehow take cpuset_mutex in cgroup_exit(),
>>>> the easy way out to avoid this possible race condition is by making
>>>> nr_deadline_tasks an atomic_t value.
>>> If css_set_lock is useless for this fields and it's going to be atomic,
>>> could you please add (presumably) a cleanup that moves dec_dl_tasks_cs()
>>> from under css_set_lock in cgroup_exit() to a (new but specific)
>>> cpuset_cgrp_subsys.exit() handler?
>> But css_set_lock is needed for updating other css data. It is true that we
>> can move dec_dl_tasks_cs() outside of the lock. I can do that in the next
>> version.
> Not sure if you had a chance to check my last question/comment on your
> previous posting?
>
> https://lore.kernel.org/lkml/ZSjfBWgZf15TchA5@localhost.localdomain/

Thanks for the reminder. I look at your comment again. Even though 
dl_rebuild_rd_accounting() operates on css(es) via css_task_iter_start() 
and css_task_iter_next(), the css_set_lock is released at the end of it. 
So it is still possible that a task can call cgroup_exit() after 
css_task_iter_next() and is being processed by 
dl_add_task_root_domain(). Is there a helper in the do_exit() path to 
nullify the dl_task() check. Or maybe we can also check for PF_EXITING 
in dl_add_task_root_domain() under the pi_lock and do the dl_task() 
check the under pi_lock to synchronize with dl_add_task_root_domain(). 
What do you think?

I still believe that it doesn't really matter if we call 
dec_dl_tasks_cs() inside or outside the css_set_lock.

Cheers,
Longman

Cheers,
Longman

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Waiman Long]

On 11/2/23 09:01, Waiman Long wrote:
>
> On 11/2/23 06:26, Juri Lelli wrote:
>> Hi Waiman,
>>
>> On 01/11/23 13:59, Waiman Long wrote:
>>> On 11/1/23 12:34, Michal Koutný wrote:
>>>> On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long 
>>>> <longman@redhat.com> wrote:
>>>>> The nr_deadline_tasks field in cpuset structure was introduced by
>>>>> commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
>>>>> in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
>>>>> cpuset_mutex, nr_deadline_tasks can be updated under two different
>>>>> locks - cpuset_mutex in most cases or css_set_lock in 
>>>>> cgroup_exit(). As
>>>>> a result, data races can happen leading to incorrect 
>>>>> nr_deadline_tasks
>>>>> value.
>>>> The effect is that dl_update_tasks_root_domain() processes tasks
>>>> unnecessarily or that it incorrectly skips dl_add_task_root_domain()?
>>> The effect is that dl_update_tasks_root_domain() may return 
>>> incorrectly or
>>> it is doing unnecessary work. Will update the commit log to reflect 
>>> that.
>>>>> Since it is not practical to somehow take cpuset_mutex in 
>>>>> cgroup_exit(),
>>>>> the easy way out to avoid this possible race condition is by making
>>>>> nr_deadline_tasks an atomic_t value.
>>>> If css_set_lock is useless for this fields and it's going to be 
>>>> atomic,
>>>> could you please add (presumably) a cleanup that moves 
>>>> dec_dl_tasks_cs()
>>>> from under css_set_lock in cgroup_exit() to a (new but specific)
>>>> cpuset_cgrp_subsys.exit() handler?
>>> But css_set_lock is needed for updating other css data. It is true 
>>> that we
>>> can move dec_dl_tasks_cs() outside of the lock. I can do that in the 
>>> next
>>> version.
>> Not sure if you had a chance to check my last question/comment on your
>> previous posting?
>>
>> https://lore.kernel.org/lkml/ZSjfBWgZf15TchA5@localhost.localdomain/
>
> Thanks for the reminder. I look at your comment again. Even though 
> dl_rebuild_rd_accounting() operates on css(es) via 
> css_task_iter_start() and css_task_iter_next(), the css_set_lock is 
> released at the end of it. So it is still possible that a task can 
> call cgroup_exit() after css_task_iter_next() and is being processed 
> by dl_add_task_root_domain(). Is there a helper in the do_exit() path 
> to nullify the dl_task() check. Or maybe we can also check for 
> PF_EXITING in dl_add_task_root_domain() under the pi_lock and do the 
> dl_task() check the under pi_lock to synchronize with 
> dl_add_task_root_domain(). What do you think?
>
> I still believe that it doesn't really matter if we call 
> dec_dl_tasks_cs() inside or outside the css_set_lock.

Just curious. Does the deadline code remove the deadline quota of an 
exiting task?

Regards,
Longman

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Juri Lelli]

On 02/11/23 14:08, Waiman Long wrote:
> On 11/2/23 09:01, Waiman Long wrote:
> > 
> > On 11/2/23 06:26, Juri Lelli wrote:
> > > Hi Waiman,
> > > 
> > > On 01/11/23 13:59, Waiman Long wrote:
> > > > On 11/1/23 12:34, Michal Koutný wrote:
> > > > > On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long
> > > > > <longman@redhat.com> wrote:
> > > > > > The nr_deadline_tasks field in cpuset structure was introduced by
> > > > > > commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
> > > > > > in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
> > > > > > cpuset_mutex, nr_deadline_tasks can be updated under two different
> > > > > > locks - cpuset_mutex in most cases or css_set_lock in
> > > > > > cgroup_exit(). As
> > > > > > a result, data races can happen leading to incorrect
> > > > > > nr_deadline_tasks
> > > > > > value.
> > > > > The effect is that dl_update_tasks_root_domain() processes tasks
> > > > > unnecessarily or that it incorrectly skips dl_add_task_root_domain()?
> > > > The effect is that dl_update_tasks_root_domain() may return
> > > > incorrectly or
> > > > it is doing unnecessary work. Will update the commit log to
> > > > reflect that.
> > > > > > Since it is not practical to somehow take cpuset_mutex
> > > > > > in cgroup_exit(),
> > > > > > the easy way out to avoid this possible race condition is by making
> > > > > > nr_deadline_tasks an atomic_t value.
> > > > > If css_set_lock is useless for this fields and it's going to
> > > > > be atomic,
> > > > > could you please add (presumably) a cleanup that moves
> > > > > dec_dl_tasks_cs()
> > > > > from under css_set_lock in cgroup_exit() to a (new but specific)
> > > > > cpuset_cgrp_subsys.exit() handler?
> > > > But css_set_lock is needed for updating other css data. It is
> > > > true that we
> > > > can move dec_dl_tasks_cs() outside of the lock. I can do that in
> > > > the next
> > > > version.
> > > Not sure if you had a chance to check my last question/comment on your
> > > previous posting?
> > > 
> > > https://lore.kernel.org/lkml/ZSjfBWgZf15TchA5@localhost.localdomain/
> > 
> > Thanks for the reminder. I look at your comment again. Even though
> > dl_rebuild_rd_accounting() operates on css(es) via css_task_iter_start()
> > and css_task_iter_next(), the css_set_lock is released at the end of it.
> > So it is still possible that a task can call cgroup_exit() after
> > css_task_iter_next() and is being processed by
> > dl_add_task_root_domain(). Is there a helper in the do_exit() path to
> > nullify the dl_task() check. Or maybe we can also check for PF_EXITING
> > in dl_add_task_root_domain() under the pi_lock and do the dl_task()
> > check the under pi_lock to synchronize with dl_add_task_root_domain().
> > What do you think?
> > 
> > I still believe that it doesn't really matter if we call
> > dec_dl_tasks_cs() inside or outside the css_set_lock.
> 
> Just curious. Does the deadline code remove the deadline quota of an exiting
> task?

Ah, interesting observation. We do indeed remove a DL tasks bandwidth
from either within task_non_contending (if zerolag time has passed at
the time the task is dying) or a bit later when the inactive timer fires
(check related paths with TASK_DEAD in task_non_contending and
inactive_task_timer). So, maybe we could do the cs subtraction at this
point as well? Maybe it's even more correct I'm now thinking (or maybe it's
just Friday :).

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Juri Lelli]

On 02/11/23 09:01, Waiman Long wrote:
> 
> On 11/2/23 06:26, Juri Lelli wrote:
> > Hi Waiman,
> > 
> > On 01/11/23 13:59, Waiman Long wrote:
> > > On 11/1/23 12:34, Michal Koutný wrote:
> > > > On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long <longman@redhat.com> wrote:
> > > > > The nr_deadline_tasks field in cpuset structure was introduced by
> > > > > commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
> > > > > in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
> > > > > cpuset_mutex, nr_deadline_tasks can be updated under two different
> > > > > locks - cpuset_mutex in most cases or css_set_lock in cgroup_exit(). As
> > > > > a result, data races can happen leading to incorrect nr_deadline_tasks
> > > > > value.
> > > > The effect is that dl_update_tasks_root_domain() processes tasks
> > > > unnecessarily or that it incorrectly skips dl_add_task_root_domain()?
> > > The effect is that dl_update_tasks_root_domain() may return incorrectly or
> > > it is doing unnecessary work. Will update the commit log to reflect that.
> > > > > Since it is not practical to somehow take cpuset_mutex in cgroup_exit(),
> > > > > the easy way out to avoid this possible race condition is by making
> > > > > nr_deadline_tasks an atomic_t value.
> > > > If css_set_lock is useless for this fields and it's going to be atomic,
> > > > could you please add (presumably) a cleanup that moves dec_dl_tasks_cs()
> > > > from under css_set_lock in cgroup_exit() to a (new but specific)
> > > > cpuset_cgrp_subsys.exit() handler?
> > > But css_set_lock is needed for updating other css data. It is true that we
> > > can move dec_dl_tasks_cs() outside of the lock. I can do that in the next
> > > version.
> > Not sure if you had a chance to check my last question/comment on your
> > previous posting?
> > 
> > https://lore.kernel.org/lkml/ZSjfBWgZf15TchA5@localhost.localdomain/
> 
> Thanks for the reminder. I look at your comment again. Even though
> dl_rebuild_rd_accounting() operates on css(es) via css_task_iter_start() and
> css_task_iter_next(), the css_set_lock is released at the end of it. So it
> is still possible that a task can call cgroup_exit() after
> css_task_iter_next() and is being processed by dl_add_task_root_domain(). Is
> there a helper in the do_exit() path to nullify the dl_task() check. Or
> maybe we can also check for PF_EXITING in dl_add_task_root_domain() under
> the pi_lock and do the dl_task() check the under pi_lock to synchronize with
> dl_add_task_root_domain(). What do you think?
> 
> I still believe that it doesn't really matter if we call dec_dl_tasks_cs()
> inside or outside the css_set_lock.

Hummm, what if we move dec_dl_tasks_cs outside css_set_lock guard in
cgroup_exit and we grab cpuset_mutex (for dl_tasks) before doing the
decrement in there?

Re: [PATCH v2] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t value

[Waiman Long]

On 11/3/23 10:29, Juri Lelli wrote:
> On 02/11/23 09:01, Waiman Long wrote:
>> On 11/2/23 06:26, Juri Lelli wrote:
>>> Hi Waiman,
>>>
>>> On 01/11/23 13:59, Waiman Long wrote:
>>>> On 11/1/23 12:34, Michal Koutný wrote:
>>>>> On Tue, Oct 24, 2023 at 10:18:34AM -0400, Waiman Long <longman@redhat.com> wrote:
>>>>>> The nr_deadline_tasks field in cpuset structure was introduced by
>>>>>> commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
>>>>>> in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
>>>>>> cpuset_mutex, nr_deadline_tasks can be updated under two different
>>>>>> locks - cpuset_mutex in most cases or css_set_lock in cgroup_exit(). As
>>>>>> a result, data races can happen leading to incorrect nr_deadline_tasks
>>>>>> value.
>>>>> The effect is that dl_update_tasks_root_domain() processes tasks
>>>>> unnecessarily or that it incorrectly skips dl_add_task_root_domain()?
>>>> The effect is that dl_update_tasks_root_domain() may return incorrectly or
>>>> it is doing unnecessary work. Will update the commit log to reflect that.
>>>>>> Since it is not practical to somehow take cpuset_mutex in cgroup_exit(),
>>>>>> the easy way out to avoid this possible race condition is by making
>>>>>> nr_deadline_tasks an atomic_t value.
>>>>> If css_set_lock is useless for this fields and it's going to be atomic,
>>>>> could you please add (presumably) a cleanup that moves dec_dl_tasks_cs()
>>>>> from under css_set_lock in cgroup_exit() to a (new but specific)
>>>>> cpuset_cgrp_subsys.exit() handler?
>>>> But css_set_lock is needed for updating other css data. It is true that we
>>>> can move dec_dl_tasks_cs() outside of the lock. I can do that in the next
>>>> version.
>>> Not sure if you had a chance to check my last question/comment on your
>>> previous posting?
>>>
>>> https://lore.kernel.org/lkml/ZSjfBWgZf15TchA5@localhost.localdomain/
>> Thanks for the reminder. I look at your comment again. Even though
>> dl_rebuild_rd_accounting() operates on css(es) via css_task_iter_start() and
>> css_task_iter_next(), the css_set_lock is released at the end of it. So it
>> is still possible that a task can call cgroup_exit() after
>> css_task_iter_next() and is being processed by dl_add_task_root_domain(). Is
>> there a helper in the do_exit() path to nullify the dl_task() check. Or
>> maybe we can also check for PF_EXITING in dl_add_task_root_domain() under
>> the pi_lock and do the dl_task() check the under pi_lock to synchronize with
>> dl_add_task_root_domain(). What do you think?
>>
>> I still believe that it doesn't really matter if we call dec_dl_tasks_cs()
>> inside or outside the css_set_lock.
> Hummm, what if we move dec_dl_tasks_cs outside css_set_lock guard in
> cgroup_exit and we grab cpuset_mutex (for dl_tasks) before doing the
> decrement in there?

It is a possibility, but acquiring a mutex in the exit path may cause it 
to sleep. A dying task in the sleep state may be a problem.

Cheers,
Longman