From: Zefan Li <lizefan-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
To: Tetsuo Handa
<penguin-kernel-JPay3/Yim36HaxMnTkn67Xf5DAMn2ifp@public.gmane.org>
Cc: tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
miaox-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org,
peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org,
mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org,
cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
fernando_b1-Zyj7fXuS5i5L9jVzuh4AOg@public.gmane.org
Subject: Re: Racy manipulation of task_struct->flags in cgroups code causes hard to reproduce kernel panics
Date: Mon, 22 Sep 2014 10:15:50 +0800 [thread overview]
Message-ID: <541F8656.2030409@huawei.com> (raw)
In-Reply-To: <201409211415.GJG26578.MFQOHtSFVJLOOF-JPay3/Yim36HaxMnTkn67Xf5DAMn2ifp@public.gmane.org>
>> On Sat, Sep 20, 2014 at 01:55:54PM +0800, Zefan Li wrote:
>>>> Then, what made current->flags to unexpectedly preserve PF_USED_MATH flag?
>>>> The user is running cgrulesengd process in order to utilize cpuset cgroup.
>>>> Thus, cpuset_update_task_spread_flag() is called when cgrulesengd process
>>>> writes someone's pid to /cgroup/cpuset/$group/tasks interface.
>>>>
>>>> cpuset_update_task_spread_flag() is updating other thread's
>>>> "struct task_struct"->flags without exclusion control or atomic
>>>> operations!
>>>>
>>>> ---------- linux-2.6.32-358.23.2.el6/kernel/cpuset.c ----------
>>>> 300:/*
>>>> 301: * update task's spread flag if cpuset's page/slab spread flag is set
>>>> 302: *
>>>> 303: * Called with callback_mutex/cgroup_mutex held
>>>> 304: */
>>>> 305:static void cpuset_update_task_spread_flag(struct cpuset *cs,
>>>> 306: struct task_struct *tsk)
>>>> 307:{
>>>> 308: if (is_spread_page(cs))
>>>> 309: tsk->flags |= PF_SPREAD_PAGE;
>>>> 310: else
>>>> 311: tsk->flags &= ~PF_SPREAD_PAGE;
>>>> 312: if (is_spread_slab(cs))
>>>> 313: tsk->flags |= PF_SPREAD_SLAB;
>>>> 314: else
>>>> 315: tsk->flags &= ~PF_SPREAD_SLAB;
>>>> 316:}
>>>
>>> We should make the updating of this flag atomic.
>>
>> Ugh, why do we even implement that in cpuset. This should be handled
>> by MPOL_INTERLEAVE. It feels like people have been using cpuset as
>> the dumpsite that people used w/o thinking much. Going forward, let's
>> please confine cpuset to collective cpu and memory affinity
>> configuration. It really shouldn't be implementing novel features for
>> scheduler or mm.
>>
>> Anyways, yeah, the patch looks correct to me. Can you please send a
>> version w/ proper description and sob?
>>
>
> This race condition exists since commit 950592f7b991 ("cpusets: update
> tasks' page/slab spread flags in time") (i.e. Linux 2.6.31 and later)
> but "struct task_struct"->atomic_flags was added in Linux 3.17.
>
> If use of ->atomic_flags for cpuset is acceptable, how should we fix
> older kernels? Backport ->atomic_flags?
>
Yeah, we'll just add tsk->atomic_flags to struct task_struct when
backporting this patch for stable trees.
WARNING: multiple messages have this Message-ID (diff)
From: Zefan Li <lizefan@huawei.com>
To: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: <tj@kernel.org>, <miaox@cn.fujitsu.com>, <peterz@infradead.org>,
<mingo@redhat.com>, <akpm@linux-foundation.org>,
<cgroups@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<fernando_b1@lab.ntt.co.jp>
Subject: Re: Racy manipulation of task_struct->flags in cgroups code causes hard to reproduce kernel panics
Date: Mon, 22 Sep 2014 10:15:50 +0800 [thread overview]
Message-ID: <541F8656.2030409@huawei.com> (raw)
In-Reply-To: <201409211415.GJG26578.MFQOHtSFVJLOOF@I-love.SAKURA.ne.jp>
>> On Sat, Sep 20, 2014 at 01:55:54PM +0800, Zefan Li wrote:
>>>> Then, what made current->flags to unexpectedly preserve PF_USED_MATH flag?
>>>> The user is running cgrulesengd process in order to utilize cpuset cgroup.
>>>> Thus, cpuset_update_task_spread_flag() is called when cgrulesengd process
>>>> writes someone's pid to /cgroup/cpuset/$group/tasks interface.
>>>>
>>>> cpuset_update_task_spread_flag() is updating other thread's
>>>> "struct task_struct"->flags without exclusion control or atomic
>>>> operations!
>>>>
>>>> ---------- linux-2.6.32-358.23.2.el6/kernel/cpuset.c ----------
>>>> 300:/*
>>>> 301: * update task's spread flag if cpuset's page/slab spread flag is set
>>>> 302: *
>>>> 303: * Called with callback_mutex/cgroup_mutex held
>>>> 304: */
>>>> 305:static void cpuset_update_task_spread_flag(struct cpuset *cs,
>>>> 306: struct task_struct *tsk)
>>>> 307:{
>>>> 308: if (is_spread_page(cs))
>>>> 309: tsk->flags |= PF_SPREAD_PAGE;
>>>> 310: else
>>>> 311: tsk->flags &= ~PF_SPREAD_PAGE;
>>>> 312: if (is_spread_slab(cs))
>>>> 313: tsk->flags |= PF_SPREAD_SLAB;
>>>> 314: else
>>>> 315: tsk->flags &= ~PF_SPREAD_SLAB;
>>>> 316:}
>>>
>>> We should make the updating of this flag atomic.
>>
>> Ugh, why do we even implement that in cpuset. This should be handled
>> by MPOL_INTERLEAVE. It feels like people have been using cpuset as
>> the dumpsite that people used w/o thinking much. Going forward, let's
>> please confine cpuset to collective cpu and memory affinity
>> configuration. It really shouldn't be implementing novel features for
>> scheduler or mm.
>>
>> Anyways, yeah, the patch looks correct to me. Can you please send a
>> version w/ proper description and sob?
>>
>
> This race condition exists since commit 950592f7b991 ("cpusets: update
> tasks' page/slab spread flags in time") (i.e. Linux 2.6.31 and later)
> but "struct task_struct"->atomic_flags was added in Linux 3.17.
>
> If use of ->atomic_flags for cpuset is acceptable, how should we fix
> older kernels? Backport ->atomic_flags?
>
Yeah, we'll just add tsk->atomic_flags to struct task_struct when
backporting this patch for stable trees.
next prev parent reply other threads:[~2014-09-22 2:15 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-19 11:53 Racy manipulation of task_struct->flags in cgroups code causes hard to reproduce kernel panics Tetsuo Handa
2014-09-19 11:53 ` Tetsuo Handa
[not found] ` <201409192053.IHJ35462.JLOMOSOFFVtQFH-JPay3/Yim36HaxMnTkn67Xf5DAMn2ifp@public.gmane.org>
2014-09-19 22:21 ` Peter Zijlstra
2014-09-19 22:21 ` Peter Zijlstra
2014-09-20 5:55 ` Zefan Li
2014-09-20 5:55 ` Zefan Li
2014-09-20 10:40 ` [PATCH 3.17-rc5] Fix confusing PFA_NO_NEW_PRIVS constant Tetsuo Handa
2014-09-20 10:40 ` Tetsuo Handa
2014-09-20 10:40 ` Tetsuo Handa
[not found] ` <201409201940.AHG21834.LJOFFHSFQOtVMO-JPay3/Yim36HaxMnTkn67Xf5DAMn2ifp@public.gmane.org>
2014-09-20 17:19 ` Kees Cook
2014-09-20 17:19 ` Kees Cook
2014-09-20 17:19 ` Kees Cook
[not found] ` <541D16EA.70407-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2014-09-20 14:30 ` Racy manipulation of task_struct->flags in cgroups code causes hard to reproduce kernel panics Peter Zijlstra
2014-09-20 14:30 ` Peter Zijlstra
[not found] ` <20140920143012.GL2832-IIpfhp3q70wB9AHHLWeGtNQXobZC6xk2@public.gmane.org>
2014-09-20 17:15 ` Kees Cook
2014-09-20 17:15 ` Kees Cook
[not found] ` <CAGXu5j+P_kcgZuqYsemgL0KU_zRhz5HGJ6seh2oLyyst=cZP6g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-20 18:04 ` Peter Zijlstra
2014-09-20 18:04 ` Peter Zijlstra
2014-09-20 17:28 ` Tejun Heo
2014-09-20 17:28 ` Tejun Heo
[not found] ` <20140920172819.GD3681-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-09-21 5:15 ` Tetsuo Handa
2014-09-21 5:15 ` Tetsuo Handa
[not found] ` <201409211415.GJG26578.MFQOHtSFVJLOOF-JPay3/Yim36HaxMnTkn67Xf5DAMn2ifp@public.gmane.org>
2014-09-22 2:15 ` Zefan Li [this message]
2014-09-22 2:15 ` Zefan Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=541F8656.2030409@huawei.com \
--to=lizefan-hv44wf8li93qt0dzr+alfa@public.gmane.org \
--cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
--cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=fernando_b1-Zyj7fXuS5i5L9jVzuh4AOg@public.gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=miaox-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org \
--cc=mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
--cc=penguin-kernel-JPay3/Yim36HaxMnTkn67Xf5DAMn2ifp@public.gmane.org \
--cc=peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org \
--cc=tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.