Re: [PATCH v2 2/5] sched, numa: Document usages of mm->numa_scan_seq

Davidlohr Bueso
2015-04-30T18:54:41Z

On Thu, 2015-04-30 at 14:42 -0400, Waiman Long wrote:
> On 04/29/2015 02:45 PM, Jason Low wrote:
> > On Wed, 2015-04-29 at 14:14 -0400, Waiman Long wrote:
> >> On 04/28/2015 04:00 PM, Jason Low wrote:
> >>> The p->mm->numa_scan_seq is accessed using READ_ONCE/WRITE_ONCE
> >>> and modified without exclusive access. It is not clear why it is
> >>> accessed this way. This patch provides some documentation on that.
> >>>
> >>> Signed-off-by: Jason Low<jason.low2@hp.com>
> >>> ---
> >>> kernel/sched/fair.c | 12 ++++++++++++
> >>> 1 files changed, 12 insertions(+), 0 deletions(-)
> >>>
> >>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> >>> index 5a44371..794f7d7 100644
> >>> --- a/kernel/sched/fair.c
> >>> +++ b/kernel/sched/fair.c
> >>> @@ -1794,6 +1794,11 @@ static void task_numa_placement(struct task_struct *p)
> >>> u64 runtime, period;
> >>> spinlock_t *group_lock = NULL;
> >>>
> >>> + /*
> >>> + * The p->mm->numa_scan_seq gets updated without
> >>> + * exclusive access. Use READ_ONCE() here to ensure
> >>> + * that the field is read in a single access.
> >>> + */
> >>> seq = READ_ONCE(p->mm->numa_scan_seq);
> >>> if (p->numa_scan_seq == seq)
> >>> return;
> >>> @@ -2107,6 +2112,13 @@ void task_numa_fault(int last_cpupid, int mem_node, int pages, int flags)
> >>>
> >>> static void reset_ptenuma_scan(struct task_struct *p)
> >>> {
> >>> + /*
> >>> + * We only did a read acquisition of the mmap sem, so
> >>> + * p->mm->numa_scan_seq is written to without exclusive access.
> >>> + * That's not much of an issue though, since this is just used
> >>> + * for statistical sampling. Use WRITE_ONCE and READ_ONCE, which
> >>> + * are not expensive, to avoid load/store tearing.
> >>> + */
> >>> WRITE_ONCE(p->mm->numa_scan_seq, READ_ONCE(p->mm->numa_scan_seq) + 1);
> >>> p->mm->numa_scan_offset = 0;
> >>> }
> >> READ_ONCE followed by a WRITE_ONCE won't stop load/store tearing from
> >> happening unless you use an atomic instruction to do the increment. So I
> >> think your comment may be a bit misleading.
> > Right, the READ and WRITE operations will still be done separately and
> > won't be atomic. Here, we're saying that this prevents load/store
> > tearing on each of those individual write/read operations. Please let me
> > know if you prefer this to be worded differently.
>
> I do have a question of what kind of tearing you are talking about. Do
> you mean the tearing due to mm being changed in the middle of the
> access? The reason why I don't like this kind of construct is that I am
> not sure if
> the address translation p->mm->numa_scan_seq is being done once or
> twice. I looked at the compiled code and the translation is done only once.
>
> Anyway, the purpose of READ_ONCE and WRITE_ONCE is not for eliminating
> data tearing. They are to make sure that the compiler won't compile away
> data access and they are done in the order they appear in the program. I
> don't think it is a good idea to associate tearing elimination with
> those macros. So I would suggest removing the last sentence in your comment.

I agree. Related, Linus also had some thoughts about the _very specific_
purposes of these macros:
http://www.spinics.net/lists/linux-next/msg32494.html

I also wonder why this patch is included in a set called
"sched, timer: Improve scalability of itimers" ;)

Thanks,
Davidlohr