Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and updating to be called once

Byungchul Park
2015-11-17T00:44:28Z

On Mon, Nov 16, 2015 at 01:53:51PM +0100, Peter Zijlstra wrote:
> On Wed, Nov 11, 2015 at 08:51:47AM +0900, Byungchul Park wrote:
> > On Tue, Nov 10, 2015 at 01:16:47PM +0100, Peter Zijlstra wrote:
> > > So the problem is that as soon as that ->cpu store comes through, the
> > > other rq->lock can happen, even though we might still hold a rq->lock
> > > thinking we're serialized.
> > >
> > > Take for instance move_queued_tasks(), it does:
> > >
> > > dequeue_task(rq, p, 0);
> > > p->on_rq = TASK_ON_RQ_MIGRATING;
> > > set_task_cpu(p, new_cpu) {
> > > __set_task_cpu();
> > >
> > > ^^^ here holding rq->lock is insufficient and the below:
> > >
> > > p->sched_class->migrate_task_rq()
> >
> > Thank you for explaning in detail, but this's why i asked you.
>
> > Yes, rq->lock is insufficient in this place as you said, but
> > should migrate_task_rq() be serialized by rq->lock? I might have
> > agreed with you if the migrate_task_rq() should be serialized by
> > rq->lock, but I think it's not the case. I think it would be of
> > if task->pi_lock can work correcly within *if statement* in
> > set_task_cpu(). Wrong?
>
> So currently, set_task_cpu() is serialized by:
>
> - p->pi_lock; on wakeup
> - rq->lock; otherwise
>
> (see the #ifdef CONFIG_LOCKDEP comment in set_task_cpu())

I already read the comment.. Then do you mean the comment above
migrate_task_rq_fair() is wrong and should be fixed? I thought the
comment above migrate_task_rq_fair() is correct rather than
CONFIG_LOCKDEP comment in set_task_cpu(), when I read it. I think
these two comments are conflict each other a little bit, so one of
those should be fixed.

* the comment above migrate_task_rq_fair() describes it like,
Caller SHOULD HOLD (&p->pi_lock)

* the CONFIG_LOCKDEP comment in set_task_cpu() describes it like,
Caller SHOULD HOLD (&p->pi_lock || &rq->lock)

>
> This means that sched_class::migrate_task() cannot indeed rely on
> rq->lock for full serialization, however it still means that
> task_rq_lock() will fully serialize against the thing.

Yes I also think this is true.

>
> By changing this, it no longer will.

???

>
> Even without that; I think such a change, if correct, is very fragile
> and prone to creating problems later on, and sets bad precedent.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/