Re: [PATCH v2 0/5] sched: Add on_rq states and remove several double rq locks

[Oleg Nesterov <oleg@redhat.com>]

Hi Kirill,

I'll try to read this series later, just one silly question for now.

On 07/26, Kirill Tkhai wrote:
>
> Patch [2/5] is main in the series. It introduces new state: ONRQ_MIGRATING
> and teaches scheduler to understand it (we need a little changes predominantly
> in try_to_wake_up()). This will be used in the following way:
>
>         (we are changing task's rq)
>
>         raw_spin_lock(&src_rq->lock);
>         dequeue_task(src_rq, p, 0);
>         p->on_rq = ONRQ_MIGRATING;
>         set_task_cpu(p, dst_cpu);
>         raw_spin_unlock(&src_rq->lock);
>
>         raw_spin_lock(&dst_rq->lock);
>         p->on_rq = ONRQ_QUEUED;
>         enqueue_task(dst_rq, p, 0);
>         raw_spin_unlock(&dst_rq->lock);

Hmm. And what if the code above doesn't hold p->pi_lock (4/5) and, say,
__sched_setscheduler() does fair_sched_class->rt_sched_class transition
in between?

ONRQ_MIGRATING helps to avoid the wrong dequeue + enqueue, but I am not
sure about check_class_changed().

Say, switched_from_fair() will use dst_rq even if p was never queued on
this rq... This only affects the .decay_count logic, perhaps this is fine,
I simply do not know what this code does.

What about switched_to_rt() ? we lose the push_rt_task() logic... Hmm,
which I can't understand too ;)

And we also lose ENQUEUE_HEAD in this case, but this looks fine.

In short: could you confirm there are no problems here?

Oleg.