Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

[Peter Zijlstra <peterz@infradead.org>]

On Tue, Jul 02, 2024 at 02:56:51AM +0000, Libo Chen wrote:

> Hi Peter,
> 
> We are observing massive hackbench regressions with this patchset as
> well as the newer version from your queue.  I don’t know if you are
> still working on complete EEVDF, just want to bring this
> issue to your attention if it’s still in play.

Yeha, I'm still (slowly) poking at this. I was planning to post a new
set in a week or so.

> The test system is a two-socket Zen4 EYPC with total of 192C/384T,
> intel does suffer too but to a lesser degree.  Kernel is v6.10 based
> off the latest schedule/core. From 48 groups to 296 groups,
> process/threads via socket degrades <= 60%, process/threads via pipe
> can have regression at whopping 110% or more.
> 
> It turns out in deactivate_task(), this patchset changes the order of
> dequeue_task() and setting p->on_rq such that p->on_rq is set after
> dequeue_task() gets called. 

Bah, I knew I had to double check that,.. but then I never did :/

> My understanding is TASK_ON_RQ_MIGRATING
> allows src rq lock to be released to other tasks since
> soon-to-be-dequeued task is migrating anyway, so holding two rq locks
> while dequeuing seems to be a quite horrible thing to do. The RFC
> patch is a bit tricky to get the order back, but a fix can be easily
> done to your newer version specifically to “sched: Split DEQUEUE_SLEEP
> from deactivate_task()” like below:
> 
>  void deactivate_task(struct rq *rq, struct task_struct *p, int flags)
>  {
> -       dequeue_task(rq, p, flags);
> -
>         WRITE_ONCE(p->on_rq, TASK_ON_RQ_MIGRATING);
>         ASSERT_EXCLUSIVE_WRITER(p->on_rq);
> +
> +       dequeue_task(rq, p, flags);
>  }

Thanks, I'll fold that in and try and put your name somewhere.