Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

K Prateek Nayak
2024-04-18T17:09:03Z

Hello Mike,

On 4/18/2024 9:54 PM, Mike Galbraith wrote:
> Greetings,
>
> I tossed a couple rocks at it today, and seem to have hit the little
> bugger. The root cause seems to be doing the delay dequeue business on
> exiting tasks. Hunk #1 of hacklet below seems to quell the explosions.
>
> crash> bt -sx
> PID: 21722 TASK: ffff88815710ee40 CPU: 6 COMMAND: "hackbench"
> #0 [ffff88822773bb90] machine_kexec+0x156 at ffffffff810642d6
> #1 [ffff88822773bbe0] __crash_kexec+0xd7 at ffffffff81152a07
> #2 [ffff88822773bc98] crash_kexec+0x23 at ffffffff81152eb3
> #3 [ffff88822773bca8] oops_end+0xbe at ffffffff810329be
> #4 [ffff88822773bcc8] page_fault_oops+0x81 at ffffffff81071951
> #5 [ffff88822773bd28] exc_page_fault+0x62 at ffffffff8194e9e2
> #6 [ffff88822773bd50] asm_exc_page_fault+0x22 at ffffffff81a00ba2
> [exception RIP: pick_next_task_fair+178]
> RIP: ffffffff810d8d12 RSP: ffff88822773be00 RFLAGS: 00010006
> RAX: ffff88813cb780b8 RBX: ffff88840edb0e80 RCX: 0000000000000000
> RDX: 0000000000000000 RSI: ffff88813cb78080 RDI: ffff88840ec30f00
> RBP: ffff88813cb78000 R8: ffff88815710eec0 R9: 0000000000000001
> R10: ffff88822773bdc8 R11: 0000000000000013 R12: 0000000000030e80
> R13: ffff88815710ee40 R14: ffff88813cb78080 R15: ffff88815710ee40
> ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
> #7 [ffff88822773be28] __schedule+0x10d at ffffffff8195865d
> #8 [ffff88822773be98] do_task_dead+0x3e at ffffffff810cc00e
> #9 [ffff88822773beb0] do_exit+0x770 at ffffffff8108f0e0
> #10 [ffff88822773bf00] do_group_exit+0x2c at ffffffff8108f64c
> #11 [ffff88822773bf28] __x64_sys_exit_group+0x14 at ffffffff8108f6f4
> #12 [ffff88822773bf30] do_syscall_64+0x57 at ffffffff8194ac37
> #13 [ffff88822773bf50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b
> RIP: 00007f4f2aa76136 RSP: 00007ffcbba84748 RFLAGS: 00000246
> RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f4f2aa76136
> RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
> RBP: 00007f4f2ab86970 R8: 00000000000000e7 R9: ffffffffffffff80
> R10: 0000000000000004 R11: 0000000000000246 R12: 00007f4f2ab86970
> R13: 0000000000000001 R14: 00007f4f2ab8a328 R15: 0000000000000000
> ORIG_RAX: 00000000000000e7 CS: 0033 SS: 002b
> crash> task_struct ffff88815710ee40 | grep sched_delayed
> sched_delayed = 1 '\001',
> crash>
>

My machine too survived the spawn test with the below patch. Thank you
for looking into this. I'll resume testing with the below patch
included.

--
Thanks and Regards,
Prateek

> ---
> kernel/sched/fair.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
> update_curr(cfs_rq);
>
> if (sched_feat(DELAY_DEQUEUE) && sleep &&
> + !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
> !entity_eligible(cfs_rq, se)) {
> if (cfs_rq->next == se)
> cfs_rq->next = NULL;
> @@ -5495,14 +5496,14 @@ pick_next_entity(struct rq *rq, struct c
> }
>
> struct sched_entity *se = pick_eevdf(cfs_rq);
> - if (se->sched_delayed) {
> + while (se && se->sched_delayed) {
> dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
> SCHED_WARN_ON(se->sched_delayed);
> SCHED_WARN_ON(se->on_rq);
> if (sched_feat(DELAY_ZERO) && se->vlag > 0)
> se->vlag = 0;
>
> - return NULL;
> + se = pick_eevdf(cfs_rq);
> }
> return se;
> }
>