Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Mike Galbraith
2024-04-18T16:25:55Z

Greetings,

I tossed a couple rocks at it today, and seem to have hit the little
bugger. The root cause seems to be doing the delay dequeue business on
exiting tasks. Hunk #1 of hacklet below seems to quell the explosions.

crash> bt -sx
PID: 21722 TASK: ffff88815710ee40 CPU: 6 COMMAND: "hackbench"
#0 [ffff88822773bb90] machine_kexec+0x156 at ffffffff810642d6
#1 [ffff88822773bbe0] __crash_kexec+0xd7 at ffffffff81152a07
#2 [ffff88822773bc98] crash_kexec+0x23 at ffffffff81152eb3
#3 [ffff88822773bca8] oops_end+0xbe at ffffffff810329be
#4 [ffff88822773bcc8] page_fault_oops+0x81 at ffffffff81071951
#5 [ffff88822773bd28] exc_page_fault+0x62 at ffffffff8194e9e2
#6 [ffff88822773bd50] asm_exc_page_fault+0x22 at ffffffff81a00ba2
[exception RIP: pick_next_task_fair+178]
RIP: ffffffff810d8d12 RSP: ffff88822773be00 RFLAGS: 00010006
RAX: ffff88813cb780b8 RBX: ffff88840edb0e80 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff88813cb78080 RDI: ffff88840ec30f00
RBP: ffff88813cb78000 R8: ffff88815710eec0 R9: 0000000000000001
R10: ffff88822773bdc8 R11: 0000000000000013 R12: 0000000000030e80
R13: ffff88815710ee40 R14: ffff88813cb78080 R15: ffff88815710ee40
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffff88822773be28] __schedule+0x10d at ffffffff8195865d
#8 [ffff88822773be98] do_task_dead+0x3e at ffffffff810cc00e
#9 [ffff88822773beb0] do_exit+0x770 at ffffffff8108f0e0
#10 [ffff88822773bf00] do_group_exit+0x2c at ffffffff8108f64c
#11 [ffff88822773bf28] __x64_sys_exit_group+0x14 at ffffffff8108f6f4
#12 [ffff88822773bf30] do_syscall_64+0x57 at ffffffff8194ac37
#13 [ffff88822773bf50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b
RIP: 00007f4f2aa76136 RSP: 00007ffcbba84748 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f4f2aa76136
RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
RBP: 00007f4f2ab86970 R8: 00000000000000e7 R9: ffffffffffffff80
R10: 0000000000000004 R11: 0000000000000246 R12: 00007f4f2ab86970
R13: 0000000000000001 R14: 00007f4f2ab8a328 R15: 0000000000000000
ORIG_RAX: 00000000000000e7 CS: 0033 SS: 002b
crash> task_struct ffff88815710ee40 | grep sched_delayed
sched_delayed = 1 '\001',
crash>

---
kernel/sched/fair.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
update_curr(cfs_rq);

if (sched_feat(DELAY_DEQUEUE) && sleep &&
+ !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
!entity_eligible(cfs_rq, se)) {
if (cfs_rq->next == se)
cfs_rq->next = NULL;
@@ -5495,14 +5496,14 @@ pick_next_entity(struct rq *rq, struct c
}

struct sched_entity *se = pick_eevdf(cfs_rq);
- if (se->sched_delayed) {
+ while (se && se->sched_delayed) {
dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
SCHED_WARN_ON(se->sched_delayed);
SCHED_WARN_ON(se->on_rq);
if (sched_feat(DELAY_ZERO) && se->vlag > 0)
se->vlag = 0;

- return NULL;
+ se = pick_eevdf(cfs_rq);
}
return se;
}