Re: [PATCH 17/24] sched/fair: Implement delayed dequeue

Peter Zijlstra
2024-11-06T14:14:28Z

On Wed, Nov 06, 2024 at 02:53:46PM +0100, Peter Zijlstra wrote:

> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 54d82c21fc8e..b083c6385e88 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3774,28 +3774,38 @@ ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
> */
> static int ttwu_runnable(struct task_struct *p, int wake_flags)
> {
> + CLASS(__task_rq_lock, rq_guard)(p);
> + struct rq *rq = rq_guard.rq;
>
> + if (!task_on_rq_queued(p))
> + return 0;
> +
> + update_rq_clock(rq);
> + if (p->se.sched_delayed) {
> + int queue_flags = ENQUEUE_DELAYED | ENQUEUE_NOCLOCK;
> +
> + /*
> + * Since sched_delayed means we cannot be current anywhere,
> + * dequeue it here and have it fall through to the
> + * select_task_rq() case further along the ttwu() path.
> + */
> + if (rq->nr_running > 1 && p->nr_cpus_allowed > 1) {
> + dequeue_task(rq, p, DEQUEUE_SLEEP | queue_flags);
> + return 0;
> }
> +
> + enqueue_task(rq, p, queue_flags);

And then I wondered... this means that !task_on_cpu() is true for
sched_delayed, and thus we can move this in the below branch.

But also, we can probably dequeue every such task, not only
sched_delayed ones.

> }
> + if (!task_on_cpu(rq, p)) {
> + /*
> + * When on_rq && !on_cpu the task is preempted, see if
> + * it should preempt the task that is current now.
> + */
> + wakeup_preempt(rq, p, wake_flags);
> + }
> + ttwu_do_wakeup(p);
>
> + return 1;
> }

Yielding something like this on top... which boots. But since I forgot
to make it a feature, I can't actually tell at this point.. *sigh*

Anyway, more toys to poke at I suppose.

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b083c6385e88..69b19ba77598 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3781,28 +3781,32 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
return 0;

update_rq_clock(rq);
- if (p->se.sched_delayed) {
- int queue_flags = ENQUEUE_DELAYED | ENQUEUE_NOCLOCK;
+ if (!task_on_cpu(rq, p)) {
+ int queue_flags = DEQUEUE_NOCLOCK;
+
+ if (p->se.sched_delayed)
+ queue_flags |= DEQUEUE_DELAYED;

/*
- * Since sched_delayed means we cannot be current anywhere,
- * dequeue it here and have it fall through to the
- * select_task_rq() case further along the ttwu() path.
+ * Since we're not current anywhere *AND* hold pi_lock, dequeue
+ * it here and have it fall through to the select_task_rq()
+ * case further along the ttwu() path.
*/
if (rq->nr_running > 1 && p->nr_cpus_allowed > 1) {
dequeue_task(rq, p, DEQUEUE_SLEEP | queue_flags);
return 0;
}

- enqueue_task(rq, p, queue_flags);
- }
- if (!task_on_cpu(rq, p)) {
+ if (p->se.sched_delayed)
+ enqueue_task(rq, p, queue_flags);
+
/*
* When on_rq && !on_cpu the task is preempted, see if
* it should preempt the task that is current now.
*/
wakeup_preempt(rq, p, wake_flags);
}
+ SCHED_WARN_ON(p->se.sched_delayed);
ttwu_do_wakeup(p);

return 1;