[PATCH 1/2] sched/fair: Renew slice protection when current is picked again

[Fernand Sieber <sieberf@amazon.com>]

Commit 0b9ca2dcabc3 ("sched/fair: Always trigger resched at the end of a
protected period") made update_curr() request a reschedule as soon as the
running entity's protected period has run out:

	if (resched || !protect_slice(curr)) {
		resched_curr_lazy(rq);
		clear_buddies(cfs_rq, curr);
	}

se->vprot, which protect_slice() tests against, is armed by
set_protect_slice() in set_next_task_fair() only for a fresh pick
(SNT_PICK). An entity that is picked again - prev == next, so
put_prev_set_next_task() uses SNT_REPICK, see commit c72945693b90
("sched: Restart fair hrtick after same-task repicks") - keeps its expired
vprot. Every later update_curr() on it therefore sets TIF_NEED_RESCHED
again, and keeps doing so until the entity is switched out.

On a CPU that picks only for itself this is mostly harmless, because the
pick runs inside __schedule(), which clears TIF_NEED_RESCHED right after
it.

With core scheduling it is not. pick_next_task() does a core wide pick and
calls pick_task(rq_i) - hence pick_task_fair() and update_curr() - on every
SMT sibling's runqueue. resched_curr_lazy() on a remote runqueue ends in
smp_send_reschedule(), a real rescheduling IPI to that sibling. The sibling
schedules, re-picks the same task, renews nothing, and its own core wide
pick sends the IPI straight back. The two siblings then trade rescheduling
IPIs for as long as both run a task whose protected period has expired.

This needs CONFIG_PREEMPT_LAZY=n. __resched_curr() only calls
smp_send_reschedule() for the eager TIF_NEED_RESCHED, and
resched_curr_lazy() asks for TIF_NEED_RESCHED_LAZY instead once
PREEMPT_LAZY is built in, so a lazy kernel sets a remote flag and sends no
IPI.

To reproduce on any SMT machine built with CONFIG_SCHED_CORE=y and
CONFIG_PREEMPT_LAZY=n:

 1) take an SMT sibling pair from
    /sys/devices/system/cpu/cpu0/topology/thread_siblings_list;
 2) give the process group a core scheduling cookie with
    prctl(PR_SCHED_CORE, PR_SCHED_CORE_CREATE, 0, PIDTYPE_PGID, 0);
 3) fork four children that spin, two pinned to each sibling. Two per
    runqueue is the minimum: update_curr() returns early while
    cfs_rq->h_nr_queued == 1;
 4) read the RES row of /proc/interrupts for both siblings, wait ten
    seconds and read it again.

Renewing on a repick does not undo 0b9ca2dcabc3: the resched leg of the
condition is untouched, so the reschedule at the end of a protected period
still happens. A repicked entity gets the same bounded protection a fresh
pick gets - set_protect_slice() caps vprot at the entity's own deadline, at
the shortest slice enqueued on the cfs_rq under RUN_TO_PARITY, and at
ineligible_vruntime() under PREEMPT_SHORT - and only once its previous
protection has expired.

Fixes: 0b9ca2dcabc3 ("sched/fair: Always trigger resched at the end of a protected period")
Signed-off-by: Fernand Sieber <sieberf@amazon.com>
---
 kernel/sched/fair.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d4cc0caee6265..237908a8c4cd1 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -15645,6 +15645,13 @@ static void set_next_task_fair(struct rq *rq, struct task_struct *p, enum snt_e
 	sched_fair_update_stop_tick(rq, p);
 
 repick:
+	/*
+	 * A repick is a pick: renew a protected period that ran out, else
+	 * update_curr() reschedules on every update until p is switched out.
+	 */
+	if (type == SNT_REPICK && on_rq && !protect_slice(se))
+		set_protect_slice(cfs_rq, se);
+
 	/*
 	 * A same-task repick skips put_prev_task_fair(), but
 	 * pick_task_fair() refreshed the entity hrtick_start_fair() reads
-- 
2.47.3




Amazon Development Centre (South Africa) (Proprietary) Limited
29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa
Registration Number: 2004 / 034463 / 07