* [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu
2021-08-04 11:58 [PATCH 0/2] Reduce SIS scanning Mel Gorman
@ 2021-08-04 11:58 ` Mel Gorman
2021-08-05 9:34 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2021-08-04 11:58 ` [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu Mel Gorman
2021-08-04 13:11 ` [PATCH 0/2] Reduce SIS scanning Peter Zijlstra
2 siblings, 1 reply; 6+ messages in thread
From: Mel Gorman @ 2021-08-04 11:58 UTC (permalink / raw)
To: Ingo Molnar
Cc: Peter Zijlstra, Vincent Guittot, Valentin Schneider, Song Bao Hua,
LKML, Mel Gorman
After select_idle_sibling, p->recent_used_cpu is set to the
new target. However on the next wakeup, prev will be the same as
recent_used_cpu unless the load balancer has moved the task since the
last wakeup. It still works, but is less efficient than it could be.
This patch preserves recent_used_cpu for longer.
The impact on SIS efficiency is tiny so the SIS statistic patches were
used to track the hit rate for using recent_used_cpu. With perf bench
pipe on a 2-socket Cascadelake machine, the hit rate went from 57.14%
to 85.32%. For more intensive wakeup loads like hackbench, the hit rate
is almost negligible but rose from 0.21% to 6.64%. For scaling loads
like tbench, the hit rate goes from almost 0% to 25.42% overall. Broadly
speaking, on tbench, the success rate is much higher for lower thread
counts and drops to almost 0 as the workload scales to towards saturation.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/fair.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 44c452072a1b..8ad7666f387c 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6376,6 +6376,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
/* Check a recently used CPU as a potential idle candidate: */
recent_used_cpu = p->recent_used_cpu;
+ p->recent_used_cpu = prev;
if (recent_used_cpu != prev &&
recent_used_cpu != target &&
cpus_share_cache(recent_used_cpu, target) &&
@@ -6902,9 +6903,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
} else if (wake_flags & WF_TTWU) { /* XXX always ? */
/* Fast path */
new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
-
- if (want_affine)
- current->recent_used_cpu = cpu;
}
rcu_read_unlock();
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [tip: sched/core] sched/fair: Use prev instead of new target as recent_used_cpu
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
@ 2021-08-05 9:34 ` tip-bot2 for Mel Gorman
0 siblings, 0 replies; 6+ messages in thread
From: tip-bot2 for Mel Gorman @ 2021-08-05 9:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Mel Gorman, Peter Zijlstra (Intel), x86, linux-kernel
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 89aafd67f28c9e3b725aa30b44b7f61ad3e348ce
Gitweb: https://git.kernel.org/tip/89aafd67f28c9e3b725aa30b44b7f61ad3e348ce
Author: Mel Gorman <mgorman@techsingularity.net>
AuthorDate: Wed, 04 Aug 2021 12:58:56 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Wed, 04 Aug 2021 15:16:44 +02:00
sched/fair: Use prev instead of new target as recent_used_cpu
After select_idle_sibling, p->recent_used_cpu is set to the
new target. However on the next wakeup, prev will be the same as
recent_used_cpu unless the load balancer has moved the task since the
last wakeup. It still works, but is less efficient than it could be.
This patch preserves recent_used_cpu for longer.
The impact on SIS efficiency is tiny so the SIS statistic patches were
used to track the hit rate for using recent_used_cpu. With perf bench
pipe on a 2-socket Cascadelake machine, the hit rate went from 57.14%
to 85.32%. For more intensive wakeup loads like hackbench, the hit rate
is almost negligible but rose from 0.21% to 6.64%. For scaling loads
like tbench, the hit rate goes from almost 0% to 25.42% overall. Broadly
speaking, on tbench, the success rate is much higher for lower thread
counts and drops to almost 0 as the workload scales to towards saturation.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210804115857.6253-2-mgorman@techsingularity.net
---
kernel/sched/fair.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 13c3fd4..7ee394b 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6347,6 +6347,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
/* Check a recently used CPU as a potential idle candidate: */
recent_used_cpu = p->recent_used_cpu;
+ p->recent_used_cpu = prev;
if (recent_used_cpu != prev &&
recent_used_cpu != target &&
cpus_share_cache(recent_used_cpu, target) &&
@@ -6873,9 +6874,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
} else if (wake_flags & WF_TTWU) { /* XXX always ? */
/* Fast path */
new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
-
- if (want_affine)
- current->recent_used_cpu = cpu;
}
rcu_read_unlock();
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu
2021-08-04 11:58 [PATCH 0/2] Reduce SIS scanning Mel Gorman
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
@ 2021-08-04 11:58 ` Mel Gorman
2021-08-05 9:34 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2021-08-04 13:11 ` [PATCH 0/2] Reduce SIS scanning Peter Zijlstra
2 siblings, 1 reply; 6+ messages in thread
From: Mel Gorman @ 2021-08-04 11:58 UTC (permalink / raw)
To: Ingo Molnar
Cc: Peter Zijlstra, Vincent Guittot, Valentin Schneider, Song Bao Hua,
LKML, Mel Gorman
When select_idle_cpu starts scanning for an idle CPU, it starts with
a target CPU that has already been checked by select_idle_sibling.
This patch starts with the next CPU instead.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 8ad7666f387c..f412b4504378 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6249,7 +6249,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
time = cpu_clock(this);
}
- for_each_cpu_wrap(cpu, cpus, target) {
+ for_each_cpu_wrap(cpu, cpus, target + 1) {
if (has_idle_core) {
i = select_idle_core(p, cpu, cpus, &idle_cpu);
if ((unsigned int)i < nr_cpumask_bits)
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [tip: sched/core] sched/fair: Avoid a second scan of target in select_idle_cpu
2021-08-04 11:58 ` [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu Mel Gorman
@ 2021-08-05 9:34 ` tip-bot2 for Mel Gorman
0 siblings, 0 replies; 6+ messages in thread
From: tip-bot2 for Mel Gorman @ 2021-08-05 9:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Mel Gorman, Peter Zijlstra (Intel), x86, linux-kernel
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 56498cfb045d7147cdcba33795d19429afcd1d00
Gitweb: https://git.kernel.org/tip/56498cfb045d7147cdcba33795d19429afcd1d00
Author: Mel Gorman <mgorman@techsingularity.net>
AuthorDate: Wed, 04 Aug 2021 12:58:57 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Wed, 04 Aug 2021 15:16:44 +02:00
sched/fair: Avoid a second scan of target in select_idle_cpu
When select_idle_cpu starts scanning for an idle CPU, it starts with
a target CPU that has already been checked by select_idle_sibling.
This patch starts with the next CPU instead.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210804115857.6253-3-mgorman@techsingularity.net
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 7ee394b..47a0fbf 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6220,7 +6220,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
time = cpu_clock(this);
}
- for_each_cpu_wrap(cpu, cpus, target) {
+ for_each_cpu_wrap(cpu, cpus, target + 1) {
if (has_idle_core) {
i = select_idle_core(p, cpu, cpus, &idle_cpu);
if ((unsigned int)i < nr_cpumask_bits)
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 0/2] Reduce SIS scanning
2021-08-04 11:58 [PATCH 0/2] Reduce SIS scanning Mel Gorman
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
2021-08-04 11:58 ` [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu Mel Gorman
@ 2021-08-04 13:11 ` Peter Zijlstra
2 siblings, 0 replies; 6+ messages in thread
From: Peter Zijlstra @ 2021-08-04 13:11 UTC (permalink / raw)
To: Mel Gorman
Cc: Ingo Molnar, Vincent Guittot, Valentin Schneider, Song Bao Hua,
LKML
On Wed, Aug 04, 2021 at 12:58:55PM +0100, Mel Gorman wrote:
> This is the first two patches from "Modify and/or delete SIS_PROP" and
> focues on improving the hit rate for recent_used_cpu and avoids rescanning
> the target CPU when it's known to be busy. These are relatively simply
> patches in comparison to the full series which could be a regression
> magnet and that series had at least one mistake in it.
>
> In general, the usual suspects showed mostly small gains with a few
> exceptions. Usual suspects were NAS, hackbench, schbench (Facebook
> latency-sensitive workload), perf pipe, kernel building, git test suite,
> dbench and redis.
>
> From here, the next obvious candidates are either improving has_idle_cores
> or continuing to try remove/improve SIS_PROP.
Thanks!
^ permalink raw reply [flat|nested] 6+ messages in thread