* [PATCH] s390/topology: Enable ASYM_PACKING to establish cpu priority
@ 2026-09-01 14:40 Mete Durlu
2026-09-01 14:52 ` sashiko-bot
2026-09-01 15:30 ` Heiko Carstens
0 siblings, 2 replies; 3+ messages in thread
From: Mete Durlu @ 2026-09-01 14:40 UTC (permalink / raw)
To: Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger, Sven Schnelle
Cc: linux-s390, Mete Durlu
On s390 LPARs the hypervisor assigns varying CPU runtime capacities
under vertical polarization, making CPUs unequal. Add SD_ASYM_PACKING
into topology level flag callbacks so the scheduler keeps busier
scheduling groups packed onto higher-capacity CPUs.
Implement arch_asym_cpu_priority() using smp_cpu_get_capacity() as
the decision factor. Subtract the CPU number so that the CPUs with
lower ids are prioritized.
Only enabled on LPAR level as other guest levels either do not have
vertical polarization (which causes asymmetric CPUs), or machine backed
topology.
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
---
Under vertical polarization, LPAR cpus get different capacities assgined
to them by the underlying hypervisor. This creates an asymmetric
configuration where the scheduler should prefer packing work onto
higher capacity CPUs rather than spreading it uniformly.
Introduces arch_asym_cpu_priority() backed by smp_cpu_get_capacity()
so the scheduler has a per-CPU priority signal. Add SD_ASYM_PACKING
to topology levels, enabling asym_packing load balancing.
Note: I initially started working to add both SD_ASYM_PACKING and
SD_SHARE_LLC to multiple topology levels but now I discovered that there
are still a bit more that I am unsure about how cache is being managed
by kernel so I am sticking to adding SD_ASYM_PACKING only for now.
Wether to introduce SD_SHARE_LLC flag to more topology levels is left
for future work.
Benchmark Results:
----------------------------------------------------------------------
I ran experiements with different combination of flags on different
topology levels. On an LPAR with 24 COREs spanning multiple DCMs
(BOOKS);
CPU NODE DRAWER BOOK SOCKET CORE POLARIZATION
0 0 0 0 0 0 vert-high
1 0 0 0 0 0 vert-high
2 0 0 0 0 1 vert-high
3 0 0 0 0 1 vert-high
4 0 0 0 0 2 vert-medium
5 0 0 0 0 2 vert-medium
6 0 0 0 0 3 vert-low
7 0 0 0 0 3 vert-low
8 0 0 0 0 4 vert-low
9 0 0 0 0 4 vert-low
10 0 0 0 0 5 vert-low
11 0 0 0 0 5 vert-low
12 0 0 0 1 6 vert-low
13 0 0 0 1 6 vert-low
14 0 0 0 1 7 vert-low
15 0 0 0 1 7 vert-low
16 0 0 0 1 8 vert-low
17 0 0 0 1 8 vert-low
18 0 0 0 1 9 vert-low
19 0 0 0 1 9 vert-low
20 0 0 0 1 10 vert-low
21 0 0 0 1 10 vert-low
22 0 0 0 1 11 vert-low
23 0 0 0 1 11 vert-low
24 0 0 1 2 12 vert-low
25 0 0 1 2 12 vert-low
26 0 0 1 2 13 vert-low
27 0 0 1 2 13 vert-low
28 0 0 1 2 14 vert-low
29 0 0 1 2 14 vert-low
30 0 0 1 3 15 vert-low
31 0 0 1 3 15 vert-low
32 0 0 1 3 16 vert-low
33 0 0 1 3 16 vert-low
34 0 0 1 3 17 vert-low
35 0 0 1 3 17 vert-low
36 0 0 2 4 18 vert-low
37 0 0 2 4 18 vert-low
38 0 0 2 4 19 vert-low
39 0 0 2 4 19 vert-low
40 0 0 2 4 20 vert-low
41 0 0 2 4 20 vert-low
42 0 0 2 5 21 vert-low
43 0 0 2 5 21 vert-low
44 0 0 2 5 22 vert-low
45 0 0 2 5 22 vert-low
46 0 0 2 5 23 vert-low
47 0 0 2 5 23 vert-low
Overall the results from different benchmarks are all positive;
baseline : mc flags -> SD_SHARE_LLC
others -> 0
mc_all : mc flags -> SD_ASYM_PACKING | SD_SHARE_LLC
others -> 0
set_all : mc flags -> SD_ASYM_PACKING | SD_SHARE_LLC
book flags -> SD_ASYM_PACKING | SD_SHARE_LLC
drawer flags -> SD_ASYM_PACKING | SD_SHARE_LLC
book_asym: mc flags -> SD_ASYM_PACKING | SD_SHARE_LLC
book flags -> SD_ASYM_PACKING
others -> 0
patch : mc flags -> SD_ASYM_PACKING | SD_SHARE_LLC
book flags -> SD_ASYM_PACKING
drawer -> SD_ASYM_PACKING
Hackbench
======================================================================
$ hackbench -T -p -l $loops -f $fd -g $g
group/fd baseline mc_all set_all book_asym patch
-------- -------- ------ ------ --------- ------
g1 fd1 2.020 +22.7% +6.4% +45.3% +10.3%
g1 fd2 4.719 -16.6% -18.6% -22.6% -20.7%
g1 fd4 6.703 -1.5% -19.1% -21.6% -14.9%
g1 fd8 8.001 -0.5% -8.5% -9.3% -14.3%
g2 fd1 1.516 -17.5% -9.4% -31.7% -31.9%
g2 fd2 6.590 -11.6% -29.9% -32.4% -26.9%
g2 fd4 8.245 -14.1% -24.4% -23.4% -23.9%
g2 fd8 8.827 -3.0% +1.5% +1.0% -4.1%
g4 fd1* 0.566 +10.2% +6.4% -1.8% +33.9%
g4 fd2 3.077 -10.8% -20.6% -20.2% -14.9%
g4 fd4 8.822 -16.7% -17.4% -18.3% -9.9%
g4 fd8 11.078 -1.8% -14.6% -14.8% +1.9%
g8 fd1 0.540 -13.1% -34.8% -39.3% -20.9%
g8 fd2 1.718 -6.2% -14.8% -15.0% -4.4%
g8 fd4 4.732 -0.6% -11.6% -11.3% -1.5%
g8 fd8 18.278 -2.8% -7.3% -8.0% -4.0%
Results are in seconds of completion, lower is better
* "4 groups 1 file descriptor" case run for too short of and amount of
time causing inaccurate results. Repeated individual runs with longer
runtimes show positive impact of patched kernels.
Stress-ng
=====================================================================
Repeated stress-ng experiments from 6 stressors with an increment of
6 after each run until $(nproc) stressors. Metrics is bogo-ops higher is
better.
$ stress-ng -M --matrix-3d $cpu --matrix-3d-method mult --timeout 10
cpu baseline mc_all set_all book_asym patch
-----------------------------------------------------------------
6 110059.00 -0.2053% -0.5579% -0.5161% -0.4425%
12 218074.00 -0.2164% -1.554% -1.122% -1.554%
18 303547.00 +0.0602% +6.943% +7.037% +6.532%
24 370629.00 -0.6416% +12.17% +12.13% +4.945%
30 426237.00 +0.4155% +0.8521% +1.031% +0.2311%
36 426766.00 -1.336% -0.5235% -0.1186% -0.0314%
42 424059.00 -1.558% -0.6162% -0.5308% -1.294%
48 416659.00 -1.363% -0.3108% -0.6797% -1.244%
$ stress-ng -M --vm $cpu --vm-bytes 128M --timeout 10
cpu baseline mc_all set_all book_asym patch
-------------------------------------------------------------
6 4257228.00 -1.39% +5.296% +4.911% -0.5117%
12 8630030.00 +1.375% -3.333% 0.03611% -3.967%
18 12137905.00 +6.66% +6.756% -2.597% +9.711%
24 15610271.00 -0.5227% -5.556% +0.8526% -0.6858%
30 18481251.00 -6.841% -6.63% +1.429% -6.102%
36 19837178.00 -0.7821% +1.292% -0.9471% +1.121%
42 20220039.00 +1.137% +9.447% +0.2035% +9.083%
48 23093876.00 -0.255% -1.515% -1.032% -0.7902%
$ stress-ng -M --mmapfork $cpu --mmapfork-bytes 128M --timeout 10
cpu baseline mc_all set_all book_asym patch
-----------------------------------------------------------
6 3355.00 +8.346% +9.538% +15.86% +14.22%
12 5883.00 +6.612% +6.986% +12.99% +11.68%
18 8045.00 +4.214% +6.986% +6.439% +7.744%
24 9682.00 +3.47% +5.206% +5.412% +8.056%
30 10889.00 +3.958% +0.8173% +3.049% +4.656%
36 12070.00 -0.3148% -2.61% +2.005% +1.939%
42 12792.00 +0.7739% +1.845% +2.955% +0.8521%
48 13609.00 -1.367% +1.19% +0.316% +4.321%
OpenBlas
=====================================================================
Openblas testcase is a series of matrix operations with different
sizes of matrices. The distribution of the measurements is important to
notice in these results. (Unit: FLOPS/s)
Metrics baseline mc_all set_all book_asym patch
---------------------------------------------------------------------------
Mean 66.07 MF/s 72.26 MF/s 82.33 MF/s 79.83 MF/s 82.87 MF/s
Median 1.66 MF/s 18.19 MF/s 38.12 MF/s 30.56 MFs/s 42.64 MF/s
Min 36.39 KF/s 50.99 KF/s 77.50 KF/s 41.84 KFs/s 34.07 KF/s
Max 831.80 MF/s 832.73 MF/s 828.87 MF/s 858.38 MFs/s 842.45 MF/s
Percentile baseline mc_all set_all book_asym patch
---------------------------------------------------------------- ------------
25th pct 717.60 KF/s 642.21 KF/s 17.93 MF/s 15.33 MF/s 23.99 MF/s
50th pct 1.66 MF/s 18.19 MF/s 38.12 MF/s 30.56 MF/s 42.64 MF/s
75th pct 79.16 MF/s 98.09 MF/s 94.34 MF/s 95.89 MF/s 77.97 MF/s
90th pct 219.63 MF/s 210.93 MF/s 214.10 MF/s 212.97 MF/s 209.31 MF/s
99th pct 477.66 MF/s 494.59 MF/s 520.22 MF/s 491.37 MF/s 524.82 MF/s
The results are similar when hiperdispatch is modifying the cpu
capacities and when capacities stay the same.
---
arch/s390/kernel/topology.c | 34 +++++++++++++++++++++++++++++++---
1 file changed, 31 insertions(+), 3 deletions(-)
diff --git a/arch/s390/kernel/topology.c b/arch/s390/kernel/topology.c
index 1377c6f3f670..6d4b7be029c3 100644
--- a/arch/s390/kernel/topology.c
+++ b/arch/s390/kernel/topology.c
@@ -527,11 +527,39 @@ static const struct cpumask *tl_drawer_mask(struct sched_domain_topology_level *
return &cpu_topology[cpu].drawer_mask;
}
+static int s390_core_flags(void)
+{
+ int flags = cpu_core_flags();
+
+ if (machine_is_lpar())
+ flags |= SD_ASYM_PACKING;
+ return flags;
+}
+
+static int s390_book_flags(void)
+{
+ if (machine_is_lpar())
+ return SD_ASYM_PACKING;
+ return 0;
+}
+
+static int s390_drawer_flags(void)
+{
+ if (machine_is_lpar())
+ return SD_ASYM_PACKING;
+ return 0;
+}
+
+int arch_asym_cpu_priority(int cpu)
+{
+ return smp_cpu_get_capacity(cpu) - cpu;
+}
+
static struct sched_domain_topology_level s390_topology[] = {
SDTL_INIT(tl_smt_mask, cpu_smt_flags, SMT),
- SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
- SDTL_INIT(tl_book_mask, NULL, BOOK),
- SDTL_INIT(tl_drawer_mask, NULL, DRAWER),
+ SDTL_INIT(tl_mc_mask, s390_core_flags, MC),
+ SDTL_INIT(tl_book_mask, s390_book_flags, BOOK),
+ SDTL_INIT(tl_drawer_mask, s390_drawer_flags, DRAWER),
SDTL_INIT(tl_pkg_mask, NULL, PKG),
{ NULL, },
};
---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260831-s390_sd_flags-e44bc519f932
Best regards,
--
Mete Durlu <meted@linux.ibm.com>
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] s390/topology: Enable ASYM_PACKING to establish cpu priority
2026-09-01 14:40 [PATCH] s390/topology: Enable ASYM_PACKING to establish cpu priority Mete Durlu
@ 2026-09-01 14:52 ` sashiko-bot
2026-09-01 15:30 ` Heiko Carstens
1 sibling, 0 replies; 3+ messages in thread
From: sashiko-bot @ 2026-09-01 14:52 UTC (permalink / raw)
To: Mete Durlu
Cc: Christian Borntraeger, Heiko Carstens, Alexander Gordeev,
linux-s390, Vasily Gorbik
> On s390 LPARs the hypervisor assigns varying CPU runtime capacities
> under vertical polarization, making CPUs unequal. Add SD_ASYM_PACKING
> into topology level flag callbacks so the scheduler keeps busier
> scheduling groups packed onto higher-capacity CPUs.
> Implement arch_asym_cpu_priority() using smp_cpu_get_capacity() as
> the decision factor. Subtract the CPU number so that the CPUs with
> lower ids are prioritized.
>
> Only enabled on LPAR level as other guest levels either do not have
> vertical polarization (which causes asymmetric CPUs), or machine backed
> topology.
>
> Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260901-s390_sd_flags-v1-1-6755ba191d5a@linux.ibm.com?part=1
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] s390/topology: Enable ASYM_PACKING to establish cpu priority
2026-09-01 14:40 [PATCH] s390/topology: Enable ASYM_PACKING to establish cpu priority Mete Durlu
2026-09-01 14:52 ` sashiko-bot
@ 2026-09-01 15:30 ` Heiko Carstens
1 sibling, 0 replies; 3+ messages in thread
From: Heiko Carstens @ 2026-09-01 15:30 UTC (permalink / raw)
To: Mete Durlu
Cc: Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, linux-s390
On Tue, Sep 01, 2026 at 04:40:26PM +0200, Mete Durlu wrote:
> On s390 LPARs the hypervisor assigns varying CPU runtime capacities
> under vertical polarization, making CPUs unequal. Add SD_ASYM_PACKING
> into topology level flag callbacks so the scheduler keeps busier
> scheduling groups packed onto higher-capacity CPUs.
> Implement arch_asym_cpu_priority() using smp_cpu_get_capacity() as
> the decision factor.
...
> Subtract the CPU number so that the CPUs with lower ids are prioritized.
Why?
> Only enabled on LPAR level as other guest levels either do not have
> vertical polarization (which causes asymmetric CPUs), or machine backed
> topology.
...
> +static int s390_core_flags(void)
> +{
> + int flags = cpu_core_flags();
> +
> + if (machine_is_lpar())
> + flags |= SD_ASYM_PACKING;
> + return flags;
> +}
Why is this machine_is_lpar()? This should actually test if cpu
polarization is available to make this independent of some random
environment test.
Furthermore to me this reads like the above only makes sense in case of
vertical polarization. What happens in case of horizontal polarization?
Does SD_ASYM_PACKING have any negative impact for that case?
> +int arch_asym_cpu_priority(int cpu)
> +{
> + return smp_cpu_get_capacity(cpu) - cpu;
> +}
As commented above. The rationale for this needs to documented.
Does this make sense for horizontal polarization?
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-09-01 15:30 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-01 14:40 [PATCH] s390/topology: Enable ASYM_PACKING to establish cpu priority Mete Durlu
2026-09-01 14:52 ` sashiko-bot
2026-09-01 15:30 ` Heiko Carstens
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox