From: Tim Chen <tim.c.chen@linux.intel.com>
To: Jianyong Wu <wujianyong@hygon.cn>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Valentin Schneider <vschneid@redhat.com>,
K Prateek Nayak <kprateek.nayak@amd.com>,
Chen Yu <yu.c.chen@intel.com>, Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Randy Dunlap <rdunlap@infradead.org>,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
jianyong.wu@outlook.com, wangfengyu@hygon.cn, zhongyuan@hygon.cn,
huangsj@hygon.cn
Subject: Re: [PATCH] sched/topology: Add new kernel parameter to tune NUMA domain flags
Date: Tue, 17 Nov 2026 11:10:19 -0800 [thread overview]
Message-ID: <0f6295ad41cce262149b939c8a245df76bfe5719.camel@linux.intel.com> (raw)
In-Reply-To: <20261117131854.924760-1-wujianyong@hygon.cn>
On Tue, 2026-11-17 at 21:18 +0800, Jianyong Wu wrote:
> Sched domain flags can largely affect performance on systems with
> multiple NUMA levels, especially SD_BALANCE_FORK, SD_BALANCE_EXEC,
> SD_WAKE_AFFINE and SD_BALANCE_NEWIDLE.
Jianyong,
A few thoughts on the three flags.
>
> SD_BALANCE_FORK and SD_BALANCE_EXEC decide how far the slow path of
> select_task_rq_fair() searches. Stripping them from the high NUMA levels
> reduces the number of CPUs to traverse and speeds up fork/exec.
> SD_WAKE_AFFINE decides the wakeup scope: if the waker and the wakee are
> farther apart than the highest level carrying it, the wakee cannot be
> pulled next to the waker on wakeup, which may hurt performance.
>
On your data later for Machine B, you say that levels at 32, 35 and 38 are
beyond node_reclaim_distance, so probably Hygon is running with the generic
node reclaim distance 30.
AMD sets node reclaim_distance=32 in init_amd_zen_common() for this reason.
You can probably do the same for Hygon, to retain SD_BALANCE_FORK and
SD_BALANCE_EXEC at the right level.
> However,
> these three flags are all controlled by a single static threshold,
> node_reclaim_distance, which can neither tell them apart nor be changed.
> Different workloads on the same system favour different settings, so no
> single static value fits them all.
>
> The same applies to SD_BALANCE_NEWIDLE, which is set on every NUMA level
> and cannot be stripped. Newidle balance on a high NUMA level can take a
> long time, even when there is no task in the system to pull.
There is an existing boot knob relax_domain_level= ,
which clears SD_BALANCE_NEWIDLE on every domain at or above
the given level, see set_domain_attribute(). On Machine A,
relax_domain_level=<level of the distance-25 NUMA domain> should give
the same domain flags as newidle:20. Can you confirm with the
/sys/kernel/debug/sched/domains/ flags and rerun schbench with that
boot parameter and NI_RANDOM set? I think we shouldn't introduce
another boot parameter to control the same flag.
I've made similar suggestions before to Kunyu in his test on Hygon system:
https://lore.kernel.org/lkml/6d8ff4593cc93c0033c6537eeac281085726c220.camel@linux.intel.com/
Will you achieve most of the gain for your workload after adjusting
node_reclaim_distance and setting appropriate relax_domain_level,
instead of introducing a new kernel parameter?
Tim
>
> Introduce a new kernel parameter, sched_numa_flag_distance, to let users
> tune the flags above separately on the NUMA levels. A group is stripped
> from the levels whose node distance is greater than the given value;
> once set, this overrides the default for that group:
>
> sched_numa_flag_distance=forkexec:<d>,newidle:<d>,wakeaffine:<d>
>
> Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
> ---
>
> Notes:
> Test results
>
> All results are normalized to the default (no parameter) = 1.00.
> UnixBench: higher is better; latency and time: lower is better.
> kernel.numa_balancing=0 for all tests.
>
> Machine A: Hygon, 2 sockets, 256 CPUs, 8 nodes. All NUMA levels (15,
> 25, 27) are below node_reclaim_distance, so by default every NUMA level
> has fork/exec, newidle and wake_affine.
>
> node 0 1 2 3 4 5 6 7
> 0: 10 15 15 15 25 27 27 27
> 1: 15 10 15 15 27 25 27 27
> 2: 15 15 10 15 27 27 25 27
> 3: 15 15 15 10 27 27 27 25
> 4: 25 27 27 27 10 15 15 15
> 5: 27 25 27 27 15 10 15 15
> 6: 27 27 25 27 15 15 10 15
> 7: 27 27 27 25 15 15 15 10
>
> Each setting was tested in its own boot of the same kernel:
> forkexec:10 drops fork/exec balancing on every NUMA level, newidle:20
> drops newidle balancing at distance 25 and 27. The default column is
> the average of two boots without the parameter.
>
> forkexec:10, tested with C2 disabled on all CPUs
> (echo 1 > /sys/devices/system/cpu/cpu*/cpuidle/state2/disable). The
> two default boots differ by at most 2.9%.
>
> UnixBench (byte-unixbench), 1 copy, average of 3 rounds:
>
> ./Run -c 1 -i 3 execl spawn shell1 shell8
>
> default forkexec:10
> Execl Throughput 1.00 1.27
> Process Creation 1.00 1.29
> Shell Scripts (1 concurrent) 1.00 1.35
> Shell Scripts (8 concurrent) 1.00 2.07
>
> Time spent in select_task_rq_fair() (function_graph) while one task
> forks and reaps 10000 children, then forks 5000 children that exec
> /bin/true, 3 times:
>
> default forkexec:10
> fork 1.00 0.05
> exec 1.00 0.05
>
> newidle:20, tested with all C-states enabled. The two default boots
> differ by at most 1%.
>
> schbench (git.kernel.org/pub/scm/linux/kernel/git/mason/schbench.git),
> 8 message threads with 32 workers each, i.e. 256 worker threads, 30s
> run after a 5s warmup, average of 3 runs:
>
> schbench -m 8 -t 32 -r 30 -w 5
>
> default newidle:20
> p99 wakeup latency 1.00 0.56
> requests per second 1.00 1.00
>
> Machine B: Hygon, 2 sockets, 384 CPUs, 12 nodes. The NUMA levels at 32,
> 35 and 38 are beyond node_reclaim_distance, so by default they have no
> fork/exec and no wake_affine.
>
> node 0 1 2 3 4 5 6 7 8 9 10 11
> 0: 10 19 15 12 15 12 32 35 32 29 38 35
> 1: 19 10 12 15 12 15 35 32 35 38 29 32
> 2: 15 12 10 12 15 12 32 35 38 35 32 35
> 3: 12 15 12 10 19 15 29 38 35 32 35 32
> 4: 15 12 15 19 10 12 38 29 32 35 32 35
> 5: 12 15 12 15 12 10 35 32 35 32 35 38
> 6: 32 35 32 29 38 35 10 19 15 12 15 12
> 7: 35 32 35 38 29 32 19 10 12 15 12 15
> 8: 32 35 38 35 32 35 15 12 10 12 15 12
> 9: 29 38 35 32 35 32 12 15 12 10 19 15
> 10: 38 29 32 35 32 35 15 12 15 19 10 12
> 11: 35 32 35 32 35 38 12 15 12 15 12 10
>
> Booted with sched_numa_flag_distance=wakeaffine:255, i.e. wake_affine
> on every NUMA level.
>
> hackbench from rt-tests (v2.20), pipes, each group being 20 senders and
> 20 receivers (40 tasks), 3 rounds of 3 runs:
>
> hackbench -p -g 4 -l 5000 (processes, 160 tasks)
> hackbench -p -g 16 -l 1500 (processes, 640 tasks)
> hackbench -p -T -g 4 -l 5000 (threads, 160 tasks)
> hackbench -p -T -g 16 -l 1500 (threads, 640 tasks)
>
> time default wakeaffine:255
> processes, 4 groups 1.00 0.80
> processes, 16 groups 1.00 0.69
> threads, 4 groups 1.00 0.77
> threads, 16 groups 1.00 0.69
>
> Open question
>
> This version only adds a boot parameter. Since the best setting follows
> the workload, a runtime knob would be more useful. Would one be
> acceptable, and if so, where: sysfs, or debugfs next to the other
> scheduler tunables? Changing it at runtime would rebuild the sched
> domains.
>
> .../admin-guide/kernel-parameters.txt | 15 +++
> kernel/sched/topology.c | 92 ++++++++++++++++++-
> 2 files changed, 102 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 6169787d3805..e4d504992cce 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -6798,6 +6798,21 @@ Kernel parameters
> cost of significant additional memory use for tables.
>
>
> + sched_numa_flag_distance= [KNL,NUMA]
> + Format: <group>:<distance>[,<group>:<distance>...]
> + Strip a group of scheduler balance flags from the
> + NUMA sched domain levels whose node distance is
> + greater than <distance>. <group> is one of:
> + forkexec - SD_BALANCE_FORK and SD_BALANCE_EXEC,
> + default: node_reclaim_distance
> + newidle - SD_BALANCE_NEWIDLE, default: never
> + wakeaffine - SD_WAKE_AFFINE,
> + default: node_reclaim_distance
> + node_reclaim_distance is 30 (RECLAIM_DISTANCE), or
> + 32 on AMD Zen. A distance of 255 keeps the group on
> + every NUMA level.
> + Example: sched_numa_flag_distance=forkexec:10,newidle:20
> +
> sched_proxy_exec= [KNL]
> Enables or disables "proxy execution" style
> solution to mutex-based priority inversion.
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 919d0fb00bd9..3bcf0db0800a 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -1947,6 +1947,92 @@ static struct cpumask ***sched_domains_numa_masks;
> SD_NUMA | \
> SD_ASYM_PACKING)
>
> +#ifdef CONFIG_NUMA
> +/*
> + * sched_numa_flag_distance=<group>:<distance>[,...]
> + *
> + * Strip a group of balance flags from the NUMA sched domain levels whose
> + * node distance is greater than <distance>. Groups left unset keep the
> + * default: fork/exec and wake_affine are stripped beyond
> + * node_reclaim_distance, newidle is never stripped.
> + *
> + * Node distances grow with the level, so a group is stripped from a level
> + * and every level above it, as the SDF_SHARED_CHILD flags require.
> + */
> +enum {
> + SD_DIST_FORKEXEC,
> + SD_DIST_NEWIDLE,
> + SD_DIST_WAKEAFFINE,
> + SD_DIST_NR,
> +};
> +
> +static const struct {
> + const char *name;
> + unsigned int flags;
> +} sd_dist_groups[SD_DIST_NR] = {
> + [SD_DIST_FORKEXEC] = { "forkexec", SD_BALANCE_FORK | SD_BALANCE_EXEC },
> + [SD_DIST_NEWIDLE] = { "newidle", SD_BALANCE_NEWIDLE },
> + [SD_DIST_WAKEAFFINE] = { "wakeaffine", SD_WAKE_AFFINE },
> +};
> +
> +/* -1: not set on the command line. */
> +static int sd_dist[SD_DIST_NR] __ro_after_init = { -1, -1, -1 };
> +
> +static int __init setup_sched_numa_flag_distance(char *str)
> +{
> + int dist[SD_DIST_NR] = { -1, -1, -1 };
> + char *tok;
> + int i;
> +
> + while ((tok = strsep(&str, ",")) != NULL) {
> + char *val = strchr(tok, ':');
> + int d;
> +
> + if (!val)
> + goto invalid;
> + *val++ = '\0';
> +
> + for (i = 0; i < SD_DIST_NR; i++) {
> + if (!strcmp(tok, sd_dist_groups[i].name))
> + break;
> + }
> + if (i == SD_DIST_NR || kstrtoint(val, 10, &d) || d < 0)
> + goto invalid;
> +
> + dist[i] = d;
> + }
> +
> + memcpy(sd_dist, dist, sizeof(sd_dist));
> + return 1;
> +
> +invalid:
> + pr_warn("sched_numa_flag_distance=: invalid entry \"%s\", ignored\n", tok);
> + return 1;
> +}
> +__setup("sched_numa_flag_distance=", setup_sched_numa_flag_distance);
> +
> +/* Balance flags to strip from a NUMA level of node distance @distance. */
> +static unsigned int sd_numa_strip_flags(int distance)
> +{
> + unsigned int flags = 0;
> + int i;
> +
> + for (i = 0; i < SD_DIST_NR; i++) {
> + int d = sd_dist[i];
> +
> + if (d < 0) {
> + if (i == SD_DIST_NEWIDLE)
> + continue;
> + d = node_reclaim_distance;
> + }
> + if (distance > d)
> + flags |= sd_dist_groups[i].flags;
> + }
> +
> + return flags;
> +}
> +#endif /* CONFIG_NUMA */
> +
> static struct sched_domain *
> sd_init(struct sched_domain_topology_level *tl,
> const struct cpumask *cpu_map,
> @@ -2030,11 +2116,7 @@ sd_init(struct sched_domain_topology_level *tl,
>
> sd->flags &= ~SD_PREFER_SIBLING;
> sd->flags |= SD_SERIALIZE;
> - if (sched_domains_numa_distance[tl->numa_level] > node_reclaim_distance) {
> - sd->flags &= ~(SD_BALANCE_EXEC |
> - SD_BALANCE_FORK |
> - SD_WAKE_AFFINE);
> - }
> + sd->flags &= ~sd_numa_strip_flags(sched_domains_numa_distance[tl->numa_level]);
>
> #endif /* CONFIG_NUMA */
> } else {
prev parent reply other threads:[~2026-11-17 19:10 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 13:18 [PATCH] sched/topology: Add new kernel parameter to tune NUMA domain flags Jianyong Wu
2026-11-17 19:10 ` Tim Chen [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0f6295ad41cce262149b939c8a245df76bfe5719.camel@linux.intel.com \
--to=tim.c.chen@linux.intel.com \
--cc=bsegall@google.com \
--cc=corbet@lwn.net \
--cc=dietmar.eggemann@arm.com \
--cc=huangsj@hygon.cn \
--cc=jianyong.wu@outlook.com \
--cc=juri.lelli@redhat.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rostedt@goodmis.org \
--cc=skhan@linuxfoundation.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=wangfengyu@hygon.cn \
--cc=wujianyong@hygon.cn \
--cc=yu.c.chen@intel.com \
--cc=zhongyuan@hygon.cn \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox