The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Tim Chen <tim.c.chen@linux.intel.com>
To: Jianyong Wu <wujianyong@hygon.cn>,
	Peter Zijlstra <peterz@infradead.org>,
	 Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot	 <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt	 <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>, Mel Gorman	 <mgorman@suse.de>,
	Valentin Schneider <vschneid@redhat.com>,
	K Prateek Nayak	 <kprateek.nayak@amd.com>,
	Chen Yu <yu.c.chen@intel.com>, Jonathan Corbet	 <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Randy Dunlap	 <rdunlap@infradead.org>,
	linux-doc@vger.kernel.org,  linux-kernel@vger.kernel.org,
	jianyong.wu@outlook.com, wangfengyu@hygon.cn, zhongyuan@hygon.cn,
	huangsj@hygon.cn
Subject: Re: [PATCH] sched/topology: Add new kernel parameter to tune NUMA domain flags
Date: Tue, 17 Nov 2026 11:10:19 -0800	[thread overview]
Message-ID: <0f6295ad41cce262149b939c8a245df76bfe5719.camel@linux.intel.com> (raw)
In-Reply-To: <20261117131854.924760-1-wujianyong@hygon.cn>

On Tue, 2026-11-17 at 21:18 +0800, Jianyong Wu wrote:
> Sched domain flags can largely affect performance on systems with
> multiple NUMA levels, especially SD_BALANCE_FORK, SD_BALANCE_EXEC,
> SD_WAKE_AFFINE and SD_BALANCE_NEWIDLE.

Jianyong,

A few thoughts on the three flags.

> 
> SD_BALANCE_FORK and SD_BALANCE_EXEC decide how far the slow path of
> select_task_rq_fair() searches. Stripping them from the high NUMA levels
> reduces the number of CPUs to traverse and speeds up fork/exec.
> SD_WAKE_AFFINE decides the wakeup scope: if the waker and the wakee are
> farther apart than the highest level carrying it, the wakee cannot be
> pulled next to the waker on wakeup, which may hurt performance. 
> 

On your data later for Machine B, you say that levels at 32, 35 and 38 are
beyond node_reclaim_distance, so probably Hygon is running with the generic
node reclaim distance 30.

AMD sets node reclaim_distance=32 in init_amd_zen_common() for this reason.
You can probably do the same for Hygon, to retain SD_BALANCE_FORK and
SD_BALANCE_EXEC at the right level.

> However,
> these three flags are all controlled by a single static threshold,
> node_reclaim_distance, which can neither tell them apart nor be changed.
> Different workloads on the same system favour different settings, so no
> single static value fits them all.
> 
> The same applies to SD_BALANCE_NEWIDLE, which is set on every NUMA level
> and cannot be stripped. Newidle balance on a high NUMA level can take a
> long time, even when there is no task in the system to pull.

There is an existing boot knob relax_domain_level= ,
which clears SD_BALANCE_NEWIDLE on every domain at or above 
the given level, see set_domain_attribute(). On Machine A,                                                                                                             
relax_domain_level=<level of the distance-25 NUMA domain> should give                                                                                                              
the same domain flags as newidle:20. Can you confirm with the                                                                                                                      
/sys/kernel/debug/sched/domains/ flags and rerun schbench with that                                                                                                                
boot parameter and NI_RANDOM set?  I think we shouldn't introduce
another boot parameter to control the same flag.

I've made similar suggestions before to Kunyu in his test on Hygon system:
https://lore.kernel.org/lkml/6d8ff4593cc93c0033c6537eeac281085726c220.camel@linux.intel.com/

Will you achieve most of the gain for your workload after adjusting
node_reclaim_distance and setting appropriate relax_domain_level,
instead of introducing a new kernel parameter? 

Tim
                                                
> 
> Introduce a new kernel parameter, sched_numa_flag_distance, to let users
> tune the flags above separately on the NUMA levels. A group is stripped
> from the levels whose node distance is greater than the given value;
> once set, this overrides the default for that group:
> 
>   sched_numa_flag_distance=forkexec:<d>,newidle:<d>,wakeaffine:<d>
> 
> Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
> ---
> 
> Notes:
>     Test results
>     
>     All results are normalized to the default (no parameter) = 1.00.
>     UnixBench: higher is better; latency and time: lower is better.
>     kernel.numa_balancing=0 for all tests.
>     
>     Machine A: Hygon, 2 sockets, 256 CPUs, 8 nodes. All NUMA levels (15,
>     25, 27) are below node_reclaim_distance, so by default every NUMA level
>     has fork/exec, newidle and wake_affine.
>     
>       node   0   1   2   3   4   5   6   7
>         0:  10  15  15  15  25  27  27  27
>         1:  15  10  15  15  27  25  27  27
>         2:  15  15  10  15  27  27  25  27
>         3:  15  15  15  10  27  27  27  25
>         4:  25  27  27  27  10  15  15  15
>         5:  27  25  27  27  15  10  15  15
>         6:  27  27  25  27  15  15  10  15
>         7:  27  27  27  25  15  15  15  10
>     
>     Each setting was tested in its own boot of the same kernel:
>     forkexec:10 drops fork/exec balancing on every NUMA level, newidle:20
>     drops newidle balancing at distance 25 and 27. The default column is
>     the average of two boots without the parameter.
>     
>     forkexec:10, tested with C2 disabled on all CPUs
>     (echo 1 > /sys/devices/system/cpu/cpu*/cpuidle/state2/disable). The
>     two default boots differ by at most 2.9%.
>     
>     UnixBench (byte-unixbench), 1 copy, average of 3 rounds:
>     
>       ./Run -c 1 -i 3 execl spawn shell1 shell8
>     
>                                     default  forkexec:10
>       Execl Throughput                 1.00      1.27
>       Process Creation                 1.00      1.29
>       Shell Scripts (1 concurrent)     1.00      1.35
>       Shell Scripts (8 concurrent)     1.00      2.07
>     
>     Time spent in select_task_rq_fair() (function_graph) while one task
>     forks and reaps 10000 children, then forks 5000 children that exec
>     /bin/true, 3 times:
>     
>                                     default  forkexec:10
>       fork                             1.00      0.05
>       exec                             1.00      0.05
>     
>     newidle:20, tested with all C-states enabled. The two default boots
>     differ by at most 1%.
>     
>     schbench (git.kernel.org/pub/scm/linux/kernel/git/mason/schbench.git),
>     8 message threads with 32 workers each, i.e. 256 worker threads, 30s
>     run after a 5s warmup, average of 3 runs:
>     
>       schbench -m 8 -t 32 -r 30 -w 5
>     
>                                     default  newidle:20
>       p99 wakeup latency               1.00      0.56
>       requests per second              1.00      1.00
>     
>     Machine B: Hygon, 2 sockets, 384 CPUs, 12 nodes. The NUMA levels at 32,
>     35 and 38 are beyond node_reclaim_distance, so by default they have no
>     fork/exec and no wake_affine.
>     
>       node   0   1   2   3   4   5   6   7   8   9  10  11
>         0:  10  19  15  12  15  12  32  35  32  29  38  35
>         1:  19  10  12  15  12  15  35  32  35  38  29  32
>         2:  15  12  10  12  15  12  32  35  38  35  32  35
>         3:  12  15  12  10  19  15  29  38  35  32  35  32
>         4:  15  12  15  19  10  12  38  29  32  35  32  35
>         5:  12  15  12  15  12  10  35  32  35  32  35  38
>         6:  32  35  32  29  38  35  10  19  15  12  15  12
>         7:  35  32  35  38  29  32  19  10  12  15  12  15
>         8:  32  35  38  35  32  35  15  12  10  12  15  12
>         9:  29  38  35  32  35  32  12  15  12  10  19  15
>        10:  38  29  32  35  32  35  15  12  15  19  10  12
>        11:  35  32  35  32  35  38  12  15  12  15  12  10
>     
>     Booted with sched_numa_flag_distance=wakeaffine:255, i.e. wake_affine
>     on every NUMA level.
>     
>     hackbench from rt-tests (v2.20), pipes, each group being 20 senders and
>     20 receivers (40 tasks), 3 rounds of 3 runs:
>     
>       hackbench -p -g 4 -l 5000          (processes, 160 tasks)
>       hackbench -p -g 16 -l 1500         (processes, 640 tasks)
>       hackbench -p -T -g 4 -l 5000       (threads, 160 tasks)
>       hackbench -p -T -g 16 -l 1500      (threads, 640 tasks)
>     
>       time                          default  wakeaffine:255
>       processes, 4 groups              1.00      0.80
>       processes, 16 groups             1.00      0.69
>       threads, 4 groups                1.00      0.77
>       threads, 16 groups               1.00      0.69
>     
>     Open question
>     
>     This version only adds a boot parameter. Since the best setting follows
>     the workload, a runtime knob would be more useful. Would one be
>     acceptable, and if so, where: sysfs, or debugfs next to the other
>     scheduler tunables? Changing it at runtime would rebuild the sched
>     domains.
> 
>  .../admin-guide/kernel-parameters.txt         | 15 +++
>  kernel/sched/topology.c                       | 92 ++++++++++++++++++-
>  2 files changed, 102 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 6169787d3805..e4d504992cce 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -6798,6 +6798,21 @@ Kernel parameters
>  			cost of significant additional memory use for tables.
>  
>  
> +	sched_numa_flag_distance= [KNL,NUMA]
> +			Format: <group>:<distance>[,<group>:<distance>...]
> +			Strip a group of scheduler balance flags from the
> +			NUMA sched domain levels whose node distance is
> +			greater than <distance>. <group> is one of:
> +			forkexec   - SD_BALANCE_FORK and SD_BALANCE_EXEC,
> +				     default: node_reclaim_distance
> +			newidle    - SD_BALANCE_NEWIDLE, default: never
> +			wakeaffine - SD_WAKE_AFFINE,
> +				     default: node_reclaim_distance
> +			node_reclaim_distance is 30 (RECLAIM_DISTANCE), or
> +			32 on AMD Zen. A distance of 255 keeps the group on
> +			every NUMA level.
> +			Example: sched_numa_flag_distance=forkexec:10,newidle:20
> +
>  	sched_proxy_exec= [KNL]
>  			Enables or disables "proxy execution" style
>  			solution to mutex-based priority inversion.
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 919d0fb00bd9..3bcf0db0800a 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -1947,6 +1947,92 @@ static struct cpumask		***sched_domains_numa_masks;
>  	 SD_NUMA		|	\
>  	 SD_ASYM_PACKING)
>  
> +#ifdef CONFIG_NUMA
> +/*
> + * sched_numa_flag_distance=<group>:<distance>[,...]
> + *
> + * Strip a group of balance flags from the NUMA sched domain levels whose
> + * node distance is greater than <distance>. Groups left unset keep the
> + * default: fork/exec and wake_affine are stripped beyond
> + * node_reclaim_distance, newidle is never stripped.
> + *
> + * Node distances grow with the level, so a group is stripped from a level
> + * and every level above it, as the SDF_SHARED_CHILD flags require.
> + */
> +enum {
> +	SD_DIST_FORKEXEC,
> +	SD_DIST_NEWIDLE,
> +	SD_DIST_WAKEAFFINE,
> +	SD_DIST_NR,
> +};
> +
> +static const struct {
> +	const char	*name;
> +	unsigned int	flags;
> +} sd_dist_groups[SD_DIST_NR] = {
> +	[SD_DIST_FORKEXEC]	= { "forkexec",	  SD_BALANCE_FORK | SD_BALANCE_EXEC },
> +	[SD_DIST_NEWIDLE]	= { "newidle",	  SD_BALANCE_NEWIDLE },
> +	[SD_DIST_WAKEAFFINE]	= { "wakeaffine", SD_WAKE_AFFINE },
> +};
> +
> +/* -1: not set on the command line. */
> +static int sd_dist[SD_DIST_NR] __ro_after_init = { -1, -1, -1 };
> +
> +static int __init setup_sched_numa_flag_distance(char *str)
> +{
> +	int dist[SD_DIST_NR] = { -1, -1, -1 };
> +	char *tok;
> +	int i;
> +
> +	while ((tok = strsep(&str, ",")) != NULL) {
> +		char *val = strchr(tok, ':');
> +		int d;
> +
> +		if (!val)
> +			goto invalid;
> +		*val++ = '\0';
> +
> +		for (i = 0; i < SD_DIST_NR; i++) {
> +			if (!strcmp(tok, sd_dist_groups[i].name))
> +				break;
> +		}
> +		if (i == SD_DIST_NR || kstrtoint(val, 10, &d) || d < 0)
> +			goto invalid;
> +
> +		dist[i] = d;
> +	}
> +
> +	memcpy(sd_dist, dist, sizeof(sd_dist));
> +	return 1;
> +
> +invalid:
> +	pr_warn("sched_numa_flag_distance=: invalid entry \"%s\", ignored\n", tok);
> +	return 1;
> +}
> +__setup("sched_numa_flag_distance=", setup_sched_numa_flag_distance);
> +
> +/* Balance flags to strip from a NUMA level of node distance @distance. */
> +static unsigned int sd_numa_strip_flags(int distance)
> +{
> +	unsigned int flags = 0;
> +	int i;
> +
> +	for (i = 0; i < SD_DIST_NR; i++) {
> +		int d = sd_dist[i];
> +
> +		if (d < 0) {
> +			if (i == SD_DIST_NEWIDLE)
> +				continue;
> +			d = node_reclaim_distance;
> +		}
> +		if (distance > d)
> +			flags |= sd_dist_groups[i].flags;
> +	}
> +
> +	return flags;
> +}
> +#endif /* CONFIG_NUMA */
> +
>  static struct sched_domain *
>  sd_init(struct sched_domain_topology_level *tl,
>  	const struct cpumask *cpu_map,
> @@ -2030,11 +2116,7 @@ sd_init(struct sched_domain_topology_level *tl,
>  
>  		sd->flags &= ~SD_PREFER_SIBLING;
>  		sd->flags |= SD_SERIALIZE;
> -		if (sched_domains_numa_distance[tl->numa_level] > node_reclaim_distance) {
> -			sd->flags &= ~(SD_BALANCE_EXEC |
> -				       SD_BALANCE_FORK |
> -				       SD_WAKE_AFFINE);
> -		}
> +		sd->flags &= ~sd_numa_strip_flags(sched_domains_numa_distance[tl->numa_level]);
>  
>  #endif /* CONFIG_NUMA */
>  	} else {

      reply	other threads:[~2026-11-17 19:10 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 13:18 [PATCH] sched/topology: Add new kernel parameter to tune NUMA domain flags Jianyong Wu
2026-11-17 19:10 ` Tim Chen [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0f6295ad41cce262149b939c8a245df76bfe5719.camel@linux.intel.com \
    --to=tim.c.chen@linux.intel.com \
    --cc=bsegall@google.com \
    --cc=corbet@lwn.net \
    --cc=dietmar.eggemann@arm.com \
    --cc=huangsj@hygon.cn \
    --cc=jianyong.wu@outlook.com \
    --cc=juri.lelli@redhat.com \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rdunlap@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=skhan@linuxfoundation.org \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    --cc=wangfengyu@hygon.cn \
    --cc=wujianyong@hygon.cn \
    --cc=yu.c.chen@intel.com \
    --cc=zhongyuan@hygon.cn \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox