[PATCH RESEND] x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled

[PATCH RESEND] x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled

[Li Chen]

From: Li Chen <chenl311@chinatelecom.cn>

Currently, the SMT domain is added into sched_domain_topology
by default if CONFIG_SCHED_SMT is enabled.

If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight
is just 1, it will destroy_sched_domain it.

On a large machine, such as one with 512 cores, this results in
512 redundant domain attach/destroy operations.

We can avoid these unnecessary operations by simply checking
cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT
is not enabled.

Signed-off-by: Li Chen <chenl311@chinatelecom.cn>
---

[RESEND] because I forgot to add any mailing list previously.

arch/x86/kernel/smpboot.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index d6cf1e23c2a3..da6192e1af12 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -485,9 +485,11 @@ static void __init build_sched_topology(void)
    int i = 0;

#ifdef CONFIG_SCHED_SMT
-    x86_topology[i++] = (struct sched_domain_topology_level){
-        cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
-    };
+    if (cpu_smt_num_threads > 1) {
+        x86_topology[i++] = (struct sched_domain_topology_level){
+            cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
+        };
+    }
#endif
#ifdef CONFIG_SCHED_CLUSTER
    x86_topology[i++] = (struct sched_domain_topology_level){
--
2.48.1

Re: [PATCH RESEND] x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled

[Li Chen]

Gentle ping.

 ---- On Tue, 22 Apr 2025 16:47:18 +0800  Li Chen <me@linux.beauty> wrote --- 
 > From: Li Chen <chenl311@chinatelecom.cn>
 > 
 > Currently, the SMT domain is added into sched_domain_topology
 > by default if CONFIG_SCHED_SMT is enabled.
 > 
 > If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight
 > is just 1, it will destroy_sched_domain it.
 > 
 > On a large machine, such as one with 512 cores, this results in
 > 512 redundant domain attach/destroy operations.
 > 
 > We can avoid these unnecessary operations by simply checking
 > cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT
 > is not enabled.
 > 
 > Signed-off-by: Li Chen <chenl311@chinatelecom.cn>
 > ---
 > 
 > [RESEND] because I forgot to add any mailing list previously.
 > 
 > arch/x86/kernel/smpboot.c | 8 +++++---
 > 1 file changed, 5 insertions(+), 3 deletions(-)
 > 
 > diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
 > index d6cf1e23c2a3..da6192e1af12 100644
 > --- a/arch/x86/kernel/smpboot.c
 > +++ b/arch/x86/kernel/smpboot.c
 > @@ -485,9 +485,11 @@ static void __init build_sched_topology(void)
 >     int i = 0;
 > 
 > #ifdef CONFIG_SCHED_SMT
 > -    x86_topology[i++] = (struct sched_domain_topology_level){
 > -        cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
 > -    };
 > +    if (cpu_smt_num_threads > 1) {
 > +        x86_topology[i++] = (struct sched_domain_topology_level){
 > +            cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
 > +        };
 > +    }
 > #endif
 > #ifdef CONFIG_SCHED_CLUSTER
 >     x86_topology[i++] = (struct sched_domain_topology_level){
 > --
 > 2.48.1
 > 
 > 

Regards,
Li

Re: [PATCH RESEND] x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled

[Thomas Gleixner]

On Tue, Apr 22 2025 at 16:47, Li Chen wrote:
> Currently, the SMT domain is added into sched_domain_topology
> by default if CONFIG_SCHED_SMT is enabled.
>
> If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight
> is just 1, it will destroy_sched_domain it.

If cpu_attach_domain() ..., it will destroy it.

> On a large machine, such as one with 512 cores, this results in
> 512 redundant domain attach/destroy operations.
>
> We can avoid these unnecessary operations by simply checking

s/We can avoid/Avoid/

> cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT

the SMT domain 

> is not enabled.
>
> #ifdef CONFIG_SCHED_SMT
> -    x86_topology[i++] = (struct sched_domain_topology_level){
> -        cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
> -    };
> +    if (cpu_smt_num_threads > 1) {
> +        x86_topology[i++] = (struct sched_domain_topology_level){
> +            cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
> +        };
> +    }

Looks about right, though I really detest this coding style. I'm not
blaming you, as you just followed the already existing bad taste...

Thanks,

        tglx

[PATCH] x86/smpboot: Decrapify build_sched_topology()

[Thomas Gleixner]

The #ifdeffery and the initializers in build_sched_topology() are just
disgusting. The SCHED_SMT #ifdef is also pointless because SCHED_SMT is
unconditionally enabled when SMP is enabled.

Statically initialize the domain levels in the topology array and let
build_sched_topology() invalidate the package domain level when NUMA in
package is available.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
Note: This allows to remove the SMT level with a simple memmove()
---
 arch/x86/kernel/smpboot.c |   45 +++++++++++++++++----------------------------
 1 file changed, 17 insertions(+), 28 deletions(-)

--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -478,43 +478,32 @@ static int x86_cluster_flags(void)
  */
 static bool x86_has_numa_in_package;
 
-static struct sched_domain_topology_level x86_topology[6];
+#define DOMAIN(maskfn, flagsfn, dname) { .mask = maskfn, .sd_flags = flagsfn, .name = #dname }
 
-static void __init build_sched_topology(void)
-{
-	int i = 0;
-
-#ifdef CONFIG_SCHED_SMT
-	x86_topology[i++] = (struct sched_domain_topology_level){
-		cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
-	};
-#endif
+static struct sched_domain_topology_level x86_topology[] = {
+	DOMAIN(cpu_smt_mask, cpu_smt_flags, SMT),
 #ifdef CONFIG_SCHED_CLUSTER
-	x86_topology[i++] = (struct sched_domain_topology_level){
-		cpu_clustergroup_mask, x86_cluster_flags, SD_INIT_NAME(CLS)
-	};
+	DOMAIN(cpu_clustergroup_mask, x86_cluster_flags, CLS),
 #endif
 #ifdef CONFIG_SCHED_MC
-	x86_topology[i++] = (struct sched_domain_topology_level){
-		cpu_coregroup_mask, x86_core_flags, SD_INIT_NAME(MC)
-	};
+	DOMAIN(cpu_coregroup_mask, x86_core_flags, MC),
 #endif
-	/*
-	 * When there is NUMA topology inside the package skip the PKG domain
-	 * since the NUMA domains will auto-magically create the right spanning
-	 * domains based on the SLIT.
-	 */
-	if (!x86_has_numa_in_package) {
-		x86_topology[i++] = (struct sched_domain_topology_level){
-			cpu_cpu_mask, x86_sched_itmt_flags, SD_INIT_NAME(PKG)
-		};
-	}
+	DOMAIN(cpu_cpu_mask, x86_sched_itmt_flags, PKG),
+	{ NULL },
+};
 
+static void __init build_sched_topology(void)
+{
 	/*
-	 * There must be one trailing NULL entry left.
+	 * When there is NUMA topology inside the package invalidate the
+	 * PKG domain since the NUMA domains will auto-magically create the
+	 * right spanning domains based on the SLIT.
 	 */
-	BUG_ON(i >= ARRAY_SIZE(x86_topology)-1);
+	if (x86_has_numa_in_package) {
+		unsigned int pkgdom = ARRAY_SIZE(x86_topology) - 2;
 
+		memset(&x86_topology[pkgdom], 0, sizeof(x86_topology[pkgdom]));
+	}
 	set_sched_topology(x86_topology);
 }
 

Re: [PATCH RESEND] x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled

[Li Chen]

Hi Thomas,

 ---- On Fri, 20 Jun 2025 21:54:27 +0800  Thomas Gleixner <tglx@linutronix.de> wrote --- 
 > On Tue, Apr 22 2025 at 16:47, Li Chen wrote:
 > > Currently, the SMT domain is added into sched_domain_topology
 > > by default if CONFIG_SCHED_SMT is enabled.
 > >
 > > If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight
 > > is just 1, it will destroy_sched_domain it.
 > 
 > If cpu_attach_domain() ..., it will destroy it.
 > 
 > > On a large machine, such as one with 512 cores, this results in
 > > 512 redundant domain attach/destroy operations.
 > >
 > > We can avoid these unnecessary operations by simply checking
 > 
 > s/We can avoid/Avoid/
 > 
 > > cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT
 > 
 > the SMT domain 
 > 
 > > is not enabled.
 > >
 > > #ifdef CONFIG_SCHED_SMT
 > > -    x86_topology[i++] = (struct sched_domain_topology_level){
 > > -        cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
 > > -    };
 > > +    if (cpu_smt_num_threads > 1) {
 > > +        x86_topology[i++] = (struct sched_domain_topology_level){
 > > +            cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
 > > +        };
 > > +    }
 > 
 > Looks about right, though I really detest this coding style. I'm not
 > blaming you, as you just followed the already existing bad taste...
 > 
 > Thanks,
 > 
 >         tglx
 > 

Thanks for your review, I have fixed the wording issues here: https://lore.kernel.org/all/20250624085559.69436-1-me@linux.beauty/T/#t

Regards,
Li