* [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages
@ 2026-10-07 7:58 Breno Leitao
2026-10-27 11:59 ` Breno Leitao
0 siblings, 1 reply; 6+ messages in thread
From: Breno Leitao @ 2026-10-07 7:58 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Mark Rutland
Cc: linux-arm-kernel, linux-kernel, kas, kernel-team, Breno Leitao
arm64 selects CPUMASK_OFFSTACK when NR_CPUS is above 256, which keeps
cpumask_var_t off the kernel stack. With 64K pages the kernel stack is
64K as well, we have enough room to keep cpumask on the stack, avoiding
extra alocation and possible performance penalties.
Skip the select when ARM64_64K_PAGES is set. A 64K kernel can then raise
NR_CPUS above 256 and keep cpumask_var_t as an inline array. 4K and 16K
page kernels are unchanged.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
Based on Will presentation at LPC, I've decided to have a look at Meta's
stack usage, and I found that this might be an small optimization!?
---
arch/arm64/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 5f2db2cecf9772..b73371327d2a47 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -111,7 +111,7 @@ config ARM64
select CLONE_BACKWARDS
select COMMON_CLK
select CPU_PM if (SUSPEND || CPU_IDLE)
- select CPUMASK_OFFSTACK if NR_CPUS > 256
+ select CPUMASK_OFFSTACK if NR_CPUS > 256 && !ARM64_64K_PAGES
select DCACHE_WORD_ACCESS
select HAVE_EXTRA_IPI_TRACEPOINTS
select DYNAMIC_FTRACE if FUNCTION_TRACER
---
base-commit: eea3fef32a9cf36abcb5975a5a594e4135a6b026
change-id: 20261007-arm_stack-fa76c2d0d69a
Best regards,
--
Breno Leitao <leitao@debian.org>
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages
2026-10-07 7:58 [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages Breno Leitao
@ 2026-10-27 11:59 ` Breno Leitao
2026-10-30 10:28 ` Will Deacon
0 siblings, 1 reply; 6+ messages in thread
From: Breno Leitao @ 2026-10-27 11:59 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Mark Rutland
Cc: linux-arm-kernel, linux-kernel, kas, kernel-team
On Wed, Oct 07, 2026 at 12:58:16AM -0700, Breno Leitao wrote:
> arm64 selects CPUMASK_OFFSTACK when NR_CPUS is above 256, which keeps
> cpumask_var_t off the kernel stack. With 64K pages the kernel stack is
> 64K as well, we have enough room to keep cpumask on the stack, avoiding
> extra alocation and possible performance penalties.
>
> Skip the select when ARM64_64K_PAGES is set. A 64K kernel can then raise
> NR_CPUS above 256 and keep cpumask_var_t as an inline array. 4K and 16K
> page kernels are unchanged.
>
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
> Based on Will presentation at LPC, I've decided to have a look at Meta's
> stack usage, and I found that this might be an small optimization!?
> ---
> arch/arm64/Kconfig | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 5f2db2cecf9772..b73371327d2a47 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -111,7 +111,7 @@ config ARM64
> select CLONE_BACKWARDS
> select COMMON_CLK
> select CPU_PM if (SUSPEND || CPU_IDLE)
> - select CPUMASK_OFFSTACK if NR_CPUS > 256
> + select CPUMASK_OFFSTACK if NR_CPUS > 256 && !ARM64_64K_PAGES
Does this one sound sane?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages
2026-10-27 11:59 ` Breno Leitao
@ 2026-10-30 10:28 ` Will Deacon
2026-10-30 10:46 ` Breno Leitao
0 siblings, 1 reply; 6+ messages in thread
From: Will Deacon @ 2026-10-30 10:28 UTC (permalink / raw)
To: Breno Leitao
Cc: Catalin Marinas, Mark Rutland, linux-arm-kernel, linux-kernel,
kas, kernel-team
On Tue, Oct 27, 2026 at 04:59:52AM -0700, Breno Leitao wrote:
> On Wed, Oct 07, 2026 at 12:58:16AM -0700, Breno Leitao wrote:
> > arm64 selects CPUMASK_OFFSTACK when NR_CPUS is above 256, which keeps
> > cpumask_var_t off the kernel stack. With 64K pages the kernel stack is
> > 64K as well, we have enough room to keep cpumask on the stack, avoiding
> > extra alocation and possible performance penalties.
> >
> > Skip the select when ARM64_64K_PAGES is set. A 64K kernel can then raise
> > NR_CPUS above 256 and keep cpumask_var_t as an inline array. 4K and 16K
> > page kernels are unchanged.
> >
> > Signed-off-by: Breno Leitao <leitao@debian.org>
> > ---
> > Based on Will presentation at LPC, I've decided to have a look at Meta's
> > stack usage, and I found that this might be an small optimization!?
> > ---
> > arch/arm64/Kconfig | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 5f2db2cecf9772..b73371327d2a47 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -111,7 +111,7 @@ config ARM64
> > select CLONE_BACKWARDS
> > select COMMON_CLK
> > select CPU_PM if (SUSPEND || CPU_IDLE)
> > - select CPUMASK_OFFSTACK if NR_CPUS > 256
> > + select CPUMASK_OFFSTACK if NR_CPUS > 256 && !ARM64_64K_PAGES
>
> Does this one sound sane?
It doesn't make an awful lot of sense to me, to be honest. The page size
and the number of CPUs are two independent variables, so tying them
together like this looks a little odd.
Will
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages
2026-10-30 10:28 ` Will Deacon
@ 2026-10-30 10:46 ` Breno Leitao
2026-10-30 11:53 ` Will Deacon
0 siblings, 1 reply; 6+ messages in thread
From: Breno Leitao @ 2026-10-30 10:46 UTC (permalink / raw)
To: Will Deacon
Cc: Catalin Marinas, Mark Rutland, linux-arm-kernel, linux-kernel,
kas, kernel-team
Hello Will,
On Fri, Oct 30, 2026 at 10:28:04AM +0000, Will Deacon wrote:
> On Tue, Oct 27, 2026 at 04:59:52AM -0700, Breno Leitao wrote:
> > On Wed, Oct 07, 2026 at 12:58:16AM -0700, Breno Leitao wrote:
> > > arm64 selects CPUMASK_OFFSTACK when NR_CPUS is above 256, which keeps
> > > cpumask_var_t off the kernel stack. With 64K pages the kernel stack is
> > > 64K as well, we have enough room to keep cpumask on the stack, avoiding
> > > extra alocation and possible performance penalties.
> > >
> > > Skip the select when ARM64_64K_PAGES is set. A 64K kernel can then raise
> > > NR_CPUS above 256 and keep cpumask_var_t as an inline array. 4K and 16K
> > > page kernels are unchanged.
> > >
> > > Signed-off-by: Breno Leitao <leitao@debian.org>
> > > ---
> > > Based on Will presentation at LPC, I've decided to have a look at Meta's
> > > stack usage, and I found that this might be an small optimization!?
> > > ---
> > > arch/arm64/Kconfig | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > > index 5f2db2cecf9772..b73371327d2a47 100644
> > > --- a/arch/arm64/Kconfig
> > > +++ b/arch/arm64/Kconfig
> > > @@ -111,7 +111,7 @@ config ARM64
> > > select CLONE_BACKWARDS
> > > select COMMON_CLK
> > > select CPU_PM if (SUSPEND || CPU_IDLE)
> > > - select CPUMASK_OFFSTACK if NR_CPUS > 256
> > > + select CPUMASK_OFFSTACK if NR_CPUS > 256 && !ARM64_64K_PAGES
> >
> > Does this one sound sane?
>
> It doesn't make an awful lot of sense to me, to be honest. The page size
> and the number of CPUs are two independent variables, so tying them
> together like this looks a little odd.
Sure, but with 64K pages the kernel stack is _huge_. Why do we still
need to keep the cpumask off the stack frame in that case?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages
2026-10-30 10:46 ` Breno Leitao
@ 2026-10-30 11:53 ` Will Deacon
2026-10-30 13:50 ` Breno Leitao
0 siblings, 1 reply; 6+ messages in thread
From: Will Deacon @ 2026-10-30 11:53 UTC (permalink / raw)
To: Breno Leitao
Cc: Catalin Marinas, Mark Rutland, linux-arm-kernel, linux-kernel,
kas, kernel-team
On Fri, Oct 30, 2026 at 03:46:03AM -0700, Breno Leitao wrote:
> Hello Will,
>
> On Fri, Oct 30, 2026 at 10:28:04AM +0000, Will Deacon wrote:
> > On Tue, Oct 27, 2026 at 04:59:52AM -0700, Breno Leitao wrote:
> > > On Wed, Oct 07, 2026 at 12:58:16AM -0700, Breno Leitao wrote:
> > > > arm64 selects CPUMASK_OFFSTACK when NR_CPUS is above 256, which keeps
> > > > cpumask_var_t off the kernel stack. With 64K pages the kernel stack is
> > > > 64K as well, we have enough room to keep cpumask on the stack, avoiding
> > > > extra alocation and possible performance penalties.
> > > >
> > > > Skip the select when ARM64_64K_PAGES is set. A 64K kernel can then raise
> > > > NR_CPUS above 256 and keep cpumask_var_t as an inline array. 4K and 16K
> > > > page kernels are unchanged.
> > > >
> > > > Signed-off-by: Breno Leitao <leitao@debian.org>
> > > > ---
> > > > Based on Will presentation at LPC, I've decided to have a look at Meta's
> > > > stack usage, and I found that this might be an small optimization!?
> > > > ---
> > > > arch/arm64/Kconfig | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > > > index 5f2db2cecf9772..b73371327d2a47 100644
> > > > --- a/arch/arm64/Kconfig
> > > > +++ b/arch/arm64/Kconfig
> > > > @@ -111,7 +111,7 @@ config ARM64
> > > > select CLONE_BACKWARDS
> > > > select COMMON_CLK
> > > > select CPU_PM if (SUSPEND || CPU_IDLE)
> > > > - select CPUMASK_OFFSTACK if NR_CPUS > 256
> > > > + select CPUMASK_OFFSTACK if NR_CPUS > 256 && !ARM64_64K_PAGES
> > >
> > > Does this one sound sane?
> >
> > It doesn't make an awful lot of sense to me, to be honest. The page size
> > and the number of CPUs are two independent variables, so tying them
> > together like this looks a little odd.
>
> Sure, but with 64K pages the kernel stack is _huge_. Why do we still
> need to keep the cpumask off the stack frame in that case?
Because you might have thousands of CPUs? Because copying these things by
value could be expensive?
Will
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages
2026-10-30 11:53 ` Will Deacon
@ 2026-10-30 13:50 ` Breno Leitao
0 siblings, 0 replies; 6+ messages in thread
From: Breno Leitao @ 2026-10-30 13:50 UTC (permalink / raw)
To: Will Deacon
Cc: Catalin Marinas, Mark Rutland, linux-arm-kernel, linux-kernel,
kas, kernel-team
On Fri, Oct 30, 2026 at 11:53:56AM +0000, Will Deacon wrote:
> Because you might have thousands of CPUs?
I checked that case. I built defconfig with 64K pages and NR_CPUS=4096,
with and without this patch, and compared clang's -fstack-usage output.
With the patch, 62 functions grow by 512 bytes or more. The worst are in
cpuset: update_prstate(), cpuset_write_resmask() and
cpuset_handle_hotplug() grow by ~1.5K, to 1648 bytes. Nothing crosses
FRAME_WARN, and the largest frame in the build goes from 1312 to 1648
bytes, out of a 64K stack.
I believe this higher frame usage is still way better than the kmalloc
at function invocation (the reason for this patch), but honestly I don't
have real numbers, and I am happy to take them, if it is worth.
> Because copying these things by value could be expensive?
Hmm, I _think_ having the cpumask on the stack or not doesn't change how
it is passed. cpumask_var_t is either a pointer or a one-element array,
which decays to a pointer, so callees get a struct cpumask * in both
cases.
Which copies do you have in mind?
Thanks,
--breno
--
PS: I am not pushing on this patch at all, I am just trying to
understand the reasoning and learn.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-10-30 13:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-07 7:58 [PATCH] arm64: Kconfig: Don't select CPUMASK_OFFSTACK with 64K pages Breno Leitao
2026-10-27 11:59 ` Breno Leitao
2026-10-30 10:28 ` Will Deacon
2026-10-30 10:46 ` Breno Leitao
2026-10-30 11:53 ` Will Deacon
2026-10-30 13:50 ` Breno Leitao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox