From: Rahul Rameshbabu <rrameshbabu@nvidia.com>
To: Michael Ellerman <mpe@ellerman.id.au>
Cc: Gal Pressman <gal@nvidia.com>,
linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] powerpc/64s: Increase default stack size to 32KB
Date: Wed, 14 Feb 2024 12:47:39 -0800 [thread overview]
Message-ID: <87ttmau5mg.fsf@nvidia.com> (raw)
In-Reply-To: <20231215124449.317597-1-mpe@ellerman.id.au>
On Fri, 15 Dec, 2023 23:44:49 +1100 Michael Ellerman <mpe@ellerman.id.au> wrote:
> There are reports of kernels crashing due to stack overflow while
> running OpenShift (Kubernetes). The primary contributor to the stack
> usage seems to be openvswitch, which is used by OVN-Kubernetes (based on
> OVN (Open Virtual Network)), but NFS also contributes in some stack
> traces.
>
> There may be some opportunities to reduce stack usage in the openvswitch
> code, but doing so potentially require tradeoffs vs performance, and
> also requires testing across architectures.
>
> Looking at stack usage across the kernel (using -fstack-usage), shows
> that ppc64le stack frames are on average 50-100% larger than the
> equivalent function built for x86-64. Which is not surprising given the
> minimum stack frame size is 32 bytes on ppc64le vs 16 bytes on x86-64.
>
> So increase the default stack size to 32KB for the modern 64-bit Book3S
> platforms, ie. pseries (virtualised) and powernv (bare metal). That
> leaves the older systems like G5s, and the AmigaOne (pasemi) with a 16KB
> stack which should be sufficient on those machines.
>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> ---
We noticed this change is causing assembler issues for us when building
the kernel.
make ARCH=powerpc KERNELRELEASE=6.8.0-rc2_for_upstream_debug_2024_02_06_20_01 KBUILD_BUILD_VERSION=1
arch/powerpc/kernel/switch.S: Assembler messages:
arch/powerpc/kernel/switch.S:249: Error: operand out of range (0x000000000000fe50 is not between 0xffffffffffff8000 and 0x0000000000007fff)
make[6]: *** [scripts/Makefile.build:361: arch/powerpc/kernel/switch.o] Error 1
make[5]: *** [scripts/Makefile.build:481: arch/powerpc/kernel] Error 2
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [scripts/Makefile.build:481: arch/powerpc] Error 2
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [Makefile:1921: .] Error 2
The issue lies with the do_switch_64 macro.
--
Thanks,
Rahul Rameshbabu
WARNING: multiple messages have this Message-ID (diff)
From: Rahul Rameshbabu <rrameshbabu@nvidia.com>
To: Michael Ellerman <mpe@ellerman.id.au>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
Gal Pressman <gal@nvidia.com>
Subject: Re: [PATCH] powerpc/64s: Increase default stack size to 32KB
Date: Wed, 14 Feb 2024 12:47:39 -0800 [thread overview]
Message-ID: <87ttmau5mg.fsf@nvidia.com> (raw)
In-Reply-To: <20231215124449.317597-1-mpe@ellerman.id.au>
On Fri, 15 Dec, 2023 23:44:49 +1100 Michael Ellerman <mpe@ellerman.id.au> wrote:
> There are reports of kernels crashing due to stack overflow while
> running OpenShift (Kubernetes). The primary contributor to the stack
> usage seems to be openvswitch, which is used by OVN-Kubernetes (based on
> OVN (Open Virtual Network)), but NFS also contributes in some stack
> traces.
>
> There may be some opportunities to reduce stack usage in the openvswitch
> code, but doing so potentially require tradeoffs vs performance, and
> also requires testing across architectures.
>
> Looking at stack usage across the kernel (using -fstack-usage), shows
> that ppc64le stack frames are on average 50-100% larger than the
> equivalent function built for x86-64. Which is not surprising given the
> minimum stack frame size is 32 bytes on ppc64le vs 16 bytes on x86-64.
>
> So increase the default stack size to 32KB for the modern 64-bit Book3S
> platforms, ie. pseries (virtualised) and powernv (bare metal). That
> leaves the older systems like G5s, and the AmigaOne (pasemi) with a 16KB
> stack which should be sufficient on those machines.
>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> ---
We noticed this change is causing assembler issues for us when building
the kernel.
make ARCH=powerpc KERNELRELEASE=6.8.0-rc2_for_upstream_debug_2024_02_06_20_01 KBUILD_BUILD_VERSION=1
arch/powerpc/kernel/switch.S: Assembler messages:
arch/powerpc/kernel/switch.S:249: Error: operand out of range (0x000000000000fe50 is not between 0xffffffffffff8000 and 0x0000000000007fff)
make[6]: *** [scripts/Makefile.build:361: arch/powerpc/kernel/switch.o] Error 1
make[5]: *** [scripts/Makefile.build:481: arch/powerpc/kernel] Error 2
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [scripts/Makefile.build:481: arch/powerpc] Error 2
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [Makefile:1921: .] Error 2
The issue lies with the do_switch_64 macro.
--
Thanks,
Rahul Rameshbabu
next prev parent reply other threads:[~2024-02-15 0:36 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-15 12:44 [PATCH] powerpc/64s: Increase default stack size to 32KB Michael Ellerman
2023-12-15 12:44 ` Michael Ellerman
2023-12-19 12:32 ` Michael Ellerman
2024-02-14 20:47 ` Rahul Rameshbabu [this message]
2024-02-14 20:47 ` Rahul Rameshbabu
2024-02-14 21:56 ` Michael Ellerman
2024-02-14 21:56 ` Michael Ellerman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ttmau5mg.fsf@nvidia.com \
--to=rrameshbabu@nvidia.com \
--cc=gal@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.