From: David Laight <david.laight.linux@gmail.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Usama Arif <usama.arif@linux.dev>,
tglx@kernel.org, mingo@redhat.com, bp@alien8.de,
dave.hansen@linux.intel.com, x86@kernel.org, kees@kernel.org,
hpa@zytor.com, gustavoars@kernel.org, ardb@kernel.org,
hca@linux.ibm.com, nathan@kernel.org, ndesaulniers@google.com,
morbo@google.com, justinstitt@google.com,
linux-hardening@vger.kernel.org, llvm@lists.linux.dev,
linux-kernel@vger.kernel.org, riel@surriel.com,
shakeel.butt@linux.dev, kernel-team@meta.com, rkeilty@meta.com,
atish.patra@linux.dev, dgreid@meta.com
Subject: Re: [PATCH 2/2] x86/kstack_erase: Use REP STOSQ for large stack fills
Date: Fri, 20 Nov 2026 14:02:00 +0000 [thread overview]
Message-ID: <20261120140200.53ec2557@pumpkin> (raw)
In-Reply-To: <20261120104534.GO2668564@noisy.programming.kicks-ass.net>
On Fri, 20 Nov 2026 11:45:34 +0100
Peter Zijlstra <peterz@infradead.org> wrote:
> On Thu, Nov 19, 2026 at 01:31:08PM -0800, Usama Arif wrote:
> > +#ifdef CONFIG_X86_64
> > +#define __stackleak_poison __stackleak_poison
> > +static __always_inline void __stackleak_poison(unsigned long erase_low,
> > + unsigned long erase_high,
> > + unsigned long poison)
> > +{
> > + unsigned long count;
> > +
> > + /* Short spans do not amortize the startup cost of REP STOSQ. */
> > + if (erase_high <= erase_low + 512) {
> > + while (erase_low < erase_high) {
> > + *(unsigned long *)erase_low = poison;
> > + erase_low += sizeof(unsigned long);
> > + }
> > + return;
> > + }
> > +
> > + count = (erase_high - erase_low) / sizeof(unsigned long);
> > + asm volatile("rep stosq"
> > + : "+D" (erase_low), "+c" (count)
> > + : "a" (poison)
> > + : "memory");
> > +}
> > +#endif
>
> Gemini seems to suggest 16-32 words would be sufficient to amortize the
> REP STOS setup.
That really does sound like wishful thinking.
Especially if you go back a few generations on the cpu architecture.
> It was also adamant I try and unroll that earlier loop.
>
> How about something like so, does that make sense? Also, I suppose we
> should make memset64() __always_inline if we use it here. I'm not at all
> sold on the unroll here, the asm is pretty horrid for what it does.
>
> ---
> --- a/arch/x86/include/asm/processor.h
> +++ b/arch/x86/include/asm/processor.h
> @@ -28,6 +28,7 @@ struct vm86;
> #include <asm/vmxfeatures.h>
> #include <asm/vdso/processor.h>
> #include <asm/shstk.h>
> +#include <asm/string.h>
>
> #include <linux/personality.h>
> #include <linux/cache.h>
> @@ -580,6 +581,33 @@ static __always_inline bool on_thread_st
> current_stack_pointer) < THREAD_SIZE;
> }
>
> +#ifdef CONFIG_X86_64
> +#define __stackleak_poison __stackleak_poison
> +static __always_inline void __stackleak_poison(unsigned long erase_low,
> + unsigned long erase_high,
> + unsigned long poison)
> +{
> + unsigned long count = (erase_high - erase_low) / sizeof(unsigned long);
> +
> + /* Amortize the startup cost of REP STOSQ. */
> + if (count < 32) {
> + unsigned long *ptr = (unsigned long *)erase_low;
> +
> + while (count) {
> + switch ((count-1) % 4) {
> + case 3: ptr[--count] = poison; fallthrough;
> + case 2: ptr[--count] = poison; fallthrough;
> + case 1: ptr[--count] = poison; fallthrough;
> + case 0: ptr[--count] = poison; break;
> + }
> + }
That looks horrid to me.
You want as few instructions (especially conditionals) inside the
loop as possible.
The Intel cpu will execute a two clock loop and the later ones will
do two writes per clock
(zen5 will execute a one clock loop and do two writes/clock).
So you want a loop with just 4 writes and the loop control code.
The 'pointer add' just needs to execute at the same time as some
of the stores - to avoid lengthening the register dependency chain.
For this code, can the length be a multiple of 32?
Also on some cpu the 'rep stos' might be twice as fast (per byte) if
the destination in 64 byte aligned - true for 'rep movsb'.
David
> + return;
> + }
> +
> + memset64((void *)erase_low, poison, count);
> +}
> +#endif
> +
> #ifdef CONFIG_PARAVIRT_XXL
> #include <asm/paravirt.h>
> #else
>
next prev parent reply other threads:[~2026-11-20 14:02 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 21:31 [PATCH 0/2] x86/kstack_erase: Faster stack poisoning, plus a startup build fix Usama Arif
2026-11-19 21:31 ` [PATCH 1/2] x86/boot/startup: Disable kernel stack depth tracking Usama Arif
2026-11-19 22:30 ` Nick Desaulniers
2026-11-19 21:31 ` [PATCH 2/2] x86/kstack_erase: Use REP STOSQ for large stack fills Usama Arif
2026-11-19 22:28 ` Nick Desaulniers
2026-11-20 11:38 ` David Laight
2026-11-20 12:42 ` David Laight
2026-11-20 18:46 ` Usama Arif
2026-11-20 10:45 ` Peter Zijlstra
2026-11-20 12:26 ` Peter Zijlstra
2026-11-20 18:43 ` Usama Arif
2026-11-20 14:02 ` David Laight [this message]
2026-11-20 15:35 ` Peter Zijlstra
2026-11-20 16:18 ` David Laight
2026-11-20 18:55 ` David Laight
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120140200.53ec2557@pumpkin \
--to=david.laight.linux@gmail.com \
--cc=ardb@kernel.org \
--cc=atish.patra@linux.dev \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=dgreid@meta.com \
--cc=gustavoars@kernel.org \
--cc=hca@linux.ibm.com \
--cc=hpa@zytor.com \
--cc=justinstitt@google.com \
--cc=kees@kernel.org \
--cc=kernel-team@meta.com \
--cc=linux-hardening@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=mingo@redhat.com \
--cc=morbo@google.com \
--cc=nathan@kernel.org \
--cc=ndesaulniers@google.com \
--cc=peterz@infradead.org \
--cc=riel@surriel.com \
--cc=rkeilty@meta.com \
--cc=shakeel.butt@linux.dev \
--cc=tglx@kernel.org \
--cc=usama.arif@linux.dev \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.