The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: David Laight <david.laight.linux@gmail.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Usama Arif <usama.arif@linux.dev>,
	tglx@kernel.org, mingo@redhat.com, bp@alien8.de,
	dave.hansen@linux.intel.com, x86@kernel.org, kees@kernel.org,
	hpa@zytor.com, gustavoars@kernel.org, ardb@kernel.org,
	hca@linux.ibm.com, nathan@kernel.org, ndesaulniers@google.com,
	morbo@google.com, justinstitt@google.com,
	linux-hardening@vger.kernel.org, llvm@lists.linux.dev,
	linux-kernel@vger.kernel.org, riel@surriel.com,
	shakeel.butt@linux.dev, kernel-team@meta.com, rkeilty@meta.com,
	atish.patra@linux.dev, dgreid@meta.com
Subject: Re: [PATCH 2/2] x86/kstack_erase: Use REP STOSQ for large stack fills
Date: Fri, 20 Nov 2026 14:02:00 +0000	[thread overview]
Message-ID: <20261120140200.53ec2557@pumpkin> (raw)
In-Reply-To: <20261120104534.GO2668564@noisy.programming.kicks-ass.net>

On Fri, 20 Nov 2026 11:45:34 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> On Thu, Nov 19, 2026 at 01:31:08PM -0800, Usama Arif wrote:
> > +#ifdef CONFIG_X86_64
> > +#define __stackleak_poison __stackleak_poison
> > +static __always_inline void __stackleak_poison(unsigned long erase_low,
> > +					       unsigned long erase_high,
> > +					       unsigned long poison)
> > +{
> > +	unsigned long count;
> > +
> > +	/* Short spans do not amortize the startup cost of REP STOSQ. */
> > +	if (erase_high <= erase_low + 512) {
> > +		while (erase_low < erase_high) {
> > +			*(unsigned long *)erase_low = poison;
> > +			erase_low += sizeof(unsigned long);
> > +		}
> > +		return;
> > +	}
> > +
> > +	count = (erase_high - erase_low) / sizeof(unsigned long);
> > +	asm volatile("rep stosq"
> > +		     : "+D" (erase_low), "+c" (count)
> > +		     : "a" (poison)
> > +		     : "memory");
> > +}
> > +#endif  
> 
> Gemini seems to suggest 16-32 words would be sufficient to amortize the
> REP STOS setup.

That really does sound like wishful thinking.
Especially if you go back a few generations on the cpu architecture.

> It was also adamant I try and unroll that earlier loop.
> 
> How about something like so, does that make sense? Also, I suppose we
> should make memset64() __always_inline if we use it here. I'm not at all
> sold on the unroll here, the asm is pretty horrid for what it does.
> 
> ---
> --- a/arch/x86/include/asm/processor.h
> +++ b/arch/x86/include/asm/processor.h
> @@ -28,6 +28,7 @@ struct vm86;
>  #include <asm/vmxfeatures.h>
>  #include <asm/vdso/processor.h>
>  #include <asm/shstk.h>
> +#include <asm/string.h>
>  
>  #include <linux/personality.h>
>  #include <linux/cache.h>
> @@ -580,6 +581,33 @@ static __always_inline bool on_thread_st
>  			       current_stack_pointer) < THREAD_SIZE;
>  }
>  
> +#ifdef CONFIG_X86_64
> +#define __stackleak_poison __stackleak_poison
> +static __always_inline void __stackleak_poison(unsigned long erase_low,
> +					       unsigned long erase_high,
> +					       unsigned long poison)
> +{
> +	unsigned long count = (erase_high - erase_low) / sizeof(unsigned long);
> +
> +	/* Amortize the startup cost of REP STOSQ. */
> +	if (count < 32) {
> +		unsigned long *ptr = (unsigned long *)erase_low;
> +
> +		while (count) {
> +			switch ((count-1) % 4) {
> +			case 3: ptr[--count] = poison; fallthrough;
> +			case 2: ptr[--count] = poison; fallthrough;
> +			case 1: ptr[--count] = poison; fallthrough;
> +			case 0: ptr[--count] = poison; break;
> +			}
> +		}

That looks horrid to me.
You want as few instructions (especially conditionals) inside the
loop as possible.
The Intel cpu will execute a two clock loop and the later ones will
do two writes per clock
(zen5 will execute a one clock loop and do two writes/clock).
So you want a loop with just 4 writes and the loop control code.
The 'pointer add' just needs to execute at the same time as some
of the stores - to avoid lengthening the register dependency chain.

For this code, can the length be a multiple of 32?
Also on some cpu the 'rep stos' might be twice as fast (per byte) if
the destination in 64 byte aligned - true for 'rep movsb'.

David


> +		return;
> +	}
> +
> +	memset64((void *)erase_low, poison, count);
> +}
> +#endif
> +
>  #ifdef CONFIG_PARAVIRT_XXL
>  #include <asm/paravirt.h>
>  #else
> 


      parent reply	other threads:[~2026-11-20 14:02 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 21:31 [PATCH 0/2] x86/kstack_erase: Faster stack poisoning, plus a startup build fix Usama Arif
2026-11-19 21:31 ` [PATCH 1/2] x86/boot/startup: Disable kernel stack depth tracking Usama Arif
2026-11-19 22:30   ` Nick Desaulniers
2026-11-19 21:31 ` [PATCH 2/2] x86/kstack_erase: Use REP STOSQ for large stack fills Usama Arif
2026-11-19 22:28   ` Nick Desaulniers
2026-11-20 11:38     ` David Laight
2026-11-20 12:42       ` David Laight
2026-11-20 10:45   ` Peter Zijlstra
2026-11-20 12:26     ` Peter Zijlstra
2026-11-20 14:02     ` David Laight [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120140200.53ec2557@pumpkin \
    --to=david.laight.linux@gmail.com \
    --cc=ardb@kernel.org \
    --cc=atish.patra@linux.dev \
    --cc=bp@alien8.de \
    --cc=dave.hansen@linux.intel.com \
    --cc=dgreid@meta.com \
    --cc=gustavoars@kernel.org \
    --cc=hca@linux.ibm.com \
    --cc=hpa@zytor.com \
    --cc=justinstitt@google.com \
    --cc=kees@kernel.org \
    --cc=kernel-team@meta.com \
    --cc=linux-hardening@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=llvm@lists.linux.dev \
    --cc=mingo@redhat.com \
    --cc=morbo@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=peterz@infradead.org \
    --cc=riel@surriel.com \
    --cc=rkeilty@meta.com \
    --cc=shakeel.butt@linux.dev \
    --cc=tglx@kernel.org \
    --cc=usama.arif@linux.dev \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox