Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Dave Hansen]

On 5/4/26 05:09, Mathias Krause wrote:
> +static int __init shstk_configure(char *str)
> +{
> +	if (!strcmp(str, "off"))
> +		setup_clear_cpu_cap(X86_FEATURE_SHSTK);
> +
> +	return 1;
> +}
> +__setup("shstk=", shstk_configure);

Is there a reason that clearcpuid=shstk doesn't work in this case? I
guess shstk and ibt are peers, but I was kinda hoping we'd stop adding
these for every single CPU feature at _some_ point.

Adding the documentation for ibt= is definitely a good idea, though.

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Edgecombe, Rick P]

On Wed, 2026-05-06 at 12:03 -0700, Dave Hansen wrote:
> Is there a reason that clearcpuid=shstk doesn't work in this case? I
> guess shstk and ibt are peers, but I was kinda hoping we'd stop adding
> these for every single CPU feature at _some_ point.

Oh yea, for the reason of "debugging related issues during early boot"
clearcpuid of shstk and ibt should be fine. It taints the kernel, but should be
fine for debugging? If I'm reading this right, the kernel does the clearcpuid
processing before setting up CET bits.

I'm remembering we actually already have a "nousershstk" too, which covers the
"userspace init cet violations break boot" usage.

What that doesn't do though, is clear CR4.CET. With nousershstk, KVM can still
use CET. So that is what is missing. A way to clear CR4.CET without tainting the
kernel when HW supports CET. Do we need it?

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Mathias Krause]

On 07.05.26 00:45, Edgecombe, Rick P wrote:
> On Wed, 2026-05-06 at 12:03 -0700, Dave Hansen wrote:
>> Is there a reason that clearcpuid=shstk doesn't work in this case? I
>> guess shstk and ibt are peers, but I was kinda hoping we'd stop adding
>> these for every single CPU feature at _some_ point.
> 
> Oh yea, for the reason of "debugging related issues during early boot"
> clearcpuid of shstk and ibt should be fine. It taints the kernel, but should be
> fine for debugging? If I'm reading this right, the kernel does the clearcpuid
> processing before setting up CET bits.

Unfortunately, neither 'clearcpuid=shstk' nor 'clearcpuid=user_shstk'
are of any help.

The former doesn't work because X86_FEATURE_SHSTK has no procfs-visible
string attached, therefore no entry in x86_cap_flags[] and therefore
can't be found via "shstk" in parse_set_clear_cpuid().

The latter only clears X86_FEATURE_USER_SHSTK which is a synthetic
feature bit but setup_cet() only looks for X86_FEATURE_SHSTK.

> 
> I'm remembering we actually already have a "nousershstk" too, which covers the
> "userspace init cet violations break boot" usage.

Oh, interesting. That'd be the equivalent of 'clearcpuid=user_shstk', right?

> 
> What that doesn't do though, is clear CR4.CET. With nousershstk, KVM can still
> use CET. So that is what is missing. A way to clear CR4.CET without tainting the
> kernel when HW supports CET. Do we need it?
> 

Right! Clearing, or, moreover, not setting CR4.CET=1 is what I need for
the debugging use cases I have in mind and had to hack around a few
times in the past.

Case in point, the last debugging session involved a bug with CPU
hotplug where the E-cores did not reset their IA32_S_CET MSR on #INIT
but the P-cores did (which wasn't the bug, as that's perfectly fine
SDM-documented behaviour (not resetting, that is)).

I had a system with CPU hotplug working on the P-cores but not on the
E-cores which took a while to debug because the E-cores died hard very
early on. CET was in effect way too early (not only in CR4 but also all
features via the (stale) MSR), calling a function that didn't had the
required ENDBR, causing an early, unhandled #CP.

So, a way to prevent CR4.CET enabling from happening would be nice to
have for future debugging sessions.

Thanks,
Mathias

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Edgecombe, Rick P]

On Thu, 2026-05-07 at 15:39 +0200, Mathias Krause wrote:
> On 07.05.26 00:45, Edgecombe, Rick P wrote:
> > On Wed, 2026-05-06 at 12:03 -0700, Dave Hansen wrote:
> > > Is there a reason that clearcpuid=shstk doesn't work in this case? I
> > > guess shstk and ibt are peers, but I was kinda hoping we'd stop adding
> > > these for every single CPU feature at _some_ point.
> > 
> > Oh yea, for the reason of "debugging related issues during early boot"
> > clearcpuid of shstk and ibt should be fine. It taints the kernel, but should be
> > fine for debugging? If I'm reading this right, the kernel does the clearcpuid
> > processing before setting up CET bits.
> 
> Unfortunately, neither 'clearcpuid=shstk' nor 'clearcpuid=user_shstk'
> are of any help.
> 
> The former doesn't work because X86_FEATURE_SHSTK has no procfs-visible
> string attached, therefore no entry in x86_cap_flags[] and therefore
> can't be found via "shstk" in parse_set_clear_cpuid().
> 
> The latter only clears X86_FEATURE_USER_SHSTK which is a synthetic
> feature bit but setup_cet() only looks for X86_FEATURE_SHSTK.

So alternatively we could just do:

diff --git a/tools/arch/x86/include/asm/cpufeatures.h
b/tools/arch/x86/include/asm/cpufeatures.h
index c3b53beb13007..270341e786f28 100644
--- a/tools/arch/x86/include/asm/cpufeatures.h
+++ b/tools/arch/x86/include/asm/cpufeatures.h
@@ -392,7 +392,7 @@
 #define X86_FEATURE_OSPKE              (16*32+ 4) /* "ospke" OS Protection Keys
Enable */
 #define X86_FEATURE_WAITPKG            (16*32+ 5) /* "waitpkg"
UMONITOR/UMWAIT/TPAUSE Instructions */
 #define X86_FEATURE_AVX512_VBMI2       (16*32+ 6) /* "avx512_vbmi2" Additional
AVX512 Vector Bit Manipulation s/
-#define X86_FEATURE_SHSTK              (16*32+ 7) /* Shadow stack */
+#define X86_FEATURE_SHSTK              (16*32+ 7) /* "shstk" Shadow stack */
 #define X86_FEATURE_GFNI               (16*32+ 8) /* "gfni" Galois Field New
Instructions */
 #define X86_FEATURE_VAES               (16*32+ 9) /* "vaes" Vector AES */
 #define X86_FEATURE_VPCLMULQDQ         (16*32+10) /* "vpclmulqdq" Carry-Less
Multiplication Double Quadword */


Now that KVM uses this this feature independently of X86_FEATURE_USER_SHSTK, it
might be good to have the plain HW shstk feature exposed for just normal runtime
user use. (+Chao, for KVM CET)

> 
> > 
> > I'm remembering we actually already have a "nousershstk" too, which covers the
> > "userspace init cet violations break boot" usage.
> 
> Oh, interesting. That'd be the equivalent of 'clearcpuid=user_shstk', right?

Right, except for no taint.

> 
> > 
> > What that doesn't do though, is clear CR4.CET. With nousershstk, KVM can still
> > use CET. So that is what is missing. A way to clear CR4.CET without tainting the
> > kernel when HW supports CET. Do we need it?
> > 
> 
> Right! Clearing, or, moreover, not setting CR4.CET=1 is what I need for
> the debugging use cases I have in mind and had to hack around a few
> times in the past.
> 
> Case in point, the last debugging session involved a bug with CPU
> hotplug where the E-cores did not reset their IA32_S_CET MSR on #INIT
> but the P-cores did (which wasn't the bug, as that's perfectly fine
> SDM-documented behaviour (not resetting, that is)).

But the above works for this case, right? The taint doesn't matter for
debugging?

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Mathias Krause]

On 07.05.26 21:53, Edgecombe, Rick P wrote:
> On Thu, 2026-05-07 at 15:39 +0200, Mathias Krause wrote:
>> On 07.05.26 00:45, Edgecombe, Rick P wrote:
>>> On Wed, 2026-05-06 at 12:03 -0700, Dave Hansen wrote:
>>>> Is there a reason that clearcpuid=shstk doesn't work in this case? I
>>>> guess shstk and ibt are peers, but I was kinda hoping we'd stop adding
>>>> these for every single CPU feature at _some_ point.
>>>
>>> Oh yea, for the reason of "debugging related issues during early boot"
>>> clearcpuid of shstk and ibt should be fine. It taints the kernel, but should be
>>> fine for debugging? If I'm reading this right, the kernel does the clearcpuid
>>> processing before setting up CET bits.
>>
>> Unfortunately, neither 'clearcpuid=shstk' nor 'clearcpuid=user_shstk'
>> are of any help.
>>
>> The former doesn't work because X86_FEATURE_SHSTK has no procfs-visible
>> string attached, therefore no entry in x86_cap_flags[] and therefore
>> can't be found via "shstk" in parse_set_clear_cpuid().
>>
>> The latter only clears X86_FEATURE_USER_SHSTK which is a synthetic
>> feature bit but setup_cet() only looks for X86_FEATURE_SHSTK.
> 
> So alternatively we could just do:
> 
> diff --git a/tools/arch/x86/include/asm/cpufeatures.h
> b/tools/arch/x86/include/asm/cpufeatures.h
> index c3b53beb13007..270341e786f28 100644
> --- a/tools/arch/x86/include/asm/cpufeatures.h
> +++ b/tools/arch/x86/include/asm/cpufeatures.h
> @@ -392,7 +392,7 @@
>  #define X86_FEATURE_OSPKE              (16*32+ 4) /* "ospke" OS Protection Keys
> Enable */
>  #define X86_FEATURE_WAITPKG            (16*32+ 5) /* "waitpkg"
> UMONITOR/UMWAIT/TPAUSE Instructions */
>  #define X86_FEATURE_AVX512_VBMI2       (16*32+ 6) /* "avx512_vbmi2" Additional
> AVX512 Vector Bit Manipulation s/
> -#define X86_FEATURE_SHSTK              (16*32+ 7) /* Shadow stack */
> +#define X86_FEATURE_SHSTK              (16*32+ 7) /* "shstk" Shadow stack */
>  #define X86_FEATURE_GFNI               (16*32+ 8) /* "gfni" Galois Field New
> Instructions */
>  #define X86_FEATURE_VAES               (16*32+ 9) /* "vaes" Vector AES */
>  #define X86_FEATURE_VPCLMULQDQ         (16*32+10) /* "vpclmulqdq" Carry-Less
> Multiplication Double Quadword */

I explicitly didn't propose that as I was under the assumption, hiding
that feature bit is intentional. But, as it was you who added that bit
like that in 701fb66d576e ("x86/cpufeatures: Add CPU feature flags for
shadow stacks") and is now proposing otherwise, I won't object either.

> 
> Now that KVM uses this this feature independently of X86_FEATURE_USER_SHSTK, it
> might be good to have the plain HW shstk feature exposed for just normal runtime
> user use. (+Chao, for KVM CET)

But that sounds more like having the need for an official chicken bit,
like I was proposing, no? Using 'clearcpuid=shstk' as a workaround for
whatever KVM bugs, similar in spirit to 'nousershstk', but without the
kernel taint?

>>>
>>> I'm remembering we actually already have a "nousershstk" too, which covers the
>>> "userspace init cet violations break boot" usage.
>>
>> Oh, interesting. That'd be the equivalent of 'clearcpuid=user_shstk', right?
> 
> Right, except for no taint.
> 
>>
>>>
>>> What that doesn't do though, is clear CR4.CET. With nousershstk, KVM can still
>>> use CET. So that is what is missing. A way to clear CR4.CET without tainting the
>>> kernel when HW supports CET. Do we need it?
>>>
>>
>> Right! Clearing, or, moreover, not setting CR4.CET=1 is what I need for
>> the debugging use cases I have in mind and had to hack around a few
>> times in the past.
>>
>> Case in point, the last debugging session involved a bug with CPU
>> hotplug where the E-cores did not reset their IA32_S_CET MSR on #INIT
>> but the P-cores did (which wasn't the bug, as that's perfectly fine
>> SDM-documented behaviour (not resetting, that is)).
> 
> But the above works for this case, right? The taint doesn't matter for
> debugging?

For *me*, 'clearcpuid=shstk,ibt' would be sufficient for my debugging
needs. It's just a question if there's more demand beside some random
kernel hacker needing a knob to disable potential problematic features,
i.e. do we expect actual *end users* having a need to fully disable CET
shadow stacks too?


Thanks,
Mathias

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Dave Hansen]

On 5/8/26 00:23, Mathias Krause wrote:
> For *me*, 'clearcpuid=shstk,ibt' would be sufficient for my debugging
> needs. It's just a question if there's more demand beside some random
> kernel hacker needing a knob to disable potential problematic features,
> i.e. do we expect actual *end users* having a need to fully disable CET
> shadow stacks too?

Last I checked, end users aren't frequently in the business of editing
the kernel command line to turn off CPU features. Maybe once in a blue
moon to work around bugs. But those folks can live with taint until they
get fixes.

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Edgecombe, Rick P]

On Fri, 2026-05-08 at 09:23 +0200, Mathias Krause wrote:
> 
> I explicitly didn't propose that as I was under the assumption, hiding
> that feature bit is intentional. But, as it was you who added that bit
> like that in 701fb66d576e ("x86/cpufeatures: Add CPU feature flags for
> shadow stacks") and is now proposing otherwise, I won't object either.

It sort of was intentional back then. CET has a weird user/supervisor split,
where it is split user/supervisor in some sense but locked together in other
senses. There was a suggestion to make up some user and kernel feature bits so
Linux could differentiate which it actually supports or is using. Which is where
user_shstk came from. However, ibt already has just "ibt" to control the kernel
support. And the split idea only works part ways, because user and supervisor
are linked together in some ways. Really that one should have been be kernel_ibt
or something to be clearer, but it predated the user_shstk discussion.

But from where we are today, I guess adding "shstk" seems ok. I guess your patch
has the benefit of not stirring the messy waters. Not stirring the waters is
maybe a valid answer to Dave's question. I'm ok either way, maybe lean towards
adding "shstk".

> 
> > 
> > Now that KVM uses this this feature independently of X86_FEATURE_USER_SHSTK,
> > it might be good to have the plain HW shstk feature exposed for just normal
> > runtime user use. (+Chao, for KVM CET)
> 
> But that sounds more like having the need for an official chicken bit,
> like I was proposing, no? Using 'clearcpuid=shstk' as a workaround for
> whatever KVM bugs, similar in spirit to 'nousershstk', but without the
> kernel taint?

For users to turn off shadow stack for guests? You can do this via the KVM API
in the normal way you customize guests.

> 
> > > 
> > 
> > But the above works for this case, right? The taint doesn't matter for
> > debugging?
> 
> For *me*, 'clearcpuid=shstk,ibt' would be sufficient for my debugging
> needs. It's just a question if there's more demand beside some random
> kernel hacker needing a knob to disable potential problematic features,
> i.e. do we expect actual *end users* having a need to fully disable CET
> shadow stacks too?

Today I think no? We have nousershstk for shadow stack users, and ibt=off for
ibt. So everything is covered from the user's perspective.

Re: [PATCH v2] x86/shstk: Provide kernel command line knob to disable

[Mathias Krause]

On 08.05.26 18:34, Dave Hansen wrote:
> On 5/8/26 00:23, Mathias Krause wrote:
>> For *me*, 'clearcpuid=shstk,ibt' would be sufficient for my debugging
>> needs. It's just a question if there's more demand beside some random
>> kernel hacker needing a knob to disable potential problematic features,
>> i.e. do we expect actual *end users* having a need to fully disable CET
>> shadow stacks too?
> 
> Last I checked, end users aren't frequently in the business of editing
> the kernel command line to turn off CPU features. Maybe once in a blue
> moon to work around bugs. But those folks can live with taint until they
> get fixes.

Ok, I'll drop this one and go for the 'clearcpuid=shstk' approach.

Thanks,
Mathias