[PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing

[PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing

[Akihiro Suda]

Even when X86_FEATURE_PKU and X86_FEATURE_OSPKE are available,
XFEATURE_PKRU can be missing.
In such a case, pkeys has to be disabled to avoid hanging up.

  WARNING: CPU: 0 PID: 1 at arch/x86/kernel/fpu/xstate.c:1003 get_xsave_addr_user+0x28/0x40
  (...)
  Call Trace:
   <TASK>
   ? get_xsave_addr_user+0x28/0x40
   ? __warn.cold+0x8e/0xea
   ? get_xsave_addr_user+0x28/0x40
   ? report_bug+0xff/0x140
   ? handle_bug+0x3b/0x70
   ? exc_invalid_op+0x17/0x70
   ? asm_exc_invalid_op+0x1a/0x20
   ? get_xsave_addr_user+0x28/0x40
   copy_fpstate_to_sigframe+0x1be/0x380
   ? __put_user_8+0x11/0x20
   get_sigframe+0xf1/0x280
   x64_setup_rt_frame+0x67/0x2c0
   arch_do_signal_or_restart+0x1b3/0x240
   syscall_exit_to_user_mode+0xb0/0x130
   do_syscall_64+0xab/0x1a0
   entry_SYSCALL_64_after_hwframe+0x77/0x7f

This fix is known to be needed on Apple Virtualization.
Tested with macOS 13.5.2 running on MacBook Pro 2020 with
Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz.

Fixes: 70044df250d0 ("x86/pkeys: Update PKRU to enable all pkeys before XSAVE")
Link: https://lore.kernel.org/regressions/CAG8fp8QvH71Wi_y7b7tgFp7knK38rfrF7rRHh-gFKqeS0gxY6Q@mail.gmail.com/T/#u
Link: https://github.com/lima-vm/lima/issues/3334

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
---
 arch/x86/kernel/cpu/common.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index e9464fe411ac..4c2c268af214 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -517,7 +517,8 @@ static bool pku_disabled;
 static __always_inline void setup_pku(struct cpuinfo_x86 *c)
 {
 	if (c == &boot_cpu_data) {
-		if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU))
+		if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU) ||
+		    !cpu_has_xfeatures(XFEATURE_PKRU, NULL))
 			return;
 		/*
 		 * Setting CR4.PKE will cause the X86_FEATURE_OSPKE cpuid
-- 
2.45.2

Re: [PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing

[kernel test robot]

Hi,

Thanks for your patch.

FYI: kernel test robot notices the stable kernel rule is not satisfied.

The check is based on https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html#option-1

Rule: add the tag "Cc: stable@vger.kernel.org" in the sign-off area to have the patch automatically included in the stable tree.
Subject: [PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing
Link: https://lore.kernel.org/stable/20250314084818.2826-1-akihiro.suda.cz%40hco.ntt.co.jp

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing

[Ingo Molnar]

* Akihiro Suda <suda.gitsendemail@gmail.com> wrote:

> Even when X86_FEATURE_PKU and X86_FEATURE_OSPKE are available,
> XFEATURE_PKRU can be missing.
> In such a case, pkeys has to be disabled to avoid hanging up.
> 
>   WARNING: CPU: 0 PID: 1 at arch/x86/kernel/fpu/xstate.c:1003 get_xsave_addr_user+0x28/0x40
>   (...)
>   Call Trace:
>    <TASK>
>    ? get_xsave_addr_user+0x28/0x40
>    ? __warn.cold+0x8e/0xea
>    ? get_xsave_addr_user+0x28/0x40
>    ? report_bug+0xff/0x140
>    ? handle_bug+0x3b/0x70
>    ? exc_invalid_op+0x17/0x70
>    ? asm_exc_invalid_op+0x1a/0x20
>    ? get_xsave_addr_user+0x28/0x40
>    copy_fpstate_to_sigframe+0x1be/0x380
>    ? __put_user_8+0x11/0x20
>    get_sigframe+0xf1/0x280
>    x64_setup_rt_frame+0x67/0x2c0
>    arch_do_signal_or_restart+0x1b3/0x240
>    syscall_exit_to_user_mode+0xb0/0x130
>    do_syscall_64+0xab/0x1a0
>    entry_SYSCALL_64_after_hwframe+0x77/0x7f
> 
> This fix is known to be needed on Apple Virtualization.
> Tested with macOS 13.5.2 running on MacBook Pro 2020 with
> Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz.
> 
> Fixes: 70044df250d0 ("x86/pkeys: Update PKRU to enable all pkeys before XSAVE")
> Link: https://lore.kernel.org/regressions/CAG8fp8QvH71Wi_y7b7tgFp7knK38rfrF7rRHh-gFKqeS0gxY6Q@mail.gmail.com/T/#u
> Link: https://github.com/lima-vm/lima/issues/3334
> 
> Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
> ---
>  arch/x86/kernel/cpu/common.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
> index e9464fe411ac..4c2c268af214 100644
> --- a/arch/x86/kernel/cpu/common.c
> +++ b/arch/x86/kernel/cpu/common.c
> @@ -517,7 +517,8 @@ static bool pku_disabled;
>  static __always_inline void setup_pku(struct cpuinfo_x86 *c)
>  {
>  	if (c == &boot_cpu_data) {
> -		if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU))
> +		if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU) ||
> +		    !cpu_has_xfeatures(XFEATURE_PKRU, NULL))
>  			return;

Note that silent quirks are counterproductive, as they don't give VM 
vendors any incentives to fix their VM for such bugs.

So I changed your quirk to be:

--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -519,6 +519,17 @@ static __always_inline void setup_pku(struct cpuinfo_x86 *c)
 	if (c == &boot_cpu_data) {
 		if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU))
 			return;
+		if (!cpu_has_xfeatures(XFEATURE_PKRU, NULL)) {
+			/*
+			 * Missing XFEATURE_PKRU is not really a valid CPU
+			 * configuration at this point, but apparently
+			 * Apple Virtualization is affected by this,
+			 * so return with a FW warning instead of crashing
+			 * the bootup:
+			 */
+			WARN_ONCE(1, FW_BUG "Invalid XFEATURE_PKRU configuration.\n");
+			return;
+		}
 		/*
 		 * Setting CR4.PKE will cause the X86_FEATURE_OSPKE cpuid
 		 * bit to be set.  Enforce it.

This is noisy in the syslog, but it's a WARN_ONCE() and it doesn't 
crash the bootup.

Thanks,

	Ingo

Re: [PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing

[Akihiro Suda]

Thanks Ingo, but we may have to reconsider whether cpu_has_xfeatures
works in this place
https://lore.kernel.org/all/1b8745e0-ae80-4add-b015-affdaa69b369@intel.com/

The current code might be accidentally disabling PKU on other
PKU-compatible environments?

2025年3月20日(木) 6:39 Ingo Molnar <mingo@kernel.org>:
>
>
> * Akihiro Suda <suda.gitsendemail@gmail.com> wrote:
>
> > Even when X86_FEATURE_PKU and X86_FEATURE_OSPKE are available,
> > XFEATURE_PKRU can be missing.
> > In such a case, pkeys has to be disabled to avoid hanging up.
> >
> >   WARNING: CPU: 0 PID: 1 at arch/x86/kernel/fpu/xstate.c:1003 get_xsave_addr_user+0x28/0x40
> >   (...)
> >   Call Trace:
> >    <TASK>
> >    ? get_xsave_addr_user+0x28/0x40
> >    ? __warn.cold+0x8e/0xea
> >    ? get_xsave_addr_user+0x28/0x40
> >    ? report_bug+0xff/0x140
> >    ? handle_bug+0x3b/0x70
> >    ? exc_invalid_op+0x17/0x70
> >    ? asm_exc_invalid_op+0x1a/0x20
> >    ? get_xsave_addr_user+0x28/0x40
> >    copy_fpstate_to_sigframe+0x1be/0x380
> >    ? __put_user_8+0x11/0x20
> >    get_sigframe+0xf1/0x280
> >    x64_setup_rt_frame+0x67/0x2c0
> >    arch_do_signal_or_restart+0x1b3/0x240
> >    syscall_exit_to_user_mode+0xb0/0x130
> >    do_syscall_64+0xab/0x1a0
> >    entry_SYSCALL_64_after_hwframe+0x77/0x7f
> >
> > This fix is known to be needed on Apple Virtualization.
> > Tested with macOS 13.5.2 running on MacBook Pro 2020 with
> > Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz.
> >
> > Fixes: 70044df250d0 ("x86/pkeys: Update PKRU to enable all pkeys before XSAVE")
> > Link: https://lore.kernel.org/regressions/CAG8fp8QvH71Wi_y7b7tgFp7knK38rfrF7rRHh-gFKqeS0gxY6Q@mail.gmail.com/T/#u
> > Link: https://github.com/lima-vm/lima/issues/3334
> >
> > Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
> > ---
> >  arch/x86/kernel/cpu/common.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
> > index e9464fe411ac..4c2c268af214 100644
> > --- a/arch/x86/kernel/cpu/common.c
> > +++ b/arch/x86/kernel/cpu/common.c
> > @@ -517,7 +517,8 @@ static bool pku_disabled;
> >  static __always_inline void setup_pku(struct cpuinfo_x86 *c)
> >  {
> >       if (c == &boot_cpu_data) {
> > -             if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU))
> > +             if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU) ||
> > +                 !cpu_has_xfeatures(XFEATURE_PKRU, NULL))
> >                       return;
>
> Note that silent quirks are counterproductive, as they don't give VM
> vendors any incentives to fix their VM for such bugs.
>
> So I changed your quirk to be:
>
> --- a/arch/x86/kernel/cpu/common.c
> +++ b/arch/x86/kernel/cpu/common.c
> @@ -519,6 +519,17 @@ static __always_inline void setup_pku(struct cpuinfo_x86 *c)
>         if (c == &boot_cpu_data) {
>                 if (pku_disabled || !cpu_feature_enabled(X86_FEATURE_PKU))
>                         return;
> +               if (!cpu_has_xfeatures(XFEATURE_PKRU, NULL)) {
> +                       /*
> +                        * Missing XFEATURE_PKRU is not really a valid CPU
> +                        * configuration at this point, but apparently
> +                        * Apple Virtualization is affected by this,
> +                        * so return with a FW warning instead of crashing
> +                        * the bootup:
> +                        */
> +                       WARN_ONCE(1, FW_BUG "Invalid XFEATURE_PKRU configuration.\n");
> +                       return;
> +               }
>                 /*
>                  * Setting CR4.PKE will cause the X86_FEATURE_OSPKE cpuid
>                  * bit to be set.  Enforce it.
>
> This is noisy in the syslog, but it's a WARN_ONCE() and it doesn't
> crash the bootup.
>
> Thanks,
>
>         Ingo

Re: [PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing

[Borislav Petkov]

On Wed, Mar 19, 2025 at 10:39:33PM +0100, Ingo Molnar wrote:
> Note that silent quirks are counterproductive, as they don't give VM 
> vendors any incentives to fix their VM for such bugs.
> 
> So I changed your quirk to be:

This fires on my Zen3 now :-P

[    2.411315] x86/cpu: User Mode Instruction Prevention (UMIP) activated
[    2.415307] ------------[ cut here ]------------
[    2.419306] [Firmware Bug]: Invalid XFEATURE_PKRU configuration.
[    2.423307] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/cpu/common.c:530 identify_cpu+0x82a/0x840
[    2.427306] Modules linked in:
[    2.431307] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.14.0-rc7+ #1 PREEMPT(full) 
[    2.435306] Hardware name: Micro-Star International Co., Ltd. MS-7A38/B450M PRO-VDH MAX (MS-7A38), BIOS B.G0 07/26/2022
[    2.439306] RIP: 0010:identify_cpu+0x82a/0x840
[    2.443306] Code: e8 bb f2 ff ff e9 4f ff ff ff 80 3d 07 4e 7b 01 00 0f 85 af fb ff ff 48 c7 c7 a8 fd f0 81 c6 05 f3 4d 7b 01 01 e8 e6 49 04 00 <0f> 0b e9 95 fb ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00
[    2.447306] RSP: 0000:ffffffff82203ec8 EFLAGS: 00010296
[    2.451306] RAX: 0000000000000034 RBX: 0000000000000000 RCX: 0000000000000000
[    2.455306] RDX: 0000000080000003 RSI: 00000000ffffffea RDI: 0000000000000001
[    2.459306] RBP: ffffffff82a09f40 R08: ffff88883e1fafe8 R09: 000000000027fffb
[    2.463306] R10: 00000000000000ee R11: ffff88883d5fb000 R12: 0000000000000000
[    2.467306] R13: ffff88883f373180 R14: ffffffff8220ba78 R15: 000000000008b000
[    2.471306] FS:  0000000000000000(0000) GS:ffff88889742b000(0000) knlGS:0000000000000000
[    2.475306] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    2.479306] CR2: ffff88883f1ff000 CR3: 000000000221a000 CR4: 00000000003108b0
[    2.483306] Call Trace:
[    2.487307]  <TASK>
[    2.489459]  ? __warn+0x85/0x150
[    2.491306]  ? identify_cpu+0x82a/0x840
[    2.495306]  ? report_bug+0x1c3/0x1d0
[    2.499306]  ? identify_cpu+0x82a/0x840
[    2.503306]  ? identify_cpu+0x82c/0x840
[    2.507306]  ? handle_bug+0xec/0x120
[    2.511306]  ? exc_invalid_op+0x14/0x70
[    2.515306]  ? asm_exc_invalid_op+0x16/0x20
[    2.519306]  ? identify_cpu+0x82a/0x840
[    2.523306]  ? identify_cpu+0x82a/0x840
[    2.527306]  arch_cpu_finalize_init+0x23/0x150
[    2.531307]  start_kernel+0x40a/0x720
[    2.535306]  x86_64_start_reservations+0x14/0x30
[    2.539306]  x86_64_start_kernel+0xa8/0xc0
[    2.543306]  common_startup_64+0x12c/0x138
[    2.547307]  </TASK>
[    2.551306] ---[ end trace 0000000000000000 ]---

Zapping it for the time being.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/pkeys: Disable PKU when XFEATURE_PKRU is missing

[Ingo Molnar]

* Borislav Petkov <bp@alien8.de> wrote:

> On Wed, Mar 19, 2025 at 10:39:33PM +0100, Ingo Molnar wrote:
> > Note that silent quirks are counterproductive, as they don't give VM 
> > vendors any incentives to fix their VM for such bugs.
> > 
> > So I changed your quirk to be:
> 
> This fires on my Zen3 now :-P
> 
> [    2.411315] x86/cpu: User Mode Instruction Prevention (UMIP) activated
> [    2.415307] ------------[ cut here ]------------
> [    2.419306] [Firmware Bug]: Invalid XFEATURE_PKRU configuration.
> [    2.423307] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/cpu/common.c:530 identify_cpu+0x82a/0x840
> [    2.427306] Modules linked in:
> [    2.431307] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.14.0-rc7+ #1 PREEMPT(full) 
> [    2.435306] Hardware name: Micro-Star International Co., Ltd. MS-7A38/B450M PRO-VDH MAX (MS-7A38), BIOS B.G0 07/26/2022
> [    2.439306] RIP: 0010:identify_cpu+0x82a/0x840
> [    2.443306] Code: e8 bb f2 ff ff e9 4f ff ff ff 80 3d 07 4e 7b 01 00 0f 85 af fb ff ff 48 c7 c7 a8 fd f0 81 c6 05 f3 4d 7b 01 01 e8 e6 49 04 00 <0f> 0b e9 95 fb ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00
> [    2.447306] RSP: 0000:ffffffff82203ec8 EFLAGS: 00010296
> [    2.451306] RAX: 0000000000000034 RBX: 0000000000000000 RCX: 0000000000000000
> [    2.455306] RDX: 0000000080000003 RSI: 00000000ffffffea RDI: 0000000000000001
> [    2.459306] RBP: ffffffff82a09f40 R08: ffff88883e1fafe8 R09: 000000000027fffb
> [    2.463306] R10: 00000000000000ee R11: ffff88883d5fb000 R12: 0000000000000000
> [    2.467306] R13: ffff88883f373180 R14: ffffffff8220ba78 R15: 000000000008b000
> [    2.471306] FS:  0000000000000000(0000) GS:ffff88889742b000(0000) knlGS:0000000000000000
> [    2.475306] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    2.479306] CR2: ffff88883f1ff000 CR3: 000000000221a000 CR4: 00000000003108b0
> [    2.483306] Call Trace:
> [    2.487307]  <TASK>
> [    2.489459]  ? __warn+0x85/0x150
> [    2.491306]  ? identify_cpu+0x82a/0x840
> [    2.495306]  ? report_bug+0x1c3/0x1d0
> [    2.499306]  ? identify_cpu+0x82a/0x840
> [    2.503306]  ? identify_cpu+0x82c/0x840
> [    2.507306]  ? handle_bug+0xec/0x120
> [    2.511306]  ? exc_invalid_op+0x14/0x70
> [    2.515306]  ? asm_exc_invalid_op+0x16/0x20
> [    2.519306]  ? identify_cpu+0x82a/0x840
> [    2.523306]  ? identify_cpu+0x82a/0x840
> [    2.527306]  arch_cpu_finalize_init+0x23/0x150
> [    2.531307]  start_kernel+0x40a/0x720
> [    2.535306]  x86_64_start_reservations+0x14/0x30
> [    2.539306]  x86_64_start_kernel+0xa8/0xc0
> [    2.543306]  common_startup_64+0x12c/0x138
> [    2.547307]  </TASK>
> [    2.551306] ---[ end trace 0000000000000000 ]---
> 
> Zapping it for the time being.

Thanks!

	Ingo