* [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging
@ 2026-06-10 17:56 Ihor Solodrai
2026-06-10 18:39 ` Andrey Konovalov
2026-06-12 16:30 ` Kiryl Shutsemau
0 siblings, 2 replies; 5+ messages in thread
From: Ihor Solodrai @ 2026-06-10 17:56 UTC (permalink / raw)
To: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Andrey Ryabinin,
Andrew Morton
Cc: bpf, kasan-dev, linux-mm, linux-kernel
On x86_64 with 5-level paging (LA57) and inline generic KASAN, the
following flaky splat may be observed on boot:
BUG: KASAN: wild-memory-access in do_raw_spin_lock+0xcf/0x260
Write of size 4 at addr ff110001000c90b8 by task swapper/0/0
CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.1.0-rc5-gcba33e0b2907 #1 PREEMPT(full)
Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Call Trace:
<IRQ>
dump_stack_lvl+0x54/0x70
kasan_report+0x117/0x150
? do_raw_spin_lock+0xcf/0x260
kasan_check_range+0x264/0x2c0
do_raw_spin_lock+0xcf/0x260
handle_edge_irq+0x35/0x770
? do_raw_spin_unlock+0x51/0x2a0
__common_interrupt+0xae/0x120
common_interrupt+0x7c/0x90
</IRQ>
<TASK>
asm_common_interrupt+0x26/0x40
RIP: 0010:identify_cpu+0x2b2/0x3460
Code: 00 41 c7 07 00 00 00 00 4d 89 e6 49 c1 ee 03 43 0f b6 04 06 84 c0 0f 85 a3 1c 00 00 41 c7 04 24 00 00 00 00 31 c0 31 c9 0f a2 <89> c7 42 0f b6 44 05 00 84 c0 0f 85 ad 1c 00 00 41 89 3f 48 8b 44
RSP: 0000:ffffffff97807df0 EFLAGS: 00000246
RAX: 0000000000000020 RBX: 00000000756e6547 RCX: 000000006c65746e
RDX: 0000000049656e69 RSI: 0000000000000000 RDI: ffffffff98632fd8
RBP: 1ffffffff30c65fc R08: dffffc0000000000 R09: 0000000000000004
R10: ffffffff98632fc4 R11: fffffbfff30c65fb R12: ffffffff98633050
R13: ffffffff98633048 R14: 1ffffffff30c660a R15: ffffffff98632fe0
identify_boot_cpu+0xd/0xd0
arch_cpu_finalize_init+0x24/0x1f0
start_kernel+0x31e/0x3e0
x86_64_start_reservations+0x24/0x30
x86_64_start_kernel+0x13a/0x140
common_startup_64+0x12c/0x137
</TASK>
It fires very early in boot. If kasan_multi_shot is set, the reports
are non-fatal and keep repeating, and the boot CPU wedges before
userspace is reached. The accessed addresses are valid 5-level kernel
pointers, so the report is a false positive.
The root cause is in generic KASAN not seeing
cpu_feature_enabled(X86_FEATURE_LA57) set, because the bit is cleared
in identify_cpu() when the offending interrupt happens [1]:
memset(&c->x86_capability, 0, ...); /* clears X86_FEATURE_LA57 */
...
get_cpu_cap(c); /* re-reads CPUID, restores it */
addr_has_metadata() then uses the 4-level threshold, and 5-level
kernel addresses fall below it, so kasan_check_range() reports them as
wild-memory-access.
Define USE_EARLY_PGTABLE_L5 in mm/kasan/generic.c so
addr_has_metadata() uses the stable variable, as
arch/x86/mm/kasan_init_64.c already does.
Some context on how this was noticed and reproduced below.
We started seeing flaky splats as above [2][3] on BPF CI runs after
runner hardware has been upgraded. Specifically, new x86 runners are
c7i.metal-24xl AWS EC2 instances, which are Intel Sapphire Rapids
machines that support LA57 feature, and have it enabled.
The splats can be reproduced with qemu on any x86_64 host with
-cpu max -accel tcg
Build a kernel with:
CONFIG_KASAN=y
CONFIG_KASAN_GENERIC=y
CONFIG_KASAN_INLINE=y
Boot it with kasan_multi_shot. The fault fires fast before userspace,
so no rootfs is required. For example:
qemu-system-x86_64 -display none -serial stdio -no-reboot \
-smp 4 -m 5G -cpu max -accel tcg \
-kernel arch/x86/boot/bzImage \
-append "console=ttyS0,115200 earlyprintk=serial,0,115200 panic=-1 kasan_multi_shot nokaslr"
It's a timing race, so a single boot hits it only sometimes.
However running several qemu instances in parallel on the same host
significantly increases the hitrate.
I confirmed the proposed fix eliminates the splats.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/cpu/common.c?h=v7.1-rc7#n2001
[2] https://github.com/kernel-patches/bpf/actions/runs/27271262414/job/80542509369
[3] https://github.com/kernel-patches/bpf/actions/runs/27260143782/job/80505353689
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
---
mm/kasan/generic.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index 2b8e73f5f6a7..b5f430f2dbb6 100644
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -9,6 +9,13 @@
* Andrey Konovalov <andreyknvl@gmail.com>
*/
+/*
+ * check_region_inline() and addr_has_metadata() can run very early.
+ * For example, in an interrupt taken while identify_cpu() has the CPU
+ * capability bits temporarily cleared.
+ */
+#define USE_EARLY_PGTABLE_L5
+
#include <linux/export.h>
#include <linux/interrupt.h>
#include <linux/init.h>
--
2.54.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging 2026-06-10 17:56 [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging Ihor Solodrai @ 2026-06-10 18:39 ` Andrey Konovalov 2026-06-10 21:55 ` Ihor Solodrai 2026-06-12 16:30 ` Kiryl Shutsemau 1 sibling, 1 reply; 5+ messages in thread From: Andrey Konovalov @ 2026-06-10 18:39 UTC (permalink / raw) To: Ihor Solodrai Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann, Eduard Zingerman, Kumar Kartikeya Dwivedi, Andrey Ryabinin, Andrew Morton, bpf, kasan-dev, linux-mm, linux-kernel On Wed, Jun 10, 2026 at 7:57 PM Ihor Solodrai <ihor.solodrai@linux.dev> wrote: > > On x86_64 with 5-level paging (LA57) and inline generic KASAN, the > following flaky splat may be observed on boot: > > BUG: KASAN: wild-memory-access in do_raw_spin_lock+0xcf/0x260 > Write of size 4 at addr ff110001000c90b8 by task swapper/0/0 > > CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.1.0-rc5-gcba33e0b2907 #1 PREEMPT(full) > Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014 > Call Trace: > <IRQ> > dump_stack_lvl+0x54/0x70 > kasan_report+0x117/0x150 > ? do_raw_spin_lock+0xcf/0x260 > kasan_check_range+0x264/0x2c0 > do_raw_spin_lock+0xcf/0x260 > handle_edge_irq+0x35/0x770 > ? do_raw_spin_unlock+0x51/0x2a0 > __common_interrupt+0xae/0x120 > common_interrupt+0x7c/0x90 > </IRQ> > <TASK> > asm_common_interrupt+0x26/0x40 > RIP: 0010:identify_cpu+0x2b2/0x3460 > Code: 00 41 c7 07 00 00 00 00 4d 89 e6 49 c1 ee 03 43 0f b6 04 06 84 c0 0f 85 a3 1c 00 00 41 c7 04 24 00 00 00 00 31 c0 31 c9 0f a2 <89> c7 42 0f b6 44 05 00 84 c0 0f 85 ad 1c 00 00 41 89 3f 48 8b 44 > RSP: 0000:ffffffff97807df0 EFLAGS: 00000246 > RAX: 0000000000000020 RBX: 00000000756e6547 RCX: 000000006c65746e > RDX: 0000000049656e69 RSI: 0000000000000000 RDI: ffffffff98632fd8 > RBP: 1ffffffff30c65fc R08: dffffc0000000000 R09: 0000000000000004 > R10: ffffffff98632fc4 R11: fffffbfff30c65fb R12: ffffffff98633050 > R13: ffffffff98633048 R14: 1ffffffff30c660a R15: ffffffff98632fe0 > identify_boot_cpu+0xd/0xd0 > arch_cpu_finalize_init+0x24/0x1f0 > start_kernel+0x31e/0x3e0 > x86_64_start_reservations+0x24/0x30 > x86_64_start_kernel+0x13a/0x140 > common_startup_64+0x12c/0x137 > </TASK> > > It fires very early in boot. If kasan_multi_shot is set, the reports > are non-fatal and keep repeating, and the boot CPU wedges before > userspace is reached. The accessed addresses are valid 5-level kernel > pointers, so the report is a false positive. > > The root cause is in generic KASAN not seeing > cpu_feature_enabled(X86_FEATURE_LA57) set, because the bit is cleared > in identify_cpu() when the offending interrupt happens [1]: > > memset(&c->x86_capability, 0, ...); /* clears X86_FEATURE_LA57 */ > ... > get_cpu_cap(c); /* re-reads CPUID, restores it */ > > addr_has_metadata() then uses the 4-level threshold, and 5-level > kernel addresses fall below it, so kasan_check_range() reports them as > wild-memory-access. > > Define USE_EARLY_PGTABLE_L5 in mm/kasan/generic.c so > addr_has_metadata() uses the stable variable, as > arch/x86/mm/kasan_init_64.c already does. It took me some digging to find which stable variable you refer to here; I think this needs to be better explained. > > Some context on how this was noticed and reproduced below. > > We started seeing flaky splats as above [2][3] on BPF CI runs after > runner hardware has been upgraded. Specifically, new x86 runners are > c7i.metal-24xl AWS EC2 instances, which are Intel Sapphire Rapids > machines that support LA57 feature, and have it enabled. > > The splats can be reproduced with qemu on any x86_64 host with > -cpu max -accel tcg > > Build a kernel with: > CONFIG_KASAN=y > CONFIG_KASAN_GENERIC=y > CONFIG_KASAN_INLINE=y > > Boot it with kasan_multi_shot. The fault fires fast before userspace, > so no rootfs is required. For example: > > qemu-system-x86_64 -display none -serial stdio -no-reboot \ > -smp 4 -m 5G -cpu max -accel tcg \ > -kernel arch/x86/boot/bzImage \ > -append "console=ttyS0,115200 earlyprintk=serial,0,115200 panic=-1 kasan_multi_shot nokaslr" > > It's a timing race, so a single boot hits it only sometimes. > However running several qemu instances in parallel on the same host > significantly increases the hitrate. > > I confirmed the proposed fix eliminates the splats. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/cpu/common.c?h=v7.1-rc7#n2001 > [2] https://github.com/kernel-patches/bpf/actions/runs/27271262414/job/80542509369 > [3] https://github.com/kernel-patches/bpf/actions/runs/27260143782/job/80505353689 > > Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev> > --- > mm/kasan/generic.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c > index 2b8e73f5f6a7..b5f430f2dbb6 100644 > --- a/mm/kasan/generic.c > +++ b/mm/kasan/generic.c > @@ -9,6 +9,13 @@ > * Andrey Konovalov <andreyknvl@gmail.com> > */ > > +/* > + * check_region_inline() and addr_has_metadata() can run very early. > + * For example, in an interrupt taken while identify_cpu() has the CPU > + * capability bits temporarily cleared. > + */ > +#define USE_EARLY_PGTABLE_L5 There are other users of addr_has_metadata(), besides ones in generic.c. Thus, should we place this definition into mm/kasan/kasan.h? And likely guard it with ifdef KASAN_GENERIC || KASAN_SW_TAGS (HW_TAGS should not be affected by this AFAIU). And let's also extend the comment to explain better what's going on. I like the way the issue is explained here: https://elixir.bootlin.com/linux/v7.0.12/source/arch/x86/boot/startup/sme.c#L36 > + > #include <linux/export.h> > #include <linux/interrupt.h> > #include <linux/init.h> > -- > 2.54.0 > > -- > You received this message because you are subscribed to the Google Groups "kasan-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com. > To view this discussion visit https://groups.google.com/d/msgid/kasan-dev/20260610175651.647515-1-ihor.solodrai%40linux.dev. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging 2026-06-10 18:39 ` Andrey Konovalov @ 2026-06-10 21:55 ` Ihor Solodrai 0 siblings, 0 replies; 5+ messages in thread From: Ihor Solodrai @ 2026-06-10 21:55 UTC (permalink / raw) To: Andrey Konovalov Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann, Eduard Zingerman, Kumar Kartikeya Dwivedi, Andrey Ryabinin, Andrew Morton, bpf, kasan-dev, linux-mm, linux-kernel On 6/10/26 11:39 AM, Andrey Konovalov wrote: > On Wed, Jun 10, 2026 at 7:57 PM Ihor Solodrai <ihor.solodrai@linux.dev> wrote: >> >> On x86_64 with 5-level paging (LA57) and inline generic KASAN, the >> following flaky splat may be observed on boot: >> >> BUG: KASAN: wild-memory-access in do_raw_spin_lock+0xcf/0x260 >> Write of size 4 at addr ff110001000c90b8 by task swapper/0/0 >> >> CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.1.0-rc5-gcba33e0b2907 #1 PREEMPT(full) >> Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014 >> Call Trace: >> <IRQ> >> dump_stack_lvl+0x54/0x70 >> kasan_report+0x117/0x150 >> ? do_raw_spin_lock+0xcf/0x260 >> kasan_check_range+0x264/0x2c0 >> do_raw_spin_lock+0xcf/0x260 >> handle_edge_irq+0x35/0x770 >> ? do_raw_spin_unlock+0x51/0x2a0 >> __common_interrupt+0xae/0x120 >> common_interrupt+0x7c/0x90 >> </IRQ> >> <TASK> >> asm_common_interrupt+0x26/0x40 >> RIP: 0010:identify_cpu+0x2b2/0x3460 >> Code: 00 41 c7 07 00 00 00 00 4d 89 e6 49 c1 ee 03 43 0f b6 04 06 84 c0 0f 85 a3 1c 00 00 41 c7 04 24 00 00 00 00 31 c0 31 c9 0f a2 <89> c7 42 0f b6 44 05 00 84 c0 0f 85 ad 1c 00 00 41 89 3f 48 8b 44 >> RSP: 0000:ffffffff97807df0 EFLAGS: 00000246 >> RAX: 0000000000000020 RBX: 00000000756e6547 RCX: 000000006c65746e >> RDX: 0000000049656e69 RSI: 0000000000000000 RDI: ffffffff98632fd8 >> RBP: 1ffffffff30c65fc R08: dffffc0000000000 R09: 0000000000000004 >> R10: ffffffff98632fc4 R11: fffffbfff30c65fb R12: ffffffff98633050 >> R13: ffffffff98633048 R14: 1ffffffff30c660a R15: ffffffff98632fe0 >> identify_boot_cpu+0xd/0xd0 >> arch_cpu_finalize_init+0x24/0x1f0 >> start_kernel+0x31e/0x3e0 >> x86_64_start_reservations+0x24/0x30 >> x86_64_start_kernel+0x13a/0x140 >> common_startup_64+0x12c/0x137 >> </TASK> >> >> It fires very early in boot. If kasan_multi_shot is set, the reports >> are non-fatal and keep repeating, and the boot CPU wedges before >> userspace is reached. The accessed addresses are valid 5-level kernel >> pointers, so the report is a false positive. >> >> The root cause is in generic KASAN not seeing >> cpu_feature_enabled(X86_FEATURE_LA57) set, because the bit is cleared >> in identify_cpu() when the offending interrupt happens [1]: >> >> memset(&c->x86_capability, 0, ...); /* clears X86_FEATURE_LA57 */ >> ... >> get_cpu_cap(c); /* re-reads CPUID, restores it */ >> >> addr_has_metadata() then uses the 4-level threshold, and 5-level >> kernel addresses fall below it, so kasan_check_range() reports them as >> wild-memory-access. >> >> Define USE_EARLY_PGTABLE_L5 in mm/kasan/generic.c so >> addr_has_metadata() uses the stable variable, as >> arch/x86/mm/kasan_init_64.c already does. > > It took me some digging to find which stable variable you refer to > here; I think this needs to be better explained. Hi Andrey, thank you for the quick review. Yeah, it's: unsigned int __section(".data") __pgtable_l5_enabled; I'll add more details in next revision. > >> >> Some context on how this was noticed and reproduced below. >> >> We started seeing flaky splats as above [2][3] on BPF CI runs after >> runner hardware has been upgraded. Specifically, new x86 runners are >> c7i.metal-24xl AWS EC2 instances, which are Intel Sapphire Rapids >> machines that support LA57 feature, and have it enabled. >> >> The splats can be reproduced with qemu on any x86_64 host with >> -cpu max -accel tcg >> >> Build a kernel with: >> CONFIG_KASAN=y >> CONFIG_KASAN_GENERIC=y >> CONFIG_KASAN_INLINE=y >> >> Boot it with kasan_multi_shot. The fault fires fast before userspace, >> so no rootfs is required. For example: >> >> qemu-system-x86_64 -display none -serial stdio -no-reboot \ >> -smp 4 -m 5G -cpu max -accel tcg \ >> -kernel arch/x86/boot/bzImage \ >> -append "console=ttyS0,115200 earlyprintk=serial,0,115200 panic=-1 kasan_multi_shot nokaslr" >> >> It's a timing race, so a single boot hits it only sometimes. >> However running several qemu instances in parallel on the same host >> significantly increases the hitrate. >> >> I confirmed the proposed fix eliminates the splats. >> >> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/cpu/common.c?h=v7.1-rc7#n2001 >> [2] https://github.com/kernel-patches/bpf/actions/runs/27271262414/job/80542509369 >> [3] https://github.com/kernel-patches/bpf/actions/runs/27260143782/job/80505353689 >> >> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev> >> --- >> mm/kasan/generic.c | 7 +++++++ >> 1 file changed, 7 insertions(+) >> >> diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c >> index 2b8e73f5f6a7..b5f430f2dbb6 100644 >> --- a/mm/kasan/generic.c >> +++ b/mm/kasan/generic.c >> @@ -9,6 +9,13 @@ >> * Andrey Konovalov <andreyknvl@gmail.com> >> */ >> >> +/* >> + * check_region_inline() and addr_has_metadata() can run very early. >> + * For example, in an interrupt taken while identify_cpu() has the CPU >> + * capability bits temporarily cleared. >> + */ >> +#define USE_EARLY_PGTABLE_L5 > > There are other users of addr_has_metadata(), besides ones in generic.c. > > Thus, should we place this definition into mm/kasan/kasan.h? And > likely guard it with ifdef KASAN_GENERIC || KASAN_SW_TAGS (HW_TAGS > should not be affected by this AFAIU). It appears setting USE_EARLY_PGTABLE_L5 in kasan.h will not work. USE_EARLY_PGTABLE_L5 needs to be set before arch/x86/include/asm/pgtable_64_types.h is included, which is very early. For example, it arrives to kasan/generic.c through a long path starting at linux/interrupt.h: In file included from ./arch/x86/include/asm/pgtable_types.h:277, from ./arch/x86/include/asm/processor.h:21, from ./arch/x86/include/asm/cpufeature.h:5, from ./arch/x86/include/asm/thread_info.h:59, from ./include/linux/thread_info.h:62, from ./include/linux/smp.h:119, from ./include/linux/alloc_tag.h:14, from ./include/linux/percpu.h:5, from ./include/linux/context_tracking_state.h:5, from ./include/linux/hardirq.h:5, from ./include/linux/interrupt.h:11, from mm/kasan/generic.c:13: So if we want to cover all users of addr_has_metadata(), we'll have to add #define USE_EARLY_PGTABLE_L5 in multiple files: generic.c, report.c, report_generic.c, maybe others? Any other ideas? I determined the include path like follows: cd linux # at 9716c086c8e8 torvalds/master make defconfig ./scripts/config -e KASAN -e KASAN_GENERIC -e KASAN_INLINE make olddefconfig make mm/kasan/generic.i # then modify pgtable_64_types.h diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h index 7eb61ef6a185..c6b754143fb4 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ #ifndef _ASM_X86_PGTABLE_64_DEFS_H #define _ASM_X86_PGTABLE_64_DEFS_H +#error TRACE #include <asm/sparsemem.h> And then: eval "$(grep '^savedcmd_' mm/kasan/.generic.i.cmd | head -1 | sed 's/^savedcmd_[^ ]* := //')" > > And let's also extend the comment to explain better what's going on. I > like the way the issue is explained here: > > https://elixir.bootlin.com/linux/v7.0.12/source/arch/x86/boot/startup/sme.c#L36 ok > >> + >> #include <linux/export.h> >> #include <linux/interrupt.h> >> #include <linux/init.h> >> -- >> 2.54.0 >> >> -- >> You received this message because you are subscribed to the Google Groups "kasan-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com. >> To view this discussion visit https://groups.google.com/d/msgid/kasan-dev/20260610175651.647515-1-ihor.solodrai%40linux.dev. ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging 2026-06-10 17:56 [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging Ihor Solodrai 2026-06-10 18:39 ` Andrey Konovalov @ 2026-06-12 16:30 ` Kiryl Shutsemau 2026-06-12 19:42 ` Ihor Solodrai 1 sibling, 1 reply; 5+ messages in thread From: Kiryl Shutsemau @ 2026-06-12 16:30 UTC (permalink / raw) To: Ihor Solodrai Cc: Borislav Petkov, Thomas Gleixner, Ingo Molnar, Dave Hansen, x86, H. Peter Anvin, Andrey Konovalov, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann, Eduard Zingerman, Kumar Kartikeya Dwivedi, Andrey Ryabinin, Andrew Morton, bpf, kasan-dev, linux-mm, linux-kernel On Wed, Jun 10, 2026 at 10:56:51AM -0700, Ihor Solodrai wrote: > On x86_64 with 5-level paging (LA57) and inline generic KASAN, the > following flaky splat may be observed on boot: > > BUG: KASAN: wild-memory-access in do_raw_spin_lock+0xcf/0x260 > Write of size 4 at addr ff110001000c90b8 by task swapper/0/0 > > CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.1.0-rc5-gcba33e0b2907 #1 PREEMPT(full) > Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014 > Call Trace: > <IRQ> > dump_stack_lvl+0x54/0x70 > kasan_report+0x117/0x150 > ? do_raw_spin_lock+0xcf/0x260 > kasan_check_range+0x264/0x2c0 > do_raw_spin_lock+0xcf/0x260 > handle_edge_irq+0x35/0x770 > ? do_raw_spin_unlock+0x51/0x2a0 > __common_interrupt+0xae/0x120 > common_interrupt+0x7c/0x90 > </IRQ> > <TASK> > asm_common_interrupt+0x26/0x40 > RIP: 0010:identify_cpu+0x2b2/0x3460 > Code: 00 41 c7 07 00 00 00 00 4d 89 e6 49 c1 ee 03 43 0f b6 04 06 84 c0 0f 85 a3 1c 00 00 41 c7 04 24 00 00 00 00 31 c0 31 c9 0f a2 <89> c7 42 0f b6 44 05 00 84 c0 0f 85 ad 1c 00 00 41 89 3f 48 8b 44 > RSP: 0000:ffffffff97807df0 EFLAGS: 00000246 > RAX: 0000000000000020 RBX: 00000000756e6547 RCX: 000000006c65746e > RDX: 0000000049656e69 RSI: 0000000000000000 RDI: ffffffff98632fd8 > RBP: 1ffffffff30c65fc R08: dffffc0000000000 R09: 0000000000000004 > R10: ffffffff98632fc4 R11: fffffbfff30c65fb R12: ffffffff98633050 > R13: ffffffff98633048 R14: 1ffffffff30c660a R15: ffffffff98632fe0 > identify_boot_cpu+0xd/0xd0 > arch_cpu_finalize_init+0x24/0x1f0 > start_kernel+0x31e/0x3e0 > x86_64_start_reservations+0x24/0x30 > x86_64_start_kernel+0x13a/0x140 > common_startup_64+0x12c/0x137 > </TASK> > > It fires very early in boot. If kasan_multi_shot is set, the reports > are non-fatal and keep repeating, and the boot CPU wedges before > userspace is reached. The accessed addresses are valid 5-level kernel > pointers, so the report is a false positive. > > The root cause is in generic KASAN not seeing > cpu_feature_enabled(X86_FEATURE_LA57) set, because the bit is cleared > in identify_cpu() when the offending interrupt happens [1]: > > memset(&c->x86_capability, 0, ...); /* clears X86_FEATURE_LA57 */ > ... > get_cpu_cap(c); /* re-reads CPUID, restores it */ > > addr_has_metadata() then uses the 4-level threshold, and 5-level > kernel addresses fall below it, so kasan_check_range() reports them as > wild-memory-access. > > Define USE_EARLY_PGTABLE_L5 in mm/kasan/generic.c so > addr_has_metadata() uses the stable variable, as > arch/x86/mm/kasan_init_64.c already does. > I'd rather not push USE_EARLY_PGTABLE_L5 into generic KASAN code. It's an x86 paging detail in arch-independent files. It's incomplete (report.c and report_generic.c also call addr_has_metadata()). And it's a permanent slowdown on the KASAN hot path -- pgtable_l5_enabled() becomes a runtime load of __pgtable_l5_enabled on every check, whereas cpu_feature_enabled() gets patched to a constant after alternatives. And it leaves the real bug in place: the window where boot_cpu_data.x86_capability reads back zero is visible to *any* cpu_feature_enabled() caller in interrupt context, not just KASAN. The window is opened by identify_cpu() itself, so fix it there: diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2003,6 +2003,7 @@ static void generic_identify(struct cpuinfo_x86 *c) */ static void identify_cpu(struct cpuinfo_x86 *c) { + unsigned long flags; int i; @@ -2022,12 +2023,21 @@ static void identify_cpu(struct cpuinfo_x86 *c) c->x86_cache_alignment = c->x86_clflush_size; + + /* + * x86_capability is cleared and repopulated from CPUID below. On + * the boot CPU this runs with IRQs on and before alternatives are + * patched, so cpu_feature_enabled() reads the live bits; an + * interrupt in this window sees e.g. X86_FEATURE_LA57 as disabled. + */ + local_irq_save(flags); memset(&c->x86_capability, 0, sizeof(c->x86_capability)); #ifdef CONFIG_X86_VMX_FEATURE_NAMES memset(&c->vmx_capability, 0, sizeof(c->vmx_capability)); #endif generic_identify(c); + local_irq_restore(flags); save/restore keeps it correct for the secondary-CPU callers that already run with IRQs off. I reproduced your splat with parallel TCG guests (-cpu max, kasan_multi_shot): ~10% of boots hit it, 0/~200 with the above. I am not sure how wide the irq-off window suppose to be. I scoped it to memset() .. generic_identify(), where LA57 is restored. Later code (apply_forced_caps(), ->c_init(), setup_sm*p()) only refines bits. Widen it to be defensive, or keep it tight? Any better solution? -- Kiryl Shutsemau / Kirill A. Shutemov ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging 2026-06-12 16:30 ` Kiryl Shutsemau @ 2026-06-12 19:42 ` Ihor Solodrai 0 siblings, 0 replies; 5+ messages in thread From: Ihor Solodrai @ 2026-06-12 19:42 UTC (permalink / raw) To: Kiryl Shutsemau Cc: Borislav Petkov, Thomas Gleixner, Ingo Molnar, Dave Hansen, x86, H. Peter Anvin, Andrey Konovalov, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann, Eduard Zingerman, Kumar Kartikeya Dwivedi, Andrey Ryabinin, Andrew Morton, bpf, kasan-dev, linux-mm, linux-kernel On 6/12/26 9:30 AM, Kiryl Shutsemau wrote: > On Wed, Jun 10, 2026 at 10:56:51AM -0700, Ihor Solodrai wrote: >> On x86_64 with 5-level paging (LA57) and inline generic KASAN, the >> following flaky splat may be observed on boot: >> >> BUG: KASAN: wild-memory-access in do_raw_spin_lock+0xcf/0x260 >> Write of size 4 at addr ff110001000c90b8 by task swapper/0/0 >> >> CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.1.0-rc5-gcba33e0b2907 #1 PREEMPT(full) >> Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014 >> Call Trace: >> <IRQ> >> dump_stack_lvl+0x54/0x70 >> kasan_report+0x117/0x150 >> ? do_raw_spin_lock+0xcf/0x260 >> kasan_check_range+0x264/0x2c0 >> do_raw_spin_lock+0xcf/0x260 >> handle_edge_irq+0x35/0x770 >> ? do_raw_spin_unlock+0x51/0x2a0 >> __common_interrupt+0xae/0x120 >> common_interrupt+0x7c/0x90 >> </IRQ> >> <TASK> >> asm_common_interrupt+0x26/0x40 >> RIP: 0010:identify_cpu+0x2b2/0x3460 >> Code: 00 41 c7 07 00 00 00 00 4d 89 e6 49 c1 ee 03 43 0f b6 04 06 84 c0 0f 85 a3 1c 00 00 41 c7 04 24 00 00 00 00 31 c0 31 c9 0f a2 <89> c7 42 0f b6 44 05 00 84 c0 0f 85 ad 1c 00 00 41 89 3f 48 8b 44 >> RSP: 0000:ffffffff97807df0 EFLAGS: 00000246 >> RAX: 0000000000000020 RBX: 00000000756e6547 RCX: 000000006c65746e >> RDX: 0000000049656e69 RSI: 0000000000000000 RDI: ffffffff98632fd8 >> RBP: 1ffffffff30c65fc R08: dffffc0000000000 R09: 0000000000000004 >> R10: ffffffff98632fc4 R11: fffffbfff30c65fb R12: ffffffff98633050 >> R13: ffffffff98633048 R14: 1ffffffff30c660a R15: ffffffff98632fe0 >> identify_boot_cpu+0xd/0xd0 >> arch_cpu_finalize_init+0x24/0x1f0 >> start_kernel+0x31e/0x3e0 >> x86_64_start_reservations+0x24/0x30 >> x86_64_start_kernel+0x13a/0x140 >> common_startup_64+0x12c/0x137 >> </TASK> >> >> It fires very early in boot. If kasan_multi_shot is set, the reports >> are non-fatal and keep repeating, and the boot CPU wedges before >> userspace is reached. The accessed addresses are valid 5-level kernel >> pointers, so the report is a false positive. >> >> The root cause is in generic KASAN not seeing >> cpu_feature_enabled(X86_FEATURE_LA57) set, because the bit is cleared >> in identify_cpu() when the offending interrupt happens [1]: >> >> memset(&c->x86_capability, 0, ...); /* clears X86_FEATURE_LA57 */ >> ... >> get_cpu_cap(c); /* re-reads CPUID, restores it */ >> >> addr_has_metadata() then uses the 4-level threshold, and 5-level >> kernel addresses fall below it, so kasan_check_range() reports them as >> wild-memory-access. >> >> Define USE_EARLY_PGTABLE_L5 in mm/kasan/generic.c so >> addr_has_metadata() uses the stable variable, as >> arch/x86/mm/kasan_init_64.c already does. >> > > I'd rather not push USE_EARLY_PGTABLE_L5 into generic KASAN code. > > It's an x86 paging detail in arch-independent files. It's incomplete > (report.c and report_generic.c also call addr_has_metadata()). And it's > a permanent slowdown on the KASAN hot path -- pgtable_l5_enabled() > becomes a runtime load of __pgtable_l5_enabled on every check, whereas > cpu_feature_enabled() gets patched to a constant after alternatives. > > And it leaves the real bug in place: the window where > boot_cpu_data.x86_capability reads back zero is visible to *any* > cpu_feature_enabled() caller in interrupt context, not just KASAN. > > The window is opened by identify_cpu() itself, so fix it there: > > diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c > --- a/arch/x86/kernel/cpu/common.c > +++ b/arch/x86/kernel/cpu/common.c > @@ -2003,6 +2003,7 @@ static void generic_identify(struct cpuinfo_x86 *c) > */ > static void identify_cpu(struct cpuinfo_x86 *c) > { > + unsigned long flags; > int i; > @@ -2022,12 +2023,21 @@ static void identify_cpu(struct cpuinfo_x86 *c) > c->x86_cache_alignment = c->x86_clflush_size; > + > + /* > + * x86_capability is cleared and repopulated from CPUID below. On > + * the boot CPU this runs with IRQs on and before alternatives are > + * patched, so cpu_feature_enabled() reads the live bits; an > + * interrupt in this window sees e.g. X86_FEATURE_LA57 as disabled. > + */ > + local_irq_save(flags); > memset(&c->x86_capability, 0, sizeof(c->x86_capability)); > #ifdef CONFIG_X86_VMX_FEATURE_NAMES > memset(&c->vmx_capability, 0, sizeof(c->vmx_capability)); > #endif > generic_identify(c); > + local_irq_restore(flags); > > save/restore keeps it correct for the secondary-CPU callers that already run > with IRQs off. > > I reproduced your splat with parallel TCG guests (-cpu max, kasan_multi_shot): > ~10% of boots hit it, 0/~200 with the above. Hi Kiryl, thank you for testing. I thought of disabling IRQs here too, and AI bot flagged this as well [1]. I wasn't sure it'd be a good fix, because I don't know if it has any undesirable consequences, and whether it's actually a complete fix. For example, can a NMI hit this? [1] https://lore.kernel.org/bpf/7b6d449f-c70c-4e8e-bfc4-a2f75517395c@linux.dev/ > > I am not sure how wide the irq-off window suppose to be. I scoped it to > memset() .. generic_identify(), where LA57 is restored. Later code > (apply_forced_caps(), ->c_init(), setup_sm*p()) only refines bits. > > Widen it to be defensive, or keep it tight? If we scope this narrowly, then it remains LA57-specific, because we protect only a portion of the capabilities array. IIUC to ensure x86_capability is "ready", a proper scope should be entire identify_cpu(). And this may be a problem, because mcheck_cpu_init() does kmalloc(GFP_KERNEL) [2], for example. So I have doubts disabling IRQ is the right approach. [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/cpu/mce/genpool.c?h=v7.1-rc7#n119 > > Any better solution? > What if we define a static key for LA57 feature for KASAN? This will avoid the performance hit we get with USE_EARLY_PGTABLE_L5. And KASAN won't care about x86_capability being cleared. Is that feasible/acceptable? That will be a KASAN-specific LA57-specific fix however. The underlying "capabilities are temporarily cleared on boot" issue would remain. Although it wasn't a problem until this bug surfaced. Something like this: diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index d7e33c7f096b..6c1ebcb4a417 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -36,6 +36,35 @@ static inline void kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid) { } #endif +#ifdef CONFIG_KASAN_GENERIC +#include <linux/jump_label.h> + +DECLARE_STATIC_KEY_TRUE(kasan_la57_enabled); + +#define addr_has_metadata addr_has_metadata +static __always_inline bool addr_has_metadata(const void *addr) +{ + unsigned int shift = static_branch_likely(&kasan_la57_enabled) ? 56 : 47; + + return (unsigned long)addr >= (-1UL << shift); +} +#endif /* CONFIG_KASAN_GENERIC */ + #endif #endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index a4268c47f2bc..79bad61f4891 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -28,6 +28,7 @@ #include <linux/stackprotector.h> #include <linux/utsname.h> #include <linux/efi.h> +#include <linux/kasan.h> #include <asm/alternative.h> #include <asm/cmdline.h> @@ -2581,6 +2582,19 @@ void __init arch_cpu_finalize_init(void) { struct cpuinfo_x86 *c = this_cpu_ptr(&cpu_info); +#ifdef CONFIG_KASAN_GENERIC + if (!__pgtable_l5_enabled) + static_branch_disable(&kasan_la57_enabled); +#endif + identify_boot_cpu(); select_idle_routine(); diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 998b6010d6d3..3cb20b4d8799 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -11,6 +11,7 @@ #include <linux/sched.h> #include <linux/sched/task.h> #include <linux/vmalloc.h> +#include <linux/jump_label.h> #include <asm/e820/types.h> #include <asm/pgalloc.h> @@ -20,6 +21,12 @@ extern struct range pfn_mapped[E820_MAX_ENTRIES]; +DEFINE_STATIC_KEY_TRUE(kasan_la57_enabled); + static p4d_t tmp_p4d_table[MAX_PTRS_PER_P4D] __initdata __aligned(PAGE_SIZE); static __init void *early_alloc(size_t size, int nid, bool should_panic) ^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-06-12 19:42 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-06-10 17:56 [PATCH v1] kasan: Fix false-positive wild-memory-access on x86 under 5-level paging Ihor Solodrai 2026-06-10 18:39 ` Andrey Konovalov 2026-06-10 21:55 ` Ihor Solodrai 2026-06-12 16:30 ` Kiryl Shutsemau 2026-06-12 19:42 ` Ihor Solodrai
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox