* CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
@ 2024-12-13 19:01 Matt Fleming
2024-12-14 1:17 ` Uros Bizjak
0 siblings, 1 reply; 9+ messages in thread
From: Matt Fleming @ 2024-12-13 19:01 UTC (permalink / raw)
To: Uros Bizjak, Ingo Molnar
Cc: Jakub Jelinek, linux-kernel, kernel-team, Matt Fleming
Hi everyone,
I've run into following Oops when running with KASAN enabled:
[ 22.938710][ T0] Oops: general protection fault, probably for non-canonical address 0xdffffc00000087c8: 0000 [#1] PREEMPT SMP KASAN NOPTI
[ 22.939369][ T0] KASAN: probably user-memory-access in range [0x0000000000043e40-0x0000000000043e47]
[ 22.939369][ T0] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.12.1-cloudflare-kasan-2024.11.20 #1
[ 22.939369][ T0] Hardware name: MACHINE, BIOS VERSION 09/04/2024
[ 22.939369][ T0] RIP: 0010:switch_mm_irqs_off+0x43/0xd70
[ 22.939369][ T0] Code: 48 83 ec 20 48 c7 c0 40 3e 04 00 65 48 8b 1d 14 41 91 77 48 ba 00 00 00 00 00 fc ff df 65 44 0f b7 25 11 41 91 77 48 c1 e8 03 <0f> b6 04 10 84 c0 74 06 0f 8e be 09 00 00 65 44 0f b6 2d a6 41 91
[ 22.939369][ T0] RSP: 0000:ffffffff8ce07e00 EFLAGS: 00010012
[ 22.939369][ T0] RAX: 00000000000087c8 RBX: ffffffff8d20a740 RCX: 0000001850076000
[ 22.939369][ T0] RDX: dffffc0000000000 RSI: ffffffff8d7a28c0 RDI: 0000000000000000
[ 22.939369][ T0] RBP: ffffffff8d7a28c0 R08: 00000000aa299018 R09: e4977f26b7bc541a
[ 22.939369][ T0] R10: ffffffff8ce07e00 R11: 8000000000000063 R12: 0000000000000000
[ 22.939369][ T0] R13: 0000001850076000 R14: 0000000000000000 R15: ffff889850076000
[ 22.939369][ T0] FS: 0000000000000000(0000) GS:ffff8887efa00000(0000) knlGS:0000000000000000
[ 22.939369][ T0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 22.939369][ T0] CR2: ffff88c04f1ff000 CR3: 00000037f4852001 CR4: 0000000000770ef0
[ 22.939369][ T0] PKRU: 55555554
[ 22.939369][ T0] Call Trace:
[ 22.939369][ T0] <TASK>
[ 22.939369][ T0] ? __die_body.cold+0x19/0x21
[ 22.939369][ T0] ? die_addr+0x46/0x70
[ 22.939369][ T0] ? exc_general_protection+0x119/0x210
[ 22.939369][ T0] ? asm_exc_general_protection+0x26/0x30
[ 22.939369][ T0] ? switch_mm_irqs_off+0x43/0xd70
[ 22.939369][ T0] ? __pfx_efi_memmap_init_late+0x10/0x10
[ 22.939369][ T0] switch_mm+0x14/0x20
[ 22.939369][ T0] efi_set_virtual_address_map+0x75/0x180
[ 22.939369][ T0] ? srso_alias_return_thunk+0x5/0xfbef5
[ 22.939369][ T0] efi_enter_virtual_mode+0x6fb/0x7c0
[ 22.939369][ T0] ? alt_reloc_selftest+0x1f/0x50
[ 22.939369][ T0] start_kernel+0x323/0x3a0
[ 22.939369][ T0] x86_64_start_reservations+0x24/0x30
[ 22.939369][ T0] x86_64_start_kernel+0x7f/0x80
[ 22.939369][ T0] common_startup_64+0x13e/0x141
[ 22.939369][ T0] </TASK>
This was supposed to be fixed by the compiler version check in
f61f02d1ff78 ("x86/percpu: Re-enable named address spaces with KASAN for
GCC 13.3+"), but I'm still able to trigger this problem with both GCC
14.1.0 and GCC 13.3.0 which include fixes for PR sanitizer/111736.
(Reverting f61f02d1ff78 obviously prevents the oops)
Here's the assembly that shows the ASAN protection kicking in for the
per-CPU addresses (cpu_tlbstate):
ffffffff8112fc40 <switch_mm_irqs_off>:
ffffffff8112fc40: f3 0f 1e fa endbr64
ffffffff8112fc44: e8 e7 79 fd ff call ffffffff81107630 <__fentry__>
ffffffff8112fc49: 41 57 push %r15
ffffffff8112fc4b: 41 56 push %r14
ffffffff8112fc4d: 49 89 d6 mov %rdx,%r14
ffffffff8112fc50: 41 55 push %r13
ffffffff8112fc52: 41 54 push %r12
ffffffff8112fc54: 55 push %rbp
ffffffff8112fc55: 48 89 f5 mov %rsi,%rbp
ffffffff8112fc58: 53 push %rbx
ffffffff8112fc59: 48 83 ec 20 sub $0x20,%rsp
ffffffff8112fc5d: 48 c7 c0 40 3e 04 00 mov $0x43e40,%rax
ffffffff8112fc64: 65 48 8b 1d 14 41 f1 mov %gs:0x7ef14114(%rip),%rbx # 43d80 <cpu_tlbstate>
ffffffff8112fc6b: 7e
ffffffff8112fc6c: 48 ba 00 00 00 00 00 movabs $0xdffffc0000000000,%rdx
ffffffff8112fc73: fc ff df
ffffffff8112fc76: 65 44 0f b7 25 11 41 movzwl %gs:0x7ef14111(%rip),%r12d # 43d90 <cpu_tlbstate+0x10>
ffffffff8112fc7d: f1 7e
ffffffff8112fc7f: 48 c1 e8 03 shr $0x3,%rax
ffffffff8112fc83: 0f b6 04 10 movzbl (%rax,%rdx,1),%eax
Fault occurs here due to ASAN -----------^
Anyone got any ideas how to debug this further?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2024-12-13 19:01 CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0 Matt Fleming
@ 2024-12-14 1:17 ` Uros Bizjak
2024-12-16 16:20 ` Matt Fleming
0 siblings, 1 reply; 9+ messages in thread
From: Uros Bizjak @ 2024-12-14 1:17 UTC (permalink / raw)
To: Matt Fleming; +Cc: Ingo Molnar, Jakub Jelinek, linux-kernel, kernel-team
On Fri, Dec 13, 2024 at 8:01 PM Matt Fleming <matt@readmodwrite.com> wrote:
>
> Hi everyone,
>
> I've run into following Oops when running with KASAN enabled:
>
> [ 22.938710][ T0] Oops: general protection fault, probably for non-canonical address 0xdffffc00000087c8: 0000 [#1] PREEMPT SMP KASAN NOPTI
> [ 22.939369][ T0] KASAN: probably user-memory-access in range [0x0000000000043e40-0x0000000000043e47]
> [ 22.939369][ T0] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.12.1-cloudflare-kasan-2024.11.20 #1
> [ 22.939369][ T0] Hardware name: MACHINE, BIOS VERSION 09/04/2024
> [ 22.939369][ T0] RIP: 0010:switch_mm_irqs_off+0x43/0xd70
> [ 22.939369][ T0] Code: 48 83 ec 20 48 c7 c0 40 3e 04 00 65 48 8b 1d 14 41 91 77 48 ba 00 00 00 00 00 fc ff df 65 44 0f b7 25 11 41 91 77 48 c1 e8 03 <0f> b6 04 10 84 c0 74 06 0f 8e be 09 00 00 65 44 0f b6 2d a6 41 91
> [ 22.939369][ T0] RSP: 0000:ffffffff8ce07e00 EFLAGS: 00010012
> [ 22.939369][ T0] RAX: 00000000000087c8 RBX: ffffffff8d20a740 RCX: 0000001850076000
> [ 22.939369][ T0] RDX: dffffc0000000000 RSI: ffffffff8d7a28c0 RDI: 0000000000000000
> [ 22.939369][ T0] RBP: ffffffff8d7a28c0 R08: 00000000aa299018 R09: e4977f26b7bc541a
> [ 22.939369][ T0] R10: ffffffff8ce07e00 R11: 8000000000000063 R12: 0000000000000000
> [ 22.939369][ T0] R13: 0000001850076000 R14: 0000000000000000 R15: ffff889850076000
> [ 22.939369][ T0] FS: 0000000000000000(0000) GS:ffff8887efa00000(0000) knlGS:0000000000000000
> [ 22.939369][ T0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 22.939369][ T0] CR2: ffff88c04f1ff000 CR3: 00000037f4852001 CR4: 0000000000770ef0
> [ 22.939369][ T0] PKRU: 55555554
> [ 22.939369][ T0] Call Trace:
> [ 22.939369][ T0] <TASK>
> [ 22.939369][ T0] ? __die_body.cold+0x19/0x21
> [ 22.939369][ T0] ? die_addr+0x46/0x70
> [ 22.939369][ T0] ? exc_general_protection+0x119/0x210
> [ 22.939369][ T0] ? asm_exc_general_protection+0x26/0x30
> [ 22.939369][ T0] ? switch_mm_irqs_off+0x43/0xd70
> [ 22.939369][ T0] ? __pfx_efi_memmap_init_late+0x10/0x10
> [ 22.939369][ T0] switch_mm+0x14/0x20
> [ 22.939369][ T0] efi_set_virtual_address_map+0x75/0x180
> [ 22.939369][ T0] ? srso_alias_return_thunk+0x5/0xfbef5
> [ 22.939369][ T0] efi_enter_virtual_mode+0x6fb/0x7c0
> [ 22.939369][ T0] ? alt_reloc_selftest+0x1f/0x50
> [ 22.939369][ T0] start_kernel+0x323/0x3a0
> [ 22.939369][ T0] x86_64_start_reservations+0x24/0x30
> [ 22.939369][ T0] x86_64_start_kernel+0x7f/0x80
> [ 22.939369][ T0] common_startup_64+0x13e/0x141
> [ 22.939369][ T0] </TASK>
>
> This was supposed to be fixed by the compiler version check in
> f61f02d1ff78 ("x86/percpu: Re-enable named address spaces with KASAN for
> GCC 13.3+"), but I'm still able to trigger this problem with both GCC
> 14.1.0 and GCC 13.3.0 which include fixes for PR sanitizer/111736.
> (Reverting f61f02d1ff78 obviously prevents the oops)
>
> Here's the assembly that shows the ASAN protection kicking in for the
> per-CPU addresses (cpu_tlbstate):
>
> ffffffff8112fc40 <switch_mm_irqs_off>:
> ffffffff8112fc40: f3 0f 1e fa endbr64
> ffffffff8112fc44: e8 e7 79 fd ff call ffffffff81107630 <__fentry__>
> ffffffff8112fc49: 41 57 push %r15
> ffffffff8112fc4b: 41 56 push %r14
> ffffffff8112fc4d: 49 89 d6 mov %rdx,%r14
> ffffffff8112fc50: 41 55 push %r13
> ffffffff8112fc52: 41 54 push %r12
> ffffffff8112fc54: 55 push %rbp
> ffffffff8112fc55: 48 89 f5 mov %rsi,%rbp
> ffffffff8112fc58: 53 push %rbx
> ffffffff8112fc59: 48 83 ec 20 sub $0x20,%rsp
> ffffffff8112fc5d: 48 c7 c0 40 3e 04 00 mov $0x43e40,%rax
> ffffffff8112fc64: 65 48 8b 1d 14 41 f1 mov %gs:0x7ef14114(%rip),%rbx # 43d80 <cpu_tlbstate>
> ffffffff8112fc6b: 7e
> ffffffff8112fc6c: 48 ba 00 00 00 00 00 movabs $0xdffffc0000000000,%rdx
> ffffffff8112fc73: fc ff df
> ffffffff8112fc76: 65 44 0f b7 25 11 41 movzwl %gs:0x7ef14111(%rip),%r12d # 43d90 <cpu_tlbstate+0x10>
> ffffffff8112fc7d: f1 7e
> ffffffff8112fc7f: 48 c1 e8 03 shr $0x3,%rax
> ffffffff8112fc83: 0f b6 04 10 movzbl (%rax,%rdx,1),%eax
>
> Fault occurs here due to ASAN -----------^
>
>
> Anyone got any ideas how to debug this further?
Does your config include CONFIG_UBSAN_BOOL=y ?
There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
(aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
[2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
Otherwise, please attach your .config, and I'll look into this issue.
Thanks,
Uros.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2024-12-14 1:17 ` Uros Bizjak
@ 2024-12-16 16:20 ` Matt Fleming
2024-12-16 16:56 ` Uros Bizjak
0 siblings, 1 reply; 9+ messages in thread
From: Matt Fleming @ 2024-12-16 16:20 UTC (permalink / raw)
To: Uros Bizjak; +Cc: Ingo Molnar, Jakub Jelinek, linux-kernel, kernel-team
On Sat, Dec 14, 2024 at 1:17 AM Uros Bizjak <ubizjak@gmail.com> wrote:
>
> Does your config include CONFIG_UBSAN_BOOL=y ?
Yes, it does!
> There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
> (aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
>
> [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
>
> [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
>
> Otherwise, please attach your .config, and I'll look into this issue.
Thanks. Disabling CONFIG_UBSAN_BOOL does indeed make the kernels boot again.
Should CONFIG_UBSAN_BOOL have a dependency on GCC 14.4+ ?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2024-12-16 16:20 ` Matt Fleming
@ 2024-12-16 16:56 ` Uros Bizjak
2025-02-27 12:22 ` Ingo Molnar
0 siblings, 1 reply; 9+ messages in thread
From: Uros Bizjak @ 2024-12-16 16:56 UTC (permalink / raw)
To: Matt Fleming; +Cc: Ingo Molnar, Jakub Jelinek, linux-kernel, kernel-team
On Mon, Dec 16, 2024 at 5:20 PM Matt Fleming <matt@readmodwrite.com> wrote:
>
> On Sat, Dec 14, 2024 at 1:17 AM Uros Bizjak <ubizjak@gmail.com> wrote:
> >
> > Does your config include CONFIG_UBSAN_BOOL=y ?
>
> Yes, it does!
>
> > There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
> > (aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
> >
> > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
> >
> > [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
> >
> > Otherwise, please attach your .config, and I'll look into this issue.
>
> Thanks. Disabling CONFIG_UBSAN_BOOL does indeed make the kernels boot again.
>
> Should CONFIG_UBSAN_BOOL have a dependency on GCC 14.4+ ?
No, this is a very rare Oops that triggers only with gcc-14.1 version
and only when both KASAN and UBSAN are enabled. This is actually the
problem with sanitization of the percpu address when named address
spaces are enabled (IOW, sanitization of __seg_gs prefixed address).
UBSAN creates a temporary in memory, but forgets to copy memory tags,
including named address space qualifier from the original. Later ASAN
sanitizes this location as a normal variable (due to missing
qualifier), but actually should be disabled for __seg_gs prefixed
addresses.
Your report is only *the second* since sanitizers were re-enabled with
named address spaces. gcc-14.2 that includes the fix is available
since August 2024, and since sanitizers are strictly development
tools, my proposed solution would be to simply upgrade the compiler to
gcc-14.2 release.
Uros.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2024-12-16 16:56 ` Uros Bizjak
@ 2025-02-27 12:22 ` Ingo Molnar
2025-02-27 12:30 ` Uros Bizjak
0 siblings, 1 reply; 9+ messages in thread
From: Ingo Molnar @ 2025-02-27 12:22 UTC (permalink / raw)
To: Uros Bizjak; +Cc: Matt Fleming, Jakub Jelinek, linux-kernel, kernel-team
* Uros Bizjak <ubizjak@gmail.com> wrote:
> On Mon, Dec 16, 2024 at 5:20 PM Matt Fleming <matt@readmodwrite.com> wrote:
> >
> > On Sat, Dec 14, 2024 at 1:17 AM Uros Bizjak <ubizjak@gmail.com> wrote:
> > >
> > > Does your config include CONFIG_UBSAN_BOOL=y ?
> >
> > Yes, it does!
> >
> > > There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
> > > (aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
> > >
> > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
> > >
> > > [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
> > >
> > > Otherwise, please attach your .config, and I'll look into this issue.
> >
> > Thanks. Disabling CONFIG_UBSAN_BOOL does indeed make the kernels boot again.
> >
> > Should CONFIG_UBSAN_BOOL have a dependency on GCC 14.4+ ?
>
> No, this is a very rare Oops that triggers only with gcc-14.1 version
> and only when both KASAN and UBSAN are enabled. This is actually the
> problem with sanitization of the percpu address when named address
> spaces are enabled (IOW, sanitization of __seg_gs prefixed address).
> UBSAN creates a temporary in memory, but forgets to copy memory tags,
> including named address space qualifier from the original. Later ASAN
> sanitizes this location as a normal variable (due to missing
> qualifier), but actually should be disabled for __seg_gs prefixed
> addresses.
>
> Your report is only *the second* since sanitizers were re-enabled with
> named address spaces. gcc-14.2 that includes the fix is available
> since August 2024, and since sanitizers are strictly development
> tools, my proposed solution would be to simply upgrade the compiler to
> gcc-14.2 release.
So unless this is difficult to test for, it would be nice to have a
compiler version cutoff for the feature. Especially if it's been
reported twice already, chances are that a lot more people have
experienced it already.
The kernel build should avoid this known oops automatically.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2025-02-27 12:22 ` Ingo Molnar
@ 2025-02-27 12:30 ` Uros Bizjak
2025-02-27 12:46 ` Ingo Molnar
0 siblings, 1 reply; 9+ messages in thread
From: Uros Bizjak @ 2025-02-27 12:30 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Matt Fleming, Jakub Jelinek, linux-kernel, kernel-team
On Thu, Feb 27, 2025 at 1:22 PM Ingo Molnar <mingo@kernel.org> wrote:
>
>
> * Uros Bizjak <ubizjak@gmail.com> wrote:
>
> > On Mon, Dec 16, 2024 at 5:20 PM Matt Fleming <matt@readmodwrite.com> wrote:
> > >
> > > On Sat, Dec 14, 2024 at 1:17 AM Uros Bizjak <ubizjak@gmail.com> wrote:
> > > >
> > > > Does your config include CONFIG_UBSAN_BOOL=y ?
> > >
> > > Yes, it does!
> > >
> > > > There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
> > > > (aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
> > > >
> > > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
> > > >
> > > > [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
> > > >
> > > > Otherwise, please attach your .config, and I'll look into this issue.
> > >
> > > Thanks. Disabling CONFIG_UBSAN_BOOL does indeed make the kernels boot again.
> > >
> > > Should CONFIG_UBSAN_BOOL have a dependency on GCC 14.4+ ?
> >
> > No, this is a very rare Oops that triggers only with gcc-14.1 version
> > and only when both KASAN and UBSAN are enabled. This is actually the
> > problem with sanitization of the percpu address when named address
> > spaces are enabled (IOW, sanitization of __seg_gs prefixed address).
> > UBSAN creates a temporary in memory, but forgets to copy memory tags,
> > including named address space qualifier from the original. Later ASAN
> > sanitizes this location as a normal variable (due to missing
> > qualifier), but actually should be disabled for __seg_gs prefixed
> > addresses.
> >
> > Your report is only *the second* since sanitizers were re-enabled with
> > named address spaces. gcc-14.2 that includes the fix is available
> > since August 2024, and since sanitizers are strictly development
> > tools, my proposed solution would be to simply upgrade the compiler to
> > gcc-14.2 release.
>
> So unless this is difficult to test for, it would be nice to have a
> compiler version cutoff for the feature. Especially if it's been
> reported twice already, chances are that a lot more people have
> experienced it already.
>
> The kernel build should avoid this known oops automatically.
The patch could be as simple as:
--cut here--
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 95ea2b4b95db..c94c37889917 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2370,7 +2370,7 @@ config CC_HAS_NAMED_AS
depends on CC_IS_GCC
config CC_HAS_NAMED_AS_FIXED_SANITIZERS
- def_bool CC_IS_GCC && GCC_VERSION >= 130300
+ def_bool CC_IS_GCC && GCC_VERSION >= 140200
config USE_X86_SEG_SUPPORT
def_bool y
--cut here--
but it will disable all sanitizers for a very rare Oops that needs the
combination of CONFIG_KASAN and CONFIG_UBSAN_BOOL. The fix is
available in gcc-14.2, released in August 2024.
Uros.
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2025-02-27 12:30 ` Uros Bizjak
@ 2025-02-27 12:46 ` Ingo Molnar
2025-02-27 13:35 ` Uros Bizjak
0 siblings, 1 reply; 9+ messages in thread
From: Ingo Molnar @ 2025-02-27 12:46 UTC (permalink / raw)
To: Uros Bizjak; +Cc: Matt Fleming, Jakub Jelinek, linux-kernel, kernel-team
* Uros Bizjak <ubizjak@gmail.com> wrote:
> On Thu, Feb 27, 2025 at 1:22 PM Ingo Molnar <mingo@kernel.org> wrote:
> >
> >
> > * Uros Bizjak <ubizjak@gmail.com> wrote:
> >
> > > On Mon, Dec 16, 2024 at 5:20 PM Matt Fleming <matt@readmodwrite.com> wrote:
> > > >
> > > > On Sat, Dec 14, 2024 at 1:17 AM Uros Bizjak <ubizjak@gmail.com> wrote:
> > > > >
> > > > > Does your config include CONFIG_UBSAN_BOOL=y ?
> > > >
> > > > Yes, it does!
> > > >
> > > > > There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
> > > > > (aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
> > > > >
> > > > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
> > > > >
> > > > > [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
> > > > >
> > > > > Otherwise, please attach your .config, and I'll look into this issue.
> > > >
> > > > Thanks. Disabling CONFIG_UBSAN_BOOL does indeed make the kernels boot again.
> > > >
> > > > Should CONFIG_UBSAN_BOOL have a dependency on GCC 14.4+ ?
> > >
> > > No, this is a very rare Oops that triggers only with gcc-14.1 version
> > > and only when both KASAN and UBSAN are enabled. This is actually the
> > > problem with sanitization of the percpu address when named address
> > > spaces are enabled (IOW, sanitization of __seg_gs prefixed address).
> > > UBSAN creates a temporary in memory, but forgets to copy memory tags,
> > > including named address space qualifier from the original. Later ASAN
> > > sanitizes this location as a normal variable (due to missing
> > > qualifier), but actually should be disabled for __seg_gs prefixed
> > > addresses.
> > >
> > > Your report is only *the second* since sanitizers were re-enabled with
> > > named address spaces. gcc-14.2 that includes the fix is available
> > > since August 2024, and since sanitizers are strictly development
> > > tools, my proposed solution would be to simply upgrade the compiler to
> > > gcc-14.2 release.
> >
> > So unless this is difficult to test for, it would be nice to have a
> > compiler version cutoff for the feature. Especially if it's been
> > reported twice already, chances are that a lot more people have
> > experienced it already.
> >
> > The kernel build should avoid this known oops automatically.
>
> The patch could be as simple as:
>
> --cut here--
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 95ea2b4b95db..c94c37889917 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -2370,7 +2370,7 @@ config CC_HAS_NAMED_AS
> depends on CC_IS_GCC
>
> config CC_HAS_NAMED_AS_FIXED_SANITIZERS
> - def_bool CC_IS_GCC && GCC_VERSION >= 130300
> + def_bool CC_IS_GCC && GCC_VERSION >= 140200
>
> config USE_X86_SEG_SUPPORT
> def_bool y
> --cut here--
>
> but it will disable all sanitizers for a very rare Oops that needs the
> combination of CONFIG_KASAN and CONFIG_UBSAN_BOOL.
Can we not limit the version quirk to KASAN && UBSAN_BOOL?
> The fix is available in gcc-14.2, released in August 2024.
And this is a contradictory argument: if the fix is so available then
it shouldn't really matter whether we have to quirk or not? Everyone
should have the fix already and the quirk shouldn't affect them, right?
Thanks,
Ingo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2025-02-27 12:46 ` Ingo Molnar
@ 2025-02-27 13:35 ` Uros Bizjak
2025-02-27 18:27 ` Ingo Molnar
0 siblings, 1 reply; 9+ messages in thread
From: Uros Bizjak @ 2025-02-27 13:35 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Matt Fleming, Jakub Jelinek, linux-kernel, kernel-team
[-- Attachment #1: Type: text/plain, Size: 3321 bytes --]
On Thu, Feb 27, 2025 at 1:46 PM Ingo Molnar <mingo@kernel.org> wrote:
>
>
> * Uros Bizjak <ubizjak@gmail.com> wrote:
>
> > On Thu, Feb 27, 2025 at 1:22 PM Ingo Molnar <mingo@kernel.org> wrote:
> > >
> > >
> > > * Uros Bizjak <ubizjak@gmail.com> wrote:
> > >
> > > > On Mon, Dec 16, 2024 at 5:20 PM Matt Fleming <matt@readmodwrite.com> wrote:
> > > > >
> > > > > On Sat, Dec 14, 2024 at 1:17 AM Uros Bizjak <ubizjak@gmail.com> wrote:
> > > > > >
> > > > > > Does your config include CONFIG_UBSAN_BOOL=y ?
> > > > >
> > > > > Yes, it does!
> > > > >
> > > > > > There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
> > > > > > (aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
> > > > > >
> > > > > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
> > > > > >
> > > > > > [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
> > > > > >
> > > > > > Otherwise, please attach your .config, and I'll look into this issue.
> > > > >
> > > > > Thanks. Disabling CONFIG_UBSAN_BOOL does indeed make the kernels boot again.
> > > > >
> > > > > Should CONFIG_UBSAN_BOOL have a dependency on GCC 14.4+ ?
> > > >
> > > > No, this is a very rare Oops that triggers only with gcc-14.1 version
> > > > and only when both KASAN and UBSAN are enabled. This is actually the
> > > > problem with sanitization of the percpu address when named address
> > > > spaces are enabled (IOW, sanitization of __seg_gs prefixed address).
> > > > UBSAN creates a temporary in memory, but forgets to copy memory tags,
> > > > including named address space qualifier from the original. Later ASAN
> > > > sanitizes this location as a normal variable (due to missing
> > > > qualifier), but actually should be disabled for __seg_gs prefixed
> > > > addresses.
> > > >
> > > > Your report is only *the second* since sanitizers were re-enabled with
> > > > named address spaces. gcc-14.2 that includes the fix is available
> > > > since August 2024, and since sanitizers are strictly development
> > > > tools, my proposed solution would be to simply upgrade the compiler to
> > > > gcc-14.2 release.
> > >
> > > So unless this is difficult to test for, it would be nice to have a
> > > compiler version cutoff for the feature. Especially if it's been
> > > reported twice already, chances are that a lot more people have
> > > experienced it already.
> > >
> > > The kernel build should avoid this known oops automatically.
> >
> > The patch could be as simple as:
> >
> > --cut here--
> > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> > index 95ea2b4b95db..c94c37889917 100644
> > --- a/arch/x86/Kconfig
> > +++ b/arch/x86/Kconfig
> > @@ -2370,7 +2370,7 @@ config CC_HAS_NAMED_AS
> > depends on CC_IS_GCC
> >
> > config CC_HAS_NAMED_AS_FIXED_SANITIZERS
> > - def_bool CC_IS_GCC && GCC_VERSION >= 130300
> > + def_bool CC_IS_GCC && GCC_VERSION >= 140200
> >
> > config USE_X86_SEG_SUPPORT
> > def_bool y
> > --cut here--
> >
> > but it will disable all sanitizers for a very rare Oops that needs the
> > combination of CONFIG_KASAN and CONFIG_UBSAN_BOOL.
>
> Can we not limit the version quirk to KASAN && UBSAN_BOOL?
I am testing the attached patch that resolves the issue.
Thanks,
Uros.
[-- Attachment #2: p.diff.txt --]
[-- Type: text/plain, Size: 1166 bytes --]
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 95ea2b4b95db..b92e0c3f7f19 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2369,18 +2369,20 @@ config CC_HAS_NAMED_AS
def_bool $(success,echo 'int __seg_fs fs; int __seg_gs gs;' | $(CC) -x c - -S -o /dev/null)
depends on CC_IS_GCC
+#
+# -fsanitize=kernel-address (KASAN) and -fsanitize=thread (KCSAN)
+# are incompatible with named address spaces with GCC < 13.3
+# (see GCC PR sanitizer/111736 and also PR sanitizer/115172).
+#
+
config CC_HAS_NAMED_AS_FIXED_SANITIZERS
- def_bool CC_IS_GCC && GCC_VERSION >= 130300
+ def_bool y
+ depends on !(KASAN || KCSAN) || GCC_VERSION >= 130300
+ depends on !(KASAN && UBSAN_BOOL) || GCC_VERSION >= 140200
config USE_X86_SEG_SUPPORT
- def_bool y
- depends on CC_HAS_NAMED_AS
- #
- # -fsanitize=kernel-address (KASAN) and -fsanitize=thread
- # (KCSAN) are incompatible with named address spaces with
- # GCC < 13.3 - see GCC PR sanitizer/111736.
- #
- depends on !(KASAN || KCSAN) || CC_HAS_NAMED_AS_FIXED_SANITIZERS
+ def_bool CC_HAS_NAMED_AS
+ depends on CC_HAS_NAMED_AS_FIXED_SANITIZERS
config CC_HAS_SLS
def_bool $(cc-option,-mharden-sls=all)
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0
2025-02-27 13:35 ` Uros Bizjak
@ 2025-02-27 18:27 ` Ingo Molnar
0 siblings, 0 replies; 9+ messages in thread
From: Ingo Molnar @ 2025-02-27 18:27 UTC (permalink / raw)
To: Uros Bizjak; +Cc: Matt Fleming, Jakub Jelinek, linux-kernel, kernel-team
* Uros Bizjak <ubizjak@gmail.com> wrote:
> On Thu, Feb 27, 2025 at 1:46 PM Ingo Molnar <mingo@kernel.org> wrote:
> >
> >
> > * Uros Bizjak <ubizjak@gmail.com> wrote:
> >
> > > On Thu, Feb 27, 2025 at 1:22 PM Ingo Molnar <mingo@kernel.org> wrote:
> > > >
> > > >
> > > > * Uros Bizjak <ubizjak@gmail.com> wrote:
> > > >
> > > > > On Mon, Dec 16, 2024 at 5:20 PM Matt Fleming <matt@readmodwrite.com> wrote:
> > > > > >
> > > > > > On Sat, Dec 14, 2024 at 1:17 AM Uros Bizjak <ubizjak@gmail.com> wrote:
> > > > > > >
> > > > > > > Does your config include CONFIG_UBSAN_BOOL=y ?
> > > > > >
> > > > > > Yes, it does!
> > > > > >
> > > > > > > There is a rare interaction between CONFIG_KASAN and CONFIG_UBSAN_BOOL
> > > > > > > (aka -fsanitize=bool), reported in [1] and fixed for gcc-14.2 in [2].
> > > > > > >
> > > > > > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111736#c42
> > > > > > >
> > > > > > > [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115172
> > > > > > >
> > > > > > > Otherwise, please attach your .config, and I'll look into this issue.
> > > > > >
> > > > > > Thanks. Disabling CONFIG_UBSAN_BOOL does indeed make the kernels boot again.
> > > > > >
> > > > > > Should CONFIG_UBSAN_BOOL have a dependency on GCC 14.4+ ?
> > > > >
> > > > > No, this is a very rare Oops that triggers only with gcc-14.1 version
> > > > > and only when both KASAN and UBSAN are enabled. This is actually the
> > > > > problem with sanitization of the percpu address when named address
> > > > > spaces are enabled (IOW, sanitization of __seg_gs prefixed address).
> > > > > UBSAN creates a temporary in memory, but forgets to copy memory tags,
> > > > > including named address space qualifier from the original. Later ASAN
> > > > > sanitizes this location as a normal variable (due to missing
> > > > > qualifier), but actually should be disabled for __seg_gs prefixed
> > > > > addresses.
> > > > >
> > > > > Your report is only *the second* since sanitizers were re-enabled with
> > > > > named address spaces. gcc-14.2 that includes the fix is available
> > > > > since August 2024, and since sanitizers are strictly development
> > > > > tools, my proposed solution would be to simply upgrade the compiler to
> > > > > gcc-14.2 release.
> > > >
> > > > So unless this is difficult to test for, it would be nice to have a
> > > > compiler version cutoff for the feature. Especially if it's been
> > > > reported twice already, chances are that a lot more people have
> > > > experienced it already.
> > > >
> > > > The kernel build should avoid this known oops automatically.
> > >
> > > The patch could be as simple as:
> > >
> > > --cut here--
> > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> > > index 95ea2b4b95db..c94c37889917 100644
> > > --- a/arch/x86/Kconfig
> > > +++ b/arch/x86/Kconfig
> > > @@ -2370,7 +2370,7 @@ config CC_HAS_NAMED_AS
> > > depends on CC_IS_GCC
> > >
> > > config CC_HAS_NAMED_AS_FIXED_SANITIZERS
> > > - def_bool CC_IS_GCC && GCC_VERSION >= 130300
> > > + def_bool CC_IS_GCC && GCC_VERSION >= 140200
> > >
> > > config USE_X86_SEG_SUPPORT
> > > def_bool y
> > > --cut here--
> > >
> > > but it will disable all sanitizers for a very rare Oops that needs the
> > > combination of CONFIG_KASAN and CONFIG_UBSAN_BOOL.
> >
> > Can we not limit the version quirk to KASAN && UBSAN_BOOL?
>
> I am testing the attached patch that resolves the issue.
Thank you!
Ingo
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2025-02-27 18:27 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-12-13 19:01 CONFIG_KASAN triggers ASAN bug in GCC 13.3.0 and 14.1.0 Matt Fleming
2024-12-14 1:17 ` Uros Bizjak
2024-12-16 16:20 ` Matt Fleming
2024-12-16 16:56 ` Uros Bizjak
2025-02-27 12:22 ` Ingo Molnar
2025-02-27 12:30 ` Uros Bizjak
2025-02-27 12:46 ` Ingo Molnar
2025-02-27 13:35 ` Uros Bizjak
2025-02-27 18:27 ` Ingo Molnar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox