* Re: [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node
[not found] ` <20261026224305.1254-1-hdanton@sina.com>
@ 2026-10-27 14:00 ` Sean Christopherson
2026-10-27 18:48 ` Harry Yoo
0 siblings, 1 reply; 6+ messages in thread
From: Sean Christopherson @ 2026-10-27 14:00 UTC (permalink / raw)
To: Hillf Danton
Cc: Boqun Feng, syzbot, kvm, linux-kernel, Tetsuo Handa,
syzkaller-bugs, Vlastimil Babka, Harry Yoo, Andrew Morton, Hao Li,
Christoph Lameter, David Rientjes, Roman Gushchin, linux-mm
+slub folks
On Tue, Oct 27, 2026, Hillf Danton wrote:
> Ask the locking people to shed light on the warning.
get_partial_node_bulk() is waiting on a non-raw spinlock while in hard IRQ context,
and it looks like __refill_objects_node() would do the same later on. On PREEMPT_RT,
those spinlocks become sleepable locks, which is what triggers the splat.
The alloction coming from KVM-as-a-guest code is GFP_ATOMIC, so I think that wonky
code is fine.
> > Date: Mon, 26 Oct 2026 03:19:37 -0700 [thread overview]
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit: c03c1a14507d Linux 7.3
> > git tree: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> > console output: https://syzkaller.appspot.com/x/log.txt?x=128d8ca5580000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=129d675874dbf896
> > dashboard link: https://syzkaller.appspot.com/bug?extid=f11512d1e5878ba5e60e
> > compiler: gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=168d8ca5580000
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+f11512d1e5878ba5e60e@syzkaller.appspotmail.com
> >
> > =============================
> > [ BUG: Invalid wait context ]
> > syzkaller #0 Tainted: G S
> > -----------------------------
> > syz-executor177/6041 is trying to lock:
> > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: get_partial_node_bulk mm/slub.c:3831 [inline]
> > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > other info that might help us debug this:
> > context-{2:2}
> > locks held by syz-executor177/6041: 1, last CPU#1:
> > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_lock_acquire include/linux/srcu.h:198 [inline]
> > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_read_lock include/linux/srcu.h:305 [inline]
> > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_read_lock security/tomoyo/common.h:1112 [inline]
> > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_check_open_permission+0x169/0x3c0 security/tomoyo/file.c:772
> > stack backtrace:
> > CPU: 1 UID: 0 PID: 6041 Comm: syz-executor177 Tainted: G S syzkaller #0 PREEMPT(full)
> > Tainted: [S]=CPU_OUT_OF_SPEC
> > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
> > Call Trace:
> > <IRQ>
> > __dump_stack lib/dump_stack.c:94 [inline]
> > dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120
> > print_lock_invalid_wait_context kernel/locking/lockdep.c:4874 [inline]
> > check_wait_context kernel/locking/lockdep.c:4946 [inline]
> > __lock_acquire+0xfcc/0x1f40 kernel/locking/lockdep.c:5238
> > lock_acquire kernel/locking/lockdep.c:5942 [inline]
> > lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
> > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:155 [inline]
> > _raw_spin_lock_irqsave+0x3a/0x60 kernel/locking/spinlock.c:181
> > get_partial_node_bulk mm/slub.c:3831 [inline]
> > __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > refill_objects+0x58/0x410 mm/slub.c:7405
> > refill_sheaf mm/slub.c:2885 [inline]
> > __pcs_replace_empty_main+0x35e/0x650 mm/slub.c:4774
> > alloc_from_pcs mm/slub.c:4850 [inline]
> > slab_alloc_node mm/slub.c:4984 [inline]
> > __kmalloc_cache_noprof+0x4f4/0x6d0 mm/slub.c:5559
> > _kmalloc_noprof include/linux/slab.h:991 [inline]
> > _kzalloc_noprof include/linux/slab.h:1312 [inline]
> > kvm_async_pf_task_wake arch/x86/kernel/kvm.c:224 [inline]
> > __sysvec_kvm_asyncpf_interrupt+0x334/0x610 arch/x86/kernel/kvm.c:338
> > instr_sysvec_kvm_asyncpf_interrupt arch/x86/kernel/kvm.c:327 [inline]
> > sysvec_kvm_asyncpf_interrupt+0x9e/0xc0 arch/x86/kernel/kvm.c:327
> > </IRQ>
> > <TASK>
> > asm_sysvec_kvm_asyncpf_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:753
> > RIP: 0010:memcmp+0x156/0x1d0 lib/string.c:665
> > Code: 0f 86 fb fe ff ff 48 89 d8 48 89 de 48 c1 e8 03 83 e6 07 42 0f b6 14 28 48 8d 43 07 48 89 c1 48 c1 e9 03 40 38 f2 40 0f 9e c6 <84> d2 42 0f b6 0c 29 0f 95 c2 40 84 d6 0f 84 45 ff ff ff be 08 00
> > RSP: 0018:ffffc90003ab7538 EFLAGS: 00000297
> > RAX: ffff8880220be413 RBX: ffff8880220be40c RCX: 1ffff11004417c82
> > RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8880220be40c
> > RBP: ffffffff8c58a940 R08: 0000000000000001 R09: 0000000000000066
> > R10: 0000000000000066 R11: 0000000000000001 R12: 0000000000000012
> > R13: dffffc0000000000 R14: dffffc0000000000 R15: 0000000000000000
> > strstr+0x109/0x170 lib/string.c:743
> > tomoyo_add_entry security/tomoyo/common.c:2157 [inline]
> > tomoyo_supervisor+0xf47/0x1340 security/tomoyo/common.c:2238
> > tomoyo_audit_path_log security/tomoyo/file.c:169 [inline]
> > tomoyo_path_permission security/tomoyo/file.c:592 [inline]
> > tomoyo_path_permission+0x270/0x3b0 security/tomoyo/file.c:577
> > tomoyo_check_open_permission+0x37f/0x3c0 security/tomoyo/file.c:782
> > tomoyo_file_open+0x6b/0x90 security/tomoyo/tomoyo.c:334
> > security_file_open+0xb5/0x1e0 security/security.c:2739
> > do_dentry_open+0x588/0x14d0 fs/open.c:973
> > vfs_open+0x82/0x3f0 fs/open.c:1101
> > do_open fs/namei.c:4836 [inline]
> > path_openat+0x1a23/0x2450 fs/namei.c:4999
> > do_file_open+0x20e/0x430 fs/namei.c:5028
> > do_sys_openat2+0x10f/0x1e0 fs/open.c:1417
> > do_sys_open fs/open.c:1423 [inline]
> > __do_sys_openat fs/open.c:1439 [inline]
> > __se_sys_openat fs/open.c:1434 [inline]
> > __x64_sys_openat+0x12d/0x210 fs/open.c:1434
> > do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
> > do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
> > entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node
2026-10-27 14:00 ` [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node Sean Christopherson
@ 2026-10-27 18:48 ` Harry Yoo
2026-10-27 20:39 ` Steven Rostedt
0 siblings, 1 reply; 6+ messages in thread
From: Harry Yoo @ 2026-10-27 18:48 UTC (permalink / raw)
To: Sean Christopherson
Cc: Hillf Danton, Boqun Feng, syzbot, kvm, linux-kernel, Tetsuo Handa,
syzkaller-bugs, Vlastimil Babka, Andrew Morton, Hao Li,
Christoph Lameter, David Rientjes, Roman Gushchin, linux-mm,
Sebastian Andrzej Siewior, Clark Williams, Steven Rostedt,
linux-rt-devel
[+Cc RT folks]
On Tue, Oct 27, 2026 at 07:00:16AM -0700, Sean Christopherson wrote:
> +slub folks
>
> On Tue, Oct 27, 2026, Hillf Danton wrote:
> > Ask the locking people to shed light on the warning.
>
> get_partial_node_bulk() is waiting on a non-raw spinlock while in hard IRQ context,
> and it looks like __refill_objects_node() would do the same later on. On PREEMPT_RT,
> those spinlocks become sleepable locks, which is what triggers the splat.
>
> The alloction coming from KVM-as-a-guest code is GFP_ATOMIC, so I think that wonky
> code is fine.
But memory allocation w/ GFP_ATOMIC is fine only on non-RT?
From the documentation:
https://docs.kernel.org/core-api/real-time/differences.html#memory-allocation
> The memory allocation APIs, such as kmalloc() and alloc_pages(),
> require a gfp_t flag to indicate the allocation context.
> On non-PREEMPT_RT kernels, it is necessary to use GFP_ATOMIC when
> allocating memory from interrupt context or from sections where
> preemption is disabled. This is because the allocator must not
> sleep in these contexts waiting for memory to become available.
Here it mentions GFP_ATOMIC allocations in preemption disabled
contexts on non-RT, and...
> However, this approach does not work on PREEMPT_RT kernels.
> The memory allocator in PREEMPT_RT uses sleeping locks internally,
> which cannot be acquired when preemption is disabled.
it says we shoud not allocate memory in such a context on RT?
> Fortunately,
> this is generally not a problem, because PREEMPT_RT moves most contexts
> that would traditionally run with preemption or interrupts disabled
> into threaded context, where sleeping is allowed.
Not sure if this applies here as I don't understand what's
going in in KVM world.
> > > Date: Mon, 26 Oct 2026 03:19:37 -0700 [thread overview]
> > > Hello,
> > >
> > > syzbot found the following issue on:
> > >
> > > HEAD commit: c03c1a14507d Linux 7.3
> > > git tree: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=128d8ca5580000
> > > kernel config: https://syzkaller.appspot.com/x/.config?x=129d675874dbf896
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=f11512d1e5878ba5e60e
> > > compiler: gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44
> > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=168d8ca5580000
> > >
> > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > Reported-by: syzbot+f11512d1e5878ba5e60e@syzkaller.appspotmail.com
> > >
> > > =============================
> > > [ BUG: Invalid wait context ]
> > > syzkaller #0 Tainted: G S
> > > -----------------------------
> > > syz-executor177/6041 is trying to lock:
> > > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: get_partial_node_bulk mm/slub.c:3831 [inline]
> > > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > > other info that might help us debug this:
> > > context-{2:2}
> > > locks held by syz-executor177/6041: 1, last CPU#1:
> > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_lock_acquire include/linux/srcu.h:198 [inline]
> > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_read_lock include/linux/srcu.h:305 [inline]
> > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_read_lock security/tomoyo/common.h:1112 [inline]
> > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_check_open_permission+0x169/0x3c0 security/tomoyo/file.c:772
> > > stack backtrace:
> > > CPU: 1 UID: 0 PID: 6041 Comm: syz-executor177 Tainted: G S syzkaller #0 PREEMPT(full)
> > > Tainted: [S]=CPU_OUT_OF_SPEC
> > > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
> > > Call Trace:
> > > <IRQ>
> > > __dump_stack lib/dump_stack.c:94 [inline]
> > > dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120
> > > print_lock_invalid_wait_context kernel/locking/lockdep.c:4874 [inline]
> > > check_wait_context kernel/locking/lockdep.c:4946 [inline]
> > > __lock_acquire+0xfcc/0x1f40 kernel/locking/lockdep.c:5238
> > > lock_acquire kernel/locking/lockdep.c:5942 [inline]
> > > lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
> > > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:155 [inline]
> > > _raw_spin_lock_irqsave+0x3a/0x60 kernel/locking/spinlock.c:181
> > > get_partial_node_bulk mm/slub.c:3831 [inline]
> > > __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > > refill_objects+0x58/0x410 mm/slub.c:7405
> > > refill_sheaf mm/slub.c:2885 [inline]
> > > __pcs_replace_empty_main+0x35e/0x650 mm/slub.c:4774
> > > alloc_from_pcs mm/slub.c:4850 [inline]
> > > slab_alloc_node mm/slub.c:4984 [inline]
> > > __kmalloc_cache_noprof+0x4f4/0x6d0 mm/slub.c:5559
> > > _kmalloc_noprof include/linux/slab.h:991 [inline]
> > > _kzalloc_noprof include/linux/slab.h:1312 [inline]
> > > kvm_async_pf_task_wake arch/x86/kernel/kvm.c:224 [inline]
> > > __sysvec_kvm_asyncpf_interrupt+0x334/0x610 arch/x86/kernel/kvm.c:338
> > > instr_sysvec_kvm_asyncpf_interrupt arch/x86/kernel/kvm.c:327 [inline]
> > > sysvec_kvm_asyncpf_interrupt+0x9e/0xc0 arch/x86/kernel/kvm.c:327
> > > </IRQ>
> > > <TASK>
> > > asm_sysvec_kvm_asyncpf_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:753
> > > RIP: 0010:memcmp+0x156/0x1d0 lib/string.c:665
> > > Code: 0f 86 fb fe ff ff 48 89 d8 48 89 de 48 c1 e8 03 83 e6 07 42 0f b6 14 28 48 8d 43 07 48 89 c1 48 c1 e9 03 40 38 f2 40 0f 9e c6 <84> d2 42 0f b6 0c 29 0f 95 c2 40 84 d6 0f 84 45 ff ff ff be 08 00
> > > RSP: 0018:ffffc90003ab7538 EFLAGS: 00000297
> > > RAX: ffff8880220be413 RBX: ffff8880220be40c RCX: 1ffff11004417c82
> > > RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8880220be40c
> > > RBP: ffffffff8c58a940 R08: 0000000000000001 R09: 0000000000000066
> > > R10: 0000000000000066 R11: 0000000000000001 R12: 0000000000000012
> > > R13: dffffc0000000000 R14: dffffc0000000000 R15: 0000000000000000
> > > strstr+0x109/0x170 lib/string.c:743
> > > tomoyo_add_entry security/tomoyo/common.c:2157 [inline]
> > > tomoyo_supervisor+0xf47/0x1340 security/tomoyo/common.c:2238
> > > tomoyo_audit_path_log security/tomoyo/file.c:169 [inline]
> > > tomoyo_path_permission security/tomoyo/file.c:592 [inline]
> > > tomoyo_path_permission+0x270/0x3b0 security/tomoyo/file.c:577
> > > tomoyo_check_open_permission+0x37f/0x3c0 security/tomoyo/file.c:782
> > > tomoyo_file_open+0x6b/0x90 security/tomoyo/tomoyo.c:334
> > > security_file_open+0xb5/0x1e0 security/security.c:2739
> > > do_dentry_open+0x588/0x14d0 fs/open.c:973
> > > vfs_open+0x82/0x3f0 fs/open.c:1101
> > > do_open fs/namei.c:4836 [inline]
> > > path_openat+0x1a23/0x2450 fs/namei.c:4999
> > > do_file_open+0x20e/0x430 fs/namei.c:5028
> > > do_sys_openat2+0x10f/0x1e0 fs/open.c:1417
> > > do_sys_open fs/open.c:1423 [inline]
> > > __do_sys_openat fs/open.c:1439 [inline]
> > > __se_sys_openat fs/open.c:1434 [inline]
> > > __x64_sys_openat+0x12d/0x210 fs/open.c:1434
> > > do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
> > > do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
> > > entry_SYSCALL_64_after_hwframe+0x77/0x7f
> >
--
Cheers,
Harry / Hyeonggon
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node
2026-10-27 18:48 ` Harry Yoo
@ 2026-10-27 20:39 ` Steven Rostedt
2026-11-02 17:43 ` Sean Christopherson
0 siblings, 1 reply; 6+ messages in thread
From: Steven Rostedt @ 2026-10-27 20:39 UTC (permalink / raw)
To: Harry Yoo
Cc: Sean Christopherson, Hillf Danton, Boqun Feng, syzbot, kvm,
linux-kernel, Tetsuo Handa, syzkaller-bugs, Vlastimil Babka,
Andrew Morton, Hao Li, Christoph Lameter, David Rientjes,
Roman Gushchin, linux-mm, Sebastian Andrzej Siewior,
Clark Williams, linux-rt-devel
On Tue, 27 Oct 2026 18:48:28 +0000
Harry Yoo <harry@kernel.org> wrote:
> [+Cc RT folks]
>
> On Tue, Oct 27, 2026 at 07:00:16AM -0700, Sean Christopherson wrote:
> > +slub folks
> >
> > On Tue, Oct 27, 2026, Hillf Danton wrote:
> > > Ask the locking people to shed light on the warning.
> >
> > get_partial_node_bulk() is waiting on a non-raw spinlock while in hard IRQ context,
> > and it looks like __refill_objects_node() would do the same later on. On PREEMPT_RT,
> > those spinlocks become sleepable locks, which is what triggers the splat.
> >
> > The alloction coming from KVM-as-a-guest code is GFP_ATOMIC, so I think that wonky
> > code is fine.
>
> But memory allocation w/ GFP_ATOMIC is fine only on non-RT?
Correct. GFP_ATOMIC is meaningless on RT. It still cannot be called with
preemption disabled. Most places that disable preemption in the non-RT case
do not disable it in the RT case. For example, spin_lock() disables
preemption on non-RT but does not in RT. Same for interrupt handlers, as in
RT interrupt handlers are run in task context.
That is, in RT only "special" places disable preemption for real.
>
> >From the documentation:
> https://docs.kernel.org/core-api/real-time/differences.html#memory-allocation
> > The memory allocation APIs, such as kmalloc() and alloc_pages(),
> > require a gfp_t flag to indicate the allocation context.
> > On non-PREEMPT_RT kernels, it is necessary to use GFP_ATOMIC when
> > allocating memory from interrupt context or from sections where
> > preemption is disabled. This is because the allocator must not
> > sleep in these contexts waiting for memory to become available.
>
> Here it mentions GFP_ATOMIC allocations in preemption disabled
> contexts on non-RT, and...
>
> > However, this approach does not work on PREEMPT_RT kernels.
> > The memory allocator in PREEMPT_RT uses sleeping locks internally,
> > which cannot be acquired when preemption is disabled.
>
> it says we shoud not allocate memory in such a context on RT?
>
> > Fortunately,
> > this is generally not a problem, because PREEMPT_RT moves most contexts
> > that would traditionally run with preemption or interrupts disabled
> > into threaded context, where sleeping is allowed.
>
> Not sure if this applies here as I don't understand what's
> going in in KVM world.
>
> > > > Date: Mon, 26 Oct 2026 03:19:37 -0700 [thread overview]
> > > > Hello,
> > > >
> > > > syzbot found the following issue on:
> > > >
> > > > HEAD commit: c03c1a14507d Linux 7.3
> > > > git tree: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=128d8ca5580000
> > > > kernel config: https://syzkaller.appspot.com/x/.config?x=129d675874dbf896
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=f11512d1e5878ba5e60e
> > > > compiler: gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44
> > > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=168d8ca5580000
> > > >
> > > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > > Reported-by: syzbot+f11512d1e5878ba5e60e@syzkaller.appspotmail.com
> > > >
> > > > =============================
> > > > [ BUG: Invalid wait context ]
> > > > syzkaller #0 Tainted: G S
> > > > -----------------------------
> > > > syz-executor177/6041 is trying to lock:
> > > > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: get_partial_node_bulk mm/slub.c:3831 [inline]
> > > > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > > > other info that might help us debug this:
> > > > context-{2:2}
> > > > locks held by syz-executor177/6041: 1, last CPU#1:
> > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_lock_acquire include/linux/srcu.h:198 [inline]
> > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_read_lock include/linux/srcu.h:305 [inline]
> > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_read_lock security/tomoyo/common.h:1112 [inline]
> > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_check_open_permission+0x169/0x3c0 security/tomoyo/file.c:772
> > > > stack backtrace:
> > > > CPU: 1 UID: 0 PID: 6041 Comm: syz-executor177 Tainted: G S syzkaller #0 PREEMPT(full)
> > > > Tainted: [S]=CPU_OUT_OF_SPEC
> > > > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
> > > > Call Trace:
> > > > <IRQ>
> > > > __dump_stack lib/dump_stack.c:94 [inline]
> > > > dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120
> > > > print_lock_invalid_wait_context kernel/locking/lockdep.c:4874 [inline]
> > > > check_wait_context kernel/locking/lockdep.c:4946 [inline]
> > > > __lock_acquire+0xfcc/0x1f40 kernel/locking/lockdep.c:5238
> > > > lock_acquire kernel/locking/lockdep.c:5942 [inline]
> > > > lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
> > > > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:155 [inline]
> > > > _raw_spin_lock_irqsave+0x3a/0x60 kernel/locking/spinlock.c:181
> > > > get_partial_node_bulk mm/slub.c:3831 [inline]
> > > > __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > > > refill_objects+0x58/0x410 mm/slub.c:7405
> > > > refill_sheaf mm/slub.c:2885 [inline]
> > > > __pcs_replace_empty_main+0x35e/0x650 mm/slub.c:4774
> > > > alloc_from_pcs mm/slub.c:4850 [inline]
> > > > slab_alloc_node mm/slub.c:4984 [inline]
> > > > __kmalloc_cache_noprof+0x4f4/0x6d0 mm/slub.c:5559
> > > > _kmalloc_noprof include/linux/slab.h:991 [inline]
> > > > _kzalloc_noprof include/linux/slab.h:1312 [inline]
> > > > kvm_async_pf_task_wake arch/x86/kernel/kvm.c:224 [inline]
Hmm, this part is OK:
if (!dummy) {
raw_spin_unlock(&b->lock);
dummy = kzalloc_obj(*dummy, GFP_ATOMIC);
As it releases the raw_spin_lock() before calling kzalloc_obj().
So something else must have disabled preemption/interrupts.
> > > > __sysvec_kvm_asyncpf_interrupt+0x334/0x610 arch/x86/kernel/kvm.c:338
DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_asyncpf_interrupt)
{
struct pt_regs *old_regs = set_irq_regs(regs);
u32 token;
apic_eoi();
inc_irq_stat(HYPERVISOR_CALLBACK);
if (__this_cpu_read(async_pf_enabled)) {
token = __this_cpu_read(apf_reason.token);
kvm_async_pf_task_wake(token);
__this_cpu_write(apf_reason.token, 0);
wrmsrq(MSR_KVM_ASYNC_PF_ACK, 1);
}
set_irq_regs(old_regs);
}
Is that run in hard interrupt context? If so, we need to make that a thread
for RT kernels if you want to call kzalloc_obj().
-- Steve
> > > > instr_sysvec_kvm_asyncpf_interrupt arch/x86/kernel/kvm.c:327 [inline]
> > > > sysvec_kvm_asyncpf_interrupt+0x9e/0xc0 arch/x86/kernel/kvm.c:327
> > > > </IRQ>
> > > > <TASK>
> > > > asm_sysvec_kvm_asyncpf_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:753
> > > > RIP: 0010:memcmp+0x156/0x1d0 lib/string.c:665
> > > > Code: 0f 86 fb fe ff ff 48 89 d8 48 89 de 48 c1 e8 03 83 e6 07 42 0f b6 14 28 48 8d 43 07 48 89 c1 48 c1 e9 03 40 38 f2 40 0f 9e c6 <84> d2 42 0f b6 0c 29 0f 95 c2 40 84 d6 0f 84 45 ff ff ff be 08 00
> > > > RSP: 0018:ffffc90003ab7538 EFLAGS: 00000297
> > > > RAX: ffff8880220be413 RBX: ffff8880220be40c RCX: 1ffff11004417c82
> > > > RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8880220be40c
> > > > RBP: ffffffff8c58a940 R08: 0000000000000001 R09: 0000000000000066
> > > > R10: 0000000000000066 R11: 0000000000000001 R12: 0000000000000012
> > > > R13: dffffc0000000000 R14: dffffc0000000000 R15: 0000000000000000
> > > > strstr+0x109/0x170 lib/string.c:743
> > > > tomoyo_add_entry security/tomoyo/common.c:2157 [inline]
> > > > tomoyo_supervisor+0xf47/0x1340 security/tomoyo/common.c:2238
> > > > tomoyo_audit_path_log security/tomoyo/file.c:169 [inline]
> > > > tomoyo_path_permission security/tomoyo/file.c:592 [inline]
> > > > tomoyo_path_permission+0x270/0x3b0 security/tomoyo/file.c:577
> > > > tomoyo_check_open_permission+0x37f/0x3c0 security/tomoyo/file.c:782
> > > > tomoyo_file_open+0x6b/0x90 security/tomoyo/tomoyo.c:334
> > > > security_file_open+0xb5/0x1e0 security/security.c:2739
> > > > do_dentry_open+0x588/0x14d0 fs/open.c:973
> > > > vfs_open+0x82/0x3f0 fs/open.c:1101
> > > > do_open fs/namei.c:4836 [inline]
> > > > path_openat+0x1a23/0x2450 fs/namei.c:4999
> > > > do_file_open+0x20e/0x430 fs/namei.c:5028
> > > > do_sys_openat2+0x10f/0x1e0 fs/open.c:1417
> > > > do_sys_open fs/open.c:1423 [inline]
> > > > __do_sys_openat fs/open.c:1439 [inline]
> > > > __se_sys_openat fs/open.c:1434 [inline]
> > > > __x64_sys_openat+0x12d/0x210 fs/open.c:1434
> > > > do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
> > > > do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
> > > > entry_SYSCALL_64_after_hwframe+0x77/0x7f
> > >
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node
2026-10-27 20:39 ` Steven Rostedt
@ 2026-11-02 17:43 ` Sean Christopherson
2026-11-03 11:18 ` Paolo Bonzini
0 siblings, 1 reply; 6+ messages in thread
From: Sean Christopherson @ 2026-11-02 17:43 UTC (permalink / raw)
To: Steven Rostedt
Cc: Harry Yoo, Hillf Danton, Boqun Feng, syzbot, kvm, linux-kernel,
Tetsuo Handa, syzkaller-bugs, Vlastimil Babka, Andrew Morton,
Hao Li, Christoph Lameter, David Rientjes, Roman Gushchin,
linux-mm, Sebastian Andrzej Siewior, Clark Williams,
linux-rt-devel, Paolo Bonzini
+Paolo
On Tue, Oct 27, 2026, Steven Rostedt wrote:
> On Tue, 27 Oct 2026 18:48:28 +0000
> Harry Yoo <harry@kernel.org> wrote:
>
> > [+Cc RT folks]
> >
> > On Tue, Oct 27, 2026 at 07:00:16AM -0700, Sean Christopherson wrote:
> > > +slub folks
> > >
> > > On Tue, Oct 27, 2026, Hillf Danton wrote:
> > > > Ask the locking people to shed light on the warning.
> > >
> > > get_partial_node_bulk() is waiting on a non-raw spinlock while in hard IRQ context,
> > > and it looks like __refill_objects_node() would do the same later on. On PREEMPT_RT,
> > > those spinlocks become sleepable locks, which is what triggers the splat.
> > >
> > > The alloction coming from KVM-as-a-guest code is GFP_ATOMIC, so I think that wonky
> > > code is fine.
> >
> > But memory allocation w/ GFP_ATOMIC is fine only on non-RT?
>
> Correct. GFP_ATOMIC is meaningless on RT. It still cannot be called with
> preemption disabled. Most places that disable preemption in the non-RT case
> do not disable it in the RT case. For example, spin_lock() disables
> preemption on non-RT but does not in RT. Same for interrupt handlers, as in
> RT interrupt handlers are run in task context.
>
> That is, in RT only "special" places disable preemption for real.
...
> > > > > =============================
> > > > > [ BUG: Invalid wait context ]
> > > > > syzkaller #0 Tainted: G S
> > > > > -----------------------------
> > > > > syz-executor177/6041 is trying to lock:
> > > > > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: get_partial_node_bulk mm/slub.c:3831 [inline]
> > > > > ffff88801d040618 (&n->list_lock){-.-.}-{3:3}, at: __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > > > > other info that might help us debug this:
> > > > > context-{2:2}
> > > > > locks held by syz-executor177/6041: 1, last CPU#1:
> > > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_lock_acquire include/linux/srcu.h:198 [inline]
> > > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: srcu_read_lock include/linux/srcu.h:305 [inline]
> > > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_read_lock security/tomoyo/common.h:1112 [inline]
> > > > > #0: ffffffff9090b438 (tomoyo_ss){.+.+}-{0:0}, at: tomoyo_check_open_permission+0x169/0x3c0 security/tomoyo/file.c:772
> > > > > stack backtrace:
> > > > > CPU: 1 UID: 0 PID: 6041 Comm: syz-executor177 Tainted: G S syzkaller #0 PREEMPT(full)
> > > > > Tainted: [S]=CPU_OUT_OF_SPEC
> > > > > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
> > > > > Call Trace:
> > > > > <IRQ>
> > > > > __dump_stack lib/dump_stack.c:94 [inline]
> > > > > dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120
> > > > > print_lock_invalid_wait_context kernel/locking/lockdep.c:4874 [inline]
> > > > > check_wait_context kernel/locking/lockdep.c:4946 [inline]
> > > > > __lock_acquire+0xfcc/0x1f40 kernel/locking/lockdep.c:5238
> > > > > lock_acquire kernel/locking/lockdep.c:5942 [inline]
> > > > > lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
> > > > > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:155 [inline]
> > > > > _raw_spin_lock_irqsave+0x3a/0x60 kernel/locking/spinlock.c:181
> > > > > get_partial_node_bulk mm/slub.c:3831 [inline]
> > > > > __refill_objects_node+0x104/0xab0 mm/slub.c:7279
> > > > > refill_objects+0x58/0x410 mm/slub.c:7405
> > > > > refill_sheaf mm/slub.c:2885 [inline]
> > > > > __pcs_replace_empty_main+0x35e/0x650 mm/slub.c:4774
> > > > > alloc_from_pcs mm/slub.c:4850 [inline]
> > > > > slab_alloc_node mm/slub.c:4984 [inline]
> > > > > __kmalloc_cache_noprof+0x4f4/0x6d0 mm/slub.c:5559
> > > > > _kmalloc_noprof include/linux/slab.h:991 [inline]
> > > > > _kzalloc_noprof include/linux/slab.h:1312 [inline]
> > > > > kvm_async_pf_task_wake arch/x86/kernel/kvm.c:224 [inline]
>
> Hmm, this part is OK:
>
> if (!dummy) {
> raw_spin_unlock(&b->lock);
> dummy = kzalloc_obj(*dummy, GFP_ATOMIC);
>
> As it releases the raw_spin_lock() before calling kzalloc_obj().
>
> So something else must have disabled preemption/interrupts.
Heh, yeah, the IRQ that kicked all of this off.
> > > > > __sysvec_kvm_asyncpf_interrupt+0x334/0x610 arch/x86/kernel/kvm.c:338
>
> DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_asyncpf_interrupt)
> {
> struct pt_regs *old_regs = set_irq_regs(regs);
> u32 token;
>
> apic_eoi();
>
> inc_irq_stat(HYPERVISOR_CALLBACK);
>
> if (__this_cpu_read(async_pf_enabled)) {
> token = __this_cpu_read(apf_reason.token);
> kvm_async_pf_task_wake(token);
> __this_cpu_write(apf_reason.token, 0);
> wrmsrq(MSR_KVM_ASYNC_PF_ACK, 1);
> }
>
> set_irq_regs(old_regs);
> }
>
>
> Is that run in hard interrupt context?
Yep.
> If so, we need to make that a thread for RT kernels if you want to call
> kzalloc_obj().
Eww. I can't imagine there's a sane usecase for memory overcommitting a real-time
guest. I'd rather just disable PV async #PF support, e.g.
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 6b0a5861ccb8..b14b03720525 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -51,7 +51,7 @@
DEFINE_STATIC_KEY_FALSE_RO(kvm_async_pf_enabled);
-static int kvmapf = 1;
+static int kvmapf = !IS_ENABLED(CONFIG_PREEMPT_RT);
static int __init parse_no_kvmapf(char *arg)
{
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node
2026-11-02 17:43 ` Sean Christopherson
@ 2026-11-03 11:18 ` Paolo Bonzini
2026-11-03 15:17 ` Sean Christopherson
0 siblings, 1 reply; 6+ messages in thread
From: Paolo Bonzini @ 2026-11-03 11:18 UTC (permalink / raw)
To: Sean Christopherson
Cc: Steven Rostedt, Harry Yoo, Hillf Danton, Boqun Feng, syzbot, kvm,
linux-kernel, Tetsuo Handa, syzkaller-bugs, Vlastimil Babka,
Andrew Morton, Hao Li, Christoph Lameter, David Rientjes,
Roman Gushchin, linux-mm, Sebastian Andrzej Siewior,
Clark Williams, linux-rt-devel
On Mon, Nov 2, 2026 at 6:43 PM Sean Christopherson <seanjc@google.com> wrote:
> > If so, we need to make that a thread for RT kernels if you want to call
> > kzalloc_obj().
>
> Eww. I can't imagine there's a sane usecase for memory overcommitting a real-time
> guest.
While I agree with the feeling, the dummy allocation path should be
mostly dead, because page fault should never (?) enable interrupts
before waiting:
#PF interrupt gate
exc_page_fault()
kvm_handle_async_pf()
__kvm_handle_async_pf()
irqentry_enter()
kvm_async_pf_task_wait_schedule()
kvm_async_pf_queue_task() <- registers waiter
prepare_to_swait_exclusive()
local_irq_enable()
The dummy allocation path is only reachable if it's an L1 hypervisor
that is getting an async page fault while L2 is running, because it
enables irq before the call to kvm_async_pf_task_wait_schedule(). I
think we can drop the dummy allocation for good, which would be even
better.
Whether you would still want to disable APF for RT kernels is up to you.
Paolo
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node
2026-11-03 11:18 ` Paolo Bonzini
@ 2026-11-03 15:17 ` Sean Christopherson
0 siblings, 0 replies; 6+ messages in thread
From: Sean Christopherson @ 2026-11-03 15:17 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Steven Rostedt, Harry Yoo, Hillf Danton, Boqun Feng, syzbot, kvm,
linux-kernel, Tetsuo Handa, syzkaller-bugs, Vlastimil Babka,
Andrew Morton, Hao Li, Christoph Lameter, David Rientjes,
Roman Gushchin, linux-mm, Sebastian Andrzej Siewior,
Clark Williams, linux-rt-devel
On Tue, Nov 03, 2026, Paolo Bonzini wrote:
> On Mon, Nov 2, 2026 at 6:43 PM Sean Christopherson <seanjc@google.com> wrote:
> > > If so, we need to make that a thread for RT kernels if you want to call
> > > kzalloc_obj().
> >
> > Eww. I can't imagine there's a sane usecase for memory overcommitting a real-time
> > guest.
>
> While I agree with the feeling, the dummy allocation path should be
> mostly dead, because page fault should never (?) enable interrupts
> before waiting:
>
> #PF interrupt gate
> exc_page_fault()
> kvm_handle_async_pf()
> __kvm_handle_async_pf()
> irqentry_enter()
> kvm_async_pf_task_wait_schedule()
> kvm_async_pf_queue_task() <- registers waiter
> prepare_to_swait_exclusive()
> local_irq_enable()
>
> The dummy allocation path is only reachable if it's an L1 hypervisor
> that is getting an async page fault while L2 is running, because it
> enables irq before the call to kvm_async_pf_task_wait_schedule(). I
> think we can drop the dummy allocation for good, which would be even
> better.
+1, I completely forgot what this code is doing (although that implies I ever
knew in the first place...).
> Whether you would still want to disable APF for RT kernels is up to you.
Nah, on second thought there are use cases for async #PF other than memory
overcommit. Whether or not they coincide with PREEMPT_RT is debatable, but I
don't think it makes sense to go looking for trouble.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-03 15:17 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <6adf2939.67317f00.38c4bc.00ae.GAE@google.com>
[not found] ` <20261026224305.1254-1-hdanton@sina.com>
2026-10-27 14:00 ` [syzbot] [kvm?] WARNING: locking bug in __refill_objects_node Sean Christopherson
2026-10-27 18:48 ` Harry Yoo
2026-10-27 20:39 ` Steven Rostedt
2026-11-02 17:43 ` Sean Christopherson
2026-11-03 11:18 ` Paolo Bonzini
2026-11-03 15:17 ` Sean Christopherson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox