[BUG] kernel.sysrq sysctl race condition(s) / list corruptions

[BUG] kernel.sysrq sysctl race condition(s) / list corruptions

[Stefan Bühler]

Hi,

we have `kernel.sysrq = 0` in /etc/sysctl.d/, and run `sysctl --system` 
multiple times, possibly concurrently.  This week I found the system in 
a weird state, and one `sysctl --system` process that couldn't be 
killed, with a "list_del corruption" in the kernel log pointing to 
`input_unregister_handle`.

This was on a debian stable 6.12.115+deb13.1-amd64, I think the most 
interesting part of the logs is:

---
list_del corruption. prev->next should be ffff8d6e4324d040, but was 
ffffffff915cf038. (prev=ffffffff915cf038)
------------[ cut here ]------------
kernel BUG at lib/list_debug.c:62!
Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
CPU: 0 UID: 0 PID: 1454 Comm: sysctl Not tainted 6.12.115+deb13.1-amd64 
#1  Debian 6.12.115-2
...
Call Trace:
  <TASK>
  input_unregister_handle+0x1b/0x90
  sysrq_disconnect+0x33/0x40
  input_unregister_handler+0x3d/0xc0
  sysrq_toggle_support+0x48/0x70
---

I can easily reproduce it by running two copies of this script (even on 
a single CPU):

---
#!/usr/bin/env python3

with open("/proc/sys/kernel/sysrq", "w") as sysctl:
     while True:
         sysctl.write("0\n")
         sysctl.seek(0)
         sysctl.write("1\n")
         sysctl.seek(0)
---

I think I can see how `sysrq_toggle_support` has a race condition; I 
don't understand how that (in my original scenario that only tried 
setting it to 0) could end in problems, even if it ran 
sysrq_unregister_handler() multiple times without 
sysrq_register_handler() between (the code looks like it should leave a 
clean input_handler, and not do anything bad when passed a clean 
input_handler).

I see how it could run sysrq_register_handler() multiple times without 
sysrq_unregister_handler() between when using my python script, and how 
that is going to be a problem - and when I tried it with vanilla 
7.4.0~rc1-g05441d697f7f (on a different system), I actually get a 
"list_add double add":

---
list_add double add: new=ffffffffa9bea108, prev=ffffffffa9bea108, 
next=ffffffffa9c09b20.
------------[ cut here ]------------
kernel BUG at lib/list_debug.c:35!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 1392 Comm: python3 Not tainted 7.4.0-rc1+ #3 
PREEMPT(lazy)
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop 
Reference Platform, BIOS 6.00 11/12/2020
RIP: 0010:__list_add_valid_or_report.cold+0x15/0x33
Code: 36 fe ff e9 de 84 7d 00 4c 89 e6 bf 0f 00 00 00 e8 2e ff ff ff 48 
89 f2 48 89 d9 48 89 fe 48 c7 c7 d0 d4 7b a9 e8 69 36 fe ff <0f> 0b 48 
c7 c7 f8 d3 7b a9 e8 5b 36 fe ff 0f 0b 48 c7 c7 d0 d3 7b
RSP: 0018:ffffd485c2f97a20 EFLAGS: 00010246
RAX: 0000000000000058 RBX: ffffffffa9c09b20 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffffa9a5b680
RBP: ffffffffa9bea0a0 R08: 3fffffffffff7fff R09: 0000000000000577
R10: ffffd485c2f97840 R11: 0000000000000002 R12: ffffffffaa78ad50
R13: ffffffffa9bea108 R14: ffffffffa92f4e00 R15: 0000000000000002
FS:  00007f78eb9b9780(0000) GS:ffff8ce9cf828000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000007132b0 CR3: 00000001041a2006 CR4: 0000000000770ef0
PKRU: 55555554
Call Trace:
  <TASK>
  input_register_handler+0x89/0x130
  sysrq_toggle_support+0x5f/0x70
  sysrq_sysctl_handler+0xb0/0xc0
  ? __pfx_sysrq_sysctl_handler+0x10/0x10
...
---

It seems `input_register_handler` is usually called from module __init, 
with freshly allocated data, or with a mutex being held - afaict only 
sysrq is potentially calling it with an already registered handler.

`sysrq_toggle_support` probably should use a mutex too.


Summary:

* my python script (run two instances) triggers a kernel BUG on both
   debian stable 6.12.115+deb13.1 and vanilla 7.4.0~rc1-g05441d697f7f
   within seconds
* 6.12.155 shows a "list_del corruption" in `input_unregister_handle`,
   which I can't explain (especially not in the original environment)
* 7.4.0~rc1-g05441d697f7f shows "list_add double add" in
   `input_register_handler`, which is kinda expected
* sometimes a python script segfaults, sometimes it ends up unkillable
* I don't think this is accessible from unprivileged userns / containers

cheers,
Stefan

Re: [BUG] kernel.sysrq sysctl race condition(s) / list corruptions

[Greg Kroah-Hartman]

On Thu, Nov 05, 2026 at 03:29:54PM +0100, Stefan Bühler wrote:
> Hi,
> 
> we have `kernel.sysrq = 0` in /etc/sysctl.d/, and run `sysctl --system`
> multiple times, possibly concurrently.  This week I found the system in a
> weird state, and one `sysctl --system` process that couldn't be killed, with
> a "list_del corruption" in the kernel log pointing to
> `input_unregister_handle`.
> 
> This was on a debian stable 6.12.115+deb13.1-amd64, I think the most
> interesting part of the logs is:
> 
> ---
> list_del corruption. prev->next should be ffff8d6e4324d040, but was
> ffffffff915cf038. (prev=ffffffff915cf038)
> ------------[ cut here ]------------
> kernel BUG at lib/list_debug.c:62!
> Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
> CPU: 0 UID: 0 PID: 1454 Comm: sysctl Not tainted 6.12.115+deb13.1-amd64 #1
> Debian 6.12.115-2
> ...
> Call Trace:
>  <TASK>
>  input_unregister_handle+0x1b/0x90
>  sysrq_disconnect+0x33/0x40
>  input_unregister_handler+0x3d/0xc0
>  sysrq_toggle_support+0x48/0x70
> ---
> 
> I can easily reproduce it by running two copies of this script (even on a
> single CPU):
> 
> ---
> #!/usr/bin/env python3
> 
> with open("/proc/sys/kernel/sysrq", "w") as sysctl:
>     while True:
>         sysctl.write("0\n")
>         sysctl.seek(0)
>         sysctl.write("1\n")
>         sysctl.seek(0)
> ---
> 
> I think I can see how `sysrq_toggle_support` has a race condition; I don't
> understand how that (in my original scenario that only tried setting it to
> 0) could end in problems, even if it ran sysrq_unregister_handler() multiple
> times without sysrq_register_handler() between (the code looks like it
> should leave a clean input_handler, and not do anything bad when passed a
> clean input_handler).
> 
> I see how it could run sysrq_register_handler() multiple times without
> sysrq_unregister_handler() between when using my python script, and how that
> is going to be a problem - and when I tried it with vanilla
> 7.4.0~rc1-g05441d697f7f (on a different system), I actually get a "list_add
> double add":
> 
> ---
> list_add double add: new=ffffffffa9bea108, prev=ffffffffa9bea108,
> next=ffffffffa9c09b20.
> ------------[ cut here ]------------
> kernel BUG at lib/list_debug.c:35!
> Oops: invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 0 UID: 0 PID: 1392 Comm: python3 Not tainted 7.4.0-rc1+ #3
> PREEMPT(lazy)
> Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference
> Platform, BIOS 6.00 11/12/2020
> RIP: 0010:__list_add_valid_or_report.cold+0x15/0x33
> Code: 36 fe ff e9 de 84 7d 00 4c 89 e6 bf 0f 00 00 00 e8 2e ff ff ff 48 89
> f2 48 89 d9 48 89 fe 48 c7 c7 d0 d4 7b a9 e8 69 36 fe ff <0f> 0b 48 c7 c7 f8
> d3 7b a9 e8 5b 36 fe ff 0f 0b 48 c7 c7 d0 d3 7b
> RSP: 0018:ffffd485c2f97a20 EFLAGS: 00010246
> RAX: 0000000000000058 RBX: ffffffffa9c09b20 RCX: 0000000000000000
> RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffffa9a5b680
> RBP: ffffffffa9bea0a0 R08: 3fffffffffff7fff R09: 0000000000000577
> R10: ffffd485c2f97840 R11: 0000000000000002 R12: ffffffffaa78ad50
> R13: ffffffffa9bea108 R14: ffffffffa92f4e00 R15: 0000000000000002
> FS:  00007f78eb9b9780(0000) GS:ffff8ce9cf828000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00000000007132b0 CR3: 00000001041a2006 CR4: 0000000000770ef0
> PKRU: 55555554
> Call Trace:
>  <TASK>
>  input_register_handler+0x89/0x130
>  sysrq_toggle_support+0x5f/0x70
>  sysrq_sysctl_handler+0xb0/0xc0
>  ? __pfx_sysrq_sysctl_handler+0x10/0x10
> ...
> ---
> 
> It seems `input_register_handler` is usually called from module __init, with
> freshly allocated data, or with a mutex being held - afaict only sysrq is
> potentially calling it with an already registered handler.
> 
> `sysrq_toggle_support` probably should use a mutex too.
> 
> 
> Summary:
> 
> * my python script (run two instances) triggers a kernel BUG on both
>   debian stable 6.12.115+deb13.1 and vanilla 7.4.0~rc1-g05441d697f7f
>   within seconds
> * 6.12.155 shows a "list_del corruption" in `input_unregister_handle`,
>   which I can't explain (especially not in the original environment)
> * 7.4.0~rc1-g05441d697f7f shows "list_add double add" in
>   `input_register_handler`, which is kinda expected
> * sometimes a python script segfaults, sometimes it ends up unkillable
> * I don't think this is accessible from unprivileged userns / containers

Can you try this patch out:
	https://lore.kernel.org/r/20261030123640.263164-1-zhanxusheng1024@gmail.com
as it seems to describe the issue you are seeing here.

Or am I mistaken?

thanks,

greg k-h