* [syzbot] [pm?] possible deadlock in lock_system_sleep
@ 2025-02-20 15:16 syzbot
2025-02-21 1:45 ` [syzbot] " syzbot
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: syzbot @ 2025-02-20 15:16 UTC (permalink / raw)
To: len.brown, linux-kernel, linux-pm, pavel, rafael, syzkaller-bugs
Hello,
syzbot found the following issue on:
HEAD commit: 6537cfb395f3 Merge tag 'sound-6.14-rc4' of git://git.kerne..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1746afdf980000
kernel config: https://syzkaller.appspot.com/x/.config?x=61cbf5ac8a063ad4
dashboard link: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13e71498580000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1432cba4580000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/ae2568ec9e34/disk-6537cfb3.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/19be7179f649/vmlinux-6537cfb3.xz
kernel image: https://storage.googleapis.com/syzbot-assets/bd8a168d2027/bzImage-6537cfb3.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
======================================================
WARNING: possible circular locking dependency detected
6.14.0-rc3-syzkaller-00060-g6537cfb395f3 #0 Not tainted
------------------------------------------------------
syz-executor895/5833 is trying to acquire lock:
ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
but task is already holding lock:
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (param_lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
rate_control_alloc net/mac80211/rate.c:266 [inline]
ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
do_one_initcall+0x128/0x700 init/main.c:1257
do_initcall_level init/main.c:1319 [inline]
do_initcalls init/main.c:1335 [inline]
do_basic_setup init/main.c:1354 [inline]
kernel_init_freeable+0x5c7/0x900 init/main.c:1568
kernel_init+0x1c/0x2b0 init/main.c:1457
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
-> #2 (rtnl_mutex){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
notifier_call_chain_robust kernel/notifier.c:120 [inline]
blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #0 (system_transition_mutex){+.+.}-{4:4}:
check_prev_add kernel/locking/lockdep.c:3163 [inline]
check_prevs_add kernel/locking/lockdep.c:3282 [inline]
validate_chain kernel/locking/lockdep.c:3906 [inline]
__lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
param_attr_store+0x18f/0x300 kernel/params.c:588
module_attr_store+0x55/0x80 kernel/params.c:924
sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
new_sync_write fs/read_write.c:586 [inline]
vfs_write+0x5ae/0x1150 fs/read_write.c:679
ksys_write+0x12b/0x250 fs/read_write.c:731
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
other info that might help us debug this:
Chain exists of:
system_transition_mutex --> rtnl_mutex --> param_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(param_lock);
lock(rtnl_mutex);
lock(param_lock);
lock(system_transition_mutex);
*** DEADLOCK ***
4 locks held by syz-executor895/5833:
#0: ffff8880335ec420 (sb_writers#8){.+.+}-{0:0}, at: ksys_write+0x12b/0x250 fs/read_write.c:731
#1: ffff88803398c488 (&of->mutex){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x27b/0x500 fs/kernfs/file.c:325
#2: ffff8880307af788 (kn->active#4){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x500 fs/kernfs/file.c:326
#3: ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
#3: ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
stack backtrace:
CPU: 0 UID: 0 PID: 5833 Comm: syz-executor895 Not tainted 6.14.0-rc3-syzkaller-00060-g6537cfb395f3 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:120
print_circular_bug+0x490/0x760 kernel/locking/lockdep.c:2076
check_noncircular+0x31a/0x400 kernel/locking/lockdep.c:2208
check_prev_add kernel/locking/lockdep.c:3163 [inline]
check_prevs_add kernel/locking/lockdep.c:3282 [inline]
validate_chain kernel/locking/lockdep.c:3906 [inline]
__lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
param_attr_store+0x18f/0x300 kernel/params.c:588
module_attr_store+0x55/0x80 kernel/params.c:924
sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
new_sync_write fs/read_write.c:586 [inline]
vfs_write+0x5ae/0x1150 fs/read_write.c:679
ksys_write+0x12b/0x250 fs/read_write.c:731
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f5f71a94a79
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 c1 17 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fffb2ae1de8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f5f71a94a79
RDX: 00000000000040db RSI: 0000400000005900 RDI: 0000000000000004
RBP: 00007f5f71b075f0 R08: 0000000000000006 R09: 0000000000000006
R10: 0000000000000006 R11: 0000000000000246 R12: 0000000000000001
R13: 431bde82d7b634db R14: 0000000000000001 R15: 0000000000000001
</TASK>
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [syzbot] Re: [syzbot] [pm?] possible deadlock in lock_system_sleep
2025-02-20 15:16 [syzbot] [pm?] possible deadlock in lock_system_sleep syzbot
@ 2025-02-21 1:45 ` syzbot
2025-02-21 2:57 ` syzbot
` (2 subsequent siblings)
3 siblings, 0 replies; 14+ messages in thread
From: syzbot @ 2025-02-21 1:45 UTC (permalink / raw)
To: linux-kernel
For archival purposes, forwarding an incoming command email to
linux-kernel@vger.kernel.org.
***
Subject: Re: [syzbot] [pm?] possible deadlock in lock_system_sleep
Author: lizhi.xu@windriver.com
#syz test
diff --git a/kernel/params.c b/kernel/params.c
index 0074d29c9b80..d19881fbb2ec 100644
--- a/kernel/params.c
+++ b/kernel/params.c
@@ -583,7 +583,9 @@ static ssize_t param_attr_store(const struct module_attribute *mattr,
if (!attribute->param->ops->set)
return -EPERM;
- kernel_param_lock(mk->mod);
+ if (!kernel_param_trylock(mk->mod))
+ return -EPERM;
+
if (param_check_unsafe(attribute->param))
err = attribute->param->ops->set(buf, attribute->param);
else
@@ -607,6 +609,11 @@ void kernel_param_lock(struct module *mod)
mutex_lock(KPARAM_MUTEX(mod));
}
+int kernel_param_trylock(struct module *mod)
+{
+ return mutex_trylock(KPARAM_MUTEX(mod));
+}
+
void kernel_param_unlock(struct module *mod)
{
mutex_unlock(KPARAM_MUTEX(mod));
diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
index bfb85fd13e1f..c7b1da7b9563 100644
--- a/include/linux/moduleparam.h
+++ b/include/linux/moduleparam.h
@@ -306,11 +306,15 @@ struct kparam_array
#ifdef CONFIG_SYSFS
extern void kernel_param_lock(struct module *mod);
+extern void kernel_param_trylock(struct module *mod);
extern void kernel_param_unlock(struct module *mod);
#else
static inline void kernel_param_lock(struct module *mod)
{
}
+static inline void kernel_param_trylock(struct module *mod)
+{
+}
static inline void kernel_param_unlock(struct module *mod)
{
}
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [syzbot] Re: [syzbot] [pm?] possible deadlock in lock_system_sleep
2025-02-20 15:16 [syzbot] [pm?] possible deadlock in lock_system_sleep syzbot
2025-02-21 1:45 ` [syzbot] " syzbot
@ 2025-02-21 2:57 ` syzbot
2025-02-21 6:27 ` syzbot
2025-02-21 10:10 ` [PATCH] module: replace the mutex lock acquisition method Lizhi Xu
3 siblings, 0 replies; 14+ messages in thread
From: syzbot @ 2025-02-21 2:57 UTC (permalink / raw)
To: linux-kernel
For archival purposes, forwarding an incoming command email to
linux-kernel@vger.kernel.org.
***
Subject: Re: [syzbot] [pm?] possible deadlock in lock_system_sleep
Author: lizhi.xu@windriver.com
#syz test
diff --git a/kernel/params.c b/kernel/params.c
index 0074d29c9b80..d19881fbb2ec 100644
--- a/kernel/params.c
+++ b/kernel/params.c
@@ -583,7 +583,9 @@ static ssize_t param_attr_store(const struct module_attribute *mattr,
if (!attribute->param->ops->set)
return -EPERM;
- kernel_param_lock(mk->mod);
+ if (!kernel_param_trylock(mk->mod))
+ return -EPERM;
+
if (param_check_unsafe(attribute->param))
err = attribute->param->ops->set(buf, attribute->param);
else
@@ -607,6 +609,11 @@ void kernel_param_lock(struct module *mod)
mutex_lock(KPARAM_MUTEX(mod));
}
+int kernel_param_trylock(struct module *mod)
+{
+ return mutex_trylock(KPARAM_MUTEX(mod));
+}
+
void kernel_param_unlock(struct module *mod)
{
mutex_unlock(KPARAM_MUTEX(mod));
diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
index bfb85fd13e1f..cbcbfd8db721 100644
--- a/include/linux/moduleparam.h
+++ b/include/linux/moduleparam.h
@@ -306,11 +306,15 @@ struct kparam_array
#ifdef CONFIG_SYSFS
extern void kernel_param_lock(struct module *mod);
+extern int kernel_param_trylock(struct module *mod);
extern void kernel_param_unlock(struct module *mod);
#else
static inline void kernel_param_lock(struct module *mod)
{
}
+static inline int kernel_param_trylock(struct module *mod)
+{
+}
static inline void kernel_param_unlock(struct module *mod)
{
}
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [syzbot] Re: [syzbot] [pm?] possible deadlock in lock_system_sleep
2025-02-20 15:16 [syzbot] [pm?] possible deadlock in lock_system_sleep syzbot
2025-02-21 1:45 ` [syzbot] " syzbot
2025-02-21 2:57 ` syzbot
@ 2025-02-21 6:27 ` syzbot
2025-02-21 10:10 ` [PATCH] module: replace the mutex lock acquisition method Lizhi Xu
3 siblings, 0 replies; 14+ messages in thread
From: syzbot @ 2025-02-21 6:27 UTC (permalink / raw)
To: linux-kernel
For archival purposes, forwarding an incoming command email to
linux-kernel@vger.kernel.org.
***
Subject: Re: [syzbot] [pm?] possible deadlock in lock_system_sleep
Author: lizhi.xu@windriver.com
#syz test
diff --git a/kernel/params.c b/kernel/params.c
index 0074d29c9b80..d19881fbb2ec 100644
--- a/kernel/params.c
+++ b/kernel/params.c
@@ -583,7 +583,9 @@ static ssize_t param_attr_store(const struct module_attribute *mattr,
if (!attribute->param->ops->set)
return -EPERM;
- kernel_param_lock(mk->mod);
+ if (!kernel_param_trylock(mk->mod))
+ return -EPERM;
+
if (param_check_unsafe(attribute->param))
err = attribute->param->ops->set(buf, attribute->param);
else
@@ -607,6 +609,11 @@ void kernel_param_lock(struct module *mod)
mutex_lock(KPARAM_MUTEX(mod));
}
+int kernel_param_trylock(struct module *mod)
+{
+ return mutex_trylock(KPARAM_MUTEX(mod));
+}
+
void kernel_param_unlock(struct module *mod)
{
mutex_unlock(KPARAM_MUTEX(mod));
diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
index bfb85fd13e1f..cbcbfd8db721 100644
--- a/include/linux/moduleparam.h
+++ b/include/linux/moduleparam.h
@@ -306,11 +306,15 @@ struct kparam_array
#ifdef CONFIG_SYSFS
extern void kernel_param_lock(struct module *mod);
+extern int kernel_param_trylock(struct module *mod);
extern void kernel_param_unlock(struct module *mod);
#else
static inline void kernel_param_lock(struct module *mod)
{
}
+static inline int kernel_param_trylock(struct module *mod)
+{
+}
static inline void kernel_param_unlock(struct module *mod)
{
}
diff --git a/net/mac80211/rate.c b/net/mac80211/rate.c
index 0d056db9f81e..aecf7ff51cd9 100644
--- a/net/mac80211/rate.c
+++ b/net/mac80211/rate.c
@@ -217,7 +217,9 @@ ieee80211_rate_control_ops_get(const char *name)
const struct rate_control_ops *ops;
const char *alg_name;
- kernel_param_lock(THIS_MODULE);
+ if (!kernel_param_trylock(THIS_MODULE))
+ return NULL;
+
if (!name)
alg_name = ieee80211_default_rc_algo;
else
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH] module: replace the mutex lock acquisition method
2025-02-20 15:16 [syzbot] [pm?] possible deadlock in lock_system_sleep syzbot
` (2 preceding siblings ...)
2025-02-21 6:27 ` syzbot
@ 2025-02-21 10:10 ` Lizhi Xu
2025-02-21 11:41 ` Rafael J. Wysocki
3 siblings, 1 reply; 14+ messages in thread
From: Lizhi Xu @ 2025-02-21 10:10 UTC (permalink / raw)
To: syzbot+ace60642828c074eb913
Cc: len.brown, linux-kernel, linux-pm, pavel, rafael, syzkaller-bugs
syzbot reported a deadlock in lock_system_sleep. [1]
The write operation to "/sys/module/hibernate/parameters/compressor"
conflicts with the registration of ieee80211 device, resulting in a deadlock
in the lock param_lock.
Since the conflict cannot be avoided, the way to obtain param_lock is changed
to trylock to avoid deadlock.
[1]
syz-executor895/5833 is trying to acquire lock:
ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
but task is already holding lock:
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (param_lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
rate_control_alloc net/mac80211/rate.c:266 [inline]
ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
do_one_initcall+0x128/0x700 init/main.c:1257
do_initcall_level init/main.c:1319 [inline]
do_initcalls init/main.c:1335 [inline]
do_basic_setup init/main.c:1354 [inline]
kernel_init_freeable+0x5c7/0x900 init/main.c:1568
kernel_init+0x1c/0x2b0 init/main.c:1457
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
-> #2 (rtnl_mutex){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
notifier_call_chain_robust kernel/notifier.c:120 [inline]
blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #0 (system_transition_mutex){+.+.}-{4:4}:
check_prev_add kernel/locking/lockdep.c:3163 [inline]
check_prevs_add kernel/locking/lockdep.c:3282 [inline]
validate_chain kernel/locking/lockdep.c:3906 [inline]
__lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
param_attr_store+0x18f/0x300 kernel/params.c:588
module_attr_store+0x55/0x80 kernel/params.c:924
sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
new_sync_write fs/read_write.c:586 [inline]
vfs_write+0x5ae/0x1150 fs/read_write.c:679
ksys_write+0x12b/0x250 fs/read_write.c:731
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
other info that might help us debug this:
Chain exists of:
system_transition_mutex --> rtnl_mutex --> param_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(param_lock);
lock(rtnl_mutex);
lock(param_lock);
lock(system_transition_mutex);
*** DEADLOCK ***
Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
Tested-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
---
include/linux/moduleparam.h | 4 ++++
kernel/params.c | 9 ++++++++-
net/mac80211/rate.c | 4 +++-
3 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
index bfb85fd13e1f..cbcbfd8db721 100644
--- a/include/linux/moduleparam.h
+++ b/include/linux/moduleparam.h
@@ -306,11 +306,15 @@ struct kparam_array
#ifdef CONFIG_SYSFS
extern void kernel_param_lock(struct module *mod);
+extern int kernel_param_trylock(struct module *mod);
extern void kernel_param_unlock(struct module *mod);
#else
static inline void kernel_param_lock(struct module *mod)
{
}
+static inline int kernel_param_trylock(struct module *mod)
+{
+}
static inline void kernel_param_unlock(struct module *mod)
{
}
diff --git a/kernel/params.c b/kernel/params.c
index 0074d29c9b80..d19881fbb2ec 100644
--- a/kernel/params.c
+++ b/kernel/params.c
@@ -583,7 +583,9 @@ static ssize_t param_attr_store(const struct module_attribute *mattr,
if (!attribute->param->ops->set)
return -EPERM;
- kernel_param_lock(mk->mod);
+ if (!kernel_param_trylock(mk->mod))
+ return -EPERM;
+
if (param_check_unsafe(attribute->param))
err = attribute->param->ops->set(buf, attribute->param);
else
@@ -607,6 +609,11 @@ void kernel_param_lock(struct module *mod)
mutex_lock(KPARAM_MUTEX(mod));
}
+int kernel_param_trylock(struct module *mod)
+{
+ return mutex_trylock(KPARAM_MUTEX(mod));
+}
+
void kernel_param_unlock(struct module *mod)
{
mutex_unlock(KPARAM_MUTEX(mod));
diff --git a/net/mac80211/rate.c b/net/mac80211/rate.c
index 0d056db9f81e..aecf7ff51cd9 100644
--- a/net/mac80211/rate.c
+++ b/net/mac80211/rate.c
@@ -217,7 +217,9 @@ ieee80211_rate_control_ops_get(const char *name)
const struct rate_control_ops *ops;
const char *alg_name;
- kernel_param_lock(THIS_MODULE);
+ if (!kernel_param_trylock(THIS_MODULE))
+ return NULL;
+
if (!name)
alg_name = ieee80211_default_rc_algo;
else
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH] module: replace the mutex lock acquisition method
2025-02-21 10:10 ` [PATCH] module: replace the mutex lock acquisition method Lizhi Xu
@ 2025-02-21 11:41 ` Rafael J. Wysocki
2025-02-21 12:32 ` Lizhi Xu
2025-02-21 12:35 ` [PATCH V2] " Lizhi Xu
0 siblings, 2 replies; 14+ messages in thread
From: Rafael J. Wysocki @ 2025-02-21 11:41 UTC (permalink / raw)
To: Lizhi Xu
Cc: syzbot+ace60642828c074eb913, len.brown, linux-kernel, linux-pm,
pavel, rafael, syzkaller-bugs
On Fri, Feb 21, 2025 at 11:10 AM Lizhi Xu <lizhi.xu@windriver.com> wrote:
>
> syzbot reported a deadlock in lock_system_sleep. [1]
>
> The write operation to "/sys/module/hibernate/parameters/compressor"
> conflicts with the registration of ieee80211 device, resulting in a deadlock
> in the lock param_lock.
>
> Since the conflict cannot be avoided, the way to obtain param_lock is changed
> to trylock to avoid deadlock.
>
> [1]
> syz-executor895/5833 is trying to acquire lock:
> ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
>
> but task is already holding lock:
> ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
> ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
>
> which lock already depends on the new lock.
>
>
> the existing dependency chain (in reverse order) is:
>
> -> #3 (param_lock){+.+.}-{4:4}:
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
> rate_control_alloc net/mac80211/rate.c:266 [inline]
> ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
> ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
> mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
> init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
> do_one_initcall+0x128/0x700 init/main.c:1257
> do_initcall_level init/main.c:1319 [inline]
> do_initcalls init/main.c:1335 [inline]
> do_basic_setup init/main.c:1354 [inline]
> kernel_init_freeable+0x5c7/0x900 init/main.c:1568
> kernel_init+0x1c/0x2b0 init/main.c:1457
> ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
>
> -> #2 (rtnl_mutex){+.+.}-{4:4}:
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
> wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
> notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
> notifier_call_chain_robust kernel/notifier.c:120 [inline]
> blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
> blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
> pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> misc_open+0x35a/0x420 drivers/char/misc.c:179
> chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> do_dentry_open+0x735/0x1c40 fs/open.c:956
> vfs_open+0x82/0x3f0 fs/open.c:1086
> do_open fs/namei.c:3830 [inline]
> path_openat+0x1e88/0x2d80 fs/namei.c:3989
> do_filp_open+0x20c/0x470 fs/namei.c:4016
> do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> do_sys_open fs/open.c:1443 [inline]
> __do_sys_openat fs/open.c:1459 [inline]
> __se_sys_openat fs/open.c:1454 [inline]
> __x64_sys_openat+0x175/0x210 fs/open.c:1454
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> -> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
> down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
> blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
> blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
> pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> misc_open+0x35a/0x420 drivers/char/misc.c:179
> chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> do_dentry_open+0x735/0x1c40 fs/open.c:956
> vfs_open+0x82/0x3f0 fs/open.c:1086
> do_open fs/namei.c:3830 [inline]
> path_openat+0x1e88/0x2d80 fs/namei.c:3989
> do_filp_open+0x20c/0x470 fs/namei.c:4016
> do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> do_sys_open fs/open.c:1443 [inline]
> __do_sys_openat fs/open.c:1459 [inline]
> __se_sys_openat fs/open.c:1454 [inline]
> __x64_sys_openat+0x175/0x210 fs/open.c:1454
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> -> #0 (system_transition_mutex){+.+.}-{4:4}:
> check_prev_add kernel/locking/lockdep.c:3163 [inline]
> check_prevs_add kernel/locking/lockdep.c:3282 [inline]
> validate_chain kernel/locking/lockdep.c:3906 [inline]
> __lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
> lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
> hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
> param_attr_store+0x18f/0x300 kernel/params.c:588
> module_attr_store+0x55/0x80 kernel/params.c:924
> sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
> kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
> new_sync_write fs/read_write.c:586 [inline]
> vfs_write+0x5ae/0x1150 fs/read_write.c:679
> ksys_write+0x12b/0x250 fs/read_write.c:731
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> other info that might help us debug this:
>
> Chain exists of:
> system_transition_mutex --> rtnl_mutex --> param_lock
>
> Possible unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(param_lock);
> lock(rtnl_mutex);
> lock(param_lock);
> lock(system_transition_mutex);
>
> *** DEADLOCK ***
>
> Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
> Tested-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
> Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
> ---
> include/linux/moduleparam.h | 4 ++++
> kernel/params.c | 9 ++++++++-
> net/mac80211/rate.c | 4 +++-
> 3 files changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
> index bfb85fd13e1f..cbcbfd8db721 100644
> --- a/include/linux/moduleparam.h
> +++ b/include/linux/moduleparam.h
> @@ -306,11 +306,15 @@ struct kparam_array
>
> #ifdef CONFIG_SYSFS
> extern void kernel_param_lock(struct module *mod);
> +extern int kernel_param_trylock(struct module *mod);
> extern void kernel_param_unlock(struct module *mod);
> #else
> static inline void kernel_param_lock(struct module *mod)
> {
> }
> +static inline int kernel_param_trylock(struct module *mod)
> +{
> +}
> static inline void kernel_param_unlock(struct module *mod)
> {
> }
> diff --git a/kernel/params.c b/kernel/params.c
> index 0074d29c9b80..d19881fbb2ec 100644
> --- a/kernel/params.c
> +++ b/kernel/params.c
> @@ -583,7 +583,9 @@ static ssize_t param_attr_store(const struct module_attribute *mattr,
> if (!attribute->param->ops->set)
> return -EPERM;
>
> - kernel_param_lock(mk->mod);
> + if (!kernel_param_trylock(mk->mod))
> + return -EPERM;
-EAGAIN would be better I think?
> +
> if (param_check_unsafe(attribute->param))
> err = attribute->param->ops->set(buf, attribute->param);
> else
> @@ -607,6 +609,11 @@ void kernel_param_lock(struct module *mod)
> mutex_lock(KPARAM_MUTEX(mod));
> }
>
> +int kernel_param_trylock(struct module *mod)
> +{
> + return mutex_trylock(KPARAM_MUTEX(mod));
> +}
> +
> void kernel_param_unlock(struct module *mod)
> {
> mutex_unlock(KPARAM_MUTEX(mod));
> diff --git a/net/mac80211/rate.c b/net/mac80211/rate.c
> index 0d056db9f81e..aecf7ff51cd9 100644
> --- a/net/mac80211/rate.c
> +++ b/net/mac80211/rate.c
> @@ -217,7 +217,9 @@ ieee80211_rate_control_ops_get(const char *name)
> const struct rate_control_ops *ops;
> const char *alg_name;
>
> - kernel_param_lock(THIS_MODULE);
> + if (!kernel_param_trylock(THIS_MODULE))
> + return NULL;
> +
> if (!name)
> alg_name = ieee80211_default_rc_algo;
> else
> --
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] module: replace the mutex lock acquisition method
2025-02-21 11:41 ` Rafael J. Wysocki
@ 2025-02-21 12:32 ` Lizhi Xu
2025-02-21 12:35 ` [PATCH V2] " Lizhi Xu
1 sibling, 0 replies; 14+ messages in thread
From: Lizhi Xu @ 2025-02-21 12:32 UTC (permalink / raw)
To: rafael
Cc: len.brown, linux-kernel, linux-pm, lizhi.xu, pavel,
syzbot+ace60642828c074eb913, syzkaller-bugs
On Fri, 21 Feb 2025 12:41:32 +0100, Rafael J. Wysocki" <rafael@kernel.org> wrote:
> > syzbot reported a deadlock in lock_system_sleep. [1]
> >
> > The write operation to "/sys/module/hibernate/parameters/compressor"
> > conflicts with the registration of ieee80211 device, resulting in a deadlock
> > in the lock param_lock.
> >
> > Since the conflict cannot be avoided, the way to obtain param_lock is changed
> > to trylock to avoid deadlock.
> >
> > [1]
> > syz-executor895/5833 is trying to acquire lock:
> > ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
> >
> > but task is already holding lock:
> > ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
> > ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
> >
> > which lock already depends on the new lock.
> >
> >
> > the existing dependency chain (in reverse order) is:
> >
> > -> #3 (param_lock){+.+.}-{4:4}:
> > __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> > __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> > ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
> > rate_control_alloc net/mac80211/rate.c:266 [inline]
> > ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
> > ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
> > mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
> > init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
> > do_one_initcall+0x128/0x700 init/main.c:1257
> > do_initcall_level init/main.c:1319 [inline]
> > do_initcalls init/main.c:1335 [inline]
> > do_basic_setup init/main.c:1354 [inline]
> > kernel_init_freeable+0x5c7/0x900 init/main.c:1568
> > kernel_init+0x1c/0x2b0 init/main.c:1457
> > ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
> > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
> >
> > -> #2 (rtnl_mutex){+.+.}-{4:4}:
> > __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> > __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> > wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
> > wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
> > notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
> > notifier_call_chain_robust kernel/notifier.c:120 [inline]
> > blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
> > blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
> > pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> > snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> > misc_open+0x35a/0x420 drivers/char/misc.c:179
> > chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> > do_dentry_open+0x735/0x1c40 fs/open.c:956
> > vfs_open+0x82/0x3f0 fs/open.c:1086
> > do_open fs/namei.c:3830 [inline]
> > path_openat+0x1e88/0x2d80 fs/namei.c:3989
> > do_filp_open+0x20c/0x470 fs/namei.c:4016
> > do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> > do_sys_open fs/open.c:1443 [inline]
> > __do_sys_openat fs/open.c:1459 [inline]
> > __se_sys_openat fs/open.c:1454 [inline]
> > __x64_sys_openat+0x175/0x210 fs/open.c:1454
> > do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> > do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> > entry_SYSCALL_64_after_hwframe+0x77/0x7f
> >
> > -> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
> > down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
> > blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
> > blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
> > pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> > snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> > misc_open+0x35a/0x420 drivers/char/misc.c:179
> > chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> > do_dentry_open+0x735/0x1c40 fs/open.c:956
> > vfs_open+0x82/0x3f0 fs/open.c:1086
> > do_open fs/namei.c:3830 [inline]
> > path_openat+0x1e88/0x2d80 fs/namei.c:3989
> > do_filp_open+0x20c/0x470 fs/namei.c:4016
> > do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> > do_sys_open fs/open.c:1443 [inline]
> > __do_sys_openat fs/open.c:1459 [inline]
> > __se_sys_openat fs/open.c:1454 [inline]
> > __x64_sys_openat+0x175/0x210 fs/open.c:1454
> > do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> > do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> > entry_SYSCALL_64_after_hwframe+0x77/0x7f
> >
> > -> #0 (system_transition_mutex){+.+.}-{4:4}:
> > check_prev_add kernel/locking/lockdep.c:3163 [inline]
> > check_prevs_add kernel/locking/lockdep.c:3282 [inline]
> > validate_chain kernel/locking/lockdep.c:3906 [inline]
> > __lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
> > lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
> > __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> > __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> > lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
> > hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
> > param_attr_store+0x18f/0x300 kernel/params.c:588
> > module_attr_store+0x55/0x80 kernel/params.c:924
> > sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
> > kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
> > new_sync_write fs/read_write.c:586 [inline]
> > vfs_write+0x5ae/0x1150 fs/read_write.c:679
> > ksys_write+0x12b/0x250 fs/read_write.c:731
> > do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> > do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> > entry_SYSCALL_64_after_hwframe+0x77/0x7f
> >
> > other info that might help us debug this:
> >
> > Chain exists of:
> > system_transition_mutex --> rtnl_mutex --> param_lock
> >
> > Possible unsafe locking scenario:
> >
> > CPU0 CPU1
> > ---- ----
> > lock(param_lock);
> > lock(rtnl_mutex);
> > lock(param_lock);
> > lock(system_transition_mutex);
> >
> > *** DEADLOCK ***
> >
> > Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
> > Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
> > Tested-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
> > Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
> > ---
> > include/linux/moduleparam.h | 4 ++++
> > kernel/params.c | 9 ++++++++-
> > net/mac80211/rate.c | 4 +++-
> > 3 files changed, 15 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
> > index bfb85fd13e1f..cbcbfd8db721 100644
> > --- a/include/linux/moduleparam.h
> > +++ b/include/linux/moduleparam.h
> > @@ -306,11 +306,15 @@ struct kparam_array
> >
> > #ifdef CONFIG_SYSFS
> > extern void kernel_param_lock(struct module *mod);
> > +extern int kernel_param_trylock(struct module *mod);
> > extern void kernel_param_unlock(struct module *mod);
> > #else
> > static inline void kernel_param_lock(struct module *mod)
> > {
> > }
> > +static inline int kernel_param_trylock(struct module *mod)
> > +{
> > +}
> > static inline void kernel_param_unlock(struct module *mod)
> > {
> > }
> > diff --git a/kernel/params.c b/kernel/params.c
> > index 0074d29c9b80..d19881fbb2ec 100644
> > --- a/kernel/params.c
> > +++ b/kernel/params.c
> > @@ -583,7 +583,9 @@ static ssize_t param_attr_store(const struct module_attribute *mattr,
> > if (!attribute->param->ops->set)
> > return -EPERM;
> >
> > - kernel_param_lock(mk->mod);
> > + if (!kernel_param_trylock(mk->mod))
> > + return -EPERM;
>
> -EAGAIN would be better I think?
Agree, -EAGAIN is more clear.
I will send a V2 patch to use -EAGAIN.
>
> > +
> > if (param_check_unsafe(attribute->param))
> > err = attribute->param->ops->set(buf, attribute->param);
> > else
> > @@ -607,6 +609,11 @@ void kernel_param_lock(struct module *mod)
> > mutex_lock(KPARAM_MUTEX(mod));
> > }
> >
> > +int kernel_param_trylock(struct module *mod)
> > +{
> > + return mutex_trylock(KPARAM_MUTEX(mod));
> > +}
> > +
> > void kernel_param_unlock(struct module *mod)
> > {
> > mutex_unlock(KPARAM_MUTEX(mod));
> > diff --git a/net/mac80211/rate.c b/net/mac80211/rate.c
> > index 0d056db9f81e..aecf7ff51cd9 100644
> > --- a/net/mac80211/rate.c
> > +++ b/net/mac80211/rate.c
> > @@ -217,7 +217,9 @@ ieee80211_rate_control_ops_get(const char *name)
> > const struct rate_control_ops *ops;
> > const char *alg_name;
> >
> > - kernel_param_lock(THIS_MODULE);
> > + if (!kernel_param_trylock(THIS_MODULE))
> > + return NULL;
> > +
> > if (!name)
> > alg_name = ieee80211_default_rc_algo;
> > else
> > --
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH V2] module: replace the mutex lock acquisition method
2025-02-21 11:41 ` Rafael J. Wysocki
2025-02-21 12:32 ` Lizhi Xu
@ 2025-02-21 12:35 ` Lizhi Xu
2025-02-21 20:07 ` Rafael J. Wysocki
1 sibling, 1 reply; 14+ messages in thread
From: Lizhi Xu @ 2025-02-21 12:35 UTC (permalink / raw)
To: rafael; +Cc: len.brown, linux-kernel, linux-pm, pavel, syzkaller-bugs
syzbot reported a deadlock in lock_system_sleep. [1]
The write operation to "/sys/module/hibernate/parameters/compressor"
conflicts with the registration of ieee80211 device, resulting in a deadlock
in the lock param_lock.
Since the conflict cannot be avoided, the way to obtain param_lock is changed
to trylock to avoid deadlock.
[1]
syz-executor895/5833 is trying to acquire lock:
ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
but task is already holding lock:
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (param_lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
rate_control_alloc net/mac80211/rate.c:266 [inline]
ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
do_one_initcall+0x128/0x700 init/main.c:1257
do_initcall_level init/main.c:1319 [inline]
do_initcalls init/main.c:1335 [inline]
do_basic_setup init/main.c:1354 [inline]
kernel_init_freeable+0x5c7/0x900 init/main.c:1568
kernel_init+0x1c/0x2b0 init/main.c:1457
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
-> #2 (rtnl_mutex){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
notifier_call_chain_robust kernel/notifier.c:120 [inline]
blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #0 (system_transition_mutex){+.+.}-{4:4}:
check_prev_add kernel/locking/lockdep.c:3163 [inline]
check_prevs_add kernel/locking/lockdep.c:3282 [inline]
validate_chain kernel/locking/lockdep.c:3906 [inline]
__lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
param_attr_store+0x18f/0x300 kernel/params.c:588
module_attr_store+0x55/0x80 kernel/params.c:924
sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
new_sync_write fs/read_write.c:586 [inline]
vfs_write+0x5ae/0x1150 fs/read_write.c:679
ksys_write+0x12b/0x250 fs/read_write.c:731
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
other info that might help us debug this:
Chain exists of:
system_transition_mutex --> rtnl_mutex --> param_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(param_lock);
lock(rtnl_mutex);
lock(param_lock);
lock(system_transition_mutex);
*** DEADLOCK ***
Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
Tested-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
---
V1 -> V2: use -EAGAIN to replace -EPERM.
include/linux/moduleparam.h | 4 ++++
kernel/params.c | 9 ++++++++-
net/mac80211/rate.c | 4 +++-
3 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
index bfb85fd13e1f..cbcbfd8db721 100644
--- a/include/linux/moduleparam.h
+++ b/include/linux/moduleparam.h
@@ -306,11 +306,15 @@ struct kparam_array
#ifdef CONFIG_SYSFS
extern void kernel_param_lock(struct module *mod);
+extern int kernel_param_trylock(struct module *mod);
extern void kernel_param_unlock(struct module *mod);
#else
static inline void kernel_param_lock(struct module *mod)
{
}
+static inline int kernel_param_trylock(struct module *mod)
+{
+}
static inline void kernel_param_unlock(struct module *mod)
{
}
diff --git a/kernel/params.c b/kernel/params.c
index 0074d29c9b80..d19881fbb2ec 100644
--- a/kernel/params.c
+++ b/kernel/params.c
@@ -583,7 +583,9 @@ static ssize_t param_attr_store(const struct module_attribute *mattr,
if (!attribute->param->ops->set)
return -EPERM;
- kernel_param_lock(mk->mod);
+ if (!kernel_param_trylock(mk->mod))
+ return -EAGAIN;
+
if (param_check_unsafe(attribute->param))
err = attribute->param->ops->set(buf, attribute->param);
else
@@ -607,6 +609,11 @@ void kernel_param_lock(struct module *mod)
mutex_lock(KPARAM_MUTEX(mod));
}
+int kernel_param_trylock(struct module *mod)
+{
+ return mutex_trylock(KPARAM_MUTEX(mod));
+}
+
void kernel_param_unlock(struct module *mod)
{
mutex_unlock(KPARAM_MUTEX(mod));
diff --git a/net/mac80211/rate.c b/net/mac80211/rate.c
index 0d056db9f81e..aecf7ff51cd9 100644
--- a/net/mac80211/rate.c
+++ b/net/mac80211/rate.c
@@ -217,7 +217,9 @@ ieee80211_rate_control_ops_get(const char *name)
const struct rate_control_ops *ops;
const char *alg_name;
- kernel_param_lock(THIS_MODULE);
+ if (!kernel_param_trylock(THIS_MODULE))
+ return NULL;
+
if (!name)
alg_name = ieee80211_default_rc_algo;
else
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH V2] module: replace the mutex lock acquisition method
2025-02-21 12:35 ` [PATCH V2] " Lizhi Xu
@ 2025-02-21 20:07 ` Rafael J. Wysocki
2025-02-22 1:02 ` [PATCH] " Lizhi Xu
2025-02-22 4:45 ` [PATCH V3] " Lizhi Xu
0 siblings, 2 replies; 14+ messages in thread
From: Rafael J. Wysocki @ 2025-02-21 20:07 UTC (permalink / raw)
To: Lizhi Xu; +Cc: rafael, len.brown, linux-kernel, linux-pm, pavel, syzkaller-bugs
On Fri, Feb 21, 2025 at 1:35 PM Lizhi Xu <lizhi.xu@windriver.com> wrote:
>
> syzbot reported a deadlock in lock_system_sleep. [1]
>
> The write operation to "/sys/module/hibernate/parameters/compressor"
> conflicts with the registration of ieee80211 device, resulting in a deadlock
> in the lock param_lock.
>
> Since the conflict cannot be avoided, the way to obtain param_lock is changed
> to trylock to avoid deadlock.
An alternative way to avoid the deadlock would be to replace
lock_system_sleep() in hibernate_compressor_param_set() with
mutex_trylock(&system_transition_mutex) (and analogously for the
unlock operation). Why have you not done that?
It is arguably better to fail a write to the module param with -EBUSY
than to fail ieee80211_register_hw() IMV.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] module: replace the mutex lock acquisition method
2025-02-21 20:07 ` Rafael J. Wysocki
@ 2025-02-22 1:02 ` Lizhi Xu
2025-02-22 4:45 ` [PATCH V3] " Lizhi Xu
1 sibling, 0 replies; 14+ messages in thread
From: Lizhi Xu @ 2025-02-22 1:02 UTC (permalink / raw)
To: rafael
Cc: len.brown, linux-kernel, linux-pm, lizhi.xu, pavel,
syzbot+ace60642828c074eb913, syzkaller-bugs
On Fri, 21 Feb 2025 21:07:59 +0100, Rafael J. Wysocki" <rafael@kernel.org> wrote:
> > syzbot reported a deadlock in lock_system_sleep. [1]
> >
> > The write operation to "/sys/module/hibernate/parameters/compressor"
> > conflicts with the registration of ieee80211 device, resulting in a deadlock
> > in the lock param_lock.
> >
> > Since the conflict cannot be avoided, the way to obtain param_lock is changed
> > to trylock to avoid deadlock.
>
> An alternative way to avoid the deadlock would be to replace
> lock_system_sleep() in hibernate_compressor_param_set() with
> mutex_trylock(&system_transition_mutex) (and analogously for the
> unlock operation). Why have you not done that?
Yes, you are right, I have confirmed that can work fine.
I will send V2 patch for using mutex_trylock(&system_transition_mutex).
BR,
Lizhi
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH V3] module: replace the mutex lock acquisition method
2025-02-21 20:07 ` Rafael J. Wysocki
2025-02-22 1:02 ` [PATCH] " Lizhi Xu
@ 2025-02-22 4:45 ` Lizhi Xu
2025-02-23 13:01 ` Rafael J. Wysocki
1 sibling, 1 reply; 14+ messages in thread
From: Lizhi Xu @ 2025-02-22 4:45 UTC (permalink / raw)
To: rafael; +Cc: len.brown, linux-kernel, linux-pm, pavel, syzkaller-bugs
syzbot reported a deadlock in lock_system_sleep. [1]
The write operation to "/sys/module/hibernate/parameters/compressor"
conflicts with the registration of ieee80211 device, resulting in a deadlock
in the lock param_lock.
Replace the method of acquiring the lock system_transition_mutex with trylock,
it is arguably better to fail a write to the module param with -EBUSY than to
fail ieee80211_register_hw() IMV.
[1]
syz-executor895/5833 is trying to acquire lock:
ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
but task is already holding lock:
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (param_lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
rate_control_alloc net/mac80211/rate.c:266 [inline]
ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
do_one_initcall+0x128/0x700 init/main.c:1257
do_initcall_level init/main.c:1319 [inline]
do_initcalls init/main.c:1335 [inline]
do_basic_setup init/main.c:1354 [inline]
kernel_init_freeable+0x5c7/0x900 init/main.c:1568
kernel_init+0x1c/0x2b0 init/main.c:1457
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
-> #2 (rtnl_mutex){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
notifier_call_chain_robust kernel/notifier.c:120 [inline]
blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #0 (system_transition_mutex){+.+.}-{4:4}:
check_prev_add kernel/locking/lockdep.c:3163 [inline]
check_prevs_add kernel/locking/lockdep.c:3282 [inline]
validate_chain kernel/locking/lockdep.c:3906 [inline]
__lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
param_attr_store+0x18f/0x300 kernel/params.c:588
module_attr_store+0x55/0x80 kernel/params.c:924
sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
new_sync_write fs/read_write.c:586 [inline]
vfs_write+0x5ae/0x1150 fs/read_write.c:679
ksys_write+0x12b/0x250 fs/read_write.c:731
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
other info that might help us debug this:
Chain exists of:
system_transition_mutex --> rtnl_mutex --> param_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(param_lock);
lock(rtnl_mutex);
lock(param_lock);
lock(system_transition_mutex);
*** DEADLOCK ***
Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
Tested-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
---
V1 -> V2: use -EAGAIN to replace -EPERM.
V2 -> V3: replace lock_system_sleep to trylock and update comments
include/linux/suspend.h | 2 ++
kernel/power/hibernate.c | 4 +++-
kernel/power/main.c | 12 ++++++++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/include/linux/suspend.h b/include/linux/suspend.h
index da6ebca3ff77..a9ea3c0b44d7 100644
--- a/include/linux/suspend.h
+++ b/include/linux/suspend.h
@@ -468,6 +468,7 @@ extern void pm_wakep_autosleep_enabled(bool set);
extern void pm_print_active_wakeup_sources(void);
extern unsigned int lock_system_sleep(void);
+extern unsigned int trylock_system_sleep(void);
extern void unlock_system_sleep(unsigned int);
#else /* !CONFIG_PM_SLEEP */
@@ -496,6 +497,7 @@ static inline void pm_wakeup_clear(bool reset) {}
static inline void pm_system_irq_wakeup(unsigned int irq_number) {}
static inline unsigned int lock_system_sleep(void) { return 0; }
+static inline unsigned int trylock_system_sleep(void) { return 0; }
static inline void unlock_system_sleep(unsigned int flags) {}
#endif /* !CONFIG_PM_SLEEP */
diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
index 10a01af63a80..eb2c424a0577 100644
--- a/kernel/power/hibernate.c
+++ b/kernel/power/hibernate.c
@@ -1449,7 +1449,9 @@ static int hibernate_compressor_param_set(const char *compressor,
unsigned int sleep_flags;
int index, ret;
- sleep_flags = lock_system_sleep();
+ sleep_flags = trylock_system_sleep();
+ if (!sleep_flags)
+ return -EBUSY;
index = sysfs_match_string(comp_alg_enabled, compressor);
if (index >= 0) {
diff --git a/kernel/power/main.c b/kernel/power/main.c
index 6254814d4817..6122c652638d 100644
--- a/kernel/power/main.c
+++ b/kernel/power/main.c
@@ -58,6 +58,18 @@ unsigned int lock_system_sleep(void)
}
EXPORT_SYMBOL_GPL(lock_system_sleep);
+unsigned int trylock_system_sleep(void)
+{
+ unsigned int flags = current->flags;
+ current->flags |= PF_NOFREEZE;
+ if (!mutex_trylock(&system_transition_mutex)) {
+ current->flags &= ~PF_NOFREEZE;
+ return 0;
+ }
+ return flags;
+}
+EXPORT_SYMBOL_GPL(trylock_system_sleep);
+
void unlock_system_sleep(unsigned int flags)
{
if (!(flags & PF_NOFREEZE))
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH V3] module: replace the mutex lock acquisition method
2025-02-22 4:45 ` [PATCH V3] " Lizhi Xu
@ 2025-02-23 13:01 ` Rafael J. Wysocki
2025-02-24 1:31 ` [PATCH V4] " Lizhi Xu
0 siblings, 1 reply; 14+ messages in thread
From: Rafael J. Wysocki @ 2025-02-23 13:01 UTC (permalink / raw)
To: Lizhi Xu; +Cc: rafael, len.brown, linux-kernel, linux-pm, pavel, syzkaller-bugs
On Sat, Feb 22, 2025 at 5:45 AM Lizhi Xu <lizhi.xu@windriver.com> wrote:
>
> syzbot reported a deadlock in lock_system_sleep. [1]
>
> The write operation to "/sys/module/hibernate/parameters/compressor"
> conflicts with the registration of ieee80211 device, resulting in a deadlock
> in the lock param_lock.
>
> Replace the method of acquiring the lock system_transition_mutex with trylock,
> it is arguably better to fail a write to the module param with -EBUSY than to
> fail ieee80211_register_hw() IMV.
>
> [1]
> syz-executor895/5833 is trying to acquire lock:
> ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
>
> but task is already holding lock:
> ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
> ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
>
> which lock already depends on the new lock.
>
>
> the existing dependency chain (in reverse order) is:
>
> -> #3 (param_lock){+.+.}-{4:4}:
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
> rate_control_alloc net/mac80211/rate.c:266 [inline]
> ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
> ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
> mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
> init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
> do_one_initcall+0x128/0x700 init/main.c:1257
> do_initcall_level init/main.c:1319 [inline]
> do_initcalls init/main.c:1335 [inline]
> do_basic_setup init/main.c:1354 [inline]
> kernel_init_freeable+0x5c7/0x900 init/main.c:1568
> kernel_init+0x1c/0x2b0 init/main.c:1457
> ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
>
> -> #2 (rtnl_mutex){+.+.}-{4:4}:
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
> wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
> notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
> notifier_call_chain_robust kernel/notifier.c:120 [inline]
> blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
> blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
> pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> misc_open+0x35a/0x420 drivers/char/misc.c:179
> chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> do_dentry_open+0x735/0x1c40 fs/open.c:956
> vfs_open+0x82/0x3f0 fs/open.c:1086
> do_open fs/namei.c:3830 [inline]
> path_openat+0x1e88/0x2d80 fs/namei.c:3989
> do_filp_open+0x20c/0x470 fs/namei.c:4016
> do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> do_sys_open fs/open.c:1443 [inline]
> __do_sys_openat fs/open.c:1459 [inline]
> __se_sys_openat fs/open.c:1454 [inline]
> __x64_sys_openat+0x175/0x210 fs/open.c:1454
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> -> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
> down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
> blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
> blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
> pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> misc_open+0x35a/0x420 drivers/char/misc.c:179
> chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> do_dentry_open+0x735/0x1c40 fs/open.c:956
> vfs_open+0x82/0x3f0 fs/open.c:1086
> do_open fs/namei.c:3830 [inline]
> path_openat+0x1e88/0x2d80 fs/namei.c:3989
> do_filp_open+0x20c/0x470 fs/namei.c:4016
> do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> do_sys_open fs/open.c:1443 [inline]
> __do_sys_openat fs/open.c:1459 [inline]
> __se_sys_openat fs/open.c:1454 [inline]
> __x64_sys_openat+0x175/0x210 fs/open.c:1454
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> -> #0 (system_transition_mutex){+.+.}-{4:4}:
> check_prev_add kernel/locking/lockdep.c:3163 [inline]
> check_prevs_add kernel/locking/lockdep.c:3282 [inline]
> validate_chain kernel/locking/lockdep.c:3906 [inline]
> __lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
> lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
> hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
> param_attr_store+0x18f/0x300 kernel/params.c:588
> module_attr_store+0x55/0x80 kernel/params.c:924
> sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
> kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
> new_sync_write fs/read_write.c:586 [inline]
> vfs_write+0x5ae/0x1150 fs/read_write.c:679
> ksys_write+0x12b/0x250 fs/read_write.c:731
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> other info that might help us debug this:
>
> Chain exists of:
> system_transition_mutex --> rtnl_mutex --> param_lock
>
> Possible unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(param_lock);
> lock(rtnl_mutex);
> lock(param_lock);
> lock(system_transition_mutex);
>
> *** DEADLOCK ***
>
> Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
> Tested-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
> Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
> ---
> V1 -> V2: use -EAGAIN to replace -EPERM.
> V2 -> V3: replace lock_system_sleep to trylock and update comments
>
> include/linux/suspend.h | 2 ++
> kernel/power/hibernate.c | 4 +++-
> kernel/power/main.c | 12 ++++++++++++
> 3 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/suspend.h b/include/linux/suspend.h
> index da6ebca3ff77..a9ea3c0b44d7 100644
> --- a/include/linux/suspend.h
> +++ b/include/linux/suspend.h
> @@ -468,6 +468,7 @@ extern void pm_wakep_autosleep_enabled(bool set);
> extern void pm_print_active_wakeup_sources(void);
>
> extern unsigned int lock_system_sleep(void);
> +extern unsigned int trylock_system_sleep(void);
> extern void unlock_system_sleep(unsigned int);
>
> #else /* !CONFIG_PM_SLEEP */
> @@ -496,6 +497,7 @@ static inline void pm_wakeup_clear(bool reset) {}
> static inline void pm_system_irq_wakeup(unsigned int irq_number) {}
>
> static inline unsigned int lock_system_sleep(void) { return 0; }
> +static inline unsigned int trylock_system_sleep(void) { return 0; }
> static inline void unlock_system_sleep(unsigned int flags) {}
>
> #endif /* !CONFIG_PM_SLEEP */
> diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
> index 10a01af63a80..eb2c424a0577 100644
> --- a/kernel/power/hibernate.c
> +++ b/kernel/power/hibernate.c
> @@ -1449,7 +1449,9 @@ static int hibernate_compressor_param_set(const char *compressor,
> unsigned int sleep_flags;
> int index, ret;
>
> - sleep_flags = lock_system_sleep();
> + sleep_flags = trylock_system_sleep();
Since this is not a kthread path and it doesn't call set_freezable()
on itself anywhere, the wrapper is not needed.
mutex_trylock(&system_transition_mutex) can be called from here
directly.
> + if (!sleep_flags)
> + return -EBUSY;
>
> index = sysfs_match_string(comp_alg_enabled, compressor);
> if (index >= 0) {
> diff --git a/kernel/power/main.c b/kernel/power/main.c
> index 6254814d4817..6122c652638d 100644
> --- a/kernel/power/main.c
> +++ b/kernel/power/main.c
> @@ -58,6 +58,18 @@ unsigned int lock_system_sleep(void)
> }
> EXPORT_SYMBOL_GPL(lock_system_sleep);
>
> +unsigned int trylock_system_sleep(void)
> +{
> + unsigned int flags = current->flags;
> + current->flags |= PF_NOFREEZE;
> + if (!mutex_trylock(&system_transition_mutex)) {
> + current->flags &= ~PF_NOFREEZE;
> + return 0;
> + }
> + return flags;
> +}
> +EXPORT_SYMBOL_GPL(trylock_system_sleep);
> +
> void unlock_system_sleep(unsigned int flags)
> {
> if (!(flags & PF_NOFREEZE))
> --
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH V4] module: replace the mutex lock acquisition method
2025-02-23 13:01 ` Rafael J. Wysocki
@ 2025-02-24 1:31 ` Lizhi Xu
2025-02-26 15:01 ` Rafael J. Wysocki
0 siblings, 1 reply; 14+ messages in thread
From: Lizhi Xu @ 2025-02-24 1:31 UTC (permalink / raw)
To: rafael; +Cc: len.brown, linux-kernel, linux-pm, pavel, lizhi.xu,
syzkaller-bugs
syzbot reported a deadlock in lock_system_sleep. [1]
The write operation to "/sys/module/hibernate/parameters/compressor"
conflicts with the registration of ieee80211 device, resulting in a deadlock
in the lock param_lock.
Replace the method of acquiring the lock system_transition_mutex with trylock,
it is arguably better to fail a write to the module param with -EBUSY than to
fail ieee80211_register_hw() IMV.
Since this is not a kthread path and it doesn't call set_freezable()
on itself anywhere, mutex_trylock(&system_transition_mutex) can be called
from here directly.
[1]
syz-executor895/5833 is trying to acquire lock:
ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
but task is already holding lock:
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (param_lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
rate_control_alloc net/mac80211/rate.c:266 [inline]
ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
do_one_initcall+0x128/0x700 init/main.c:1257
do_initcall_level init/main.c:1319 [inline]
do_initcalls init/main.c:1335 [inline]
do_basic_setup init/main.c:1354 [inline]
kernel_init_freeable+0x5c7/0x900 init/main.c:1568
kernel_init+0x1c/0x2b0 init/main.c:1457
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
-> #2 (rtnl_mutex){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
notifier_call_chain_robust kernel/notifier.c:120 [inline]
blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
snapshot_open+0x189/0x2b0 kernel/power/user.c:77
misc_open+0x35a/0x420 drivers/char/misc.c:179
chrdev_open+0x237/0x6a0 fs/char_dev.c:414
do_dentry_open+0x735/0x1c40 fs/open.c:956
vfs_open+0x82/0x3f0 fs/open.c:1086
do_open fs/namei.c:3830 [inline]
path_openat+0x1e88/0x2d80 fs/namei.c:3989
do_filp_open+0x20c/0x470 fs/namei.c:4016
do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
do_sys_open fs/open.c:1443 [inline]
__do_sys_openat fs/open.c:1459 [inline]
__se_sys_openat fs/open.c:1454 [inline]
__x64_sys_openat+0x175/0x210 fs/open.c:1454
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #0 (system_transition_mutex){+.+.}-{4:4}:
check_prev_add kernel/locking/lockdep.c:3163 [inline]
check_prevs_add kernel/locking/lockdep.c:3282 [inline]
validate_chain kernel/locking/lockdep.c:3906 [inline]
__lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
__mutex_lock_common kernel/locking/mutex.c:585 [inline]
__mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
param_attr_store+0x18f/0x300 kernel/params.c:588
module_attr_store+0x55/0x80 kernel/params.c:924
sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
new_sync_write fs/read_write.c:586 [inline]
vfs_write+0x5ae/0x1150 fs/read_write.c:679
ksys_write+0x12b/0x250 fs/read_write.c:731
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
other info that might help us debug this:
Chain exists of:
system_transition_mutex --> rtnl_mutex --> param_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(param_lock);
lock(rtnl_mutex);
lock(param_lock);
lock(system_transition_mutex);
*** DEADLOCK ***
Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
---
V1 -> V2: use -EAGAIN to replace -EPERM.
V2 -> V3: replace lock_system_sleep to trylock and update comments
V3 -> V4: use system_transition_mutex directly
kernel/power/hibernate.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
index 10a01af63a80..b129ed1d25a8 100644
--- a/kernel/power/hibernate.c
+++ b/kernel/power/hibernate.c
@@ -1446,10 +1446,10 @@ static const char * const comp_alg_enabled[] = {
static int hibernate_compressor_param_set(const char *compressor,
const struct kernel_param *kp)
{
- unsigned int sleep_flags;
int index, ret;
- sleep_flags = lock_system_sleep();
+ if (!mutex_trylock(&system_transition_mutex))
+ return -EBUSY;
index = sysfs_match_string(comp_alg_enabled, compressor);
if (index >= 0) {
@@ -1461,7 +1461,7 @@ static int hibernate_compressor_param_set(const char *compressor,
ret = index;
}
- unlock_system_sleep(sleep_flags);
+ mutex_unlock(&system_transition_mutex);
if (ret)
pr_debug("Cannot set specified compressor %s\n",
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH V4] module: replace the mutex lock acquisition method
2025-02-24 1:31 ` [PATCH V4] " Lizhi Xu
@ 2025-02-26 15:01 ` Rafael J. Wysocki
0 siblings, 0 replies; 14+ messages in thread
From: Rafael J. Wysocki @ 2025-02-26 15:01 UTC (permalink / raw)
To: Lizhi Xu; +Cc: rafael, len.brown, linux-kernel, linux-pm, pavel, syzkaller-bugs
On Mon, Feb 24, 2025 at 2:31 AM Lizhi Xu <lizhi.xu@windriver.com> wrote:
>
> syzbot reported a deadlock in lock_system_sleep. [1]
>
> The write operation to "/sys/module/hibernate/parameters/compressor"
> conflicts with the registration of ieee80211 device, resulting in a deadlock
> in the lock param_lock.
>
> Replace the method of acquiring the lock system_transition_mutex with trylock,
> it is arguably better to fail a write to the module param with -EBUSY than to
> fail ieee80211_register_hw() IMV.
>
> Since this is not a kthread path and it doesn't call set_freezable()
> on itself anywhere, mutex_trylock(&system_transition_mutex) can be called
> from here directly.
>
> [1]
> syz-executor895/5833 is trying to acquire lock:
> ffffffff8e0828c8 (system_transition_mutex){+.+.}-{4:4}, at: lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
>
> but task is already holding lock:
> ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: kernel_param_lock kernel/params.c:607 [inline]
> ffffffff8e07dc68 (param_lock){+.+.}-{4:4}, at: param_attr_store+0xe6/0x300 kernel/params.c:586
>
> which lock already depends on the new lock.
>
>
> the existing dependency chain (in reverse order) is:
>
> -> #3 (param_lock){+.+.}-{4:4}:
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> ieee80211_rate_control_ops_get net/mac80211/rate.c:220 [inline]
> rate_control_alloc net/mac80211/rate.c:266 [inline]
> ieee80211_init_rate_ctrl_alg+0x18d/0x6b0 net/mac80211/rate.c:1015
> ieee80211_register_hw+0x20cd/0x4060 net/mac80211/main.c:1531
> mac80211_hwsim_new_radio+0x304e/0x54e0 drivers/net/wireless/virtual/mac80211_hwsim.c:5558
> init_mac80211_hwsim+0x432/0x8c0 drivers/net/wireless/virtual/mac80211_hwsim.c:6910
> do_one_initcall+0x128/0x700 init/main.c:1257
> do_initcall_level init/main.c:1319 [inline]
> do_initcalls init/main.c:1335 [inline]
> do_basic_setup init/main.c:1354 [inline]
> kernel_init_freeable+0x5c7/0x900 init/main.c:1568
> kernel_init+0x1c/0x2b0 init/main.c:1457
> ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
>
> -> #2 (rtnl_mutex){+.+.}-{4:4}:
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> wg_pm_notification drivers/net/wireguard/device.c:80 [inline]
> wg_pm_notification+0x49/0x180 drivers/net/wireguard/device.c:64
> notifier_call_chain+0xb7/0x410 kernel/notifier.c:85
> notifier_call_chain_robust kernel/notifier.c:120 [inline]
> blocking_notifier_call_chain_robust kernel/notifier.c:345 [inline]
> blocking_notifier_call_chain_robust+0xc9/0x170 kernel/notifier.c:333
> pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> misc_open+0x35a/0x420 drivers/char/misc.c:179
> chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> do_dentry_open+0x735/0x1c40 fs/open.c:956
> vfs_open+0x82/0x3f0 fs/open.c:1086
> do_open fs/namei.c:3830 [inline]
> path_openat+0x1e88/0x2d80 fs/namei.c:3989
> do_filp_open+0x20c/0x470 fs/namei.c:4016
> do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> do_sys_open fs/open.c:1443 [inline]
> __do_sys_openat fs/open.c:1459 [inline]
> __se_sys_openat fs/open.c:1454 [inline]
> __x64_sys_openat+0x175/0x210 fs/open.c:1454
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> -> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
> down_read+0x9a/0x330 kernel/locking/rwsem.c:1524
> blocking_notifier_call_chain_robust kernel/notifier.c:344 [inline]
> blocking_notifier_call_chain_robust+0xa9/0x170 kernel/notifier.c:333
> pm_notifier_call_chain_robust+0x27/0x60 kernel/power/main.c:102
> snapshot_open+0x189/0x2b0 kernel/power/user.c:77
> misc_open+0x35a/0x420 drivers/char/misc.c:179
> chrdev_open+0x237/0x6a0 fs/char_dev.c:414
> do_dentry_open+0x735/0x1c40 fs/open.c:956
> vfs_open+0x82/0x3f0 fs/open.c:1086
> do_open fs/namei.c:3830 [inline]
> path_openat+0x1e88/0x2d80 fs/namei.c:3989
> do_filp_open+0x20c/0x470 fs/namei.c:4016
> do_sys_openat2+0x17a/0x1e0 fs/open.c:1428
> do_sys_open fs/open.c:1443 [inline]
> __do_sys_openat fs/open.c:1459 [inline]
> __se_sys_openat fs/open.c:1454 [inline]
> __x64_sys_openat+0x175/0x210 fs/open.c:1454
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> -> #0 (system_transition_mutex){+.+.}-{4:4}:
> check_prev_add kernel/locking/lockdep.c:3163 [inline]
> check_prevs_add kernel/locking/lockdep.c:3282 [inline]
> validate_chain kernel/locking/lockdep.c:3906 [inline]
> __lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5228
> lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5851
> __mutex_lock_common kernel/locking/mutex.c:585 [inline]
> __mutex_lock+0x19b/0xb10 kernel/locking/mutex.c:730
> lock_system_sleep+0x87/0xa0 kernel/power/main.c:56
> hibernate_compressor_param_set+0x1c/0x210 kernel/power/hibernate.c:1452
> param_attr_store+0x18f/0x300 kernel/params.c:588
> module_attr_store+0x55/0x80 kernel/params.c:924
> sysfs_kf_write+0x117/0x170 fs/sysfs/file.c:139
> kernfs_fop_write_iter+0x33d/0x500 fs/kernfs/file.c:334
> new_sync_write fs/read_write.c:586 [inline]
> vfs_write+0x5ae/0x1150 fs/read_write.c:679
> ksys_write+0x12b/0x250 fs/read_write.c:731
> do_syscall_x64 arch/x86/entry/common.c:52 [inline]
> do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> other info that might help us debug this:
>
> Chain exists of:
> system_transition_mutex --> rtnl_mutex --> param_lock
>
> Possible unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(param_lock);
> lock(rtnl_mutex);
> lock(param_lock);
> lock(system_transition_mutex);
>
> *** DEADLOCK ***
>
> Reported-by: syzbot+ace60642828c074eb913@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=ace60642828c074eb913
> Signed-off-by: Lizhi Xu <lizhi.xu@windriver.com>
> ---
> V1 -> V2: use -EAGAIN to replace -EPERM.
> V2 -> V3: replace lock_system_sleep to trylock and update comments
> V3 -> V4: use system_transition_mutex directly
>
> kernel/power/hibernate.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
> index 10a01af63a80..b129ed1d25a8 100644
> --- a/kernel/power/hibernate.c
> +++ b/kernel/power/hibernate.c
> @@ -1446,10 +1446,10 @@ static const char * const comp_alg_enabled[] = {
> static int hibernate_compressor_param_set(const char *compressor,
> const struct kernel_param *kp)
> {
> - unsigned int sleep_flags;
> int index, ret;
>
> - sleep_flags = lock_system_sleep();
> + if (!mutex_trylock(&system_transition_mutex))
> + return -EBUSY;
>
> index = sysfs_match_string(comp_alg_enabled, compressor);
> if (index >= 0) {
> @@ -1461,7 +1461,7 @@ static int hibernate_compressor_param_set(const char *compressor,
> ret = index;
> }
>
> - unlock_system_sleep(sleep_flags);
> + mutex_unlock(&system_transition_mutex);
>
> if (ret)
> pr_debug("Cannot set specified compressor %s\n",
> --
Applied as 6.15 material under a new subject and with edits in the changelog.
Thanks!
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2025-02-26 15:01 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-20 15:16 [syzbot] [pm?] possible deadlock in lock_system_sleep syzbot
2025-02-21 1:45 ` [syzbot] " syzbot
2025-02-21 2:57 ` syzbot
2025-02-21 6:27 ` syzbot
2025-02-21 10:10 ` [PATCH] module: replace the mutex lock acquisition method Lizhi Xu
2025-02-21 11:41 ` Rafael J. Wysocki
2025-02-21 12:32 ` Lizhi Xu
2025-02-21 12:35 ` [PATCH V2] " Lizhi Xu
2025-02-21 20:07 ` Rafael J. Wysocki
2025-02-22 1:02 ` [PATCH] " Lizhi Xu
2025-02-22 4:45 ` [PATCH V3] " Lizhi Xu
2025-02-23 13:01 ` Rafael J. Wysocki
2025-02-24 1:31 ` [PATCH V4] " Lizhi Xu
2025-02-26 15:01 ` Rafael J. Wysocki
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox