[syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    f9f24ca362a4 Add linux-next specific files for 20241031
git tree:       linux-next
console+strace: https://syzkaller.appspot.com/x/log.txt?x=14886630580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=328572ed4d152be9
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=174432a7980000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ffe55f980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/eb84549dd6b3/disk-f9f24ca3.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/beb29bdfa297/vmlinux-f9f24ca3.xz
kernel image: https://storage.googleapis.com/syzbot-assets/8881fe3245ad/bzImage-f9f24ca3.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+d2adb332fe371b0595e3@syzkaller.appspotmail.com

=============================
[ BUG: Invalid wait context ]
6.12.0-rc5-next-20241031-syzkaller #0 Not tainted
-----------------------------
syz-executor304/5844 is trying to lock:
ffffffff8e9ba4b8 (map_idr_lock){+...}-{3:3}, at: bpf_map_free_id kernel/bpf/syscall.c:468 [inline]
ffffffff8e9ba4b8 (map_idr_lock){+...}-{3:3}, at: bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:902
other info that might help us debug this:
context-{5:5}
2 locks held by syz-executor304/5844:
 #0: ffffffff8e939f20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
 #0: ffffffff8e939f20 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:849 [inline]
 #0: ffffffff8e939f20 (rcu_read_lock){....}-{1:3}, at: map_delete_elem+0x338/0x5c0 kernel/bpf/syscall.c:1777
 #1: ffff88807b870410 (&htab->lockdep_key){....}-{2:2}, at: htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
stack backtrace:
CPU: 1 UID: 0 PID: 5844 Comm: syz-executor304 Not tainted 6.12.0-rc5-next-20241031-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 print_lock_invalid_wait_context kernel/locking/lockdep.c:4826 [inline]
 check_wait_context kernel/locking/lockdep.c:4898 [inline]
 __lock_acquire+0x15a8/0x2100 kernel/locking/lockdep.c:5176
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
 bpf_map_free_id kernel/bpf/syscall.c:468 [inline]
 bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:902
 htab_put_fd_value kernel/bpf/hashtab.c:911 [inline]
 free_htab_elem+0xbb/0x460 kernel/bpf/hashtab.c:946
 htab_map_delete_elem+0x576/0x6b0 kernel/bpf/hashtab.c:1438
 map_delete_elem+0x431/0x5c0 kernel/bpf/syscall.c:1778
 __sys_bpf+0x598/0x810 kernel/bpf/syscall.c:5745
 __do_sys_bpf kernel/bpf/syscall.c:5861 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:5859 [inline]
 __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5859
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f9ad03385e9
Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffd14d58828 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ffd14d589f8 RCX: 00007f9ad03385e9
RDX: 0000000000000020 RSI: 0000000020000300 RDI: 0000000000000003
R


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[Edward Adam Davis]

#syz test: upstream master

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
WARNING: locking bug in bpf_map_put

=============================
[ BUG: Invalid wait context ]
6.12.0-rc5-syzkaller-00291-g05b92660cdfe #0 Not tainted
-----------------------------
syz.0.15/6647 is trying to lock:
ffffffff8e9b57d8 (map_idr_lock){+...}-{3:3}, at: bpf_map_free_id kernel/bpf/syscall.c:380 [inline]
ffffffff8e9b57d8 (map_idr_lock){+...}-{3:3}, at: bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:808
other info that might help us debug this:
context-{5:5}
2 locks held by syz.0.15/6647:
 #0: ffffffff8e937de0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
 #0: ffffffff8e937de0 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:849 [inline]
 #0: ffffffff8e937de0 (rcu_read_lock){....}-{1:3}, at: map_delete_elem+0x338/0x5c0 kernel/bpf/syscall.c:1677
 #1: ffff88807caca0f8 (&htab->lockdep_key){....}-{2:2}, at: htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
stack backtrace:
CPU: 0 UID: 0 PID: 6647 Comm: syz.0.15 Not tainted 6.12.0-rc5-syzkaller-00291-g05b92660cdfe #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 print_lock_invalid_wait_context kernel/locking/lockdep.c:4802 [inline]
 check_wait_context kernel/locking/lockdep.c:4874 [inline]
 __lock_acquire+0x15a8/0x2100 kernel/locking/lockdep.c:5152
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
 bpf_map_free_id kernel/bpf/syscall.c:380 [inline]
 bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:808
 htab_put_fd_value kernel/bpf/hashtab.c:911 [inline]
 free_htab_elem+0xbb/0x460 kernel/bpf/hashtab.c:946
 htab_map_delete_elem+0x576/0x6b0 kernel/bpf/hashtab.c:1438
 map_delete_elem+0x431/0x5c0 kernel/bpf/syscall.c:1678
 __sys_bpf+0x598/0x810 kernel/bpf/syscall.c:5644
 __do_sys_bpf kernel/bpf/syscall.c:5760 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:5758 [inline]
 __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5758
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7ff3cef7e719
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ff3cfda7038 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ff3cf135f80 RCX: 00007ff3cef7e719
RDX: 0000000000000020 RSI: 0000000020000300 RDI: 0000000000000003
RBP: 00007ff3ceff132e R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007ff3cf135f80 R15: 00007ffd7e2203e8
 </TASK>


Tested on:

commit:         05b92660 Merge tag 'pci-v6.12-fixes-2' of git://git.ke..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=145a9630580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a6b0d046ffd0e9c1
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[Edward Adam Davis]

there is a raw_spin_lock and spin_lock:

      raw_spin_lock(&b->raw_lock);
      spin_lock(&map_idr_lock);
      spin_unlock(&map_idr_lock);
      raw_spin_unlock(&b->raw_lock);

#syz test: upstream master

diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index a8f1808a1ca5..4891904ee1fc 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -54,7 +54,7 @@ DEFINE_PER_CPU(int, bpf_prog_active);
 static DEFINE_IDR(prog_idr);
 static DEFINE_SPINLOCK(prog_idr_lock);
 static DEFINE_IDR(map_idr);
-static DEFINE_SPINLOCK(map_idr_lock);
+static DEFINE_RAW_SPINLOCK(map_idr_lock);
 static DEFINE_IDR(link_idr);
 static DEFINE_SPINLOCK(link_idr_lock);
 
@@ -352,11 +352,11 @@ static int bpf_map_alloc_id(struct bpf_map *map)
 	int id;
 
 	idr_preload(GFP_KERNEL);
-	spin_lock_bh(&map_idr_lock);
+	raw_spin_lock_bh(&map_idr_lock);
 	id = idr_alloc_cyclic(&map_idr, map, 1, INT_MAX, GFP_ATOMIC);
 	if (id > 0)
 		map->id = id;
-	spin_unlock_bh(&map_idr_lock);
+	raw_spin_unlock_bh(&map_idr_lock);
 	idr_preload_end();
 
 	if (WARN_ON_ONCE(!id))
@@ -377,12 +377,12 @@ void bpf_map_free_id(struct bpf_map *map)
 	if (!map->id)
 		return;
 
-	spin_lock_irqsave(&map_idr_lock, flags);
+	raw_spin_lock_irqsave(&map_idr_lock, flags);
 
 	idr_remove(&map_idr, map->id);
 	map->id = 0;
 
-	spin_unlock_irqrestore(&map_idr_lock, flags);
+	raw_spin_unlock_irqrestore(&map_idr_lock, flags);
 }
 
 #ifdef CONFIG_MEMCG
@@ -1479,9 +1479,9 @@ struct bpf_map *__bpf_map_inc_not_zero(struct bpf_map *map, bool uref)
 
 struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map)
 {
-	spin_lock_bh(&map_idr_lock);
+	raw_spin_lock_bh(&map_idr_lock);
 	map = __bpf_map_inc_not_zero(map, false);
-	spin_unlock_bh(&map_idr_lock);
+	raw_spin_unlock_bh(&map_idr_lock);
 
 	return map;
 }
@@ -4278,11 +4278,37 @@ static int bpf_obj_get_next_id(const union bpf_attr *attr,
 	return err;
 }
 
+static int bpf_obj_get_next_id_raw(const union bpf_attr *attr,
+			       union bpf_attr __user *uattr,
+			       struct idr *idr,
+			       raw_spinlock_t *lock)
+{
+	u32 next_id = attr->start_id;
+	int err = 0;
+
+	if (CHECK_ATTR(BPF_OBJ_GET_NEXT_ID) || next_id >= INT_MAX)
+		return -EINVAL;
+
+	if (!capable(CAP_SYS_ADMIN))
+		return -EPERM;
+
+	next_id++;
+	raw_spin_lock_bh(lock);
+	if (!idr_get_next(idr, &next_id))
+		err = -ENOENT;
+	raw_spin_unlock_bh(lock);
+
+	if (!err)
+		err = put_user(next_id, &uattr->next_id);
+
+	return err;
+}
+
 struct bpf_map *bpf_map_get_curr_or_next(u32 *id)
 {
 	struct bpf_map *map;
 
-	spin_lock_bh(&map_idr_lock);
+	raw_spin_lock_bh(&map_idr_lock);
 again:
 	map = idr_get_next(&map_idr, id);
 	if (map) {
@@ -4292,7 +4318,7 @@ struct bpf_map *bpf_map_get_curr_or_next(u32 *id)
 			goto again;
 		}
 	}
-	spin_unlock_bh(&map_idr_lock);
+	raw_spin_unlock_bh(&map_idr_lock);
 
 	return map;
 }
@@ -4378,13 +4404,13 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr)
 	if (f_flags < 0)
 		return f_flags;
 
-	spin_lock_bh(&map_idr_lock);
+	raw_spin_lock_bh(&map_idr_lock);
 	map = idr_find(&map_idr, id);
 	if (map)
 		map = __bpf_map_inc_not_zero(map, true);
 	else
 		map = ERR_PTR(-ENOENT);
-	spin_unlock_bh(&map_idr_lock);
+	raw_spin_unlock_bh(&map_idr_lock);
 
 	if (IS_ERR(map))
 		return PTR_ERR(map);
@@ -5656,7 +5682,7 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size)
 					  &prog_idr, &prog_idr_lock);
 		break;
 	case BPF_MAP_GET_NEXT_ID:
-		err = bpf_obj_get_next_id(&attr, uattr.user,
+		err = bpf_obj_get_next_id_raw(&attr, uattr.user,
 					  &map_idr, &map_idr_lock);
 		break;
 	case BPF_BTF_GET_NEXT_ID:
Upstream-Status: Pending

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
WARNING: locking bug in radix_tree_node_alloc

=============================
[ BUG: Invalid wait context ]
6.12.0-rc5-syzkaller-00299-g11066801dd4b-dirty #0 Not tainted
-----------------------------
syz.0.15/6617 is trying to lock:
ffff8880b8643cc0 (&c->lock){-.-.}-{3:3}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
ffff8880b8643cc0 (&c->lock){-.-.}-{3:3}, at: ___slab_alloc+0x265/0x14b0 mm/slub.c:3695
other info that might help us debug this:
context-{5:5}
2 locks held by syz.0.15/6617:
 #0: ffff8880b863d678 (lock){+.+.}-{3:3}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
 #0: ffff8880b863d678 (lock){+.+.}-{3:3}, at: __radix_tree_preload+0x80/0x860 lib/radix-tree.c:334
 #1: ffffffff8e9b57d8 (map_idr_lock){+...}-{2:2}, at: bpf_map_alloc_id+0x21/0xe0 kernel/bpf/syscall.c:355
stack backtrace:
CPU: 0 UID: 0 PID: 6617 Comm: syz.0.15 Not tainted 6.12.0-rc5-syzkaller-00299-g11066801dd4b-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 print_lock_invalid_wait_context kernel/locking/lockdep.c:4802 [inline]
 check_wait_context kernel/locking/lockdep.c:4874 [inline]
 __lock_acquire+0x15a8/0x2100 kernel/locking/lockdep.c:5152
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825
 local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
 ___slab_alloc+0x27e/0x14b0 mm/slub.c:3695
 __slab_alloc+0x58/0xa0 mm/slub.c:3908
 __slab_alloc_node mm/slub.c:3961 [inline]
 slab_alloc_node mm/slub.c:4122 [inline]
 kmem_cache_alloc_noprof+0x1c1/0x2a0 mm/slub.c:4141
 radix_tree_node_alloc+0x8b/0x3c0 lib/radix-tree.c:276
 idr_get_free+0x296/0xab0 lib/radix-tree.c:1506
 idr_alloc_u32+0x195/0x330 lib/idr.c:46
 idr_alloc_cyclic+0x106/0x300 lib/idr.c:125
 bpf_map_alloc_id+0x40/0xe0 kernel/bpf/syscall.c:356
 map_create+0xdc3/0x11c0 kernel/bpf/syscall.c:1398
 __sys_bpf+0x6c8/0x810 kernel/bpf/syscall.c:5661
 __do_sys_bpf kernel/bpf/syscall.c:5786 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:5784 [inline]
 __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5784
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fe23117e719
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fe231ee3038 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007fe231335f80 RCX: 00007fe23117e719
RDX: 0000000000000048 RSI: 00000000200004c0 RDI: 0000000000000000
RBP: 00007fe2311f132e R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000001 R14: 00007fe231335f80 R15: 00007fff2658b528
 </TASK>


Tested on:

commit:         11066801 Merge tag 'linux_kselftest-fixes-6.12-rc6' of..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=143a5630580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a6b0d046ffd0e9c1
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch:          https://syzkaller.appspot.com/x/patch.diff?x=12ce3340580000

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[Edward Adam Davis]

there is a raw_spin_lock and spin_lock:

      raw_spin_lock(&b->raw_lock);
      spin_lock(&map_idr_lock);
      spin_unlock(&map_idr_lock);
      raw_spin_unlock(&b->raw_lock);

#syz test: upstream master

diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index b14b87463ee0..c2e0b26c8053 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -78,7 +78,7 @@
  */
 struct bucket {
 	struct hlist_nulls_head head;
-	raw_spinlock_t raw_lock;
+	spinlock_t lock;
 };
 
 #define HASHTAB_MAP_LOCK_COUNT 8
@@ -140,8 +140,8 @@ static void htab_init_buckets(struct bpf_htab *htab)
 
 	for (i = 0; i < htab->n_buckets; i++) {
 		INIT_HLIST_NULLS_HEAD(&htab->buckets[i].head, i);
-		raw_spin_lock_init(&htab->buckets[i].raw_lock);
-		lockdep_set_class(&htab->buckets[i].raw_lock,
+		spin_lock_init(&htab->buckets[i].lock);
+		lockdep_set_class(&htab->buckets[i].lock,
 					  &htab->lockdep_key);
 		cond_resched();
 	}
@@ -164,7 +164,7 @@ static inline int htab_lock_bucket(const struct bpf_htab *htab,
 		return -EBUSY;
 	}
 
-	raw_spin_lock(&b->raw_lock);
+	spin_lock(&b->lock);
 	*pflags = flags;
 
 	return 0;
@@ -175,7 +175,7 @@ static inline void htab_unlock_bucket(const struct bpf_htab *htab,
 				      unsigned long flags)
 {
 	hash = hash & min_t(u32, HASHTAB_MAP_LOCK_MASK, htab->n_buckets - 1);
-	raw_spin_unlock(&b->raw_lock);
+	spin_unlock(&b->lock);
 	__this_cpu_dec(*(htab->map_locked[hash]));
 	local_irq_restore(flags);
 	preempt_enable();

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-by: syzbot+d2adb332fe371b0595e3@syzkaller.appspotmail.com
Tested-by: syzbot+d2adb332fe371b0595e3@syzkaller.appspotmail.com

Tested on:

commit:         11066801 Merge tag 'linux_kselftest-fixes-6.12-rc6' of..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=15802987980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a6b0d046ffd0e9c1
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch:          https://syzkaller.appspot.com/x/patch.diff?x=1395b55f980000

Note: testing is done by a robot and is best-effort only.

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[syzbot]

syzbot has bisected this issue to:

commit 560af5dc839eef08a273908f390cfefefb82aa04
Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Date:   Wed Oct 9 15:45:03 2024 +0000

    lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=122a4740580000
start commit:   f9f24ca362a4 Add linux-next specific files for 20241031
git tree:       linux-next
final oops:     https://syzkaller.appspot.com/x/report.txt?x=112a4740580000
console output: https://syzkaller.appspot.com/x/log.txt?x=162a4740580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=328572ed4d152be9
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=174432a7980000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ffe55f980000

Reported-by: syzbot+d2adb332fe371b0595e3@syzkaller.appspotmail.com
Fixes: 560af5dc839e ("lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[Sebastian Andrzej Siewior]

On 2024-11-03 18:29:04 [-0800], syzbot wrote:
> syzbot has bisected this issue to:
> 
> commit 560af5dc839eef08a273908f390cfefefb82aa04
> Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Date:   Wed Oct 9 15:45:03 2024 +0000
> 
>     lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.
> 
> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=122a4740580000
> start commit:   f9f24ca362a4 Add linux-next specific files for 20241031
> git tree:       linux-next
> final oops:     https://syzkaller.appspot.com/x/report.txt?x=112a4740580000
> console output: https://syzkaller.appspot.com/x/log.txt?x=162a4740580000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=328572ed4d152be9
> dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=174432a7980000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ffe55f980000
> 
> Reported-by: syzbot+d2adb332fe371b0595e3@syzkaller.appspotmail.com
> Fixes: 560af5dc839e ("lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.")
> 
> For information about bisection process see: https://goo.gl/tpsmEJ#bisection

This is due to raw_spinlock_t in bucket::lock and the acquired
spinlock_t underneath. Would it would to move free part outside of the
locked section?

diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index b14b87463ee04..1d8d09fdd2da5 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -824,13 +824,14 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node)
 	hlist_nulls_for_each_entry_rcu(l, n, head, hash_node)
 		if (l == tgt_l) {
 			hlist_nulls_del_rcu(&l->hash_node);
-			check_and_free_fields(htab, l);
 			bpf_map_dec_elem_count(&htab->map);
 			break;
 		}
 
 	htab_unlock_bucket(htab, b, tgt_l->hash, flags);
 
+	if (l == tgt_l)
+		check_and_free_fields(htab, l);
 	return l == tgt_l;
 }
 
@@ -1181,14 +1182,18 @@ static long htab_map_update_elem(struct bpf_map *map, void *key, void *value,
 	 * concurrent search will find it before old elem
 	 */
 	hlist_nulls_add_head_rcu(&l_new->hash_node, head);
-	if (l_old) {
+	if (l_old)
 		hlist_nulls_del_rcu(&l_old->hash_node);
+	htab_unlock_bucket(htab, b, hash, flags);
+
+	if (l_old) {
 		if (!htab_is_prealloc(htab))
 			free_htab_elem(htab, l_old);
 		else
 			check_and_free_fields(htab, l_old);
 	}
-	ret = 0;
+	return 0;
+
 err:
 	htab_unlock_bucket(htab, b, hash, flags);
 	return ret;
@@ -1433,14 +1438,15 @@ static long htab_map_delete_elem(struct bpf_map *map, void *key)
 
 	l = lookup_elem_raw(head, hash, key, key_size);
 
-	if (l) {
+	if (l)
 		hlist_nulls_del_rcu(&l->hash_node);
-		free_htab_elem(htab, l);
-	} else {
+	else
 		ret = -ENOENT;
-	}
 
 	htab_unlock_bucket(htab, b, hash, flags);
+
+	if (l)
+		free_htab_elem(htab, l);
 	return ret;
 }
 
@@ -1647,14 +1653,16 @@ static int __htab_map_lookup_and_delete_elem(struct bpf_map *map, void *key,
 		}
 
 		hlist_nulls_del_rcu(&l->hash_node);
-		if (!is_lru_map)
-			free_htab_elem(htab, l);
 	}
 
 	htab_unlock_bucket(htab, b, hash, bflags);
 
-	if (is_lru_map && l)
-		htab_lru_push_free(htab, l);
+	if (l) {
+		if (is_lru_map)
+			htab_lru_push_free(htab, l);
+		else
+			free_htab_elem(htab, l);
+	}
 
 	return ret;
 }
@@ -1851,15 +1859,12 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
 
 			/* bpf_lru_push_free() will acquire lru_lock, which
 			 * may cause deadlock. See comments in function
-			 * prealloc_lru_pop(). Let us do bpf_lru_push_free()
-			 * after releasing the bucket lock.
+			 * prealloc_lru_pop(). htab_lru_push_free() may allocate
+			 * sleeping locks. Let us do bpf_lru_push_free() after
+			 * releasing the bucket lock.
 			 */
-			if (is_lru_map) {
-				l->batch_flink = node_to_free;
-				node_to_free = l;
-			} else {
-				free_htab_elem(htab, l);
-			}
+			l->batch_flink = node_to_free;
+			node_to_free = l;
 		}
 		dst_key += key_size;
 		dst_val += value_size;
@@ -1871,7 +1876,10 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
 	while (node_to_free) {
 		l = node_to_free;
 		node_to_free = node_to_free->batch_flink;
-		htab_lru_push_free(htab, l);
+		if (is_lru_map)
+			htab_lru_push_free(htab, l);
+		else
+			free_htab_elem(htab, l);
 	}
 
 next_batch:

Re: [syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[Hou Tao]

Hi,

On 11/5/2024 12:28 AM, Sebastian Andrzej Siewior wrote:
> On 2024-11-03 18:29:04 [-0800], syzbot wrote:
>> syzbot has bisected this issue to:
>>
>> commit 560af5dc839eef08a273908f390cfefefb82aa04
>> Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
>> Date:   Wed Oct 9 15:45:03 2024 +0000
>>
>>     lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.
>>
>> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=122a4740580000
>> start commit:   f9f24ca362a4 Add linux-next specific files for 20241031
>> git tree:       linux-next
>> final oops:     https://syzkaller.appspot.com/x/report.txt?x=112a4740580000
>> console output: https://syzkaller.appspot.com/x/log.txt?x=162a4740580000
>> kernel config:  https://syzkaller.appspot.com/x/.config?x=328572ed4d152be9
>> dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
>> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=174432a7980000
>> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ffe55f980000
>>
>> Reported-by: syzbot+d2adb332fe371b0595e3@syzkaller.appspotmail.com
>> Fixes: 560af5dc839e ("lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.")
>>
>> For information about bisection process see: https://goo.gl/tpsmEJ#bisection
> This is due to raw_spinlock_t in bucket::lock and the acquired
> spinlock_t underneath. Would it would to move free part outside of the
> locked section?

I think moving free_htab_elem() after htab_unlock_bucket() is OK. But
the fix below is not enough, and there is some corn cases for
pre-allocated element . I had written a patch for the problem a few day
ago because the problem can be easily reproduced by running test_maps. I
am also writing a selftest patch for it.  I could post the patch and the
selftest patch if you are OK with it.
>
> diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
> index b14b87463ee04..1d8d09fdd2da5 100644
> --- a/kernel/bpf/hashtab.c
> +++ b/kernel/bpf/hashtab.c
> @@ -824,13 +824,14 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node)
>  	hlist_nulls_for_each_entry_rcu(l, n, head, hash_node)
>  		if (l == tgt_l) {
>  			hlist_nulls_del_rcu(&l->hash_node);
> -			check_and_free_fields(htab, l);
>  			bpf_map_dec_elem_count(&htab->map);
>  			break;
>  		}
>  
>  	htab_unlock_bucket(htab, b, tgt_l->hash, flags);
>  
> +	if (l == tgt_l)
> +		check_and_free_fields(htab, l);
>  	return l == tgt_l;
>  }
>  
> @@ -1181,14 +1182,18 @@ static long htab_map_update_elem(struct bpf_map *map, void *key, void *value,
>  	 * concurrent search will find it before old elem
>  	 */
>  	hlist_nulls_add_head_rcu(&l_new->hash_node, head);
> -	if (l_old) {
> +	if (l_old)
>  		hlist_nulls_del_rcu(&l_old->hash_node);
> +	htab_unlock_bucket(htab, b, hash, flags);
> +
> +	if (l_old) {
>  		if (!htab_is_prealloc(htab))
>  			free_htab_elem(htab, l_old);
>  		else
>  			check_and_free_fields(htab, l_old);
>  	}
> -	ret = 0;
> +	return 0;
> +
>  err:
>  	htab_unlock_bucket(htab, b, hash, flags);
>  	return ret;
> @@ -1433,14 +1438,15 @@ static long htab_map_delete_elem(struct bpf_map *map, void *key)
>  
>  	l = lookup_elem_raw(head, hash, key, key_size);
>  
> -	if (l) {
> +	if (l)
>  		hlist_nulls_del_rcu(&l->hash_node);
> -		free_htab_elem(htab, l);
> -	} else {
> +	else
>  		ret = -ENOENT;
> -	}
>  
>  	htab_unlock_bucket(htab, b, hash, flags);
> +
> +	if (l)
> +		free_htab_elem(htab, l);
>  	return ret;
>  }
>  
> @@ -1647,14 +1653,16 @@ static int __htab_map_lookup_and_delete_elem(struct bpf_map *map, void *key,
>  		}
>  
>  		hlist_nulls_del_rcu(&l->hash_node);
> -		if (!is_lru_map)
> -			free_htab_elem(htab, l);
>  	}
>  
>  	htab_unlock_bucket(htab, b, hash, bflags);
>  
> -	if (is_lru_map && l)
> -		htab_lru_push_free(htab, l);
> +	if (l) {
> +		if (is_lru_map)
> +			htab_lru_push_free(htab, l);
> +		else
> +			free_htab_elem(htab, l);
> +	}
>  
>  	return ret;
>  }
> @@ -1851,15 +1859,12 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
>  
>  			/* bpf_lru_push_free() will acquire lru_lock, which
>  			 * may cause deadlock. See comments in function
> -			 * prealloc_lru_pop(). Let us do bpf_lru_push_free()
> -			 * after releasing the bucket lock.
> +			 * prealloc_lru_pop(). htab_lru_push_free() may allocate
> +			 * sleeping locks. Let us do bpf_lru_push_free() after
> +			 * releasing the bucket lock.
>  			 */
> -			if (is_lru_map) {
> -				l->batch_flink = node_to_free;
> -				node_to_free = l;
> -			} else {
> -				free_htab_elem(htab, l);
> -			}
> +			l->batch_flink = node_to_free;
> +			node_to_free = l;
>  		}
>  		dst_key += key_size;
>  		dst_val += value_size;
> @@ -1871,7 +1876,10 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
>  	while (node_to_free) {
>  		l = node_to_free;
>  		node_to_free = node_to_free->batch_flink;
> -		htab_lru_push_free(htab, l);
> +		if (is_lru_map)
> +			htab_lru_push_free(htab, l);
> +		else
> +			free_htab_elem(htab, l);
>  	}
>  
>  next_batch:
>
> .