[syzbot] [jfs?] INFO: task hung in txBegin

[syzbot] [jfs?] INFO: task hung in txBegin

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    50736169ecc8 Merge tag 'for-6.10-rc4-tag' of git://git.ker..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=172777ee980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=12f98862a3c0c799
dashboard link: https://syzkaller.appspot.com/bug?extid=eda89a33c5856f66f823
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=17dccdda980000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=1644d08e980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/5cd6a6f1e8a6/disk-50736169.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/2fb1a68197bc/vmlinux-50736169.xz
kernel image: https://storage.googleapis.com/syzbot-assets/783059c8c714/bzImage-50736169.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/95d758e45724/mount_0.gz

Bisection is inconclusive: the issue happens on the oldest tested release.

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=13861201980000
final oops:     https://syzkaller.appspot.com/x/report.txt?x=10461201980000
console output: https://syzkaller.appspot.com/x/log.txt?x=17861201980000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+eda89a33c5856f66f823@syzkaller.appspotmail.com

INFO: task kworker/u8:11:2848 blocked for more than 143 seconds.
      Not tainted 6.10.0-rc4-syzkaller-00148-g50736169ecc8 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u8:11   state:D stack:23552 pid:2848  tgid:2848  ppid:2      flags:0x00004000
Workqueue: writeback wb_workfn (flush-7:0)
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5408 [inline]
 __schedule+0x1796/0x49d0 kernel/sched/core.c:6745
 __schedule_loop kernel/sched/core.c:6822 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6837
 io_schedule+0x8d/0x110 kernel/sched/core.c:9043
 TXN_SLEEP_DROP_LOCK fs/jfs/jfs_txnmgr.c:125 [inline]
 txBegin+0x260/0xce0 fs/jfs/jfs_txnmgr.c:374
 jfs_commit_inode+0x223/0x590 fs/jfs/inode.c:101
 jfs_write_inode+0x143/0x220 fs/jfs/inode.c:132
 write_inode fs/fs-writeback.c:1497 [inline]
 __writeback_single_inode+0x6b9/0x10b0 fs/fs-writeback.c:1716
 writeback_sb_inodes+0x99c/0x1380 fs/fs-writeback.c:1947
 wb_writeback+0x481/0xd40 fs/fs-writeback.c:2127
 wb_do_writeback fs/fs-writeback.c:2274 [inline]
 wb_workfn+0x410/0x1090 fs/fs-writeback.c:2314
 process_one_work kernel/workqueue.c:3231 [inline]
 process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
 worker_thread+0x86d/0xd70 kernel/workqueue.c:3393
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
INFO: task syz-executor117:5099 blocked for more than 143 seconds.
      Not tainted 6.10.0-rc4-syzkaller-00148-g50736169ecc8 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor117 state:D stack:19408 pid:5099  tgid:5099  ppid:5098   flags:0x00004002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5408 [inline]
 __schedule+0x1796/0x49d0 kernel/sched/core.c:6745
 __schedule_loop kernel/sched/core.c:6822 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6837
 wb_wait_for_completion+0x166/0x290 fs/fs-writeback.c:216
 __writeback_inodes_sb_nr+0x2d2/0x380 fs/fs-writeback.c:2706
 sync_filesystem+0xa4/0x230 fs/sync.c:54
 generic_shutdown_super+0x72/0x2d0 fs/super.c:621
 kill_block_super+0x44/0x90 fs/super.c:1676
 deactivate_locked_super+0xc4/0x130 fs/super.c:473
 cleanup_mnt+0x41f/0x4b0 fs/namespace.c:1267
 task_work_run+0x24f/0x310 kernel/task_work.c:180
 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:114 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0x168/0x370 kernel/entry/common.c:218
 do_syscall_64+0x100/0x230 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fe052acc017
RSP: 002b:00007ffd7811a4d8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a6
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fe052acc017
RDX: 0000000000000000 RSI: 0000000000000009 RDI: 00007ffd7811a590
RBP: 00007ffd7811a590 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000206 R12: 00007ffd7811b5f0
R13: 00005555816966c0 R14: 000000000000000f R15: 431bde82d7b634db
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/30:
 #0: ffffffff8e333fa0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:329 [inline]
 #0: ffffffff8e333fa0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:781 [inline]
 #0: ffffffff8e333fa0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x55/0x2a0 kernel/locking/lockdep.c:6614
2 locks held by kworker/u8:11/2848:
 #0: ffff88801ae86148 ((wq_completion)writeback){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
 #0: ffff88801ae86148 ((wq_completion)writeback){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
 #1: ffffc900098b7d00 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
 #1: ffffc900098b7d00 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
2 locks held by getty/4845:
 #0: ffff88802a5ab0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x25/0x70 drivers/tty/tty_ldisc.c:243
 #1: ffffc90002f162f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6b5/0x1e10 drivers/tty/n_tty.c:2211
1 lock held by syz-executor117/5099:
 #0: ffff88802c5e00e0 (&type->s_umount_key#46){+.+.}-{3:3}, at: __super_lock fs/super.c:56 [inline]
 #0: ffff88802c5e00e0 (&type->s_umount_key#46){+.+.}-{3:3}, at: __super_lock_excl fs/super.c:71 [inline]
 #0: ffff88802c5e00e0 (&type->s_umount_key#46){+.+.}-{3:3}, at: deactivate_super+0xb5/0xf0 fs/super.c:505

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 30 Comm: khungtaskd Not tainted 6.10.0-rc4-syzkaller-00148-g50736169ecc8 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
 nmi_cpu_backtrace+0x49c/0x4d0 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x198/0x320 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:162 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:223 [inline]
 watchdog+0xfde/0x1020 kernel/hung_task.c:379
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 51 Comm: kworker/u8:3 Not tainted 6.10.0-rc4-syzkaller-00148-g50736169ecc8 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:__sanitizer_cov_trace_const_cmp8+0x0/0x90 kernel/kcov.c:310
Code: 10 48 89 74 0a 18 4c 89 44 0a 20 c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 <f3> 0f 1e fa 4c 8b 04 24 65 48 8b 0c 25 80 d4 03 00 65 8b 05 f0 aa
RSP: 0018:ffffc90000bb7638 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 000ffffffffff000 RCX: ffff888015193c00
RDX: 000000001507c558 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc90000bb7730 R08: ffffffff81e6dded R09: 1ffffffff25edcb0
R10: dffffc0000000000 R11: fffffbfff25edcb1 R12: 1ffff92000176ed0
R13: 000000001507c067 R14: 0000000000000000 R15: 1ffff92000176ed8
FS:  0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005617e0440680 CR3: 000000000e132000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 pmd_pfn_mask arch/x86/include/asm/pgtable_types.h:453 [inline]
 pmd_pfn arch/x86/include/asm/pgtable.h:234 [inline]
 pte_lockptr include/linux/mm.h:2955 [inline]
 __pte_offset_map_lock+0x17b/0x300 mm/pgtable-generic.c:374
 get_locked_pte include/linux/mm.h:2744 [inline]
 __text_poke+0x2c5/0xd30 arch/x86/kernel/alternative.c:1883
 text_poke arch/x86/kernel/alternative.c:1968 [inline]
 text_poke_bp_batch+0x8cd/0xb30 arch/x86/kernel/alternative.c:2357
 text_poke_flush arch/x86/kernel/alternative.c:2470 [inline]
 text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2477
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate+0xb5/0x250 mm/kfence/core.c:826
 process_one_work kernel/workqueue.c:3231 [inline]
 process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
 worker_thread+0x86d/0xd70 kernel/workqueue.c:3393
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.555 msecs


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

Re: [syzbot] Re: [syzbot] [jfs?] INFO: task hung in txBegin

[syzbot]

For archival purposes, forwarding an incoming command email to
linux-kernel@vger.kernel.org.

***

Subject: Re: [syzbot] [jfs?] INFO: task hung in txBegin
Author: lizhi.xu@windriver.com

#syz test: upstream 50736169ecc8

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 92a5b8283528..0209ebe97e7f 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -2123,10 +2123,12 @@ static long wb_writeback(struct bdi_writeback *wb,
 			queue_io(wb, work, dirtied_before);
 			queued = true;
 		}
+		down_write(&work->sb->s_umount);
 		if (work->sb)
 			progress = writeback_sb_inodes(work->sb, wb, work);
 		else
 			progress = __writeback_inodes_wb(wb, work);
+		up_write(&work->sb->s_umount);
 		trace_writeback_written(wb, work);
 
 		/*

Re: [syzbot] Re: [syzbot] [jfs?] INFO: task hung in txBegin

[syzbot]

For archival purposes, forwarding an incoming command email to
linux-kernel@vger.kernel.org.

***

Subject: Re: [syzbot] [jfs?] INFO: task hung in txBegin
Author: lizhi.xu@windriver.com

#syz test: upstream 50736169ecc8

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 92a5b8283528..965bd3f109cc 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -2078,6 +2078,7 @@ static long wb_writeback(struct bdi_writeback *wb,
 	bool queued = false;
 
 	blk_start_plug(&plug);
+	down_write(&work->sb->s_umount);
 	for (;;) {
 		/*
 		 * Stop writeback when nr_pages has been consumed
@@ -2162,6 +2163,7 @@ static long wb_writeback(struct bdi_writeback *wb,
 		/* This function drops i_lock... */
 		inode_sleep_on_writeback(inode);
 	}
+	up_write(&work->sb->s_umount);
 	blk_finish_plug(&plug);
 
 	return nr_pages - work->nr_pages;

Re: [syzbot] Re: [syzbot] [jfs?] INFO: task hung in txBegin

[syzbot]

For archival purposes, forwarding an incoming command email to
linux-kernel@vger.kernel.org.

***

Subject: Re: [syzbot] [jfs?] INFO: task hung in txBegin
Author: lizhi.xu@windriver.com

#syz test: upstream 50736169ecc8

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 92a5b8283528..04ccc85c80b4 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -2076,8 +2076,11 @@ static long wb_writeback(struct bdi_writeback *wb,
 	long progress;
 	struct blk_plug plug;
 	bool queued = false;
+	struct super_block *sb = work->sb;
 
 	blk_start_plug(&plug);
+	if (sb)
+		down_write(&sb->s_umount);
 	for (;;) {
 		/*
 		 * Stop writeback when nr_pages has been consumed
@@ -2162,6 +2165,8 @@ static long wb_writeback(struct bdi_writeback *wb,
 		/* This function drops i_lock... */
 		inode_sleep_on_writeback(inode);
 	}
+	if (sb)
+		up_write(&sb->s_umount);
 	blk_finish_plug(&plug);
 
 	return nr_pages - work->nr_pages;

Re: [syzbot] [jfs?] INFO: task hung in txBegin

[Edward Adam Davis]

with sb rwsem lock to order syncfs and wb worker

#syz test: upstream 50736169ecc8

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 92a5b8283528..660a1ac8912f 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -2123,8 +2123,12 @@ static long wb_writeback(struct bdi_writeback *wb,
 			queue_io(wb, work, dirtied_before);
 			queued = true;
 		}
-		if (work->sb)
+		if (work->sb) {
+			printk("sb: %p, %s\n", work->sb, __func__);
+			down_write(&work->sb->s_umount);
 			progress = writeback_sb_inodes(work->sb, wb, work);
+			up_write(&work->sb->s_umount);
+		}
 		else
 			progress = __writeback_inodes_wb(wb, work);
 		trace_writeback_written(wb, work);
diff --git a/fs/sync.c b/fs/sync.c
index dc725914e1ed..111c3c29b139 100644
--- a/fs/sync.c
+++ b/fs/sync.c
@@ -51,6 +51,7 @@ int sync_filesystem(struct super_block *sb)
 	 * methods call sync_dirty_buffer() and thus effectively write one block
 	 * at a time.
 	 */
+	printk("sb: %p, %s\n", sb, __func__);
 	writeback_inodes_sb(sb, WB_REASON_SYNC);
 	if (sb->s_op->sync_fs) {
 		ret = sb->s_op->sync_fs(sb, 0);

Re: [syzbot] [jfs?] INFO: task hung in txBegin

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
BUG: sleeping function called from invalid context in wb_writeback

sb: ffff88802058c000, wb_writeback
BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:1578
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 2837, name: kworker/u8:7
preempt_count: 1, expected: 0
RCU nest depth: 0, expected: 0
3 locks held by kworker/u8:7/2837:
 #0: ffff8880186ea948 ((wq_completion)writeback){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
 #0: ffff8880186ea948 ((wq_completion)writeback){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
 #1: ffffc90009a67d00 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
 #1: ffffc90009a67d00 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
 #2: ffff888020b920d8 (&wb->list_lock){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:351 [inline]
 #2: ffff888020b920d8 (&wb->list_lock){+.+.}-{2:2}, at: wb_writeback+0x29e/0xdb0 fs/fs-writeback.c:2106
Preemption disabled at:
[<0000000000000000>] 0x0
CPU: 1 PID: 2837 Comm: kworker/u8:7 Not tainted 6.10.0-rc4-syzkaller-00148-g50736169ecc8-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
Workqueue: writeback wb_workfn (flush-7:0)
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
 __might_resched+0x5d4/0x780 kernel/sched/core.c:10196
 down_write+0x19/0x50 kernel/locking/rwsem.c:1578
 wb_writeback+0x836/0xdb0 fs/fs-writeback.c:2128
 wb_do_writeback fs/fs-writeback.c:2278 [inline]
 wb_workfn+0x410/0x1090 fs/fs-writeback.c:2318
 process_one_work kernel/workqueue.c:3231 [inline]
 process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
 worker_thread+0x86d/0xd70 kernel/workqueue.c:3393
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

=============================
[ BUG: Invalid wait context ]
6.10.0-rc4-syzkaller-00148-g50736169ecc8-dirty #0 Tainted: G        W         
-----------------------------
kworker/u8:7/2837 is trying to lock:
ffff88802058c0e0 (&type->s_umount_key#52){+.+.}-{3:3}, at: wb_writeback+0x836/0xdb0 fs/fs-writeback.c:2128
other info that might help us debug this:
context-{4:4}
3 locks held by kworker/u8:7/2837:
 #0: ffff8880186ea948 ((wq_completion)writeback){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
 #0: ffff8880186ea948 ((wq_completion)writeback){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
 #1: ffffc90009a67d00 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
 #1: ffffc90009a67d00 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
 #2: ffff888020b920d8 (&wb->list_lock){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:351 [inline]
 #2: ffff888020b920d8 (&wb->list_lock){+.+.}-{2:2}, at: wb_writeback+0x29e/0xdb0 fs/fs-writeback.c:2106
stack backtrace:
CPU: 1 PID: 2837 Comm: kworker/u8:7 Tainted: G        W          6.10.0-rc4-syzkaller-00148-g50736169ecc8-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
Workqueue: writeback wb_workfn (flush-7:0)
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
 print_lock_invalid_wait_context kernel/locking/lockdep.c:4751 [inline]
 check_wait_context kernel/locking/lockdep.c:4821 [inline]
 __lock_acquire+0x1507/0x1fd0 kernel/locking/lockdep.c:5087
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
 down_write+0x3a/0x50 kernel/locking/rwsem.c:1579
 wb_writeback+0x836/0xdb0 fs/fs-writeback.c:2128
 wb_do_writeback fs/fs-writeback.c:2278 [inline]
 wb_workfn+0x410/0x1090 fs/fs-writeback.c:2318
 process_one_work kernel/workqueue.c:3231 [inline]
 process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
 worker_thread+0x86d/0xd70 kernel/workqueue.c:3393
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
BUG: scheduling while atomic: kworker/u8:7/2837/0x00000002
INFO: lockdep is turned off.
Modules linked in:
Preemption disabled at:
[<0000000000000000>] 0x0


Tested on:

commit:         50736169 Merge tag 'for-6.10-rc4-tag' of git://git.ker..
git tree:       git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=1212f7a5980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=12f98862a3c0c799
dashboard link: https://syzkaller.appspot.com/bug?extid=eda89a33c5856f66f823
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch:          https://syzkaller.appspot.com/x/patch.diff?x=1442b72d980000

Re: [syzbot] [jfs?] INFO: task hung in txBegin

[Edward Adam Davis]

with sb rwsem lock to order syncfs and wb worker

#syz test: upstream 50736169ecc8

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index b865a3fa52f3..172ca9e1eed5 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -1865,6 +1865,7 @@ static long writeback_sb_inodes(struct super_block *sb,
 	long total_wrote = 0;  /* count both pages and inodes */
 	unsigned long dirtied_before = jiffies;
 
+	printk("path1, sb: %p, %s\n", sb, __func__);
 	if (work->for_kupdate)
 		dirtied_before = jiffies -
 			msecs_to_jiffies(dirty_expire_interval * 10);
@@ -1993,6 +1994,7 @@ static long writeback_sb_inodes(struct super_block *sb,
 				break;
 		}
 	}
+	printk("path1,end sb: %p, total_wrote:%lu, %s\n", sb, total_wrote, __func__);
 	return total_wrote;
 }
 
@@ -2720,6 +2722,7 @@ void writeback_inodes_sb_nr(struct super_block *sb,
 			    unsigned long nr,
 			    enum wb_reason reason)
 {
+	printk("path2: nr: %lu, sb: %p, %s\n", nr, sb, __func__);
 	__writeback_inodes_sb_nr(sb, nr, reason, false);
 }
 EXPORT_SYMBOL(writeback_inodes_sb_nr);

Re: [syzbot] [jfs?] INFO: task hung in txBegin

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
general protection fault in lmLogSync

Oops: general protection fault, probably for non-canonical address 0xdffffc0000000006: 0000 [#1] PREEMPT SMP KASAN PTI
KASAN: null-ptr-deref in range [0x0000000000000030-0x0000000000000037]
CPU: 1 PID: 114 Comm: jfsCommit Not tainted 6.10.0-rc4-syzkaller-00148-g50736169ecc8-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
RIP: 0010:write_special_inodes fs/jfs/jfs_logmgr.c:208 [inline]
RIP: 0010:lmLogSync+0x130/0xae0 fs/jfs/jfs_logmgr.c:935
Code: a6 fe 49 8d 5f f0 48 89 d8 48 c1 e8 03 42 80 3c 28 00 74 08 48 89 df e8 1e 97 d3 fe 48 8b 1b 48 83 c3 30 48 89 d8 48 c1 e8 03 <42> 80 3c 28 00 74 08 48 89 df e8 01 97 d3 fe 48 8b 3b e8 b9 c5 a6
RSP: 0018:ffffc90002cefc00 EFLAGS: 00010206
RAX: 0000000000000006 RBX: 0000000000000030 RCX: 328fe0b8e03e5b00
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc90002cefd18 R08: ffffffff81cbb49a R09: 0000000000000000
R10: ffffc90002cef9e8 R11: fffff5200059df6b R12: ffff88806a112800
R13: dffffc0000000000 R14: 0000000000000001 R15: ffff88807cb0ba38
FS:  0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055559488b938 CR3: 000000000e132000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 jfs_syncpt+0x7d/0xa0 fs/jfs/jfs_logmgr.c:1041
 txEnd+0x30f/0x560 fs/jfs/jfs_txnmgr.c:549
 txLazyCommit fs/jfs/jfs_txnmgr.c:2684 [inline]
 jfs_lazycommit+0x634/0xb80 fs/jfs/jfs_txnmgr.c:2733
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:write_special_inodes fs/jfs/jfs_logmgr.c:208 [inline]
RIP: 0010:lmLogSync+0x130/0xae0 fs/jfs/jfs_logmgr.c:935
Code: a6 fe 49 8d 5f f0 48 89 d8 48 c1 e8 03 42 80 3c 28 00 74 08 48 89 df e8 1e 97 d3 fe 48 8b 1b 48 83 c3 30 48 89 d8 48 c1 e8 03 <42> 80 3c 28 00 74 08 48 89 df e8 01 97 d3 fe 48 8b 3b e8 b9 c5 a6
RSP: 0018:ffffc90002cefc00 EFLAGS: 00010206
RAX: 0000000000000006 RBX: 0000000000000030 RCX: 328fe0b8e03e5b00
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc90002cefd18 R08: ffffffff81cbb49a R09: 0000000000000000
R10: ffffc90002cef9e8 R11: fffff5200059df6b R12: ffff88806a112800
R13: dffffc0000000000 R14: 0000000000000001 R15: ffff88807cb0ba38
FS:  0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055559488b938 CR3: 000000002bf6a000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
----------------
Code disassembly (best guess):
   0:	a6                   	cmpsb  %es:(%rdi),%ds:(%rsi)
   1:	fe 49 8d             	decb   -0x73(%rcx)
   4:	5f                   	pop    %rdi
   5:	f0 48 89 d8          	lock mov %rbx,%rax
   9:	48 c1 e8 03          	shr    $0x3,%rax
   d:	42 80 3c 28 00       	cmpb   $0x0,(%rax,%r13,1)
  12:	74 08                	je     0x1c
  14:	48 89 df             	mov    %rbx,%rdi
  17:	e8 1e 97 d3 fe       	call   0xfed3973a
  1c:	48 8b 1b             	mov    (%rbx),%rbx
  1f:	48 83 c3 30          	add    $0x30,%rbx
  23:	48 89 d8             	mov    %rbx,%rax
  26:	48 c1 e8 03          	shr    $0x3,%rax
* 2a:	42 80 3c 28 00       	cmpb   $0x0,(%rax,%r13,1) <-- trapping instruction
  2f:	74 08                	je     0x39
  31:	48 89 df             	mov    %rbx,%rdi
  34:	e8 01 97 d3 fe       	call   0xfed3973a
  39:	48 8b 3b             	mov    (%rbx),%rdi
  3c:	e8                   	.byte 0xe8
  3d:	b9                   	.byte 0xb9
  3e:	c5                   	.byte 0xc5
  3f:	a6                   	cmpsb  %es:(%rdi),%ds:(%rsi)


Tested on:

commit:         50736169 Merge tag 'for-6.10-rc4-tag' of git://git.ker..
git tree:       git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=16ca70f3980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=12f98862a3c0c799
dashboard link: https://syzkaller.appspot.com/bug?extid=eda89a33c5856f66f823
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch:          https://syzkaller.appspot.com/x/patch.diff?x=1688319d980000

Re: [syzbot] [jfs?] INFO: task hung in txBegin

[Edward Adam Davis]

with sb rwsem lock to order syncfs and wb worker

#syz test: upstream 50736169ecc8

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index b865a3fa52f3..682c5a237edc 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -985,6 +985,7 @@ static void bdi_split_work_to_wbs(struct backing_dev_info *bdi,
 	struct bdi_writeback *last_wb = NULL;
 	struct bdi_writeback *wb = list_entry(&bdi->wb_list,
 					      struct bdi_writeback, bdi_node);
+	bool no_queue = true;
 
 	might_sleep();
 restart:
@@ -1015,6 +1016,7 @@ static void bdi_split_work_to_wbs(struct backing_dev_info *bdi,
 			*work = *base_work;
 			work->nr_pages = nr_pages;
 			work->auto_free = 1;
+			no_queue = false;
 			wb_queue_work(wb, work);
 			continue;
 		}
@@ -1043,6 +1045,12 @@ static void bdi_split_work_to_wbs(struct backing_dev_info *bdi,
 		wb_wait_for_completion(&fallback_work_done);
 		goto restart;
 	}
+
+	if (no_queue) {
+		printk("no queue\n");
+		base_work->auto_free = 1;
+		finish_writeback_work(base_work);
+	}
 	rcu_read_unlock();
 
 	if (last_wb)

Re: [syzbot] [jfs?] INFO: task hung in txBegin

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
general protection fault in lmLogSync

Oops: general protection fault, probably for non-canonical address 0xdffffc0000000006: 0000 [#1] PREEMPT SMP KASAN PTI
KASAN: null-ptr-deref in range [0x0000000000000030-0x0000000000000037]
CPU: 1 PID: 112 Comm: jfsCommit Not tainted 6.10.0-rc4-syzkaller-00148-g50736169ecc8-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
RIP: 0010:write_special_inodes fs/jfs/jfs_logmgr.c:208 [inline]
RIP: 0010:lmLogSync+0x130/0xae0 fs/jfs/jfs_logmgr.c:935
Code: a6 fe 49 8d 5f f0 48 89 d8 48 c1 e8 03 42 80 3c 28 00 74 08 48 89 df e8 fe 96 d3 fe 48 8b 1b 48 83 c3 30 48 89 d8 48 c1 e8 03 <42> 80 3c 28 00 74 08 48 89 df e8 e1 96 d3 fe 48 8b 3b e8 99 c5 a6
RSP: 0018:ffffc90002c8fc00 EFLAGS: 00010206
RAX: 0000000000000006 RBX: 0000000000000030 RCX: cec33d79cc59c000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc90002c8fd18 R08: ffffffff81cbb49a R09: 0000000000000000
R10: ffffc90002c8f9e8 R11: fffff52000591f6b R12: ffff88807720a800
R13: dffffc0000000000 R14: 0000000000000001 R15: ffff88807dc3f238
FS:  0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0017f8000 CR3: 000000002f11c000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 jfs_syncpt+0x7d/0xa0 fs/jfs/jfs_logmgr.c:1041
 txEnd+0x30f/0x560 fs/jfs/jfs_txnmgr.c:549
 txLazyCommit fs/jfs/jfs_txnmgr.c:2684 [inline]
 jfs_lazycommit+0x634/0xb80 fs/jfs/jfs_txnmgr.c:2733
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:write_special_inodes fs/jfs/jfs_logmgr.c:208 [inline]
RIP: 0010:lmLogSync+0x130/0xae0 fs/jfs/jfs_logmgr.c:935
Code: a6 fe 49 8d 5f f0 48 89 d8 48 c1 e8 03 42 80 3c 28 00 74 08 48 89 df e8 fe 96 d3 fe 48 8b 1b 48 83 c3 30 48 89 d8 48 c1 e8 03 <42> 80 3c 28 00 74 08 48 89 df e8 e1 96 d3 fe 48 8b 3b e8 99 c5 a6
RSP: 0018:ffffc90002c8fc00 EFLAGS: 00010206
RAX: 0000000000000006 RBX: 0000000000000030 RCX: cec33d79cc59c000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc90002c8fd18 R08: ffffffff81cbb49a R09: 0000000000000000
R10: ffffc90002c8f9e8 R11: fffff52000591f6b R12: ffff88807720a800
R13: dffffc0000000000 R14: 0000000000000001 R15: ffff88807dc3f238
FS:  0000000000000000(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f43bc3ff000 CR3: 000000002f11c000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
----------------
Code disassembly (best guess):
   0:	a6                   	cmpsb  %es:(%rdi),%ds:(%rsi)
   1:	fe 49 8d             	decb   -0x73(%rcx)
   4:	5f                   	pop    %rdi
   5:	f0 48 89 d8          	lock mov %rbx,%rax
   9:	48 c1 e8 03          	shr    $0x3,%rax
   d:	42 80 3c 28 00       	cmpb   $0x0,(%rax,%r13,1)
  12:	74 08                	je     0x1c
  14:	48 89 df             	mov    %rbx,%rdi
  17:	e8 fe 96 d3 fe       	call   0xfed3971a
  1c:	48 8b 1b             	mov    (%rbx),%rbx
  1f:	48 83 c3 30          	add    $0x30,%rbx
  23:	48 89 d8             	mov    %rbx,%rax
  26:	48 c1 e8 03          	shr    $0x3,%rax
* 2a:	42 80 3c 28 00       	cmpb   $0x0,(%rax,%r13,1) <-- trapping instruction
  2f:	74 08                	je     0x39
  31:	48 89 df             	mov    %rbx,%rdi
  34:	e8 e1 96 d3 fe       	call   0xfed3971a
  39:	48 8b 3b             	mov    (%rbx),%rdi
  3c:	e8                   	.byte 0xe8
  3d:	99                   	cltd
  3e:	c5                   	.byte 0xc5
  3f:	a6                   	cmpsb  %es:(%rdi),%ds:(%rsi)


Tested on:

commit:         50736169 Merge tag 'for-6.10-rc4-tag' of git://git.ker..
git tree:       git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=1500d4d3980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=12f98862a3c0c799
dashboard link: https://syzkaller.appspot.com/bug?extid=eda89a33c5856f66f823
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch:          https://syzkaller.appspot.com/x/patch.diff?x=13a018f3980000