All of lore.kernel.org
 help / color / mirror / Atom feed
From: syzbot <syzbot+5218c85078236fc46227@syzkaller.appspotmail.com>
To: boqun.feng@gmail.com, hdanton@sina.com,
	linux-block@vger.kernel.org,  linux-kernel@vger.kernel.org,
	ming.lei@redhat.com,  syzkaller-bugs@googlegroups.com
Subject: Re: [syzbot] [block?] possible deadlock in blk_mq_submit_bio
Date: Tue, 10 Dec 2024 03:12:02 -0800	[thread overview]
Message-ID: <67582202.050a0220.a30f1.01cb.GAE@google.com> (raw)
In-Reply-To: <20241210104439.1770-1-hdanton@sina.com>

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
possible deadlock in __submit_bio

======================================================
WARNING: possible circular locking dependency detected
6.13.0-rc1-syzkaller-00011-gc018ec9dd144 #0 Not tainted
------------------------------------------------------
syz.0.15/7623 is trying to acquire lock:
ffff0000ca7b1de8 (&q->q_usage_counter(io)#17){++++}-{0:0}, at: __submit_bio+0x1a0/0x4f8 block/blk-core.c:629

but task is already holding lock:
ffff0000d771a0b0 (&tree->tree_lock){+.+.}-{4:4}, at: hfsplus_find_init+0x144/0x1bc fs/hfsplus/bfind.c:28

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (&tree->tree_lock){+.+.}-{4:4}:
       __mutex_lock_common+0x218/0x28f4 kernel/locking/mutex.c:585
       __mutex_lock kernel/locking/mutex.c:735 [inline]
       mutex_lock_nested+0x2c/0x38 kernel/locking/mutex.c:787
       hfsplus_find_init+0x144/0x1bc fs/hfsplus/bfind.c:28
       hfsplus_cat_write_inode+0x1a4/0xd48 fs/hfsplus/inode.c:589
       hfsplus_write_inode+0x15c/0x4dc fs/hfsplus/super.c:161
       write_inode fs/fs-writeback.c:1525 [inline]
       __writeback_single_inode+0x5a0/0x15a4 fs/fs-writeback.c:1745
       writeback_single_inode+0x18c/0x554 fs/fs-writeback.c:1801
       sync_inode_metadata+0xc4/0x12c fs/fs-writeback.c:2871
       hfsplus_file_fsync+0xe4/0x4c8 fs/hfsplus/inode.c:316
       vfs_fsync_range fs/sync.c:187 [inline]
       vfs_fsync+0x154/0x18c fs/sync.c:201
       __loop_update_dio+0x248/0x420 drivers/block/loop.c:204
       loop_set_status+0x538/0x7f4 drivers/block/loop.c:1289
       lo_ioctl+0xf10/0x1c48
       blkdev_ioctl+0x3a8/0xa8c block/ioctl.c:693
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:906 [inline]
       __se_sys_ioctl fs/ioctl.c:892 [inline]
       __arm64_sys_ioctl+0x14c/0x1cc fs/ioctl.c:892
       __invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
       el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
       do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
       el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
       el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
       el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600

-> #1 (&sb->s_type->i_mutex_key#20){+.+.}-{4:4}:
       down_write+0x50/0xc0 kernel/locking/rwsem.c:1577
       inode_lock include/linux/fs.h:818 [inline]
       hfsplus_file_fsync+0xd8/0x4c8 fs/hfsplus/inode.c:311
       vfs_fsync_range fs/sync.c:187 [inline]
       vfs_fsync+0x154/0x18c fs/sync.c:201
       __loop_update_dio+0x248/0x420 drivers/block/loop.c:204
       loop_set_status+0x538/0x7f4 drivers/block/loop.c:1289
       lo_ioctl+0xf10/0x1c48
       blkdev_ioctl+0x3a8/0xa8c block/ioctl.c:693
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:906 [inline]
       __se_sys_ioctl fs/ioctl.c:892 [inline]
       __arm64_sys_ioctl+0x14c/0x1cc fs/ioctl.c:892
       __invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
       el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
       do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
       el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
       el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
       el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600

-> #0 (&q->q_usage_counter(io)#17){++++}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3161 [inline]
       check_prevs_add kernel/locking/lockdep.c:3280 [inline]
       validate_chain kernel/locking/lockdep.c:3904 [inline]
       __lock_acquire+0x34f0/0x7904 kernel/locking/lockdep.c:5226
       lock_acquire+0x23c/0x724 kernel/locking/lockdep.c:5849
       bio_queue_enter block/blk.h:75 [inline]
       blk_mq_submit_bio+0x1254/0x2070 block/blk-mq.c:3093
       __submit_bio+0x1a0/0x4f8 block/blk-core.c:629
       __submit_bio_noacct_mq block/blk-core.c:710 [inline]
       submit_bio_noacct_nocheck+0x3bc/0xcbc block/blk-core.c:739
       submit_bio_noacct+0xc6c/0x166c block/blk-core.c:868
       submit_bio+0x374/0x564 block/blk-core.c:910
       submit_bh_wbc+0x3f8/0x4c8 fs/buffer.c:2814
       submit_bh fs/buffer.c:2819 [inline]
       block_read_full_folio+0x7d0/0x950 fs/buffer.c:2446
       hfsplus_read_folio+0x28/0x38 fs/hfsplus/inode.c:28
       filemap_read_folio+0x108/0x318 mm/filemap.c:2366
       do_read_cache_folio+0x368/0x5c0 mm/filemap.c:3826
       do_read_cache_page mm/filemap.c:3892 [inline]
       read_cache_page+0x6c/0x15c mm/filemap.c:3901
       read_mapping_page include/linux/pagemap.h:1005 [inline]
       __hfs_bnode_create+0x3dc/0x6d4 fs/hfsplus/bnode.c:440
       hfsplus_bnode_find+0x200/0xe60 fs/hfsplus/bnode.c:486
       hfsplus_brec_find+0x134/0x4a0 fs/hfsplus/bfind.c:172
       hfsplus_brec_read+0x38/0x128 fs/hfsplus/bfind.c:211
       hfsplus_find_cat+0x140/0x4a0 fs/hfsplus/catalog.c:202
       hfsplus_iget+0x34c/0x584 fs/hfsplus/super.c:83
       hfsplus_fill_super+0xa5c/0x16f8 fs/hfsplus/super.c:504
       get_tree_bdev_flags+0x38c/0x494 fs/super.c:1636
       get_tree_bdev+0x2c/0x3c fs/super.c:1659
       hfsplus_get_tree+0x28/0x38 fs/hfsplus/super.c:640
       vfs_get_tree+0x90/0x28c fs/super.c:1814
       do_new_mount+0x278/0x900 fs/namespace.c:3507
       path_mount+0x590/0xe04 fs/namespace.c:3834
       do_mount fs/namespace.c:3847 [inline]
       __do_sys_mount fs/namespace.c:4057 [inline]
       __se_sys_mount fs/namespace.c:4034 [inline]
       __arm64_sys_mount+0x4d4/0x5ac fs/namespace.c:4034
       __invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
       el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
       do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
       el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
       el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
       el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600

other info that might help us debug this:

Chain exists of:
  &q->q_usage_counter(io)#17 --> &sb->s_type->i_mutex_key#20 --> &tree->tree_lock

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&tree->tree_lock);
                               lock(&sb->s_type->i_mutex_key#20);
                               lock(&tree->tree_lock);
  rlock(&q->q_usage_counter(io)#17);

 *** DEADLOCK ***

2 locks held by syz.0.15/7623:
 #0: ffff0000cb7e60e0 (&type->s_umount_key#51/1){+.+.}-{4:4}, at: alloc_super+0x1b0/0x834 fs/super.c:344
 #1: ffff0000d771a0b0 (&tree->tree_lock){+.+.}-{4:4}, at: hfsplus_find_init+0x144/0x1bc fs/hfsplus/bfind.c:28

stack backtrace:
CPU: 1 UID: 0 PID: 7623 Comm: syz.0.15 Not tainted 6.13.0-rc1-syzkaller-00011-gc018ec9dd144 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call trace:
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:484 (C)
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0xe4/0x150 lib/dump_stack.c:120
 dump_stack+0x1c/0x28 lib/dump_stack.c:129
 print_circular_bug+0x154/0x1c0 kernel/locking/lockdep.c:2074
 check_noncircular+0x310/0x404 kernel/locking/lockdep.c:2206
 check_prev_add kernel/locking/lockdep.c:3161 [inline]
 check_prevs_add kernel/locking/lockdep.c:3280 [inline]
 validate_chain kernel/locking/lockdep.c:3904 [inline]
 __lock_acquire+0x34f0/0x7904 kernel/locking/lockdep.c:5226
 lock_acquire+0x23c/0x724 kernel/locking/lockdep.c:5849
 bio_queue_enter block/blk.h:75 [inline]
 blk_mq_submit_bio+0x1254/0x2070 block/blk-mq.c:3093
 __submit_bio+0x1a0/0x4f8 block/blk-core.c:629
 __submit_bio_noacct_mq block/blk-core.c:710 [inline]
 submit_bio_noacct_nocheck+0x3bc/0xcbc block/blk-core.c:739
 submit_bio_noacct+0xc6c/0x166c block/blk-core.c:868
 submit_bio+0x374/0x564 block/blk-core.c:910
 submit_bh_wbc+0x3f8/0x4c8 fs/buffer.c:2814
 submit_bh fs/buffer.c:2819 [inline]
 block_read_full_folio+0x7d0/0x950 fs/buffer.c:2446
 hfsplus_read_folio+0x28/0x38 fs/hfsplus/inode.c:28
 filemap_read_folio+0x108/0x318 mm/filemap.c:2366
 do_read_cache_folio+0x368/0x5c0 mm/filemap.c:3826
 do_read_cache_page mm/filemap.c:3892 [inline]
 read_cache_page+0x6c/0x15c mm/filemap.c:3901
 read_mapping_page include/linux/pagemap.h:1005 [inline]
 __hfs_bnode_create+0x3dc/0x6d4 fs/hfsplus/bnode.c:440
 hfsplus_bnode_find+0x200/0xe60 fs/hfsplus/bnode.c:486
 hfsplus_brec_find+0x134/0x4a0 fs/hfsplus/bfind.c:172
 hfsplus_brec_read+0x38/0x128 fs/hfsplus/bfind.c:211
 hfsplus_find_cat+0x140/0x4a0 fs/hfsplus/catalog.c:202
 hfsplus_iget+0x34c/0x584 fs/hfsplus/super.c:83
 hfsplus_fill_super+0xa5c/0x16f8 fs/hfsplus/super.c:504
 get_tree_bdev_flags+0x38c/0x494 fs/super.c:1636
 get_tree_bdev+0x2c/0x3c fs/super.c:1659
 hfsplus_get_tree+0x28/0x38 fs/hfsplus/super.c:640
 vfs_get_tree+0x90/0x28c fs/super.c:1814
 do_new_mount+0x278/0x900 fs/namespace.c:3507
 path_mount+0x590/0xe04 fs/namespace.c:3834
 do_mount fs/namespace.c:3847 [inline]
 __do_sys_mount fs/namespace.c:4057 [inline]
 __se_sys_mount fs/namespace.c:4034 [inline]
 __arm64_sys_mount+0x4d4/0x5ac fs/namespace.c:4034
 __invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
 el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
 do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
 el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
 el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
 el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
loop0: detected capacity change from 1024 to 3
Dev loop0: unable to read RDB block 3
 loop0: unable to read partition table
loop0: partition table beyond EOD, truncated
loop_reread_partitions: partition scan of loop0 (�Rt�\v*�3\f!6{\x06bO�0�\x7f�\x17.�Qʝ�\x03�	H�"Uqd\�'�Lz�8�\b���w1�A\bH��\x10�\x19��) failed (rc=-5)


Tested on:

commit:         c018ec9d block: rnull: Initialize the module in place
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git for-6.14/block
console output: https://syzkaller.appspot.com/x/log.txt?x=124c68f8580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=bd60186d08e947a5
dashboard link: https://syzkaller.appspot.com/bug?extid=5218c85078236fc46227
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
userspace arch: arm64

Note: no patches were applied.

  reply	other threads:[~2024-12-10 11:12 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-23 15:37 [syzbot] [block?] possible deadlock in blk_mq_submit_bio syzbot
2024-11-23 23:59 ` Hillf Danton
2024-11-27  5:59 ` Ming Lei
2024-11-27  5:59   ` syzbot
2024-12-09 19:19 ` syzbot
2024-12-10 10:44   ` Hillf Danton
2024-12-10 11:12     ` syzbot [this message]
2024-12-10 11:17     ` Ming Lei

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=67582202.050a0220.a30f1.01cb.GAE@google.com \
    --to=syzbot+5218c85078236fc46227@syzkaller.appspotmail.com \
    --cc=boqun.feng@gmail.com \
    --cc=hdanton@sina.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ming.lei@redhat.com \
    --cc=syzkaller-bugs@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.