From: "Lai, Yi" <yi1.lai@linux.intel.com>
To: Ming Lei <ming.lei@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
linux-block@vger.kernel.org, Christoph Hellwig <hch@lst.de>,
Peter Zijlstra <peterz@infradead.org>,
Waiman Long <longman@redhat.com>,
Boqun Feng <boqun.feng@gmail.com>, Ingo Molnar <mingo@redhat.com>,
Will Deacon <will@kernel.org>,
linux-kernel@vger.kernel.org,
Bart Van Assche <bvanassche@acm.org>,
yi1.lai@intel.com, syzkaller-bugs@googlegroups.com
Subject: Re: [PATCH V2 3/3] block: model freeze & enter queue as lock for supporting lockdep
Date: Wed, 30 Oct 2024 16:50:55 +0800 [thread overview]
Message-ID: <ZyHzb8ExdDG4b8lo@ly-workstation> (raw)
In-Reply-To: <ZyHchfaUe2cEzFMm@fedora>
On Wed, Oct 30, 2024 at 03:13:09PM +0800, Ming Lei wrote:
> On Wed, Oct 30, 2024 at 02:45:03PM +0800, Lai, Yi wrote:
> > Hi Ming,
> >
> > Greetings!
> >
> > I used Syzkaller and found that there is possible deadlock in __submit_bio in linux-next next-20241029.
> >
> > After bisection and the first bad commit is:
> > "
> > f1be1788a32e block: model freeze & enter queue as lock for supporting lockdep
> > "
> >
> > All detailed into can be found at:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio
> > Syzkaller repro code:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/repro.c
> > Syzkaller repro syscall steps:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/repro.prog
> > Syzkaller report:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/repro.report
> > Kconfig(make olddefconfig):
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/kconfig_origin
> > Bisect info:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/bisect_info.log
> > bzImage:
> > https://github.com/laifryiee/syzkaller_logs/raw/refs/heads/main/241029_183511___submit_bio/bzImage_6fb2fa9805c501d9ade047fc511961f3273cdcb5
> > Issue dmesg:
> > https://github.com/laifryiee/syzkaller_logs/blob/main/241029_183511___submit_bio/6fb2fa9805c501d9ade047fc511961f3273cdcb5_dmesg.log
> >
> > "
> > [ 22.219103] 6.12.0-rc5-next-20241029-6fb2fa9805c5 #1 Not tainted
> > [ 22.219512] ------------------------------------------------------
> > [ 22.219827] repro/735 is trying to acquire lock:
> > [ 22.220066] ffff888010f1a768 (&q->q_usage_counter(io)#25){++++}-{0:0}, at: __submit_bio+0x39f/0x550
> > [ 22.220568]
> > [ 22.220568] but task is already holding lock:
> > [ 22.220884] ffffffff872322a0 (fs_reclaim){+.+.}-{0:0}, at: __alloc_pages_slowpath.constprop.0+0x76b/0x21e0
> > [ 22.221453]
> > [ 22.221453] which lock already depends on the new lock.
> > [ 22.221453]
> > [ 22.221862]
> > [ 22.221862] the existing dependency chain (in reverse order) is:
> > [ 22.222247]
> > [ 22.222247] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [ 22.222630] lock_acquire+0x80/0xb0
> > [ 22.222920] fs_reclaim_acquire+0x116/0x160
> > [ 22.223244] __kmalloc_cache_node_noprof+0x59/0x470
> > [ 22.223528] blk_mq_init_tags+0x79/0x1a0
> > [ 22.223771] blk_mq_alloc_map_and_rqs+0x1f4/0xdd0
> > [ 22.224127] blk_mq_init_sched+0x33d/0x6d0
> > [ 22.224376] elevator_init_mq+0x2b2/0x400
>
> It should be addressed by the following patch:
>
> https://lore.kernel.org/linux-block/ZyEGLdg744U_xBjp@fedora/
>
I have applied proposed fix patch on top of next-20241029. Issue can
still be reproduced.
It seems the dependency chain is different from Marek's log and mine.
> Thanks,
> Ming
>
next prev parent reply other threads:[~2024-10-30 8:51 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-10-25 0:37 [PATCH V2 0/3] block: model freeze/enter queue as lock for lockdep Ming Lei
2024-10-25 0:37 ` [PATCH V2 1/3] blk-mq: add non_owner variant of start_freeze/unfreeze queue APIs Ming Lei
2024-10-25 0:37 ` [PATCH V2 2/3] nvme: core: switch to non_owner variant of start_freeze/unfreeze queue Ming Lei
2024-10-25 0:37 ` [PATCH V2 3/3] block: model freeze & enter queue as lock for supporting lockdep Ming Lei
2024-10-29 11:13 ` Marek Szyprowski
2024-10-29 15:58 ` Ming Lei
2024-10-29 16:59 ` Marek Szyprowski
2024-11-12 8:36 ` Marek Szyprowski
2024-11-12 10:15 ` Ming Lei
2024-11-12 11:32 ` Marek Szyprowski
2024-11-12 11:48 ` Ming Lei
2024-10-30 6:45 ` Lai, Yi
2024-10-30 7:13 ` Ming Lei
2024-10-30 8:50 ` Lai, Yi [this message]
2024-10-30 9:50 ` Ming Lei
2024-10-30 10:39 ` Lai, Yi
2024-10-30 11:08 ` Ming Lei
2024-12-04 3:21 ` Lai, Yi
2024-12-04 3:30 ` Ming Lei
2025-01-13 14:39 ` Chris Bainbridge
2025-01-13 15:11 ` Ming Lei
2025-01-13 15:33 ` Chris Bainbridge
2025-01-13 15:52 ` Jens Axboe
2025-01-13 15:23 ` Chris Bainbridge
2024-10-26 13:15 ` [PATCH V2 0/3] block: model freeze/enter queue as lock for lockdep Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZyHzb8ExdDG4b8lo@ly-workstation \
--to=yi1.lai@linux.intel.com \
--cc=axboe@kernel.dk \
--cc=boqun.feng@gmail.com \
--cc=bvanassche@acm.org \
--cc=hch@lst.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=ming.lei@redhat.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=syzkaller-bugs@googlegroups.com \
--cc=will@kernel.org \
--cc=yi1.lai@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).