linux-block.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Lai, Yi" <yi1.lai@linux.intel.com>
To: Ming Lei <ming.lei@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
	linux-block@vger.kernel.org, Christoph Hellwig <hch@lst.de>,
	Peter Zijlstra <peterz@infradead.org>,
	Waiman Long <longman@redhat.com>,
	Boqun Feng <boqun.feng@gmail.com>, Ingo Molnar <mingo@redhat.com>,
	Will Deacon <will@kernel.org>,
	linux-kernel@vger.kernel.org,
	Bart Van Assche <bvanassche@acm.org>,
	yi1.lai@intel.com, syzkaller-bugs@googlegroups.com
Subject: Re: [PATCH V2 3/3] block: model freeze & enter queue as lock for supporting lockdep
Date: Wed, 30 Oct 2024 16:50:55 +0800	[thread overview]
Message-ID: <ZyHzb8ExdDG4b8lo@ly-workstation> (raw)
In-Reply-To: <ZyHchfaUe2cEzFMm@fedora>

On Wed, Oct 30, 2024 at 03:13:09PM +0800, Ming Lei wrote:
> On Wed, Oct 30, 2024 at 02:45:03PM +0800, Lai, Yi wrote:
> > Hi Ming,
> > 
> > Greetings!
> > 
> > I used Syzkaller and found that there is possible deadlock in __submit_bio in linux-next next-20241029.
> > 
> > After bisection and the first bad commit is:
> > "
> > f1be1788a32e block: model freeze & enter queue as lock for supporting lockdep
> > "
> > 
> > All detailed into can be found at:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio
> > Syzkaller repro code:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/repro.c
> > Syzkaller repro syscall steps:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/repro.prog
> > Syzkaller report:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/repro.report
> > Kconfig(make olddefconfig):
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/kconfig_origin
> > Bisect info:
> > https://github.com/laifryiee/syzkaller_logs/tree/main/241029_183511___submit_bio/bisect_info.log
> > bzImage:
> > https://github.com/laifryiee/syzkaller_logs/raw/refs/heads/main/241029_183511___submit_bio/bzImage_6fb2fa9805c501d9ade047fc511961f3273cdcb5
> > Issue dmesg:
> > https://github.com/laifryiee/syzkaller_logs/blob/main/241029_183511___submit_bio/6fb2fa9805c501d9ade047fc511961f3273cdcb5_dmesg.log
> > 
> > "
> > [   22.219103] 6.12.0-rc5-next-20241029-6fb2fa9805c5 #1 Not tainted
> > [   22.219512] ------------------------------------------------------
> > [   22.219827] repro/735 is trying to acquire lock:
> > [   22.220066] ffff888010f1a768 (&q->q_usage_counter(io)#25){++++}-{0:0}, at: __submit_bio+0x39f/0x550
> > [   22.220568] 
> > [   22.220568] but task is already holding lock:
> > [   22.220884] ffffffff872322a0 (fs_reclaim){+.+.}-{0:0}, at: __alloc_pages_slowpath.constprop.0+0x76b/0x21e0
> > [   22.221453] 
> > [   22.221453] which lock already depends on the new lock.
> > [   22.221453] 
> > [   22.221862] 
> > [   22.221862] the existing dependency chain (in reverse order) is:
> > [   22.222247] 
> > [   22.222247] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [   22.222630]        lock_acquire+0x80/0xb0
> > [   22.222920]        fs_reclaim_acquire+0x116/0x160
> > [   22.223244]        __kmalloc_cache_node_noprof+0x59/0x470
> > [   22.223528]        blk_mq_init_tags+0x79/0x1a0
> > [   22.223771]        blk_mq_alloc_map_and_rqs+0x1f4/0xdd0
> > [   22.224127]        blk_mq_init_sched+0x33d/0x6d0
> > [   22.224376]        elevator_init_mq+0x2b2/0x400
> 
> It should be addressed by the following patch:
> 
> https://lore.kernel.org/linux-block/ZyEGLdg744U_xBjp@fedora/
>

I have applied proposed fix patch on top of next-20241029. Issue can
still be reproduced.

It seems the dependency chain is different from Marek's log and mine.


> Thanks,
> Ming
> 

  reply	other threads:[~2024-10-30  8:51 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-10-25  0:37 [PATCH V2 0/3] block: model freeze/enter queue as lock for lockdep Ming Lei
2024-10-25  0:37 ` [PATCH V2 1/3] blk-mq: add non_owner variant of start_freeze/unfreeze queue APIs Ming Lei
2024-10-25  0:37 ` [PATCH V2 2/3] nvme: core: switch to non_owner variant of start_freeze/unfreeze queue Ming Lei
2024-10-25  0:37 ` [PATCH V2 3/3] block: model freeze & enter queue as lock for supporting lockdep Ming Lei
2024-10-29 11:13   ` Marek Szyprowski
2024-10-29 15:58     ` Ming Lei
2024-10-29 16:59       ` Marek Szyprowski
2024-11-12  8:36       ` Marek Szyprowski
2024-11-12 10:15         ` Ming Lei
2024-11-12 11:32           ` Marek Szyprowski
2024-11-12 11:48             ` Ming Lei
2024-10-30  6:45   ` Lai, Yi
2024-10-30  7:13     ` Ming Lei
2024-10-30  8:50       ` Lai, Yi [this message]
2024-10-30  9:50         ` Ming Lei
2024-10-30 10:39           ` Lai, Yi
2024-10-30 11:08             ` Ming Lei
2024-12-04  3:21               ` Lai, Yi
2024-12-04  3:30                 ` Ming Lei
2025-01-13 14:39                 ` Chris Bainbridge
2025-01-13 15:11                   ` Ming Lei
2025-01-13 15:33                     ` Chris Bainbridge
2025-01-13 15:52                       ` Jens Axboe
2025-01-13 15:23                   ` Chris Bainbridge
2024-10-26 13:15 ` [PATCH V2 0/3] block: model freeze/enter queue as lock for lockdep Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZyHzb8ExdDG4b8lo@ly-workstation \
    --to=yi1.lai@linux.intel.com \
    --cc=axboe@kernel.dk \
    --cc=boqun.feng@gmail.com \
    --cc=bvanassche@acm.org \
    --cc=hch@lst.de \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=longman@redhat.com \
    --cc=ming.lei@redhat.com \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=syzkaller-bugs@googlegroups.com \
    --cc=will@kernel.org \
    --cc=yi1.lai@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).