From: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
To: Logan Gunthorpe <logang@deltatee.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
Keith Busch <kbusch@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>,
Keith Busch <Keith.Busch@wdc.com>, Jens Axboe <axboe@kernel.dk>
Subject: Re: [PATCH v2] nvme: Use spin_lock_irq() when taking the ctrl->lock
Date: Thu, 13 Aug 2020 02:38:20 +0000 [thread overview]
Message-ID: <BYAPR04MB496599E8B5ECCEBEE3E461CE86430@BYAPR04MB4965.namprd04.prod.outlook.com> (raw)
In-Reply-To: 20200812232444.18310-1-logang@deltatee.com
On 8/12/20 16:25, Logan Gunthorpe wrote:
> When locking the ctrl->lock spinlock IRQs need to be disabled to avoid a
> dead lock. The new spin_lock() calls recently added produce the
> following lockdep warning when running the blktest nvme/003:
>
> ================================
> WARNING: inconsistent lock state
> --------------------------------
> inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
> ksoftirqd/2/22 [HC0[0]:SC1[1]:HE0:SE0] takes:
> ffff888276a8c4c0 (&ctrl->lock){+.?.}-{2:2}, at: nvme_keep_alive_end_io+0x50/0xc0
> {SOFTIRQ-ON-W} state was registered at:
> lock_acquire+0x164/0x500
> _raw_spin_lock+0x28/0x40
> nvme_get_effects_log+0x37/0x1c0
> nvme_init_identify+0x9e4/0x14f0
> nvme_reset_work+0xadd/0x2360
> process_one_work+0x66b/0xb70
> worker_thread+0x6e/0x6c0
> kthread+0x1e7/0x210
> ret_from_fork+0x22/0x30
> irq event stamp: 1449221
> hardirqs last enabled at (1449220): [<ffffffff81c58e69>] ktime_get+0xf9/0x140
> hardirqs last disabled at (1449221): [<ffffffff83129665>] _raw_spin_lock_irqsave+0x25/0x60
> softirqs last enabled at (1449210): [<ffffffff83400447>] __do_softirq+0x447/0x595
> softirqs last disabled at (1449215): [<ffffffff81b489b5>] run_ksoftirqd+0x35/0x50
>
> other info that might help us debug this:
> Possible unsafe locking scenario:
>
> CPU0
> ----
> lock(&ctrl->lock);
> <Interrupt>
> lock(&ctrl->lock);
>
> *** DEADLOCK ***
>
> no locks held by ksoftirqd/2/22.
>
> stack backtrace:
> CPU: 2 PID: 22 Comm: ksoftirqd/2 Not tainted 5.8.0-rc4-eid-vmlocalyes-dbg-00157-g7236657c6b3a #1450
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.12.0-1 04/01/2014
> Call Trace:
> dump_stack+0xc8/0x11a
> print_usage_bug.cold.63+0x235/0x23e
> mark_lock+0xa9c/0xcf0
> __lock_acquire+0xd9a/0x2b50
> lock_acquire+0x164/0x500
> _raw_spin_lock_irqsave+0x40/0x60
> nvme_keep_alive_end_io+0x50/0xc0
> blk_mq_end_request+0x158/0x210
> nvme_complete_rq+0x146/0x500
> nvme_loop_complete_rq+0x26/0x30 [nvme_loop]
> blk_done_softirq+0x187/0x1e0
> __do_softirq+0x118/0x595
> run_ksoftirqd+0x35/0x50
> smpboot_thread_fn+0x1d3/0x310
> kthread+0x1e7/0x210
> ret_from_fork+0x22/0x30
>
> Fixes: be93e87e7802 ("nvme: support for multiple Command Sets Supported and Effects log pages")
> Signed-off-by: Logan Gunthorpe<logang@deltatee.com>
Thanks for this fix, looks good.
Tested-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
prev parent reply other threads:[~2020-08-13 2:38 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-12 23:24 [PATCH v2] nvme: Use spin_lock_irq() when taking the ctrl->lock Logan Gunthorpe
2020-08-13 0:32 ` Keith Busch
2020-08-13 3:55 ` Logan Gunthorpe
2020-08-14 6:04 ` Sagi Grimberg
2020-08-13 2:38 ` Chaitanya Kulkarni [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BYAPR04MB496599E8B5ECCEBEE3E461CE86430@BYAPR04MB4965.namprd04.prod.outlook.com \
--to=chaitanya.kulkarni@wdc.com \
--cc=Keith.Busch@wdc.com \
--cc=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=logang@deltatee.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox