From: Hannes Reinecke <hare@kernel.org>
To: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>, Keith Busch <kbusch@kernel.org>,
linux-nvme@lists.infradead.org, Hannes Reinecke <hare@kernel.org>
Subject: [PATCH 3/4] nvme-multipath: check for NVME_NSHEAD_DISK_LIVE when selecting paths
Date: Mon, 2 Sep 2024 13:15:47 +0200 [thread overview]
Message-ID: <20240902111548.41430-4-hare@kernel.org> (raw)
In-Reply-To: <20240902111548.41430-1-hare@kernel.org>
During repetitive namespace remapping operations on the target the
namespace might have changed between the time the initial scan
was performed, and partition scan was invoked by device_add_disk()
in nvme_mpath_set_live(). We then end up with a stuck scanning process:
[<0>] folio_wait_bit_common+0x12a/0x310
[<0>] filemap_read_folio+0x97/0xd0
[<0>] do_read_cache_folio+0x108/0x390
[<0>] read_part_sector+0x31/0xa0
[<0>] read_lba+0xc5/0x160
[<0>] efi_partition+0xd9/0x8f0
[<0>] bdev_disk_changed+0x23d/0x6d0
[<0>] blkdev_get_whole+0x78/0xc0
[<0>] bdev_open+0x2c6/0x3b0
[<0>] bdev_file_open_by_dev+0xcb/0x120
[<0>] disk_scan_partitions+0x5d/0x100
[<0>] device_add_disk+0x402/0x420
[<0>] nvme_mpath_set_live+0x4f/0x1f0 [nvme_core]
[<0>] nvme_mpath_add_disk+0x107/0x120 [nvme_core]
[<0>] nvme_alloc_ns+0xac6/0xe60 [nvme_core]
[<0>] nvme_scan_ns+0x2dd/0x3e0 [nvme_core]
[<0>] nvme_scan_work+0x1a3/0x490 [nvme_core]
and another scanning process (triggered by AEN) trying to remove the namespaces:
[<0>] __synchronize_srcu+0x17c/0x1b0
[<0>] nvme_ns_remove+0x12e/0x210 [nvme_core]
[<0>] nvme_ns_remove_by_nsid+0x21/0x70 [nvme_core]
[<0>] nvme_scan_work+0x1b7/0x490 [nvme_core]
[<0>] process_scheduled_works+0x37d/0x6d0
none of which is able to make progress as the first process cannot submit I/O
(as all the namespace inforation is stale), and the second process is stalled
waiting on the srcu to be released by the first process.
This patch checks for the NVME_NSHEAD_DISK_LIVE bit when selecting a path,
causing I/O to be aborted and allowing the first process to make progress.
Signed-off-by: Hannes Reinecke <hare@kernel.org>
---
drivers/nvme/host/multipath.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 01dbbe866d61..f5f2e1667c64 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -407,6 +407,9 @@ static struct nvme_ns *nvme_numa_path(struct nvme_ns_head *head)
inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)
{
+ if (!test_bit(NVME_NSHEAD_DISK_LIVE, &head->flags))
+ return NULL;
+
switch (READ_ONCE(head->subsys->iopolicy)) {
case NVME_IOPOLICY_QD:
return nvme_queue_depth_path(head);
@@ -421,6 +424,9 @@ static bool nvme_available_path(struct nvme_ns_head *head)
{
struct nvme_ns *ns;
+ if (!test_bit(NVME_NSHEAD_DISK_LIVE, &head->flags))
+ return NULL;
+
list_for_each_entry_rcu(ns, &head->list, siblings) {
if (test_bit(NVME_CTRL_FAILFAST_EXPIRED, &ns->ctrl->flags))
continue;
--
2.35.3
next prev parent reply other threads:[~2024-09-02 11:16 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-02 11:15 [PATCH 0/4] nvme: NSHEAD_DISK_LIVE fixes Hannes Reinecke
2024-09-02 11:15 ` [PATCH 1/4] nvme-multipath: fixup typo when clearing DISK_LIVE Hannes Reinecke
2024-09-02 16:48 ` Sagi Grimberg
2024-09-02 11:15 ` [PATCH 2/4] nvme-multipath: clear 'NVME_NSHEAD_DISK_LIVE' bit on shutdown Hannes Reinecke
2024-09-02 16:51 ` Sagi Grimberg
2024-09-02 11:15 ` Hannes Reinecke [this message]
2024-09-02 16:53 ` [PATCH 3/4] nvme-multipath: check for NVME_NSHEAD_DISK_LIVE when selecting paths Sagi Grimberg
2024-09-02 11:15 ` [PATCH 4/4] nvme: remove existing namespace on ID mismatch Hannes Reinecke
2024-09-02 17:01 ` Sagi Grimberg
2024-09-03 12:30 ` Hannes Reinecke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240902111548.41430-4-hare@kernel.org \
--to=hare@kernel.org \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.