[PATCH v6 00/27] Native SCSI Multipath support

[PATCH v6 00/27] Native SCSI Multipath support

[John Garry]

NATIVE SCSI MULTIPATH:

This series introduces native SCSI multipath support. It is intended as
an alternative to dm-mpath.

This support aims to provide a multipath-enabled SCSI block device/
gendisk.

For a SCSI device to support native multipath, either of the following
conditions must be satisfied:
a. unique ID in VPD page 83 and ALUA support and scsi_multipath modparam
   enabled
b. unique ID in VPD page 83 and scsi_multipath_always modparam enabled

This series relies on reading sdev->access_state to get path information.
This path information would be provided by ALUA. ALUA support which does
not rely on device handlers has already been discussed at
https://lore.kernel.org/linux-scsi/7755e98f-5619-48ba-bcfc-b64eec930c40@oracle.com/
and support will be added in the next phase.

New classes of devices are added:
- scsi_mpath_device
- scsi_mpath_disk

These are required since a multipath scsi_device has no common scsi host.
An example of the sysfs files and directories for these new classes is as
follows:

$ ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/
total 0
-rw-r--r--    1 root     root          4096 Feb 25 11:59 iopolicy
drwxr-xr-x    2 root     root             0 Feb 25 11:59 multipath
drwxr-xr-x    2 root     root             0 Feb 25 11:59 power
lrwxrwxrwx    1 root     root             0 Feb 25 11:59 subsystem ->
../../../../class/scsi_mpath_device
-rw-r--r--    1 root     root          4096 Feb 25 11:58 uevent
-r--r--r--    1 root     root          4096 Feb 25 11:59 vpd_id
$ ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/multipath/
total 0
lrwxrwxrwx    1 root     root             0 Feb 25 11:59 8:0:0:0 ->
../../../../platform/host8/session1/target8:0:0/8:0:0:0
lrwxrwxrwx    1 root     root             0 Feb 25 11:59 9:0:0:0 ->
../../../../platform/host9/session2/target9:0:0/9:0:0:0
$ cat /sys/class/scsi_mpath_device/scsi_mpath_device0/vpd_id
naa.600140505200a986f0043c9afa1fd077
$ cat /sys/class/scsi_mpath_device/scsi_mpath_device0/iopolicy
numa
$

$ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/
total 0
drwxr-xr-x    2 root     root             0 Feb 25 12:00 power
drwxr-xr-x   11 root     root             0 Feb 25 11:58 sdc
lrwxrwxrwx    1 root     root             0 Feb 25 11:58 subsystem ->
../../../../class/scsi_mpath_disk
-rw-r--r--    1 root     root          4096 Feb 25 11:58 uevent
$ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/sdc/multipath/
total 0
lrwxrwxrwx    1 root     root             0 Feb 25 12:00 sdc:0 ->
../../../../../platform/host8/session1/target8:0:0/8:0:0:0/block/sdc:0
lrwxrwxrwx    1 root     root             0 Feb 25 12:00 sdc:1 ->
../../../../../platform/host9/session2/target9:0:0/9:0:0:0/block/sdc:1

$ ls -l /dev/sdc
brw-rw----    1 root     disk        8,  32 Feb 25 11:58 /dev/sdc

The scsi_device and scsi_disk classes otherwise remain unmodified.
However, the per-path block device is hidden in /dev/. Furthermore,
multipathed block devices have a new naming scheme, sdX:Y, where
X is the scsi multipath device index and Y is the path index.

No multipath sg support is added. We still have a per-path sg device.
Since the SCSI block device is multipath enabled, we can access
multipathed scsi_ioctl() through that block device.

For failover, we take the approach of cloning bio's and re-submitting them
in full (for failover errors).

LIBMULTIPATH:

This series introduces also libmultipath. It is essentially a refactoring
of NVME multipath support, so we can have a common library to also support
native SCSI multipath.

Much of the code is taken directly from the NVMe multipath code. However,
NVMe specifics are removed. A template structure is provided so the driver
-may provide callbacks for driver specifics, like ANA support for NVMe.

Important new structures introduced include:

- mpath_head
These contain much of the multipath-specific functionality from
nvme_ns_head, including a pointer to the gendisk structure and
a path SRCU-based array.

- mpath_device
This is the per-path structure, and contains much the same
multipath-specific functionality in nvme_ns

libmultipath provides functionality for path management, path selection,
data path, and failover handling.

Full series also available at
https://github.com/johnpgarry/linux/commits/scsi-multipath-v7.2-v6/

Differences to v5:
- deal with issues sashiko found in libmultipath:
 - include wait_bit.h
 - don't store error in mpath_head->disk
 - use check_mul_overflow() in mpath_can_remove_head()
 - use -ENOTTY in mpath_bdev_ioctl()
- deal with issues sashiko found in scsi mpath support:
 - check 0 returned from scsi_vpd_lun_id()
 - call flush_work() in scsi_mpath_device_iopolicy_store()
 - clear SCMD_MPATH_IO_STATS for !bio in scsi_mpath_start_request()
 - add checks for calling scsi_mpath_{start,end}_request()
 - tweak sd_mpath_dev_show()

Differences to v4:
- deal with issues sashiko/codex found in libmultipath:
 - SPDX spelling mistake
 - make driver set mpath_device->mpath_head
 - change delayed support to use unsigned int for delay
 - check for partition in bdev ioctl
 - drop zoned support for now
- deal with issues sashiko/codex found in scsi mpath support:
 - use ida destory in 2x places
 - set vpd_id earlier
 - iter through all possible mpath heads when searching
 - remove scsi mpath device later in __scsi_remove_device
 - call scsi_mpath_clear_path() outside state mutex
 - add revalidate paths caller to sd revalidate
 - fix sdev put in failover handling
 - use READ_ONCE() fo reading sdev state
 - fix scsi mpath start/end request
 - take scsi mpath head ref in scsi_mpath_remove_head_work()
 - fix sd_mpath_probe error path
 - fix sd_mpath_dev_show() for deleted multipath disk
- Add RB tag from Hannes (thanks!)

John Garry (27):
  libmultipath: Add initial framework
  libmultipath: Add basic gendisk support
  libmultipath: Add path selection support
  libmultipath: Add bio handling
  libmultipath: Add support for mpath_device management
  libmultipath: Add delayed removal support
  libmultipath: Add sysfs helpers
  libmultipath: Add support for block device IOCTL
  libmultipath: Add mpath_bdev_getgeo()
  libmultipath: Add mpath_bdev_get_unique_id()
  scsi-multipath: introduce basic SCSI device support
  scsi-multipath: introduce scsi_device head structure
  scsi-multipath: provide sysfs link from to scsi_device
  scsi-multipath: support iopolicy
  scsi-multipath: clone each bio
  scsi-multipath: clear path when device is blocked
  scsi-multipath: revalidate paths upon device unblock
  scsi-multipath: failover handling
  scsi-multipath: provide callbacks for path state
  scsi-multipath: add scsi_mpath_{start,end}_request()
  scsi-multipath: add delayed disk removal support
  scsi: sd: add multipath disk class
  scsi: sd: add multipath disk attr groups
  scsi: sd: support multipath disk
  scsi: sd: add mpath_dev file
  scsi: sd: add mpath_numa_nodes dev attribute
  scsi: sd: add mpath_queue_depth dev attribute

 drivers/scsi/Kconfig          |  10 +
 drivers/scsi/Makefile         |   1 +
 drivers/scsi/scsi.c           |   8 +-
 drivers/scsi/scsi_lib.c       |  13 +-
 drivers/scsi/scsi_multipath.c | 673 ++++++++++++++++++++++++
 drivers/scsi/scsi_scan.c      |   4 +
 drivers/scsi/scsi_sysfs.c     |  10 +
 drivers/scsi/sd.c             | 610 +++++++++++++++++++++-
 drivers/scsi/sd.h             |   3 +
 include/linux/multipath.h     | 156 ++++++
 include/scsi/scsi_cmnd.h      |   9 +-
 include/scsi/scsi_device.h    |   2 +
 include/scsi/scsi_driver.h    |   4 +
 include/scsi/scsi_host.h      |   4 +
 include/scsi/scsi_multipath.h | 119 +++++
 lib/Kconfig                   |   6 +
 lib/Makefile                  |   2 +
 lib/multipath.c               | 948 ++++++++++++++++++++++++++++++++++
 18 files changed, 2558 insertions(+), 24 deletions(-)
 create mode 100644 drivers/scsi/scsi_multipath.c
 create mode 100644 include/linux/multipath.h
 create mode 100644 include/scsi/scsi_multipath.h
 create mode 100644 lib/multipath.c

-- 
2.43.7

[PATCH v6 01/27] libmultipath: Add initial framework

[John Garry]

Add initial framework for libmultipath. libmultipath is a library for
multipath-capable block drivers, such as NVMe. The main function is to
support path management, path selection, and failover handling.

Basic support to add and remove the head structure - mpath_head - is
included.

This main purpose of this structure is to manage available paths and path
selection. It is quite similar to the multipath functionality in
nvme_ns_head. It also manages the multipath gendisk.

Each path is represented by the mpath_device structure. It should hold a
pointer to the per-path gendisk and also a list element for all siblings
of paths. For NVMe, there would be a mpath_device per nvme_ns.

All the libmultipath code is more or less taken from
drivers/nvme/host/multipath.c, which was originally authored by Christoph
Hellwig <hch@lst.de>.

Signed-off-by: John Garry <john.g.garry@oracle.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
---
 include/linux/multipath.h | 28 ++++++++++++++++
 lib/Kconfig               |  6 ++++
 lib/Makefile              |  2 ++
 lib/multipath.c           | 67 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 103 insertions(+)
 create mode 100644 include/linux/multipath.h
 create mode 100644 lib/multipath.c

diff --git a/include/linux/multipath.h b/include/linux/multipath.h
new file mode 100644
index 0000000000000..407d985cd31f5
--- /dev/null
+++ b/include/linux/multipath.h
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#ifndef _LIBMULTIPATH_H
+#define _LIBMULTIPATH_H
+
+#include <linux/blkdev.h>
+#include <linux/srcu.h>
+
+struct mpath_device {
+	struct list_head	siblings;
+	struct gendisk		*disk;
+};
+
+struct mpath_head {
+	struct srcu_struct	srcu;
+	struct list_head	dev_list;	/* list of all mpath_devs */
+	struct mutex		lock;
+
+	refcount_t		refcount;
+
+	struct mpath_device __rcu 		*current_path[MAX_NUMNODES];
+};
+
+int mpath_get_head(struct mpath_head *mpath_head);
+void mpath_put_head(struct mpath_head *mpath_head);
+int mpath_head_init(struct mpath_head *mpath_head);
+void mpath_head_uninit(struct mpath_head *mpath_head);
+
+#endif // _LIBMULTIPATH_H
diff --git a/lib/Kconfig b/lib/Kconfig
index 55748b68714e0..d0258bef374a1 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -636,3 +636,9 @@ config UNION_FIND
 
 config MIN_HEAP
 	bool
+
+config LIBMULTIPATH
+	bool "MULTIPATH BLOCK DRIVER LIBRARY"
+	depends on BLOCK
+	help
+	  If you say yes here then you get a multipath lib for block drivers
diff --git a/lib/Makefile b/lib/Makefile
index 7f75cc6edf94a..7ba5e13be4171 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -334,3 +334,5 @@ CONTEXT_ANALYSIS_test_context-analysis.o := y
 obj-$(CONFIG_CONTEXT_ANALYSIS_TEST) += test_context-analysis.o
 
 subdir-$(CONFIG_FORTIFY_SOURCE) += test_fortify
+
+obj-$(CONFIG_LIBMULTIPATH)	+= multipath.o
diff --git a/lib/multipath.c b/lib/multipath.c
new file mode 100644
index 0000000000000..f7b26c66eac7c
--- /dev/null
+++ b/lib/multipath.c
@@ -0,0 +1,67 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2017-2018 Christoph Hellwig.
+ * Copyright (c) 2026 Oracle and/or its affiliates.
+ */
+#include <linux/module.h>
+#include <linux/multipath.h>
+#include <linux/wait_bit.h>
+
+static struct workqueue_struct *mpath_wq;
+
+int mpath_get_head(struct mpath_head *mpath_head)
+{
+	if (!refcount_inc_not_zero(&mpath_head->refcount))
+		return -ENXIO;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(mpath_get_head);
+
+void mpath_put_head(struct mpath_head *mpath_head)
+{
+	refcount_t *refcount = &mpath_head->refcount;
+
+	if (refcount_dec_and_test(&mpath_head->refcount))
+		wake_up_var(refcount);
+}
+EXPORT_SYMBOL_GPL(mpath_put_head);
+
+void mpath_head_uninit(struct mpath_head *mpath_head)
+{
+	refcount_t *refcount = &mpath_head->refcount;
+
+	if (!refcount_dec_and_test(refcount))
+		wait_var_event(refcount, !refcount_read(refcount));
+	cleanup_srcu_struct(&mpath_head->srcu);
+}
+EXPORT_SYMBOL_GPL(mpath_head_uninit);
+
+int mpath_head_init(struct mpath_head *mpath_head)
+{
+	memset(mpath_head, 0, sizeof(*mpath_head));
+	INIT_LIST_HEAD(&mpath_head->dev_list);
+	mutex_init(&mpath_head->lock);
+	refcount_set(&mpath_head->refcount, 1);
+
+	return init_srcu_struct(&mpath_head->srcu);
+}
+EXPORT_SYMBOL_GPL(mpath_head_init);
+
+static int __init mpath_init(void)
+{
+	mpath_wq = alloc_workqueue("mpath-wq",
+			WQ_UNBOUND | WQ_MEM_RECLAIM | WQ_SYSFS, 0);
+	if (!mpath_wq)
+		return -ENOMEM;
+	return 0;
+}
+
+static void __exit mpath_exit(void)
+{
+	destroy_workqueue(mpath_wq);
+}
+
+module_init(mpath_init);
+module_exit(mpath_exit);
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("libmultipath");
-- 
2.43.7

[PATCH v6 02/27] libmultipath: Add basic gendisk support

[John Garry]

Add support to allocate and free a multipath gendisk.

NVMe has almost like-for-like equivalents here:
- mpath_alloc_head_disk() -> nvme_mpath_alloc_disk()
- multipath_partition_scan_work() -> nvme_partition_scan_work()
- mpath_remove_disk() -> nvme_remove_head()
- mpath_device_set_live() -> nvme_mpath_set_live()

struct mpath_head_template is introduced as a method for drivers to
provide custom multipath functionality.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 include/linux/multipath.h |  38 ++++++++++++++
 lib/multipath.c           | 104 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 142 insertions(+)

diff --git a/include/linux/multipath.h b/include/linux/multipath.h
index 407d985cd31f5..ca6e02505be43 100644
--- a/include/linux/multipath.h
+++ b/include/linux/multipath.h
@@ -5,11 +5,19 @@
 #include <linux/blkdev.h>
 #include <linux/srcu.h>
 
+extern const struct block_device_operations mpath_ops;
+
 struct mpath_device {
+	struct mpath_head	*mpath_head;
 	struct list_head	siblings;
 	struct gendisk		*disk;
 };
 
+struct mpath_head_template {
+};
+
+#define MPATH_HEAD_DISK_LIVE 			0
+
 struct mpath_head {
 	struct srcu_struct	srcu;
 	struct list_head	dev_list;	/* list of all mpath_devs */
@@ -17,12 +25,42 @@ struct mpath_head {
 
 	refcount_t		refcount;
 
+	unsigned long		flags;
+	struct gendisk		*disk;
+	struct work_struct	partition_scan_work;
+	struct device		*parent;
+	const struct attribute_group 		**disk_groups;
+	const struct mpath_head_template	*mpdt;
 	struct mpath_device __rcu 		*current_path[MAX_NUMNODES];
 };
 
+static inline struct mpath_head *mpath_bd_device_to_head(struct device *dev)
+{
+	return dev_get_drvdata(dev);
+}
+
+static inline struct mpath_head *mpath_gendisk_to_head(struct gendisk *disk)
+{
+	return mpath_bd_device_to_head(disk_to_dev(disk));
+}
+
 int mpath_get_head(struct mpath_head *mpath_head);
 void mpath_put_head(struct mpath_head *mpath_head);
 int mpath_head_init(struct mpath_head *mpath_head);
 void mpath_head_uninit(struct mpath_head *mpath_head);
 
+void mpath_put_disk(struct mpath_head *mpath_head);
+void mpath_remove_disk(struct mpath_head *mpath_head);
+int mpath_alloc_head_disk(struct mpath_head *mpath_head,
+			struct queue_limits *lim, int numa_node);
+void mpath_device_set_live(struct mpath_device *mpath_device);
+
+static inline bool is_mpath_disk(struct gendisk *disk)
+{
+	#if IS_ENABLED(CONFIG_LIBMULTIPATH)
+	return disk->fops == &mpath_ops;
+	#else
+	return false;
+	#endif
+}
 #endif // _LIBMULTIPATH_H
diff --git a/lib/multipath.c b/lib/multipath.c
index f7b26c66eac7c..1cf20b0f62d07 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -36,6 +36,107 @@ void mpath_head_uninit(struct mpath_head *mpath_head)
 }
 EXPORT_SYMBOL_GPL(mpath_head_uninit);
 
+static int mpath_bdev_open(struct gendisk *disk, blk_mode_t mode)
+{
+	struct mpath_head *mpath_head = disk->private_data;
+
+	return mpath_get_head(mpath_head);
+}
+
+static void mpath_bdev_release(struct gendisk *disk)
+{
+	struct mpath_head *mpath_head = disk->private_data;
+
+	mpath_put_head(mpath_head);
+}
+
+const struct block_device_operations mpath_ops = {
+	.owner          = THIS_MODULE,
+	.open		= mpath_bdev_open,
+	.release	= mpath_bdev_release,
+};
+EXPORT_SYMBOL_GPL(mpath_ops);
+
+static void multipath_partition_scan_work(struct work_struct *work)
+{
+	struct mpath_head *mpath_head =
+		container_of(work, struct mpath_head, partition_scan_work);
+
+	if (WARN_ON_ONCE(!test_and_clear_bit(GD_SUPPRESS_PART_SCAN,
+					     &mpath_head->disk->state)))
+		return;
+
+	mutex_lock(&mpath_head->disk->open_mutex);
+	bdev_disk_changed(mpath_head->disk, false);
+	mutex_unlock(&mpath_head->disk->open_mutex);
+}
+
+void mpath_remove_disk(struct mpath_head *mpath_head)
+{
+	if (test_and_clear_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags)) {
+		struct gendisk *disk = mpath_head->disk;
+
+		del_gendisk(disk);
+	}
+}
+EXPORT_SYMBOL_GPL(mpath_remove_disk);
+
+void mpath_put_disk(struct mpath_head *mpath_head)
+{
+	if (!mpath_head->disk)
+		return;
+
+	/* make sure all pending bios are cleaned up */
+	flush_work(&mpath_head->partition_scan_work);
+	put_disk(mpath_head->disk);
+	mpath_head->disk = NULL;
+}
+EXPORT_SYMBOL_GPL(mpath_put_disk);
+
+int mpath_alloc_head_disk(struct mpath_head *mpath_head,
+			struct queue_limits *lim, int numa_node)
+{
+	struct gendisk *disk;
+
+	if (!mpath_head->disk_groups || !mpath_head->parent ||
+	    mpath_head->disk)
+		return -EINVAL;
+
+	disk = blk_alloc_disk(lim, numa_node);
+	if (IS_ERR(disk))
+		return PTR_ERR(disk);
+
+	mpath_head->disk = disk;
+	mpath_head->disk->private_data = mpath_head;
+	mpath_head->disk->fops = &mpath_ops;
+
+	set_bit(GD_SUPPRESS_PART_SCAN, &mpath_head->disk->state);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(mpath_alloc_head_disk);
+
+void mpath_device_set_live(struct mpath_device *mpath_device)
+{
+	struct mpath_head *mpath_head = mpath_device->mpath_head;
+	int ret;
+
+	if (!mpath_head->disk)
+		return;
+
+	if (!test_and_set_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags)) {
+		dev_set_drvdata(disk_to_dev(mpath_head->disk), mpath_head);
+		ret = device_add_disk(mpath_head->parent, mpath_head->disk,
+				mpath_head->disk_groups);
+		if (ret) {
+			clear_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags);
+			return;
+		}
+		queue_work(mpath_wq, &mpath_head->partition_scan_work);
+	}
+}
+EXPORT_SYMBOL_GPL(mpath_device_set_live);
+
 int mpath_head_init(struct mpath_head *mpath_head)
 {
 	memset(mpath_head, 0, sizeof(*mpath_head));
@@ -43,6 +144,9 @@ int mpath_head_init(struct mpath_head *mpath_head)
 	mutex_init(&mpath_head->lock);
 	refcount_set(&mpath_head->refcount, 1);
 
+	INIT_WORK(&mpath_head->partition_scan_work,
+		multipath_partition_scan_work);
+
 	return init_srcu_struct(&mpath_head->srcu);
 }
 EXPORT_SYMBOL_GPL(mpath_head_init);
-- 
2.43.7

[PATCH v6 03/27] libmultipath: Add path selection support

[John Garry]

From: John Garry <john.garry@linux.dev>

Add code for path selection.

NVMe ANA is abstracted into enum mpath_access_state. The motivation here is
so that SCSI ALUA can be used. Callbacks .is_disabled and .is_optimized
are added to get the path access state.

Path selection modes round-robin, NUMA, and queue-depth are added, same
as NVMe supports.

NVMe has almost like-for-like equivalents here:
- __mpath_find_path() -> __nvme_find_path()
- mpath_find_path() -> nvme_find_path()

and similar for all introduced callee functions.

Functions mpath_set_iopolicy() and mpath_get_iopolicy() are added for
setting default iopolicy.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 include/linux/multipath.h |  33 +++++
 lib/multipath.c           | 253 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 285 insertions(+), 1 deletion(-)

diff --git a/include/linux/multipath.h b/include/linux/multipath.h
index ca6e02505be43..9bbab0aa8961a 100644
--- a/include/linux/multipath.h
+++ b/include/linux/multipath.h
@@ -7,13 +7,31 @@
 
 extern const struct block_device_operations mpath_ops;
 
+enum mpath_iopolicy_e {
+	MPATH_IOPOLICY_NUMA,
+	MPATH_IOPOLICY_RR,
+	MPATH_IOPOLICY_QD,
+};
+
+enum mpath_access_state {
+	MPATH_STATE_OPTIMIZED,
+	MPATH_STATE_NONOPTIMIZED,
+	MPATH_STATE_OTHER
+};
+
 struct mpath_device {
 	struct mpath_head	*mpath_head;
 	struct list_head	siblings;
 	struct gendisk		*disk;
+	int			numa_node;
+	atomic_t		*nr_active;
+	enum mpath_access_state access_state;
 };
 
 struct mpath_head_template {
+	bool (*is_disabled)(struct mpath_device *);
+	bool (*is_optimized)(struct mpath_device *);
+	const struct attribute_group **device_groups;
 };
 
 #define MPATH_HEAD_DISK_LIVE 			0
@@ -25,6 +43,7 @@ struct mpath_head {
 
 	refcount_t		refcount;
 
+	enum mpath_iopolicy_e	*iopolicy;
 	unsigned long		flags;
 	struct gendisk		*disk;
 	struct work_struct	partition_scan_work;
@@ -44,6 +63,14 @@ static inline struct mpath_head *mpath_gendisk_to_head(struct gendisk *disk)
 	return mpath_bd_device_to_head(disk_to_dev(disk));
 }
 
+static inline enum mpath_iopolicy_e mpath_read_iopolicy(
+			struct mpath_head *mpath_head)
+{
+	return READ_ONCE(*mpath_head->iopolicy);
+}
+void mpath_synchronize(struct mpath_head *mpath_head);
+int mpath_set_iopolicy(const char *str, enum mpath_iopolicy_e *iopolicy);
+int mpath_get_iopolicy(char *buf, int iopolicy);
 int mpath_get_head(struct mpath_head *mpath_head);
 void mpath_put_head(struct mpath_head *mpath_head);
 int mpath_head_init(struct mpath_head *mpath_head);
@@ -63,4 +90,10 @@ static inline bool is_mpath_disk(struct gendisk *disk)
 	return false;
 	#endif
 }
+
+static inline bool mpath_qd_iopolicy(enum mpath_iopolicy_e *iopolicy)
+{
+	return READ_ONCE(*iopolicy) == MPATH_IOPOLICY_QD;
+}
+
 #endif // _LIBMULTIPATH_H
diff --git a/lib/multipath.c b/lib/multipath.c
index 1cf20b0f62d07..26714116023a3 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -7,8 +7,245 @@
 #include <linux/multipath.h>
 #include <linux/wait_bit.h>
 
+static struct mpath_device *mpath_find_path(struct mpath_head *mpath_head);
+
 static struct workqueue_struct *mpath_wq;
 
+static const char * const mpath_iopolicy_names[] = {
+	[MPATH_IOPOLICY_NUMA]	= "numa",
+	[MPATH_IOPOLICY_RR]	= "round-robin",
+	[MPATH_IOPOLICY_QD]	= "queue-depth",
+};
+
+static int mpath_iopolicy_parse(const char *str)
+{
+	return __sysfs_match_string(mpath_iopolicy_names,
+		ARRAY_SIZE(mpath_iopolicy_names), str);
+}
+
+int mpath_set_iopolicy(const char *str, enum mpath_iopolicy_e *iopolicy)
+{
+	int policy;
+
+	if (!str)
+		return -EINVAL;
+	policy = mpath_iopolicy_parse(str);
+	if (policy < 0)
+		return policy;
+	WRITE_ONCE(*iopolicy, policy);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(mpath_set_iopolicy);
+
+int mpath_get_iopolicy(char *buf, int iopolicy)
+{
+	return sprintf(buf, "%s\n", mpath_iopolicy_names[iopolicy]);
+}
+EXPORT_SYMBOL_GPL(mpath_get_iopolicy);
+
+
+void mpath_synchronize(struct mpath_head *mpath_head)
+{
+	synchronize_srcu(&mpath_head->srcu);
+}
+EXPORT_SYMBOL_GPL(mpath_synchronize);
+
+static bool mpath_path_is_disabled(struct mpath_head *mpath_head,
+				struct mpath_device *mpath_device)
+{
+	return mpath_head->mpdt->is_disabled(mpath_device);
+}
+
+static struct mpath_device *__mpath_find_path(struct mpath_head *mpath_head,
+					int node)
+{
+	int found_distance = INT_MAX, fallback_distance = INT_MAX, distance;
+	struct mpath_device *found = NULL, *fallback = NULL, *mpath_device;
+
+	list_for_each_entry_srcu(mpath_device, &mpath_head->dev_list, siblings,
+		srcu_read_lock_held(&mpath_head->srcu)) {
+		if (mpath_path_is_disabled(mpath_head, mpath_device))
+			continue;
+
+		if (mpath_device->numa_node != NUMA_NO_NODE &&
+		    (mpath_read_iopolicy(mpath_head) ==
+			MPATH_IOPOLICY_NUMA))
+			distance = node_distance(node,
+					mpath_device->numa_node);
+		else
+			distance = LOCAL_DISTANCE;
+
+		switch(mpath_device->access_state) {
+		case MPATH_STATE_OPTIMIZED:
+		    if (distance < found_distance) {
+			    found_distance = distance;
+			    found = mpath_device;
+		    }
+		    break;
+		case MPATH_STATE_NONOPTIMIZED:
+		    if (distance < fallback_distance) {
+			    fallback_distance = distance;
+			    fallback = mpath_device;
+		    }
+		    break;
+		default:
+		    break;
+		}
+	}
+
+	if (!found)
+		found = fallback;
+
+	if (found)
+		rcu_assign_pointer(mpath_head->current_path[node], found);
+
+	return found;
+}
+
+static struct mpath_device *mpath_next_dev(struct mpath_head *mpath_head,
+				struct mpath_device *mpath_dev)
+{
+	mpath_dev = list_next_or_null_rcu(&mpath_head->dev_list,
+			&mpath_dev->siblings, struct mpath_device,
+			siblings);
+
+	if (mpath_dev)
+		return mpath_dev;
+	return list_first_or_null_rcu(&mpath_head->dev_list,
+				struct mpath_device, siblings);
+}
+
+static struct mpath_device *mpath_round_robin_path(
+				struct mpath_head *mpath_head)
+{
+	struct mpath_device *mpath_device, *found = NULL;
+	int node = numa_node_id();
+	enum mpath_access_state access_state_old;
+	struct mpath_device *old =
+			srcu_dereference(mpath_head->current_path[node],
+				&mpath_head->srcu);
+
+	if (unlikely(!old))
+		return __mpath_find_path(mpath_head, node);
+
+	if (list_is_singular(&mpath_head->dev_list)) {
+		if (mpath_path_is_disabled(mpath_head, old))
+			return NULL;
+		return old;
+	}
+
+	for (mpath_device = mpath_next_dev(mpath_head, old);
+	    mpath_device && mpath_device != old;
+	    mpath_device = mpath_next_dev(mpath_head, mpath_device)) {
+
+		if (mpath_path_is_disabled(mpath_head, mpath_device))
+			continue;
+		if (mpath_device->access_state == MPATH_STATE_OPTIMIZED) {
+			found = mpath_device;
+			goto out;
+		}
+		if (mpath_device->access_state == MPATH_STATE_NONOPTIMIZED)
+			found = mpath_device;
+	}
+
+	/*
+	 * The loop above skips the current path for round-robin semantics.
+	 * Fall back to the current path if either:
+	 *  - no other optimized path found and current is optimized,
+	 *  - no other usable path found and current is usable.
+	 */
+	access_state_old = old->access_state;
+	if (!mpath_path_is_disabled(mpath_head, old) &&
+	    (access_state_old == MPATH_STATE_OPTIMIZED ||
+	    (!found && access_state_old == MPATH_STATE_NONOPTIMIZED)))
+		return old;
+
+	if (!found)
+		return NULL;
+out:
+	rcu_assign_pointer(mpath_head->current_path[node], found);
+
+	return found;
+}
+
+static struct mpath_device *mpath_queue_depth_path(
+				struct mpath_head *mpath_head)
+{
+	struct mpath_device *best_opt = NULL, *mpath_device;
+	struct mpath_device *best_nonopt = NULL;
+	unsigned int min_depth_opt = UINT_MAX, min_depth_nonopt = UINT_MAX;
+	unsigned int depth;
+
+	list_for_each_entry_srcu(mpath_device, &mpath_head->dev_list, siblings,
+				 srcu_read_lock_held(&mpath_head->srcu)) {
+
+		if (mpath_path_is_disabled(mpath_head, mpath_device))
+			continue;
+
+		depth = atomic_read(mpath_device->nr_active);
+
+		switch (mpath_device->access_state) {
+		case MPATH_STATE_OPTIMIZED:
+			if (depth < min_depth_opt) {
+				min_depth_opt = depth;
+				best_opt = mpath_device;
+			}
+			break;
+		case MPATH_STATE_NONOPTIMIZED:
+			if (depth < min_depth_nonopt) {
+				min_depth_nonopt = depth;
+				best_nonopt = mpath_device;
+			}
+			break;
+		default:
+			break;
+		}
+
+		if (min_depth_opt == 0)
+			return best_opt;
+	}
+
+	return best_opt ? best_opt : best_nonopt;
+}
+
+static inline bool mpath_path_is_optimized(struct mpath_head *mpath_head,
+				struct mpath_device *mpath_device)
+{
+	return mpath_head->mpdt->is_optimized(mpath_device);
+}
+
+static struct mpath_device *mpath_numa_path(struct mpath_head *mpath_head)
+{
+	int node = numa_node_id();
+	struct mpath_device *mpath_device;
+
+	mpath_device = srcu_dereference(mpath_head->current_path[node],
+					&mpath_head->srcu);
+	if (unlikely(!mpath_device))
+		return __mpath_find_path(mpath_head, node);
+	if (unlikely(mpath_path_is_disabled(mpath_head, mpath_device)))
+		return __mpath_find_path(mpath_head, node);
+	if (unlikely(!mpath_path_is_optimized(mpath_head, mpath_device)))
+		return __mpath_find_path(mpath_head, node);
+	return mpath_device;
+}
+
+__maybe_unused
+static struct mpath_device *mpath_find_path(struct mpath_head *mpath_head)
+{
+	enum mpath_iopolicy_e iopolicy = mpath_read_iopolicy(mpath_head);
+
+	switch (iopolicy) {
+	case MPATH_IOPOLICY_QD:
+		return mpath_queue_depth_path(mpath_head);
+	case MPATH_IOPOLICY_RR:
+		return mpath_round_robin_path(mpath_head);
+	default:
+		return mpath_numa_path(mpath_head);
+	}
+}
+
 int mpath_get_head(struct mpath_head *mpath_head)
 {
 	if (!refcount_inc_not_zero(&mpath_head->refcount))
@@ -76,6 +313,7 @@ void mpath_remove_disk(struct mpath_head *mpath_head)
 	if (test_and_clear_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags)) {
 		struct gendisk *disk = mpath_head->disk;
 
+		mpath_synchronize(mpath_head);
 		del_gendisk(disk);
 	}
 }
@@ -99,7 +337,7 @@ int mpath_alloc_head_disk(struct mpath_head *mpath_head,
 	struct gendisk *disk;
 
 	if (!mpath_head->disk_groups || !mpath_head->parent ||
-	    mpath_head->disk)
+	     !mpath_head->iopolicy || mpath_head->disk)
 		return -EINVAL;
 
 	disk = blk_alloc_disk(lim, numa_node);
@@ -134,6 +372,19 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
 		}
 		queue_work(mpath_wq, &mpath_head->partition_scan_work);
 	}
+
+	mutex_lock(&mpath_head->lock);
+	if (mpath_path_is_optimized(mpath_head, mpath_device)) {
+		int node, srcu_idx;
+
+		srcu_idx = srcu_read_lock(&mpath_head->srcu);
+		for_each_online_node(node)
+			__mpath_find_path(mpath_head, node);
+		srcu_read_unlock(&mpath_head->srcu, srcu_idx);
+	}
+	mutex_unlock(&mpath_head->lock);
+
+	mpath_synchronize(mpath_head);
 }
 EXPORT_SYMBOL_GPL(mpath_device_set_live);
 
-- 
2.43.7

[PATCH v6 04/27] libmultipath: Add bio handling

[John Garry]

From: John Garry <john.garry@linux.dev>

Add support to submit a bio per-path. In addition, for failover, add
support to requeue a failed bio.

NVMe has almost like-for-like equivalents here:
    - nvme_available_path() -> mpath_available_path()
    - nvme_requeue_work() -> mpath_requeue_work()
    - nvme_ns_head_submit_bio() -> mpath_bdev_submit_bio()

For failover, a driver may want to re-submit a bio, so add support to
clone a bio prior to submission.

A bio which is submitted to a per-path device has flag REQ_MPATH set,
same as what is done for NVMe with REQ_NVME_MPATH.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 include/linux/multipath.h |  22 ++++++++
 lib/multipath.c           | 108 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 128 insertions(+), 2 deletions(-)

diff --git a/include/linux/multipath.h b/include/linux/multipath.h
index 9bbab0aa8961a..374f0531f7e07 100644
--- a/include/linux/multipath.h
+++ b/include/linux/multipath.h
@@ -3,6 +3,7 @@
 #define _LIBMULTIPATH_H
 
 #include <linux/blkdev.h>
+#include <linux/blk-mq.h>
 #include <linux/srcu.h>
 
 extern const struct block_device_operations mpath_ops;
@@ -29,8 +30,10 @@ struct mpath_device {
 };
 
 struct mpath_head_template {
+	bool (*available_path)(struct mpath_device *);
 	bool (*is_disabled)(struct mpath_device *);
 	bool (*is_optimized)(struct mpath_device *);
+	struct bio *(*clone_bio)(struct bio *);
 	const struct attribute_group **device_groups;
 };
 
@@ -44,6 +47,14 @@ struct mpath_head {
 	refcount_t		refcount;
 
 	enum mpath_iopolicy_e	*iopolicy;
+
+	struct bio_list		requeue_list; /* list for requeing bio */
+	spinlock_t		requeue_lock;
+	struct work_struct	requeue_work; /* work struct for requeue */
+
+	atomic_long_t		requeue_no_usable_path_cnt;
+	atomic_long_t		fail_no_avail_path_cnt;
+
 	unsigned long		flags;
 	struct gendisk		*disk;
 	struct work_struct	partition_scan_work;
@@ -53,6 +64,13 @@ struct mpath_head {
 	struct mpath_device __rcu 		*current_path[MAX_NUMNODES];
 };
 
+#define REQ_MPATH		REQ_DRV
+
+static inline bool is_mpath_request(struct request *req)
+{
+	return req->cmd_flags & REQ_MPATH;
+}
+
 static inline struct mpath_head *mpath_bd_device_to_head(struct device *dev)
 {
 	return dev_get_drvdata(dev);
@@ -96,4 +114,8 @@ static inline bool mpath_qd_iopolicy(enum mpath_iopolicy_e *iopolicy)
 	return READ_ONCE(*iopolicy) == MPATH_IOPOLICY_QD;
 }
 
+static inline void mpath_schedule_requeue_work(struct mpath_head *mpath_head)
+{
+	kblockd_schedule_work(&mpath_head->requeue_work);
+}
 #endif // _LIBMULTIPATH_H
diff --git a/lib/multipath.c b/lib/multipath.c
index 26714116023a3..93b82eea7d38b 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -6,6 +6,7 @@
 #include <linux/module.h>
 #include <linux/multipath.h>
 #include <linux/wait_bit.h>
+#include <trace/events/block.h>
 
 static struct mpath_device *mpath_find_path(struct mpath_head *mpath_head);
 
@@ -44,7 +45,6 @@ int mpath_get_iopolicy(char *buf, int iopolicy)
 }
 EXPORT_SYMBOL_GPL(mpath_get_iopolicy);
 
-
 void mpath_synchronize(struct mpath_head *mpath_head)
 {
 	synchronize_srcu(&mpath_head->srcu);
@@ -231,7 +231,6 @@ static struct mpath_device *mpath_numa_path(struct mpath_head *mpath_head)
 	return mpath_device;
 }
 
-__maybe_unused
 static struct mpath_device *mpath_find_path(struct mpath_head *mpath_head)
 {
 	enum mpath_iopolicy_e iopolicy = mpath_read_iopolicy(mpath_head);
@@ -246,6 +245,81 @@ static struct mpath_device *mpath_find_path(struct mpath_head *mpath_head)
 	}
 }
 
+static bool mpath_available_path(struct mpath_head *mpath_head)
+{
+	struct mpath_device *mpath_device;
+
+	if (!test_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags))
+		return false;
+
+	list_for_each_entry_srcu(mpath_device, &mpath_head->dev_list, siblings,
+				 srcu_read_lock_held(&mpath_head->srcu)) {
+		if (mpath_head->mpdt->available_path(mpath_device))
+			return true;
+	}
+
+	return false;
+}
+
+static void mpath_bdev_submit_bio(struct bio *bio)
+{
+	struct mpath_head *mpath_head = bio->bi_bdev->bd_disk->private_data;
+	struct device *dev = mpath_head->parent;
+	struct mpath_device *mpath_device;
+	int srcu_idx;
+
+	/*
+	 * The mpath_device might be going away and the bio might be moved to a
+	 * different queue in failover, so we need to use the bio_split
+	 * pool from the original queue to allocate the bvecs from.
+	 */
+	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;
+
+	srcu_idx = srcu_read_lock(&mpath_head->srcu);
+	mpath_device = mpath_find_path(mpath_head);
+
+	if (likely(mpath_device)) {
+		if (mpath_head->mpdt->clone_bio) {
+			struct bio *orig = bio;
+
+			bio = mpath_head->mpdt->clone_bio(bio);
+			if (!bio) {
+				bio_io_error(orig);
+				goto out;
+			}
+		}
+
+		bio_set_dev(bio, mpath_device->disk->part0);
+		/*
+		 * Use BIO_REMAPPED to skip bio_check_eod() when this bio
+		 * enters submit_bio_noacct() for the per-path device. The EOD
+		 * check already passed on the multipath head.
+		 */
+		bio_set_flag(bio, BIO_REMAPPED);
+		bio->bi_opf |= REQ_MPATH;
+		trace_block_bio_remap(bio, disk_devt(mpath_head->disk),
+				      bio->bi_iter.bi_sector);
+		submit_bio_noacct(bio);
+	} else if (mpath_available_path(mpath_head)) {
+		dev_warn_ratelimited(dev, "no usable path - requeuing I/O\n");
+
+		spin_lock_irq(&mpath_head->requeue_lock);
+		bio_list_add(&mpath_head->requeue_list, bio);
+		spin_unlock_irq(&mpath_head->requeue_lock);
+		atomic_long_inc(&mpath_head->requeue_no_usable_path_cnt);
+	} else {
+		dev_warn_ratelimited(dev, "no available path - failing I/O\n");
+
+		bio_io_error(bio);
+		atomic_long_inc(&mpath_head->fail_no_avail_path_cnt);
+	}
+
+out:
+	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
+}
+
 int mpath_get_head(struct mpath_head *mpath_head)
 {
 	if (!refcount_inc_not_zero(&mpath_head->refcount))
@@ -291,6 +365,7 @@ const struct block_device_operations mpath_ops = {
 	.owner          = THIS_MODULE,
 	.open		= mpath_bdev_open,
 	.release	= mpath_bdev_release,
+	.submit_bio	= mpath_bdev_submit_bio,
 };
 EXPORT_SYMBOL_GPL(mpath_ops);
 
@@ -308,12 +383,35 @@ static void multipath_partition_scan_work(struct work_struct *work)
 	mutex_unlock(&mpath_head->disk->open_mutex);
 }
 
+static void mpath_requeue_work(struct work_struct *work)
+{
+	struct mpath_head *mpath_head =
+	    container_of(work, struct mpath_head, requeue_work);
+	struct bio *bio, *next;
+
+	spin_lock_irq(&mpath_head->requeue_lock);
+	next = bio_list_get(&mpath_head->requeue_list);
+	spin_unlock_irq(&mpath_head->requeue_lock);
+
+	while ((bio = next) != NULL) {
+		next = bio->bi_next;
+		bio->bi_next = NULL;
+		submit_bio_noacct(bio);
+	}
+}
+
 void mpath_remove_disk(struct mpath_head *mpath_head)
 {
 	if (test_and_clear_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags)) {
 		struct gendisk *disk = mpath_head->disk;
 
+		/*
+		 * requeue I/O after MPATH_HEAD_DISK_LIVE has been cleared
+		 * to allow multipath to fail all I/O.
+		 */
 		mpath_synchronize(mpath_head);
+		mpath_schedule_requeue_work(mpath_head);
+
 		del_gendisk(disk);
 	}
 }
@@ -325,6 +423,8 @@ void mpath_put_disk(struct mpath_head *mpath_head)
 		return;
 
 	/* make sure all pending bios are cleaned up */
+	kblockd_schedule_work(&mpath_head->requeue_work);
+	flush_work(&mpath_head->requeue_work);
 	flush_work(&mpath_head->partition_scan_work);
 	put_disk(mpath_head->disk);
 	mpath_head->disk = NULL;
@@ -385,6 +485,7 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
 	mutex_unlock(&mpath_head->lock);
 
 	mpath_synchronize(mpath_head);
+	mpath_schedule_requeue_work(mpath_head);
 }
 EXPORT_SYMBOL_GPL(mpath_device_set_live);
 
@@ -397,6 +498,9 @@ int mpath_head_init(struct mpath_head *mpath_head)
 
 	INIT_WORK(&mpath_head->partition_scan_work,
 		multipath_partition_scan_work);
+	INIT_WORK(&mpath_head->requeue_work, mpath_requeue_work);
+	spin_lock_init(&mpath_head->requeue_lock);
+	bio_list_init(&mpath_head->requeue_list);
 
 	return init_srcu_struct(&mpath_head->srcu);
 }
-- 
2.43.7

[PATCH v6 05/27] libmultipath: Add support for mpath_device management

[John Garry]

Add support to add or remove a mpath_device as a path.

NVMe has almost like-for-like equivalents here:
- nvme_mpath_clear_current_path() -> mpath_clear_current_path()
- nvme_mpath_add_sysfs_link() -> mpath_add_sysfs_link()
- nvme_mpath_remove_sysfs_link() -> mpath_remove_sysfs_link()
- nvme_mpath_revalidate_paths() -> mpath_revalidate_paths()

The functionality in mpath_clear_paths() and mpath_synchronize() have the
same pattern which is frequently used in the NVMe code.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 include/linux/multipath.h |  12 +++
 lib/multipath.c           | 154 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 166 insertions(+)

diff --git a/include/linux/multipath.h b/include/linux/multipath.h
index 374f0531f7e07..033ee8d54bbe9 100644
--- a/include/linux/multipath.h
+++ b/include/linux/multipath.h
@@ -20,10 +20,13 @@ enum mpath_access_state {
 	MPATH_STATE_OTHER
 };
 
+#define MPATH_DEVICE_SYSFS_ATTR_LINK      0
+
 struct mpath_device {
 	struct mpath_head	*mpath_head;
 	struct list_head	siblings;
 	struct gendisk		*disk;
+	unsigned long		flags;
 	int			numa_node;
 	atomic_t		*nr_active;
 	enum mpath_access_state access_state;
@@ -89,6 +92,15 @@ static inline enum mpath_iopolicy_e mpath_read_iopolicy(
 void mpath_synchronize(struct mpath_head *mpath_head);
 int mpath_set_iopolicy(const char *str, enum mpath_iopolicy_e *iopolicy);
 int mpath_get_iopolicy(char *buf, int iopolicy);
+bool mpath_clear_current_path(struct mpath_device *mpath_device);
+int mpath_add_device(struct mpath_device *mpath_device, struct gendisk *disk,
+		int numa_node, atomic_t *nr_active);
+bool mpath_delete_device(struct mpath_device *mpath_device);
+bool mpath_head_devices_empty(struct mpath_head *mpath_head);
+void mpath_clear_paths(struct mpath_head *mpath_head);
+void mpath_revalidate_paths(struct mpath_head *mpath_head);
+void mpath_add_sysfs_link(struct mpath_head *mpath_head);
+void mpath_remove_sysfs_link(struct mpath_device *mpath_device);
 int mpath_get_head(struct mpath_head *mpath_head);
 void mpath_put_head(struct mpath_head *mpath_head);
 int mpath_head_init(struct mpath_head *mpath_head);
diff --git a/lib/multipath.c b/lib/multipath.c
index 93b82eea7d38b..6b8f6e846987a 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -51,6 +51,87 @@ void mpath_synchronize(struct mpath_head *mpath_head)
 }
 EXPORT_SYMBOL_GPL(mpath_synchronize);
 
+int mpath_add_device(struct mpath_device *mpath_device, struct gendisk *disk,
+		int numa_node, atomic_t *nr_active)
+{
+	struct mpath_head *mpath_head = mpath_device->mpath_head;
+
+	if (!disk || !nr_active)
+		return -EINVAL;
+
+	mpath_device->disk = disk;
+	mpath_device->numa_node = numa_node;
+	mpath_device->nr_active = nr_active;
+	mutex_lock(&mpath_head->lock);
+	list_add_tail_rcu(&mpath_device->siblings, &mpath_head->dev_list);
+	mutex_unlock(&mpath_head->lock);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(mpath_add_device);
+
+bool mpath_delete_device(struct mpath_device *mpath_device)
+{
+	bool empty;
+
+	mutex_lock(&mpath_device->mpath_head->lock);
+	list_del_rcu(&mpath_device->siblings);
+	empty = list_empty(&mpath_device->mpath_head->dev_list);
+	mutex_unlock(&mpath_device->mpath_head->lock);
+
+	mpath_synchronize(mpath_device->mpath_head);
+
+	return empty;
+}
+EXPORT_SYMBOL_GPL(mpath_delete_device);
+
+bool mpath_head_devices_empty(struct mpath_head *mpath_head)
+{
+	bool empty;
+
+	mutex_lock(&mpath_head->lock);
+	empty = list_empty(&mpath_head->dev_list);
+	mutex_unlock(&mpath_head->lock);
+
+	return empty;
+}
+EXPORT_SYMBOL_GPL(mpath_head_devices_empty);
+
+bool mpath_clear_current_path(struct mpath_device *mpath_device)
+{
+	struct mpath_head *mpath_head = mpath_device->mpath_head;
+	bool changed = false;
+	int node;
+
+	for_each_node(node) {
+		if (mpath_device ==
+			rcu_access_pointer(mpath_head->current_path[node])) {
+			rcu_assign_pointer(mpath_head->current_path[node],
+				NULL);
+			changed = true;
+		}
+	}
+
+	return changed;
+}
+EXPORT_SYMBOL_GPL(mpath_clear_current_path);
+
+void mpath_clear_paths(struct mpath_head *mpath_head)
+{
+	int node;
+
+	for_each_node(node)
+		rcu_assign_pointer(mpath_head->current_path[node], NULL);
+}
+EXPORT_SYMBOL_GPL(mpath_clear_paths);
+
+void mpath_revalidate_paths(struct mpath_head *mpath_head)
+{
+	mpath_clear_paths(mpath_head);
+	mpath_schedule_requeue_work(mpath_head);
+}
+EXPORT_SYMBOL_GPL(mpath_revalidate_paths);
+
 static bool mpath_path_is_disabled(struct mpath_head *mpath_head,
 				struct mpath_device *mpath_device)
 {
@@ -473,6 +554,8 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
 		queue_work(mpath_wq, &mpath_head->partition_scan_work);
 	}
 
+	mpath_add_sysfs_link(mpath_head);
+
 	mutex_lock(&mpath_head->lock);
 	if (mpath_path_is_optimized(mpath_head, mpath_device)) {
 		int node, srcu_idx;
@@ -489,6 +572,77 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
 }
 EXPORT_SYMBOL_GPL(mpath_device_set_live);
 
+void mpath_add_sysfs_link(struct mpath_head *mpath_head)
+{
+	struct device *target;
+	struct device *source;
+	int rc, srcu_idx;
+	struct kobject *mpath_gd_kobj;
+	struct mpath_device *mpath_device;
+
+	/*
+	 * Ensure head disk node is already added otherwise we may get invalid
+	 * kobj for head disk node
+	 */
+	if (!test_bit(GD_ADDED, &mpath_head->disk->state))
+		return;
+
+	mpath_gd_kobj = &disk_to_dev(mpath_head->disk)->kobj;
+	srcu_idx = srcu_read_lock(&mpath_head->srcu);
+
+	list_for_each_entry_srcu(mpath_device, &mpath_head->dev_list, siblings,
+				 srcu_read_lock_held(&mpath_head->srcu)) {
+		if (!test_bit(GD_ADDED, &mpath_device->disk->state))
+			continue;
+
+		if (test_and_set_bit(MPATH_DEVICE_SYSFS_ATTR_LINK,
+					&mpath_device->flags))
+			continue;
+
+		target = disk_to_dev(mpath_device->disk);
+		source = disk_to_dev(mpath_head->disk);
+		/*
+		 * Create sysfs link from head gendisk kobject @kobj to the
+		 * ns path gendisk kobject @target->kobj.
+		 */
+		rc = sysfs_add_link_to_group(mpath_gd_kobj, "multipath",
+				&target->kobj, dev_name(target));
+
+		if (unlikely(rc)) {
+			dev_err(disk_to_dev(mpath_head->disk),
+					"failed to create link to %s rc=%d\n",
+					dev_name(target), rc);
+			clear_bit(MPATH_DEVICE_SYSFS_ATTR_LINK,
+					&mpath_device->flags);
+		} else {
+			dev_info(source, "Created multipath sysfs link to %s\n",
+					mpath_device->disk->disk_name);
+		}
+	}
+
+	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
+}
+EXPORT_SYMBOL_GPL(mpath_add_sysfs_link);
+
+void mpath_remove_sysfs_link(struct mpath_device *mpath_device)
+{
+	struct device *target;
+	struct kobject *mpath_gd_kobj;
+	struct mpath_head *mpath_head = mpath_device->mpath_head;
+
+	if (!test_bit(MPATH_DEVICE_SYSFS_ATTR_LINK, &mpath_device->flags))
+		return;
+
+	target = disk_to_dev(mpath_device->disk);
+	mpath_gd_kobj = &disk_to_dev(mpath_head->disk)->kobj;
+
+	sysfs_remove_link_from_group(mpath_gd_kobj, "multipath",
+			dev_name(target));
+
+	clear_bit(MPATH_DEVICE_SYSFS_ATTR_LINK, &mpath_device->flags);
+}
+EXPORT_SYMBOL_GPL(mpath_remove_sysfs_link);
+
 int mpath_head_init(struct mpath_head *mpath_head)
 {
 	memset(mpath_head, 0, sizeof(*mpath_head));
-- 
2.43.7

[PATCH v6 06/27] libmultipath: Add delayed removal support

[John Garry]

From: John Garry <john.garry@linux.dev>

Add support for delayed removal, same as exists for NVMe.

The purpose of this feature is to keep the multipath disk and cdev present
for intermittent periods of no available path.

Helpers mpath_delayed_removal_secs_show() and
mpath_delayed_removal_secs_store() may be used in the driver sysfs code.

The driver is responsible for supplying the removal work callback for
the delayed work.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 include/linux/multipath.h | 18 ++++++++
 lib/multipath.c           | 91 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 108 insertions(+), 1 deletion(-)

diff --git a/include/linux/multipath.h b/include/linux/multipath.h
index 033ee8d54bbe9..66b6ad9c2219a 100644
--- a/include/linux/multipath.h
+++ b/include/linux/multipath.h
@@ -34,6 +34,7 @@ struct mpath_device {
 
 struct mpath_head_template {
 	bool (*available_path)(struct mpath_device *);
+	void (*remove_head)(struct mpath_head *);
 	bool (*is_disabled)(struct mpath_device *);
 	bool (*is_optimized)(struct mpath_device *);
 	struct bio *(*clone_bio)(struct bio *);
@@ -41,6 +42,7 @@ struct mpath_head_template {
 };
 
 #define MPATH_HEAD_DISK_LIVE 			0
+#define MPATH_HEAD_QUEUE_IF_NO_PATH		1
 
 struct mpath_head {
 	struct srcu_struct	srcu;
@@ -58,6 +60,10 @@ struct mpath_head {
 	atomic_long_t		requeue_no_usable_path_cnt;
 	atomic_long_t		fail_no_avail_path_cnt;
 
+	struct delayed_work	remove_work;
+	unsigned int		delayed_removal_secs;
+	struct module		*drv_module;
+
 	unsigned long		flags;
 	struct gendisk		*disk;
 	struct work_struct	partition_scan_work;
@@ -111,6 +117,11 @@ void mpath_remove_disk(struct mpath_head *mpath_head);
 int mpath_alloc_head_disk(struct mpath_head *mpath_head,
 			struct queue_limits *lim, int numa_node);
 void mpath_device_set_live(struct mpath_device *mpath_device);
+bool mpath_can_remove_head(struct mpath_head *mpath_head);
+ssize_t mpath_delayed_removal_secs_show(struct mpath_head *mpath_head,
+			char *buf);
+ssize_t mpath_delayed_removal_secs_store(struct mpath_head *mpath_head,
+			const char *buf, size_t count);
 
 static inline bool is_mpath_disk(struct gendisk *disk)
 {
@@ -126,6 +137,13 @@ static inline bool mpath_qd_iopolicy(enum mpath_iopolicy_e *iopolicy)
 	return READ_ONCE(*iopolicy) == MPATH_IOPOLICY_QD;
 }
 
+static inline bool mpath_head_queue_if_no_path(struct mpath_head *mpath_head)
+{
+	if (test_bit(MPATH_HEAD_QUEUE_IF_NO_PATH, &mpath_head->flags))
+		return true;
+	return false;
+}
+
 static inline void mpath_schedule_requeue_work(struct mpath_head *mpath_head)
 {
 	kblockd_schedule_work(&mpath_head->requeue_work);
diff --git a/lib/multipath.c b/lib/multipath.c
index 6b8f6e846987a..fada92bf9754c 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -66,6 +66,9 @@ int mpath_add_device(struct mpath_device *mpath_device, struct gendisk *disk,
 	list_add_tail_rcu(&mpath_device->siblings, &mpath_head->dev_list);
 	mutex_unlock(&mpath_head->lock);
 
+	if (cancel_delayed_work(&mpath_head->remove_work))
+		module_put(mpath_head->drv_module);
+
 	return 0;
 }
 EXPORT_SYMBOL_GPL(mpath_add_device);
@@ -339,7 +342,16 @@ static bool mpath_available_path(struct mpath_head *mpath_head)
 			return true;
 	}
 
-	return false;
+	/*
+	 * If "mpath_head->delayed_removal_secs" is set (i.e., non-zero), do
+	 * not immediately fail I/O. Instead, requeue the I/O for the configured
+	 * duration, anticipating that if there's a transient link failure then
+	 * it may recover within this time window. This parameter is exported to
+	 * userspace via sysfs, and its default value is zero. It is internally
+	 * mapped to MPATH_HEAD_QUEUE_IF_NO_PATH. When delayed_removal_secs is
+	 * non-zero, this flag is set to true. When zero, the flag is cleared.
+	 */
+	return mpath_head_queue_if_no_path(mpath_head);
 }
 
 static void mpath_bdev_submit_bio(struct bio *bio)
@@ -481,6 +493,39 @@ static void mpath_requeue_work(struct work_struct *work)
 	}
 }
 
+bool mpath_can_remove_head(struct mpath_head *mpath_head)
+{
+	unsigned long delay;
+	bool remove = false;
+
+	mutex_lock(&mpath_head->lock);
+	/*
+	 * Ensure that no one could remove this module while the head
+	 * remove work is pending.
+	 */
+	if (mpath_head_queue_if_no_path(mpath_head) &&
+	    !check_mul_overflow(mpath_head->delayed_removal_secs, HZ, &delay) &&
+	    try_module_get(mpath_head->drv_module)) {
+		mod_delayed_work(mpath_wq, &mpath_head->remove_work, delay);
+	} else {
+		remove = true;
+	}
+
+	mutex_unlock(&mpath_head->lock);
+	return remove;
+}
+EXPORT_SYMBOL_GPL(mpath_can_remove_head);
+
+static void mpath_remove_head_work(struct work_struct *work)
+{
+	struct mpath_head *mpath_head = container_of(to_delayed_work(work),
+			struct mpath_head, remove_work);
+	struct module *drv_module = mpath_head->drv_module;
+
+	mpath_head->mpdt->remove_head(mpath_head);
+	module_put(drv_module);
+}
+
 void mpath_remove_disk(struct mpath_head *mpath_head)
 {
 	if (test_and_clear_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags)) {
@@ -529,6 +574,9 @@ int mpath_alloc_head_disk(struct mpath_head *mpath_head,
 	mpath_head->disk->private_data = mpath_head;
 	mpath_head->disk->fops = &mpath_ops;
 
+	INIT_DELAYED_WORK(&mpath_head->remove_work, mpath_remove_head_work);
+	mpath_head->delayed_removal_secs = 0;
+
 	set_bit(GD_SUPPRESS_PART_SCAN, &mpath_head->disk->state);
 
 	return 0;
@@ -572,6 +620,47 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
 }
 EXPORT_SYMBOL_GPL(mpath_device_set_live);
 
+ssize_t mpath_delayed_removal_secs_show(struct mpath_head *mpath_head,
+					char *buf)
+{
+	int ret;
+
+	mutex_lock(&mpath_head->lock);
+	ret = sysfs_emit(buf, "%u\n", mpath_head->delayed_removal_secs);
+	mutex_unlock(&mpath_head->lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(mpath_delayed_removal_secs_show);
+
+ssize_t mpath_delayed_removal_secs_store(struct mpath_head *mpath_head,
+			const char *buf, size_t count)
+{
+	unsigned int sec;
+	ssize_t ret;
+
+	ret = kstrtouint(buf, 0, &sec);
+	if (ret < 0)
+		return ret;
+
+	mutex_lock(&mpath_head->lock);
+	mpath_head->delayed_removal_secs = sec;
+	if (sec)
+		set_bit(MPATH_HEAD_QUEUE_IF_NO_PATH, &mpath_head->flags);
+	else
+		clear_bit(MPATH_HEAD_QUEUE_IF_NO_PATH, &mpath_head->flags);
+	mutex_unlock(&mpath_head->lock);
+
+	/*
+	 * Ensure that update to MPATH_HEAD_QUEUE_IF_NO_PATH is seen
+	 * by its reader.
+	 */
+	mpath_synchronize(mpath_head);
+
+	return count;
+}
+EXPORT_SYMBOL_GPL(mpath_delayed_removal_secs_store);
+
 void mpath_add_sysfs_link(struct mpath_head *mpath_head)
 {
 	struct device *target;
-- 
2.43.7

[PATCH v6 07/27] libmultipath: Add sysfs helpers

[John Garry]

Add helpers for driver sysfs code for the following functionality:
- get/set iopolicy with mpath_iopolicy_store() and mpath_iopolicy_show()
- show device path per NUMA node
- "multipath" attribute group, equivalent to nvme_ns_mpath_attr_group
- device groups attribute array, similar to nvme_ns_attr_groups but not
  containing NVMe members.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 include/linux/multipath.h |  5 +++
 lib/multipath.c           | 94 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)

diff --git a/include/linux/multipath.h b/include/linux/multipath.h
index 66b6ad9c2219a..7a3708ff37254 100644
--- a/include/linux/multipath.h
+++ b/include/linux/multipath.h
@@ -7,6 +7,8 @@
 #include <linux/srcu.h>
 
 extern const struct block_device_operations mpath_ops;
+extern const struct attribute_group mpath_attr_group;
+extern const struct attribute_group *mpath_device_groups[];
 
 enum mpath_iopolicy_e {
 	MPATH_IOPOLICY_NUMA,
@@ -118,6 +120,9 @@ int mpath_alloc_head_disk(struct mpath_head *mpath_head,
 			struct queue_limits *lim, int numa_node);
 void mpath_device_set_live(struct mpath_device *mpath_device);
 bool mpath_can_remove_head(struct mpath_head *mpath_head);
+ssize_t mpath_numa_nodes_show(struct mpath_device *mpath_device, char *buf);
+ssize_t mpath_iopolicy_show(enum mpath_iopolicy_e *iopolicy, char *buf);
+bool mpath_iopolicy_store(enum mpath_iopolicy_e *iopolicy, const char *buf);
 ssize_t mpath_delayed_removal_secs_show(struct mpath_head *mpath_head,
 			char *buf);
 ssize_t mpath_delayed_removal_secs_store(struct mpath_head *mpath_head,
diff --git a/lib/multipath.c b/lib/multipath.c
index fada92bf9754c..58eb9be5bebbe 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -620,6 +620,100 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
 }
 EXPORT_SYMBOL_GPL(mpath_device_set_live);
 
+static struct attribute dummy_attr = {
+	.name = "dummy",
+};
+
+static struct attribute *mpath_attrs[] = {
+	&dummy_attr,
+	NULL
+};
+
+static bool multipath_sysfs_group_visible(struct kobject *kobj)
+{
+	struct device *dev = container_of(kobj, struct device, kobj);
+	struct gendisk *disk = dev_to_disk(dev);
+
+	return is_mpath_disk(disk);
+}
+DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(multipath_sysfs)
+
+const struct attribute_group mpath_attr_group = {
+	.name           = "multipath",
+	.attrs		= mpath_attrs,
+	.is_visible     = SYSFS_GROUP_VISIBLE(multipath_sysfs),
+};
+EXPORT_SYMBOL_GPL(mpath_attr_group);
+
+const struct attribute_group *mpath_device_groups[] = {
+	&mpath_attr_group,
+	NULL
+};
+EXPORT_SYMBOL_GPL(mpath_device_groups);
+
+ssize_t mpath_iopolicy_show(enum mpath_iopolicy_e *iopolicy, char *buf)
+{
+	return sysfs_emit(buf, "%s\n",
+		mpath_iopolicy_names[READ_ONCE(*iopolicy)]);
+}
+EXPORT_SYMBOL_GPL(mpath_iopolicy_show);
+
+static void mpath_iopolicy_update(enum mpath_iopolicy_e *iopolicy,
+		int new)
+{
+	int old = READ_ONCE(*iopolicy);
+
+	if (old == new)
+		return;
+
+	WRITE_ONCE(*iopolicy, new);
+
+	pr_info("iopolicy changed from %s to %s\n",
+		mpath_iopolicy_names[old],
+		mpath_iopolicy_names[new]);
+}
+
+bool mpath_iopolicy_store(enum mpath_iopolicy_e *iopolicy, const char *buf)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(mpath_iopolicy_names); i++) {
+		if (sysfs_streq(buf, mpath_iopolicy_names[i])) {
+			mpath_iopolicy_update(iopolicy, i);
+			return true;
+		}
+	}
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(mpath_iopolicy_store);
+
+ssize_t mpath_numa_nodes_show(struct mpath_device *mpath_device, char *buf)
+{
+	struct mpath_head *mpath_head = mpath_device->mpath_head;
+	int node, srcu_idx;
+	nodemask_t numa_nodes;
+	struct mpath_device *current_mpath_dev;
+
+	if (mpath_read_iopolicy(mpath_head) != MPATH_IOPOLICY_NUMA)
+		return 0;
+
+	nodes_clear(numa_nodes);
+
+	srcu_idx = srcu_read_lock(&mpath_head->srcu);
+	for_each_node(node) {
+		current_mpath_dev =
+			srcu_dereference(mpath_head->current_path[node],
+				&mpath_head->srcu);
+		if (current_mpath_dev == mpath_device)
+			node_set(node, numa_nodes);
+	}
+	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
+
+	return sysfs_emit(buf, "%*pbl\n", nodemask_pr_args(&numa_nodes));
+}
+EXPORT_SYMBOL_GPL(mpath_numa_nodes_show);
+
 ssize_t mpath_delayed_removal_secs_show(struct mpath_head *mpath_head,
 					char *buf)
 {
-- 
2.43.7

[PATCH v6 08/27] libmultipath: Add support for block device IOCTL

[John Garry]

From: John Garry <john.garry@linux.dev>

Add mpath_bdev_ioctl() as a multipath block device IOCTL handler. This
handler calls into the mpath_device bdev fops handler.

The .compat_ioctl handler is given the standard handler.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 lib/multipath.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/lib/multipath.c b/lib/multipath.c
index 58eb9be5bebbe..f1758d1e60a60 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -454,11 +454,53 @@ static void mpath_bdev_release(struct gendisk *disk)
 	mpath_put_head(mpath_head);
 }
 
+static int mpath_bdev_ioctl(struct block_device *bdev, blk_mode_t mode,
+		    unsigned int cmd, unsigned long arg)
+{
+	struct gendisk *disk = bdev->bd_disk;
+	struct mpath_head *mpath_head = mpath_gendisk_to_head(disk);
+	struct mpath_device *mpath_device;
+	int srcu_idx, err;
+
+	/*
+	 * This check is duplicated from sd_ioctl() as we don't pass the
+	 * partition bdev to fops->ioctl. That is not yet possible as the
+	 * per-path disk is hidden and, as such, does not have partitions
+	 * scanned.
+	 */
+	if (bdev_is_partition(bdev) && !capable(CAP_SYS_RAWIO))
+		return -ENOIOCTLCMD;
+
+	srcu_idx = srcu_read_lock(&mpath_head->srcu);
+	mpath_device = mpath_find_path(mpath_head);
+	if (!mpath_device) {
+		err = -EWOULDBLOCK;
+		goto out_unlock;
+	}
+
+	if (!mpath_device->disk->fops->ioctl) {
+		err = -ENOTTY;
+		goto out_unlock;
+	}
+
+	err = mpath_device->disk->fops->ioctl(
+			mpath_device->disk->part0, mode, cmd, arg);
+out_unlock:
+	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
+	return err;
+}
+
 const struct block_device_operations mpath_ops = {
 	.owner          = THIS_MODULE,
 	.open		= mpath_bdev_open,
 	.release	= mpath_bdev_release,
 	.submit_bio	= mpath_bdev_submit_bio,
+	.ioctl		= mpath_bdev_ioctl,
+	/*
+	 * Both NVMe and SCSI use generic blkdev_compat_ptr_ioctl, so would
+	 * avoid their custom compat_ioctl implementation.
+	 */
+	.compat_ioctl	= blkdev_compat_ptr_ioctl,
 };
 EXPORT_SYMBOL_GPL(mpath_ops);
 
-- 
2.43.7

[PATCH v6 09/27] libmultipath: Add mpath_bdev_getgeo()

[John Garry]

From: John Garry <john.garry@linux.dev>

Add mpath_bdev_getgeo() as a multipath block device .getgeo handler.

Here we just redirect into the selected mpath_device disk fops->getgeo
handler.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 lib/multipath.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/multipath.c b/lib/multipath.c
index f1758d1e60a60..2017c4d0a210c 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -490,6 +490,26 @@ static int mpath_bdev_ioctl(struct block_device *bdev, blk_mode_t mode,
 	return err;
 }
 
+static int mpath_bdev_getgeo(struct gendisk *disk, struct hd_geometry *geo)
+{
+	struct mpath_head *mpath_head = mpath_gendisk_to_head(disk);
+	int srcu_idx, ret = -EWOULDBLOCK;
+	struct mpath_device *mpath_device;
+
+	srcu_idx = srcu_read_lock(&mpath_head->srcu);
+	mpath_device = mpath_find_path(mpath_head);
+	if (mpath_device) {
+		if (mpath_device->disk->fops->getgeo)
+			ret = mpath_device->disk->fops->getgeo(
+					mpath_device->disk, geo);
+		else
+			ret = -ENOTTY; /* See blkdev_getgeo */
+	}
+	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
+
+	return ret;
+}
+
 const struct block_device_operations mpath_ops = {
 	.owner          = THIS_MODULE,
 	.open		= mpath_bdev_open,
@@ -501,6 +521,7 @@ const struct block_device_operations mpath_ops = {
 	 * avoid their custom compat_ioctl implementation.
 	 */
 	.compat_ioctl	= blkdev_compat_ptr_ioctl,
+	.getgeo		= mpath_bdev_getgeo,
 };
 EXPORT_SYMBOL_GPL(mpath_ops);
 
-- 
2.43.7

[PATCH v6 10/27] libmultipath: Add mpath_bdev_get_unique_id()

[John Garry]

From: John Garry <john.garry@linux.dev>

Add mpath_bdev_get_unique_id() as a multipath block device .get_unique_id
handler.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 lib/multipath.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/lib/multipath.c b/lib/multipath.c
index 2017c4d0a210c..66b3338cf8389 100644
--- a/lib/multipath.c
+++ b/lib/multipath.c
@@ -454,6 +454,27 @@ static void mpath_bdev_release(struct gendisk *disk)
 	mpath_put_head(mpath_head);
 }
 
+static int mpath_bdev_get_unique_id(struct gendisk *disk, u8 id[16],
+    enum blk_unique_id type)
+{
+	struct mpath_head *mpath_head = mpath_gendisk_to_head(disk);
+	int srcu_idx, ret = -EWOULDBLOCK;
+	struct mpath_device *mpath_device;
+
+	srcu_idx = srcu_read_lock(&mpath_head->srcu);
+	mpath_device = mpath_find_path(mpath_head);
+	if (mpath_device) {
+		if (mpath_device->disk->fops->get_unique_id)
+			ret = mpath_device->disk->fops->get_unique_id(
+					mpath_device->disk, id, type);
+		else
+			ret = 0; /* referencing __dm_get_unique_id() */
+	}
+	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
+
+	return ret;
+}
+
 static int mpath_bdev_ioctl(struct block_device *bdev, blk_mode_t mode,
 		    unsigned int cmd, unsigned long arg)
 {
@@ -521,6 +542,7 @@ const struct block_device_operations mpath_ops = {
 	 * avoid their custom compat_ioctl implementation.
 	 */
 	.compat_ioctl	= blkdev_compat_ptr_ioctl,
+	.get_unique_id	= mpath_bdev_get_unique_id,
 	.getgeo		= mpath_bdev_getgeo,
 };
 EXPORT_SYMBOL_GPL(mpath_ops);
-- 
2.43.7

[PATCH v6 11/27] scsi-multipath: introduce basic SCSI device support

[John Garry]

For a scsi_device to support multipath, introduce structure
scsi_mpath_device to hold multipath-specific details.

Like NS structure for NVME, scsi_mpath_device holds the mpath_device
structure to device management and path selection.

A module param are introduced to enable multipath - the following modes
are available:
- on
- off
- always

SCSI multipath will only be available until the following conditions:
- scsi_multipath enabled and implicit ALUA supported and unique ID
  available in VPD page 83.
- scsi_multipath always mode and unique ID available in VPD page 83

The scsi_device structure contains a pointer to scsi_mpath_device; having
this pointer set or unset indicates whether multipath is enabled or
disabled for the scsi_device.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/Kconfig          |  10 +++
 drivers/scsi/Makefile         |   1 +
 drivers/scsi/scsi.c           |   8 +-
 drivers/scsi/scsi_multipath.c | 148 ++++++++++++++++++++++++++++++++++
 drivers/scsi/scsi_scan.c      |   4 +
 drivers/scsi/scsi_sysfs.c     |   2 +
 include/scsi/scsi_device.h    |   2 +
 include/scsi/scsi_multipath.h |  55 +++++++++++++
 8 files changed, 229 insertions(+), 1 deletion(-)
 create mode 100644 drivers/scsi/scsi_multipath.c
 create mode 100644 include/scsi/scsi_multipath.h

diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index c3042393af234..d6c31df454825 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -76,6 +76,16 @@ config SCSI_LIB_KUNIT_TEST
 
 	  If unsure say N.
 
+config SCSI_MULTIPATH
+	bool "SCSI multipath support (EXPERIMENTAL)"
+	depends on SCSI_MOD
+	select LIBMULTIPATH
+	help
+	  This option enables support for native SCSI multipath support for
+	  SCSI host.
+
+	  If unsure say N.
+
 comment "SCSI support type (disk, tape, CD-ROM)"
 	depends on SCSI
 
diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile
index 842c254bb2269..04f9c2e51fcbd 100644
--- a/drivers/scsi/Makefile
+++ b/drivers/scsi/Makefile
@@ -168,6 +168,7 @@ scsi_mod-y			+= scsi_trace.o scsi_logging.o
 scsi_mod-$(CONFIG_PM)		+= scsi_pm.o
 scsi_mod-$(CONFIG_SCSI_DH)	+= scsi_dh.o
 scsi_mod-$(CONFIG_BLK_DEV_BSG)	+= scsi_bsg.o
+scsi_mod-$(CONFIG_SCSI_MULTIPATH)	+= scsi_multipath.o
 
 hv_storvsc-y			:= storvsc_drv.o
 
diff --git a/drivers/scsi/scsi.c b/drivers/scsi/scsi.c
index 76cdad063f7bc..70aa1dbeacebf 100644
--- a/drivers/scsi/scsi.c
+++ b/drivers/scsi/scsi.c
@@ -64,6 +64,7 @@
 #include <scsi/scsi_driver.h>
 #include <scsi/scsi_eh.h>
 #include <scsi/scsi_host.h>
+#include <scsi/scsi_multipath.h>
 #include <scsi/scsi_tcq.h>
 
 #include "scsi_priv.h"
@@ -1042,12 +1043,16 @@ static int __init init_scsi(void)
 	error = scsi_sysfs_register();
 	if (error)
 		goto cleanup_sysctl;
+	error =  scsi_multipath_init();
+	if (error)
+		goto cleanup_sysfs;
 
 	scsi_netlink_init();
 
 	printk(KERN_NOTICE "SCSI subsystem initialized\n");
 	return 0;
-
+cleanup_sysfs:
+	scsi_sysfs_unregister();
 cleanup_sysctl:
 	scsi_exit_sysctl();
 cleanup_hosts:
@@ -1066,6 +1071,7 @@ static int __init init_scsi(void)
 static void __exit exit_scsi(void)
 {
 	scsi_netlink_exit();
+	scsi_multipath_exit();
 	scsi_sysfs_unregister();
 	scsi_exit_sysctl();
 	scsi_exit_hosts();
diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
new file mode 100644
index 0000000000000..5559e2e8d64a8
--- /dev/null
+++ b/drivers/scsi/scsi_multipath.c
@@ -0,0 +1,148 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 Oracle Corp
+ *
+ */
+
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_driver.h>
+#include <scsi/scsi_proto.h>
+#include <scsi/scsi_host.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_multipath.h>
+
+#include "scsi_priv.h"
+
+#define TPGS_MODE_IMPLICIT		0x1
+
+enum {
+	SCSI_MULTIPATH_OFF,
+	SCSI_MULTIPATH_ON,
+	SCSI_MULTIPATH_ALWAYS,
+};
+
+static const char *scsi_multipath_modes[] = {
+	[SCSI_MULTIPATH_OFF]	= "off",
+	[SCSI_MULTIPATH_ON]	= "on",
+	[SCSI_MULTIPATH_ALWAYS]	= "always",
+};
+
+static int scsi_multipath = SCSI_MULTIPATH_OFF;
+
+static int scsi_multipath_param_set(const char *val, const struct kernel_param *kp)
+{
+	int mode;
+
+	if (!val)
+		return -EINVAL;
+	mode = __sysfs_match_string(scsi_multipath_modes,
+		ARRAY_SIZE(scsi_multipath_modes), val);
+
+	if (mode < 0)
+		return mode;
+	scsi_multipath = mode;
+	return 0;
+}
+
+static int scsi_multipath_param_get(char *buf, const struct kernel_param *kp)
+{
+	return sprintf(buf, "%s\n", scsi_multipath_modes[scsi_multipath]);
+}
+
+static const struct kernel_param_ops multipath_param_ops = {
+	.set = scsi_multipath_param_set,
+	.get = scsi_multipath_param_get,
+};
+
+module_param_cb(multipath, &multipath_param_ops, &scsi_multipath, 0444);
+MODULE_PARM_DESC(multipath, "turn on native multipath support, options: on, off, always");
+
+static int scsi_mpath_unique_lun_id(struct scsi_device *sdev)
+{
+	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
+	int ret;
+
+	ret = scsi_vpd_lun_id(sdev, scsi_mpath_dev->device_id_str,
+				SCSI_MPATH_DEVICE_ID_LEN);
+	if (ret < 0)
+		return ret;
+	else if (ret == 0)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int scsi_multipath_sdev_init(struct scsi_device *sdev)
+{
+	struct Scsi_Host *shost = sdev->host;
+	struct scsi_mpath_device *scsi_mpath_dev;
+	struct mpath_device *mpath_device;
+
+	scsi_mpath_dev = kzalloc(sizeof(*scsi_mpath_dev), GFP_KERNEL);
+	if (!scsi_mpath_dev)
+		return -ENOMEM;
+	scsi_mpath_dev->sdev = sdev;
+	sdev->scsi_mpath_dev = scsi_mpath_dev;
+
+	mpath_device = &scsi_mpath_dev->mpath_device;
+	mpath_device->numa_node = dev_to_node(shost->dma_dev);
+	mpath_device->access_state = MPATH_STATE_OPTIMIZED;
+
+	return 0;
+}
+
+static void scsi_multipath_sdev_uninit(struct scsi_device *sdev)
+{
+	kfree(sdev->scsi_mpath_dev);
+	sdev->scsi_mpath_dev = NULL;
+}
+
+int scsi_mpath_dev_alloc(struct scsi_device *sdev)
+{
+	int ret;
+
+	if (scsi_multipath == SCSI_MULTIPATH_OFF)
+		return 0;
+
+	if (!(scsi_device_tpgs(sdev) & TPGS_MODE_IMPLICIT) &&
+	    (scsi_multipath != SCSI_MULTIPATH_ALWAYS)) {
+		sdev_printk(KERN_DEBUG, sdev, "IMPLICIT TPGS are required for multipath support\n");
+		return 0;
+	}
+
+	ret = scsi_multipath_sdev_init(sdev);
+	if (ret)
+		return ret;
+
+	ret = scsi_mpath_unique_lun_id(sdev);
+	if (ret < 0)
+		goto out_uninit;
+
+	return 0;
+
+out_uninit:
+	scsi_multipath_sdev_uninit(sdev);
+	return 0;
+}
+
+void scsi_mpath_dev_release(struct scsi_device *sdev)
+{
+	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
+
+	if (!scsi_mpath_dev)
+		return;
+
+	scsi_multipath_sdev_uninit(sdev);
+}
+
+int __init scsi_multipath_init(void)
+{
+	return 0;
+}
+
+void __exit scsi_multipath_exit(void)
+{
+}
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("scsi_multipath");
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index e27da038603a2..3f1ac302719c2 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -46,6 +46,7 @@
 #include <scsi/scsi_transport.h>
 #include <scsi/scsi_dh.h>
 #include <scsi/scsi_eh.h>
+#include <scsi/scsi_multipath.h>
 
 #include "scsi_priv.h"
 #include "scsi_logging.h"
@@ -1130,6 +1131,9 @@ static int scsi_add_lun(struct scsi_device *sdev, unsigned char *inq_result,
 	sdev->max_queue_depth = sdev->queue_depth;
 	WARN_ON_ONCE(sdev->max_queue_depth > sdev->budget_map.depth);
 
+	if (scsi_mpath_dev_alloc(sdev))
+		return SCSI_SCAN_NO_RESPONSE;
+
 	/*
 	 * Ok, the device is now all set up, we can
 	 * register it and tell the rest of the kernel
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index dfc3559e7e04f..2f80d703ce640 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -23,6 +23,7 @@
 #include <scsi/scsi_transport.h>
 #include <scsi/scsi_driver.h>
 #include <scsi/scsi_devinfo.h>
+#include <scsi/scsi_multipath.h>
 
 #include "scsi_priv.h"
 #include "scsi_logging.h"
@@ -455,6 +456,7 @@ static void scsi_device_dev_release(struct device *dev)
 	might_sleep();
 
 	scsi_dh_release_device(sdev);
+	scsi_mpath_dev_release(sdev);
 
 	parent = sdev->sdev_gendev.parent;
 
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index 029f5115b2ea0..40a435e66b34e 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -280,6 +280,8 @@ struct scsi_device {
 	struct device		sdev_gendev,
 				sdev_dev;
 
+	struct scsi_mpath_device *scsi_mpath_dev;
+
 	struct work_struct	requeue_work;
 
 	struct scsi_device_handler *handler;
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
new file mode 100644
index 0000000000000..d3d410dafd17a
--- /dev/null
+++ b/include/scsi/scsi_multipath.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _SCSI_SCSI_MULTIPATH_H
+#define _SCSI_SCSI_MULTIPATH_H
+
+#include <linux/list.h>
+#include <linux/types.h>
+#include <linux/rcupdate.h>
+#include <linux/workqueue.h>
+#include <linux/mutex.h>
+#include <linux/blk-mq.h>
+#include <linux/multipath.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_dbg.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_devinfo.h>
+#include <scsi/scsi_driver.h>
+
+#ifdef CONFIG_SCSI_MULTIPATH
+#define SCSI_MPATH_DEVICE_ID_LEN 256
+
+struct scsi_mpath_device {
+	struct mpath_device	mpath_device;
+	struct scsi_device 	*sdev;
+
+	char			device_id_str[SCSI_MPATH_DEVICE_ID_LEN];
+};
+#define to_scsi_mpath_device(d) \
+	container_of(d, struct scsi_mpath_device, mpath_device)
+
+int scsi_mpath_dev_alloc(struct scsi_device *sdev);
+void scsi_mpath_dev_release(struct scsi_device *sdev);
+int scsi_multipath_init(void);
+void scsi_multipath_exit(void);
+#else /* CONFIG_SCSI_MULTIPATH */
+
+struct scsi_mpath_device {
+};
+
+static inline int scsi_mpath_dev_alloc(struct scsi_device *sdev)
+{
+	return 0;
+}
+static inline void scsi_mpath_dev_release(struct scsi_device *sdev)
+{
+}
+static inline int scsi_multipath_init(void)
+{
+	return 0;
+}
+static inline void scsi_multipath_exit(void)
+{
+}
+#endif /* CONFIG_SCSI_MULTIPATH */
+#endif /* _SCSI_SCSI_MULTIPATH_H */
-- 
2.43.7

[PATCH v6 12/27] scsi-multipath: introduce scsi_device head structure

[John Garry]

From: John Garry <john.garry@linux.dev>

Introduce a scsi_device head structure - scsi_mpath_head - to manage
multipathing for a scsi_device. This is similar to nvme_ns_head structure.

There is no reference in scsi_mpath_head to any disk, as this would be
managed by the scsi_disk driver.

A list of scsi_mpath_head structures is managed to lookup for matching
multipathed scsi_device's. Matching is done through the scsi_device
unique id.

A new class for multipathed devices is added, scsi_mpath_device_class.

The purpose of this class is for managing the scsi_mpath_head.dev member.

The naming for the scsi_device structure is in form H:C:I:L,
where H is host, C is channel, I is ID, and L is lun.

However, for a multipathed scsi_device, all the naming members may be
different between member scsi_device's. As such, just use a simple
single-number naming index for each scsi_mpath_head.

The sysfs device folder will have links to the scsi_device's so, it will
be possible to lookup the member scsi_device's.

An example sysfs entry is as follows:
# ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/
total 0
drwxr-xr-x    2 root     root             0 Apr 13 15:48 power
lrwxrwxrwx    1 root     root             0 Apr 13 15:48 subsystem -> ../../../../class/scsi_mpath_device
-rw-r--r--    1 root     root          4096 Apr 13 15:48 uevent
-r--r--r--    1 root     root          4096 Apr 13 15:48 vpd_id

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_multipath.c | 199 +++++++++++++++++++++++++++++++++-
 drivers/scsi/scsi_sysfs.c     |   3 +
 include/scsi/scsi_multipath.h |  31 ++++++
 3 files changed, 231 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 5559e2e8d64a8..30a2d01be8aaf 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -29,6 +29,10 @@ static const char *scsi_multipath_modes[] = {
 
 static int scsi_multipath = SCSI_MULTIPATH_OFF;
 
+static LIST_HEAD(scsi_mpath_heads_list);
+static DEFINE_MUTEX(scsi_mpath_heads_lock);
+static DEFINE_IDA(scsi_multipath_dev_ida);
+
 static int scsi_multipath_param_set(const char *val, const struct kernel_param *kp)
 {
 	int mode;
@@ -72,6 +76,55 @@ static int scsi_mpath_unique_lun_id(struct scsi_device *sdev)
 	return 0;
 }
 
+static void scsi_mpath_head_release(struct device *dev)
+{
+	struct scsi_mpath_head *scsi_mpath_head =
+		container_of(dev, struct scsi_mpath_head, dev);
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+
+	ida_free(&scsi_multipath_dev_ida, scsi_mpath_head->index);
+	ida_destroy(&scsi_mpath_head->ida);
+	mpath_head_uninit(mpath_head);
+	kfree(scsi_mpath_head);
+}
+
+static ssize_t scsi_mpath_device_vpd_id_show(struct device *dev,
+			struct device_attribute *attr,
+			char *buf)
+{
+	struct scsi_mpath_head *scsi_mpath_head =
+		container_of(dev, struct scsi_mpath_head, dev);
+
+	return sysfs_emit(buf, "%s\n", scsi_mpath_head->vpd_id);
+}
+static DEVICE_ATTR(vpd_id, S_IRUGO, scsi_mpath_device_vpd_id_show, NULL);
+
+static struct attribute *scsi_mpath_device_attrs[] = {
+	&dev_attr_vpd_id.attr,
+	NULL
+};
+
+static const struct attribute_group scsi_mpath_device_attrs_group = {
+	.attrs = scsi_mpath_device_attrs,
+};
+
+static bool scsi_multipath_sysfs_group_visible(struct kobject *kobj)
+{
+	return true;
+}
+DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(scsi_multipath_sysfs)
+
+static const struct attribute_group *scsi_mpath_device_groups[] = {
+	&scsi_mpath_device_attrs_group,
+	NULL
+};
+
+static const struct class scsi_mpath_device_class = {
+	.name = "scsi_mpath_device",
+	.dev_groups = scsi_mpath_device_groups,
+	.dev_release = scsi_mpath_head_release,
+};
+
 static int scsi_multipath_sdev_init(struct scsi_device *sdev)
 {
 	struct Scsi_Host *shost = sdev->host;
@@ -91,6 +144,74 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
 	return 0;
 }
 
+static struct mpath_head_template smpdt = {
+};
+
+static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
+{
+	struct scsi_mpath_head *scsi_mpath_head;
+	int ret;
+
+	scsi_mpath_head = kzalloc(sizeof(*scsi_mpath_head), GFP_KERNEL);
+	if (!scsi_mpath_head)
+		return NULL;
+
+	ida_init(&scsi_mpath_head->ida);
+
+	if (mpath_head_init(&scsi_mpath_head->mpath_head))
+		goto out_free;
+	scsi_mpath_head->mpath_head.mpdt = &smpdt;
+
+	strscpy(scsi_mpath_head->vpd_id, vpd_id,
+		SCSI_MPATH_DEVICE_ID_LEN);
+
+	scsi_mpath_head->index = ida_alloc(&scsi_multipath_dev_ida, GFP_KERNEL);
+	if (scsi_mpath_head->index < 0)
+		goto out_uninit_head;
+	kref_init(&scsi_mpath_head->ref);
+
+	device_initialize(&scsi_mpath_head->dev);
+	scsi_mpath_head->dev.class = &scsi_mpath_device_class;
+	ret = dev_set_name(&scsi_mpath_head->dev, "scsi_mpath_device%d",
+				scsi_mpath_head->index);
+	if (ret) {
+		put_device(&scsi_mpath_head->dev);
+		return NULL;
+	}
+
+	ret = device_add(&scsi_mpath_head->dev);
+	if (ret) {
+		put_device(&scsi_mpath_head->dev);
+		return NULL;
+	}
+
+	return scsi_mpath_head;
+
+out_uninit_head:
+	mpath_head_uninit(&scsi_mpath_head->mpath_head);
+out_free:
+	kfree(scsi_mpath_head);
+	return NULL;
+}
+
+static struct scsi_mpath_head *scsi_mpath_find_head(
+			struct scsi_mpath_device *scsi_mpath_dev)
+{
+	struct scsi_mpath_head *scsi_mpath_head;
+
+	list_for_each_entry(scsi_mpath_head, &scsi_mpath_heads_list, entry) {
+		if (strncmp(scsi_mpath_head->vpd_id,
+			scsi_mpath_dev->device_id_str,
+			SCSI_MPATH_DEVICE_ID_LEN) == 0) {
+			if (scsi_mpath_get_head(scsi_mpath_head))
+				continue;
+			return scsi_mpath_head;
+		}
+	}
+
+	return NULL;
+}
+
 static void scsi_multipath_sdev_uninit(struct scsi_device *sdev)
 {
 	kfree(sdev->scsi_mpath_dev);
@@ -99,6 +220,7 @@ static void scsi_multipath_sdev_uninit(struct scsi_device *sdev)
 
 int scsi_mpath_dev_alloc(struct scsi_device *sdev)
 {
+	struct scsi_mpath_head *scsi_mpath_head;
 	int ret;
 
 	if (scsi_multipath == SCSI_MULTIPATH_OFF)
@@ -118,13 +240,51 @@ int scsi_mpath_dev_alloc(struct scsi_device *sdev)
 	if (ret < 0)
 		goto out_uninit;
 
-	return 0;
+	mutex_lock(&scsi_mpath_heads_lock);
+	scsi_mpath_head = scsi_mpath_find_head(sdev->scsi_mpath_dev);
+	if (scsi_mpath_head)
+		goto found;
+	scsi_mpath_head =
+		scsi_mpath_alloc_head(sdev->scsi_mpath_dev->device_id_str);
+	if (!scsi_mpath_head) {
+		sdev_printk(KERN_NOTICE, sdev, "could not allocate multipath head, device multipathing disabled\n");
+		mutex_unlock(&scsi_mpath_heads_lock);
+		goto out_uninit;
+	}
 
+	list_add_tail(&scsi_mpath_head->entry, &scsi_mpath_heads_list);
+found:
+	mutex_unlock(&scsi_mpath_heads_lock);
+	ret = ida_alloc(&scsi_mpath_head->ida, GFP_KERNEL);
+	if (ret < 0)
+		goto out_put_head;
+	sdev->scsi_mpath_dev->index = ret;
+	sdev->scsi_mpath_dev->scsi_mpath_head = scsi_mpath_head;
+	sdev->scsi_mpath_dev->mpath_device.mpath_head =
+				&scsi_mpath_head->mpath_head;
+	return 0;
+out_put_head:
+	scsi_mpath_put_head(scsi_mpath_head);
 out_uninit:
 	scsi_multipath_sdev_uninit(sdev);
 	return 0;
 }
 
+static void scsi_mpath_remove_head(struct scsi_mpath_device *scsi_mpath_dev)
+{
+	scsi_mpath_put_head(scsi_mpath_dev->scsi_mpath_head);
+	scsi_mpath_dev->scsi_mpath_head = NULL;
+}
+
+void scsi_mpath_remove_device(struct scsi_mpath_device *scsi_mpath_dev)
+{
+	struct scsi_mpath_head *scsi_mpath_head = scsi_mpath_dev->scsi_mpath_head;
+
+	ida_free(&scsi_mpath_head->ida, scsi_mpath_dev->index);
+
+	scsi_mpath_remove_head(scsi_mpath_dev);
+}
+
 void scsi_mpath_dev_release(struct scsi_device *sdev)
 {
 	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
@@ -135,13 +295,48 @@ void scsi_mpath_dev_release(struct scsi_device *sdev)
 	scsi_multipath_sdev_uninit(sdev);
 }
 
-int __init scsi_multipath_init(void)
+int scsi_mpath_get_head(struct scsi_mpath_head *scsi_mpath_head)
 {
+	if (!kref_get_unless_zero(&scsi_mpath_head->ref))
+		return -ENXIO;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(scsi_mpath_get_head);
+
+static void scsi_mpath_free_head(struct kref *ref)
+{
+	struct scsi_mpath_head *scsi_mpath_head =
+		container_of(ref, struct scsi_mpath_head, ref);
+
+	/*
+	 * If we race with scsi_mpath_find_head(), then that function may
+	 * find this scsi_mpath_head in the heads list; however we would fail
+	 * to take a reference to this scsi_mpath_head and continue the search.
+	 * As such, it is safe to call device_unregister (and free
+	 * scsi_mpath_head) after we delete this head from the list.
+	 */
+	mutex_lock(&scsi_mpath_heads_lock);
+	list_del_init(&scsi_mpath_head->entry);
+	mutex_unlock(&scsi_mpath_heads_lock);
+
+	device_unregister(&scsi_mpath_head->dev);
+}
+
+void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head)
+{
+	kref_put(&scsi_mpath_head->ref, scsi_mpath_free_head);
+}
+EXPORT_SYMBOL_GPL(scsi_mpath_put_head);
+
+int __init scsi_multipath_init(void)
+{
+	return class_register(&scsi_mpath_device_class);
+}
 
 void __exit scsi_multipath_exit(void)
 {
+	ida_destroy(&scsi_multipath_dev_ida);
+	class_unregister(&scsi_mpath_device_class);
 }
 
 MODULE_LICENSE("GPL");
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 2f80d703ce640..799569b74b535 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1525,6 +1525,9 @@ void __scsi_remove_device(struct scsi_device *sdev)
 	 */
 	scsi_target_reap(scsi_target(sdev));
 
+	if (sdev->scsi_mpath_dev)
+		scsi_mpath_remove_device(sdev->scsi_mpath_dev);
+
 	put_device(dev);
 }
 
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
index d3d410dafd17a..50298056181f9 100644
--- a/include/scsi/scsi_multipath.h
+++ b/include/scsi/scsi_multipath.h
@@ -19,12 +19,25 @@
 #ifdef CONFIG_SCSI_MULTIPATH
 #define SCSI_MPATH_DEVICE_ID_LEN 256
 
+struct scsi_mpath_head {
+	struct mpath_head	mpath_head;
+	char			vpd_id[SCSI_MPATH_DEVICE_ID_LEN];
+	struct list_head	entry;
+	struct ida		ida;
+	struct kref		ref;
+	struct device		dev;
+	int			index;
+};
+
 struct scsi_mpath_device {
 	struct mpath_device	mpath_device;
 	struct scsi_device 	*sdev;
+	int			index;
+	struct scsi_mpath_head	*scsi_mpath_head;
 
 	char			device_id_str[SCSI_MPATH_DEVICE_ID_LEN];
 };
+
 #define to_scsi_mpath_device(d) \
 	container_of(d, struct scsi_mpath_device, mpath_device)
 
@@ -32,8 +45,13 @@ int scsi_mpath_dev_alloc(struct scsi_device *sdev);
 void scsi_mpath_dev_release(struct scsi_device *sdev);
 int scsi_multipath_init(void);
 void scsi_multipath_exit(void);
+void scsi_mpath_remove_device(struct scsi_mpath_device *scsi_mpath_dev);
+int scsi_mpath_get_head(struct scsi_mpath_head *scsi_mpath_head);
+void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head);
 #else /* CONFIG_SCSI_MULTIPATH */
 
+struct scsi_mpath_head {
+};
 struct scsi_mpath_device {
 };
 
@@ -51,5 +69,18 @@ static inline int scsi_multipath_init(void)
 static inline void scsi_multipath_exit(void)
 {
 }
+static inline
+void scsi_mpath_remove_device(struct scsi_mpath_device *scsi_mpath_dev)
+{
+}
+static inline
+int scsi_mpath_get_head(struct scsi_mpath_head *scsi_mpath_head)
+{
+	return 0;
+}
+static inline
+void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head)
+{
+}
 #endif /* CONFIG_SCSI_MULTIPATH */
 #endif /* _SCSI_SCSI_MULTIPATH_H */
-- 
2.43.7

[PATCH v6 13/27] scsi-multipath: provide sysfs link from to scsi_device

[John Garry]

From: John Garry <john.garry@linux.dev>

Provide a link in sysfs from a scsi_mpath_device to member scsi_device's.

An example is as follows:
# ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/multipath/
total 0
lrwxrwxrwx    1 root     root             0 Feb 24 12:01 8:0:0:0 -> ../../../../platform/host8/session1/target8:0:0/8:0:0:0
lrwxrwxrwx    1 root     root             0 Feb 24 12:01 9:0:0:0 -> ../../../../platform/host9/session2/target9:0:0/9:0:0:0

Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_multipath.c | 43 +++++++++++++++++++++++++++++++++++
 drivers/scsi/scsi_sysfs.c     |  5 ++++
 include/scsi/scsi_multipath.h |  8 +++++++
 3 files changed, 56 insertions(+)

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 30a2d01be8aaf..1ec7bf1835de0 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -114,11 +114,54 @@ static bool scsi_multipath_sysfs_group_visible(struct kobject *kobj)
 }
 DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(scsi_multipath_sysfs)
 
+static struct attribute dummy_attr = {
+	.name = "dummy",
+};
+
+static struct attribute *scsi_mpath_attrs[] = {
+	&dummy_attr,
+	NULL
+};
+
+static const struct attribute_group scsi_mpath_attr_group = {
+	.name           = "multipath",
+	.attrs		= scsi_mpath_attrs,
+	.is_visible     = SYSFS_GROUP_VISIBLE(scsi_multipath_sysfs),
+};
+
 static const struct attribute_group *scsi_mpath_device_groups[] = {
 	&scsi_mpath_device_attrs_group,
+	&scsi_mpath_attr_group,
 	NULL
 };
 
+void scsi_mpath_add_sysfs_link(struct scsi_device *sdev)
+{
+	struct device *target = &sdev->sdev_gendev;
+	struct scsi_mpath_head *scsi_mpath_head =
+		sdev->scsi_mpath_dev->scsi_mpath_head;
+	struct device *source = &scsi_mpath_head->dev;
+	int error;
+
+	error = sysfs_add_link_to_group(&source->kobj, "multipath",
+			&target->kobj, dev_name(target));
+	if (error) {
+		sdev_printk(KERN_INFO, sdev, "Failed to create mpath sysfs link, error=%d\n",
+				    error);
+	}
+}
+
+void scsi_mpath_remove_sysfs_link(struct scsi_device *sdev)
+{
+	struct device *target = &sdev->sdev_gendev;
+	struct scsi_mpath_head *scsi_mpath_head =
+		sdev->scsi_mpath_dev->scsi_mpath_head;
+	struct device *source = &scsi_mpath_head->dev;
+
+	sysfs_remove_link_from_group(&source->kobj, "multipath",
+		dev_name(target));
+}
+
 static const struct class scsi_mpath_device_class = {
 	.name = "scsi_mpath_device",
 	.dev_groups = scsi_mpath_device_groups,
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 799569b74b535..ce51fcc002bbf 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1443,6 +1443,9 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
 	transport_add_device(&sdev->sdev_gendev);
 	sdev->is_visible = 1;
 
+	if (sdev->scsi_mpath_dev)
+		scsi_mpath_add_sysfs_link(sdev);
+
 	if (IS_ENABLED(CONFIG_BLK_DEV_BSG)) {
 		sdev->bsg_dev = scsi_bsg_register_queue(sdev);
 		if (IS_ERR(sdev->bsg_dev)) {
@@ -1495,6 +1498,8 @@ void __scsi_remove_device(struct scsi_device *sdev)
 
 		if (IS_ENABLED(CONFIG_BLK_DEV_BSG) && sdev->bsg_dev)
 			bsg_unregister_queue(sdev->bsg_dev);
+		if (sdev->scsi_mpath_dev)
+			scsi_mpath_remove_sysfs_link(sdev);
 		device_unregister(&sdev->sdev_dev);
 		transport_remove_device(dev);
 		device_del(dev);
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
index 50298056181f9..a9fd02bc42371 100644
--- a/include/scsi/scsi_multipath.h
+++ b/include/scsi/scsi_multipath.h
@@ -46,6 +46,8 @@ void scsi_mpath_dev_release(struct scsi_device *sdev);
 int scsi_multipath_init(void);
 void scsi_multipath_exit(void);
 void scsi_mpath_remove_device(struct scsi_mpath_device *scsi_mpath_dev);
+void scsi_mpath_add_sysfs_link(struct scsi_device *sdev);
+void scsi_mpath_remove_sysfs_link(struct scsi_device *sdev);
 int scsi_mpath_get_head(struct scsi_mpath_head *scsi_mpath_head);
 void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head);
 #else /* CONFIG_SCSI_MULTIPATH */
@@ -82,5 +84,11 @@ static inline
 void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head)
 {
 }
+static inline void scsi_mpath_add_sysfs_link(struct scsi_device *sdev)
+{
+}
+static inline void scsi_mpath_remove_sysfs_link(struct scsi_device *sdev)
+{
+}
 #endif /* CONFIG_SCSI_MULTIPATH */
 #endif /* _SCSI_SCSI_MULTIPATH_H */
-- 
2.43.7

[PATCH v6 14/27] scsi-multipath: support iopolicy

[John Garry]

Add support to set the multipath iopolicy.

The iopolicy member is per scsi_mpath_head structure.

A module param is added so that the default iopolicy may be set.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_multipath.c | 52 +++++++++++++++++++++++++++++++++++
 include/scsi/scsi_multipath.h |  3 ++
 2 files changed, 55 insertions(+)

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 1ec7bf1835de0..593f94aac70e9 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -61,6 +61,23 @@ static const struct kernel_param_ops multipath_param_ops = {
 module_param_cb(multipath, &multipath_param_ops, &scsi_multipath, 0444);
 MODULE_PARM_DESC(multipath, "turn on native multipath support, options: on, off, always");
 
+static enum mpath_iopolicy_e iopolicy = MPATH_IOPOLICY_NUMA;
+
+static int scsi_mpath_set_iopolicy_param(const char *val, const struct kernel_param *kp)
+{
+	return mpath_set_iopolicy(val, &iopolicy);
+}
+
+static int scsi_mpath_get_iopolicy_param(char *buf, const struct kernel_param *kp)
+{
+	return mpath_get_iopolicy(buf, iopolicy);
+}
+
+module_param_call(multipath_iopolicy, scsi_mpath_set_iopolicy_param,
+		scsi_mpath_get_iopolicy_param, &iopolicy, 0644);
+MODULE_PARM_DESC(multipath_iopolicy,
+	"Default multipath I/O policy; 'numa' (default), 'round-robin' or 'queue-depth'");
+
 static int scsi_mpath_unique_lun_id(struct scsi_device *sdev)
 {
 	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
@@ -99,8 +116,41 @@ static ssize_t scsi_mpath_device_vpd_id_show(struct device *dev,
 }
 static DEVICE_ATTR(vpd_id, S_IRUGO, scsi_mpath_device_vpd_id_show, NULL);
 
+static ssize_t scsi_mpath_device_iopolicy_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct scsi_mpath_head *scsi_mpath_head =
+		container_of(dev, struct scsi_mpath_head, dev);
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+
+	if (!mpath_iopolicy_store(&scsi_mpath_head->iopolicy, buf))
+		return -EINVAL;
+
+	mpath_clear_paths(mpath_head);
+	mpath_schedule_requeue_work(mpath_head);
+	/*
+	 * Ensure requeue work completes, as this work could run later when
+	 * the mpath_head is gone.
+	 */
+	flush_work(&mpath_head->requeue_work);
+	return count;
+}
+
+static ssize_t scsi_mpath_device_iopolicy_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct scsi_mpath_head *scsi_mpath_head =
+		container_of(dev, struct scsi_mpath_head, dev);
+
+	return mpath_iopolicy_show(&scsi_mpath_head->iopolicy, buf);
+}
+
+static DEVICE_ATTR(iopolicy, S_IRUGO | S_IWUSR,
+		scsi_mpath_device_iopolicy_show, scsi_mpath_device_iopolicy_store);
+
 static struct attribute *scsi_mpath_device_attrs[] = {
 	&dev_attr_vpd_id.attr,
+	&dev_attr_iopolicy.attr,
 	NULL
 };
 
@@ -204,6 +254,8 @@ static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
 	if (mpath_head_init(&scsi_mpath_head->mpath_head))
 		goto out_free;
 	scsi_mpath_head->mpath_head.mpdt = &smpdt;
+	scsi_mpath_head->iopolicy = iopolicy;
+	scsi_mpath_head->mpath_head.iopolicy = &scsi_mpath_head->iopolicy;
 
 	strscpy(scsi_mpath_head->vpd_id, vpd_id,
 		SCSI_MPATH_DEVICE_ID_LEN);
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
index a9fd02bc42371..2dc02313b0496 100644
--- a/include/scsi/scsi_multipath.h
+++ b/include/scsi/scsi_multipath.h
@@ -25,6 +25,7 @@ struct scsi_mpath_head {
 	struct list_head	entry;
 	struct ida		ida;
 	struct kref		ref;
+	enum mpath_iopolicy_e	iopolicy;
 	struct device		dev;
 	int			index;
 };
@@ -40,6 +41,8 @@ struct scsi_mpath_device {
 
 #define to_scsi_mpath_device(d) \
 	container_of(d, struct scsi_mpath_device, mpath_device)
+#define to_scsi_mpath_head(d) \
+	container_of(d, struct scsi_mpath_head, mpath_head)
 
 int scsi_mpath_dev_alloc(struct scsi_device *sdev);
 void scsi_mpath_dev_release(struct scsi_device *sdev);
-- 
2.43.7

[PATCH v6 15/27] scsi-multipath: clone each bio

[John Garry]

From: John Garry <john.garry@linux.dev>

For failover handling, we will take the approach to resubmit each
bio.

However, unlike NVMe, for SCSI there is no guarantee that any bio submitted
is either all or none completed.

As such, for SCSI, for failover handling we will take the approach to
just re-submit the original bio. For this, clone and submit each bio.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_multipath.c | 33 +++++++++++++++++++++++++++++++++
 include/scsi/scsi_multipath.h |  1 +
 2 files changed, 34 insertions(+)

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 593f94aac70e9..0d439cac6447a 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -99,6 +99,7 @@ static void scsi_mpath_head_release(struct device *dev)
 		container_of(dev, struct scsi_mpath_head, dev);
 	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
 
+	bioset_exit(&scsi_mpath_head->bio_pool);
 	ida_free(&scsi_multipath_dev_ida, scsi_mpath_head->index);
 	ida_destroy(&scsi_mpath_head->ida);
 	mpath_head_uninit(mpath_head);
@@ -237,7 +238,34 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
 	return 0;
 }
 
+static void scsi_mpath_clone_end_io(struct bio *clone)
+{
+	struct bio *master_bio = clone->bi_private;
+
+	master_bio->bi_status = clone->bi_status;
+	bio_put(clone);
+	bio_endio(master_bio);
+}
+
+static struct bio *scsi_mpath_clone_bio(struct bio *bio)
+{
+	struct mpath_head *mpath_head = bio->bi_bdev->bd_disk->private_data;
+	struct scsi_mpath_head *scsi_mpath_head = to_scsi_mpath_head(mpath_head);
+	struct bio *clone;
+
+	clone = bio_alloc_clone(bio->bi_bdev, bio, GFP_NOIO,
+				&scsi_mpath_head->bio_pool);
+	if (!clone)
+		return NULL;
+
+	clone->bi_end_io = scsi_mpath_clone_end_io;
+	clone->bi_private = bio;
+
+	return clone;
+}
+
 static struct mpath_head_template smpdt = {
+	.clone_bio = scsi_mpath_clone_bio,
 };
 
 static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
@@ -263,6 +291,9 @@ static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
 	scsi_mpath_head->index = ida_alloc(&scsi_multipath_dev_ida, GFP_KERNEL);
 	if (scsi_mpath_head->index < 0)
 		goto out_uninit_head;
+	if (bioset_init(&scsi_mpath_head->bio_pool, BIO_POOL_SIZE,
+			0, BIOSET_PERCPU_CACHE))
+		goto out_ida_free;
 	kref_init(&scsi_mpath_head->ref);
 
 	device_initialize(&scsi_mpath_head->dev);
@@ -282,6 +313,8 @@ static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
 
 	return scsi_mpath_head;
 
+out_ida_free:
+	ida_free(&scsi_multipath_dev_ida, scsi_mpath_head->index);
 out_uninit_head:
 	mpath_head_uninit(&scsi_mpath_head->mpath_head);
 out_free:
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
index 2dc02313b0496..5cc37acaa664f 100644
--- a/include/scsi/scsi_multipath.h
+++ b/include/scsi/scsi_multipath.h
@@ -26,6 +26,7 @@ struct scsi_mpath_head {
 	struct ida		ida;
 	struct kref		ref;
 	enum mpath_iopolicy_e	iopolicy;
+	struct bio_set		bio_pool;
 	struct device		dev;
 	int			index;
 };
-- 
2.43.7

[PATCH v6 16/27] scsi-multipath: clear path when device is blocked

[John Garry]

From: John Garry <john.garry@linux.dev>

Add scsi_mpath_dev_clear_path() to clear a device path when it becomes
blocked.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_lib.c       |  3 +++
 drivers/scsi/scsi_multipath.c | 10 ++++++++++
 include/scsi/scsi_multipath.h |  5 +++++
 3 files changed, 18 insertions(+)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 22e2e3223440d..e2452392671f5 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -35,6 +35,7 @@
 #include <scsi/scsi_host.h>
 #include <scsi/scsi_transport.h> /* scsi_init_limits() */
 #include <scsi/scsi_dh.h>
+#include <scsi/scsi_multipath.h>
 
 #include <trace/events/scsi.h>
 
@@ -3016,6 +3017,8 @@ static void scsi_device_block(struct scsi_device *sdev, void *data)
 		scsi_stop_queue(sdev);
 
 	mutex_unlock(&sdev->state_mutex);
+	if ((err == 0) && sdev->scsi_mpath_dev)
+		scsi_mpath_dev_clear_path(sdev->scsi_mpath_dev);
 
 	WARN_ONCE(err, "%s: failed to block %s in state %d\n",
 		  __func__, dev_name(&sdev->sdev_gendev), state);
diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 0d439cac6447a..b23cd6f12c602 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -238,6 +238,16 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
 	return 0;
 }
 
+void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
+{
+       struct mpath_device *mpath_device = &scsi_mpath_dev->mpath_device;
+       struct scsi_mpath_head *scsi_mpath_head = scsi_mpath_dev->scsi_mpath_head;
+       struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+
+       if (mpath_clear_current_path(mpath_device))
+               mpath_synchronize(mpath_head);
+}
+
 static void scsi_mpath_clone_end_io(struct bio *clone)
 {
 	struct bio *master_bio = clone->bi_private;
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
index 5cc37acaa664f..8f58c44d96a77 100644
--- a/include/scsi/scsi_multipath.h
+++ b/include/scsi/scsi_multipath.h
@@ -54,6 +54,7 @@ void scsi_mpath_add_sysfs_link(struct scsi_device *sdev);
 void scsi_mpath_remove_sysfs_link(struct scsi_device *sdev);
 int scsi_mpath_get_head(struct scsi_mpath_head *scsi_mpath_head);
 void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head);
+void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev);
 #else /* CONFIG_SCSI_MULTIPATH */
 
 struct scsi_mpath_head {
@@ -88,6 +89,10 @@ static inline
 void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head)
 {
 }
+static inline
+void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
+{
+}
 static inline void scsi_mpath_add_sysfs_link(struct scsi_device *sdev)
 {
 }
-- 
2.43.7

[PATCH v6 17/27] scsi-multipath: revalidate paths upon device unblock

[John Garry]

From: John Garry <john.garry@linux.dev>

When a scsi_device is unblocked, we need revalidate paths which means that
we clear paths and requeue any queued bios.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_lib.c       | 2 ++
 drivers/scsi/scsi_multipath.c | 8 ++++++++
 include/scsi/scsi_multipath.h | 5 +++++
 3 files changed, 15 insertions(+)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index e2452392671f5..41d201b7e7def 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -3100,6 +3100,8 @@ static int scsi_internal_device_unblock(struct scsi_device *sdev,
 	mutex_lock(&sdev->state_mutex);
 	ret = scsi_internal_device_unblock_nowait(sdev, new_state);
 	mutex_unlock(&sdev->state_mutex);
+	if ((ret == 0) && sdev->scsi_mpath_dev)
+		scsi_mpath_revalidate_paths(sdev->scsi_mpath_dev);
 
 	return ret;
 }
diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index b23cd6f12c602..12750621a7fc0 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -238,6 +238,14 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
 	return 0;
 }
 
+void scsi_mpath_revalidate_paths(struct scsi_mpath_device *scsi_mpath_dev)
+{
+       struct scsi_mpath_head *scsi_mpath_head = scsi_mpath_dev->scsi_mpath_head;
+       struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+
+       mpath_revalidate_paths(mpath_head);
+}
+
 void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
 {
        struct mpath_device *mpath_device = &scsi_mpath_dev->mpath_device;
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
index 8f58c44d96a77..cc77e63c43da3 100644
--- a/include/scsi/scsi_multipath.h
+++ b/include/scsi/scsi_multipath.h
@@ -55,6 +55,7 @@ void scsi_mpath_remove_sysfs_link(struct scsi_device *sdev);
 int scsi_mpath_get_head(struct scsi_mpath_head *scsi_mpath_head);
 void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head);
 void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev);
+void scsi_mpath_revalidate_paths(struct scsi_mpath_device *scsi_mpath_dev);
 #else /* CONFIG_SCSI_MULTIPATH */
 
 struct scsi_mpath_head {
@@ -93,6 +94,10 @@ static inline
 void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
 {
 }
+static inline
+void scsi_mpath_revalidate_paths(struct scsi_mpath_device *scsi_mpath_dev)
+{
+}
 static inline void scsi_mpath_add_sysfs_link(struct scsi_device *sdev)
 {
 }
-- 
2.43.7

[PATCH v6 18/27] scsi-multipath: failover handling

[John Garry]

From: John Garry <john.garry@linux.dev>

Failover occurs when the scsi_cmnd has failed and it is discovered that the
target scsi_device has transport down.

For a scsi command which suffers failover, requeue the master bio of each
bio attached to its request.

A bio which for which failover occurs is handled in
scsi_mpath_clone_end_io(). Failover is detected for blk_path_error()
occuring, same as how dm-mpath detects this.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_multipath.c | 39 +++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 12750621a7fc0..6ced89576368c 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -256,10 +256,49 @@ void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
                mpath_synchronize(mpath_head);
 }
 
+static inline void bio_list_add_clone(struct bio_list *bl,
+				struct bio *clone)
+{
+	struct bio *master_bio = clone->bi_private;
+
+	if (bl->tail)
+		bl->tail->bi_next = master_bio;
+	else
+		bl->head = master_bio;
+	bl->tail = master_bio;
+	bio_put(clone);
+}
+
 static void scsi_mpath_clone_end_io(struct bio *clone)
 {
 	struct bio *master_bio = clone->bi_private;
 
+	if (clone->bi_status && blk_path_error(clone->bi_status)) {
+		struct block_device *bi_bdev = clone->bi_bdev;
+		struct request_queue *q = bi_bdev->bd_queue;
+		struct scsi_device *sdev = scsi_device_from_queue(q);
+		struct scsi_mpath_device *scsi_mpath_dev;
+		struct mpath_device *mpath_device;
+		struct mpath_head *mpath_head;
+		unsigned long flags;
+
+		if (!sdev)
+			goto end_bio;
+
+		scsi_mpath_dev = sdev->scsi_mpath_dev;
+		mpath_device = &scsi_mpath_dev->mpath_device;
+		mpath_head = mpath_device->mpath_head;
+
+		spin_lock_irqsave(&mpath_head->requeue_lock, flags);
+		bio_list_add_clone(&mpath_head->requeue_list, clone);
+		spin_unlock_irqrestore(&mpath_head->requeue_lock, flags);
+
+		mpath_schedule_requeue_work(mpath_head);
+		put_device(&sdev->sdev_gendev);
+		return;
+	}
+
+end_bio:
 	master_bio->bi_status = clone->bi_status;
 	bio_put(clone);
 	bio_endio(master_bio);
-- 
2.43.7

[PATCH v6 19/27] scsi-multipath: provide callbacks for path state

[John Garry]

Provide callbacks for .is_disabled, .is_optimized, and .available_path.

These all use scsi_device.sdev_state and scsi_device.access_state.

Member scsi_device.access_state will be driven by ALUA. Currently
only device handlers support this, and in future we will have core
SCSI support for implicit ALUA (not relying on device handlers).

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_multipath.c | 49 +++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 6ced89576368c..2a05f177a1dcd 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -321,7 +321,56 @@ static struct bio *scsi_mpath_clone_bio(struct bio *bio)
 	return clone;
 }
 
+static bool scsi_mpath_is_disabled(struct mpath_device *mpath_device)
+{
+	struct scsi_mpath_device *scsi_mpath_dev =
+				to_scsi_mpath_device(mpath_device);
+	struct scsi_device *sdev = scsi_mpath_dev->sdev;
+	unsigned char access_state = READ_ONCE(sdev->access_state) &
+					SCSI_ACCESS_STATE_MASK;
+
+	if (READ_ONCE(sdev->sdev_state) != SDEV_RUNNING)
+		return true;
+
+	if (access_state == SCSI_ACCESS_STATE_OPTIMAL ||
+	    access_state == SCSI_ACCESS_STATE_ACTIVE)
+		return false;
+
+	return true;
+}
+
+static bool scsi_mpath_is_optimized(struct mpath_device *mpath_device)
+{
+	struct scsi_mpath_device *scsi_mpath_dev =
+				to_scsi_mpath_device(mpath_device);
+	struct scsi_device *sdev = scsi_mpath_dev->sdev;
+	unsigned char access_state = READ_ONCE(sdev->access_state) &
+					SCSI_ACCESS_STATE_MASK;
+
+	if (READ_ONCE(sdev->sdev_state) != SDEV_RUNNING)
+		return false;
+
+	return access_state == SCSI_ACCESS_STATE_OPTIMAL;
+}
+
+static bool scsi_mpath_available_path(struct mpath_device *mpath_device)
+{
+	struct scsi_mpath_device *scsi_mpath_dev =
+				to_scsi_mpath_device(mpath_device);
+	struct scsi_device *sdev = scsi_mpath_dev->sdev;
+	enum scsi_device_state sdev_state = READ_ONCE(sdev->sdev_state);
+
+	if (sdev_state == SDEV_RUNNING || sdev_state == SDEV_QUIESCE ||
+	    sdev_state == SDEV_BLOCK || sdev_state == SDEV_CREATED_BLOCK)
+		return true;
+
+	return false;
+}
+
 static struct mpath_head_template smpdt = {
+	.is_disabled = scsi_mpath_is_disabled,
+	.is_optimized = scsi_mpath_is_optimized,
+	.available_path = scsi_mpath_available_path,
 	.clone_bio = scsi_mpath_clone_bio,
 };
 
-- 
2.43.7

[PATCH v6 20/27] scsi-multipath: add scsi_mpath_{start,end}_request()

[John Garry]

From: John Garry <john.garry@linux.dev>

Add scsi_mpath_{start,end}_request() to handle updating private multipath
request data, like nvme_mpath_{start,end}_request().

New member Scsi_Host.mpath_nr_active is added. This is required for queue
depth multipath iopolicy.

For NVMe, this count is per controller. The reason is that many NSes may
be connected to a controller, so congestion should be judged at
controller level.

SCSI has no definition of a controller, but SCSI host is a comparable
concept.

Indeed, many SCSI disks may be connected to the same SCSI host, so it
makes sense to count number of active requests at this point. However,
for a transport like iSCSI Initiator over TCP/IP, we have a separate SCSI
host per SCSI device (so there the count would be same at SCSI device
level).

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_lib.c       |  8 +++-
 drivers/scsi/scsi_multipath.c | 69 +++++++++++++++++++++++++++++++++++
 include/scsi/scsi_cmnd.h      |  9 ++++-
 include/scsi/scsi_host.h      |  4 ++
 include/scsi/scsi_multipath.h | 11 ++++++
 5 files changed, 99 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 41d201b7e7def..baafca4eea866 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -669,7 +669,10 @@ static bool scsi_end_request(struct request *req, blk_status_t error,
 	struct scsi_device *sdev = cmd->device;
 	struct request_queue *q = sdev->request_queue;
 
-	if (blk_update_request(req, error, bytes))
+	if (sdev->scsi_mpath_dev && is_mpath_request(req)) {
+		if (scsi_mpath_end_request(req, error, bytes))
+			return true;
+	} else if (blk_update_request(req, error, bytes))
 		return true;
 
 	if (q->limits.features & BLK_FEAT_ADD_RANDOM)
@@ -1917,6 +1920,9 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 	memset(cmd->sense_buffer, 0, SCSI_SENSE_BUFFERSIZE);
 	cmd->submitter = SUBMITTED_BY_BLOCK_LAYER;
 
+	if (sdev->scsi_mpath_dev && is_mpath_request(req))
+		scsi_mpath_start_request(req);
+
 	blk_mq_start_request(req);
 	if (blk_mq_is_reserved_rq(req)) {
 		reason = shost->hostt->queue_reserved_command(shost, cmd);
diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 2a05f177a1dcd..0adc5adb4b234 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -562,6 +562,75 @@ void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head)
 }
 EXPORT_SYMBOL_GPL(scsi_mpath_put_head);
 
+void scsi_mpath_start_request(struct request *req)
+{
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
+	struct scsi_device *sdev = scmd->device;
+	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
+	struct scsi_mpath_head *scsi_mpath_head =
+				scsi_mpath_dev->scsi_mpath_head;
+	struct bio *clone = req->bio, *master;
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+	struct gendisk *disk = mpath_head->disk;
+
+	if (mpath_qd_iopolicy(&scsi_mpath_head->iopolicy) &&
+	    !(scmd->flags & SCMD_MPATH_CNT_ACTIVE)) {
+		struct Scsi_Host *shost = sdev->host;
+
+		atomic_inc(&shost->mpath_nr_active);
+		scmd->flags |= SCMD_MPATH_CNT_ACTIVE;
+	}
+
+	if (blk_rq_is_passthrough(req) || !blk_queue_io_stat(disk->queue))
+		return;
+
+	if (!clone) {
+		scmd->flags &= ~SCMD_MPATH_IO_STATS;
+		return;
+	}
+
+	if (scmd->flags & SCMD_MPATH_IO_STATS)
+		return;
+
+	master = clone->bi_private;
+
+	scmd->flags |= SCMD_MPATH_IO_STATS;
+	scmd->start_time = bdev_start_io_acct(master->bi_bdev, req_op(req),
+				jiffies);
+	scmd->start_bytes = blk_rq_bytes(req);
+}
+
+bool scsi_mpath_end_request(struct request *req, blk_status_t error,
+				unsigned int nr_bytes)
+{
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
+	struct scsi_device *sdev = scmd->device;
+	struct block_device *bi_bdev = NULL;
+
+	if (scmd->flags & SCMD_MPATH_IO_STATS) {
+		struct bio *clone = req->bio, *master = clone->bi_private;
+
+		bi_bdev = master->bi_bdev;
+	}
+
+	if (blk_update_request(req, error, nr_bytes))
+		return true;
+
+	if (scmd->flags & SCMD_MPATH_CNT_ACTIVE) {
+		struct Scsi_Host *shost = sdev->host;
+
+		atomic_dec_if_positive(&shost->mpath_nr_active);
+	}
+
+	if (!(scmd->flags & SCMD_MPATH_IO_STATS))
+		return false;
+	bdev_end_io_acct(bi_bdev, req_op(req),
+			 scmd->start_bytes >> SECTOR_SHIFT,
+			 scmd->start_time);
+
+	return false;
+}
+
 int __init scsi_multipath_init(void)
 {
 	return class_register(&scsi_mpath_device_class);
diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h
index 8ecfb94049db5..52227e2cd9749 100644
--- a/include/scsi/scsi_cmnd.h
+++ b/include/scsi/scsi_cmnd.h
@@ -58,8 +58,11 @@ struct scsi_pointer {
  */
 #define SCMD_FORCE_EH_SUCCESS	(1 << 3)
 #define SCMD_FAIL_IF_RECOVERING	(1 << 4)
+#define SCMD_MPATH_IO_STATS	(1 << 5)
+#define SCMD_MPATH_CNT_ACTIVE	(1 << 6)
 /* flags preserved across unprep / reprep */
-#define SCMD_PRESERVED_FLAGS	(SCMD_INITIALIZED | SCMD_FAIL_IF_RECOVERING)
+#define SCMD_PRESERVED_FLAGS	(SCMD_INITIALIZED | SCMD_FAIL_IF_RECOVERING | \
+				SCMD_MPATH_IO_STATS | SCMD_MPATH_CNT_ACTIVE)
 
 /* for scmd->state */
 #define SCMD_STATE_COMPLETE	0
@@ -139,6 +142,10 @@ struct scsi_cmnd {
 					 * to release this memory.  (The memory
 					 * obtained by scsi_malloc is guaranteed
 					 * to be at an address < 16Mb). */
+	#ifdef CONFIG_SCSI_MULTIPATH
+	unsigned long		start_time;
+	unsigned int		start_bytes;
+	#endif
 
 	int result;		/* Status code from lower level driver */
 };
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 7e2011830ba4b..cf4fff3ea1a8e 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -753,6 +753,10 @@ struct Scsi_Host {
 	/* Delay for runtime autosuspend */
 	int rpm_autosuspend_delay;
 
+	#ifdef CONFIG_SCSI_MULTIPATH
+	atomic_t mpath_nr_active;
+	#endif
+
 	/*
 	 * We should ensure that this is aligned, both for better performance
 	 * and also because some compilers (m68k) don't automatically force
diff --git a/include/scsi/scsi_multipath.h b/include/scsi/scsi_multipath.h
index cc77e63c43da3..c3398e9f03f16 100644
--- a/include/scsi/scsi_multipath.h
+++ b/include/scsi/scsi_multipath.h
@@ -56,6 +56,9 @@ int scsi_mpath_get_head(struct scsi_mpath_head *scsi_mpath_head);
 void scsi_mpath_put_head(struct scsi_mpath_head *scsi_mpath_head);
 void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev);
 void scsi_mpath_revalidate_paths(struct scsi_mpath_device *scsi_mpath_dev);
+void scsi_mpath_start_request(struct request *req);
+bool scsi_mpath_end_request(struct request *req, blk_status_t error,
+			       unsigned int nr_bytes);
 #else /* CONFIG_SCSI_MULTIPATH */
 
 struct scsi_mpath_head {
@@ -98,6 +101,14 @@ static inline
 void scsi_mpath_revalidate_paths(struct scsi_mpath_device *scsi_mpath_dev)
 {
 }
+static inline void scsi_mpath_start_request(struct request *req)
+{
+}
+static inline bool scsi_mpath_end_request(struct request *req, blk_status_t error,
+			       unsigned int nr_bytes)
+{
+	return false;
+}
 static inline void scsi_mpath_add_sysfs_link(struct scsi_device *sdev)
 {
 }
-- 
2.43.7

[PATCH v6 21/27] scsi-multipath: add delayed disk removal support

[John Garry]

Add support in core code for delayed disk removal support. In this, the
callback calls into the scsi_driver to do the necessary removal work.

The scsi_disk driver (sd) must ensure that the scsi_mpath_device does not
go away while the delayed removal work is active, i.e. it must keep a
reference.

No reference to the scsi_disk multipath structures are kept outside that
driver, so that driver needs to provide a scsi_driver.mpath_remove_head
callback to do the necessary work.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/scsi_multipath.c | 27 +++++++++++++++++++++++++++
 include/scsi/scsi_driver.h    |  4 ++++
 2 files changed, 31 insertions(+)

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 0adc5adb4b234..a42e20d33e422 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c
@@ -367,7 +367,34 @@ static bool scsi_mpath_available_path(struct mpath_device *mpath_device)
 	return false;
 }
 
+static int scsi_mpath_remove_head_drv(struct device_driver *drv, void *data)
+{
+	struct scsi_mpath_head *scsi_mpath_head = data;
+	struct scsi_driver *scsi_driver = to_scsi_driver(drv);
+
+	if (scsi_driver->mpath_remove_head)
+		scsi_driver->mpath_remove_head(scsi_mpath_head);
+
+	return 0;
+}
+
+static void scsi_mpath_remove_head_work(struct mpath_head *mpath_head)
+{
+	struct scsi_mpath_head *scsi_mpath_head = to_scsi_mpath_head(mpath_head);
+
+	/*
+	 * scsi_mpath_remove_head_drv() could drop the final reference to
+	 * scsi_mpath_head, meaning that we pass a free'd pointer further
+	 * drivers - take a reference so that won't happen.
+	 */
+	scsi_mpath_get_head(scsi_mpath_head);
+	bus_for_each_drv(&scsi_bus_type, NULL, scsi_mpath_head,
+		scsi_mpath_remove_head_drv);
+	scsi_mpath_put_head(scsi_mpath_head);
+}
+
 static struct mpath_head_template smpdt = {
+	.remove_head = scsi_mpath_remove_head_work,
 	.is_disabled = scsi_mpath_is_disabled,
 	.is_optimized = scsi_mpath_is_optimized,
 	.available_path = scsi_mpath_available_path,
diff --git a/include/scsi/scsi_driver.h b/include/scsi/scsi_driver.h
index 249cea724abd1..d92b63d357f2a 100644
--- a/include/scsi/scsi_driver.h
+++ b/include/scsi/scsi_driver.h
@@ -8,6 +8,7 @@
 
 struct module;
 struct request;
+struct scsi_mpath_head;
 
 struct scsi_driver {
 	struct device_driver	gendrv;
@@ -22,6 +23,9 @@ struct scsi_driver {
 	int (*done)(struct scsi_cmnd *);
 	int (*eh_action)(struct scsi_cmnd *, int);
 	void (*eh_reset)(struct scsi_cmnd *);
+	#ifdef CONFIG_SCSI_MULTIPATH
+	void (*mpath_remove_head)(struct scsi_mpath_head *);
+	#endif
 };
 #define to_scsi_driver(drv) \
 	container_of((drv), struct scsi_driver, gendrv)
-- 
2.43.7

[PATCH v6 22/27] scsi: sd: add multipath disk class

[John Garry]

Add a new class, sd_mpath_disk_class, which is the multipath version of
the scsi_disk class.

Structure sd_mpath_disk is introduced to manage the multipath gendisk.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/sd.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 drivers/scsi/sd.h |  3 +++
 2 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 599e75f333343..242a15bc2c5bb 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -70,6 +70,7 @@
 #include <scsi/scsi_ioctl.h>
 #include <scsi/scsicam.h>
 #include <scsi/scsi_common.h>
+#include <scsi/scsi_multipath.h>
 
 #include "sd.h"
 #include "scsi_priv.h"
@@ -113,6 +114,39 @@ static mempool_t *sd_page_pool;
 static mempool_t *sd_large_page_pool;
 static atomic_t sd_large_page_pool_users = ATOMIC_INIT(0);
 static struct lock_class_key sd_bio_compl_lkclass;
+#ifdef CONFIG_SCSI_MULTIPATH
+struct sd_mpath_disk {
+	struct scsi_mpath_head		*scsi_mpath_head;
+};
+
+static void sd_mpath_disk_release(struct device *dev)
+{
+}
+
+static const struct class sd_mpath_disk_class = {
+	.name = "scsi_mpath_disk",
+	.dev_release = sd_mpath_disk_release,
+};
+
+static int sd_mpath_class_register(void)
+{
+	return class_register(&sd_mpath_disk_class);
+}
+
+static void sd_mpath_class_unregister(void)
+{
+	class_unregister(&sd_mpath_disk_class);
+}
+#else /* CONFIG_SCSI_MULTIPATH */
+static int sd_mpath_class_register(void)
+{
+	return 0;
+}
+
+static void sd_mpath_class_unregister(void)
+{
+}
+#endif
 
 static const char *sd_cache_types[] = {
 	"write through", "none", "write back",
@@ -4453,11 +4487,15 @@ static int __init init_sd(void)
 	if (err)
 		goto err_out;
 
+	err = sd_mpath_class_register();
+	if (err)
+		goto err_out_class;
+
 	sd_page_pool = mempool_create_page_pool(SD_MEMPOOL_SIZE, 0);
 	if (!sd_page_pool) {
 		printk(KERN_ERR "sd: can't init discard page pool\n");
 		err = -ENOMEM;
-		goto err_out_class;
+		goto err_out_mpath_class;
 	}
 
 	err = scsi_register_driver(&sd_template);
@@ -4468,6 +4506,8 @@ static int __init init_sd(void)
 
 err_out_driver:
 	mempool_destroy(sd_page_pool);
+err_out_mpath_class:
+	sd_mpath_class_unregister();
 err_out_class:
 	class_unregister(&sd_disk_class);
 err_out:
@@ -4493,6 +4533,7 @@ static void __exit exit_sd(void)
 		mempool_destroy(sd_large_page_pool);
 
 	class_unregister(&sd_disk_class);
+	sd_mpath_class_unregister();
 
 	for (i = 0; i < SD_MAJORS; i++)
 		unregister_blkdev(sd_major(i), "sd");
diff --git a/drivers/scsi/sd.h b/drivers/scsi/sd.h
index 574af82430169..22438173ea2f8 100644
--- a/drivers/scsi/sd.h
+++ b/drivers/scsi/sd.h
@@ -83,6 +83,9 @@ struct zoned_disk_info {
 
 struct scsi_disk {
 	struct scsi_device *device;
+#ifdef CONFIG_SCSI_MULTIPATH
+	struct sd_mpath_disk *sd_mpath_disk;
+#endif
 
 	/*
 	 * disk_dev is used to show attributes in /sys/class/scsi_disk/,
-- 
2.43.7

[PATCH v6 23/27] scsi: sd: add multipath disk attr groups

[John Garry]

From: John Garry <john.garry@linux.dev>

Set multipath disk attr groups, which includes delayed disk removal and
everything from mpath_attr_group.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/sd.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 242a15bc2c5bb..36c745cbb2619 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -4006,6 +4006,42 @@ static int sd_format_disk_name(char *prefix, int index, char *buf, int buflen)
 	return 0;
 }
 
+static ssize_t sd_mpath_device_delayed_removal_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct mpath_head *mpath_head = mpath_bd_device_to_head(dev);
+
+	return mpath_delayed_removal_secs_store(mpath_head, buf, count);
+}
+
+static ssize_t sd_mpath_device_delayed_removal_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct mpath_head *mpath_head = mpath_bd_device_to_head(dev);
+
+	return mpath_delayed_removal_secs_show(mpath_head, buf);
+}
+
+static DEVICE_ATTR(delayed_removal_secs, S_IRUGO | S_IWUSR,
+		sd_mpath_device_delayed_removal_show,
+		sd_mpath_device_delayed_removal_store);
+
+static struct attribute *sd_mpath_disk_attrs[] = {
+	&dev_attr_delayed_removal_secs.attr,
+	NULL
+};
+
+static const struct attribute_group sd_mpath_disk_attr_group = {
+	.attrs		= sd_mpath_disk_attrs,
+};
+
+__maybe_unused
+static const struct attribute_group *sd_mpath_disk_attr_groups[] = {
+	&sd_mpath_disk_attr_group,
+	&mpath_attr_group,
+	NULL
+};
+
 /**
  *	sd_probe - called during driver initialization and whenever a
  *	new scsi device is attached to the system. It is called once
-- 
2.43.7

[PATCH v6 24/27] scsi: sd: support multipath disk

[John Garry]

From: John Garry <john.garry@linux.dev>

Add support to attach a multipath disk.

We still allocate the gendisk per path, and this is required for the
per-path submission. However, those gendisks are marked as hidden. Those
disks are named sdX:Y, where X is the multipath disk index and Y is the
per-path index.

A global list of sd_mpath_disks is kept for matching scsi_device's.

The multipath gendisk has the name and disk->major/minor set to minic a
scsi_disk.

The following is an example of relevant scsi_disk and block sysfs
directories:

$ ls -l /sys/block/ | grep sdc
lrwxrwxrwx    1 root     root             0 Feb 24 16:01 sdc -> ../devices/virtual/scsi_mpath_disk/0/sdc
lrwxrwxrwx    1 root     root             0 Feb 24 16:01 sdc:0 -> ../devices/platform/host8/session1/target8:0:0/8:0:0:0/block/sdc:0
lrwxrwxrwx    1 root     root             0 Feb 24 16:02 sdc:1 -> ../devices/platform/host9/session2/target9:0:0/9:0:0:0/block/sdc:1

$ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/
total 0
drwxr-xr-x    2 root     root             0 Feb 24 16:03 power
drwxr-xr-x   11 root     root             0 Feb 24 16:01 sdc
lrwxrwxrwx    1 root     root             0 Feb 24 16:01 subsystem -> ../../../../class/scsi_mpath_disk
-rw-r--r--    1 root     root          4096 Feb 24 16:01 uevent

$ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/sdc/multipath/
total 0
lrwxrwxrwx    1 root     root             0 Feb 24 16:20 sdc:0 -> ../../../../../platform/host8/session1/target8:0:0/8:0:0:0/block/sdc:0
lrwxrwxrwx    1 root     root             0 Feb 24 16:20 sdc:1 -> ../../../../../platform/host9/session2/target9:0:0/9:0:0:0/block/sdc:1

$ ls -l /dev/sdc*
brw-rw----    1 root     disk        8,  32 Feb 24 16:01 /dev/sdc
brw-rw----    1 root     disk        8,  33 Feb 24 16:01 /dev/sdc1
brw-rw----    1 root     disk        8,  34 Feb 24 16:01 /dev/sdc2

$ lsblk /dev/sdc
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sdc               8:32   0  600M  0 disk
|-sdc1            8:33   0    9M  0 part
`-sdc2            8:34   0  568M  0 part

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/sd.c | 425 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 405 insertions(+), 20 deletions(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index 36c745cbb2619..f044b2dfaa9fd 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -115,12 +115,30 @@ static mempool_t *sd_large_page_pool;
 static atomic_t sd_large_page_pool_users = ATOMIC_INIT(0);
 static struct lock_class_key sd_bio_compl_lkclass;
 #ifdef CONFIG_SCSI_MULTIPATH
+static LIST_HEAD(sd_mpath_disks_list);
+static DEFINE_MUTEX(sd_mpath_disks_lock);
+
 struct sd_mpath_disk {
+	struct device			dev;
+	int				disk_index;
+	int				disk_count;
+	struct list_head		entry;
 	struct scsi_mpath_head		*scsi_mpath_head;
 };
 
 static void sd_mpath_disk_release(struct device *dev)
 {
+	struct sd_mpath_disk *sd_mpath_disk =
+		container_of(dev, struct sd_mpath_disk, dev);
+	struct scsi_mpath_head *scsi_mpath_head =
+		sd_mpath_disk->scsi_mpath_head;
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+
+	mpath_put_disk(mpath_head);
+	ida_free(&sd_index_ida, sd_mpath_disk->disk_index);
+	scsi_mpath_put_head(scsi_mpath_head);
+
+	kfree(sd_mpath_disk);
 }
 
 static const struct class sd_mpath_disk_class = {
@@ -817,7 +835,8 @@ static void scsi_disk_release(struct device *dev)
 {
 	struct scsi_disk *sdkp = to_scsi_disk(dev);
 
-	ida_free(&sd_index_ida, sdkp->index);
+	if (sdkp->index >= 0)
+		ida_free(&sd_index_ida, sdkp->index);
 	put_device(&sdkp->device->sdev_gendev);
 	free_opal_dev(sdkp->opal_dev);
 
@@ -4006,6 +4025,90 @@ static int sd_format_disk_name(char *prefix, int index, char *buf, int buflen)
 	return 0;
 }
 
+#ifdef CONFIG_SCSI_MULTIPATH
+static int sd_mpath_revalidate_head(struct scsi_disk *sdkp)
+{
+	struct sd_mpath_disk *sd_mpath_disk = sdkp->sd_mpath_disk;
+	struct scsi_mpath_head *scsi_mpath_head = sd_mpath_disk->scsi_mpath_head;
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+	struct gendisk *disk = mpath_head->disk;
+	struct queue_limits *sdkp_lim = &sdkp->disk->queue->limits;
+	struct queue_limits lim;
+	unsigned int memflags;
+	int ret;
+
+	lim = queue_limits_start_update(disk->queue);
+	memflags = blk_mq_freeze_queue(disk->queue);
+
+	lim.logical_block_size = sdkp_lim->logical_block_size;
+	lim.physical_block_size = sdkp_lim->physical_block_size;
+	lim.io_min = sdkp_lim->io_min;
+	lim.io_opt = sdkp_lim->io_opt;
+
+	queue_limits_stack_bdev(&lim, sdkp->disk->part0, 0,
+					disk->disk_name);
+
+	/* TODO: setup integrity and zoned limits */
+	lim.max_write_streams = sdkp_lim->max_write_streams;
+	lim.write_stream_granularity = sdkp_lim->write_stream_granularity;
+	ret = queue_limits_commit_update(disk->queue, &lim);
+
+	set_capacity_and_notify(disk, get_capacity(sdkp->disk));
+
+	mpath_revalidate_paths(mpath_head);
+
+	blk_mq_unfreeze_queue(disk->queue, memflags);
+
+	return ret;
+}
+
+static int sd_mpath_get_disk(struct sd_mpath_disk *sd_mpath_disk)
+{
+	if (!get_device(&sd_mpath_disk->dev))
+		return -ENXIO;
+	return 0;
+}
+
+static void sd_mpath_put_disk(struct sd_mpath_disk *sd_mpath_disk)
+{
+	put_device(&sd_mpath_disk->dev);
+}
+
+static struct sd_mpath_disk *sd_mpath_find_disk(
+			struct scsi_mpath_head *scsi_mpath_head)
+{
+	struct sd_mpath_disk *sd_mpath_disk;
+	int ret;
+
+	list_for_each_entry(sd_mpath_disk, &sd_mpath_disks_list, entry) {
+		ret = sd_mpath_get_disk(sd_mpath_disk);
+		if (ret)
+			continue;
+
+		if (sd_mpath_disk->scsi_mpath_head == scsi_mpath_head)
+			return sd_mpath_disk;
+
+		sd_mpath_put_disk(sd_mpath_disk);
+	}
+
+	return NULL;
+}
+
+static int sd_mpath_add_disk(struct scsi_disk *sdkp)
+{
+	struct scsi_device *sdp = sdkp->device;
+	struct scsi_mpath_device *scsi_mpath_dev = sdp->scsi_mpath_dev;
+	struct mpath_device *mpath_device = &scsi_mpath_dev->mpath_device;
+	int ret;
+
+	ret = mpath_add_device(mpath_device, sdkp->disk,
+		dev_to_node(sdp->host->dma_dev), &sdp->host->mpath_nr_active);
+	if (ret)
+		return ret;
+	mpath_device_set_live(mpath_device);
+	return 0;
+}
+
 static ssize_t sd_mpath_device_delayed_removal_store(struct device *dev,
 		struct device_attribute *attr, const char *buf, size_t count)
 {
@@ -4035,13 +4138,259 @@ static const struct attribute_group sd_mpath_disk_attr_group = {
 	.attrs		= sd_mpath_disk_attrs,
 };
 
-__maybe_unused
 static const struct attribute_group *sd_mpath_disk_attr_groups[] = {
 	&sd_mpath_disk_attr_group,
 	&mpath_attr_group,
 	NULL
 };
 
+static int sd_mpath_probe(struct scsi_disk *sdkp)
+{
+	struct scsi_device *sdp = sdkp->device;
+	struct scsi_mpath_device *scsi_mpath_dev = sdp->scsi_mpath_dev;
+	struct device *dma_dev = sdp->host->dma_dev;
+	struct scsi_mpath_head *scsi_mpath_head =
+				scsi_mpath_dev->scsi_mpath_head;
+	struct sd_mpath_disk *sd_mpath_disk;
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+	char disk_name[DISK_NAME_LEN - 2];
+	struct queue_limits lim;
+	struct gendisk *disk;
+	int error;
+
+	/*
+	 * sd_mpath_disks_list is kept locked if no disk found.
+	 * Otherwise an extra reference is taken.
+	 */
+	mutex_lock(&sd_mpath_disks_lock);
+	sd_mpath_disk = sd_mpath_find_disk(scsi_mpath_head);
+	if (sd_mpath_disk) {
+		error = sized_strscpy(disk_name, mpath_head->disk->disk_name,
+				sizeof(disk_name));
+		if (error < 0) {
+			/*
+			 * Should not happen as would fail for the same when
+			 * allocating the sd_mpath_disk
+			 */
+			sd_mpath_put_disk(sd_mpath_disk);
+			mutex_unlock(&sd_mpath_disks_lock);
+			return error;
+		}
+		sd_mpath_disk->disk_count++;
+		mutex_unlock(&sd_mpath_disks_lock);
+
+		goto found;
+	}
+
+	sd_mpath_disk = kzalloc(sizeof(*sd_mpath_disk), GFP_KERNEL);
+	if (!sd_mpath_disk) {
+		error = -ENOMEM;
+		goto out_unlock;
+	}
+
+	sd_mpath_disk->scsi_mpath_head = scsi_mpath_head;
+
+	blk_set_stacking_limits(&lim);
+	lim.dma_alignment = 3;
+	lim.features |= BLK_FEAT_IO_STAT | BLK_FEAT_NOWAIT |
+		BLK_FEAT_POLL | BLK_FEAT_ATOMIC_WRITES;
+
+	mpath_head->parent = &sd_mpath_disk->dev;
+	mpath_head->drv_module = THIS_MODULE;
+	mpath_head->disk_groups = sd_mpath_disk_attr_groups;
+	error = mpath_alloc_head_disk(mpath_head, &lim,
+				dev_to_node(dma_dev));
+	if (error)
+		goto out_free_disk;
+	disk = mpath_head->disk;
+
+	error = ida_alloc(&sd_index_ida, GFP_KERNEL);
+	if (error < 0) {
+		sdev_printk(KERN_WARNING, sdp, "sd_mpath_probe: memory exhausted.\n");
+		goto out_put_disk;
+	}
+	sd_mpath_disk->disk_index = error;
+	error = sd_format_disk_name("sd", sd_mpath_disk->disk_index,
+				disk->disk_name, DISK_NAME_LEN);
+	if (error)
+		goto out_free_index;
+
+	error = sized_strscpy(disk_name, mpath_head->disk->disk_name,
+				sizeof(disk_name));
+	if (error < 0)
+		goto out_free_index;
+
+	device_initialize(&sd_mpath_disk->dev);
+	sd_mpath_disk->dev.class = &sd_mpath_disk_class;
+
+	/* undone in sd_mpath_disk_release() */
+	scsi_mpath_get_head(scsi_mpath_head);
+
+	error = dev_set_name(&sd_mpath_disk->dev, "scsi_mpath_disk%d",
+				scsi_mpath_head->index);
+	if (error) {
+		put_device(&sd_mpath_disk->dev);
+		goto out_unlock;
+	}
+
+	error = device_add(&sd_mpath_disk->dev);
+	if (error) {
+		put_device(&sd_mpath_disk->dev);
+		goto out_unlock;
+	}
+
+	list_add_tail(&sd_mpath_disk->entry, &sd_mpath_disks_list);
+	disk->major = sd_major((sd_mpath_disk->disk_index & 0xf0) >> 4);
+	disk->first_minor = ((sd_mpath_disk->disk_index & 0xf) << 4) |
+				(sd_mpath_disk->disk_index & 0xfff00);
+	disk->minors = SD_MINORS;
+
+	sd_mpath_disk->disk_count = 1;
+	mutex_unlock(&sd_mpath_disks_lock);
+found:
+	sdkp->sd_mpath_disk = sd_mpath_disk;
+	sdkp->disk->flags |= GENHD_FL_HIDDEN;
+	snprintf(sdkp->disk->disk_name, DISK_NAME_LEN, "%s:%d",
+		disk_name, scsi_mpath_dev->index);
+
+	sdkp->index = -1;
+	return 0;
+
+out_free_index:
+	ida_free(&sd_index_ida, sd_mpath_disk->disk_index);
+out_put_disk:
+	mpath_put_disk(mpath_head);
+out_free_disk:
+	kfree(sd_mpath_disk);
+out_unlock:
+	mutex_unlock(&sd_mpath_disks_lock);
+	return error;
+}
+
+static void sd_mpath_remove(struct scsi_disk *sdkp)
+{
+	struct sd_mpath_disk *sd_mpath_disk = sdkp->sd_mpath_disk;
+	struct scsi_device *sdp = sdkp->device;
+	struct scsi_mpath_device *scsi_mpath_dev = sdp->scsi_mpath_dev;
+	struct mpath_device *mpath_device = &scsi_mpath_dev->mpath_device;
+	struct scsi_mpath_head *scsi_mpath_head = sd_mpath_disk->scsi_mpath_head;
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+	bool remove = false;
+
+	mpath_synchronize(mpath_head);
+
+	if (mpath_clear_current_path(mpath_device))
+		mpath_synchronize(mpath_head);
+
+	mpath_delete_device(mpath_device);
+
+	mutex_lock(&sd_mpath_disks_lock);
+	sd_mpath_disk->disk_count--;
+	if (!sd_mpath_disk->disk_count && mpath_can_remove_head(mpath_head)) {
+		list_del_init(&sd_mpath_disk->entry);
+		remove = true;
+	}
+	mutex_unlock(&sd_mpath_disks_lock);
+	mpath_remove_sysfs_link(mpath_device);
+	mpath_device->disk = NULL;
+
+	if (remove) {
+		device_del(&sd_mpath_disk->dev);
+		mpath_remove_disk(mpath_head);
+	}
+	sd_mpath_put_disk(sd_mpath_disk);
+}
+
+static void sd_mpath_remove_head(struct scsi_mpath_head *scsi_mpath_head)
+{
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+	struct sd_mpath_disk *sd_mpath_disk;
+	struct device *dev = &scsi_mpath_head->dev;
+
+	mutex_lock(&sd_mpath_disks_lock);
+	sd_mpath_disk = sd_mpath_find_disk(scsi_mpath_head);
+	if (!sd_mpath_disk) {
+		dev_warn(dev, "could not find mpath disk\n");
+		mutex_unlock(&sd_mpath_disks_lock);
+		return;
+	}
+
+	if (sd_mpath_disk->disk_count) {
+		dev_dbg(dev, "non-zero multipath disk count in removal\n");
+		sd_mpath_put_disk(sd_mpath_disk);
+		mutex_unlock(&sd_mpath_disks_lock);
+		return;
+	}
+
+	list_del_init(&sd_mpath_disk->entry);
+	mutex_unlock(&sd_mpath_disks_lock);
+
+	device_del(&sd_mpath_disk->dev);
+	mpath_remove_disk(mpath_head);
+	sd_mpath_put_disk(sd_mpath_disk);
+}
+
+/*
+ * Always calls for a failed probe, so we need to handle that some structures
+ * have not been setup.
+ */
+static void sd_mpath_fail_probe(struct scsi_disk *sdkp)
+{
+	struct sd_mpath_disk *sd_mpath_disk = sdkp->sd_mpath_disk;
+	struct scsi_mpath_device *scsi_mpath_dev;
+	struct mpath_device *mpath_device;
+	struct scsi_device *sdp = sdkp->device;
+	struct scsi_mpath_head *scsi_mpath_head;
+	struct mpath_head *mpath_head;
+	bool remove = false;
+
+	if (!sd_mpath_disk)
+		return;
+
+	scsi_mpath_dev = sdp->scsi_mpath_dev;
+	mpath_device = &scsi_mpath_dev->mpath_device;
+	scsi_mpath_head = sd_mpath_disk->scsi_mpath_head;
+	mpath_head = &scsi_mpath_head->mpath_head;
+
+	mutex_lock(&sd_mpath_disks_lock);
+	sd_mpath_disk->disk_count--;
+	if (!sd_mpath_disk->disk_count) {
+		list_del_init(&sd_mpath_disk->entry);
+		remove = true;
+	}
+	mutex_unlock(&sd_mpath_disks_lock);
+	mpath_device->disk = NULL;
+
+	if (remove) {
+		device_del(&sd_mpath_disk->dev);
+		mpath_remove_disk(mpath_head);
+	}
+	sd_mpath_put_disk(sd_mpath_disk);
+}
+
+#else /* CONFIG_SCSI_MULTIPATH */
+static int sd_mpath_probe(struct scsi_disk *sdkp)
+{
+	return 0;
+}
+static void sd_mpath_remove(struct scsi_disk *sdkp)
+{
+	return;
+}
+static void sd_mpath_fail_probe(struct scsi_disk *sdkp)
+{
+
+}
+static int sd_mpath_revalidate_head(struct scsi_disk *sdkp)
+{
+	return 0;
+}
+static int sd_mpath_add_disk(struct scsi_disk *sdkp)
+{
+	return 0;
+}
+#endif
+
 /**
  *	sd_probe - called during driver initialization and whenever a
  *	new scsi device is attached to the system. It is called once
@@ -4065,7 +4414,7 @@ static int sd_probe(struct scsi_device *sdp)
 	struct device *dev = &sdp->sdev_gendev;
 	struct scsi_disk *sdkp;
 	struct gendisk *gd;
-	int index;
+	int index = -1;
 	int error;
 
 	scsi_autopm_get_device(sdp);
@@ -4094,22 +4443,33 @@ static int sd_probe(struct scsi_device *sdp)
 					 &sd_bio_compl_lkclass);
 	if (!gd)
 		goto out_free;
+	sdkp->disk = gd;
+	sdkp->device = sdp;
 
-	index = ida_alloc(&sd_index_ida, GFP_KERNEL);
-	if (index < 0) {
-		sdev_printk(KERN_WARNING, sdp, "sd_probe: memory exhausted.\n");
-		goto out_put;
-	}
+	if (sdp->scsi_mpath_dev) {
+		error = sd_mpath_probe(sdkp);
+		if (error)
+			goto out_put;
+	} else {
+		index = ida_alloc(&sd_index_ida, GFP_KERNEL);
+		if (index < 0) {
+			sdev_printk(KERN_WARNING, sdp, "sd_probe: memory exhausted.\n");
+			goto out_put;
+		}
 
-	error = sd_format_disk_name("sd", index, gd->disk_name, DISK_NAME_LEN);
-	if (error) {
-		sdev_printk(KERN_WARNING, sdp, "SCSI disk (sd) name length exceeded.\n");
-		goto out_free_index;
+		error = sd_format_disk_name("sd", index, gd->disk_name,
+					DISK_NAME_LEN);
+		if (error) {
+			sdev_printk(KERN_WARNING, sdp, "SCSI disk (sd) name length exceeded.\n");
+			goto out_free_index;
+		}
+		sdkp->index = index;
+
+		gd->major = sd_major((index & 0xf0) >> 4);
+		gd->first_minor = ((index & 0xf) << 4) | (index & 0xfff00);
+		gd->minors = SD_MINORS;
 	}
 
-	sdkp->device = sdp;
-	sdkp->disk = gd;
-	sdkp->index = index;
 	sdkp->max_retries = SD_MAX_RETRIES;
 	atomic_set(&sdkp->openers, 0);
 	atomic_set(&sdkp->device->ioerr_cnt, 0);
@@ -4129,6 +4489,7 @@ static int sd_probe(struct scsi_device *sdp)
 
 	error = device_add(&sdkp->disk_dev);
 	if (error) {
+		sd_mpath_fail_probe(sdkp);
 		put_device(&sdkp->disk_dev);
 		put_disk(gd);
 		goto out;
@@ -4136,10 +4497,6 @@ static int sd_probe(struct scsi_device *sdp)
 
 	dev_set_drvdata(dev, sdkp);
 
-	gd->major = sd_major((index & 0xf0) >> 4);
-	gd->first_minor = ((index & 0xf) << 4) | (index & 0xfff00);
-	gd->minors = SD_MINORS;
-
 	gd->fops = &sd_fops;
 	gd->private_data = sdkp;
 
@@ -4158,11 +4515,18 @@ static int sd_probe(struct scsi_device *sdp)
 	sd_revalidate_disk(gd);
 	if (sdp->sector_size > PAGE_SIZE) {
 		if (sd_large_pool_create()) {
+			sd_mpath_fail_probe(sdkp);
 			error = -ENOMEM;
 			goto out_free_index;
 		}
 	}
 
+	if (sdp->scsi_mpath_dev) {
+		error = sd_mpath_revalidate_head(sdkp);
+		if (error)
+			sdev_printk(KERN_WARNING, sdp, "could not revalidate multipath limits\n");
+	}
+
 	if (sdp->removable) {
 		gd->flags |= GENHD_FL_REMOVABLE;
 		gd->events |= DISK_EVENT_MEDIA_CHANGE;
@@ -4177,6 +4541,7 @@ static int sd_probe(struct scsi_device *sdp)
 
 	error = device_add_disk(dev, gd, NULL);
 	if (error) {
+		sd_mpath_fail_probe(sdkp);
 		device_unregister(&sdkp->disk_dev);
 		put_disk(gd);
 		if (sdp->sector_size > PAGE_SIZE)
@@ -4184,6 +4549,19 @@ static int sd_probe(struct scsi_device *sdp)
 		goto out;
 	}
 
+	if (sdp->scsi_mpath_dev) {
+		error = sd_mpath_add_disk(sdkp);
+		if (error) {
+			sd_mpath_fail_probe(sdkp);
+			device_del(&sdkp->disk_dev);
+			del_gendisk(sdkp->disk);
+
+			put_disk(sdkp->disk);
+			if (sdp->sector_size > PAGE_SIZE)
+				sd_large_pool_destroy();
+			goto out;
+		}
+	}
 	if (sdkp->security) {
 		sdkp->opal_dev = init_opal_dev(sdkp, &sd_sec_submit);
 		if (sdkp->opal_dev)
@@ -4197,7 +4575,8 @@ static int sd_probe(struct scsi_device *sdp)
 	return 0;
 
  out_free_index:
-	ida_free(&sd_index_ida, index);
+	if (index >= 0)
+		ida_free(&sd_index_ida, index);
  out_put:
 	put_disk(gd);
  out_free:
@@ -4325,6 +4704,9 @@ static void sd_remove(struct scsi_device *sdp)
 	struct device *dev = &sdp->sdev_gendev;
 	struct scsi_disk *sdkp = dev_get_drvdata(dev);
 
+	if (sdp->scsi_mpath_dev)
+		sd_mpath_remove(sdkp);
+
 	scsi_autopm_get_device(sdkp->device);
 
 	device_del(&sdkp->disk_dev);
@@ -4493,6 +4875,9 @@ static struct scsi_driver sd_template = {
 	.resume			= sd_resume,
 	.init_command		= sd_init_command,
 	.uninit_command		= sd_uninit_command,
+#ifdef CONFIG_SCSI_MULTIPATH
+	.mpath_remove_head	= sd_mpath_remove_head,
+#endif
 	.done			= sd_done,
 	.eh_action		= sd_eh_action,
 	.eh_reset		= sd_eh_reset,
-- 
2.43.7

[PATCH v6 25/27] scsi: sd: add mpath_dev file

[John Garry]

Add a mpath_dev file so that the multipath disk can be looked up from
per-path scsi_disk directory.

The following is an example of this usage:

$ ls -l /dev/sdc
brw-rw----    1 root     disk        8,  32 Feb 24 16:08 /dev/sdc
$ cat /sys/class/scsi_mpath_disk/scsi_mpath_disk0/sdc/multipath/sdc:0/mpath_dev
8:32

This can be used by a util like lsscsi, which would find that the gendisk
for the per-path scsi_disk is missing.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/sd.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 74 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index f044b2dfaa9fd..cb1bf4174248b 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -4109,6 +4109,77 @@ static int sd_mpath_add_disk(struct scsi_disk *sdkp)
 	return 0;
 }
 
+static ssize_t sd_mpath_dev_show(struct device *dev,
+			struct device_attribute *attr, char *page)
+{
+	struct gendisk *gd = dev_to_disk(dev);
+	struct scsi_disk *sdkp = gd->private_data;
+	struct scsi_device *sdev = sdkp->device;
+	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
+	struct scsi_mpath_head *scsi_mpath_head =
+				scsi_mpath_dev->scsi_mpath_head;
+	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
+	struct sd_mpath_disk *sd_mpath_disk;
+	struct gendisk *disk;
+	char devt_str[13];
+	ssize_t ret = -EINVAL;
+
+	/*
+	 * This attr is for the per-path scsi_disk block device. We need to
+	 * handle the scenario that the multipath disk has been deleted and
+	 * the per-path scsi_disk (and its attr files) is still present.
+	 */
+	mutex_lock(&sd_mpath_disks_lock);
+	sd_mpath_disk = sd_mpath_find_disk(scsi_mpath_head);
+	if (!sd_mpath_disk)
+		goto out_unlock;
+	if (!sd_mpath_disk->disk_count)
+		goto out_put_disk;
+
+	disk = mpath_head->disk;
+	/* A reference to this disk is held through sd_mpath_disk */
+	if (!disk || !test_bit(GD_ADDED, &disk->state))
+		goto out_put_disk;
+	format_dev_t(devt_str, disk_to_dev(disk)->devt);
+	ret = sysfs_emit(page, "%s\n", devt_str);
+out_put_disk:
+	sd_mpath_put_disk(sd_mpath_disk);
+out_unlock:
+	mutex_unlock(&sd_mpath_disks_lock);
+	return ret;
+}
+static DEVICE_ATTR(mpath_dev, 0444, sd_mpath_dev_show, NULL);
+
+static struct attribute *sd_mpath_dev_attrs[] = {
+	&dev_attr_mpath_dev.attr,
+	NULL
+};
+
+static umode_t sd_mpath_dev_attr_is_visible(struct kobject *kobj,
+				struct attribute *attr, int i)
+{
+	struct device *dev = kobj_to_dev(kobj);
+	struct gendisk *gd = dev_to_disk(dev);
+	struct scsi_disk *sdkp = gd->private_data;
+	struct scsi_device *sdev = sdkp->device;
+	struct scsi_mpath_device *scsi_mpath_device = sdev->scsi_mpath_dev;
+
+	if (!scsi_mpath_device)
+		return 0;
+
+	return attr->mode;
+}
+
+static const struct attribute_group sd_mpath_dev_attr_group = {
+	.is_visible = sd_mpath_dev_attr_is_visible,
+	.attrs = sd_mpath_dev_attrs,
+};
+
+static const struct attribute_group *sd_mpath_dev_groups[] = {
+	&sd_mpath_dev_attr_group,
+	NULL
+};
+
 static ssize_t sd_mpath_device_delayed_removal_store(struct device *dev,
 		struct device_attribute *attr, const char *buf, size_t count)
 {
@@ -4389,6 +4460,8 @@ static int sd_mpath_add_disk(struct scsi_disk *sdkp)
 {
 	return 0;
 }
+
+#define sd_mpath_dev_groups NULL
 #endif
 
 /**
@@ -4539,7 +4612,7 @@ static int sd_probe(struct scsi_device *sdp)
 			sdp->host->rpm_autosuspend_delay);
 	}
 
-	error = device_add_disk(dev, gd, NULL);
+	error = device_add_disk(dev, gd, sd_mpath_dev_groups);
 	if (error) {
 		sd_mpath_fail_probe(sdkp);
 		device_unregister(&sdkp->disk_dev);
-- 
2.43.7

[PATCH v6 26/27] scsi: sd: add mpath_numa_nodes dev attribute

[John Garry]

From: John Garry <john.garry@linux.dev>

Add an attribute to show multipath NUMA node per-path (scsi_disk).

The following is an example of reading the file:

$ cat /sys/devices/platform/host8/session1/target8:0:0/8:0:0:0/block/sdc:0/numa_
mpath_numa_nodes
0-3
$ cat /sys/devices/platform/host9/session2/target9:0:0/9:0:0:0/block/sdc:1/numa_
mpath_numa_nodes
$

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/sd.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index cb1bf4174248b..e63a42bc3d134 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -4150,8 +4150,22 @@ static ssize_t sd_mpath_dev_show(struct device *dev,
 }
 static DEVICE_ATTR(mpath_dev, 0444, sd_mpath_dev_show, NULL);
 
+static ssize_t sd_mpath_numa_nodes_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct gendisk *gd = dev_to_disk(dev);
+	struct scsi_disk *sdkp = gd->private_data;
+	struct scsi_device *sdev = sdkp->device;
+	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
+	struct mpath_device *mpath_device = &scsi_mpath_dev->mpath_device;
+
+	return mpath_numa_nodes_show(mpath_device, buf);
+}
+static DEVICE_ATTR(mpath_numa_nodes, 0444, sd_mpath_numa_nodes_show, NULL);
+
 static struct attribute *sd_mpath_dev_attrs[] = {
 	&dev_attr_mpath_dev.attr,
+	&dev_attr_mpath_numa_nodes.attr,
 	NULL
 };
 
-- 
2.43.7

[PATCH v6 27/27] scsi: sd: add mpath_queue_depth dev attribute

[John Garry]

From: John Garry <john.garry@linux.dev>

Add a queue_depth file so that the multipath dynamic queue depth can be
looked up from per-path gendisk (scsi_disk) directory.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/scsi/sd.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index e63a42bc3d134..6713af46f61a6 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -4163,9 +4163,28 @@ static ssize_t sd_mpath_numa_nodes_show(struct device *dev,
 }
 static DEVICE_ATTR(mpath_numa_nodes, 0444, sd_mpath_numa_nodes_show, NULL);
 
+static ssize_t sd_mpath_queue_depth_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct gendisk *gd = dev_to_disk(dev);
+	struct scsi_disk *sdkp = gd->private_data;
+	struct scsi_device *sdev = sdkp->device;
+	struct scsi_mpath_device *scsi_mpath_dev = sdev->scsi_mpath_dev;
+	struct scsi_mpath_head *scsi_mpath_head =
+				scsi_mpath_dev->scsi_mpath_head;
+	struct Scsi_Host *shost = sdev->host;
+
+	if (!mpath_qd_iopolicy(&scsi_mpath_head->iopolicy))
+		return 0;
+
+	return sysfs_emit(buf, "%d\n", atomic_read(&shost->mpath_nr_active));
+}
+static DEVICE_ATTR(mpath_queue_depth, 0444, sd_mpath_queue_depth_show, NULL);
+
 static struct attribute *sd_mpath_dev_attrs[] = {
 	&dev_attr_mpath_dev.attr,
 	&dev_attr_mpath_numa_nodes.attr,
+	&dev_attr_mpath_queue_depth.attr,
 	NULL
 };
 
-- 
2.43.7

Re: [PATCH v6 02/27] libmultipath: Add basic gendisk support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] The block_device_operations struct `mpath_ops` lacks the mandatory `submit_bio` callback, causing `device_add_disk()` to unconditionally fail and preventing the multipath disk from functioning.
- [High] Re-using `gendisk` after `device_add_disk()` failure violates the driver model and causes kobject corruption.
- [High] `mpath_head_uninit()` synchronously blocks on open file descriptors, causing device removal to hang indefinitely if userspace holds the device open.
--

--- Patch [2]: [PATCH v6 02/27] libmultipath: Add basic gendisk support ---
commit 0995e161443bbd8d5024681ae6aeaebc37bafd47
Author: John Garry <john.g.garry@oracle.com>

libmultipath: Add basic gendisk support

This commit adds support to allocate and free a multipath gendisk.
It introduces struct mpath_head_template as a method for drivers to
provide custom multipath functionality.

> diff --git a/lib/multipath.c b/lib/multipath.c
> --- a/lib/multipath.c
> +++ b/lib/multipath.c
[ ... ]
> @@ -36,6 +36,107 @@ void mpath_head_uninit(struct mpath_head *mpath_head)
[ ... ]
> +const struct block_device_operations mpath_ops = {
> +	.owner          = THIS_MODULE,

[Severity: High]
Does setting the owner to THIS_MODULE (libmultipath) rather than the caller
module (such as nvme_core) introduce a regression where the caller module can
be unloaded while the disk is held open by userspace? If the caller's module
exit calls mpath_head_uninit(), could it hang indefinitely in an
uninterruptible wait on the refcount, permanently deadlocking module_mutex?

> +	.open		= mpath_bdev_open,
> +	.release	= mpath_bdev_release,
> +};

[Severity: High]
Is this regression missing a submit_bio callback in mpath_ops? For bio-based
queues, the block layer's __add_disk() mandates that disk->fops->submit_bio
is present. If it is absent, device_add_disk() unconditionally aborts and
returns -EINVAL, which prevents the multipath block device from registering.

[ ... ]
> +void mpath_device_set_live(struct mpath_device *mpath_device)
> +{
[ ... ]
> +	if (!test_and_set_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags)) {
> +		dev_set_drvdata(disk_to_dev(mpath_head->disk), mpath_head);
> +		ret = device_add_disk(mpath_head->parent, mpath_head->disk,
> +				mpath_head->disk_groups);
> +		if (ret) {
> +			clear_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags);
> +			return;
> +		}

[Severity: High]
If device_add_disk() fails, clearing the live flag without freeing the
disk leaves the object intact. When this function is called again for another
path, won't it retry device_add_disk() on the exact same gendisk object?
Because the driver model forbids calling device_add() more than once on the
same device object and requires dropping the device upon failure, could this
regression result in kobject BUGs or memory corruption?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=2

Re: [PATCH v6 03/27] libmultipath: Add path selection support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [High] Round-robin path selection behaves as reverse round-robin for non-optimized paths due to a missing loop break.
- [High] Unsafe and logically flawed use of `list_is_singular` on an RCU-protected list routes I/O to a removed device.

Pre-existing issues:
- [High] mpath_head_uninit() blocks indefinitely waiting for user-space to close the block device.
--

--- Patch [3]: [PATCH v6 03/27] libmultipath: Add path selection support ---
commit 70a41be1b0a598af8cbd34732b889f79118d17ac
Author: John Garry <john.garry@linux.dev>

libmultipath: Add path selection support

This commit adds code for path selection support in libmultipath. It abstracts
NVMe ANA into mpath_access_state to allow SCSI ALUA usage and adds round-robin,
NUMA, and queue-depth selection modes. It also introduces functions to set and
get the default I/O policy.

> diff --git a/lib/multipath.c b/lib/multipath.c
> index 1cf20b0f62d07..26714116023a3 100644
> --- a/lib/multipath.c
> +++ b/lib/multipath.c

[ ... ]

> @@ -7,8 +7,245 @@
> +static struct mpath_device *mpath_round_robin_path(
> +				struct mpath_head *mpath_head)
> +{
> +	struct mpath_device *mpath_device, *found = NULL;
> +	int node = numa_node_id();
> +	enum mpath_access_state access_state_old;
> +	struct mpath_device *old =
> +			srcu_dereference(mpath_head->current_path[node],
> +				&mpath_head->srcu);
> +
> +	if (unlikely(!old))
> +		return __mpath_find_path(mpath_head, node);
> +
> +	if (list_is_singular(&mpath_head->dev_list)) {

[Severity: High]
Does using list_is_singular() here on an SRCU-protected list introduce a data
race on head->prev?

Additionally, if old is concurrently removed and exactly one other active
device remains in the list, list_is_singular() evaluates to true. Will this
cause the function to return the unlinked old path instead of the valid
active path?

> +		if (mpath_path_is_disabled(mpath_head, old))
> +			return NULL;
> +		return old;
> +	}
> +
> +	for (mpath_device = mpath_next_dev(mpath_head, old);
> +	    mpath_device && mpath_device != old;
> +	    mpath_device = mpath_next_dev(mpath_head, mpath_device)) {
> +
> +		if (mpath_path_is_disabled(mpath_head, mpath_device))
> +			continue;
> +		if (mpath_device->access_state == MPATH_STATE_OPTIMIZED) {
> +			found = mpath_device;
> +			goto out;
> +		}
> +		if (mpath_device->access_state == MPATH_STATE_NONOPTIMIZED)
> +			found = mpath_device;

[Severity: High]
For the MPATH_STATE_NONOPTIMIZED case, found is overwritten unconditionally.
Because there is no loop break or check like if (!found), will this
evaluate multiple non-optimized paths and incorrectly select the last
available one rather than the next available one?

> +	}

[ ... ]

[Severity: High]
This is a pre-existing issue, but I noticed a potential problem during
teardown in mpath_head_uninit():

lib/multipath.c:mpath_head_uninit() {
        ...
        if (!refcount_dec_and_test(refcount))
                wait_var_event(refcount, !refcount_read(refcount));
}

Since a reference is taken for every user-space open, can wait_var_event()
block indefinitely if a user process holds the device open while the
driver is unbound or the device is hot-unplugged?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=3

Re: [PATCH v6 02/27] libmultipath: Add basic gendisk support

[John Garry]

On 29/07/2026 12:20, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> - [High] The block_device_operations struct `mpath_ops` lacks the mandatory `submit_bio` callback, causing `device_add_disk()` to unconditionally fail and preventing the multipath disk from functioning.
> - [High] Re-using `gendisk` after `device_add_disk()` failure violates the driver model and causes kobject corruption.
> - [High] `mpath_head_uninit()` synchronously blocks on open file descriptors, causing device removal to hang indefinitely if userspace holds the device open.
> --
> 
> --- Patch [2]: [PATCH v6 02/27] libmultipath: Add basic gendisk support ---
> commit 0995e161443bbd8d5024681ae6aeaebc37bafd47
> Author: John Garry <john.g.garry@oracle.com>
> 
> libmultipath: Add basic gendisk support
> 
> This commit adds support to allocate and free a multipath gendisk.
> It introduces struct mpath_head_template as a method for drivers to
> provide custom multipath functionality.
> 
>> diff --git a/lib/multipath.c b/lib/multipath.c
>> --- a/lib/multipath.c
>> +++ b/lib/multipath.c
> [ ... ]
>> @@ -36,6 +36,107 @@ void mpath_head_uninit(struct mpath_head *mpath_head)
> [ ... ]
>> +const struct block_device_operations mpath_ops = {
>> +	.owner          = THIS_MODULE,
> 
> [Severity: High]
> Does setting the owner to THIS_MODULE (libmultipath) rather than the caller
> module (such as nvme_core) introduce a regression where the caller module can
> be unloaded while the disk is held open by userspace? If the caller's module
> exit calls mpath_head_uninit(), could it hang indefinitely in an
> uninterruptible wait on the refcount, permanently deadlocking module_mutex?

the caller module should call mpath_head_uninit(), which ensures all 
references are dropped

> 
>> +	.open		= mpath_bdev_open,
>> +	.release	= mpath_bdev_release,
>> +};
> 
> [Severity: High]
> Is this regression missing a submit_bio callback in mpath_ops? For bio-based
> queues, the block layer's __add_disk() mandates that disk->fops->submit_bio
> is present. If it is absent, device_add_disk() unconditionally aborts and
> returns -EINVAL, which prevents the multipath block device from registering.

it's added later

> 
> [ ... ]
>> +void mpath_device_set_live(struct mpath_device *mpath_device)
>> +{
> [ ... ]
>> +	if (!test_and_set_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags)) {
>> +		dev_set_drvdata(disk_to_dev(mpath_head->disk), mpath_head);
>> +		ret = device_add_disk(mpath_head->parent, mpath_head->disk,
>> +				mpath_head->disk_groups);
>> +		if (ret) {
>> +			clear_bit(MPATH_HEAD_DISK_LIVE, &mpath_head->flags);
>> +			return;
>> +		}
> 
> [Severity: High]
> If device_add_disk() fails, clearing the live flag without freeing the
> disk leaves the object intact. When this function is called again for another
> path, won't it retry device_add_disk() on the exact same gendisk object?
> Because the driver model forbids calling device_add() more than once on the
> same device object and requires dropping the device upon failure, could this
> regression result in kobject BUGs or memory corruption?

This is same as NVMe, so I won't change it now

> 

Re: [PATCH v6 03/27] libmultipath: Add path selection support

[John Garry]

On 29/07/2026 12:33, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> 
> New issues:
> - [High] Round-robin path selection behaves as reverse round-robin for non-optimized paths due to a missing loop break.
> - [High] Unsafe and logically flawed use of `list_is_singular` on an RCU-protected list routes I/O to a removed device.
> 
> Pre-existing issues:
> - [High] mpath_head_uninit() blocks indefinitely waiting for user-space to close the block device.
> --
> 
> --- Patch [3]: [PATCH v6 03/27] libmultipath: Add path selection support ---
> commit 70a41be1b0a598af8cbd34732b889f79118d17ac
> Author: John Garry <john.garry@linux.dev>
> 
> libmultipath: Add path selection support
> 
> This commit adds code for path selection support in libmultipath. It abstracts
> NVMe ANA into mpath_access_state to allow SCSI ALUA usage and adds round-robin,
> NUMA, and queue-depth selection modes. It also introduces functions to set and
> get the default I/O policy.
> 
>> diff --git a/lib/multipath.c b/lib/multipath.c
>> index 1cf20b0f62d07..26714116023a3 100644
>> --- a/lib/multipath.c
>> +++ b/lib/multipath.c
> 
> [ ... ]
> 
>> @@ -7,8 +7,245 @@
>> +static struct mpath_device *mpath_round_robin_path(
>> +				struct mpath_head *mpath_head)
>> +{
>> +	struct mpath_device *mpath_device, *found = NULL;
>> +	int node = numa_node_id();
>> +	enum mpath_access_state access_state_old;
>> +	struct mpath_device *old =
>> +			srcu_dereference(mpath_head->current_path[node],
>> +				&mpath_head->srcu);
>> +
>> +	if (unlikely(!old))
>> +		return __mpath_find_path(mpath_head, node);
>> +
>> +	if (list_is_singular(&mpath_head->dev_list)) {
> 
> [Severity: High]
> Does using list_is_singular() here on an SRCU-protected list introduce a data
> race on head->prev?

locking is added later

> 
> Additionally, if old is concurrently removed and exactly one other active
> device remains in the list, list_is_singular() evaluates to true. Will this
> cause the function to return the unlinked old path instead of the valid
> active path?

as above

> 
>> +		if (mpath_path_is_disabled(mpath_head, old))
>> +			return NULL;
>> +		return old;
>> +	}
>> +
>> +	for (mpath_device = mpath_next_dev(mpath_head, old);
>> +	    mpath_device && mpath_device != old;
>> +	    mpath_device = mpath_next_dev(mpath_head, mpath_device)) {
>> +
>> +		if (mpath_path_is_disabled(mpath_head, mpath_device))
>> +			continue;
>> +		if (mpath_device->access_state == MPATH_STATE_OPTIMIZED) {
>> +			found = mpath_device;
>> +			goto out;
>> +		}
>> +		if (mpath_device->access_state == MPATH_STATE_NONOPTIMIZED)
>> +			found = mpath_device;
> 
> [Severity: High]
> For the MPATH_STATE_NONOPTIMIZED case, found is overwritten unconditionally.
> Because there is no loop break or check like if (!found), will this
> evaluate multiple non-optimized paths and incorrectly select the last
> available one rather than the next available one?

I think so, but no one cares about non-optimized. Again, this is same as 
NVMe code.

> 
>> +	}
> 
> [ ... ]
> 
> [Severity: High]
> This is a pre-existing issue, but I noticed a potential problem during
> teardown in mpath_head_uninit():
> 
> lib/multipath.c:mpath_head_uninit() {
>          ...
>          if (!refcount_dec_and_test(refcount))
>                  wait_var_event(refcount, !refcount_read(refcount));
> }
> 
> Since a reference is taken for every user-space open, can wait_var_event()
> block indefinitely if a user process holds the device open while the
> driver is unbound or the device is hot-unplugged?

yes, but that is just the way it is - userspace needs to drop the ref

> 

Re: [PATCH v6 05/27] libmultipath: Add support for mpath_device management

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] RCU Use-After-Free Vulnerability in mpath_delete_device()
- [High] NULL Pointer Dereference in sysfs link management
- [High] Race condition causing sysfs link leak and dangling kobject references
- [Medium] TOCTOU race in mpath_remove_sysfs_link
--

--- Patch [5]: [PATCH v6 05/27] libmultipath: Add support for mpath_device management ---
commit a32651ca51a97ca7e06ffd62321d0190b9e9cfa0
Author: John Garry <john.g.garry@oracle.com>

libmultipath: Add support for mpath_device management
    
Add support to add or remove a mpath_device as a path.
    
NVMe has almost like-for-like equivalents here:
- nvme_mpath_clear_current_path() -> mpath_clear_current_path()
- nvme_mpath_add_sysfs_link() -> mpath_add_sysfs_link()
- nvme_mpath_remove_sysfs_link() -> mpath_remove_sysfs_link()
- nvme_mpath_revalidate_paths() -> mpath_revalidate_paths()

The functionality in mpath_clear_paths() and mpath_synchronize() have the
same pattern which is frequently used in the NVMe code.

> diff --git a/lib/multipath.c b/lib/multipath.c
> index 93b82eea7d38b..6b8f6e846987a 100644
> --- a/lib/multipath.c
> +++ b/lib/multipath.c

[ ... ]

> @@ -51,6 +51,87 @@ void mpath_synchronize(struct mpath_head *mpath_head)
>  }
>  EXPORT_SYMBOL_GPL(mpath_synchronize);
>  

[ ... ]

> +bool mpath_delete_device(struct mpath_device *mpath_device)
> +{
> +	bool empty;
> +
> +	mutex_lock(&mpath_device->mpath_head->lock);
> +	list_del_rcu(&mpath_device->siblings);
> +	empty = list_empty(&mpath_device->mpath_head->dev_list);
> +	mutex_unlock(&mpath_device->mpath_head->lock);
> +
> +	mpath_synchronize(mpath_device->mpath_head);

[Severity: High]
Does this sequence safely handle RCU readers accessing current_path?

Because mpath_delete_device() removes the device from dev_list and
immediately calls mpath_synchronize() without clearing it from
mpath_head->current_path, the RCU-protected current_path array may still
hold a pointer to the device. Lockless readers can then dereference the
freed device.

If callers attempt to clear current_path before calling
mpath_delete_device(), couldn't the pointer be repopulated from dev_list
by concurrent readers during the race window? If cleared after, the
synchronization has already passed, providing no protection against
use-after-free.

> +
> +	return empty;
> +}
> +EXPORT_SYMBOL_GPL(mpath_delete_device);

[ ... ]

> @@ -489,6 +572,77 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
>  }
>  EXPORT_SYMBOL_GPL(mpath_device_set_live);
>  
> +void mpath_add_sysfs_link(struct mpath_head *mpath_head)
> +{
> +	struct device *target;
> +	struct device *source;
> +	int rc, srcu_idx;
> +	struct kobject *mpath_gd_kobj;
> +	struct mpath_device *mpath_device;
> +
> +	/*
> +	 * Ensure head disk node is already added otherwise we may get invalid
> +	 * kobj for head disk node
> +	 */
> +	if (!test_bit(GD_ADDED, &mpath_head->disk->state))
> +		return;

[Severity: High]
Is it possible for mpath_head->disk to be NULL here?

When a multi-path head disk is removed, mpath_put_disk() is called, which
sets mpath_head->disk = NULL. If a path subsequently goes live,
mpath_add_sysfs_link() appears to dereference it without checking for NULL.

Could this cause a deterministic oops when path teardown outlives the head,
such as during an abrupt controller removal?

> +
> +	mpath_gd_kobj = &disk_to_dev(mpath_head->disk)->kobj;
> +	srcu_idx = srcu_read_lock(&mpath_head->srcu);
> +
> +	list_for_each_entry_srcu(mpath_device, &mpath_head->dev_list, siblings,
> +				 srcu_read_lock_held(&mpath_head->srcu)) {
> +		if (!test_bit(GD_ADDED, &mpath_device->disk->state))
> +			continue;
> +
> +		if (test_and_set_bit(MPATH_DEVICE_SYSFS_ATTR_LINK,
> +					&mpath_device->flags))
> +			continue;
> +
> +		target = disk_to_dev(mpath_device->disk);
> +		source = disk_to_dev(mpath_head->disk);
> +		/*
> +		 * Create sysfs link from head gendisk kobject @kobj to the
> +		 * ns path gendisk kobject @target->kobj.
> +		 */

[Severity: High]
Could this re-create a sysfs link for a device that is in the middle of
being deleted?

During path removal, mpath_remove_sysfs_link() clears the
MPATH_DEVICE_SYSFS_ATTR_LINK bit before mpath_delete_device() removes
the device from the RCU dev_list.

If a different path concurrently goes live, mpath_add_sysfs_link() iterates
over dev_list, observes the cleared bit on the dying device, and might
mistakenly re-create the sysfs link. When the dying device is finally
removed and freed, the re-created sysfs link is permanently leaked and
points to a freed block device kobject.

> +		rc = sysfs_add_link_to_group(mpath_gd_kobj, "multipath",
> +				&target->kobj, dev_name(target));

[ ... ]

> +void mpath_remove_sysfs_link(struct mpath_device *mpath_device)
> +{
> +	struct device *target;
> +	struct kobject *mpath_gd_kobj;
> +	struct mpath_head *mpath_head = mpath_device->mpath_head;
> +
> +	if (!test_bit(MPATH_DEVICE_SYSFS_ATTR_LINK, &mpath_device->flags))
> +		return;
> +
> +	target = disk_to_dev(mpath_device->disk);
> +	mpath_gd_kobj = &disk_to_dev(mpath_head->disk)->kobj;

[Severity: High]
Does mpath_head->disk need to be checked for NULL before dereferencing it
here?

Similar to mpath_add_sysfs_link(), if underlying paths are detached after
mpath_put_disk() sets mpath_head->disk = NULL, this will cause a NULL
pointer dereference during teardown.

> +
> +	sysfs_remove_link_from_group(mpath_gd_kobj, "multipath",
> +			dev_name(target));
> +
> +	clear_bit(MPATH_DEVICE_SYSFS_ATTR_LINK, &mpath_device->flags);

[Severity: Medium]
Is there a race condition between test_bit() at the start of this function
and clear_bit() here?

A concurrent execution for the same device could pass the test_bit() check
and issue duplicate removals to the sysfs core. Should this diverging
non-atomic pattern be replaced with test_and_clear_bit() to match the safe
pattern used in mpath_add_sysfs_link()?

> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=5

Re: [PATCH v6 06/27] libmultipath: Add delayed removal support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] mpath_remove_head_work() unconditionally removes the multipath head without verifying if paths have been concurrently added, leading to destruction of an active device.
- [High] Missing cancellation of remove_work during multipath head teardown leads to a use-after-free when the delayed work expires.
- [High] mpath_head->drv_module is never initialized, leading to silent failures or crashes on module unload.
- [High] Disabling delayed removal via sysfs leaves queued I/Os stuck indefinitely.
- [Medium] Module reference count leak when mpath_can_remove_head() is called multiple times.
- [Medium] Large delayed removal values cause immediate removal instead of a long delay.
--

--- Patch [6]: [PATCH v6 06/27] libmultipath: Add delayed removal support ---
commit 87d6b13fc148098f06b11013c5f12523eb0d477f
Author: John Garry <john.garry@linux.dev>

libmultipath: Add delayed removal support

Add support for delayed removal, same as exists for NVMe, to keep the
multipath disk and cdev present for intermittent periods of no available
path.

> diff --git a/lib/multipath.c b/lib/multipath.c
> index 6b8f6e846987a..fada92bf9754c 100644
> --- a/lib/multipath.c
> +++ b/lib/multipath.c
> @@ -66,6 +66,9 @@ int mpath_add_device(struct mpath_device *mpath_device, struct gendisk *disk,
>  	list_add_tail_rcu(&mpath_device->siblings, &mpath_head->dev_list);
>  	mutex_unlock(&mpath_head->lock);
>  
> +	if (cancel_delayed_work(&mpath_head->remove_work))
> +		module_put(mpath_head->drv_module);
> +
>  	return 0;
>  }

[Severity: High]
Can this race with the removal work?

If the delayed work is already executing, cancel_delayed_work() returns
false, but mpath_add_device() still returns success. Meanwhile, could
mpath_remove_head_work() execute without taking mpath_head->lock or
verifying if the device list is empty, unintentionally destroying the
multipath head while the driver believes a new path was successfully
added?

[ ... ]

> @@ -481,6 +493,39 @@ static void mpath_requeue_work(struct work_struct *work)
>  	}
>  }
>  
> +bool mpath_can_remove_head(struct mpath_head *mpath_head)
> +{
> +	unsigned long delay;
> +	bool remove = false;
> +
> +	mutex_lock(&mpath_head->lock);
> +	/*
> +	 * Ensure that no one could remove this module while the head
> +	 * remove work is pending.
> +	 */
> +	if (mpath_head_queue_if_no_path(mpath_head) &&
> +	    !check_mul_overflow(mpath_head->delayed_removal_secs, HZ, &delay) &&

[Severity: Medium]
What happens if a user configures a very large delayed removal value?

If check_mul_overflow() detects an overflow (e.g., on 32-bit systems where
a large sysfs input overflows unsigned long), it fails the condition and
falls through to the else block, setting remove = true. Does this cause an
immediate teardown instead of clamping the delay to a maximum value?

> +	    try_module_get(mpath_head->drv_module)) {
> +		mod_delayed_work(mpath_wq, &mpath_head->remove_work, delay);

[Severity: Medium]
Could this lead to a module reference leak?

If mod_delayed_work() returns true (meaning the work was already pending
and its timer was just modified), it doesn't queue a new work item.
Since try_module_get() was called unconditionally beforehand, and the
work callback only drops one module reference, do subsequent calls to
mpath_can_remove_head() leak a module reference permanently?

> +	} else {
> +		remove = true;
> +	}
> +
> +	mutex_unlock(&mpath_head->lock);
> +	return remove;
> +}

[ ... ]

> @@ -529,6 +574,9 @@ int mpath_alloc_head_disk(struct mpath_head *mpath_head,
>  	mpath_head->disk->private_data = mpath_head;
>  	mpath_head->disk->fops = &mpath_ops;
>  
> +	INIT_DELAYED_WORK(&mpath_head->remove_work, mpath_remove_head_work);
> +	mpath_head->delayed_removal_secs = 0;
> +

[Severity: High]
Is drv_module properly initialized anywhere?

It seems drv_module is added to struct mpath_head but never assigned in
mpath_alloc_head_disk() or elsewhere in this patch. If it defaults to
NULL, try_module_get(NULL) succeeds but pins no module. Could this allow
the driver to unload while the delayed work is still pending, leading to
a kernel panic when the timer fires?

[Severity: High]
Is the remove_work guaranteed to be cancelled during object teardown?

The standard teardown sequence calls mpath_put_disk() and
mpath_head_uninit(). If cancel_delayed_work_sync(&mpath_head->remove_work)
is omitted during teardown, could the timer remain active on the
workqueue and execute on the freed mpath_head structure later?

>  	set_bit(GD_SUPPRESS_PART_SCAN, &mpath_head->disk->state);
>  
>  	return 0;

[ ... ]

> @@ -572,6 +620,47 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
>  }
>  EXPORT_SYMBOL_GPL(mpath_device_set_live);
>  
> +ssize_t mpath_delayed_removal_secs_show(struct mpath_head *mpath_head,
> +					char *buf)
> +{
> +	int ret;
> +
> +	mutex_lock(&mpath_head->lock);
> +	ret = sysfs_emit(buf, "%u\n", mpath_head->delayed_removal_secs);
> +	mutex_unlock(&mpath_head->lock);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(mpath_delayed_removal_secs_show);
> +
> +ssize_t mpath_delayed_removal_secs_store(struct mpath_head *mpath_head,
> +			const char *buf, size_t count)
> +{
> +	unsigned int sec;
> +	ssize_t ret;
> +
> +	ret = kstrtouint(buf, 0, &sec);
> +	if (ret < 0)
> +		return ret;
> +
> +	mutex_lock(&mpath_head->lock);
> +	mpath_head->delayed_removal_secs = sec;
> +	if (sec)
> +		set_bit(MPATH_HEAD_QUEUE_IF_NO_PATH, &mpath_head->flags);
> +	else
> +		clear_bit(MPATH_HEAD_QUEUE_IF_NO_PATH, &mpath_head->flags);
> +	mutex_unlock(&mpath_head->lock);
> +
> +	/*
> +	 * Ensure that update to MPATH_HEAD_QUEUE_IF_NO_PATH is seen
> +	 * by its reader.
> +	 */
> +	mpath_synchronize(mpath_head);
> +
> +	return count;
> +}

[Severity: High]
If a user disables delayed removal by setting delayed_removal_secs to 0 via
sysfs while paths are missing, how do pending I/Os get handled?

We clear the flag and call mpath_synchronize(), but shouldn't we also
call mpath_schedule_requeue_work() here to flush and fail the I/Os that
are already parked in mpath_head->requeue_list?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=6

Re: [PATCH v6 05/27] libmultipath: Add support for mpath_device management

[John Garry]

On 29/07/2026 12:57, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
> - [High] RCU Use-After-Free Vulnerability in mpath_delete_device()
> - [High] NULL Pointer Dereference in sysfs link management
> - [High] Race condition causing sysfs link leak and dangling kobject references
> - [Medium] TOCTOU race in mpath_remove_sysfs_link
> --
> 
> --- Patch [5]: [PATCH v6 05/27] libmultipath: Add support for mpath_device management ---
> commit a32651ca51a97ca7e06ffd62321d0190b9e9cfa0
> Author: John Garry <john.g.garry@oracle.com>
> 
> libmultipath: Add support for mpath_device management
>      
> Add support to add or remove a mpath_device as a path.
>      
> NVMe has almost like-for-like equivalents here:
> - nvme_mpath_clear_current_path() -> mpath_clear_current_path()
> - nvme_mpath_add_sysfs_link() -> mpath_add_sysfs_link()
> - nvme_mpath_remove_sysfs_link() -> mpath_remove_sysfs_link()
> - nvme_mpath_revalidate_paths() -> mpath_revalidate_paths()
> 
> The functionality in mpath_clear_paths() and mpath_synchronize() have the
> same pattern which is frequently used in the NVMe code.
> 
>> diff --git a/lib/multipath.c b/lib/multipath.c
>> index 93b82eea7d38b..6b8f6e846987a 100644
>> --- a/lib/multipath.c
>> +++ b/lib/multipath.c
> 
> [ ... ]
> 
>> @@ -51,6 +51,87 @@ void mpath_synchronize(struct mpath_head *mpath_head)
>>   }
>>   EXPORT_SYMBOL_GPL(mpath_synchronize);
>>   
> 
> [ ... ]
> 
>> +bool mpath_delete_device(struct mpath_device *mpath_device)
>> +{
>> +	bool empty;
>> +
>> +	mutex_lock(&mpath_device->mpath_head->lock);
>> +	list_del_rcu(&mpath_device->siblings);
>> +	empty = list_empty(&mpath_device->mpath_head->dev_list);
>> +	mutex_unlock(&mpath_device->mpath_head->lock);
>> +
>> +	mpath_synchronize(mpath_device->mpath_head);
> 
> [Severity: High]
> Does this sequence safely handle RCU readers accessing current_path?
> 
> Because mpath_delete_device() removes the device from dev_list and
> immediately calls mpath_synchronize() without clearing it from
> mpath_head->current_path, the RCU-protected current_path array may still
> hold a pointer to the device. Lockless readers can then dereference the
> freed device.

it is removed in sd_mpath_remove() -> mpath_clear_current_path()

> 
> If callers attempt to clear current_path before calling
> mpath_delete_device(), couldn't the pointer be repopulated from dev_list
> by concurrent readers during the race window? If cleared after, the
> synchronization has already passed, providing no protection against
> use-after-free.
> 
>> +
>> +	return empty;
>> +}
>> +EXPORT_SYMBOL_GPL(mpath_delete_device);
> 
> [ ... ]
> 
>> @@ -489,6 +572,77 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
>>   }
>>   EXPORT_SYMBOL_GPL(mpath_device_set_live);
>>   
>> +void mpath_add_sysfs_link(struct mpath_head *mpath_head)
>> +{
>> +	struct device *target;
>> +	struct device *source;
>> +	int rc, srcu_idx;
>> +	struct kobject *mpath_gd_kobj;
>> +	struct mpath_device *mpath_device;
>> +
>> +	/*
>> +	 * Ensure head disk node is already added otherwise we may get invalid
>> +	 * kobj for head disk node
>> +	 */
>> +	if (!test_bit(GD_ADDED, &mpath_head->disk->state))
>> +		return;
> 
> [Severity: High]
> Is it possible for mpath_head->disk to be NULL here?
> 
> When a multi-path head disk is removed, mpath_put_disk() is called, which
> sets mpath_head->disk = NULL. If a path subsequently goes live,
> mpath_add_sysfs_link() appears to dereference it without checking for NULL.
> 
> Could this cause a deterministic oops when path teardown outlives the head,
> such as during an abrupt controller removal?

if a new device is going live, then mpath_head->disk has to be present 
as it has a reference to it

> 
>> +
>> +	mpath_gd_kobj = &disk_to_dev(mpath_head->disk)->kobj;
>> +	srcu_idx = srcu_read_lock(&mpath_head->srcu);
>> +
>> +	list_for_each_entry_srcu(mpath_device, &mpath_head->dev_list, siblings,
>> +				 srcu_read_lock_held(&mpath_head->srcu)) {
>> +		if (!test_bit(GD_ADDED, &mpath_device->disk->state))
>> +			continue;
>> +
>> +		if (test_and_set_bit(MPATH_DEVICE_SYSFS_ATTR_LINK,
>> +					&mpath_device->flags))
>> +			continue;
>> +
>> +		target = disk_to_dev(mpath_device->disk);
>> +		source = disk_to_dev(mpath_head->disk);
>> +		/*
>> +		 * Create sysfs link from head gendisk kobject @kobj to the
>> +		 * ns path gendisk kobject @target->kobj.
>> +		 */
> 
> [Severity: High]
> Could this re-create a sysfs link for a device that is in the middle of
> being deleted?
> 
> During path removal, mpath_remove_sysfs_link() clears the
> MPATH_DEVICE_SYSFS_ATTR_LINK bit before mpath_delete_device() removes
> the device from the RCU dev_list.
> 
> If a different path concurrently goes live, mpath_add_sysfs_link() iterates
> over dev_list, observes the cleared bit on the dying device, and might
> mistakenly re-create the sysfs link. When the dying device is finally
> removed and freed, the re-created sysfs link is permanently leaked and
> points to a freed block device kobject.

we call mpath_delete_device() before mpath_remove_sysfs_link() in 
sd_mpath_remove(), so this should not happen

> 
>> +		rc = sysfs_add_link_to_group(mpath_gd_kobj, "multipath",
>> +				&target->kobj, dev_name(target));
> 
> [ ... ]
> 
>> +void mpath_remove_sysfs_link(struct mpath_device *mpath_device)
>> +{
>> +	struct device *target;
>> +	struct kobject *mpath_gd_kobj;
>> +	struct mpath_head *mpath_head = mpath_device->mpath_head;
>> +
>> +	if (!test_bit(MPATH_DEVICE_SYSFS_ATTR_LINK, &mpath_device->flags))
>> +		return;
>> +
>> +	target = disk_to_dev(mpath_device->disk);
>> +	mpath_gd_kobj = &disk_to_dev(mpath_head->disk)->kobj;
> 
> [Severity: High]
> Does mpath_head->disk need to be checked for NULL before dereferencing it
> here?
> 
> Similar to mpath_add_sysfs_link(), if underlying paths are detached after
> mpath_put_disk() sets mpath_head->disk = NULL, this will cause a NULL
> pointer dereference during teardown.

the paths cannot be detached after mpath_put_disk()

> 
>> +
>> +	sysfs_remove_link_from_group(mpath_gd_kobj, "multipath",
>> +			dev_name(target));
>> +
>> +	clear_bit(MPATH_DEVICE_SYSFS_ATTR_LINK, &mpath_device->flags);
> 
> [Severity: Medium]
> Is there a race condition between test_bit() at the start of this function
> and clear_bit() here?
> 
> A concurrent execution for the same device could pass the test_bit() check
> and issue duplicate removals to the sysfs core. 

We don't call for the same device concurrently

> Should this diverging
> non-atomic pattern be replaced with test_and_clear_bit() to match the safe
> pattern used in mpath_add_sysfs_link()?
> 
>> +}
> 

Re: [PATCH v6 06/27] libmultipath: Add delayed removal support

[John Garry]

On 29/07/2026 13:08, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
> - [High] mpath_remove_head_work() unconditionally removes the multipath head without verifying if paths have been concurrently added, leading to destruction of an active device.
> - [High] Missing cancellation of remove_work during multipath head teardown leads to a use-after-free when the delayed work expires.
> - [High] mpath_head->drv_module is never initialized, leading to silent failures or crashes on module unload.
> - [High] Disabling delayed removal via sysfs leaves queued I/Os stuck indefinitely.
> - [Medium] Module reference count leak when mpath_can_remove_head() is called multiple times.
> - [Medium] Large delayed removal values cause immediate removal instead of a long delay.
> --
> 
> --- Patch [6]: [PATCH v6 06/27] libmultipath: Add delayed removal support ---
> commit 87d6b13fc148098f06b11013c5f12523eb0d477f
> Author: John Garry <john.garry@linux.dev>
> 
> libmultipath: Add delayed removal support
> 
> Add support for delayed removal, same as exists for NVMe, to keep the
> multipath disk and cdev present for intermittent periods of no available
> path.
> 
>> diff --git a/lib/multipath.c b/lib/multipath.c
>> index 6b8f6e846987a..fada92bf9754c 100644
>> --- a/lib/multipath.c
>> +++ b/lib/multipath.c
>> @@ -66,6 +66,9 @@ int mpath_add_device(struct mpath_device *mpath_device, struct gendisk *disk,
>>   	list_add_tail_rcu(&mpath_device->siblings, &mpath_head->dev_list);
>>   	mutex_unlock(&mpath_head->lock);
>>   
>> +	if (cancel_delayed_work(&mpath_head->remove_work))
>> +		module_put(mpath_head->drv_module);
>> +
>>   	return 0;
>>   }
> 
> [Severity: High]
> Can this race with the removal work?
> 
> If the delayed work is already executing, cancel_delayed_work() returns
> false, but mpath_add_device() still returns success. Meanwhile, could
> mpath_remove_head_work() execute without taking mpath_head->lock or
> verifying if the device list is empty, unintentionally destroying the
> multipath head while the driver believes a new path was successfully
> added?
> 

the removal callback should take care of this


> [ ... ]
> 
>> @@ -481,6 +493,39 @@ static void mpath_requeue_work(struct work_struct *work)
>>   	}
>>   }
>>   
>> +bool mpath_can_remove_head(struct mpath_head *mpath_head)
>> +{
>> +	unsigned long delay;
>> +	bool remove = false;
>> +
>> +	mutex_lock(&mpath_head->lock);
>> +	/*
>> +	 * Ensure that no one could remove this module while the head
>> +	 * remove work is pending.
>> +	 */
>> +	if (mpath_head_queue_if_no_path(mpath_head) &&
>> +	    !check_mul_overflow(mpath_head->delayed_removal_secs, HZ, &delay) &&
> 
> [Severity: Medium]
> What happens if a user configures a very large delayed removal value?
> 
> If check_mul_overflow() detects an overflow (e.g., on 32-bit systems where
> a large sysfs input overflows unsigned long), it fails the condition and
> falls through to the else block, setting remove = true. Does this cause an
> immediate teardown instead of clamping the delay to a maximum value?

yes, so don't use a silly big value

> 
>> +	    try_module_get(mpath_head->drv_module)) {
>> +		mod_delayed_work(mpath_wq, &mpath_head->remove_work, delay);
> 
> [Severity: Medium]
> Could this lead to a module reference leak?
> 
> If mod_delayed_work() returns true (meaning the work was already pending
> and its timer was just modified), it doesn't queue a new work item.
> Since try_module_get() was called unconditionally beforehand, and the
> work callback only drops one module reference, do subsequent calls to
> mpath_can_remove_head() leak a module reference permanently?

it should not happen - that is the job of the caller to ensure

> 
>> +	} else {
>> +		remove = true;
>> +	}
>> +
>> +	mutex_unlock(&mpath_head->lock);
>> +	return remove;
>> +}
> 
> [ ... ]
> 
>> @@ -529,6 +574,9 @@ int mpath_alloc_head_disk(struct mpath_head *mpath_head,
>>   	mpath_head->disk->private_data = mpath_head;
>>   	mpath_head->disk->fops = &mpath_ops;
>>   
>> +	INIT_DELAYED_WORK(&mpath_head->remove_work, mpath_remove_head_work);
>> +	mpath_head->delayed_removal_secs = 0;
>> +
> 
> [Severity: High]
> Is drv_module properly initialized anywhere?

yes

> 
> It seems drv_module is added to struct mpath_head but never assigned in
> mpath_alloc_head_disk() or elsewhere in this patch. If it defaults to
> NULL, try_module_get(NULL) succeeds but pins no module. Could this allow
> the driver to unload while the delayed work is still pending, leading to
> a kernel panic when the timer fires?
> 
> [Severity: High]
> Is the remove_work guaranteed to be cancelled during object teardown?

it must be assigned

> 
> The standard teardown sequence calls mpath_put_disk() and
> mpath_head_uninit(). If cancel_delayed_work_sync(&mpath_head->remove_work)
> is omitted during teardown, could the timer remain active on the
> workqueue and execute on the freed mpath_head structure later?
> 
>>   	set_bit(GD_SUPPRESS_PART_SCAN, &mpath_head->disk->state);
>>   
>>   	return 0;
> 
> [ ... ]
> 
>> @@ -572,6 +620,47 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
>>   }
>>   EXPORT_SYMBOL_GPL(mpath_device_set_live);
>>   
>> +ssize_t mpath_delayed_removal_secs_show(struct mpath_head *mpath_head,
>> +					char *buf)
>> +{
>> +	int ret;
>> +
>> +	mutex_lock(&mpath_head->lock);
>> +	ret = sysfs_emit(buf, "%u\n", mpath_head->delayed_removal_secs);
>> +	mutex_unlock(&mpath_head->lock);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(mpath_delayed_removal_secs_show);
>> +
>> +ssize_t mpath_delayed_removal_secs_store(struct mpath_head *mpath_head,
>> +			const char *buf, size_t count)
>> +{
>> +	unsigned int sec;
>> +	ssize_t ret;
>> +
>> +	ret = kstrtouint(buf, 0, &sec);
>> +	if (ret < 0)
>> +		return ret;
>> +
>> +	mutex_lock(&mpath_head->lock);
>> +	mpath_head->delayed_removal_secs = sec;
>> +	if (sec)
>> +		set_bit(MPATH_HEAD_QUEUE_IF_NO_PATH, &mpath_head->flags);
>> +	else
>> +		clear_bit(MPATH_HEAD_QUEUE_IF_NO_PATH, &mpath_head->flags);
>> +	mutex_unlock(&mpath_head->lock);
>> +
>> +	/*
>> +	 * Ensure that update to MPATH_HEAD_QUEUE_IF_NO_PATH is seen
>> +	 * by its reader.
>> +	 */
>> +	mpath_synchronize(mpath_head);
>> +
>> +	return count;
>> +}
> 
> [Severity: High]
> If a user disables delayed removal by setting delayed_removal_secs to 0 via
> sysfs while paths are missing, how do pending I/Os get handled?
> 
> We clear the flag and call mpath_synchronize(), but shouldn't we also
> call mpath_schedule_requeue_work() here to flush and fail the I/Os that
> are already parked in mpath_head->requeue_list?

I don't see how this is related to requeue work

> 

Re: [PATCH v6 07/27] libmultipath: Add sysfs helpers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The sysfs store helper `mpath_iopolicy_store` updates the `iopolicy` locklessly and without invalidating cached paths, creating a TOCTOU race on concurrent writes and leaving the system operating on stale paths.
- [Medium] Inconsistent evaluation of `iopolicy` in `__mpath_find_path` due to reading the state inside the device loop.
--

--- Patch [7]: [PATCH v6 07/27] libmultipath: Add sysfs helpers ---
commit ceaf280cc3938326ee64a5bab20fde5b9a603a83
Author: John Garry <john.g.garry@oracle.com>

libmultipath: Add sysfs helpers

This commit adds helper functions for driver sysfs code to get/set the
iopolicy, show device paths per NUMA node, and manage multipath attribute
groups.

> diff --git a/lib/multipath.c b/lib/multipath.c
> index fada92bf9754c..58eb9be5bebbe 100644
> --- a/lib/multipath.c
> +++ b/lib/multipath.c

[ ... ]

> @@ -620,6 +620,100 @@ void mpath_device_set_live(struct mpath_device *mpath_device)

[ ... ]

> +static void mpath_iopolicy_update(enum mpath_iopolicy_e *iopolicy,
> +		int new)
> +{
> +	int old = READ_ONCE(*iopolicy);
> +
> +	if (old == new)
> +		return;
> +
> +	WRITE_ONCE(*iopolicy, new);

[Severity: High]
Since kernfs does not serialize store callbacks, can this lockless
read-compare-update sequence lead to TOCTOU races on concurrent writes?

If multiple threads execute this simultaneously, could they both read the
same old state, overwrite it, and emit inconsistent logs?

Additionally, does this policy update need a way to invalidate the cached
paths? Because this helper only updates the policy and has no reference to
mpath_head, it doesn't invalidate mpath_head->current_path.

Will the new policy continue using the stale path cached by the old policy?

> +
> +	pr_info("iopolicy changed from %s to %s\n",
> +		mpath_iopolicy_names[old],
> +		mpath_iopolicy_names[new]);
> +}

[Severity: Medium]
Does introducing this concurrent sysfs update expose a TOCTOU race in
__mpath_find_path()?

If __mpath_find_path() iterates over devices and reads the IO policy on each
iteration via mpath_read_iopolicy(mpath_head):

	if (mpath_device->numa_node != NUMA_NO_NODE &&
	    (mpath_read_iopolicy(mpath_head) ==
		MPATH_IOPOLICY_NUMA))
		distance = node_distance(node,
				mpath_device->numa_node);
	else
		distance = LOCAL_DISTANCE;

Could a concurrent change here transition the policy mid-loop, causing the
function to mix incompatible node_distance metrics with LOCAL_DISTANCE and
potentially corrupting the minimum-finding logic?

Should the policy be read exactly once before the loop?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=7

Re: [PATCH v6 08/27] libmultipath: Add support for block device IOCTL

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Deadlock in `mpath_bdev_ioctl` due to holding `srcu_read_lock` while forwarding ioctls that can trigger synchronous path removal.
- [High] Bypass of underlying partition-specific IOCTL security policies because `mpath_bdev_ioctl` unconditionally passes `part0`.
--

--- Patch [8]: [PATCH v6 08/27] libmultipath: Add support for block device IOCTL ---
commit d2e45114ccfb93fccdccc60ed75e05084f9b9173
Author: John Garry <john.garry@linux.dev>

libmultipath: Add support for block device IOCTL

Add mpath_bdev_ioctl() as a multipath block device IOCTL handler. This
handler calls into the mpath_device bdev fops handler.

> diff --git a/lib/multipath.c b/lib/multipath.c
> index 58eb9be5bebbe..f1758d1e60a60 100644
> --- a/lib/multipath.c
> +++ b/lib/multipath.c
> @@ -454,11 +454,53 @@ static void mpath_bdev_release(struct gendisk *disk)
>  	mpath_put_head(mpath_head);
>  }
>  
> +static int mpath_bdev_ioctl(struct block_device *bdev, blk_mode_t mode,
> +		    unsigned int cmd, unsigned long arg)
> +{
> +	struct gendisk *disk = bdev->bd_disk;
> +	struct mpath_head *mpath_head = mpath_gendisk_to_head(disk);
> +	struct mpath_device *mpath_device;
> +	int srcu_idx, err;
> +
> +	/*
> +	 * This check is duplicated from sd_ioctl() as we don't pass the
> +	 * partition bdev to fops->ioctl. That is not yet possible as the
> +	 * per-path disk is hidden and, as such, does not have partitions
> +	 * scanned.
> +	 */
> +	if (bdev_is_partition(bdev) && !capable(CAP_SYS_RAWIO))
> +		return -ENOIOCTLCMD;
> +
> +	srcu_idx = srcu_read_lock(&mpath_head->srcu);
> +	mpath_device = mpath_find_path(mpath_head);
> +	if (!mpath_device) {
> +		err = -EWOULDBLOCK;
> +		goto out_unlock;
> +	}
> +
> +	if (!mpath_device->disk->fops->ioctl) {
> +		err = -ENOTTY;
> +		goto out_unlock;
> +	}
> +
> +	err = mpath_device->disk->fops->ioctl(
> +			mpath_device->disk->part0, mode, cmd, arg);

[Severity: High]
Does unconditionally passing part0 bypass partition-specific IOCTL security
policies in the underlying driver?

By passing the whole-disk block device (mpath_device->disk->part0),
bdev_is_partition() will evaluate to false in the underlying driver. For
drivers like NVMe, this strips the partition context and could bypass
requirements like CAP_SYS_ADMIN for passthrough commands on partitions,
allowing a user with only CAP_SYS_RAWIO to execute whole-disk commands.

[Severity: High]
Can holding the SRCU read lock across the underlying ioctl cause a deadlock?

If the underlying ioctl triggers a controller reset or device removal, the
teardown path could eventually wait on synchronize_srcu() for this exact
srcu instance. Since the ioctl is waiting for the operation to complete
while holding srcu_read_lock(), synchronize_srcu() would block forever.

> +out_unlock:
> +	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
> +	return err;
> +}

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=8

Re: [PATCH v6 07/27] libmultipath: Add sysfs helpers

[John Garry]

On 29/07/2026 13:28, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [High] The sysfs store helper `mpath_iopolicy_store` updates the `iopolicy` locklessly and without invalidating cached paths, creating a TOCTOU race on concurrent writes and leaving the system operating on stale paths.
> - [Medium] Inconsistent evaluation of `iopolicy` in `__mpath_find_path` due to reading the state inside the device loop.
> --
> 
> --- Patch [7]: [PATCH v6 07/27] libmultipath: Add sysfs helpers ---
> commit ceaf280cc3938326ee64a5bab20fde5b9a603a83
> Author: John Garry <john.g.garry@oracle.com>
> 
> libmultipath: Add sysfs helpers
> 
> This commit adds helper functions for driver sysfs code to get/set the
> iopolicy, show device paths per NUMA node, and manage multipath attribute
> groups.
> 
>> diff --git a/lib/multipath.c b/lib/multipath.c
>> index fada92bf9754c..58eb9be5bebbe 100644
>> --- a/lib/multipath.c
>> +++ b/lib/multipath.c
> 
> [ ... ]
> 
>> @@ -620,6 +620,100 @@ void mpath_device_set_live(struct mpath_device *mpath_device)
> 
> [ ... ]
> 
>> +static void mpath_iopolicy_update(enum mpath_iopolicy_e *iopolicy,
>> +		int new)
>> +{
>> +	int old = READ_ONCE(*iopolicy);
>> +
>> +	if (old == new)
>> +		return;
>> +
>> +	WRITE_ONCE(*iopolicy, new);
> 
> [Severity: High]
> Since kernfs does not serialize store callbacks, can this lockless
> read-compare-update sequence lead to TOCTOU races on concurrent writes?
> 
> If multiple threads execute this simultaneously, could they both read the
> same old state, overwrite it, and emit inconsistent logs?

Don't call it concurrently then

> 
> Additionally, does this policy update need a way to invalidate the cached
> paths? Because this helper only updates the policy and has no reference to
> mpath_head, it doesn't invalidate mpath_head->current_path.
> 
> Will the new policy continue using the stale path cached by the old policy?
> 
>> +
>> +	pr_info("iopolicy changed from %s to %s\n",
>> +		mpath_iopolicy_names[old],
>> +		mpath_iopolicy_names[new]);
>> +}
> 
> [Severity: Medium]
> Does introducing this concurrent sysfs update expose a TOCTOU race in
> __mpath_find_path()?
> 
> If __mpath_find_path() iterates over devices and reads the IO policy on each
> iteration via mpath_read_iopolicy(mpath_head):
> 
> 	if (mpath_device->numa_node != NUMA_NO_NODE &&
> 	    (mpath_read_iopolicy(mpath_head) ==
> 		MPATH_IOPOLICY_NUMA))
> 		distance = node_distance(node,
> 				mpath_device->numa_node);
> 	else
> 		distance = LOCAL_DISTANCE;
> 
> Could a concurrent change here transition the policy mid-loop, causing the
> function to mix incompatible node_distance metrics with LOCAL_DISTANCE and
> potentially corrupting the minimum-finding logic?
> 
> Should the policy be read exactly once before the loop?
> 

So we choose not the best/appropiate path once - it won't make a difference

Re: [PATCH v6 08/27] libmultipath: Add support for block device IOCTL

[John Garry]

On 29/07/2026 13:39, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [High] Deadlock in `mpath_bdev_ioctl` due to holding `srcu_read_lock` while forwarding ioctls that can trigger synchronous path removal.
> - [High] Bypass of underlying partition-specific IOCTL security policies because `mpath_bdev_ioctl` unconditionally passes `part0`.
> --
> 
> --- Patch [8]: [PATCH v6 08/27] libmultipath: Add support for block device IOCTL ---
> commit d2e45114ccfb93fccdccc60ed75e05084f9b9173
> Author: John Garry <john.garry@linux.dev>
> 
> libmultipath: Add support for block device IOCTL
> 
> Add mpath_bdev_ioctl() as a multipath block device IOCTL handler. This
> handler calls into the mpath_device bdev fops handler.
> 
>> diff --git a/lib/multipath.c b/lib/multipath.c
>> index 58eb9be5bebbe..f1758d1e60a60 100644
>> --- a/lib/multipath.c
>> +++ b/lib/multipath.c
>> @@ -454,11 +454,53 @@ static void mpath_bdev_release(struct gendisk *disk)
>>   	mpath_put_head(mpath_head);
>>   }
>>   
>> +static int mpath_bdev_ioctl(struct block_device *bdev, blk_mode_t mode,
>> +		    unsigned int cmd, unsigned long arg)
>> +{
>> +	struct gendisk *disk = bdev->bd_disk;
>> +	struct mpath_head *mpath_head = mpath_gendisk_to_head(disk);
>> +	struct mpath_device *mpath_device;
>> +	int srcu_idx, err;
>> +
>> +	/*
>> +	 * This check is duplicated from sd_ioctl() as we don't pass the
>> +	 * partition bdev to fops->ioctl. That is not yet possible as the
>> +	 * per-path disk is hidden and, as such, does not have partitions
>> +	 * scanned.
>> +	 */
>> +	if (bdev_is_partition(bdev) && !capable(CAP_SYS_RAWIO))
>> +		return -ENOIOCTLCMD;
>> +
>> +	srcu_idx = srcu_read_lock(&mpath_head->srcu);
>> +	mpath_device = mpath_find_path(mpath_head);
>> +	if (!mpath_device) {
>> +		err = -EWOULDBLOCK;
>> +		goto out_unlock;
>> +	}
>> +
>> +	if (!mpath_device->disk->fops->ioctl) {
>> +		err = -ENOTTY;
>> +		goto out_unlock;
>> +	}
>> +
>> +	err = mpath_device->disk->fops->ioctl(
>> +			mpath_device->disk->part0, mode, cmd, arg);
> 
> [Severity: High]
> Does unconditionally passing part0 bypass partition-specific IOCTL security
> policies in the underlying driver?

Check the comment above regarding sd_ioctl()

> 
> By passing the whole-disk block device (mpath_device->disk->part0),
> bdev_is_partition() will evaluate to false in the underlying driver. For
> drivers like NVMe, this strips the partition context and could bypass
> requirements like CAP_SYS_ADMIN for passthrough commands on partitions,
> allowing a user with only CAP_SYS_RAWIO to execute whole-disk commands.
> 
> [Severity: High]
> Can holding the SRCU read lock across the underlying ioctl cause a deadlock?
> 
> If the underlying ioctl triggers a controller reset or device removal, the
> teardown path could eventually wait on synchronize_srcu() for this exact
> srcu instance. Since the ioctl is waiting for the operation to complete
> while holding srcu_read_lock(), synchronize_srcu() would block forever.
> 

Currently the only ioctls supported cannot support doing controller 
reset or device removal, etc.

>> +out_unlock:
>> +	srcu_read_unlock(&mpath_head->srcu, srcu_idx);
>> +	return err;
>> +}
> 
> [ ... ]
> 

Re: [PATCH v6 12/27] scsi-multipath: introduce scsi_device head structure

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] The struct `mpath_head_template smpdt` is defined completely empty, lacking logically required callbacks, leading to NULL pointer dereferences.
- [Medium] The patch omits the logic to create sysfs links to member `scsi_device`'s, contradicting the stated commit message.
- [Low] The function `scsi_multipath_sysfs_group_visible` and the macro `DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(scsi_multipath_sysfs)` generate unused static code, causing a `-Wunused-function` warning.
--

--- Patch [12]: [PATCH v6 12/27] scsi-multipath: introduce scsi_device head structure ---
commit 41b0248171de9edac7171ac10282e0c5b3f1056c
Author: John Garry <john.garry@linux.dev>

scsi-multipath: introduce scsi_device head structure

Introduce a scsi_mpath_head structure to manage multipathing for a
scsi_device. A list of scsi_mpath_head structures is managed to lookup for
matching multipathed scsi_device's.

> The sysfs device folder will have links to the scsi_device's so, it will
> be possible to lookup the member scsi_device's.

[Severity: Medium]
I notice the patch doesn't seem to contain the code to create these sysfs links
to the member scsi_device structures. Is this implementation missing, or is it
planned for a future patch?

diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
index 5559e2e8d64a8..30a2d01be8aaf 100644
--- a/drivers/scsi/scsi_multipath.c
+++ b/drivers/scsi/scsi_multipath.c

[ ... ]

> +static const struct attribute_group scsi_mpath_device_attrs_group = {
> +	.attrs = scsi_mpath_device_attrs,
> +};
> +
> +static bool scsi_multipath_sysfs_group_visible(struct kobject *kobj)
> +{
> +	return true;
> +}
> +DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(scsi_multipath_sysfs)

[Severity: Low]
Is this function and macro used anywhere in this patch? 

It looks like DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE generates a static function 
that isn't assigned to any attribute_group.is_visible callback, which might 
cause a unused-function compiler warning.

> +
> +static const struct attribute_group *scsi_mpath_device_groups[] = {
> +	&scsi_mpath_device_attrs_group,
> +	NULL
> +};

[ ... ]

>  static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>  {
>  	struct Scsi_Host *shost = sdev->host;
> @@ -91,6 +144,74 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>  	return 0;
>  }
>  
> +static struct mpath_head_template smpdt = {
> +};

[Severity: High]
Will this cause a null pointer dereference when the generic multipath core
attempts to evaluate I/O paths? 

Since smpdt is defined completely empty without any callback assignments, 
function pointers like mpath_head->mpdt->is_disabled will be null when 
dereferenced by the framework.

> +
> +static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
> +{
> +	struct scsi_mpath_head *scsi_mpath_head;

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=12

Re: [PATCH v6 12/27] scsi-multipath: introduce scsi_device head structure

[John Garry]

On 29/07/2026 14:46, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> - [High] The struct `mpath_head_template smpdt` is defined completely empty, lacking logically required callbacks, leading to NULL pointer dereferences.
> - [Medium] The patch omits the logic to create sysfs links to member `scsi_device`'s, contradicting the stated commit message.
> - [Low] The function `scsi_multipath_sysfs_group_visible` and the macro `DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(scsi_multipath_sysfs)` generate unused static code, causing a `-Wunused-function` warning.
> --
> 
> --- Patch [12]: [PATCH v6 12/27] scsi-multipath: introduce scsi_device head structure ---
> commit 41b0248171de9edac7171ac10282e0c5b3f1056c
> Author: John Garry <john.garry@linux.dev>
> 
> scsi-multipath: introduce scsi_device head structure
> 
> Introduce a scsi_mpath_head structure to manage multipathing for a
> scsi_device. A list of scsi_mpath_head structures is managed to lookup for
> matching multipathed scsi_device's.
> 
>> The sysfs device folder will have links to the scsi_device's so, it will
>> be possible to lookup the member scsi_device's.
> 
> [Severity: Medium]
> I notice the patch doesn't seem to contain the code to create these sysfs links
> to the member scsi_device structures. Is this implementation missing, or is it
> planned for a future patch?

future patch

> 
> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> index 5559e2e8d64a8..30a2d01be8aaf 100644
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c
> 
> [ ... ]
> 
>> +static const struct attribute_group scsi_mpath_device_attrs_group = {
>> +	.attrs = scsi_mpath_device_attrs,
>> +};
>> +
>> +static bool scsi_multipath_sysfs_group_visible(struct kobject *kobj)
>> +{
>> +	return true;
>> +}
>> +DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(scsi_multipath_sysfs)
> 
> [Severity: Low]
> Is this function and macro used anywhere in this patch?
> 
> It looks like DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE generates a static function
> that isn't assigned to any attribute_group.is_visible callback, which might
> cause a unused-function compiler warning.

yes, it is used later. For me, the compiler does not complain as 
DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE uses inline in the function declaration

> 
>> +
>> +static const struct attribute_group *scsi_mpath_device_groups[] = {
>> +	&scsi_mpath_device_attrs_group,
>> +	NULL
>> +};
> 
> [ ... ]
> 
>>   static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>>   {
>>   	struct Scsi_Host *shost = sdev->host;
>> @@ -91,6 +144,74 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>>   	return 0;
>>   }
>>   
>> +static struct mpath_head_template smpdt = {
>> +};
> 
> [Severity: High]
> Will this cause a null pointer dereference when the generic multipath core
> attempts to evaluate I/O paths?
> 
> Since smpdt is defined completely empty without any callback assignments,
> function pointers like mpath_head->mpdt->is_disabled will be null when
> dereferenced by the framework.

it is filled in later

> 
>> +
>> +static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
>> +{
>> +	struct scsi_mpath_head *scsi_mpath_head;
> 
> [ ... ]
> 

Re: [PATCH v6 14/27] scsi-multipath: support iopolicy

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] The struct mpath_head_template 'smpdt' is entirely empty, lacking implementations for logically required callbacks, leading to NULL pointer dereferences.
- [Low] Missing READ_ONCE() when reading the global `iopolicy` variable, leading to a data race.
--

--- Patch [14]: [PATCH v6 14/27] scsi-multipath: support iopolicy ---
commit f1795dc3e5eb8bb57ed7783aab4a6563ae5c3b62
Author: John Garry <john.g.garry@oracle.com>

scsi-multipath: support iopolicy

Add support to set the multipath iopolicy. The iopolicy member is per
scsi_mpath_head structure, and a module param is added so that the default
iopolicy may be set.

> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> index 1ec7bf1835de0..593f94aac70e9 100644
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c
> @@ -61,6 +61,23 @@ static const struct kernel_param_ops multipath_param_ops = {
>  module_param_cb(multipath, &multipath_param_ops, &scsi_multipath, 0444);
>  MODULE_PARM_DESC(multipath, "turn on native multipath support, options: on, off, always");
>  
> +static enum mpath_iopolicy_e iopolicy = MPATH_IOPOLICY_NUMA;
> +
> +static int scsi_mpath_set_iopolicy_param(const char *val, const struct kernel_param *kp)
> +{
> +	return mpath_set_iopolicy(val, &iopolicy);
> +}
> +
> +static int scsi_mpath_get_iopolicy_param(char *buf, const struct kernel_param *kp)
> +{
> +	return mpath_get_iopolicy(buf, iopolicy);

[Severity: Low]
Is a READ_ONCE() missing here when reading the global iopolicy variable?

mpath_set_iopolicy() updates the variable using WRITE_ONCE(). Concurrent
reads here could lead to a data race and trigger KCSAN warnings.

> +}
> +
> +module_param_call(multipath_iopolicy, scsi_mpath_set_iopolicy_param,
> +		scsi_mpath_get_iopolicy_param, &iopolicy, 0644);
> +MODULE_PARM_DESC(multipath_iopolicy,
> +	"Default multipath I/O policy; 'numa' (default), 'round-robin' or 'queue-depth'");
> +

[ ... ]

> @@ -204,6 +254,8 @@ static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
>  	if (mpath_head_init(&scsi_mpath_head->mpath_head))
>  		goto out_free;
>  	scsi_mpath_head->mpath_head.mpdt = &smpdt;

[Severity: Critical]
Since the scsi_mpath_head is initialized with smpdt here, and smpdt appears
to be an empty template, could this lead to NULL pointer dereferences in the
core multipath library?

drivers/scsi/scsi_multipath.c:
static struct mpath_head_template smpdt = {
};

The core code unconditionally dereferences these callbacks. For example, during
path validation:

lib/multipath.c:mpath_path_is_disabled() {
    ...
    return mpath_head->mpdt->is_disabled(mpath_device);
}

> +	scsi_mpath_head->iopolicy = iopolicy;

[Severity: Low]
Would it be better to also use READ_ONCE() here when reading the global
iopolicy variable during initialization?

> +	scsi_mpath_head->mpath_head.iopolicy = &scsi_mpath_head->iopolicy;
>  
>  	strscpy(scsi_mpath_head->vpd_id, vpd_id,
>  		SCSI_MPATH_DEVICE_ID_LEN);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=14

Re: [PATCH v6 14/27] scsi-multipath: support iopolicy

[John Garry]

On 29/07/2026 15:07, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [Critical] The struct mpath_head_template 'smpdt' is entirely empty, lacking implementations for logically required callbacks, leading to NULL pointer dereferences.
> - [Low] Missing READ_ONCE() when reading the global `iopolicy` variable, leading to a data race.
> --
> 
> --- Patch [14]: [PATCH v6 14/27] scsi-multipath: support iopolicy ---
> commit f1795dc3e5eb8bb57ed7783aab4a6563ae5c3b62
> Author: John Garry <john.g.garry@oracle.com>
> 
> scsi-multipath: support iopolicy
> 
> Add support to set the multipath iopolicy. The iopolicy member is per
> scsi_mpath_head structure, and a module param is added so that the default
> iopolicy may be set.
> 
>> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
>> index 1ec7bf1835de0..593f94aac70e9 100644
>> --- a/drivers/scsi/scsi_multipath.c
>> +++ b/drivers/scsi/scsi_multipath.c
>> @@ -61,6 +61,23 @@ static const struct kernel_param_ops multipath_param_ops = {
>>   module_param_cb(multipath, &multipath_param_ops, &scsi_multipath, 0444);
>>   MODULE_PARM_DESC(multipath, "turn on native multipath support, options: on, off, always");
>>   
>> +static enum mpath_iopolicy_e iopolicy = MPATH_IOPOLICY_NUMA;
>> +
>> +static int scsi_mpath_set_iopolicy_param(const char *val, const struct kernel_param *kp)
>> +{
>> +	return mpath_set_iopolicy(val, &iopolicy);
>> +}
>> +
>> +static int scsi_mpath_get_iopolicy_param(char *buf, const struct kernel_param *kp)
>> +{
>> +	return mpath_get_iopolicy(buf, iopolicy);
> 
> [Severity: Low]
> Is a READ_ONCE() missing here when reading the global iopolicy variable?
> 
> mpath_set_iopolicy() updates the variable using WRITE_ONCE(). Concurrent
> reads here could lead to a data race and trigger KCSAN warnings.
> 
>> +}
>> +
>> +module_param_call(multipath_iopolicy, scsi_mpath_set_iopolicy_param,
>> +		scsi_mpath_get_iopolicy_param, &iopolicy, 0644);
>> +MODULE_PARM_DESC(multipath_iopolicy,
>> +	"Default multipath I/O policy; 'numa' (default), 'round-robin' or 'queue-depth'");
>> +
> 
> [ ... ]
> 
>> @@ -204,6 +254,8 @@ static struct scsi_mpath_head *scsi_mpath_alloc_head(char *vpd_id)
>>   	if (mpath_head_init(&scsi_mpath_head->mpath_head))
>>   		goto out_free;
>>   	scsi_mpath_head->mpath_head.mpdt = &smpdt;
> 
> [Severity: Critical]
> Since the scsi_mpath_head is initialized with smpdt here, and smpdt appears
> to be an empty template, could this lead to NULL pointer dereferences in the
> core multipath library?
> 

It is filled in later. And that is not even related to this patch.

> drivers/scsi/scsi_multipath.c:
> static struct mpath_head_template smpdt = {
> };
> 
> The core code unconditionally dereferences these callbacks. For example, during
> path validation:
> 
> lib/multipath.c:mpath_path_is_disabled() {
>      ...
>      return mpath_head->mpdt->is_disabled(mpath_device);
> }
> 
>> +	scsi_mpath_head->iopolicy = iopolicy;
> 
> [Severity: Low]
> Would it be better to also use READ_ONCE() here when reading the global
> iopolicy variable during initialization?

This won't make a difference, as iopolicy is only written once at module 
init time.

> 
>> +	scsi_mpath_head->mpath_head.iopolicy = &scsi_mpath_head->iopolicy;
>>   
>>   	strscpy(scsi_mpath_head->vpd_id, vpd_id,
>>   		SCSI_MPATH_DEVICE_ID_LEN);
> 

Re: [PATCH v6 15/27] scsi-multipath: clone each bio

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] The `smpdt` structure fails to implement mandatory `mpath_head_template` callbacks, leading to a NULL pointer dereference.
- [High] The clone completion callback entirely lacks the failover resubmission logic claimed in the commit message, breaking failover handling.
- [High] The `bio_pool` mempool is destroyed before waiting for active users to drain, leading to a use-after-free when in-flight cloned bios complete.
--

--- Patch [15]: [PATCH v6 15/27] scsi-multipath: clone each bio ---
commit ea53159f3e5b8959582afd4b035267bb7a201a6d
Author: John Garry <john.garry@linux.dev>

scsi-multipath: clone each bio

For failover handling, we will take the approach to resubmit each
bio.

However, unlike NVMe, for SCSI there is no guarantee that any bio submitted
is either all or none completed.

As such, for SCSI, for failover handling we will take the approach to
just re-submit the original bio. For this, clone and submit each bio.

> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> index 593f94aac70e9..0d439cac6447a 100644
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c
> @@ -99,6 +99,7 @@ static void scsi_mpath_head_release(struct device *dev)
>  		container_of(dev, struct scsi_mpath_head, dev);
>  	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
>  
> +	bioset_exit(&scsi_mpath_head->bio_pool);
>  	ida_free(&scsi_multipath_dev_ida, scsi_mpath_head->index);
>  	ida_destroy(&scsi_mpath_head->ida);
>  	mpath_head_uninit(mpath_head);

[Severity: High]
Does this code destroy the mempool before all users have drained?

When the last SCSI path is removed, scsi_mpath_head_release() calls
bioset_exit() immediately. It then calls mpath_head_uninit(), which
synchronously waits for all users to release their references.

If there are in-flight cloned bios completing, or if the block device is
still open and receives new I/O while waiting, they could access the
already-freed bio_pool via bio_put() or bio_alloc_clone().

Can this result in a use-after-free on the destroyed mempool?

[ ... ]

> @@ -237,7 +238,34 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>  	return 0;
>  }
>  
> +static void scsi_mpath_clone_end_io(struct bio *clone)
> +{
> +	struct bio *master_bio = clone->bi_private;
> +
> +	master_bio->bi_status = clone->bi_status;
> +	bio_put(clone);
> +	bio_endio(master_bio);
> +}

[Severity: High]
Does this callback omit the failover resubmission logic mentioned in the
commit message?

When an underlying SCSI path fails an I/O, this function unconditionally
copies the error to master_bio and calls bio_endio().

Since the commit message states the approach is to re-submit the original
bio for failover handling, shouldn't there be code here to intercept the
error and requeue it, rather than immediately returning the error to the
issuer?

[ ... ]

>  static struct mpath_head_template smpdt = {
> +	.clone_bio = scsi_mpath_clone_bio,
>  };

[Severity: High]
Is this struct missing mandatory mpath_head_template callbacks?

Any I/O submission to the multipath block device invokes mpath_find_path(),
which unconditionally calls mpath_head->mpdt->is_disabled(mpath_device).

Because smpdt only initializes .clone_bio, functions like is_disabled,
is_optimized, and available_path will be NULL.

Will this cause an immediate kernel panic due to a NULL pointer dereference
when an I/O is issued?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=15

Re: [PATCH v6 15/27] scsi-multipath: clone each bio

[John Garry]

On 29/07/2026 15:23, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> - [High] The `smpdt` structure fails to implement mandatory `mpath_head_template` callbacks, leading to a NULL pointer dereference.
> - [High] The clone completion callback entirely lacks the failover resubmission logic claimed in the commit message, breaking failover handling.
> - [High] The `bio_pool` mempool is destroyed before waiting for active users to drain, leading to a use-after-free when in-flight cloned bios complete.
> --
> 
> --- Patch [15]: [PATCH v6 15/27] scsi-multipath: clone each bio ---
> commit ea53159f3e5b8959582afd4b035267bb7a201a6d
> Author: John Garry <john.garry@linux.dev>
> 
> scsi-multipath: clone each bio
> 
> For failover handling, we will take the approach to resubmit each
> bio.
> 
> However, unlike NVMe, for SCSI there is no guarantee that any bio submitted
> is either all or none completed.
> 
> As such, for SCSI, for failover handling we will take the approach to
> just re-submit the original bio. For this, clone and submit each bio.
> 
>> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
>> index 593f94aac70e9..0d439cac6447a 100644
>> --- a/drivers/scsi/scsi_multipath.c
>> +++ b/drivers/scsi/scsi_multipath.c
>> @@ -99,6 +99,7 @@ static void scsi_mpath_head_release(struct device *dev)
>>   		container_of(dev, struct scsi_mpath_head, dev);
>>   	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
>>   
>> +	bioset_exit(&scsi_mpath_head->bio_pool);
>>   	ida_free(&scsi_multipath_dev_ida, scsi_mpath_head->index);
>>   	ida_destroy(&scsi_mpath_head->ida);
>>   	mpath_head_uninit(mpath_head);
> 
> [Severity: High]
> Does this code destroy the mempool before all users have drained?

Yes, all bios should be finished at this stage

> 
> When the last SCSI path is removed, scsi_mpath_head_release() calls
> bioset_exit() immediately. It then calls mpath_head_uninit(), which
> synchronously waits for all users to release their references.
> 
> If there are in-flight cloned bios completing, or if the block device is
> still open and receives new I/O while waiting, they could access the
> already-freed bio_pool via bio_put() or bio_alloc_clone().
> 
> Can this result in a use-after-free on the destroyed mempool?
> 
> [ ... ]
> 
>> @@ -237,7 +238,34 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>>   	return 0;
>>   }
>>   
>> +static void scsi_mpath_clone_end_io(struct bio *clone)
>> +{
>> +	struct bio *master_bio = clone->bi_private;
>> +
>> +	master_bio->bi_status = clone->bi_status;
>> +	bio_put(clone);
>> +	bio_endio(master_bio);
>> +}
> 
> [Severity: High]
> Does this callback omit the failover resubmission logic mentioned in the
> commit message?
> 
> When an underlying SCSI path fails an I/O, this function unconditionally
> copies the error to master_bio and calls bio_endio().
> 
> Since the commit message states the approach is to re-submit the original
> bio for failover handling, shouldn't there be code here to intercept the
> error and requeue it, rather than immediately returning the error to the
> issuer?
> 

this is done in a later patch

> [ ... ]
> 
>>   static struct mpath_head_template smpdt = {
>> +	.clone_bio = scsi_mpath_clone_bio,
>>   };
> 
> [Severity: High]
> Is this struct missing mandatory mpath_head_template callbacks?
> 
> Any I/O submission to the multipath block device invokes mpath_find_path(),
> which unconditionally calls mpath_head->mpdt->is_disabled(mpath_device).
> 
> Because smpdt only initializes .clone_bio, functions like is_disabled,
> is_optimized, and available_path will be NULL.
> 
> Will this cause an immediate kernel panic due to a NULL pointer dereference
> when an I/O is issued?
> 

they are added later

Re: [PATCH v6 16/27] scsi-multipath: clear path when device is blocked

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Unprotected, lockless access to `sdev->scsi_mpath_dev` in `scsi_device_block()` causes NULL pointer dereferences and Use-After-Free (UAF) bugs when racing with device initialization or teardown.
--

--- Patch [16]: [PATCH v6 16/27] scsi-multipath: clear path when device is blocked ---
commit 8f07d9f873129f9ecc9b8d3d02fee5bce667c745
Author: John Garry <john.garry@linux.dev>

scsi-multipath: clear path when device is blocked

Add scsi_mpath_dev_clear_path() to clear a device path when it becomes
blocked.

> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index daeb3693fe559..a1ec11050090c 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c

[ ... ]

> @@ -3008,6 +3009,8 @@ static void scsi_device_block(struct scsi_device *sdev, void *data)
>  		scsi_stop_queue(sdev);
>  
>  	mutex_unlock(&sdev->state_mutex);
> +	if ((err == 0) && sdev->scsi_mpath_dev)
> +		scsi_mpath_dev_clear_path(sdev->scsi_mpath_dev);

[Severity: High]
Is it safe to access sdev->scsi_mpath_dev without holding state_mutex?

When scsi_alloc_sdev() adds a new device to shost->__devices, it becomes
globally visible. Then scsi_mpath_dev_alloc() sets sdev->scsi_mpath_dev but
leaves scsi_mpath_head as NULL while issuing SCSI VPD queries.

If a concurrent error on the bus triggers scsi_device_block(), it could
read the partially initialized sdev->scsi_mpath_dev here outside of any locks.

Additionally, if scsi_mpath_dev_alloc() encounters an error and frees
sdev->scsi_mpath_dev, could reading it here cause a use-after-free?

During teardown, if __scsi_remove_device() is called concurrently with
scsi_device_block(), the teardown thread might set scsi_mpath_head to NULL.
Can this lead to a NULL pointer dereference when evaluating
scsi_mpath_dev_clear_path()?

>  
>  	WARN_ONCE(err, "%s: failed to block %s in state %d\n",
>  		  __func__, dev_name(&sdev->sdev_gendev), state);

[ ... ]

> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> index 0d439cac6447a..b23cd6f12c602 100644
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c

[ ... ]

> @@ -238,6 +238,16 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>  	return 0;
>  }
>  
> +void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
> +{
> +       struct mpath_device *mpath_device = &scsi_mpath_dev->mpath_device;
> +       struct scsi_mpath_head *scsi_mpath_head = scsi_mpath_dev->scsi_mpath_head;
> +       struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;

[Severity: High]
Does this code dereference a NULL pointer if it races with initialization or
teardown?

If scsi_mpath_head is NULL due to the races mentioned above, deriving
mpath_head from it will calculate an invalid pointer.

> +
> +       if (mpath_clear_current_path(mpath_device))

[Severity: High]
Could calling mpath_clear_current_path() with a partially initialized
mpath_device lead to a NULL pointer dereference?

If mpath_device->mpath_head is NULL, it will trigger an immediate dereference
in the multipath code:

lib/multipath.c:mpath_clear_current_path() {
    ...
	struct mpath_head *mpath_head = mpath_device->mpath_head;
    ...
		if (mpath_device ==
			rcu_access_pointer(mpath_head->current_path[node])) {
    ...
}

> +               mpath_synchronize(mpath_head);
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=16

Re: [PATCH v6 16/27] scsi-multipath: clear path when device is blocked

[John Garry]

On 29/07/2026 15:42, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Unprotected, lockless access to `sdev->scsi_mpath_dev` in `scsi_device_block()` causes NULL pointer dereferences and Use-After-Free (UAF) bugs when racing with device initialization or teardown.
> --
> 

Same as v5, there should be not races

> --- Patch [16]: [PATCH v6 16/27] scsi-multipath: clear path when device is blocked ---
> commit 8f07d9f873129f9ecc9b8d3d02fee5bce667c745
> Author: John Garry <john.garry@linux.dev>
> 
> scsi-multipath: clear path when device is blocked
> 
> Add scsi_mpath_dev_clear_path() to clear a device path when it becomes
> blocked.
> 
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index daeb3693fe559..a1ec11050090c 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
> 
> [ ... ]
> 
>> @@ -3008,6 +3009,8 @@ static void scsi_device_block(struct scsi_device *sdev, void *data)
>>   		scsi_stop_queue(sdev);
>>   
>>   	mutex_unlock(&sdev->state_mutex);
>> +	if ((err == 0) && sdev->scsi_mpath_dev)
>> +		scsi_mpath_dev_clear_path(sdev->scsi_mpath_dev);
> 
> [Severity: High]
> Is it safe to access sdev->scsi_mpath_dev without holding state_mutex?
> 
> When scsi_alloc_sdev() adds a new device to shost->__devices, it becomes
> globally visible. Then scsi_mpath_dev_alloc() sets sdev->scsi_mpath_dev but
> leaves scsi_mpath_head as NULL while issuing SCSI VPD queries.
> 
> If a concurrent error on the bus triggers scsi_device_block(), it could
> read the partially initialized sdev->scsi_mpath_dev here outside of any locks.
> 
> Additionally, if scsi_mpath_dev_alloc() encounters an error and frees
> sdev->scsi_mpath_dev, could reading it here cause a use-after-free?
> 
> During teardown, if __scsi_remove_device() is called concurrently with
> scsi_device_block(), the teardown thread might set scsi_mpath_head to NULL.
> Can this lead to a NULL pointer dereference when evaluating
> scsi_mpath_dev_clear_path()?
> 
>>   
>>   	WARN_ONCE(err, "%s: failed to block %s in state %d\n",
>>   		  __func__, dev_name(&sdev->sdev_gendev), state);
> 
> [ ... ]
> 
>> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
>> index 0d439cac6447a..b23cd6f12c602 100644
>> --- a/drivers/scsi/scsi_multipath.c
>> +++ b/drivers/scsi/scsi_multipath.c
> 
> [ ... ]
> 
>> @@ -238,6 +238,16 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>>   	return 0;
>>   }
>>   
>> +void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
>> +{
>> +       struct mpath_device *mpath_device = &scsi_mpath_dev->mpath_device;
>> +       struct scsi_mpath_head *scsi_mpath_head = scsi_mpath_dev->scsi_mpath_head;
>> +       struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
> 
> [Severity: High]
> Does this code dereference a NULL pointer if it races with initialization or
> teardown?
> 
> If scsi_mpath_head is NULL due to the races mentioned above, deriving
> mpath_head from it will calculate an invalid pointer.
> 
>> +
>> +       if (mpath_clear_current_path(mpath_device))
> 
> [Severity: High]
> Could calling mpath_clear_current_path() with a partially initialized
> mpath_device lead to a NULL pointer dereference?
> 
> If mpath_device->mpath_head is NULL, it will trigger an immediate dereference
> in the multipath code:
> 
> lib/multipath.c:mpath_clear_current_path() {
>      ...
> 	struct mpath_head *mpath_head = mpath_device->mpath_head;
>      ...
> 		if (mpath_device ==
> 			rcu_access_pointer(mpath_head->current_path[node])) {
>      ...
> }
> 
>> +               mpath_synchronize(mpath_head);
>> +}
> 

Re: [PATCH v6 17/27] scsi-multipath: revalidate paths upon device unblock

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [High] Calling `scsi_mpath_revalidate_paths()` outside `sdev->state_mutex` without a NULL check leads to TOCTOU races with device removal or allocation, causing a NULL pointer dereference.
- [High] Direct callers of `scsi_internal_device_unblock_nowait` bypass path revalidation, which can leave queued bios indefinitely stalled.

Pre-existing issues:
- [High] The patch calls `mpath_revalidate_paths()`, which schedules `requeue_work` on a background workqueue, but the teardown path fails to cancel this work, leading to a Use-After-Free.
--

--- Patch [17]: [PATCH v6 17/27] scsi-multipath: revalidate paths upon device unblock ---
commit 6253cb1fc352f4ec6d0d42ac99c1827b35592ff4
Author: John Garry <john.garry@linux.dev>

scsi-multipath: revalidate paths upon device unblock

When a scsi_device is unblocked, we need revalidate paths which means that
we clear paths and requeue any queued bios.

> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index a1ec11050090c..6a2d7dbfbe8c3 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -3092,6 +3092,8 @@ static int scsi_internal_device_unblock(struct scsi_device *sdev,
>  	mutex_lock(&sdev->state_mutex);
>  	ret = scsi_internal_device_unblock_nowait(sdev, new_state);
>  	mutex_unlock(&sdev->state_mutex);
> +	if ((ret == 0) && sdev->scsi_mpath_dev)
> +		scsi_mpath_revalidate_paths(sdev->scsi_mpath_dev);

[Severity: High]
Is it safe to access sdev->scsi_mpath_dev and call scsi_mpath_revalidate_paths()
outside of sdev->state_mutex?

It seems that concurrently, __scsi_remove_device() could nullify
scsi_mpath_dev->scsi_mpath_head. A similar race might exist during device
initialization before the head is populated.

When scsi_mpath_revalidate_paths() is then called, it blindly computes the
offset to &scsi_mpath_head->mpath_head, which could lead to a NULL pointer
dereference.

[Severity: High]
Does placing this hook in the wrapper bypass path revalidation for some drivers?

Low-level drivers like mpt3sas and scsi_transport_srp call the core API
scsi_internal_device_unblock_nowait() directly. Since the revalidation is
only added to the scsi_internal_device_unblock() wrapper, won't devices managed
by those drivers miss path revalidation when unblocked, potentially leaving
queued bios permanently stalled?

>  
>  	return ret;
>  }
> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> index b23cd6f12c602..12750621a7fc0 100644
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c
[ ... ]
> @@ -238,6 +238,14 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>  	return 0;
>  }
>  
> +void scsi_mpath_revalidate_paths(struct scsi_mpath_device *scsi_mpath_dev)
> +{
> +       struct scsi_mpath_head *scsi_mpath_head = scsi_mpath_dev->scsi_mpath_head;
> +       struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
> +
> +       mpath_revalidate_paths(mpath_head);

[Severity: High]
This is a pre-existing issue, but by introducing this new asynchronous trigger,
could we run into a use-after-free during device removal?

Calling mpath_revalidate_paths() queues mpath_head->requeue_work. If the device
is subsequently removed, the teardown path in scsi_mpath_head_release() frees
scsi_mpath_head via kfree():

drivers/scsi/scsi_multipath.c:scsi_mpath_head_release() {
    ...
    mpath_head_uninit(mpath_head);
    kfree(scsi_mpath_head);
}

Because cancel_work_sync(&mpath_head->requeue_work) is missing from the teardown
logic, could the pending work execute on a freed mpath_head structure?

> +}
> +

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=17

Re: [PATCH v6 18/27] scsi-multipath: failover handling

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] List corruption and use-after-free occur due to `bio_list_add_clone()` failing to nullify `master_bio->bi_next`.
- [High] A use-after-free crash occurs because `cancel_work_sync()` is missing during `mpath_head` teardown.
--

--- Patch [18]: [PATCH v6 18/27] scsi-multipath: failover handling ---
commit d3d46b0330ddb0dc22577a7546f9765b9e1a4651
Author: John Garry <john.garry@linux.dev>

scsi-multipath: failover handling
    
For a scsi command which suffers failover, requeue the master bio of each
bio attached to its request.

> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> index 12750621a7fc0..6ced89576368c 100644
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c
> @@ -256,10 +256,49 @@ void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
[ ... ]
> +static inline void bio_list_add_clone(struct bio_list *bl,
> +				struct bio *clone)
> +{
> +	struct bio *master_bio = clone->bi_private;
> +
> +	if (bl->tail)
> +		bl->tail->bi_next = master_bio;
> +	else
> +		bl->head = master_bio;
> +	bl->tail = master_bio;
> +	bio_put(clone);
> +}

[Severity: High]
Will this leave the newly appended list element improperly terminated?

This custom insertion into requeue_list appears to omit the standard
master_bio->bi_next = NULL assignment for the new list tail.

If master_bio->bi_next happens to contain a stale pointer when it is added
here, could iterating the list later result in memory corruption or memory
being accessed after it has been freed?

[ ... ]
> +static void scsi_mpath_clone_end_io(struct bio *clone)
> +{
> +	struct bio *master_bio = clone->bi_private;
> +
> +	if (clone->bi_status && blk_path_error(clone->bi_status)) {
[ ... ]
> +		spin_lock_irqsave(&mpath_head->requeue_lock, flags);
> +		bio_list_add_clone(&mpath_head->requeue_list, clone);
> +		spin_unlock_irqrestore(&mpath_head->requeue_lock, flags);
> +
> +		mpath_schedule_requeue_work(mpath_head);

[Severity: High]
Is it possible for the scheduled requeue work item to run after the
mpath_head structure has been torn down?

If a device hot-unplug overlaps with I/O path errors, requeue_work is
scheduled here.

When teardown proceeds to remove the device and drops the last reference,
it appears scsi_mpath_head_release() frees the scsi_mpath_head structure
without calling cancel_work_sync() first.

Could this sequence lead to a delayed kworker executing on deallocated
memory?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=18

Re: [PATCH v6 17/27] scsi-multipath: revalidate paths upon device unblock

[John Garry]

On 29/07/2026 15:54, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> 
> New issues:
> - [High] Calling `scsi_mpath_revalidate_paths()` outside `sdev->state_mutex` without a NULL check leads to TOCTOU races with device removal or allocation, causing a NULL pointer dereference.

Same as v5 response, this should not happen

> - [High] Direct callers of `scsi_internal_device_unblock_nowait` bypass path revalidation, which can leave queued bios indefinitely stalled.

can you name relevant situations?

> 
> Pre-existing issues:
> - [High] The patch calls `mpath_revalidate_paths()`, which schedules `requeue_work` on a background workqueue, but the teardown path fails to cancel this work, leading to a Use-After-Free.

this is done by (disk) sd driver through calling mpath_put_disk()

> --
> 
> --- Patch [17]: [PATCH v6 17/27] scsi-multipath: revalidate paths upon device unblock ---
> commit 6253cb1fc352f4ec6d0d42ac99c1827b35592ff4
> Author: John Garry <john.garry@linux.dev>
> 
> scsi-multipath: revalidate paths upon device unblock
> 
> When a scsi_device is unblocked, we need revalidate paths which means that
> we clear paths and requeue any queued bios.
> 
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index a1ec11050090c..6a2d7dbfbe8c3 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
>> @@ -3092,6 +3092,8 @@ static int scsi_internal_device_unblock(struct scsi_device *sdev,
>>   	mutex_lock(&sdev->state_mutex);
>>   	ret = scsi_internal_device_unblock_nowait(sdev, new_state);
>>   	mutex_unlock(&sdev->state_mutex);
>> +	if ((ret == 0) && sdev->scsi_mpath_dev)
>> +		scsi_mpath_revalidate_paths(sdev->scsi_mpath_dev);
> 
> [Severity: High]
> Is it safe to access sdev->scsi_mpath_dev and call scsi_mpath_revalidate_paths()
> outside of sdev->state_mutex?
> 
> It seems that concurrently, __scsi_remove_device() could nullify
> scsi_mpath_dev->scsi_mpath_head. A similar race might exist during device
> initialization before the head is populated.
> 
> When scsi_mpath_revalidate_paths() is then called, it blindly computes the
> offset to &scsi_mpath_head->mpath_head, which could lead to a NULL pointer
> dereference.
> 
> [Severity: High]
> Does placing this hook in the wrapper bypass path revalidation for some drivers?
> 
> Low-level drivers like mpt3sas and scsi_transport_srp call the core API
> scsi_internal_device_unblock_nowait() directly. Since the revalidation is
> only added to the scsi_internal_device_unblock() wrapper, won't devices managed
> by those drivers miss path revalidation when unblocked, potentially leaving
> queued bios permanently stalled?
> 
>>   
>>   	return ret;
>>   }
>> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
>> index b23cd6f12c602..12750621a7fc0 100644
>> --- a/drivers/scsi/scsi_multipath.c
>> +++ b/drivers/scsi/scsi_multipath.c
> [ ... ]
>> @@ -238,6 +238,14 @@ static int scsi_multipath_sdev_init(struct scsi_device *sdev)
>>   	return 0;
>>   }
>>   
>> +void scsi_mpath_revalidate_paths(struct scsi_mpath_device *scsi_mpath_dev)
>> +{
>> +       struct scsi_mpath_head *scsi_mpath_head = scsi_mpath_dev->scsi_mpath_head;
>> +       struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
>> +
>> +       mpath_revalidate_paths(mpath_head);
> 
> [Severity: High]
> This is a pre-existing issue, but by introducing this new asynchronous trigger,
> could we run into a use-after-free during device removal?
> 
> Calling mpath_revalidate_paths() queues mpath_head->requeue_work. If the device
> is subsequently removed, the teardown path in scsi_mpath_head_release() frees
> scsi_mpath_head via kfree():
> 
> drivers/scsi/scsi_multipath.c:scsi_mpath_head_release() {
>      ...
>      mpath_head_uninit(mpath_head);
>      kfree(scsi_mpath_head);
> }
> 
> Because cancel_work_sync(&mpath_head->requeue_work) is missing from the teardown
> logic, could the pending work execute on a freed mpath_head structure?
> 
>> +}
>> +
> 

Re: [PATCH v6 18/27] scsi-multipath: failover handling

[John Garry]

On 29/07/2026 16:14, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [High] List corruption and use-after-free occur due to `bio_list_add_clone()` failing to nullify `master_bio->bi_next`.

master_bio->bi_next as we never create a request for the master_bio

> - [High] A use-after-free crash occurs because `cancel_work_sync()` is missing during `mpath_head` teardown.
> --
> 
> --- Patch [18]: [PATCH v6 18/27] scsi-multipath: failover handling ---
> commit d3d46b0330ddb0dc22577a7546f9765b9e1a4651
> Author: John Garry <john.garry@linux.dev>
> 
> scsi-multipath: failover handling
>      
> For a scsi command which suffers failover, requeue the master bio of each
> bio attached to its request.
> 
>> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
>> index 12750621a7fc0..6ced89576368c 100644
>> --- a/drivers/scsi/scsi_multipath.c
>> +++ b/drivers/scsi/scsi_multipath.c
>> @@ -256,10 +256,49 @@ void scsi_mpath_dev_clear_path(struct scsi_mpath_device *scsi_mpath_dev)
> [ ... ]
>> +static inline void bio_list_add_clone(struct bio_list *bl,
>> +				struct bio *clone)
>> +{
>> +	struct bio *master_bio = clone->bi_private;
>> +
>> +	if (bl->tail)
>> +		bl->tail->bi_next = master_bio;
>> +	else
>> +		bl->head = master_bio;
>> +	bl->tail = master_bio;
>> +	bio_put(clone);
>> +}
> 
> [Severity: High]
> Will this leave the newly appended list element improperly terminated?
> 
> This custom insertion into requeue_list appears to omit the standard
> master_bio->bi_next = NULL assignment for the new list tail.
> 
> If master_bio->bi_next happens to contain a stale pointer when it is added
> here, could iterating the list later result in memory corruption or memory
> being accessed after it has been freed?
> 
> [ ... ]
>> +static void scsi_mpath_clone_end_io(struct bio *clone)
>> +{
>> +	struct bio *master_bio = clone->bi_private;
>> +
>> +	if (clone->bi_status && blk_path_error(clone->bi_status)) {
> [ ... ]
>> +		spin_lock_irqsave(&mpath_head->requeue_lock, flags);
>> +		bio_list_add_clone(&mpath_head->requeue_list, clone);
>> +		spin_unlock_irqrestore(&mpath_head->requeue_lock, flags);
>> +
>> +		mpath_schedule_requeue_work(mpath_head);
> 
> [Severity: High]
> Is it possible for the scheduled requeue work item to run after the
> mpath_head structure has been torn down?

this flush is done at the disk teardown

> 
> If a device hot-unplug overlaps with I/O path errors, requeue_work is
> scheduled here.
> 
> When teardown proceeds to remove the device and drops the last reference,
> it appears scsi_mpath_head_release() frees the scsi_mpath_head structure
> without calling cancel_work_sync() first.
> 
> Could this sequence lead to a delayed kworker executing on deallocated
> memory?
> 

Re: [PATCH v6 19/27] scsi-multipath: provide callbacks for path state

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 8 potential issue(s) to consider:

New issues:
- [High] `scsi_mpath_is_disabled()` returns true for STANDBY paths, preventing I/O and deadlocking implicit ALUA failovers.
- [High] Bios placed on the `requeue_list` during an ALUA state change are permanently stalled due to missing notification hooks.
- [High] ALUA path prioritization is broken because routing policies check a static struct field instead of the new `.is_optimized` callback.
- [High] `scsi_mpath_available_path` ignores ALUA offline state, causing indefinite I/O hangs instead of failing.
- [High] LBA-dependent paths are incorrectly flagged as disabled, preventing I/O.
- [High] SDEV_CREATED state is omitted from available paths, causing premature I/O failures.
- [High] I/O submitted during device quiesce (e.g., system suspend) will hang indefinitely because the requeue list is never flushed upon resume.

Pre-existing issues:
- [High] Missing cancellation of background `requeue_work` task before tearing down and freeing the `scsi_mpath_head` object.
--

--- Patch [19]: [PATCH v6 19/27] scsi-multipath: provide callbacks for path state ---
commit a79ca9463c83fa5a5480f800bdc6d5f3fef27829
Author: John Garry <john.g.garry@oracle.com>

scsi-multipath: provide callbacks for path state

This commit provides callbacks for .is_disabled, .is_optimized, and
.available_path. These callbacks use the scsi_device's sdev_state
and access_state, which will be driven by ALUA.

> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c

[ ... ]

> +static bool scsi_mpath_is_disabled(struct mpath_device *mpath_device)
> +{

[ ... ]

> +	if (access_state == SCSI_ACCESS_STATE_OPTIMAL ||
> +	    access_state == SCSI_ACCESS_STATE_ACTIVE)
> +		return false;
> +
> +	return true;
> +}

[Severity: High]
Will this return true for STANDBY paths?

If implicit ALUA relies on sending I/O to a STANDBY path to transition it
to ACTIVE via a Unit Attention, marking it disabled might prevent the core
from routing any I/O to it, potentially deadlocking the failover.

[Severity: High]
Does this also need to allow SCSI_ACCESS_STATE_LBA?

SPC-4 defines LBA-dependent states as active and capable of processing I/O
commands. Should this state also be permitted here to avoid stalling I/O
on storage arrays that rely on it?

> +static bool scsi_mpath_available_path(struct mpath_device *mpath_device)
> +{

[ ... ]

> +	if (sdev_state == SDEV_RUNNING || sdev_state == SDEV_QUIESCE ||
> +	    sdev_state == SDEV_BLOCK || sdev_state == SDEV_CREATED_BLOCK)
> +		return true;
> +
> +	return false;
> +}

[Severity: High]
Should SDEV_CREATED be included here?

During SCSI device probing, the device state is SDEV_CREATED and the core
SCSI layer permits I/O operations like INQUIRY. If SDEV_CREATED is omitted
here, could valid I/O be incorrectly failed as unavailable during device
scanning?

[Severity: High]
By allowing SDEV_QUIESCE here, bios will be placed on the requeue list.

When the device is later unquiesced during a system resume,
scsi_device_resume() sets the state back to SDEV_RUNNING, but it does not
seem to trigger scsi_mpath_revalidate_paths() or schedule requeue_work.

Will bios queued during suspend remain stuck on the list indefinitely?

[Severity: High]
This function only checks sdev_state.

If an ALUA path transitions to SCSI_ACCESS_STATE_OFFLINE, sdev_state remains
SDEV_RUNNING, which causes this callback to return true.

Could this cause the block layer to endlessly requeue the I/O, believing
the path is temporarily unavailable, rather than failing it immediately
for being permanently offline?

>  static struct mpath_head_template smpdt = {
> +	.is_disabled = scsi_mpath_is_disabled,
> +	.is_optimized = scsi_mpath_is_optimized,
> +	.available_path = scsi_mpath_available_path,
>  	.clone_bio = scsi_mpath_clone_bio,
>  };

[Severity: High]
How does this integration handle ALUA state changes for requeued I/O?

When no active paths are found, bios are placed on the requeue list:

lib/multipath.c:mpath_bdev_submit_bio()
    spin_lock_irq(&mpath_head->requeue_lock);
    bio_list_add(&mpath_head->requeue_list, bio);
    spin_unlock_irq(&mpath_head->requeue_lock);

When scsi_dh_alua.c handles an ALUA state change, does it notify
scsi-multipath to schedule the requeue work? If not, could these bios
remain permanently stalled?

[Severity: High]
Are the routing algorithms in lib/multipath.c actually using this new
.is_optimized callback?

lib/multipath.c:__mpath_find_path()
    switch(mpath_device->access_state) {
    case MPATH_STATE_OPTIMIZED:

It appears the path finder checks the statically initialized access_state
field instead of invoking the new callback. Will this cause all active
paths to be incorrectly treated as optimized, bypassing path prioritization?

[Severity: High]
This is a pre-existing issue, but I noticed a potential use-after-free
during device removal in scsi_mpath_head_release():

drivers/scsi/scsi_multipath.c:scsi_mpath_head_release()
static void scsi_mpath_head_release(struct device *dev)
{
    ...
    mpath_head_uninit(mpath_head);
    kfree(scsi_mpath_head);
}

Since scsi_mpath_revalidate_paths() asynchronously schedules the
requeue_work task, if the device is removed and the refcount drops to zero,
the structure is freed without calling cancel_work_sync().

Could this allow pending work to execute on the freed scsi_mpath_head
pointer?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=19

Re: [PATCH v6 20/27] scsi-multipath: add scsi_mpath_{start,end}_request()

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Permanent leak of mpath_nr_active counter and master block device in-flight IO stats when a requeued multipath request fails with a hard error.
--

--- Patch [20]: [PATCH v6 20/27] scsi-multipath: add scsi_mpath_{start,end}_request() ---
commit 1a27bdf4453d197f11533d04e7997e7873652eb4
Author: John Garry <john.garry@linux.dev>

scsi-multipath: add scsi_mpath_{start,end}_request()

Add scsi_mpath_{start,end}_request() to handle updating private multipath
request data, like nvme_mpath_{start,end}_request().

New member Scsi_Host.mpath_nr_active is added. This is required for queue
depth multipath iopolicy.

> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index 6a2d7dbfbe8c3..ce9a2c7fbe707 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
[ ... ]
> @@ -1917,6 +1920,9 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
>  	memset(cmd->sense_buffer, 0, SCSI_SENSE_BUFFERSIZE);
>  	cmd->submitter = SUBMITTED_BY_BLOCK_LAYER;
>  
> +	if (sdev->scsi_mpath_dev && is_mpath_request(req))
> +		scsi_mpath_start_request(req);
> +
>  	blk_mq_start_request(req);
>  	if (blk_mq_is_reserved_rq(req)) {
>  		reason = shost->hostt->queue_reserved_command(shost, cmd);

[Severity: High]
Can this permanently leak the mpath_nr_active counter and master block device
in-flight IO stats when a requeued multipath request fails with a hard error?

When a multipath request is dispatched, scsi_mpath_start_request() increments
mpath_nr_active and sets SCMD_MPATH_CNT_ACTIVE. If the dispatch then fails
with BLK_STS_RESOURCE (e.g. due to a full queue), the request is requeued and
the multipath flags are preserved by SCMD_PRESERVED_FLAGS.

On a subsequent retry, if the device has transitioned to an offline state
(which is common during path failure), the early state check in
scsi_queue_rq() returns an error before scsi_mpath_start_request() is
reached:

	if (unlikely(sdev->sdev_state != SDEV_RUNNING)) {
		ret = scsi_device_state_check(sdev, req);
		if (ret != BLK_STS_OK)
			goto out_put_budget;
	}

Returning an error here causes the block layer to directly complete the
request, bypassing the driver's normal scsi_end_request() completion path.
The block layer will eventually call the scsi_cleanup_rq() hook, which merely
zeroes cmd->flags:

	cmd->flags = 0;

but fails to decrement mpath_nr_active or call bdev_end_io_acct().

Would this leave the queue-depth multipath I/O policy perceiving an
artificially inflated load on the controller, permanently degrading load
balancing?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=20

Re: [PATCH v6 21/27] scsi-multipath: add delayed disk removal support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The unchecked return value of `scsi_mpath_get_head()` in `scsi_mpath_remove_head_work()` can lead to a Use-After-Free (UAF) and double-free.
--

--- Patch [21]: [PATCH v6 21/27] scsi-multipath: add delayed disk removal support ---
commit 10f5e133107a91a4b4df48e21559fe51a6b82a5e
Author: John Garry <john.g.garry@oracle.com>

scsi-multipath: add delayed disk removal support

This commit adds core code support for delayed disk removal, introducing a
worker function that iterates over the scsi_bus_type to invoke a newly
added mpath_remove_head callback. It also attempts to manage reference
counting for scsi_mpath_head during this removal process.

> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
> index 0adc5adb4b234..a42e20d33e422 100644
> --- a/drivers/scsi/scsi_multipath.c
> +++ b/drivers/scsi/scsi_multipath.c
[ ... ]
> @@ -367,7 +367,34 @@ static bool scsi_mpath_available_path(struct mpath_device *mpath_device)
> +static void scsi_mpath_remove_head_work(struct mpath_head *mpath_head)
> +{
> +	struct scsi_mpath_head *scsi_mpath_head = to_scsi_mpath_head(mpath_head);
> +
> +	/*
> +	 * scsi_mpath_remove_head_drv() could drop the final reference to
> +	 * scsi_mpath_head, meaning that we pass a free'd pointer further
> +	 * drivers - take a reference so that won't happen.
> +	 */
> +	scsi_mpath_get_head(scsi_mpath_head);
> +	bus_for_each_drv(&scsi_bus_type, NULL, scsi_mpath_head,
> +		scsi_mpath_remove_head_drv);
> +	scsi_mpath_put_head(scsi_mpath_head);
> +}

[Severity: High]
Can ignoring the return value of scsi_mpath_get_head() cause a use-after-free
regression?

Looking at scsi_mpath_get_head(), it uses kref_get_unless_zero() which can
fail and return -ENXIO. If the reference count drops to zero concurrently
just before scsi_mpath_get_head() is called, the acquisition will fail.

By proceeding unconditionally when it fails, bus_for_each_drv() could
operate on an already freed scsi_mpath_head pointer.

Furthermore, scsi_mpath_put_head() would then invoke kref_put() on an
object that had a zero reference count, causing a double-free regression.

Should this code check the return value of scsi_mpath_get_head() and exit
early if the reference could not be acquired?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=21

Re: [PATCH v6 22/27] scsi: sd: add multipath disk class

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Empty `dev_release` callback `sd_mpath_disk_release` bypasses device/kobject memory management.
- [Medium] Struct `sd_mpath_disk` is incomplete and fails to manage the multipath gendisk as claimed.
--

--- Patch [22]: [PATCH v6 22/27] scsi: sd: add multipath disk class ---
commit f604e0ae8b45c7be3d202656fafcb45e6c81e7cf
Author: John Garry <john.g.garry@oracle.com>

scsi: sd: add multipath disk class

Add a new class, sd_mpath_disk_class, which is the multipath version of
the scsi_disk class. Structure sd_mpath_disk is introduced to manage the
multipath gendisk.

> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index 599e75f333343..242a15bc2c5bb 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
[ ... ]
> @@ -113,6 +114,39 @@ static mempool_t *sd_page_pool;
>  static mempool_t *sd_large_page_pool;
>  static atomic_t sd_large_page_pool_users = ATOMIC_INIT(0);
>  static struct lock_class_key sd_bio_compl_lkclass;
> +#ifdef CONFIG_SCSI_MULTIPATH
> +struct sd_mpath_disk {
> +	struct scsi_mpath_head		*scsi_mpath_head;
> +};

[Severity: Medium]
The commit message states that sd_mpath_disk is introduced to manage the
multipath gendisk. However, does this struct lack a struct gendisk pointer
and an embedded struct device? Without these, it doesn't appear capable of
fulfilling its documented purpose of managing the multipath disk.

> +
> +static void sd_mpath_disk_release(struct device *dev)
> +{
> +}

[Severity: High]
Is it safe to provide an empty dev_release callback here?

When a device is assigned to sd_mpath_disk_class and its reference count
reaches zero, this callback is invoked. Providing an empty release callback
circumvents proper resource cleanup, silencing kobject warnings while
guaranteeing that the enclosing structure is never actually freed.

Could this cause a memory leak of the device structure during teardown?

> +
> +static const struct class sd_mpath_disk_class = {
> +	.name = "scsi_mpath_disk",
> +	.dev_release = sd_mpath_disk_release,
> +};

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=22

Re: [PATCH v6 23/27] scsi: sd: add multipath disk attr groups

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] Result: Dead Code. The newly defined sysfs attribute groups for multipath disks are never assigned or registered, rendering the patch's functionality completely dead.
- [Medium] Result: Latent Type Confusion. The `sd_mpath_disk_attr_group` lacks an `is_visible` callback, exposing it to a fatal type-confusion or NULL dereference crash if mistakenly wired up to standard SCSI disks.
- [Medium] Result: Speculative Use-After-Free. Uncanceled delayed work caused by the `delayed_removal_secs` sysfs configuration risks a UAF on `mpath_head` teardown.
--

--- Patch [23]: [PATCH v6 23/27] scsi: sd: add multipath disk attr groups ---
commit 681fea1e5bfb430f8ec567b5c630bf2528c332b3
Author: John Garry <john.garry@linux.dev>

scsi: sd: add multipath disk attr groups

Set multipath disk attr groups, which includes delayed disk removal and
everything from mpath_attr_group.

> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index 242a15bc2c5bb..36c745cbb2619 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -4006,6 +4006,42 @@ static int sd_format_disk_name(char *prefix, int index, char *buf, int buflen)
>  	return 0;
>  }
>  
> +static ssize_t sd_mpath_device_delayed_removal_store(struct device *dev,
> +		struct device_attribute *attr, const char *buf, size_t count)
> +{
> +	struct mpath_head *mpath_head = mpath_bd_device_to_head(dev);

[Severity: Medium]
Does the teardown path in mpath_head_uninit() contain an explicit
cancel_delayed_work_sync(&mpath_head->remove_work)? If a delayed work is
scheduled via this sysfs store function, could it race with mpath_head
teardown and trigger a use-after-free if the work isn't canceled?

> +
> +	return mpath_delayed_removal_secs_store(mpath_head, buf, count);
> +}
> +
> +static ssize_t sd_mpath_device_delayed_removal_show(struct device *dev,
> +		struct device_attribute *attr, char *buf)
> +{
> +	struct mpath_head *mpath_head = mpath_bd_device_to_head(dev);
> +
> +	return mpath_delayed_removal_secs_show(mpath_head, buf);
> +}
> +
> +static DEVICE_ATTR(delayed_removal_secs, S_IRUGO | S_IWUSR,
> +		sd_mpath_device_delayed_removal_show,
> +		sd_mpath_device_delayed_removal_store);
> +
> +static struct attribute *sd_mpath_disk_attrs[] = {
> +	&dev_attr_delayed_removal_secs.attr,
> +	NULL
> +};
> +
> +static const struct attribute_group sd_mpath_disk_attr_group = {

[Severity: Medium]
Should this group include an is_visible() callback? Without one, if this
attribute group is ever mistakenly attached to generic, non-multipath SCSI
disks, accessing the attribute would trigger mpath_bd_device_to_head() on
an invalid device pointer, potentially causing a type confusion or NULL
dereference crash.

> +	.attrs		= sd_mpath_disk_attrs,
> +};
> +
> +__maybe_unused
> +static const struct attribute_group *sd_mpath_disk_attr_groups[] = {

[Severity: Medium]
Are these attribute groups actually registered anywhere? The array is defined
as __maybe_unused, but it doesn't appear to be hooked up to any device
class (e.g., sd_mpath_disk_class.dev_groups) or passed to device_add_disk()
in this patch. Will these attributes ever be visible to userspace?

> +	&sd_mpath_disk_attr_group,
> +	&mpath_attr_group,
> +	NULL
> +};

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=23

Re: [PATCH v6 19/27] scsi-multipath: provide callbacks for path state

[John Garry]

On 29/07/2026 16:43, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 8 potential issue(s) to consider:
> 
> New issues:
> - [High] `scsi_mpath_is_disabled()` returns true for STANDBY paths, preventing I/O and deadlocking implicit ALUA failovers.
> - [High] Bios placed on the `requeue_list` during an ALUA state change are permanently stalled due to missing notification hooks.
> - [High] ALUA path prioritization is broken because routing policies check a static struct field instead of the new `.is_optimized` callback.
> - [High] `scsi_mpath_available_path` ignores ALUA offline state, causing indefinite I/O hangs instead of failing.
> - [High] LBA-dependent paths are incorrectly flagged as disabled, preventing I/O.
> - [High] SDEV_CREATED state is omitted from available paths, causing premature I/O failures.
> - [High] I/O submitted during device quiesce (e.g., system suspend) will hang indefinitely because the requeue list is never flushed upon resume.
> 
> Pre-existing issues:
> - [High] Missing cancellation of background `requeue_work` task before tearing down and freeing the `scsi_mpath_head` object.
> --
> 
> --- Patch [19]: [PATCH v6 19/27] scsi-multipath: provide callbacks for path state ---
> commit a79ca9463c83fa5a5480f800bdc6d5f3fef27829
> Author: John Garry <john.g.garry@oracle.com>
> 
> scsi-multipath: provide callbacks for path state
> 
> This commit provides callbacks for .is_disabled, .is_optimized, and
> .available_path. These callbacks use the scsi_device's sdev_state
> and access_state, which will be driven by ALUA.
> 
>> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
>> --- a/drivers/scsi/scsi_multipath.c
>> +++ b/drivers/scsi/scsi_multipath.c
> 
> [ ... ]
> 
>> +static bool scsi_mpath_is_disabled(struct mpath_device *mpath_device)
>> +{
> 
> [ ... ]
> 
>> +	if (access_state == SCSI_ACCESS_STATE_OPTIMAL ||
>> +	    access_state == SCSI_ACCESS_STATE_ACTIVE)
>> +		return false;
>> +
>> +	return true;
>> +}
> 
> [Severity: High]
> Will this return true for STANDBY paths?
> 
> If implicit ALUA relies on sending I/O to a STANDBY path to transition it
> to ACTIVE via a Unit Attention, marking it disabled might prevent the core
> from routing any I/O to it, potentially deadlocking the failover.

ALUA support to be improved

> 
> [Severity: High]
> Does this also need to allow SCSI_ACCESS_STATE_LBA?

As above

> 
> SPC-4 defines LBA-dependent states as active and capable of processing I/O
> commands. Should this state also be permitted here to avoid stalling I/O
> on storage arrays that rely on it?
> 
>> +static bool scsi_mpath_available_path(struct mpath_device *mpath_device)
>> +{
> 
> [ ... ]
> 
>> +	if (sdev_state == SDEV_RUNNING || sdev_state == SDEV_QUIESCE ||
>> +	    sdev_state == SDEV_BLOCK || sdev_state == SDEV_CREATED_BLOCK)
>> +		return true;
>> +
>> +	return false;
>> +}
> 
> [Severity: High]
> Should SDEV_CREATED be included here?
> 
> During SCSI device probing, the device state is SDEV_CREATED and the core
> SCSI layer permits I/O operations like INQUIRY. If SDEV_CREATED is omitted
> here, could valid I/O be incorrectly failed as unavailable during device
> scanning?

Not relevant to multipath

> 
> [Severity: High]
> By allowing SDEV_QUIESCE here, bios will be placed on the requeue list.
> 
> When the device is later unquiesced during a system resume,
> scsi_device_resume() sets the state back to SDEV_RUNNING, but it does not
> seem to trigger scsi_mpath_revalidate_paths() or schedule requeue_work.

No bios should be queued when quiesed

> 
> Will bios queued during suspend remain stuck on the list indefinitely?
> 
> [Severity: High]
> This function only checks sdev_state.
> 
> If an ALUA path transitions to SCSI_ACCESS_STATE_OFFLINE, sdev_state remains
> SDEV_RUNNING, which causes this callback to return true.

ALUA support incomplete

> 
> Could this cause the block layer to endlessly requeue the I/O, believing
> the path is temporarily unavailable, rather than failing it immediately
> for being permanently offline?
> 
>>   static struct mpath_head_template smpdt = {
>> +	.is_disabled = scsi_mpath_is_disabled,
>> +	.is_optimized = scsi_mpath_is_optimized,
>> +	.available_path = scsi_mpath_available_path,
>>   	.clone_bio = scsi_mpath_clone_bio,
>>   };
> 
> [Severity: High]
> How does this integration handle ALUA state changes for requeued I/O?
> 
> When no active paths are found, bios are placed on the requeue list:
> 
> lib/multipath.c:mpath_bdev_submit_bio()
>      spin_lock_irq(&mpath_head->requeue_lock);
>      bio_list_add(&mpath_head->requeue_list, bio);
>      spin_unlock_irq(&mpath_head->requeue_lock);
> 
> When scsi_dh_alua.c handles an ALUA state change, does it notify
> scsi-multipath to schedule the requeue work? If not, could these bios
> remain permanently stalled?

ALUA support incomplete

> 
> [Severity: High]
> Are the routing algorithms in lib/multipath.c actually using this new
> .is_optimized callback?
> 
> lib/multipath.c:__mpath_find_path()
>      switch(mpath_device->access_state) {
>      case MPATH_STATE_OPTIMIZED:
> 
> It appears the path finder checks the statically initialized access_state
> field instead of invoking the new callback. Will this cause all active
> paths to be incorrectly treated as optimized, bypassing path prioritization?

used elsewhere

Re: [PATCH v6 20/27] scsi-multipath: add scsi_mpath_{start,end}_request()

[John Garry]

On 29/07/2026 17:12, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Permanent leak of mpath_nr_active counter and master block device in-flight IO stats when a requeued multipath request fails with a hard error.
> --
> 
> --- Patch [20]: [PATCH v6 20/27] scsi-multipath: add scsi_mpath_{start,end}_request() ---
> commit 1a27bdf4453d197f11533d04e7997e7873652eb4
> Author: John Garry <john.garry@linux.dev>
> 
> scsi-multipath: add scsi_mpath_{start,end}_request()
> 
> Add scsi_mpath_{start,end}_request() to handle updating private multipath
> request data, like nvme_mpath_{start,end}_request().
> 
> New member Scsi_Host.mpath_nr_active is added. This is required for queue
> depth multipath iopolicy.
> 
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index 6a2d7dbfbe8c3..ce9a2c7fbe707 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
> [ ... ]
>> @@ -1917,6 +1920,9 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
>>   	memset(cmd->sense_buffer, 0, SCSI_SENSE_BUFFERSIZE);
>>   	cmd->submitter = SUBMITTED_BY_BLOCK_LAYER;
>>   
>> +	if (sdev->scsi_mpath_dev && is_mpath_request(req))
>> +		scsi_mpath_start_request(req);
>> +
>>   	blk_mq_start_request(req);
>>   	if (blk_mq_is_reserved_rq(req)) {
>>   		reason = shost->hostt->queue_reserved_command(shost, cmd);
> 
> [Severity: High]
> Can this permanently leak the mpath_nr_active counter and master block device
> in-flight IO stats when a requeued multipath request fails with a hard error?
> 
> When a multipath request is dispatched, scsi_mpath_start_request() increments
> mpath_nr_active and sets SCMD_MPATH_CNT_ACTIVE. If the dispatch then fails
> with BLK_STS_RESOURCE (e.g. due to a full queue), the request is requeued and
> the multipath flags are preserved by SCMD_PRESERVED_FLAGS.
> 
> On a subsequent retry, if the device has transitioned to an offline state
> (which is common during path failure), the early state check in
> scsi_queue_rq() returns an error before scsi_mpath_start_request() is
> reached:
> 
> 	if (unlikely(sdev->sdev_state != SDEV_RUNNING)) {
> 		ret = scsi_device_state_check(sdev, req);
> 		if (ret != BLK_STS_OK)
> 			goto out_put_budget;
> 	}
> 
> Returning an error here causes the block layer to directly complete the
> request, bypassing the driver's normal scsi_end_request() completion path.
> The block layer will eventually call the scsi_cleanup_rq() hook, which merely
> zeroes cmd->flags:
> 
> 	cmd->flags = 0;
> 
> but fails to decrement mpath_nr_active or call bdev_end_io_acct().
> 
> Would this leave the queue-depth multipath I/O policy perceiving an
> artificially inflated load on the controller, permanently degrading load
> balancing?

Can be improved later

> 

Re: [PATCH v6 21/27] scsi-multipath: add delayed disk removal support

[John Garry]

On 29/07/2026 17:26, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] The unchecked return value of `scsi_mpath_get_head()` in `scsi_mpath_remove_head_work()` can lead to a Use-After-Free (UAF) and double-free.
> --
> 
> --- Patch [21]: [PATCH v6 21/27] scsi-multipath: add delayed disk removal support ---
> commit 10f5e133107a91a4b4df48e21559fe51a6b82a5e
> Author: John Garry <john.g.garry@oracle.com>
> 
> scsi-multipath: add delayed disk removal support
> 
> This commit adds core code support for delayed disk removal, introducing a
> worker function that iterates over the scsi_bus_type to invoke a newly
> added mpath_remove_head callback. It also attempts to manage reference
> counting for scsi_mpath_head during this removal process.
> 
>> diff --git a/drivers/scsi/scsi_multipath.c b/drivers/scsi/scsi_multipath.c
>> index 0adc5adb4b234..a42e20d33e422 100644
>> --- a/drivers/scsi/scsi_multipath.c
>> +++ b/drivers/scsi/scsi_multipath.c
> [ ... ]
>> @@ -367,7 +367,34 @@ static bool scsi_mpath_available_path(struct mpath_device *mpath_device)
>> +static void scsi_mpath_remove_head_work(struct mpath_head *mpath_head)
>> +{
>> +	struct scsi_mpath_head *scsi_mpath_head = to_scsi_mpath_head(mpath_head);
>> +
>> +	/*
>> +	 * scsi_mpath_remove_head_drv() could drop the final reference to
>> +	 * scsi_mpath_head, meaning that we pass a free'd pointer further
>> +	 * drivers - take a reference so that won't happen.
>> +	 */
>> +	scsi_mpath_get_head(scsi_mpath_head);
>> +	bus_for_each_drv(&scsi_bus_type, NULL, scsi_mpath_head,
>> +		scsi_mpath_remove_head_drv);
>> +	scsi_mpath_put_head(scsi_mpath_head);
>> +}
> 
> [Severity: High]
> Can ignoring the return value of scsi_mpath_get_head() cause a use-after-free
> regression?

It should not fail as we already must have a reference

> 
> Looking at scsi_mpath_get_head(), it uses kref_get_unless_zero() which can
> fail and return -ENXIO. If the reference count drops to zero concurrently
> just before scsi_mpath_get_head() is called, the acquisition will fail.
> 
> By proceeding unconditionally when it fails, bus_for_each_drv() could
> operate on an already freed scsi_mpath_head pointer.
> 
> Furthermore, scsi_mpath_put_head() would then invoke kref_put() on an
> object that had a zero reference count, causing a double-free regression.
> 
> Should this code check the return value of scsi_mpath_get_head() and exit
> early if the reference could not be acquired?
> 

Re: [PATCH v6 22/27] scsi: sd: add multipath disk class

[John Garry]

On 29/07/2026 17:33, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [High] Empty `dev_release` callback `sd_mpath_disk_release` bypasses device/kobject memory management.
> - [Medium] Struct `sd_mpath_disk` is incomplete and fails to manage the multipath gendisk as claimed.
> --
> 
> --- Patch [22]: [PATCH v6 22/27] scsi: sd: add multipath disk class ---
> commit f604e0ae8b45c7be3d202656fafcb45e6c81e7cf
> Author: John Garry <john.g.garry@oracle.com>
> 
> scsi: sd: add multipath disk class
> 
> Add a new class, sd_mpath_disk_class, which is the multipath version of
> the scsi_disk class. Structure sd_mpath_disk is introduced to manage the
> multipath gendisk.
> 
>> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
>> index 599e75f333343..242a15bc2c5bb 100644
>> --- a/drivers/scsi/sd.c
>> +++ b/drivers/scsi/sd.c
> [ ... ]
>> @@ -113,6 +114,39 @@ static mempool_t *sd_page_pool;
>>   static mempool_t *sd_large_page_pool;
>>   static atomic_t sd_large_page_pool_users = ATOMIC_INIT(0);
>>   static struct lock_class_key sd_bio_compl_lkclass;
>> +#ifdef CONFIG_SCSI_MULTIPATH
>> +struct sd_mpath_disk {
>> +	struct scsi_mpath_head		*scsi_mpath_head;
>> +};
> 
> [Severity: Medium]
> The commit message states that sd_mpath_disk is introduced to manage the
> multipath gendisk. However, does this struct lack a struct gendisk pointer
> and an embedded struct device? Without these, it doesn't appear capable of
> fulfilling its documented purpose of managing the multipath disk.

This is added later

> 
>> +
>> +static void sd_mpath_disk_release(struct device *dev)
>> +{
>> +}
> 
> [Severity: High]
> Is it safe to provide an empty dev_release callback here?

This is added later

> 
> When a device is assigned to sd_mpath_disk_class and its reference count
> reaches zero, this callback is invoked. Providing an empty release callback
> circumvents proper resource cleanup, silencing kobject warnings while
> guaranteeing that the enclosing structure is never actually freed.
> 
> Could this cause a memory leak of the device structure during teardown?
> 
>> +
>> +static const struct class sd_mpath_disk_class = {
>> +	.name = "scsi_mpath_disk",
>> +	.dev_release = sd_mpath_disk_release,
>> +};
> 

Re: [PATCH v6 24/27] scsi: sd: support multipath disk

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

New issues:
- [High] Missing `put_device()` in the `sd_mpath_add_disk()` error path permanently leaks the `sdkp` structure.
- [High] Inverted device teardown order causes double `device_del()` and sysfs warnings.
- [High] Cyclic kobject reference between `sd_mpath_disk` and its child gendisk guarantees both are permanently leaked on removal.
- [Medium] Ignored return value of `scsi_mpath_get_head()` leads to potential kref underflow and use-after-free.

Pre-existing issues:
- [High] `kfree()` is called on an active kobject when `sd_large_pool_create()` fails, leading to UAF / Sysfs Corruption.
--

--- Patch [24]: [PATCH v6 24/27] scsi: sd: support multipath disk ---
commit 1896d58c043437e813525b463ccbb2249c843f33
Author: John Garry <john.garry@linux.dev>
scsi: sd: support multipath disk

This commit adds support for attaching a multipath disk in the SCSI sd
subsystem. It allocates gendisks per-path but marks them as hidden, and
maintains a global list of sd_mpath_disks to match with scsi_devices.

> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -115,12 +115,30 @@ static mempool_t *sd_large_page_pool;
>  static atomic_t sd_large_page_pool_users = ATOMIC_INIT(0);
>  static struct lock_class_key sd_bio_compl_lkclass;
>  #ifdef CONFIG_SCSI_MULTIPATH
> +static LIST_HEAD(sd_mpath_disks_list);
> +static DEFINE_MUTEX(sd_mpath_disks_lock);
> +
>  struct sd_mpath_disk {
> +	struct device			dev;
> +	int				disk_index;
> +	int				disk_count;
> +	struct list_head		entry;
>  	struct scsi_mpath_head		*scsi_mpath_head;
>  };
>  
>  static void sd_mpath_disk_release(struct device *dev)
>  {
> +	struct sd_mpath_disk *sd_mpath_disk =
> +		container_of(dev, struct sd_mpath_disk, dev);
> +	struct scsi_mpath_head *scsi_mpath_head =
> +		sd_mpath_disk->scsi_mpath_head;
> +	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
> +
> +	mpath_put_disk(mpath_head);

[Severity: High]
Does this create an unbreakable cyclic kobject reference? The child disk is
created as a child of sd_mpath_disk->dev, meaning it holds a reference to
the parent. If the child's reference is only dropped via mpath_put_disk()
inside the parent's release function (sd_mpath_disk_release), the parent's
release function will never execute because the child still holds a
reference to it. Will this permanently leak both the sd_mpath_disk and the
multipath gendisk on removal?

[ ... ]

> @@ -4035,13 +4138,259 @@ static const struct attribute_group sd_mpath_disk_attr_group = {
>  	.attrs		= sd_mpath_disk_attrs,
>  };
>  
> +static int sd_mpath_probe(struct scsi_disk *sdkp)
> +{
[ ... ]
> +	device_initialize(&sd_mpath_disk->dev);
> +	sd_mpath_disk->dev.class = &sd_mpath_disk_class;
> +
> +	/* undone in sd_mpath_disk_release() */
> +	scsi_mpath_get_head(scsi_mpath_head);

[Severity: Medium]
Is it safe to ignore the return value here? In sd_mpath_probe(),
scsi_mpath_get_head() is called but its return value is discarded. Since
scsi_mpath_get_head() explicitly uses kref_get_unless_zero() and can return
-ENXIO, could a failure lead to a refcount underflow and a use-after-free
during the release path?

[ ... ]

> +static void sd_mpath_remove(struct scsi_disk *sdkp)
> +{
[ ... ]
> +	if (remove) {
> +		device_del(&sd_mpath_disk->dev);
> +		mpath_remove_disk(mpath_head);
> +	}
> +	sd_mpath_put_disk(sd_mpath_disk);
> +}

[Severity: High]
Is this device teardown order inverted? In sd_mpath_remove() and the related
sd_mpath_remove_head(), the parent device (sd_mpath_disk->dev) is deleted
via device_del() before the child gendisk is removed via mpath_remove_disk().
Does this cause double device_del() operations or sysfs hierarchy corruption,
since the device model requires children to be unregistered before their
parents?

[ ... ]

> +static void sd_mpath_fail_probe(struct scsi_disk *sdkp)
> +{
[ ... ]
> +	if (remove) {
> +		device_del(&sd_mpath_disk->dev);
> +		mpath_remove_disk(mpath_head);
> +	}
> +	sd_mpath_put_disk(sd_mpath_disk);
> +}

[Severity: High]
Does the same inverted teardown order apply here in sd_mpath_fail_probe()?
Deleting the parent device before the child gendisk violates the device
model requirements and can trigger sysfs warnings.

[ ... ]

> @@ -4158,11 +4515,18 @@ static int sd_probe(struct scsi_device *sdp)
>  	sd_revalidate_disk(gd);
>  	if (sdp->sector_size > PAGE_SIZE) {
>  		if (sd_large_pool_create()) {
> +			sd_mpath_fail_probe(sdkp);
>  			error = -ENOMEM;
>  			goto out_free_index;
>  		}
>  	}

[Severity: High]
This is a pre-existing issue, but does jumping to out_free_index here leak
the active kobject? In sd_probe(), device_add(&sdkp->disk_dev) has already
succeeded earlier in the function. When sd_large_pool_create() fails, it
jumps to out_free_index and eventually calls kfree(sdkp). Should this error
path use device_del() and put_device() instead to prevent sysfs corruption
and a use-after-free on the active kobject?

[ ... ]

> @@ -4177,6 +4541,7 @@ static int sd_probe(struct scsi_device *sdp)
[ ... ]
>  	error = device_add_disk(dev, gd, NULL);
>  	if (error) {
> +		sd_mpath_fail_probe(sdkp);
>  		device_unregister(&sdkp->disk_dev);
>  		put_disk(gd);
>  		if (sdp->sector_size > PAGE_SIZE)
> @@ -4184,6 +4549,19 @@ static int sd_probe(struct scsi_device *sdp)
>  		goto out;
>  	}
>  
> +	if (sdp->scsi_mpath_dev) {
> +		error = sd_mpath_add_disk(sdkp);
> +		if (error) {
> +			sd_mpath_fail_probe(sdkp);
> +			device_del(&sdkp->disk_dev);
> +			del_gendisk(sdkp->disk);
> +
> +			put_disk(sdkp->disk);
> +			if (sdp->sector_size > PAGE_SIZE)
> +				sd_large_pool_destroy();
> +			goto out;
> +		}
> +	}

[Severity: High]
Does this error path leak the sdkp structure? In sd_probe(), if
sd_mpath_add_disk() fails, the code manually calls device_del() and
del_gendisk() but entirely omits put_device(&sdkp->disk_dev). Because the
initial reference count set by device_initialize() is never decremented to
zero, will this permanently leak the scsi_disk structure and its associated
resources?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260729105107.255712-1-john.g.garry@oracle.com?part=24

Re: [PATCH v6 23/27] scsi: sd: add multipath disk attr groups

[John Garry]

On 29/07/2026 17:48, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> - [Medium] Result: Dead Code. The newly defined sysfs attribute groups for multipath disks are never assigned or registered, rendering the patch's functionality completely dead.
> - [Medium] Result: Latent Type Confusion. The `sd_mpath_disk_attr_group` lacks an `is_visible` callback, exposing it to a fatal type-confusion or NULL dereference crash if mistakenly wired up to standard SCSI disks.
> - [Medium] Result: Speculative Use-After-Free. Uncanceled delayed work caused by the `delayed_removal_secs` sysfs configuration risks a UAF on `mpath_head` teardown.
> --
> 
> --- Patch [23]: [PATCH v6 23/27] scsi: sd: add multipath disk attr groups ---
> commit 681fea1e5bfb430f8ec567b5c630bf2528c332b3
> Author: John Garry <john.garry@linux.dev>
> 
> scsi: sd: add multipath disk attr groups
> 
> Set multipath disk attr groups, which includes delayed disk removal and
> everything from mpath_attr_group.
> 
>> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
>> index 242a15bc2c5bb..36c745cbb2619 100644
>> --- a/drivers/scsi/sd.c
>> +++ b/drivers/scsi/sd.c
>> @@ -4006,6 +4006,42 @@ static int sd_format_disk_name(char *prefix, int index, char *buf, int buflen)
>>   	return 0;
>>   }
>>   
>> +static ssize_t sd_mpath_device_delayed_removal_store(struct device *dev,
>> +		struct device_attribute *attr, const char *buf, size_t count)
>> +{
>> +	struct mpath_head *mpath_head = mpath_bd_device_to_head(dev);
> 
> [Severity: Medium]
> Does the teardown path in mpath_head_uninit() contain an explicit
> cancel_delayed_work_sync(&mpath_head->remove_work)? If a delayed work is
> scheduled via this sysfs store function, could it race with mpath_head
> teardown and trigger a use-after-free if the work isn't canceled?

delayed work is not scheduled via this sysfs store function

> 
>> +
>> +	return mpath_delayed_removal_secs_store(mpath_head, buf, count);
>> +}
>> +
>> +static ssize_t sd_mpath_device_delayed_removal_show(struct device *dev,
>> +		struct device_attribute *attr, char *buf)
>> +{
>> +	struct mpath_head *mpath_head = mpath_bd_device_to_head(dev);
>> +
>> +	return mpath_delayed_removal_secs_show(mpath_head, buf);
>> +}
>> +
>> +static DEVICE_ATTR(delayed_removal_secs, S_IRUGO | S_IWUSR,
>> +		sd_mpath_device_delayed_removal_show,
>> +		sd_mpath_device_delayed_removal_store);
>> +
>> +static struct attribute *sd_mpath_disk_attrs[] = {
>> +	&dev_attr_delayed_removal_secs.attr,
>> +	NULL
>> +};
>> +
>> +static const struct attribute_group sd_mpath_disk_attr_group = {
> 
> [Severity: Medium]
> Should this group include an is_visible() callback? Without one, if this
> attribute group is ever mistakenly attached to generic,

it's not used elsewhere, so I won't add a pointless is_visible callback

> non-multipath SCSI
> disks, accessing the attribute would trigger mpath_bd_device_to_head() on
> an invalid device pointer, potentially causing a type confusion or NULL
> dereference crash.
> 
>> +	.attrs		= sd_mpath_disk_attrs,
>> +};
>> +
>> +__maybe_unused
>> +static const struct attribute_group *sd_mpath_disk_attr_groups[] = {
> 
> [Severity: Medium]
> Are these attribute groups actually registered anywhere? 

They are used later

> The array is defined
> as __maybe_unused, but it doesn't appear to be hooked up to any device
> class (e.g., sd_mpath_disk_class.dev_groups) or passed to device_add_disk()
> in this patch. Will these attributes ever be visible to userspace?
> 
>> +	&sd_mpath_disk_attr_group,
>> +	&mpath_attr_group,
>> +	NULL
>> +};
> 

Re: [PATCH v6 24/27] scsi: sd: support multipath disk

[John Garry]

On 29/07/2026 18:01, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
> 
> New issues:
> - [High] Missing `put_device()` in the `sd_mpath_add_disk()` error path permanently leaks the `sdkp` structure.
> - [High] Inverted device teardown order causes double `device_del()` and sysfs warnings.
> - [High] Cyclic kobject reference between `sd_mpath_disk` and its child gendisk guarantees both are permanently leaked on removal.
> - [Medium] Ignored return value of `scsi_mpath_get_head()` leads to potential kref underflow and use-after-free.
> 
> Pre-existing issues:
> - [High] `kfree()` is called on an active kobject when `sd_large_pool_create()` fails, leading to UAF / Sysfs Corruption.

That needs to be fixed in mkp-scsi tree

> --
> 
> --- Patch [24]: [PATCH v6 24/27] scsi: sd: support multipath disk ---
> commit 1896d58c043437e813525b463ccbb2249c843f33
> Author: John Garry <john.garry@linux.dev>
> scsi: sd: support multipath disk
> 
> This commit adds support for attaching a multipath disk in the SCSI sd
> subsystem. It allocates gendisks per-path but marks them as hidden, and
> maintains a global list of sd_mpath_disks to match with scsi_devices.
> 
>> --- a/drivers/scsi/sd.c
>> +++ b/drivers/scsi/sd.c
>> @@ -115,12 +115,30 @@ static mempool_t *sd_large_page_pool;
>>   static atomic_t sd_large_page_pool_users = ATOMIC_INIT(0);
>>   static struct lock_class_key sd_bio_compl_lkclass;
>>   #ifdef CONFIG_SCSI_MULTIPATH
>> +static LIST_HEAD(sd_mpath_disks_list);
>> +static DEFINE_MUTEX(sd_mpath_disks_lock);
>> +
>>   struct sd_mpath_disk {
>> +	struct device			dev;
>> +	int				disk_index;
>> +	int				disk_count;
>> +	struct list_head		entry;
>>   	struct scsi_mpath_head		*scsi_mpath_head;
>>   };
>>   
>>   static void sd_mpath_disk_release(struct device *dev)
>>   {
>> +	struct sd_mpath_disk *sd_mpath_disk =
>> +		container_of(dev, struct sd_mpath_disk, dev);
>> +	struct scsi_mpath_head *scsi_mpath_head =
>> +		sd_mpath_disk->scsi_mpath_head;
>> +	struct mpath_head *mpath_head = &scsi_mpath_head->mpath_head;
>> +
>> +	mpath_put_disk(mpath_head);
> 
> [Severity: High]
> Does this create an unbreakable cyclic kobject reference? The child disk is
> created as a child of sd_mpath_disk->dev, meaning it holds a reference to
> the parent. If the child's reference is only dropped via mpath_put_disk()
> inside the parent's release function (sd_mpath_disk_release), the parent's
> release function will never execute because the child still holds a
> reference to it. Will this permanently leak both the sd_mpath_disk and the
> multipath gendisk on removal?

I think that the earlier del_gendisk() -> device_del(disk_to_dev(disk)) 
drops that reference

> 
> [ ... ]
> 
>> @@ -4035,13 +4138,259 @@ static const struct attribute_group sd_mpath_disk_attr_group = {
>>   	.attrs		= sd_mpath_disk_attrs,
>>   };
>>   
>> +static int sd_mpath_probe(struct scsi_disk *sdkp)
>> +{
> [ ... ]
>> +	device_initialize(&sd_mpath_disk->dev);
>> +	sd_mpath_disk->dev.class = &sd_mpath_disk_class;
>> +
>> +	/* undone in sd_mpath_disk_release() */
>> +	scsi_mpath_get_head(scsi_mpath_head);
> 
> [Severity: Medium]
> Is it safe to ignore the return value here? In sd_mpath_probe(),

the count must be non-zero here

later I can add try get variant to make this clear

> scsi_mpath_get_head() is called but its return value is discarded. Since
> scsi_mpath_get_head() explicitly uses kref_get_unless_zero() and can return
> -ENXIO, could a failure lead to a refcount underflow and a use-after-free
> during the release path?
> 
> [ ... ]
> 
>> +static void sd_mpath_remove(struct scsi_disk *sdkp)
>> +{
> [ ... ]
>> +	if (remove) {
>> +		device_del(&sd_mpath_disk->dev);
>> +		mpath_remove_disk(mpath_head);
>> +	}
>> +	sd_mpath_put_disk(sd_mpath_disk);
>> +}
> 
> [Severity: High]
> Is this device teardown order inverted? In sd_mpath_remove() and the related
> sd_mpath_remove_head(), the parent device (sd_mpath_disk->dev) is deleted
> via device_del() before the child gendisk is removed via mpath_remove_disk().
> Does this cause double device_del() operations or sysfs hierarchy corruption,
> since the device model requires children to be unregistered before their
> parents?

maybe it's a problem, but I did not experience an issue. There is a 
later put on the &sd_mpath_disk->dev in the sd_mpath_put_disk() call, so 
the parent is not gone before disk

> 
> [ ... ]
> 
>> +static void sd_mpath_fail_probe(struct scsi_disk *sdkp)
>> +{
> [ ... ]
>> +	if (remove) {
>> +		device_del(&sd_mpath_disk->dev);
>> +		mpath_remove_disk(mpath_head);
>> +	}
>> +	sd_mpath_put_disk(sd_mpath_disk);
>> +}
> 
> [Severity: High]
> Does the same inverted teardown order apply here in sd_mpath_fail_probe()?
> Deleting the parent device before the child gendisk violates the device
> model requirements and can trigger sysfs warnings.

Same answer as above

> 
> [ ... ]
> 
>> @@ -4158,11 +4515,18 @@ static int sd_probe(struct scsi_device *sdp)
>>   	sd_revalidate_disk(gd);
>>   	if (sdp->sector_size > PAGE_SIZE) {
>>   		if (sd_large_pool_create()) {
>> +			sd_mpath_fail_probe(sdkp);
>>   			error = -ENOMEM;
>>   			goto out_free_index;
>>   		}
>>   	}
> 
> [Severity: High]
> This is a pre-existing issue,


Yes, mkp-scsi tree needs fixing


> but does jumping to out_free_index here leak
> the active kobject? In sd_probe(), device_add(&sdkp->disk_dev) has already
> succeeded earlier in the function. When sd_large_pool_create() fails, it
> jumps to out_free_index and eventually calls kfree(sdkp). Should this error
> path use device_del() and put_device() instead to prevent sysfs corruption
> and a use-after-free on the active kobject?
> 
> [ ... ]
> 
>> @@ -4177,6 +4541,7 @@ static int sd_probe(struct scsi_device *sdp)
> [ ... ]
>>   	error = device_add_disk(dev, gd, NULL);
>>   	if (error) {
>> +		sd_mpath_fail_probe(sdkp);
>>   		device_unregister(&sdkp->disk_dev);
>>   		put_disk(gd);
>>   		if (sdp->sector_size > PAGE_SIZE)
>> @@ -4184,6 +4549,19 @@ static int sd_probe(struct scsi_device *sdp)
>>   		goto out;
>>   	}
>>   
>> +	if (sdp->scsi_mpath_dev) {
>> +		error = sd_mpath_add_disk(sdkp);
>> +		if (error) {
>> +			sd_mpath_fail_probe(sdkp);
>> +			device_del(&sdkp->disk_dev);
>> +			del_gendisk(sdkp->disk);
>> +
>> +			put_disk(sdkp->disk);
>> +			if (sdp->sector_size > PAGE_SIZE)
>> +				sd_large_pool_destroy();
>> +			goto out;
>> +		}
>> +	}
> 
> [Severity: High]
> Does this error path leak the sdkp structure? In sd_probe(), if
> sd_mpath_add_disk() fails, the code manually calls device_del() and
> del_gendisk() but entirely omits put_device(&sdkp->disk_dev). 

let's fix mkp-scsi tree first

> Because the
> initial reference count set by device_initialize() is never decremented to
> zero, will this permanently leak the scsi_disk structure and its associated
> resources?
>