Re: [PATCHv5 RFC 0/3] Add visibility for native NVMe multipath using sysfs

[Nilay Shroff <nilay@linux.ibm.com>]

On 12/10/24 15:04, Daniel Wagner wrote:
> On Tue, Dec 10, 2024 at 12:33:45PM +0530, Nilay Shroff wrote:
>>>>> As we know, NVMe native multipath supports three different io policies
>>>>> (numa, round-robin and queue-depth) for selecting I/O path, however, we
>>>>> don't have any visibility about which path is being selected by multipath
>>>>> code for forwarding I/O. This RFC helps add that visibility by adding new
>>>>> sysfs attribute files named "numa_nodes" and "queue_depth" under each
>>>>> namespace block device path /sys/block/nvmeXcYnZ/. We also create a
>>>>> "multipath" sysfs directory under head disk node and then from this
>>>>> directory add a link to each namespace path device this head disk node
>>>>> points to.
>>>>>
>>>>> Please find below output generated with this proposed RFC patch applied on
>>>>> a system with two multi-controller PCIe NVMe disks attached to it. This
>>>>> system is also an NVMf-TCP host which is connected to an NVMf-TCP target
>>>>> over two NIC cards. This system has four numa nodes online when the below
>>>>> output was captured:
> 
> Looks good to me from libnvme's perspective. I just my question on the
> lifetime of the new link, but I might just miss the point.
> 
When a namepsace path device is removed, we also remove the corresponding 
link (if it exists) from the head disk node to the path device.Please refer
below function, from the first patch of the series, which is invoked from 
the nvme_ns_remove() for removing the sysfs link:

void nvme_mpath_remove_sysfs_link(struct nvme_ns *ns)
{
	struct device *target;
	struct kobject *kobj;

	if (!test_bit(NVME_NS_SYSFS_ATTR_LINK, &ns->flags))
		return;

	target = disk_to_dev(ns->disk);
	kobj = &disk_to_dev(ns->head->disk)->kobj;
	sysfs_remove_link_from_group(kobj, nvme_ns_mpath_attr_group.name,
			dev_name(target));
	clear_bit(NVME_NS_SYSFS_ATTR_LINK, &ns->flags);
}

Thanks,
--Nilay