Re: [PATCH] nvme: remove multipath module parameter

[Nilay Shroff <nilay@linux.ibm.com>]

On 3/7/25 9:13 PM, Keith Busch wrote:
> On Fri, Mar 07, 2025 at 08:49:09PM +0530, Nilay Shroff wrote:
>> On 3/7/25 6:16 AM, Keith Busch wrote:
>> I think always creating multipath head node even for the disk which doesn't 
>> have CMIC/NMIC capability should be useful. That way, we may then be able 
>> to remove multipath module parameter? In fact, you already mentioned about
>> it in one of your previous message. I see two approaches (one of them you 
>> proposed and another one Christoph proposed: 
>> https://lore.kernel.org/linux-nvme/Y+1aKcQgbskA2tra@kbusch-mbp.dhcp.thefacebook.com/). 
>>
>> Maybe in first cut we should create multipath head disk node always for 
>> single/multi ported NVMe disk. Later we may enhance it and allow pinning the 
>> head node for hotplug events so that head node dev name remains consistent 
>> across disk add/remove hotplug events.
> 
> It honestly has potential to solve some real problems, like
> re-enumeration triggered by a link reset on an in-use drive. You'd
> currently need to close the old handle and open a new on, even though
> it's the same device. It may not even be possible to do that if that
> device contains your root partition, and then you can only power cycle.
> 
> The downside is we wouldn't get the short cut to blk_mq_submit_bio. We'd
> instead stack that atop an indirect call, so it's not free.
> 
Yes agreed however it seems advantages of using an indirect call outweighs 
using the short cut to blk_mq_submit_bio. Moreover it seems the cost of 
indirect call is trivial because we already cache the nexthop. 

I integrated your proposed patch (with few trivial additional changes on top) 
and I see that it's coming out nicely. I ran few tests and confirmed it's 
working well. However, in the proposed patch we *always* delay (~10 sec) the
removal of multipath head node. That means that even while removing the
nvme module (rmmod nvme) or if user delete/detache the namespace, we delay 
the removal of head node but that may not be what we want. So I'd suggest 
instead, delayed removal of multipath head not shall be configurable using a
sysfs attribute. With this attribute then we shall let user opt for pinning 
the head node (with optional delayed time as well?). And it's only when user 
shows the intent to pin the node we should delay its removal. This is what 
exactly (pinning of head node) Christoph's proposed patch implements. So I'd 
suggest a bit of amalgamation of yours as well as Christoph patch to implement
this change. 

If you and Christoph are busy with other work then in that case I'll be glad
to pursue this further if you agree.

Thanks,
--Nilay