Re: [PATCH 0/2] nvme: add reference counting for transport modules

[Nilay Shroff <nilay@linux.ibm.com>]

On 9/1/26 1:44 PM, John Garry wrote:
> On 8/31/26 16:19, Nilay Shroff wrote:
>> Hi,
>>
>> This patchset adds reference counting for NVMe transport modules while
>> the corresponding multipath head node is open. This prevents the
>> underlying transport module from being unloaded while it is still in
>> use by the multipath head node. Unloading a transport module while it
>> is still in use can result in undefined behavior.
>>
> 
> What is the undefined behaviour specifically?
> 
Yes, so far we have observed I/O errors when the transport module is
unloaded while a filesystem is mounted on the multipath NVMe disk. This
becomes particularly problematic if the root filesystem is on that
device: once the transport module is unloaded, I/O fails and we can no
longer run commands to reload the NVMe transport module. In our testing,
the only recovery option in that situation has been to power-cycle the
system.

> When the ctrl ops module ref counting was originally introduced, the commit message mentioned a crash which it solves.
> 
> So far for this problem we have seen a report that if we remove the module, the mounted FS will have IOs fail. The same can be experienced if the admin unbinds the device from the driver. However, I have not seen a mention of a crash, kernel data corruption, a hang, etc.
> 
Yes I have also not observed a kernel crash or data corruption.
Regarding unbinding, this is triggered through the PCI/device-model
unbind path. The driver unbind callback returns void so the NVMe driver
has no way to reject or otherwise prevent the unbind operation from the
driver side. The purpose of this patch is therefore to protect the transport
module from being unloaded while the multipath head is still open.

Thanks,
--Nilay