Re: [PATCH 1/2] nvme: keep transport module referenced while head node is open

[Nilay Shroff <nilay@linux.ibm.com>]

On 9/2/26 7:01 PM, Christoph Hellwig wrote:
> On Mon, Aug 31, 2026 at 08:49:54PM +0530, Nilay Shroff wrote:
>> When a user opens an NVMe multipath head node, we take a reference to
>> the head node, but this does not prevent the transport module backing
>> its paths from being unloaded. This can result in the multipath head
>> remaining open while its underlying transport module is unloaded.
> 
> Which makes sense.  Different paths can use different transports, but
> even when all paths go away, the head can stick around.
> 
>> Fix this by taking a reference to the transport module for each active
>> path when the multipath head node is opened. Keep track of the number
>> of active head node openers so that dynamically added paths acquire the
>> same number of transport module references.
> 
> I'm not sure this is correct.  Unloading the layer below should be
> just fine.
> 
> What practical problem do you want to solve with this?
> 

The problem we are trying to solve is that the multipath head node can
remain open and be used by a filesystem even though the transport module
backing its paths can be unloaded.

For example, we have a shared namespace exposed through two PCIe paths:

nvme-subsys0
     nvme0 -> nvme0n1
     nvme1 -> nvme0n1

The namespace is formatted with ext4 and mounted:

# mount -t ext4 /dev/nvme0n1 /mnt/disk

At this point, the multipath head has a reference to nvme_core, but the
nvme transport module itself has a refcount of zero:

# lsmod | grep nvme
nvme                  262144  0
nvme_core             458752  2
nvme_keyring          262144  1
nvme_auth             262144  1

Consequently, the transport module can be unloaded:

# rmmod nvme

After this, the filesystem remains mounted, but I/O starts failing:

     EXT4-fs (...): shut down requested (2)
     Aborting journal on device nvme0n1-8.
     block nvme0n1: no available path - failing I/O
     Buffer I/O error on dev nvme0n1, logical block ..., lost sync page write
     JBD2: I/O error when updating journal superblock for nvme0n1-8.

This becomes particularly problematic if the multipath NVMe device is
used as the root filesystem. Once the transport module is unloaded, I/O
fails and we can no longer run commands to reload the NVMe module. In
our testing, the only recovery option in that situation has been to
power-cycle the system.

So the practical problem is that an open multipath head can continue to
be used after its underlying transport module has been unloaded. The
proposed change keeps the transport module referenced while the head
node is open, preventing the module from being unloaded in this
scenario.

Thanks,
--Nilay