Re: Unable to reconnect namespace via NVMe/TCP

[Nilay Shroff <nilay@linux.ibm.com>]

On 8/12/25 9:18 PM, Anton Gavriliuk wrote:
> Hi
> 
> There are NVMe/TCP target and initiator servers, both running on
> RHEL10 (6.12.0-55.25.1.el10_0.x86_64)
> 
> NVMe/TCP target exports single NVMe SSD
> 
>       "namespaces": [
>         {
>           "device": {
>             "nguid": "01000000-0000-0000-8ce3-8ee3064aa4f2",
>             "path": "/dev/nvme0n1"
>           },
>           "enable": 1,
>           "nsid": 1
>         }
>       ],
> 
> If NVMe/TCP target is not available, initiator tries to reconnect
> every 10 seconds
> 
> [ 2586.071048] nvme nvme9: failed to connect socket: -111
> [ 2586.071403] nvme nvme9: Failed reconnect attempt 16/-1
> [ 2586.071565] nvme nvme9: Reconnecting in 10 seconds...
> [ 2596.310921] nvme nvme9: failed to connect socket: -111
> [ 2596.311186] nvme nvme9: Failed reconnect attempt 17/-1
> [ 2596.311349] nvme nvme9: Reconnecting in 10 seconds...
> [ 2606.550772] nvme nvme9: failed to connect socket: -111
> [ 2606.551252] nvme nvme9: Failed reconnect attempt 18/-1
> [ 2606.551592] nvme nvme9: Reconnecting in 10 seconds...
> 
> when NVMe/TCP target become available, initiator failed reconnect the namespace
> 
> [ 2606.551592] nvme nvme9: Reconnecting in 10 seconds...
> [ 2616.793080] nvme nvme9: creating 16 I/O queues.
> [ 2616.829881] nvme nvme9: mapped 16/0/0 default/read/poll queues.
> [ 2616.833685] nvme nvme9: Successfully reconnected (attempt 19/-1)
> [ 2616.834446] nvme nvme9: identifiers changed for nsid 1

It seems after the initiator re-connected to the target, the namespace
identifiers (UUID/NGUID/EUI64 or CSI(command set identifiers)) have
changed for nsid 1. So host kernel removed this (nvme9n1) namespace.

> [ 2616.835618] block nvme9n1: no usable path - requeuing I/O
> [ 2616.856602] block nvme9n1: no available path - failing I/O
> [ 2616.856811] block nvme9n1: no available path - failing I/O
> 
> and there is no nvme9n1 namespace in the "nvme list" output.
> 
So you may check on the target about what could have possibly changed
during reconnect time window for nsid 1. 

As a side note, it’s generally best to reproduce and report issues with
an upstream kernel when posting to the kernel mailing list. This helps
ensure the report gets prompt attention and makes it easier for others
to debug and assist.

Thanks,
--Nilay