nvme: Duplicate cntlid error

nvme: Duplicate cntlid error

[Alex Talker]

Using Fedora 31 as a host I'm trying to connect to 2 nodes that host
subsystem with one and the same NQN.
So when I call:
 
nvme connect -t rdma -a 10.21.0.1 -n nqn.2020-03.com.example:subsystem0
 
device nvme0 successfully allocated but when then I call:
 
nvme connect -t rdma -a 10.20.0.1 -n nqn.2020-03.com.example:subsystem0
 
I get an error:
 
[239355.661527] nvme nvme1: Duplicate cntlid 1 with nvme0, rejecting
 
what could be the problem?
Few months before I checked connection to a single node via few ports
and it worked out just fine
but attempt of H.A. setup fails here.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: nvme: Duplicate cntlid error

[Sagi Grimberg]

> Using Fedora 31 as a host I'm trying to connect to 2 nodes that host
> subsystem with one and the same NQN.
> So when I call:
>   
> nvme connect -t rdma -a 10.21.0.1 -n nqn.2020-03.com.example:subsystem0
>   
> device nvme0 successfully allocated but when then I call:
>   
> nvme connect -t rdma -a 10.20.0.1 -n nqn.2020-03.com.example:subsystem0
>   
> I get an error:
>   
> [239355.661527] nvme nvme1: Duplicate cntlid 1 with nvme0, rejecting
>   
> what could be the problem?
> Few months before I checked connection to a single node via few ports
> and it worked out just fine
> but attempt of H.A. setup fails here.

Your subsystem present two controllers with the same controller id, that
is a violation of the spec, and we properly reject such controllers.

In upstream code, you can specify a cntlid min/max range such that if
you are trying to play games with different linux targets as a single
subsystem you won't violate the spec.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: nvme: Duplicate cntlid error

[Alex Talker]

3/31/2020 9:45 AM, Sagi Grimberg wrote:
>
>> Using Fedora 31 as a host I'm trying to connect to 2 nodes that host
>> subsystem with one and the same NQN.
>> So when I call:
>>   nvme connect -t rdma -a 10.21.0.1 -n
>> nqn.2020-03.com.example:subsystem0
>>   device nvme0 successfully allocated but when then I call:
>>   nvme connect -t rdma -a 10.20.0.1 -n
>> nqn.2020-03.com.example:subsystem0
>>   I get an error:
>>   [239355.661527] nvme nvme1: Duplicate cntlid 1 with nvme0, rejecting
>>   what could be the problem?
>> Few months before I checked connection to a single node via few ports
>> and it worked out just fine
>> but attempt of H.A. setup fails here.
>
> Your subsystem present two controllers with the same controller id, that
> is a violation of the spec, and we properly reject such controllers.
>
> In upstream code, you can specify a cntlid min/max range such that if
> you are trying to play games with different linux targets as a single
> subsystem you won't violate the spec.

To summarize problem is that cntlid is automatically allocated inside
the target driver

per-connection but since there's no synchronization between the nodes,
each node counts separately

and that's why the numbers may clash, right?

I've checked and the patch
https://github.com/torvalds/linux/commit/94a39d61f80fcd679debda11e1ca02b88d90e67e
is in fact available only upstream

but not in any MLNX OFED driver back-port. Does Mellanox has plans to
include this feature in next release?



_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: nvme: Duplicate cntlid error

[Christoph Hellwig]

On Tue, Mar 31, 2020 at 01:37:18PM +0300, Alex Talker wrote:
> but not in any MLNX OFED driver back-port. Does Mellanox has plans to
> include this feature in next release?

That is completely offtopic for this list.  Also please don't use vendor
forks full of broken code, your life will be painful and of no
interest for others.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme