Re: SPDK initiators (Vmware 7.x) can not connect to nvmet-rdma.

Mark Ruijter
2021-09-01T15:08:45Z

The device is a Mellanox ConnectX-6 controller.
Vmware can connect to an SPDK target started on the exact same Ubuntu target system.

4: enp129s0f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 4096 qdisc mq state UP group default qlen 25000
link/ether b8:ce:f6:92:b7:b6 brd ff:ff:ff:ff:ff:ff
inet 192.168.100.34/24 brd 192.168.100.255 scope global enp129s0f0np0
valid_lft forever preferred_lft forever

81:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
81:00.1 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]

--
# cd /sys/kernel/config/nvmet/ports/
# ls
11290 11291 21290 21291
# cd 11290
# cat *
ipv4
192.168.100.34
not specified
4420
rdma
--

﻿On 01/09/2021, 16:58, "Sagi Grimberg" <sagi@grimberg.me> wrote:

> Hi Sagi,
>
> I am using VMware 7.x as initiator with RDMA.
> The target system is running Ubuntu 20.04.3 LTS with kernel 5.11.22+.
>
> The device that is exported is an LVM volume, however I also tested with a file backed loop device.
> Connecting with SPDK seems to be the problem and as reported on the SPDK mailing-list it can be used to reproduce the issue when VMWare is not available.
> ./perf -q 64 -P 1 -s 4096 -w read -t 300 -c 0x1 -o 4096 -r 'trtype:RDMA adrfam:IPv4 traddr:169.254.85.8 trsvcid:4420'
>
> This seems to produce a similar result:
> nvme connect --transport=rdma --queue-size=1024 --nqn=testnqn_1 --traddr=169.254.85.8 --trsvcid=4420
>
> I hope this helps,

I meant which rdma device are you using? that device is failing the
qp creation.