From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============1835851477691795542==" MIME-Version: 1.0 From: Walker, Benjamin Subject: Re: [SPDK] ctrlr.c: 534:spdk_nvmf_ctrlr_connect: *ERROR*: Subsystem 'nqn.2016-06.io.spdk:cnode2' does not allow host '2014-08.org.nvmexpress:uuid:f7124f51-379b-4752-b195-0487d39d4543' Date: Fri, 26 Jul 2019 17:34:04 +0000 Message-ID: In-Reply-To: CAPhUAXEbeS1tYeZy3102bdpQQskPbHBC9RhoJb6PmoMLHS1u_w@mail.gmail.com List-ID: To: spdk@lists.01.org --===============1835851477691795542== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Fri, 2019-07-26 at 10:19 -0700, sushilkumar gangadharan wrote: > Hi > I am trying to setup a docker container which then talks to Name drive > using SPDK. In the first phase of the experimenting, I did the following > within the docker container and it seems to work: > = > 508 app/nvmf_tgt/nvmf_tgt > 509 scripts/rpc.py nvmf_create_transport -t TCP -u 16384 -p 8 -c 8192 > 510 scripts/rpc.py nvmf_create_transport -t TCP -u 16384 -p 8 -c 8192 Creating the tcp transport a second time doesn't fail? It should. You can o= nly have one of each type of transport. > 511 scripts/rpc.py construct_malloc_bdev -b Malloc0 512 512 > 512 scripts/rpc.py nvmf_subsystem_create nqn.2016-06.io.spdk:cnode1 -a > -s SPDK00000000000001 -d SPDK_Controller1 Note for later the use of the '-a' parameter when creating the subsystem, w= hich means allow any host to connect. > 513 scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk:cnode1 > Malloc0 > 514 scripts/rpc.py nvmf_subsystem_add_listener > nqn.2016-06.io.spdk:cnode1 -t TCP -a 172.20.0.2 -s 4420 > = > Was able to verify that Identify works fine: > sh-4.2# ./identify -r 'trtype:TCP adrfam:IPv4 traddr:172.20.0.2 > trsvcid:4420' > Starting SPDK v19.04.1 / DPDK 19.05.0 initialization... > [ DPDK EAL parameters: identify --no-shconf -c 0x1 -n 1 -m 0 --no-pci > --log-level=3Dlib.eal:6 --base-virtaddr=3D0x200000000000 --match-allocati= ons > --file-prefix=3Dspdk_pid498 ] > EAL: No available hugepages reported in hugepages-1048576kB > EAL: VFIO support initialized > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > NVMe over Fabrics controller at 172.20.0.2:4420: > nqn.2014-08.org.nvmexpress.discovery > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > Controller Capabilities/Features > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > Vendor ID: 0000 > Subsystem Vendor ID: 0000 > Serial Number: .................... > Model Number: > ........................................ > Firmware Version: 19.04.1 > Recommended Arb Burst: 0 > IEEE OUI Identifier: 00 00 00 > Multi-path I/O > May have multiple subsystem ports: No > May be connected to multiple hosts: No > Associated with SR-IOV VF: No > Max Data Transfer Size: 131072 > NVMe Specification Version (VS): 1.3 > NVMe Specification Version (Identify): 1.3 > Maximum Queue Entries: 128 > Contiguous Queues Required: Yes > Arbitration Mechanisms Supported > .... > = > Then I tried I/O : > sh-4.2# /usr/local/bin/spdk_nvme_perf -q 1 -o 4096 -w write -M 0 -t 10 -r > 'trtype:TCP traddr:172.20.0.2 adrfam:IPv4 trsvcid:4420 > subnqn:nqn.2016-06.io.spdk:cnode1' > Ignoring -M option... Please use -M option only when using rw or randrw. > Starting SPDK v19.04.1 / DPDK 19.05.0 initialization... > [ DPDK EAL parameters: perf --no-shconf -c 0x1 --no-pci > --log-level=3Dlib.eal:6 --base-virtaddr=3D0x200000000000 --match-allocati= ons > --file-prefix=3Dspdk_pid549 ] > EAL: No available hugepages reported in hugepages-1048576kB > EAL: VFIO support initialized > Initializing NVMe Controllers > Attaching to NVMe over Fabrics controller at 172.20.0.2:4420: > nqn.2016-06.io.spdk:cnode1 > Attached to NVMe over Fabrics controller at 172.20.0.2:4420: > nqn.2016-06.io.spdk:cnode1 > Associating SPDK_Controller1 (SPDK00000000000001 ) with lcore 0 > Initialization complete. Launching workers. > Starting thread on core 0 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > = > Latency(us) > Device Information : IOPS > MiB/s Average min max > SPDK_Controller1 (SPDK00000000000001 ) from core 0: 37.60 > 0.15 26709.88 14961.21 39904.69 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > Total : 37.60 > 0.15 26709.88 14961.21 39904.69 > = > = > Then on the same container I tried the following steps : > = > 604 scripts/rpc.py construct_nvme_bdev -b nvme1 -t PCIe -a 0000:1d:00.= 00 > 605 scripts/rpc.py nvmf_subsystem_create nqn.2016-06.io.spdk:cnode2 a = -s > SPDK00000000000002 -d SPDK_Controller2 You forgot the '-' on the a. I believe this command failed in your script. = Try adding set -e to the top so that your script exits on error. If this doesn't fail, please file a GitHub issue. It should not have allowed you to proceed. > 606 scripts/rpc.py nvmf_subsystem_create nqn.2016-06.io.spdk:cnode2 -sa > -s SPDK00000000000002 -d SPDK_Controller2 This should be failing - you can't create a subsystem twice. Note also that= you have the -sa option, which is not valid. > 607 scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk:cnode2 > nvme1n1 > 608 scripts/rpc.py nvmf_subsystem_add_listener > nqn.2016-06.io.spdk:cnode2 -t TCP -a 172.20.0.2 -s 4421 > = > = > = > = > This time when I try I/O I get this error : > sh-4.2# /usr/local/bin/spdk_nvme_perf -q 1 -o 4096 -w write -M 0 -t 10 -r > 'trtype:TCP traddr:172.20.0.2 adrfam:IPv4 trsvcid:4421 > subnqn:nqn.2016-06.io.spdk:cnode2' > Ignoring -M option... Please use -M option only when using rw or randrw. > Starting SPDK v19.04.1 / DPDK 19.05.0 initialization... > [ DPDK EAL parameters: perf --no-shconf -c 0x1 --no-pci > --log-level=3Dlib.eal:6 --base-virtaddr=3D0x200000000000 --match-allocati= ons > --file-prefix=3Dspdk_pid561 ] > EAL: No available hugepages reported in hugepages-1048576kB > EAL: VFIO support initialized > Initializing NVMe Controllers > Attaching to NVMe over Fabrics controller at 172.20.0.2:4421: > nqn.2016-06.io.spdk:cnode2 > ctrlr.c: 534:spdk_nvmf_ctrlr_connect: *ERROR*: Subsystem > 'nqn.2016-06.io.spdk:cnode2' does not allow host > '2014-08.org.nvmexpress:uuid:f7124f51-379b-4752-b195-0487d39d4543' > nvme_fabric.c: 327:nvme_fabric_qpair_connect: *ERROR*: Connect command > failed > nvme_tcp.c:1749:nvme_tcp_qpair_connect: *ERROR*: Failed to send an NVMe-oF > Fabric CONNECT command > nvme_tcp.c:1826:nvme_tcp_ctrlr_construct: *ERROR*: failed to create admin > qpair > nvme.c: 408:nvme_ctrlr_probe: *ERROR*: Failed to construct NVMe controller > for SSD: 172.20.0.2 > nvme.c: 535:spdk_nvme_probe_internal: *ERROR*: NVMe ctrlr scan failed > nvme.c: 608:spdk_nvme_probe: *ERROR*: Create probe context failed > spdk_nvme_probe() failed for transport address '172.20.0.2' > /usr/local/bin/spdk_nvme_perf: errors occured > = > What does the above error mean and how do I correct this ? Is my second > test expected to work ? Apologies if this sort of newbie query. I am just > starting on this. It means you didn't set up the access control on the subsystem to allow this host to connect. "Subsystem [xyz] does not allow host [ijk]". That's becaus= e you didn't pass valid parameters to the command that creates the subsystem, and those commands should have failed. If they don't fail today, please do file= an issue on GitHub and we'll get that sorted out. Thanks, Ben > = > Thanks > Sushil > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk --===============1835851477691795542==--