Hi, I have the SPDK target working for AIO and Malloc. I now have configured NVMe device on the same setup, but not able to start the target, as the target is not finding the nvme0 bdev. Do we have any other dependency for configuring Nvme devices ? *Nvmf.conf* [Global] [Rpc] Enable No Listen 127.0.0.1 [Nvmf] MaxQueuesPerSession 4 AcceptorPollRate 10000 [Nvme] TransportId "trtype:PCIe traddr:0000:07:00.0" Nvme0 RetryCount 4 Timeout 0 AdminPollRate 100000 HotplugEnable Yes [Subsystem1] NQN nqn.2016-06.io.spdk:cnode1 Core 0 Listen RDMA 10.3.7.2:4420 AllowAnyHost Yes SN SPDK00000000000001 Namespace Nvme0n1 1 *# app/nvmf_tgt/nvmf_tgt -t all -c /etc/spdk/nvmf.conf * Starting DPDK 17.05.0 initialization... [ DPDK EAL parameters: nvmf -c 0x1 --file-prefix=spdk_pid97433 ] EAL: Detected 20 lcore(s) EAL: Probing VFIO support... EAL: VFIO support initialized Total cores available: 1 Occupied cpu socket mask is 0x1 reactor.c: 314:_spdk_reactor_run: *NOTICE*: Reactor started on core 0 on socket 0 EAL: PCI device 0000:00:04.0 on NUMA socket 0 EAL: probe driver: 8086:6f20 spdk_ioat EAL: PCI device 0000:00:04.1 on NUMA socket 0 EAL: probe driver: 8086:6f21 spdk_ioat EAL: PCI device 0000:00:04.2 on NUMA socket 0 EAL: probe driver: 8086:6f22 spdk_ioat EAL: PCI device 0000:00:04.3 on NUMA socket 0 EAL: probe driver: 8086:6f23 spdk_ioat EAL: PCI device 0000:00:04.4 on NUMA socket 0 EAL: probe driver: 8086:6f24 spdk_ioat EAL: PCI device 0000:00:04.5 on NUMA socket 0 EAL: probe driver: 8086:6f25 spdk_ioat EAL: PCI device 0000:00:04.6 on NUMA socket 0 EAL: probe driver: 8086:6f26 spdk_ioat EAL: PCI device 0000:00:04.7 on NUMA socket 0 EAL: probe driver: 8086:6f27 spdk_ioat copy_engine_ioat.c: 306:copy_engine_ioat_init: *NOTICE*: Ioat Copy Engine Offload Enabled EAL: PCI device 0000:00:04.0 on NUMA socket 0 EAL: probe driver: 8086:6f20 spdk_ioat EAL: PCI device 0000:00:04.1 on NUMA socket 0 EAL: probe driver: 8086:6f21 spdk_ioat EAL: PCI device 0000:00:04.2 on NUMA socket 0 EAL: probe driver: 8086:6f22 spdk_ioat EAL: PCI device 0000:00:04.3 on NUMA socket 0 EAL: probe driver: 8086:6f23 spdk_ioat EAL: PCI device 0000:00:04.4 on NUMA socket 0 EAL: probe driver: 8086:6f24 spdk_ioat EAL: PCI device 0000:00:04.5 on NUMA socket 0 EAL: probe driver: 8086:6f25 spdk_ioat EAL: PCI device 0000:00:04.6 on NUMA socket 0 EAL: probe driver: 8086:6f26 spdk_ioat EAL: PCI device 0000:00:04.7 on NUMA socket 0 EAL: probe driver: 8086:6f27 spdk_ioat EAL: PCI device 0000:07:00.0 on NUMA socket 0 EAL: probe driver: 8086:953 spdk_nvme nvmf.c: 67:spdk_nvmf_tgt_init: *INFO*: Max Queue Pairs Per Controller: 4 nvmf.c: 68:spdk_nvmf_tgt_init: *INFO*: Max Queue Depth: 128 nvmf.c: 69:spdk_nvmf_tgt_init: *INFO*: Max In Capsule Data: 4096 bytes nvmf.c: 70:spdk_nvmf_tgt_init: *INFO*: Max I/O Size: 131072 bytes nvmf_tgt.c: 215:nvmf_tgt_create_subsystem: *NOTICE*: allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0 nvmf_tgt.c: 215:nvmf_tgt_create_subsystem: *NOTICE*: allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0 transport.c: 63:spdk_nvmf_transport_create: *ERROR*: inside spdk_nvmf_transport_create 1 rdma.c: 955:spdk_nvmf_rdma_create: *NOTICE*: *** RDMA Transport Init *** rdma.c:1120:spdk_nvmf_rdma_listen: *NOTICE*: *** NVMf Target Listening on 10.3.7.2 port 4420 *** bdev.c: 214:spdk_bdev_get_by_name: *ERROR*: bdev's returned null 'Nvme0n1' *conf.c: 492:spdk_nvmf_construct_subsystem: *ERROR*: Could not find namespace bdev 'Nvme0n1'* subsystem.c: 232:spdk_nvmf_delete_subsystem: *INFO*: subsystem is 0x1218ef0 nvmf_tgt.c: 276:spdk_nvmf_startup: *ERROR*: spdk_nvmf_parse_conf() failed subsystem.c: 232:spdk_nvmf_delete_subsystem: *INFO*: subsystem is 0x1218bf0 *# lspci |grep -i vola* 07:00.0 Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01) Thanks -kiru *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Yang, Ziye *Sent:* Tuesday, September 05, 2017 10:41 AM *To:* Storage Performance Development Kit *Subject:* Re: [SPDK] NVMf target configuration issue And the other choice is to remove the Host related line, which means any host can connect. *From:* SPDK [mailto:spdk-bounces(a)lists.01.org ] *On Behalf Of *Yang, Ziye *Sent:* Tuesday, September 5, 2017 1:04 PM *To:* Storage Performance Development Kit *Subject:* Re: [SPDK] NVMf target configuration issue Hi Kiru, According to the error message, I suggest that you add the Host info in the following section. [Subsystem1] NQN nqn.2016-06.io.spdk:cnode1 Core 0 Listen RDMA 10.3.7.2:4420 Host nqn.2016-06.io.spdk:init * Host nqn.2014-08.org.nvmexpress:NVMf:uuid:5e5e0064-2ad6-4a43-9c26-063e3ef6cf14* SN SPDK00000000000001 Namespace AIO0 Thanks. *From:* SPDK [mailto:spdk-bounces(a)lists.01.org ] *On Behalf Of *Kirubakaran Kaliannan *Sent:* Tuesday, September 5, 2017 12:52 PM *To:* Storage Performance Development Kit *Subject:* [SPDK] NVMf target configuration issue Hi All, I am trying to get the nvmf configuration going with SPDK. I have the following configuration, *# ofed 2.4 on both target and initiator* *# SPDK + 3.18 kernel on target * *# 4.9 kernel on initiator* *# mellanox connect3X-Pro* */etc/spdk/nvmf.conf* [Global] [Rpc] Enable No Listen 127.0.0.1 [AIO] AIO /dev/sdd AIO0 [Nvmf] MaxQueuesPerSession 4 AcceptorPollRate 10000 [Subsystem1] NQN nqn.2016-06.io.spdk:cnode1 Core 0 Listen RDMA 10.3.7.2:4420 Host nqn.2016-06.io.spdk:init SN SPDK00000000000001 Namespace AIO0 *From initiator I tried discover and connect* *# nvme discover -t rdma -a 10.3.7.2 -s 4420* Discovery Log Number of Records 1, Generation counter 4 =====Discovery Log Entry 0====== trtype: rdma adrfam: ipv4 subtype: nvme subsystem treq: not specified portid: 0 trsvcid: 4420 subnqn: nqn.2016-06.io.spdk:cnode1 traddr: 10.3.7.2 rdma_prtype: not specified rdma_qptype: connected rdma_cms: rdma-cm rdma_pkey: 0x0000 *# nvme connect -t rdma -n "nqn.2016-06.io.spdk:cnode1" -a 10.3.7.2 -s 4420* Failed to write to /dev/nvme-fabrics: Input/output error [dmesg] [59909.541392] nvme nvme0: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 10.3.7.2:4420 [59940.389952] nvme nvme0: Connect command failed, error wo/DNR bit: 388 *On target I get the following error* # app/nvmf_tgt/nvmf_tgt -c /etc/spdk/nvmf.conf > /tmp/1 EAL: Detected 20 lcore(s) reactor.c: 314:_spdk_reactor_run: *NOTICE*: Reactor started on core 0 on socket 0 copy_engine_ioat.c: 306:copy_engine_ioat_init: *NOTICE*: Ioat Copy Engine Offload Enabled nvmf_tgt.c: 215:nvmf_tgt_create_subsystem: *NOTICE*: allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0 nvmf_tgt.c: 215:nvmf_tgt_create_subsystem: *NOTICE*: allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0 rdma.c: 955:spdk_nvmf_rdma_create: *NOTICE*: *** RDMA Transport Init *** rdma.c:1120:spdk_nvmf_rdma_listen: *NOTICE*: *** NVMf Target Listening on 10.3.7.2 port 4420 *** conf.c: 500:spdk_nvmf_construct_subsystem: *NOTICE*: Attaching block device AIO0 to subsystem nqn.2016-06.io.spdk:cnode1 nvmf_tgt.c: 290:spdk_nvmf_startup: *NOTICE*: Acceptor running on core 0 on socket 0 *request.c: 171:nvmf_process_connect: *ERROR*: Subsystem 'nqn.2016-06.io.spdk:cnode1' does not allow host 'nqn.2014-08.org.nvmexpress:NVMf:uuid:5e5e0064-2ad6-4a43-9c26-063e3ef6cf14'* can you please help on why I am not able to connect in this configuration. Regards, -kiru