Hi Gyan,

libnuma/numactl is a recently added package dependency. You can run scripts/pkgdep.sh or just install it explicitly – numactl-devel on RHEL/CentOS or libnuma-dev on Ubuntu/Debian.

-Jim

From: SPDK <spdk-bounces@lists.01.org> on behalf of Gyan Prakash <gyapra2016@gmail.com>
Reply-To: Storage Performance Development Kit <spdk@lists.01.org>
Date: Tuesday, October 17, 2017 at 5:36 PM
To: Storage Performance Development Kit <spdk@lists.01.org>
Subject: Re: [SPDK] NVMf Target configuration issue

Hi all,

I got the latest master of SPDK and getting following error:

/root/test-spdk/spdk/dpdk/lib/librte_eal/linuxapp/eal/eal_memory.c:56:18: fatal error: numa.h: No such file or directory
#include <numa.h>
^
compilation terminated.
CC eal_timer.o
CC eal_interrupts.o
CC eal_alarm.o
make[8]: *** [eal_memory.o] Error 1
make[8]: *** Waiting for unfinished jobs....
make[7]: *** [eal] Error 2
make[6]: *** [linuxapp] Error 2
make[5]: *** [librte_eal] Error 2
make[4]: *** [lib] Error 2
make[3]: *** [all] Error 2
make[3]: Leaving directory `/root/test-spdk/spdk/dpdk'
make[2]: *** [all] Error 2
make[1]: *** [all] Error 2
make: *** [dpdkbuild] Error 2

I am using the in-built DPDK and I do not see the file "numa.h" anywhere. Am I supposed to get something installed on my system to get the numa.h?

I used "make CONFIG_RDMA=y CONFIG_DEBUG=y" as my build command.

Thanks & regards,

Gyan

On Tue, Oct 17, 2017 at 1:03 PM, Gyan Prakash <gyapra2016@gmail.com> wrote:

Hi Daniel,

Thanks for the reply. You are correct, I am using SPDK v17.03. I will test with SPDK v17.07.

I will also check the referenced documents for the configuration.

Thanks,

Gyan

On Tue, Oct 17, 2017 at 12:54 PM, Verkamp, Daniel <daniel.verkamp@intel.com> wrote:

Hi Gyan,

It looks like you are using an older version of SPDK before the NVMe-oF target was changed to always use the bdev abstraction layer.

Can you retest with the latest master, or at least SPDK v17.07? To do this, you will need to update your nvmf.conf to specify the NVMe devices you want to use as bdevs – see the NVMe bdev configuration docs [1] and NVMe-oF target Getting Started guide [2] for more details.

Thanks,

-- Daniel

[1]: http://www.spdk.io/doc/bdev.html#bdev_config_nvme

[2]: http://www.spdk.io/doc/nvmf.html#nvmf_getting_started

From: SPDK [mailto:spdk-bounces@lists.01.org] On Behalf Of Gyan Prakash
Sent: Tuesday, October 17, 2017 12:05 PM
To: spdk@lists.01.org
Subject: [SPDK] NVMf Target configuration issue

Hi all,

I see issue with NVMf Target configuration. I also see the thread [SPDK] NVMf target configuration issue Thu Sep 7 01:29:33 PDT 2017 which is for the similar issue what I am seeing. I tried the suggestion from that thread, but it was not helpful for my problem.

Error Message:

# ./nvmf_tgt -t all -c nvmf.conf

Starting DPDK 17.02.0 initialization...

[ DPDK EAL parameters: nvmf -c fff --file-prefix=spdk_pid3911 ]

EAL: Detected 12 lcore(s)

EAL: No free hugepages reported in hugepages-1048576kB

EAL: Probing VFIO support...

Occupied cpu core mask is 0xfff

Occupied cpu socket mask is 0x1

EAL: PCI device 0000:00:04.0 on NUMA socket 0

EAL: probe driver: 8086:2f20 spdk_ioat

Found matching device at 0000:00:04.0 vendor:0x8086 device:0x2f20

EAL: PCI device 0000:00:04.1 on NUMA socket 0

EAL: probe driver: 8086:2f21 spdk_ioat

Found matching device at 0000:00:04.1 vendor:0x8086 device:0x2f21

EAL: PCI device 0000:00:04.2 on NUMA socket 0

EAL: probe driver: 8086:2f22 spdk_ioat

Found matching device at 0000:00:04.2 vendor:0x8086 device:0x2f22

EAL: PCI device 0000:00:04.3 on NUMA socket 0

EAL: probe driver: 8086:2f23 spdk_ioat

Found matching device at 0000:00:04.3 vendor:0x8086 device:0x2f23

EAL: PCI device 0000:00:04.4 on NUMA socket 0

EAL: probe driver: 8086:2f24 spdk_ioat

Found matching device at 0000:00:04.4 vendor:0x8086 device:0x2f24

EAL: PCI device 0000:00:04.5 on NUMA socket 0

EAL: probe driver: 8086:2f25 spdk_ioat

Found matching device at 0000:00:04.5 vendor:0x8086 device:0x2f25

EAL: PCI device 0000:00:04.6 on NUMA socket 0

EAL: probe driver: 8086:2f26 spdk_ioat

Found matching device at 0000:00:04.6 vendor:0x8086 device:0x2f26

EAL: PCI device 0000:00:04.7 on NUMA socket 0

EAL: probe driver: 8086:2f27 spdk_ioat

Found matching device at 0000:00:04.7 vendor:0x8086 device:0x2f27

Ioat Copy Engine Offload Enabled

EAL: PCI device 0000:01:00.0 on NUMA socket 0

EAL: probe driver: 8086:953 spdk_nvme

Probing device 0000:01:00.0

[nvme] nvme_ctrlr.c: 502:nvme_ctrlr_set_state: setting state to init (no timeout)

[nvme] nvme_ctrlr.c:1135:nvme_ctrlr_process_init: CC.EN = 1

[nvme] nvme_ctrlr.c:1149:nvme_ctrlr_process_init: Setting CC.EN = 0

[nvme] nvme_ctrlr.c: 506:nvme_ctrlr_set_state: setting state to disable and wait for CSTS.RDY = 0 (timeout 20000 ms)

[nvme] nvme_ctrlr.c:1206:nvme_ctrlr_process_init: CC.EN = 0 && CSTS.RDY = 0 - enabling controller

[nvme] nvme_ctrlr.c:1208:nvme_ctrlr_process_init: Setting CC.EN = 1

[nvme] nvme_ctrlr.c: 506:nvme_ctrlr_set_state: setting state to enable and wait for CSTS.RDY = 1 (timeout 20000 ms)

[nvme] nvme_ctrlr.c:1217:nvme_ctrlr_process_init: CC.EN = 1 && CSTS.RDY = 1 - controller is ready

[nvme] nvme_ctrlr.c: 594:nvme_ctrlr_identify: transport max_xfer_size 2072576

[nvme] nvme_ctrlr.c: 598:nvme_ctrlr_identify: MDTS max_xfer_size 131072

[nvme] nvme_ctrlr.c: 673:nvme_ctrlr_set_keep_alive_timeout: Controller KAS is 0 - not enabling Keep Alive

[nvme] nvme_ctrlr.c: 502:nvme_ctrlr_set_state: setting state to ready (no timeout)

[debug] bdev.c: 932:spdk_bdev_register: Inserting bdev Nvme0n1 into list

Total cores available: 12

Reactor started on core 1 on socket 0

Reactor started on core 2 on socket 0

Reactor started on core 3 on socket 0

Reactor started on core 4 on socket 0

Reactor started on core 5 on socket 0

Reactor started on core 6 on socket 0

Reactor started on core 7 on socket 0

Reactor started on core 8 on socket 0

Reactor started on core 9 on socket 0

Reactor started on core 10 on socket 0

Reactor started on core 0 on socket 0

[nvmf] nvmf.c: 68:spdk_nvmf_tgt_init: Max Queues Per Session: 4

[nvmf] nvmf.c: 69:spdk_nvmf_tgt_init: Max Queue Depth: 128

[nvmf] nvmf.c: 70:spdk_nvmf_tgt_init: Max In Capsule Data: 4096 bytes

[nvmf] nvmf.c: 71:spdk_nvmf_tgt_init: Max I/O Size: 131072 bytes

*** RDMA Transport Init ***

Reactor started on core 11 on socket 0

allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0

allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 5 on socket 0

[rdma] rdma.c:1177:spdk_nvmf_rdma_listen: For listen id 0x1720840 with context 0x1720db0, created completion channel 0x1720ad0

conf.c: 555:spdk_nvmf_construct_subsystem: ***ERROR*** Subsystem nqn.2016-06.io.spdk:cnode1: missing NVMe directive

[nvmf] subsystem.c: 231:spdk_nvmf_delete_subsystem: subsystem is 0x1720b20

nvmf_tgt.c: 313:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed

[nvmf] subsystem.c: 231:spdk_nvmf_delete_subsystem: subsystem is 0x17205b0

[nvme] nvme_ctrlr.c: 406:nvme_ctrlr_shutdown: shutdown complete

# ./setup.sh

0000:01:00.0 (8086 0953): nvme -> uio_pci_generic

0000:00:04.0 (8086 2f20): ioatdma -> uio_pci_generic

0000:00:04.1 (8086 2f21): ioatdma -> uio_pci_generic

0000:00:04.2 (8086 2f22): ioatdma -> uio_pci_generic

0000:00:04.3 (8086 2f23): ioatdma -> uio_pci_generic

0000:00:04.4 (8086 2f24): ioatdma -> uio_pci_generic

0000:00:04.5 (8086 2f25): ioatdma -> uio_pci_generic

0000:00:04.6 (8086 2f26): ioatdma -> uio_pci_generic

0000:00:04.7 (8086 2f27): ioatdma -> uio_pci_generic

# lspci | grep -i vola*

01:00.0 Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)

# lspci -v -s 0000:01:00.0

01:00.0 Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01) (prog-if 02 [NVM Express])

Subsystem: Intel Corporation DC P3700 SSD

Physical Slot: 1

Flags: fast devsel, IRQ 24, NUMA node 0

Memory at fb410000 (64-bit, non-prefetchable) [size=16K]

Expansion ROM at fb400000 [disabled] [size=64K]

Capabilities: [40] Power Management version 3

Capabilities: [50] MSI-X: Enable- Count=32 Masked-

Capabilities: [60] Express Endpoint, MSI 00

Capabilities: [100] Advanced Error Reporting

Capabilities: [150] Virtual Channel

Capabilities: [180] Power Budgeting <?>

Capabilities: [190] Alternative Routing-ID Interpretation (ARI)

Capabilities: [270] Device Serial Number 55-cd-2e-41-4d-34-1e-1b

Capabilities: [2a0] #19

Kernel driver in use: uio_pci_generic

Kernel modules: nvme

_______________________________________________
SPDK mailing list
SPDK@lists.01.org
https://lists.01.org/mailman/listinfo/spdk