Re: [SPDK] nvme_ctrlr.c:1224:nvme_ctrlr_process_init: ***ERROR*** Initialization timed out in state 3

[Harris, James R <james.r.harris at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 2527 bytes --]

Hi Oza,

> On Apr 3, 2017, at 2:01 AM, Oza Oza <oza.oza(a)broadcom.com> wrote:
> 
> I have ported SPDK for ARMv8.

Great!  Do you have patches that you can share?

> And DPDK is compiled with version: 16.11.1
> 
> init controller is failing.

Does this mean that the previous issue you reported on April 1 has been resolved?  If so, can you share details?

> 
> root(a)ubuntu:/home/oza/SPDK/spdk#
> odepth=128 --size=4G --readwrite=read --filename=0000.01.00.00/1 --bs=4096
> --i
> /home/oza/fio /home/oza/SPDK/spdk
> EAL: pci driver is being registered 0x1nreadtest: (g=0): rw=read,
> bs=4096B-4096B,4096B-4096B,4096B-4096B, ioengine=spdk_fio, iodepth=128
> fio-2.17-29-gf0ac1
> Starting 1 process
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: fio -c 1 --file-prefix=spdk_pid6448
> --base-virtaddr=0x1000000000 --proc-type=auto ]
> EAL: Detected 8 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: cannot open /proc/self/numa_maps, consider that all memory is in
> socket_id 0
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:953 spdk_nvme
> EAL:   using IOMMU type 1 (Type 1)
> EAL: vfio_group_fd=11 iommu_group_no=3 *vfio_dev_fd=13
> EAL: reg=0x2000 fd=13 cap_offset=0x50
> EAL: the msi-x bar number is 0 0x2000 0x200
> EAL: Hotplug doesn't support vfio yet
> spdk_fio_setup() is being called
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:953 spdk_nvme
> EAL: vfio_group_fd=11 iommu_group_no=3 *vfio_dev_fd=16
> EAL: reg=0x2000 fd=16 cap_offset=0x50
> EAL: the msi-x bar number is 0 0x2000 0x200
> EAL: inside pci_vfio_write_config offset=4
> nvme_ctrlr.c:1224:nvme_ctrlr_process_init: ***ERROR*** Initialization
> timed out in state 3
> nvme_ctrlr.c: 403:nvme_ctrlr_shutdown: ***ERROR*** did not shutdown within
> 5 seconds

I think this is where seeing your ARMv8 patches would help.  Then we could collaborate on if/how the ARMv8 changes you have made may be causing the initialization error.

> EAL: Hotplug doesn't support vfio yet
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:953 spdk_nvme
> EAL: vfio_group_fd=11 iommu_group_no=3 *vfio_dev_fd=18
> EAL: reg=0x2000 fd=18 cap_offset=0x50
> EAL: the msi-x bar number is 0 0x2000 0x200
> EAL: Hotplug doesn't support vfio yet
> EAL: Requested device 0000:01:00.0 cannot be used
> spdk_nvme_probe() failed

Thanks,

-Jim