[SPDK] EAL: pci_map_resource(): cannot mmap(16, 0x1080004000, 0x4000, 0x0): Invalid argument (0xffffffffffffffff)

[Oza Oza <oza.oza at broadcom.com>]

[-- Attachment #1: Type: text/plain, Size: 1930 bytes --]

Hi All,



I am using SPDK and DPDK.



I have single intel nvme SSD connected to x4 pci slot.



Setup.sh runs fine, and able to unbind kernel driver okay.

But when I run fio benchmark : as follows:



fio --randrepeat=1
--ioengine=/home/oza/SPDK/spdk/examples/nvme/fio_plugin/fio_plugin
--direct=1 --numjobs=1 --gtod_reduce=1 --name=readtest --filename=
0000.01.00.00/1 --bs=4096   --iodepth=128 --size=4G --readwrite=read



I get pci_map_resource failed.

I have debugged this to an extent where:



I figure that pci_map_resource trying to map msi-x memory and vfio is
denying it.

I see there is a call in pci_vfio_map_resource which asks for
pci_vfio_get_msix_bar.



But following log suggests that pci_vfio_map_resource is being called twice
which is strange.

On top of that, I am getting different value for msix_bar second time,
which is causing the failure.



So first time bar0 as msi-x which seems okay with respect to kernel vfio
(msix-bar)

But second time DPDK says msix_bar is 1, but kernel vfio still says it is
0, and denies to map msi-x memory to user space.



EAL: Detected 8 lcore(s)

EAL: Auto-detected process type: PRIMARY

EAL: Probing VFIO support...

EAL: VFIO support initialized

EAL: cannot open /proc/self/numa_maps, consider that all memory is in
socket_id 0

EAL: PCI device 0000:01:00.0 on NUMA socket 0

EAL:   probe driver: 8086:953 spdk_nvme

EAL:   using IOMMU type 1 (Type 1)

EAL: the msi-x bar number is 0

EAL: Hotplug doesn't support vfio yet

EAL: PCI device 0000:01:00.0 on NUMA socket 0

EAL:   probe driver: 8086:953 spdk_nvme

EAL: the msi-x bar number is 1

EAL: pci_map_resource(): cannot mmap(16, 0x1080004000, 0x4000, 0x0):
Invalid argument (0xffffffffffffffff)

EAL:   0000:01:00.0 mapping BAR0 failed: Invalid argument

EAL: Error - exiting with code: 1





Regards,

Oza.

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 3939 bytes --]