Re: [SPDK] EAL: pci_map_resource(): cannot mmap(16, 0x1080004000, 0x4000, 0x0): Invalid argument (0xffffffffffffffff)

[Oza Oza <oza.oza at broadcom.com>]

[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]

Hi All,

I am using SPDK and DPDK.

I have single intel nvme SSD connected to x4 pci slot.

Setup.sh runs fine, and able to unbind kernel driver okay.
But when I run fio benchmark : as follows:

fio --randrepeat=1
--ioengine=/home/oza/SPDK/spdk/examples/nvme/fio_plugin/fio_plugin
--direct=1 --numjobs=1 --gtod_reduce=1 --name=readtest
--filename=0000.01.00.00/1 --bs=4096   --iodepth=128 --size=4G
--readwrite=read

I get pci_map_resource failed.
I have debugged this to an extent where:

I figure that pci_map_resource trying to map msi-x memory and vfio is
denying it.
I see there is a call in pci_vfio_map_resource which asks for
pci_vfio_get_msix_bar.

But following log suggests that pci_vfio_map_resource is being called
twice which is strange.
On top of that, I am getting different value for msix_bar second time,
which is causing the failure.

So first time bar0 as msi-x which seems okay with respect to kernel vfio
(msix-bar)
But second time DPDK says msix_bar is 1, but kernel vfio still says it is
0, and denies to map msi-x memory to user space.

EAL: Detected 8 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: cannot open /proc/self/numa_maps, consider that all memory is in
socket_id 0
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL:   probe driver: 8086:953 spdk_nvme
EAL:   using IOMMU type 1 (Type 1)
EAL: the msi-x bar number is 0
EAL: Hotplug doesn't support vfio yet
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL:   probe driver: 8086:953 spdk_nvme
EAL: the msi-x bar number is 1
EAL: pci_map_resource(): cannot mmap(16, 0x1080004000, 0x4000, 0x0):
Invalid argument (0xffffffffffffffff)
EAL:   0000:01:00.0 mapping BAR0 failed: Invalid argument
EAL: Error - exiting with code: 1


Regards,
Oza.