Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Manoj Viswanath]

[-- Attachment #1: Type: text/plain, Size: 1607 bytes --]

Hi,

I have a DPDK application running on QEMU-KVM environment using DPDK 1.6.0.
I am trying to port the same to DPDK version 1.7.1.

I am using Virt-manager GUI to assign e1000 emulated port to the VM. This
works fine in DPDK 1.6.0. The device is identified by IGB_UIO and
initialized by my application as expected.

However in case of DPDK 1.7.1, the emulated e1000 devices do not seem to be
recognized.
Following is my analysis:

1. The API pci_get_uio_dev() is returning ERROR. This is called from
pci_uio_map_resource() in the flow of PCI PROBE [rte_eal_pci_probe()].

2. Due to this, the PCI device is not getting mapped to the correct driver
(EM
​ Driver​
).

3. The reason for the error in [1] appears to be that "uio" sub-directory
doesn't seem to be correctly created for interfaces assigned to this VM.

4. Upon further analysis i found that IGB_UIO probe function
["igbuio_pci_probe()"] is not getting triggered indicating the port has *not
been assigned* to the IGB_UIO.

Kindly refer to the attachments:-
- "Output of sys-bus-pci-devices" - indicating "uio" subdirectory not
created for PCI devices in case of DPDK 1.7.1
- "Output of lspci -v" - indicating device not bound to driver in case of
DPDK 1.7.1
- IGB_UIO init log snippet - indicating PCI devices not detected and
initialized by IGB_UIO in case of DPDK 1.7.1
- CONFIG file used for DPDK compilation

Not sure what has changed between 1.6.0 and 1.7.1 which is impacting this.

Could someone throw light in this regard as to what i may be missing ?

Thanks in advance.

Regards,
Manoj

[-- Attachment #2: IGB_UIO INIT Logs.txt --]
[-- Type: text/plain, Size: 1559 bytes --]

----------------------------------------------
[A] IGB UIO INIT LOGS - VM running DPDK 1.6.0
----------------------------------------------

Oct 20 05:14:31 localhost kernel: Use MSIX interrupt by default
Oct 20 05:14:31 localhost kernel: Use MSIX interrupt by default

Oct 20 05:14:31 localhost kernel: igb_uio 0000:00:04.0: setting latency timer to
 64
Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en
tries
Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en
tries
Oct 20 05:14:31 localhost kernel: uio device registered with irq b	-------------------------------------> PCI device bound to driver
Oct 20 05:14:31 localhost kernel: uio device registered with irq b

Oct 20 05:14:31 localhost kernel: igb_uio 0000:00:08.0: setting latency timer to
 64
Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en
tries
Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en
tries
Oct 20 05:14:31 localhost kernel: uio device registered with irq b	-------------------------------------> PCI device bound to driver
Oct 20 05:14:31 localhost kernel: uio device registered with irq b

----------------------------------------------
[B] IGB UIO INIT LOGS - VM running DPDK 1.7.1
----------------------------------------------

Oct 20 05:10:40 localhost kernel: igb_uio: Use MSIX interrupt by default
Oct 20 05:10:40 localhost kernel: igb_uio: Use MSIX interrupt by default
=====> No output for PCI initialization by IGB_UIO 

[-- Attachment #3: Output of lspci -v.txt --]
[-- Type: text/plain, Size: 1013 bytes --]

-------------------------
[VM-2 running DPDK 1.6.0]
-------------------------
00:08.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controll
er (rev 03)
    Subsystem: Red Hat, Inc Device 1100
    Flags: bus master, fast devsel, latency 0, IRQ 11
    Memory at f20a0000 (32-bit, non-prefetchable) [size=128K]
    I/O ports at c140 [size=64]
    Expansion ROM at f20c0000 [disabled] [size=128K]
    Kernel driver in use: igb_uio=================> Driver binding succesfull
    Kernel modules: igb_uio	==================> Driver binding succesfull

-------------------------
[VM-1 running DPDK 1.7.1]
-------------------------
00:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controll
er (rev 03)
    Subsystem: Red Hat, Inc Device 1100
    Flags: fast devsel, IRQ 11
    Memory at f2040000 (32-bit, non-prefetchable) [size=128K]
    I/O ports at c080 [size=64]
    Expansion ROM at f2060000 [disabled] [size=128K]
===> Missing driver binding info


[-- Attachment #4: Output of sys-bus-pci-devices.txt --]
[-- Type: text/plain, Size: 4777 bytes --]


@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ On VM-1 running DPDK 1.7.1 @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

bash-4.2# lspci | grep 82540EM
00:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)
00:05.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)

##SYS-BUS
bash-4.2# ls /sys/bus/pci/devices/               
0000:00:00.0  0000:00:01.2  0000:00:03.0  0000:00:06.0
0000:00:01.0  0000:00:01.3  0000:00:04.0  0000:00:07.0
0000:00:01.1  0000:00:02.0  0000:00:05.0  0000:00:08.0

bash-4.2# ls /sys/bus/pci/devices/0000\:00\:04.0/
broken_parity_status      local_cpulist             resource0
class                     local_cpus                resource1
config                    modalias                  rom
consistent_dma_mask_bits  msi_bus                   subsystem/
device                    numa_node                 subsystem_device
dma_mask_bits             power/                    subsystem_vendor
enable                    remove                    uevent
firmware_node/            rescan                    vendor
irq                       resource                  
bash-4.2# ls /sys/bus/pci/devices/0000\:00\:04.0/uio    
ls: cannot access /sys/bus/pci/devices/0000:00:04.0/uio: No such file or directory	============================> No directory

bash-4.2# ls /sys/bus/pci/devices/0000\:00\:05.0/
broken_parity_status      local_cpulist             resource0
class                     local_cpus                resource1
config                    modalias                  rom
consistent_dma_mask_bits  msi_bus                   subsystem/
device                    numa_node                 subsystem_device
dma_mask_bits             power/                    subsystem_vendor
enable                    remove                    uevent
firmware_node/            rescan                    vendor
irq                       resource                  
bash-4.2# ls /sys/bus/pci/devices/0000\:00\:05.0/uio    
ls: cannot access /sys/bus/pci/devices/0000:00:05.0/uio: No such file or directory	============================> No directory
bash-4.2#

##SYS-DEVICES
bash-4.2# ls /sys/devices/pci0000\:00/
0000:00:00.0/  0000:00:01.3/  0000:00:05.0/  firmware_node/ 
0000:00:01.0/  0000:00:02.0/  0000:00:06.0/  pci_bus/       
0000:00:01.1/  0000:00:03.0/  0000:00:07.0/  power/         
0000:00:01.2/  0000:00:04.0/  0000:00:08.0/  uevent         
bash-4.2# ls /sys/devices/pci0000\:00/0000:00:04.0/uio
ls: cannot access /sys/devices/pci0000:00/0000:00:04.0/uio: No such file or directory	============================> No directory
bash-4.2# ls /sys/devices/pci0000\:00/0000:00:04.0/   
broken_parity_status	  enable	 msi_bus    resource0	      uevent
class			  firmware_node  numa_node  resource1	      vendor
config			  irq		 power	    rom
consistent_dma_mask_bits  local_cpulist  remove     subsystem
device			  local_cpus	 rescan     subsystem_device
dma_mask_bits		  modalias	 resource   subsystem_vendor
bash-4.2# 

bash-4.2# ls /sys/devices/pci0000\:00/0000:00:05.0/uio
ls: cannot access /sys/devices/pci0000:00/0000:00:05.0/uio: No such file or directory	============================> No directory

bash-4.2# ls /sys/devices/pci0000\:00/0000:00:05.0/
broken_parity_status	  enable	 msi_bus    resource0	      uevent
class			  firmware_node  numa_node  resource1	      vendor
config			  irq		 power	    rom
consistent_dma_mask_bits  local_cpulist  remove     subsystem
device			  local_cpus	 rescan     subsystem_device
dma_mask_bits		  modalias	 resource   subsystem_vendor



--------------------------------------------------------------------------------------------------------------------------------------------------------

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ On VM-2 running DPDK 1.6.0 @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

bash-4.2# lspci | grep 82540EM
00:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)
00:08.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)

##SYS-BUS
bash-4.2# /sys/bus/pci/devices/0000\:00\:04.0/uio/uio0/			============================> Directory created correctly
device/    maps/      portio/    power/     subsystem/ 
bash-4.2# ls /sys/bus/pci/devices/0000\:00\:04.0/uio/uio0/
dev  device  event  maps  name	portio	power  subsystem  uevent  version

bash-4.2# ls /sys/bus/pci/devices/0000\:00\:08.0/uio/uio1/		============================> Directory created correctly
dev  device  event  maps  name	portio	power  subsystem  uevent  version


##SYS-DEVICES
bash-4.2# ls /sys/devices/pci0000:00/0000:00:04.0/uio/uio0
dev  device  event  maps  name	portio	power  subsystem  uevent  version
bash-4.2# 

Re: Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Bruce Richardson]

On Wed, Nov 05, 2014 at 03:28:13PM +0530, Manoj Viswanath wrote:
> Hi,
> 
> I have a DPDK application running on QEMU-KVM environment using DPDK 1.6.0.
> I am trying to port the same to DPDK version 1.7.1.
> 
> I am using Virt-manager GUI to assign e1000 emulated port to the VM. This
> works fine in DPDK 1.6.0. The device is identified by IGB_UIO and
> initialized by my application as expected.
> 
> However in case of DPDK 1.7.1, the emulated e1000 devices do not seem to be
> recognized.
> Following is my analysis:
> 
> 1. The API pci_get_uio_dev() is returning ERROR. This is called from
> pci_uio_map_resource() in the flow of PCI PROBE [rte_eal_pci_probe()].
> 
> 2. Due to this, the PCI device is not getting mapped to the correct driver
> (EM
> ​ Driver​
> ).
> 
> 3. The reason for the error in [1] appears to be that "uio" sub-directory
> doesn't seem to be correctly created for interfaces assigned to this VM.
> 
> 4. Upon further analysis i found that IGB_UIO probe function
> ["igbuio_pci_probe()"] is not getting triggered indicating the port has *not
> been assigned* to the IGB_UIO.
> 
> Kindly refer to the attachments:-
> - "Output of sys-bus-pci-devices" - indicating "uio" subdirectory not
> created for PCI devices in case of DPDK 1.7.1
> - "Output of lspci -v" - indicating device not bound to driver in case of
> DPDK 1.7.1
> - IGB_UIO init log snippet - indicating PCI devices not detected and
> initialized by IGB_UIO in case of DPDK 1.7.1
> - CONFIG file used for DPDK compilation
> 
> Not sure what has changed between 1.6.0 and 1.7.1 which is impacting this.
> 
> Could someone throw light in this regard as to what i may be missing ?
> 
> Thanks in advance.
> 
> Regards,
> Manoj

Hi Manoj,

can you perhaps give some details on how you were binding the device to the uio
module both for 1.6 and for 1.7?

/Bruce

Re: Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Manoj Viswanath]

Hi Bruce,

I was not doing anything specific for binding the NICs to IGB_UIO (like
invoking "dpdk_nic_bind.py" script explicitly) when using my application
with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM were
automatically getting picked up and initialized by IGB_UIO within each VM.

The same is not working with DPDK 1.7.1 now.

I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK
1.7.1) and tried to check the status. The emulated devices were shown as
neither bound to kernel nor to IGB_UIO as evident from below output:-

<--------------------------------------------------------------------------------------------------->
Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio

Other network devices
=====================
0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
<--------------------------------------------------------------------------------------------------->

When i tried to forcefully bind the NICs using the "--bind=igb_uio" option
of the script, i am getting the following sequence of error log
continuously:-
<--------------------------------------------------------------------------------------------------->
........
........
EAL: Error reading from file descriptor 31: Input/output error
EAL: Error reading from file descriptor 32: Input/output error
........
........
<--------------------------------------------------------------------------------------------------->

Kindly advise.

Thanks in advance !

Regards,
Manoj

On Wed, Nov 5, 2014 at 3:42 PM, Bruce Richardson <bruce.richardson-ral2JQCrhuE@public.gmane.orgm
> wrote:

> On Wed, Nov 05, 2014 at 03:28:13PM +0530, Manoj Viswanath wrote:
> > Hi,
> >
> > I have a DPDK application running on QEMU-KVM environment using DPDK
> 1.6.0.
> > I am trying to port the same to DPDK version 1.7.1.
> >
> > I am using Virt-manager GUI to assign e1000 emulated port to the VM. This
> > works fine in DPDK 1.6.0. The device is identified by IGB_UIO and
> > initialized by my application as expected.
> >
> > However in case of DPDK 1.7.1, the emulated e1000 devices do not seem to
> be
> > recognized.
> > Following is my analysis:
> >
> > 1. The API pci_get_uio_dev() is returning ERROR. This is called from
> > pci_uio_map_resource() in the flow of PCI PROBE [rte_eal_pci_probe()].
> >
> > 2. Due to this, the PCI device is not getting mapped to the correct
> driver
> > (EM
> > ​ Driver​
> > ).
> >
> > 3. The reason for the error in [1] appears to be that "uio" sub-directory
> > doesn't seem to be correctly created for interfaces assigned to this VM.
> >
> > 4. Upon further analysis i found that IGB_UIO probe function
> > ["igbuio_pci_probe()"] is not getting triggered indicating the port has
> *not
> > been assigned* to the IGB_UIO.
> >
> > Kindly refer to the attachments:-
> > - "Output of sys-bus-pci-devices" - indicating "uio" subdirectory not
> > created for PCI devices in case of DPDK 1.7.1
> > - "Output of lspci -v" - indicating device not bound to driver in case of
> > DPDK 1.7.1
> > - IGB_UIO init log snippet - indicating PCI devices not detected and
> > initialized by IGB_UIO in case of DPDK 1.7.1
> > - CONFIG file used for DPDK compilation
> >
> > Not sure what has changed between 1.6.0 and 1.7.1 which is impacting
> this.
> >
> > Could someone throw light in this regard as to what i may be missing ?
> >
> > Thanks in advance.
> >
> > Regards,
> > Manoj
>
> Hi Manoj,
>
> can you perhaps give some details on how you were binding the device to
> the uio
> module both for 1.6 and for 1.7?
>
> /Bruce
>

Re: Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Bruce Richardson]

On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote:
> Hi Bruce,
> 
> I was not doing anything specific for binding the NICs to IGB_UIO (like
> invoking "dpdk_nic_bind.py" script explicitly) when using my application
> with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM were
> automatically getting picked up and initialized by IGB_UIO within each VM.
> 
> The same is not working with DPDK 1.7.1 now.
> 
> I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK
> 1.7.1) and tried to check the status. The emulated devices were shown as
> neither bound to kernel nor to IGB_UIO as evident from below output:-
> 
> <--------------------------------------------------------------------------------------------------->
> Network devices using DPDK-compatible driver
> ============================================
> <none>
> 
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> 
> Other network devices
> =====================
> 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> <--------------------------------------------------------------------------------------------------->
> 
> When i tried to forcefully bind the NICs using the "--bind=igb_uio" option

Was there any output of the dpdk_nic_bind script? What does the output of
it with --status show afterwards?

Regards,
/Bruce

Re: Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Manoj Viswanath]

Hi Bruce,

Please find my comment in lined.

On Fri, Nov 7, 2014 at 9:00 PM, Bruce Richardson <bruce.richardson-ral2JQCrhuE@public.gmane.orgm
> wrote:

> On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote:
> > Hi Bruce,
> >
> > I was not doing anything specific for binding the NICs to IGB_UIO (like
> > invoking "dpdk_nic_bind.py" script explicitly) when using my application
> > with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM
> were
> > automatically getting picked up and initialized by IGB_UIO within each
> VM.
> >
> > The same is not working with DPDK 1.7.1 now.
> >
> > I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK
> > 1.7.1) and tried to check the status. The emulated devices were shown as
> > neither bound to kernel nor to IGB_UIO as evident from below output:-
> >
> >
> <--------------------------------------------------------------------------------------------------->
> > Network devices using DPDK-compatible driver
> > ============================================
> > <none>
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> >
> > Other network devices
> > =====================
> > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> >
> <--------------------------------------------------------------------------------------------------->
> >
> > When i tried to forcefully bind the NICs using the "--bind=igb_uio"
> option
>
> Was there any output of the dpdk_nic_bind script? What does the output of
> it with --status show afterwards?
>
> ​
[MANOJ]​

​Yes. Please refer below output:-
********************************************************************
Network devices using DPDK-compatible driver
============================================
0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
0000:00:05.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=

Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio

Other network devices
=====================
<none>​
********************************************************************

​However, when i start the DPDK application, i am getting the error log as
indicated in earlier mail. ​

The difference with DPDK 1.6.1 is that at the same stage IGB_UIO has
already bound the assigned devices without having to explicitly run the
"dpdk_nic_bind.py". Please find below the application log when run with
DPDK 1.6.0:-

​********************************************************************
Network devices using DPDK-compatible driver
============================================
0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=

Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio

Other network devices
=====================
<none>
********************************************************************​

​Kindly note that in both cases, logs have been taken after loading IGB_UIO
prior to starting DPDK application. ​

​[/MANOJ]​

Regards,
> /Bruce
>
>
​Regards,
MANOJ

Re: Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Bruce Richardson]

On Fri, Nov 07, 2014 at 11:26:08PM +0530, Manoj Viswanath wrote:
> Hi Bruce,
> 
> Please find my comment in lined.
> 
> On Fri, Nov 7, 2014 at 9:00 PM, Bruce Richardson <bruce.richardson@intel.com
> > wrote:
> 
> > On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote:
> > > Hi Bruce,
> > >
> > > I was not doing anything specific for binding the NICs to IGB_UIO (like
> > > invoking "dpdk_nic_bind.py" script explicitly) when using my application
> > > with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM
> > were
> > > automatically getting picked up and initialized by IGB_UIO within each
> > VM.
> > >
> > > The same is not working with DPDK 1.7.1 now.
> > >
> > > I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK
> > > 1.7.1) and tried to check the status. The emulated devices were shown as
> > > neither bound to kernel nor to IGB_UIO as evident from below output:-
> > >
> > >
> > <--------------------------------------------------------------------------------------------------->
> > > Network devices using DPDK-compatible driver
> > > ============================================
> > > <none>
> > >
> > > Network devices using kernel driver
> > > ===================================
> > > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> > >
> > > Other network devices
> > > =====================
> > > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > >
> > <--------------------------------------------------------------------------------------------------->
> > >
> > > When i tried to forcefully bind the NICs using the "--bind=igb_uio"
> > option
> >
> > Was there any output of the dpdk_nic_bind script? What does the output of
> > it with --status show afterwards?
> >
> > ​
> [MANOJ]​
> 
> ​Yes. Please refer below output:-
> ********************************************************************
> Network devices using DPDK-compatible driver
> ============================================
> 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> 0000:00:05.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> 
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> 
> Other network devices
> =====================
> <none>​
> ********************************************************************
> 
> ​However, when i start the DPDK application, i am getting the error log as
> indicated in earlier mail. ​
> 
> The difference with DPDK 1.6.1 is that at the same stage IGB_UIO has
> already bound the assigned devices without having to explicitly run the
> "dpdk_nic_bind.py". Please find below the application log when run with
> DPDK 1.6.0:-
> 
> ​********************************************************************
> Network devices using DPDK-compatible driver
> ============================================
> 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> 0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> 
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> 
> Other network devices
> =====================
> <none>
> ********************************************************************​
> 
> ​Kindly note that in both cases, logs have been taken after loading IGB_UIO
> prior to starting DPDK application. ​
> 
> ​[/MANOJ]​
> 
> Regards,

Ok, so it appears that after running dpdk_nic_bind to bind the devices to igb_uio
the differences between 1.6 and 1.7 are resolved for that part. The reason
why you explicitly need to bind the devices in 1.7 is due to this commit which
removes the pci id table from the igb_uio driver.

http://dpdk.org/browse/dpdk/commit/?id=629395b063e8278a05ea41908d1152fa68df098c

As for the other errors you are seeing, I'm not sure of the cause, though they
may be related to interrupt support for changes in link status. Can you perhaps
use a debugger and find out what the file descriptor in question refers to?

/Bruce

Re: Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Manoj Viswanath]

Bruce,

Thanks for the input.
Sure, will figure out the offending file behind this error and update this
thread.

Meanwhile, wanted to share one more observation regarding this issue:
The "file descriptor error" is NOT SEEN with DPDK 1.7.0 (dpdk-1.7.0.tar.gz
<http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.0.tar.gz>) when explicitly
binding NICs to IGB_UIO using the .py script.

The issue is only seen when using DPDK 1.7.1 ( dpdk-1.7.1.tar.gz
<http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.1.tar.gz>). I hope the
version i am using is the last official tag in the 1.7.x tree.

I see that this problem ("file descriptor error") in DPDK 1.7.1 and its
non-occurrence in DPDK 1.7.0 has already been reported in this group on Sep
11 by zimeiw <zimeiw-9Onoh4P/yGk@public.gmane.org> in a mail with following subject line:- "*There
are a lot of error log when run l3fwd of dpdk-1.7.1*". Couldn't find any
responses to that thread.

Regards,
Manoj

On Mon, Nov 10, 2014 at 4:28 PM, Bruce Richardson <
bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:

> On Fri, Nov 07, 2014 at 11:26:08PM +0530, Manoj Viswanath wrote:
> > Hi Bruce,
> >
> > Please find my comment in lined.
> >
> > On Fri, Nov 7, 2014 at 9:00 PM, Bruce Richardson <
> bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
> > > wrote:
> >
> > > On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote:
> > > > Hi Bruce,
> > > >
> > > > I was not doing anything specific for binding the NICs to IGB_UIO
> (like
> > > > invoking "dpdk_nic_bind.py" script explicitly) when using my
> application
> > > > with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the
> VM
> > > were
> > > > automatically getting picked up and initialized by IGB_UIO within
> each
> > > VM.
> > > >
> > > > The same is not working with DPDK 1.7.1 now.
> > > >
> > > > I tried exporting the "dpdk_nic_bind.py" script into my VM (running
> DPDK
> > > > 1.7.1) and tried to check the status. The emulated devices were
> shown as
> > > > neither bound to kernel nor to IGB_UIO as evident from below output:-
> > > >
> > > >
> > >
> <--------------------------------------------------------------------------------------------------->
> > > > Network devices using DPDK-compatible driver
> > > > ============================================
> > > > <none>
> > > >
> > > > Network devices using kernel driver
> > > > ===================================
> > > > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci
> unused=igb_uio
> > > >
> > > > Other network devices
> > > > =====================
> > > > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > > > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > > >
> > >
> <--------------------------------------------------------------------------------------------------->
> > > >
> > > > When i tried to forcefully bind the NICs using the "--bind=igb_uio"
> > > option
> > >
> > > Was there any output of the dpdk_nic_bind script? What does the output
> of
> > > it with --status show afterwards?
> > >
> > > ​
> > [MANOJ]​
> >
> > ​Yes. Please refer below output:-
> > ********************************************************************
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> >
> > Other network devices
> > =====================
> > <none>​
> > ********************************************************************
> >
> > ​However, when i start the DPDK application, i am getting the error log
> as
> > indicated in earlier mail. ​
> >
> > The difference with DPDK 1.6.1 is that at the same stage IGB_UIO has
> > already bound the assigned devices without having to explicitly run the
> > "dpdk_nic_bind.py". Please find below the application log when run with
> > DPDK 1.6.0:-
> >
> > ​********************************************************************
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> > 0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> >
> > Other network devices
> > =====================
> > <none>
> > ********************************************************************​
> >
> > ​Kindly note that in both cases, logs have been taken after loading
> IGB_UIO
> > prior to starting DPDK application. ​
> >
> > ​[/MANOJ]​
> >
> > Regards,
>
> Ok, so it appears that after running dpdk_nic_bind to bind the devices to
> igb_uio
> the differences between 1.6 and 1.7 are resolved for that part. The reason
> why you explicitly need to bind the devices in 1.7 is due to this commit
> which
> removes the pci id table from the igb_uio driver.
>
>
> http://dpdk.org/browse/dpdk/commit/?id=629395b063e8278a05ea41908d1152fa68df098c
>
> As for the other errors you are seeing, I'm not sure of the cause, though
> they
> may be related to interrupt support for changes in link status. Can you
> perhaps
> use a debugger and find out what the file descriptor in question refers to?
>
> /Bruce
>

Re: Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment

[Bruce Richardson]

On Tue, Nov 11, 2014 at 07:18:32PM +0530, Manoj Viswanath wrote:
> Bruce,
> 
> Thanks for the input.
> Sure, will figure out the offending file behind this error and update this
> thread.
> 
> Meanwhile, wanted to share one more observation regarding this issue:
> The "file descriptor error" is NOT SEEN with DPDK 1.7.0 (dpdk-1.7.0.tar.gz
> <http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.0.tar.gz>) when explicitly
> binding NICs to IGB_UIO using the .py script.
> 
> The issue is only seen when using DPDK 1.7.1 ( dpdk-1.7.1.tar.gz
> <http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.1.tar.gz>). I hope the
> version i am using is the last official tag in the 1.7.x tree.
> 

Given that it's narrowed down to a change between 1.7.0 and 1.7.1 would it also
be possible for you to use git bisect to track down the commit that introduces
the error?

/Bruce