Problems when testing l2fwd with ixgbevf pmd dirver

Problems when testing l2fwd with ixgbevf pmd dirver

[Lilijun]

Hi all,

I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
But the VF can't be used when doing probe pmd driver with following error:

EAL: PCI device 0000:02:10.0 on NUMA socket -1
EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
EAL:   PCI memory mapped at 0x7f78b181f000
EAL:   PCI memory mapped at 0x7f78b181b000
EAL: Error - exiting with code: 1
  Cause: Requested device 0000:02:10.0 cannot be used

Any suggestions?

Thanks,
Jerry

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Zhang, Helin]

Hi

> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
> Sent: Thursday, October 23, 2014 11:36 AM
> To: dev-VfR2kkLFssw@public.gmane.org
> Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
> 
> Hi all,
> 
> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
> But the VF can't be used when doing probe pmd driver with following error:
> 
> EAL: PCI device 0000:02:10.0 on NUMA socket -1
> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
> EAL:   PCI memory mapped at 0x7f78b181f000
> EAL:   PCI memory mapped at 0x7f78b181b000
> EAL: Error - exiting with code: 1
>   Cause: Requested device 0000:02:10.0 cannot be used
> 
> Any suggestions?
Did you launch DPDK host application?

> 
> Thanks,
> Jerry

Regards,
Helin

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Lilijun]

On 2014/10/23 11:44, Zhang, Helin wrote:
> Hi
> 
>> -----Original Message-----
>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
>> Sent: Thursday, October 23, 2014 11:36 AM
>> To: dev-VfR2kkLFssw@public.gmane.org
>> Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
>>
>> Hi all,
>>
>> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
>> But the VF can't be used when doing probe pmd driver with following error:
>>
>> EAL: PCI device 0000:02:10.0 on NUMA socket -1
>> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
>> EAL:   PCI memory mapped at 0x7f78b181f000
>> EAL:   PCI memory mapped at 0x7f78b181b000
>> EAL: Error - exiting with code: 1
>>   Cause: Requested device 0000:02:10.0 cannot be used
>>
>> Any suggestions?
> Did you launch DPDK host application?

My test steps:
1) rmmod ixgbe; modprobe ixgbe max_vfs=3,3
2) insmod dpdk-1.7.0/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
3) dpdk_nic_bind.py -b igb_uio 02:10.0 02:10.1
4) ./build/l2fwd -c 0xf -n 3 -- -p 3 -q 1

Then l2fwd run failed with error messages shown as above email.

Is my test steps correctly?

> 
>>
>> Thanks,
>> Jerry
> 
> Regards,
> Helin
> 
> 

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Zhang, Helin]

> -----Original Message-----
> From: Lilijun [mailto:jerry.lilijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
> Sent: Thursday, October 23, 2014 12:31 PM
> To: Zhang, Helin; dev-VfR2kkLFssw@public.gmane.org
> Subject: Re: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
> 
> On 2014/10/23 11:44, Zhang, Helin wrote:
> > Hi
> >
> >> -----Original Message-----
> >> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
> >> Sent: Thursday, October 23, 2014 11:36 AM
> >> To: dev-VfR2kkLFssw@public.gmane.org
> >> Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
> >>
> >> Hi all,
> >>
> >> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
> >> But the VF can't be used when doing probe pmd driver with following error:
> >>
> >> EAL: PCI device 0000:02:10.0 on NUMA socket -1
> >> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
> >> EAL:   PCI memory mapped at 0x7f78b181f000
> >> EAL:   PCI memory mapped at 0x7f78b181b000
> >> EAL: Error - exiting with code: 1
> >>   Cause: Requested device 0000:02:10.0 cannot be used
> >>
> >> Any suggestions?
> > Did you launch DPDK host application?
> 
> My test steps:
> 1) rmmod ixgbe; modprobe ixgbe max_vfs=3,3
> 2) insmod dpdk-1.7.0/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
> 3) dpdk_nic_bind.py -b igb_uio 02:10.0 02:10.1
> 4) ./build/l2fwd -c 0xf -n 3 -- -p 3 -q 1
> 
> Then l2fwd run failed with error messages shown as above email.
All done in guest, right?
If yes, you need to check if host runs in host. It could be standard Linux host driver, or DPDK host driver.

> 
> Is my test steps correctly?
> 
> >
> >>
> >> Thanks,
> >> Jerry
> >
> > Regards,
> > Helin
> >
> >
> 

Regards,
Helin

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Lilijun]

On 2014/10/23 12:56, Zhang, Helin wrote:
> 
> 
>> -----Original Message-----
>> From: Lilijun [mailto:jerry.lilijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
>> Sent: Thursday, October 23, 2014 12:31 PM
>> To: Zhang, Helin; dev-VfR2kkLFssw@public.gmane.org
>> Subject: Re: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
>>
>> On 2014/10/23 11:44, Zhang, Helin wrote:
>>> Hi
>>>
>>>> -----Original Message-----
>>>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
>>>> Sent: Thursday, October 23, 2014 11:36 AM
>>>> To: dev-VfR2kkLFssw@public.gmane.org
>>>> Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
>>>>
>>>> Hi all,
>>>>
>>>> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
>>>> But the VF can't be used when doing probe pmd driver with following error:
>>>>
>>>> EAL: PCI device 0000:02:10.0 on NUMA socket -1
>>>> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
>>>> EAL:   PCI memory mapped at 0x7f78b181f000
>>>> EAL:   PCI memory mapped at 0x7f78b181b000
>>>> EAL: Error - exiting with code: 1
>>>>   Cause: Requested device 0000:02:10.0 cannot be used
>>>>
>>>> Any suggestions?
>>> Did you launch DPDK host application?
>>
>> My test steps:
>> 1) rmmod ixgbe; modprobe ixgbe max_vfs=3,3
>> 2) insmod dpdk-1.7.0/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
>> 3) dpdk_nic_bind.py -b igb_uio 02:10.0 02:10.1
>> 4) ./build/l2fwd -c 0xf -n 3 -- -p 3 -q 1
>>
>> Then l2fwd run failed with error messages shown as above email.
> All done in guest, right?
> If yes, you need to check if host runs in host. It could be standard Linux host driver, or DPDK host driver.

Thanks for the reply.
The example application runs in host. Now the problem has been fixed for that I haven't make my PF NIC up yet.
> 
>>
>> Is my test steps correctly?
>>
>>>
>>>>
>>>> Thanks,
>>>> Jerry
>>>
>>> Regards,
>>> Helin
>>>
>>>
>>
> 
> Regards,
> Helin
> 
> 
> 

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Dong, Binghua]

Hi all,

It looks like that Lilijun runs the four steps are both in HOST?

Lilijun, you did not startup a VM, pass through the VF to VM, and runs l2fwd in VM?

-----Original Message-----
From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
Sent: Thursday, October 23, 2014 12:31 PM
To: Zhang, Helin; dev-VfR2kkLFssw@public.gmane.org
Subject: Re: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver

On 2014/10/23 11:44, Zhang, Helin wrote:
> Hi
> 
>> -----Original Message-----
>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
>> Sent: Thursday, October 23, 2014 11:36 AM
>> To: dev-VfR2kkLFssw@public.gmane.org
>> Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
>>
>> Hi all,
>>
>> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
>> But the VF can't be used when doing probe pmd driver with following error:
>>
>> EAL: PCI device 0000:02:10.0 on NUMA socket -1
>> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
>> EAL:   PCI memory mapped at 0x7f78b181f000
>> EAL:   PCI memory mapped at 0x7f78b181b000
>> EAL: Error - exiting with code: 1
>>   Cause: Requested device 0000:02:10.0 cannot be used
>>
>> Any suggestions?
> Did you launch DPDK host application?

My test steps:
1) rmmod ixgbe; modprobe ixgbe max_vfs=3,3
2) insmod dpdk-1.7.0/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
3) dpdk_nic_bind.py -b igb_uio 02:10.0 02:10.1
4) ./build/l2fwd -c 0xf -n 3 -- -p 3 -q 1

Then l2fwd run failed with error messages shown as above email.

Is my test steps correctly?

> 
>>
>> Thanks,
>> Jerry
> 
> Regards,
> Helin
> 
> 

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Lilijun]

On 2014/10/23 13:52, Dong, Binghua wrote:
> Hi all,
> 
> It looks like that Lilijun runs the four steps are both in HOST?
> 
> Lilijun, you did not startup a VM, pass through the VF to VM, and runs l2fwd in VM?

Yes, that's all in HOST. The problem has been fixed.

> 
> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
> Sent: Thursday, October 23, 2014 12:31 PM
> To: Zhang, Helin; dev-VfR2kkLFssw@public.gmane.org
> Subject: Re: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
> 
> On 2014/10/23 11:44, Zhang, Helin wrote:
>> Hi
>>
>>> -----Original Message-----
>>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
>>> Sent: Thursday, October 23, 2014 11:36 AM
>>> To: dev-VfR2kkLFssw@public.gmane.org
>>> Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
>>>
>>> Hi all,
>>>
>>> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
>>> But the VF can't be used when doing probe pmd driver with following error:
>>>
>>> EAL: PCI device 0000:02:10.0 on NUMA socket -1
>>> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
>>> EAL:   PCI memory mapped at 0x7f78b181f000
>>> EAL:   PCI memory mapped at 0x7f78b181b000
>>> EAL: Error - exiting with code: 1
>>>   Cause: Requested device 0000:02:10.0 cannot be used
>>>
>>> Any suggestions?
>> Did you launch DPDK host application?
> 
> My test steps:
> 1) rmmod ixgbe; modprobe ixgbe max_vfs=3,3
> 2) insmod dpdk-1.7.0/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
> 3) dpdk_nic_bind.py -b igb_uio 02:10.0 02:10.1
> 4) ./build/l2fwd -c 0xf -n 3 -- -p 3 -q 1
> 
> Then l2fwd run failed with error messages shown as above email.
> 
> Is my test steps correctly?
> 
>>
>>>
>>> Thanks,
>>> Jerry
>>
>> Regards,
>> Helin
>>
>>
> 
> 
> 
> 

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Dong, Binghua]

Hi Lilijun,

I think you need unbind the device 02:10.0 from driver ixgbevf first; In Host Linux, you also need ifconfig up the PF eth of this VF;
Please ref my script to unbind a VF device and pass into VM, such as:
PCI_eth="0000:02:10.0"
ID_eth="ffff ffff"
PCIX_eth=" -device pci-assign,host="${PCI_eth}
echo $ID_eth > /sys/bus/pci/drivers/pci-stub/new_id
echo ${PCI_eth} > /sys/bus/pci/devices/${PCI_eth}/driver/unbind
echo ${PCI_eth} > /sys/bus/pci/drivers/pci-stub/bind
CMD=${CMD}${PCIX_eth}

==============================================================================================
#!/bin/bash
ID="8086 0442"
PCI="0000:83:01"
IMG="/root/test_image/sles11sp2.img"
HOST=" -vnc 172.16.200.166:1 "
PCIX=" -device pci-assign,host="${PCI}
CDROM=" -cdrom /root/QAT/driver.iso "


CMD="qemu-system-x86_64 -m 2048 -smp 4 -hda "${IMG}${HOST}" -enable-kvm "
#ADD CCK
if [ 1 -eq 0 ];then
echo $ID > /sys/bus/pci/drivers/pci-stub/new_id
for i in 4 5 6 7
do
echo ${PCI}"."$i > /sys/bus/pci/devices/${PCI}"."$i/driver/unbind
echo ${PCI}"."$i > /sys/bus/pci/drivers/pci-stub/bind
CMD=${CMD}${PCIX}.$i
done
fi

#ADD ETH
if [ 1 -eq 0 ];then
PCI_eth="0000:02:00.1"
ID_eth="8086 1521"
PCIX_eth=" -device pci-assign,host="${PCI_eth}
echo $ID_eth > /sys/bus/pci/drivers/pci-stub/new_id
echo ${PCI_eth} > /sys/bus/pci/devices/${PCI_eth}/driver/unbind
echo ${PCI_eth} > /sys/bus/pci/drivers/pci-stub/bind
CMD=${CMD}${PCIX_eth}
fi

#ADD CDROM
#CMD=${CMD}${CDROM}
echo $CMD

#rmmod kvm_intel
#rmmod kvm
modprobe kvm
modprobe kvm_intel
$CMD


-----Original Message-----
From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
Sent: Thursday, October 23, 2014 11:36 AM
To: dev-VfR2kkLFssw@public.gmane.org
Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver

Hi all,

I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
But the VF can't be used when doing probe pmd driver with following error:

EAL: PCI device 0000:02:10.0 on NUMA socket -1
EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
EAL:   PCI memory mapped at 0x7f78b181f000
EAL:   PCI memory mapped at 0x7f78b181b000
EAL: Error - exiting with code: 1
  Cause: Requested device 0000:02:10.0 cannot be used

Any suggestions?

Thanks,
Jerry

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Lilijun]

On 2014/10/23 13:37, Dong, Binghua wrote:
> Hi Lilijun,
> 
> I think you need unbind the device 02:10.0 from driver ixgbevf first; In Host Linux, you also need ifconfig up the PF eth of this VF;

The script dpdk_nic_bind.py will unbind ixgbevf automatically when doing dpdk_nic_bind.py -b igb_uio 02:10.0.
As you said, the problem is fixed by executing  ifconfig up my PF eth.

Thanks very much.

> Please ref my script to unbind a VF device and pass into VM, such as:
> PCI_eth="0000:02:10.0"
> ID_eth="ffff ffff"
> PCIX_eth=" -device pci-assign,host="${PCI_eth}
> echo $ID_eth > /sys/bus/pci/drivers/pci-stub/new_id
> echo ${PCI_eth} > /sys/bus/pci/devices/${PCI_eth}/driver/unbind
> echo ${PCI_eth} > /sys/bus/pci/drivers/pci-stub/bind
> CMD=${CMD}${PCIX_eth}
> 
> ==============================================================================================
> #!/bin/bash
> ID="8086 0442"
> PCI="0000:83:01"
> IMG="/root/test_image/sles11sp2.img"
> HOST=" -vnc 172.16.200.166:1 "
> PCIX=" -device pci-assign,host="${PCI}
> CDROM=" -cdrom /root/QAT/driver.iso "
> 
> 
> CMD="qemu-system-x86_64 -m 2048 -smp 4 -hda "${IMG}${HOST}" -enable-kvm "
> #ADD CCK
> if [ 1 -eq 0 ];then
> echo $ID > /sys/bus/pci/drivers/pci-stub/new_id
> for i in 4 5 6 7
> do
> echo ${PCI}"."$i > /sys/bus/pci/devices/${PCI}"."$i/driver/unbind
> echo ${PCI}"."$i > /sys/bus/pci/drivers/pci-stub/bind
> CMD=${CMD}${PCIX}.$i
> done
> fi
> 
> #ADD ETH
> if [ 1 -eq 0 ];then
> PCI_eth="0000:02:00.1"
> ID_eth="8086 1521"
> PCIX_eth=" -device pci-assign,host="${PCI_eth}
> echo $ID_eth > /sys/bus/pci/drivers/pci-stub/new_id
> echo ${PCI_eth} > /sys/bus/pci/devices/${PCI_eth}/driver/unbind
> echo ${PCI_eth} > /sys/bus/pci/drivers/pci-stub/bind
> CMD=${CMD}${PCIX_eth}
> fi
> 
> #ADD CDROM
> #CMD=${CMD}${CDROM}
> echo $CMD
> 
> #rmmod kvm_intel
> #rmmod kvm
> modprobe kvm
> modprobe kvm_intel
> $CMD
> 
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Lilijun
> Sent: Thursday, October 23, 2014 11:36 AM
> To: dev-VfR2kkLFssw@public.gmane.org
> Subject: [dpdk-dev] Problems when testing l2fwd with ixgbevf pmd dirver
> 
> Hi all,
> 
> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
> But the VF can't be used when doing probe pmd driver with following error:
> 
> EAL: PCI device 0000:02:10.0 on NUMA socket -1
> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
> EAL:   PCI memory mapped at 0x7f78b181f000
> EAL:   PCI memory mapped at 0x7f78b181b000
> EAL: Error - exiting with code: 1
>   Cause: Requested device 0000:02:10.0 cannot be used
> 
> Any suggestions?
> 
> Thanks,
> Jerry
> 
> 
> .
> 

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Zhang, Jerry]

Hi lijun,

     It seems that one of the PCI BAR mapping failed. Need to identify the accurate code line of the mapping error.
     
     Is this a common issue for all 82599 VFs in your environment or just  one rare  case? If it's just one case, maybe it is caused by configure problem and please try to reconfigure your PF/VF.

     Thank you!

发自我的 iPhone

> 在 2014年10月23日，11:37，Lilijun <jerry.lilijun@huawei.com> 写道：
> 
> Hi all,
> 
> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
> But the VF can't be used when doing probe pmd driver with following error:
> 
> EAL: PCI device 0000:02:10.0 on NUMA socket -1
> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
> EAL:   PCI memory mapped at 0x7f78b181f000
> EAL:   PCI memory mapped at 0x7f78b181b000
> EAL: Error - exiting with code: 1
>  Cause: Requested device 0000:02:10.0 cannot be used
> 
> Any suggestions?
> 
> Thanks,
> Jerry
> 

Re: Problems when testing l2fwd with ixgbevf pmd dirver

[Lilijun]

On 2014/10/23 13:56, Zhang, Jerry wrote:
> Hi lijun,
> 
>      It seems that one of the PCI BAR mapping failed. Need to identify the accurate code line of the mapping error.
>      
>      Is this a common issue for all 82599 VFs in your environment or just  one rare  case? If it's just one case, maybe it is caused by configure problem and please try to reconfigure your PF/VF.

The PF nic has not been up yet, so VF pmd dirver initialize failed.
That's ok now.

Thanks.

> 
>      Thank you!
> 
> 发自我的 iPhone
> 
>> 在 2014年10月23日，11:37，Lilijun <jerry.lilijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org> 写道：
>>
>> Hi all,
>>
>> I am testing the example l2fwd of dpdk-1.7.0 using intel 82599 VF NICs.
>> But the VF can't be used when doing probe pmd driver with following error:
>>
>> EAL: PCI device 0000:02:10.0 on NUMA socket -1
>> EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
>> EAL:   PCI memory mapped at 0x7f78b181f000
>> EAL:   PCI memory mapped at 0x7f78b181b000
>> EAL: Error - exiting with code: 1
>>  Cause: Requested device 0000:02:10.0 cannot be used
>>
>> Any suggestions?
>>
>> Thanks,
>> Jerry
>>