DPDK RSS support for ixgbevf PMD

DPDK RSS support for ixgbevf PMD

[Vladislav Zolotarov]

Hi,
I'm running an ixgbevf PMD on an AWS guests with extended networking
(SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
current git tree there is no support for a multi-queue in this PMD: reta
size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues and
max_tx_queues are both 4.

Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
RSS queues: for some reason it always limits the number of RSS queues by 2.

ixgbevf_main.c: line 2539
u16 rss = min_t(u16, num_online_cpus(), 2);

The above is strange since if MRQE is set to 1010b there are 4 RSS queues
available which seems to be the case in my AWS Guest.

However, let's get back to DPDK. As I've mentioned above the SR-IOV
function i have is RSS capable (to be 100% sure I've verified both queues
are receiving packets in a multi-socket TCP test). And it's a shame I can't
utilize it with a DPDK.

I wonder if there are any blockers to add this capability to the ixgbevf
PMD and if not is it scheduled to some time soon?

Thanks in advance,
vlad

Re: DPDK RSS support for ixgbevf PMD

[Thomas Monjalon]

2014-12-15 21:11, Vladislav Zolotarov:
> Hi,
> I'm running an ixgbevf PMD on an AWS guests with extended networking
> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
> current git tree there is no support for a multi-queue in this PMD: reta
> size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues and
> max_tx_queues are both 4.
> 
> Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
> RSS queues: for some reason it always limits the number of RSS queues by 2.
> 
> ixgbevf_main.c: line 2539
> u16 rss = min_t(u16, num_online_cpus(), 2);
> 
> The above is strange since if MRQE is set to 1010b there are 4 RSS queues
> available which seems to be the case in my AWS Guest.
> 
> However, let's get back to DPDK. As I've mentioned above the SR-IOV
> function i have is RSS capable (to be 100% sure I've verified both queues
> are receiving packets in a multi-socket TCP test). And it's a shame I can't
> utilize it with a DPDK.

Yes, it is not yet supported.
But a patch was recently sent:
	http://dpdk.org/ml/archives/dev/2014-December/010028.html

> I wonder if there are any blockers to add this capability to the ixgbevf
> PMD and if not is it scheduled to some time soon?

Please jump in the above thread for your questions.

My turn to ask a question:
Does it mean DPDK is going to be supported in your system (OSv)?
I've seen it in the OSv roadmap.
Could we have more details about how it would be integrated?

Thanks
-- 
Thomas

Re: DPDK RSS support for ixgbevf PMD

[Dor Laor]

On Mon, Dec 15, 2014 at 10:33 PM, Thomas Monjalon <thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org
> wrote:
>
> 2014-12-15 21:11, Vladislav Zolotarov:
> > Hi,
> > I'm running an ixgbevf PMD on an AWS guests with extended networking
> > (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
> > current git tree there is no support for a multi-queue in this PMD: reta
> > size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues
> and
> > max_tx_queues are both 4.
> >
> > Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
> > RSS queues: for some reason it always limits the number of RSS queues by
> 2.
> >
> > ixgbevf_main.c: line 2539
> > u16 rss = min_t(u16, num_online_cpus(), 2);
> >
> > The above is strange since if MRQE is set to 1010b there are 4 RSS queues
> > available which seems to be the case in my AWS Guest.
> >
> > However, let's get back to DPDK. As I've mentioned above the SR-IOV
> > function i have is RSS capable (to be 100% sure I've verified both queues
> > are receiving packets in a multi-socket TCP test). And it's a shame I
> can't
> > utilize it with a DPDK.
>
> Yes, it is not yet supported.
> But a patch was recently sent:
>         http://dpdk.org/ml/archives/dev/2014-December/010028.html


Good to know and nice timing

>
> t
> > I wonder if there are any blockers to add this capability to the ixgbevf
> > PMD and if not is it scheduled to some time soon?
>
> Please jump in the above thread for your questions.
>
> My turn to ask a question:
> Does it mean DPDK is going to be supported in your system (OSv)?
>

In general yes. It's premature to ask now since there are components
that already implemented in OSv and won't be needed (efficient
producer-consumer, huge pages, etc). We sure like to reuse the driver code,
it's a pity not to contribute to the same code base. We'll certainly like
to let unmodified dpdk apps run as well

Initially we're playing w/ the Linux dpdk on the host and AWS to gain
familiarity w/ it


> I've seen it in the OSv roadmap.
> Could we have more details about how it would be integrated?
>
> Thanks
> --
> Thomas
>

Re: DPDK on OSv (was: RSS for ixgbevf)

[Thomas Monjalon]

2014-12-15 22:45, Dor Laor:
> On Mon, Dec 15, 2014 at 10:33 PM, Thomas Monjalon <thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org
> > wrote:
> > Does it mean DPDK is going to be supported in your system (OSv)?
> 
> In general yes. It's premature to ask now since there are components
> that already implemented in OSv and won't be needed (efficient
> producer-consumer, huge pages, etc). We sure like to reuse the driver code,
> it's a pity not to contribute to the same code base. We'll certainly like
> to let unmodified dpdk apps run as well

I'm not familiar with OSv but perhaps it has a sense to do an OSv integration
in DPDK.
We already have directories linuxapp and bsdapp for kernel-specific stuff like
memory management, PCI access, etc.

> Initially we're playing w/ the Linux dpdk on the host and AWS to gain
> familiarity w/ it
> 
> > I've seen it in the OSv roadmap.
> > Could we have more details about how it would be integrated?

-- 
Thomas

Re: DPDK on OSv (was: RSS for ixgbevf)

[Dor Laor]

On Mon, Dec 15, 2014 at 11:04 PM, Thomas Monjalon <thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org
> wrote:
>
> 2014-12-15 22:45, Dor Laor:
> > On Mon, Dec 15, 2014 at 10:33 PM, Thomas Monjalon <
> thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org
> > > wrote:
> > > Does it mean DPDK is going to be supported in your system (OSv)?
> >
> > In general yes. It's premature to ask now since there are components
> > that already implemented in OSv and won't be needed (efficient
> > producer-consumer, huge pages, etc). We sure like to reuse the driver
> code,
> > it's a pity not to contribute to the same code base. We'll certainly like
> > to let unmodified dpdk apps run as well
>
> I'm not familiar with OSv but perhaps it has a sense to do an OSv
> integration
> in DPDK.
> We already have directories linuxapp and bsdapp for kernel-specific stuff
> like
> memory management, PCI access, etc.
>

Thanks, it will definetly ease the porting


>
> > Initially we're playing w/ the Linux dpdk on the host and AWS to gain
> > familiarity w/ it
> >
> > > I've seen it in the OSv roadmap.
> > > Could we have more details about how it would be integrated?
>
> --
> Thomas
>

Re: DPDK RSS support for ixgbevf PMD

[Vlad Zolotarov]

On 12/15/14 22:33, Thomas Monjalon wrote:
> 2014-12-15 21:11, Vladislav Zolotarov:
>> Hi,
>> I'm running an ixgbevf PMD on an AWS guests with extended networking
>> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
>> current git tree there is no support for a multi-queue in this PMD: reta
>> size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues and
>> max_tx_queues are both 4.
>>
>> Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
>> RSS queues: for some reason it always limits the number of RSS queues by 2.
>>
>> ixgbevf_main.c: line 2539
>> u16 rss = min_t(u16, num_online_cpus(), 2);
>>
>> The above is strange since if MRQE is set to 1010b there are 4 RSS queues
>> available which seems to be the case in my AWS Guest.
>>
>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
>> function i have is RSS capable (to be 100% sure I've verified both queues
>> are receiving packets in a multi-socket TCP test). And it's a shame I can't
>> utilize it with a DPDK.
> Yes, it is not yet supported.
> But a patch was recently sent:
> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html

Applying this patchset seems to break the NIC fast path functionality of 
a AWS Guest NIC.
I'm still debugging it - will update u as soon as I have more specific info.

vlad

>
>> I wonder if there are any blockers to add this capability to the ixgbevf
>> PMD and if not is it scheduled to some time soon?
> Please jump in the above thread for your questions.
>
> My turn to ask a question:
> Does it mean DPDK is going to be supported in your system (OSv)?
> I've seen it in the OSv roadmap.
> Could we have more details about how it would be integrated?
>
> Thanks

Re: DPDK RSS support for ixgbevf PMD

[Ouyang, Changchun]

Hi ,

> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Vlad Zolotarov
> Sent: Tuesday, December 16, 2014 11:36 PM
> To: Thomas Monjalon
> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> 
> 
> On 12/15/14 22:33, Thomas Monjalon wrote:
> > 2014-12-15 21:11, Vladislav Zolotarov:
> >> Hi,
> >> I'm running an ixgbevf PMD on an AWS guests with extended networking
> >> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
> >> current git tree there is no support for a multi-queue in this PMD:
> >> reta size returned by rte_eth_dev_info_get() call is 0, while
> >> max_rx_queues and max_tx_queues are both 4.
> >>
> >> Linux ixgbevf-2.15.3 driver on the other hand successfully
> >> initializes 2 RSS queues: for some reason it always limits the number of
> RSS queues by 2.
> >>
> >> ixgbevf_main.c: line 2539
> >> u16 rss = min_t(u16, num_online_cpus(), 2);
> >>
> >> The above is strange since if MRQE is set to 1010b there are 4 RSS
> >> queues available which seems to be the case in my AWS Guest.
> >>
> >> However, let's get back to DPDK. As I've mentioned above the SR-IOV
> >> function i have is RSS capable (to be 100% sure I've verified both
> >> queues are receiving packets in a multi-socket TCP test). And it's a
> >> shame I can't utilize it with a DPDK.
> > Yes, it is not yet supported.
> > But a patch was recently sent:
> > 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
> 
> Applying this patchset seems to break the NIC fast path functionality of a
> AWS Guest NIC.
> I'm still debugging it - will update u as soon as I have more specific info.
> 
Pls make sure enabling and using 4 queues on guest if you enable vf rss on host and if each vf has 4 queues,
The packet could be received and put in any of 4 queues on guest depend on hash value of the packet.
You could let packet IP address increased continuously and make those packets distribute to different queues.

Thanks
Changchun

Re: DPDK RSS support for ixgbevf PMD

[Vlad Zolotarov]

On 12/17/14 03:03, Ouyang, Changchun wrote:
> Hi ,
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Vlad Zolotarov
>> Sent: Tuesday, December 16, 2014 11:36 PM
>> To: Thomas Monjalon
>> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
>>
>>
>> On 12/15/14 22:33, Thomas Monjalon wrote:
>>> 2014-12-15 21:11, Vladislav Zolotarov:
>>>> Hi,
>>>> I'm running an ixgbevf PMD on an AWS guests with extended networking
>>>> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
>>>> current git tree there is no support for a multi-queue in this PMD:
>>>> reta size returned by rte_eth_dev_info_get() call is 0, while
>>>> max_rx_queues and max_tx_queues are both 4.
>>>>
>>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
>>>> initializes 2 RSS queues: for some reason it always limits the number of
>> RSS queues by 2.
>>>> ixgbevf_main.c: line 2539
>>>> u16 rss = min_t(u16, num_online_cpus(), 2);
>>>>
>>>> The above is strange since if MRQE is set to 1010b there are 4 RSS
>>>> queues available which seems to be the case in my AWS Guest.
>>>>
>>>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
>>>> function i have is RSS capable (to be 100% sure I've verified both
>>>> queues are receiving packets in a multi-socket TCP test). And it's a
>>>> shame I can't utilize it with a DPDK.
>>> Yes, it is not yet supported.
>>> But a patch was recently sent:
>>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
>> Applying this patchset seems to break the NIC fast path functionality of a
>> AWS Guest NIC.
>> I'm still debugging it - will update u as soon as I have more specific info.

Hi, thanks for tips but I have a question below.

> Pls make sure enabling and using 4 queues on guest

Why should I enable all 4? Won't any number below or equal 4 work if (!) 
4 queues are available?
Note that there is a 82599 mode when only 2 RSS queues are available per 
VF: MRQE=1011b.

>   if you enable vf rss on host and if each vf has 4 queues,
> The packet could be received and put in any of 4 queues on guest depend on hash value of the packet.
> You could let packet IP address increased continuously and make those packets distribute to different queues.
>
> Thanks
> Changchun

Re: DPDK RSS support for ixgbevf PMD

[Ouyang, Changchun]

Hi

> -----Original Message-----
> From: Vlad Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org]
> Sent: Wednesday, December 17, 2014 4:47 PM
> To: Ouyang, Changchun; Thomas Monjalon
> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> 
> 
> On 12/17/14 03:03, Ouyang, Changchun wrote:
> > Hi ,
> >
> >> -----Original Message-----
> >> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Vlad Zolotarov
> >> Sent: Tuesday, December 16, 2014 11:36 PM
> >> To: Thomas Monjalon
> >> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
> >> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> >>
> >>
> >> On 12/15/14 22:33, Thomas Monjalon wrote:
> >>> 2014-12-15 21:11, Vladislav Zolotarov:
> >>>> Hi,
> >>>> I'm running an ixgbevf PMD on an AWS guests with extended
> >>>> networking (SR-IOV functions of 82599 Intel's NIC) and noticed that
> >>>> even in the current git tree there is no support for a multi-queue in this
> PMD:
> >>>> reta size returned by rte_eth_dev_info_get() call is 0, while
> >>>> max_rx_queues and max_tx_queues are both 4.
> >>>>
> >>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
> >>>> initializes 2 RSS queues: for some reason it always limits the
> >>>> number of
> >> RSS queues by 2.
> >>>> ixgbevf_main.c: line 2539
> >>>> u16 rss = min_t(u16, num_online_cpus(), 2);
> >>>>
> >>>> The above is strange since if MRQE is set to 1010b there are 4 RSS
> >>>> queues available which seems to be the case in my AWS Guest.
> >>>>
> >>>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
> >>>> function i have is RSS capable (to be 100% sure I've verified both
> >>>> queues are receiving packets in a multi-socket TCP test). And it's
> >>>> a shame I can't utilize it with a DPDK.
> >>> Yes, it is not yet supported.
> >>> But a patch was recently sent:
> >>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
> >> Applying this patchset seems to break the NIC fast path functionality
> >> of a AWS Guest NIC.
> >> I'm still debugging it - will update u as soon as I have more specific info.
> 
> Hi, thanks for tips but I have a question below.
> 
> > Pls make sure enabling and using 4 queues on guest
> 
> Why should I enable all 4? Won't any number below or equal 4 work if (!)
> 4 queues are available?
> Note that there is a 82599 mode when only 2 RSS queues are available per
> VF: MRQE=1011b.
> 

Yes you are right, it could be 2 queues, the actual queue number per VF depends on the number of VF:
VF number from 1~32:     4 queues per VF;
VF number from 33~64:   2 queues per VF;
The queue number is determined by the vf number at the pf init stage,
If pf setup 4 queues for each vf and distribute packets to 4 queues, but vf only poll 1 or 2 queue,
Then packets in other queues will be lost.
You can try to poll 4 queues firstly to debug your issue.
Thanks
Changchun

Re: DPDK RSS support for ixgbevf PMD

[Vlad Zolotarov]

On 12/18/14 03:32, Ouyang, Changchun wrote:
> Hi
>
>> -----Original Message-----
>> From: Vlad Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org]
>> Sent: Wednesday, December 17, 2014 4:47 PM
>> To: Ouyang, Changchun; Thomas Monjalon
>> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
>>
>>
>> On 12/17/14 03:03, Ouyang, Changchun wrote:
>>> Hi ,
>>>
>>>> -----Original Message-----
>>>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Vlad Zolotarov
>>>> Sent: Tuesday, December 16, 2014 11:36 PM
>>>> To: Thomas Monjalon
>>>> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
>>>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
>>>>
>>>>
>>>> On 12/15/14 22:33, Thomas Monjalon wrote:
>>>>> 2014-12-15 21:11, Vladislav Zolotarov:
>>>>>> Hi,
>>>>>> I'm running an ixgbevf PMD on an AWS guests with extended
>>>>>> networking (SR-IOV functions of 82599 Intel's NIC) and noticed that
>>>>>> even in the current git tree there is no support for a multi-queue in this
>> PMD:
>>>>>> reta size returned by rte_eth_dev_info_get() call is 0, while
>>>>>> max_rx_queues and max_tx_queues are both 4.
>>>>>>
>>>>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
>>>>>> initializes 2 RSS queues: for some reason it always limits the
>>>>>> number of
>>>> RSS queues by 2.
>>>>>> ixgbevf_main.c: line 2539
>>>>>> u16 rss = min_t(u16, num_online_cpus(), 2);
>>>>>>
>>>>>> The above is strange since if MRQE is set to 1010b there are 4 RSS
>>>>>> queues available which seems to be the case in my AWS Guest.
>>>>>>
>>>>>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
>>>>>> function i have is RSS capable (to be 100% sure I've verified both
>>>>>> queues are receiving packets in a multi-socket TCP test). And it's
>>>>>> a shame I can't utilize it with a DPDK.
>>>>> Yes, it is not yet supported.
>>>>> But a patch was recently sent:
>>>>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
>>>> Applying this patchset seems to break the NIC fast path functionality
>>>> of a AWS Guest NIC.
>>>> I'm still debugging it - will update u as soon as I have more specific info.
>> Hi, thanks for tips but I have a question below.
>>
>>> Pls make sure enabling and using 4 queues on guest
>> Why should I enable all 4? Won't any number below or equal 4 work if (!)
>> 4 queues are available?
>> Note that there is a 82599 mode when only 2 RSS queues are available per
>> VF: MRQE=1011b.
>>
> Yes you are right, it could be 2 queues, the actual queue number per VF depends on the number of VF:
> VF number from 1~32:     4 queues per VF;
> VF number from 33~64:   2 queues per VF;
> The queue number is determined by the vf number at the pf init stage,
> If pf setup 4 queues for each vf and distribute packets to 4 queues, but vf only poll 1 or 2 queue,
> Then packets in other queues will be lost.
> You can try to poll 4 queues firstly to debug your issue.

Sure. When I hardcoded the queues number to 4 all works like a charm. 
The problem is that we can't leave it this way. I've posted some 
comments to your patches (0 and 5). Since I've joined the list after the 
patches have been sent I had to paste the patches bodies into my email 
and they will most like not appear in the thread of the original series. 
I hope this won't be a big problem... ;)

vlad

> Thanks
> Changchun
>

Re: DPDK RSS support for ixgbevf PMD

[Ouyang, Changchun]

Hi ,

> -----Original Message-----
> From: Vlad Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org]
> Sent: Thursday, December 18, 2014 6:34 PM
> To: Ouyang, Changchun; Thomas Monjalon
> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> 
> 
> On 12/18/14 03:32, Ouyang, Changchun wrote:
> > Hi
> >
> >> -----Original Message-----
> >> From: Vlad Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org]
> >> Sent: Wednesday, December 17, 2014 4:47 PM
> >> To: Ouyang, Changchun; Thomas Monjalon
> >> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
> >> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> >>
> >>
> >> On 12/17/14 03:03, Ouyang, Changchun wrote:
> >>> Hi ,
> >>>
> >>>> -----Original Message-----
> >>>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Vlad
> Zolotarov
> >>>> Sent: Tuesday, December 16, 2014 11:36 PM
> >>>> To: Thomas Monjalon
> >>>> Cc: dev-VfR2kkLFssw@public.gmane.org; Avi Kivity; Gleb Natapov
> >>>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> >>>>
> >>>>
> >>>> On 12/15/14 22:33, Thomas Monjalon wrote:
> >>>>> 2014-12-15 21:11, Vladislav Zolotarov:
> >>>>>> Hi,
> >>>>>> I'm running an ixgbevf PMD on an AWS guests with extended
> >>>>>> networking (SR-IOV functions of 82599 Intel's NIC) and noticed
> >>>>>> that even in the current git tree there is no support for a
> >>>>>> multi-queue in this
> >> PMD:
> >>>>>> reta size returned by rte_eth_dev_info_get() call is 0, while
> >>>>>> max_rx_queues and max_tx_queues are both 4.
> >>>>>>
> >>>>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
> >>>>>> initializes 2 RSS queues: for some reason it always limits the
> >>>>>> number of
> >>>> RSS queues by 2.
> >>>>>> ixgbevf_main.c: line 2539
> >>>>>> u16 rss = min_t(u16, num_online_cpus(), 2);
> >>>>>>
> >>>>>> The above is strange since if MRQE is set to 1010b there are 4
> >>>>>> RSS queues available which seems to be the case in my AWS Guest.
> >>>>>>
> >>>>>> However, let's get back to DPDK. As I've mentioned above the
> >>>>>> SR-IOV function i have is RSS capable (to be 100% sure I've
> >>>>>> verified both queues are receiving packets in a multi-socket TCP
> >>>>>> test). And it's a shame I can't utilize it with a DPDK.
> >>>>> Yes, it is not yet supported.
> >>>>> But a patch was recently sent:
> >>>>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
> >>>> Applying this patchset seems to break the NIC fast path
> >>>> functionality of a AWS Guest NIC.
> >>>> I'm still debugging it - will update u as soon as I have more specific info.
> >> Hi, thanks for tips but I have a question below.
> >>
> >>> Pls make sure enabling and using 4 queues on guest
> >> Why should I enable all 4? Won't any number below or equal 4 work if
> >> (!)
> >> 4 queues are available?
> >> Note that there is a 82599 mode when only 2 RSS queues are available
> >> per
> >> VF: MRQE=1011b.
> >>
> > Yes you are right, it could be 2 queues, the actual queue number per VF
> depends on the number of VF:
> > VF number from 1~32:     4 queues per VF;
> > VF number from 33~64:   2 queues per VF;
> > The queue number is determined by the vf number at the pf init stage,
> > If pf setup 4 queues for each vf and distribute packets to 4 queues,
> > but vf only poll 1 or 2 queue, Then packets in other queues will be lost.
> > You can try to poll 4 queues firstly to debug your issue.
> 
> Sure. When I hardcoded the queues number to 4 all works like a charm.
> The problem is that we can't leave it this way. I've posted some comments to
> your patches (0 and 5). Since I've joined the list after the patches have been
> sent I had to paste the patches bodies into my email and they will most like
> not appear in the thread of the original series.
> I hope this won't be a big problem... ;)
> 
Yes, I have seen your comments there, and respond it already, let's use that thread to discuss from now on.

Thanks
Changchun