Re: [PATCH 0/6] Enable VF RSS for Niantic

Re: [PATCH 0/6] Enable VF RSS for Niantic

[Vlad Zolotarov]

On 12/18/14 12:11, Vlad Zolotarov wrote:
> From: Changchun Ouyang <changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>
> This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues.
> The actual queue number per VF depends on the number of VF:
> VF number from 1~32: 4 queues per VF;
> VF number from 33~max vf num: 2 queues per VF;
>
> On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS
> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated.
> It also needs config VF RSS information like hash function, RSS key, RSS key length.

This patch series is missing a few things:

 1. Taking into the consideration the number of Rx queues requested by a
    user in the rte_eth_dev_configure().
 2. dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is
    still not initialized for a VF. Thus there is no way to query the
    RSS table contents.
 3. rte_eth_dev_info_get() returns reta_size == 0 when called for a VF
    function.

thanks,
vlad
>
> Changchun Ouyang (6):
>    ixgbe: Code cleanup
>    ixgbe: Negotiate VF API version
>    ixgbe: Get VF queue number
>    ether: Check VMDq RSS mode
>    ixgbe: Config VF RSS
>    testpmd: Set Rx VMDq RSS mode
>
>   app/test-pmd/testpmd.c              |  9 ++++
>   lib/librte_ether/rte_ethdev.c       | 21 ++++++--
>   lib/librte_pmd_ixgbe/ixgbe_ethdev.h |  1 +
>   lib/librte_pmd_ixgbe/ixgbe_pf.c     | 75 ++++++++++++++++++++++++++++-
>   lib/librte_pmd_ixgbe/ixgbe_rxtx.c   | 95 +++++++++++++++++++++++++++----------
>   5 files changed, 171 insertions(+), 30 deletions(-)
>
> --

Re: [PATCH 0/6] Enable VF RSS for Niantic

[Ouyang, Changchun]

My response as below.

From: Vlad Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org]
Sent: Thursday, December 18, 2014 9:06 PM
To: dev-VfR2kkLFssw@public.gmane.org; Ouyang, Changchun
Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic


On 12/18/14 12:11, Vlad Zolotarov wrote:

From: Changchun Ouyang <changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org><mailto:changchun.ouyang@intel.com>



This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues.

The actual queue number per VF depends on the number of VF:

VF number from 1~32: 4 queues per VF;

VF number from 33~max vf num: 2 queues per VF;



On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS

or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated.

It also needs config VF RSS information like hash function, RSS key, RSS key length.

This patch series is missing a few things:

  1.  Taking into the consideration the number of Rx queues requested by a user in the rte_eth_dev_configure().
Changchun: yes, will have a v2 to consider it, as I say in my previous response.

  1.  dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is still not initialized for a VF. Thus there is no way to query the RSS table
Changchun: do you mean query rss table on guest? Niantic doesn't have separate reta for a specific vf, the reta is shared by pf and all vf.
Do you think of returning all reta contents for each vf make sense?  Or any other insight here?

  1.  rte_eth_dev_info_get() returns reta_size == 0 when called for a VF function.
Changchun:  same as above.
Your insight here are welcome here.
thanks
Changchun

Re: [PATCH 0/6] Enable VF RSS for Niantic

[Vladislav Zolotarov]

On Dec 19, 2014 3:35 AM, "Ouyang, Changchun" <changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
wrote:
>
> My response as below.
>
>
>
> From: Vlad Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org]
> Sent: Thursday, December 18, 2014 9:06 PM
> To: dev-VfR2kkLFssw@public.gmane.org; Ouyang, Changchun
> Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic
>
>
>
>
>
> On 12/18/14 12:11, Vlad Zolotarov wrote:
>>
>> From: Changchun Ouyang <changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>>
>>
>>
>> This patch enables VF RSS for Niantic, which allow each VF having at
most 4 queues.
>>
>> The actual queue number per VF depends on the number of VF:
>>
>> VF number from 1~32: 4 queues per VF;
>>
>> VF number from 33~max vf num: 2 queues per VF;
>>
>>
>>
>> On host, to enable VF RSS functionality, mq mode should be set as
ETH_MQ_RX_VMDQ_RSS
>>
>> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated.
>>
>> It also needs config VF RSS information like hash function, RSS key, RSS
key length.
>
>
> This patch series is missing a few things:
>
> Taking into the consideration the number of Rx queues requested by a user
in the rte_eth_dev_configure().
>
> Changchun: yes, will have a v2 to consider it, as I say in my previous
response.

I must have joined the mailing list after your v2 series. I'll take a look
in the web for it.

>
> dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is still
not initialized for a VF. Thus there is no way to query the RSS table
>
> Changchun: do you mean query rss table on guest?

Of course.

Niantic doesn’t have separate reta for a specific vf, the reta is shared by
pf and all vf.

AFAIK the reta is statically divided between all available functions.

So, first of all we need a way to know that is the size of a single VF
partition in order to know how many RSS queues may be configured.

Then we need an ability to query/set the contents of the corresponding reta
partition like we can for PF.

>
> Do you think of returning all reta contents for each vf make sense?  Or
any other insight here?
>
> rte_eth_dev_info_get() returns reta_size == 0 when called for a VF
function.
>
> Changchun:  same as above.
>
> Your insight here are welcome here.
>
> thanks
>
> Changchun
>
>

Re: [PATCH 0/6] Enable VF RSS for Niantic

[Ouyang, Changchun]

Hi,
I have v3 patch for this to resolve your comments,
We can discuss on v3 patch if there are further comments.
Thanks
Changchun

From: Vladislav Zolotarov [mailto:vladz@cloudius-systems.com]
Sent: Sunday, December 21, 2014 4:02 PM
To: Ouyang, Changchun
Cc: dev@dpdk.org
Subject: RE: [PATCH 0/6] Enable VF RSS for Niantic


On Dec 19, 2014 3:35 AM, "Ouyang, Changchun" <changchun.ouyang@intel.com<mailto:changchun.ouyang@intel.com>> wrote:
>
> My response as below.
>
>
>
> From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com<mailto:vladz@cloudius-systems.com>]
> Sent: Thursday, December 18, 2014 9:06 PM
> To: dev@dpdk.org<mailto:dev@dpdk.org>; Ouyang, Changchun
> Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic
>
>
>
>
>
> On 12/18/14 12:11, Vlad Zolotarov wrote:
>>
>> From: Changchun Ouyang <changchun.ouyang@intel.com<mailto:changchun.ouyang@intel.com>>
>>
>>
>>
>> This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues.
>>
>> The actual queue number per VF depends on the number of VF:
>>
>> VF number from 1~32: 4 queues per VF;
>>
>> VF number from 33~max vf num: 2 queues per VF;
>>
>>
>>
>> On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS
>>
>> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated.
>>
>> It also needs config VF RSS information like hash function, RSS key, RSS key length.
>
>
> This patch series is missing a few things:
>
> Taking into the consideration the number of Rx queues requested by a user in the rte_eth_dev_configure().
>
> Changchun: yes, will have a v2 to consider it, as I say in my previous response.

I must have joined the mailing list after your v2 series. I'll take a look in the web for it.

>
> dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is still not initialized for a VF. Thus there is no way to query the RSS table
>
> Changchun: do you mean query rss table on guest?

Of course.

Niantic doesn’t have separate reta for a specific vf, the reta is shared by pf and all vf.

AFAIK the reta is statically divided between all available functions.

So, first of all we need a way to know that is the size of a single VF partition in order to know how many RSS queues may be configured.

Then we need an ability to query/set the contents of the corresponding reta partition like we can for PF.

>
> Do you think of returning all reta contents for each vf make sense?  Or any other insight here?
>
> rte_eth_dev_info_get() returns reta_size == 0 when called for a VF function.
>
> Changchun:  same as above.
>
> Your insight here are welcome here.
>
> thanks
>
> Changchun
>
>

Re: [PATCH 0/6] Enable VF RSS for Niantic

[Vlad Zolotarov]

On 12/24/14 08:37, Ouyang, Changchun wrote:
>
> Hi,
>
> I have v3 patch for this to resolve your comments,
>
> We can discuss on v3 patch if there are further comments.
>

I'm looking at your v3 patchset right now... ;)

> Thanks
>
> Changchun
>
> *From:*Vladislav Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org]
> *Sent:* Sunday, December 21, 2014 4:02 PM
> *To:* Ouyang, Changchun
> *Cc:* dev-VfR2kkLFssw@public.gmane.org
> *Subject:* RE: [PATCH 0/6] Enable VF RSS for Niantic
>
>
> On Dec 19, 2014 3:35 AM, "Ouyang, Changchun" 
> <changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org <mailto:changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>> wrote:
> >
> > My response as below.
> >
> >
> >
> > From: Vlad Zolotarov [mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org 
> <mailto:vladz-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org>]
> > Sent: Thursday, December 18, 2014 9:06 PM
> > To: dev-VfR2kkLFssw@public.gmane.org <mailto:dev-VfR2kkLFssw@public.gmane.org>; Ouyang, Changchun
> > Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic
> >
> >
> >
> >
> >
> > On 12/18/14 12:11, Vlad Zolotarov wrote:
> >>
> >> From: Changchun Ouyang <changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org 
> <mailto:changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>>
> >>
> >>
> >>
> >> This patch enables VF RSS for Niantic, which allow each VF having 
> at most 4 queues.
> >>
> >> The actual queue number per VF depends on the number of VF:
> >>
> >> VF number from 1~32: 4 queues per VF;
> >>
> >> VF number from 33~max vf num: 2 queues per VF;
> >>
> >>
> >>
> >> On host, to enable VF RSS functionality, mq mode should be set as 
> ETH_MQ_RX_VMDQ_RSS
> >>
> >> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated.
> >>
> >> It also needs config VF RSS information like hash function, RSS 
> key, RSS key length.
> >
> >
> > This patch series is missing a few things:
> >
> > Taking into the consideration the number of Rx queues requested by a 
> user in the rte_eth_dev_configure().
> >
> > Changchun: yes, will have a v2 to consider it, as I say in my 
> previous response.
>
> I must have joined the mailing list after your v2 series. I'll take a 
> look in the web for it.
>
> >
> > dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is 
> still not initialized for a VF. Thus there is no way to query the RSS 
> table
> >
> > Changchun: do you mean query rss table on guest?
>
> Of course.
>
> Niantic doesn’t have separate reta for a specific vf, the reta is 
> shared by pf and all vf.
>
> AFAIK the reta is statically divided between all available functions.
>
> So, first of all we need a way to know that is the size of a single VF 
> partition in order to know how many RSS queues may be configured.
>
> Then we need an ability to query/set the contents of the corresponding 
> reta partition like we can for PF.
>
> >
> > Do you think of returning all reta contents for each vf make sense?  
> Or any other insight here?
> >
> > rte_eth_dev_info_get() returns reta_size == 0 when called for a VF 
> function.
> >
> > Changchun:  same as above.
> >
> > Your insight here are welcome here.
> >
> > thanks
> >
> > Changchun
> >
> >
>

[PATCH 0/6] Enable VF RSS for Niantic

[Ouyang Changchun]

This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues.
The actual queue number per VF depends on the number of VF:
VF number from 1~32: 4 queues per VF;
VF number from 33~max vf num: 2 queues per VF;
 
On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS
or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated.
It also needs config VF RSS information like hash function, RSS key, RSS key length.

Changchun Ouyang (6):
  ixgbe: Code cleanup
  ixgbe: Negotiate VF API version
  ixgbe: Get VF queue number
  ether: Check VMDq RSS mode
  ixgbe: Config VF RSS
  testpmd: Set Rx VMDq RSS mode

 app/test-pmd/testpmd.c              |  9 ++++
 lib/librte_ether/rte_ethdev.c       | 21 ++++++--
 lib/librte_pmd_ixgbe/ixgbe_ethdev.h |  1 +
 lib/librte_pmd_ixgbe/ixgbe_pf.c     | 75 ++++++++++++++++++++++++++++-
 lib/librte_pmd_ixgbe/ixgbe_rxtx.c   | 95 +++++++++++++++++++++++++++----------
 5 files changed, 171 insertions(+), 30 deletions(-)

-- 
1.8.4.2

Re: [PATCH 0/6] Enable VF RSS for Niantic

[Bruce Richardson]

On Mon, Dec 15, 2014 at 10:56:59AM +0800, Ouyang Changchun wrote:
> This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues.
> The actual queue number per VF depends on the number of VF:
> VF number from 1~32: 4 queues per VF;
> VF number from 33~max vf num: 2 queues per VF;
>  
> On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS
> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated.
> It also needs config VF RSS information like hash function, RSS key, RSS key length.
> 

Hi Changchun,

are there limitations to this support, as I understood that that RSS support for
VFs was not fully available in Niantic-based hardware?

/Bruce


> Changchun Ouyang (6):
>   ixgbe: Code cleanup
>   ixgbe: Negotiate VF API version
>   ixgbe: Get VF queue number
>   ether: Check VMDq RSS mode
>   ixgbe: Config VF RSS
>   testpmd: Set Rx VMDq RSS mode
> 
>  app/test-pmd/testpmd.c              |  9 ++++
>  lib/librte_ether/rte_ethdev.c       | 21 ++++++--
>  lib/librte_pmd_ixgbe/ixgbe_ethdev.h |  1 +
>  lib/librte_pmd_ixgbe/ixgbe_pf.c     | 75 ++++++++++++++++++++++++++++-
>  lib/librte_pmd_ixgbe/ixgbe_rxtx.c   | 95 +++++++++++++++++++++++++++----------
>  5 files changed, 171 insertions(+), 30 deletions(-)
> 
> -- 
> 1.8.4.2
> 

Re: [PATCH 0/6] Enable VF RSS for Niantic

[Ouyang, Changchun]

Hi Bruce,

> -----Original Message-----
> From: Richardson, Bruce
> Sent: Monday, December 15, 2014 6:55 PM
> To: Ouyang, Changchun
> Cc: dev-VfR2kkLFssw@public.gmane.org
> Subject: Re: [dpdk-dev] [PATCH 0/6] Enable VF RSS for Niantic
> 
> On Mon, Dec 15, 2014 at 10:56:59AM +0800, Ouyang Changchun wrote:
> > This patch enables VF RSS for Niantic, which allow each VF having at most 4
> queues.
> > The actual queue number per VF depends on the number of VF:
> > VF number from 1~32: 4 queues per VF;
> > VF number from 33~max vf num: 2 queues per VF;
> >
> > On host, to enable VF RSS functionality, mq mode should be set as
> > ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_RSS mode, and SRIOV mode
> should be activated.
> > It also needs config VF RSS information like hash function, RSS key, RSS key
> length.
> >
> 
> Hi Changchun,
> 
> are there limitations to this support, as I understood that that RSS support for
> VFs was not fully available in Niantic-based hardware?
> 

The limitation here is RETA table, RSS key, Hash function are shared by pf and vf,
This is not like FVL, in FVL each VF may has its own RETA.

DPDK can setup 4 queues per vf while Linux ixgbe driver setup only 2 queues per vf.
This is a little advantage in DPDK over Linux driver .

Thanks
Changchun