Question on i40e PCIe relaxed ordering (RO)

Question on i40e PCIe relaxed ordering (RO)

[tndave]

Hi,

Running iperf tcp test on 2 sparc systems with i40e connected back to
back, I see huge number of 'port.rx_dropped' (on iperf server). Based on
past experience with ixgbe, this could very well because of PCIe RO
(relaxed ordering) not enabled.

I am trying to confirm RO is enabled. i40e datasheet mentioned RO
settings in 3 different sections:

1. section 10.2.2.2.38 PCIe Global Config 2 - GLPCI_CNF2 register
contains global status fields of PCIe configuration. The bit 0 of the
register is "RO_DIS". If this bit is set to 1 RO is disabled.

RO_DIS in my setup is 0 imply RO is not disabled.

2. section 12.3.5.5 Device Control Register (0xA8; RW) has bit 4
that enable/disable RO. This is pcie cap register.

In my i40e pcie config space value at offset 0xA8 is "2110". i.e 4th bit
set to 1 imply RO is enabled.

3. section 3.1.2.7.2 mentions some relaxed ordering rules
e.g. "The GLLAN_RCTL.RXDESCRDROEN bit (loaded from NVM) enables relaxed
ordering for Rx descriptor reads"

However, GLLAN_RCTL register definition has not bit like RXDESCRDROEN.
Same goes for GLLAN_TCTL.TXDESCRDROEN.

Am I missing anything? please advise.

Thanks.

-Tushar

(Ref:http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xl710-10-40-controller-datasheet.pdf)

Re: Question on i40e PCIe relaxed ordering (RO)

[Greg]

On Wed, 2016-06-29 at 13:05 -0700, tndave wrote:
> Hi,
> 
> Running iperf tcp test on 2 sparc systems with i40e connected back to
> back, I see huge number of 'port.rx_dropped' (on iperf server). Based on
> past experience with ixgbe, this could very well because of PCIe RO
> (relaxed ordering) not enabled.
> 
> I am trying to confirm RO is enabled. i40e datasheet mentioned RO
> settings in 3 different sections:
> 
> 1. section 10.2.2.2.38 PCIe Global Config 2 - GLPCI_CNF2 register
> contains global status fields of PCIe configuration. The bit 0 of the
> register is "RO_DIS". If this bit is set to 1 RO is disabled.
> 
> RO_DIS in my setup is 0 imply RO is not disabled.
> 
> 2. section 12.3.5.5 Device Control Register (0xA8; RW) has bit 4
> that enable/disable RO. This is pcie cap register.
> 
> In my i40e pcie config space value at offset 0xA8 is "2110". i.e 4th bit
> set to 1 imply RO is enabled.
> 
> 3. section 3.1.2.7.2 mentions some relaxed ordering rules
> e.g. "The GLLAN_RCTL.RXDESCRDROEN bit (loaded from NVM) enables relaxed
> ordering for Rx descriptor reads"
> 
> However, GLLAN_RCTL register definition has not bit like RXDESCRDROEN.
> Same goes for GLLAN_TCTL.TXDESCRDROEN.
> 
> Am I missing anything? please advise.

I would try posting this question to the e1000 developer list over at
Source Forge.  The Intel customer support folks used to monitor that
list closely when I was there, hopefully they still are.

Regards,

- Greg

> 
> Thanks.
> 
> -Tushar
> 
> (Ref:http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xl710-10-40-controller-datasheet.pdf)
> 
> 
> 

Re: Question on i40e PCIe relaxed ordering (RO)

[Jeff Kirsher]

[-- Attachment #1.1: Type: text/plain, Size: 1707 bytes --]

On Wed, 2016-06-29 at 13:18 -0700, Greg wrote:
> On Wed, 2016-06-29 at 13:05 -0700, tndave wrote:
> > Hi,
> > 
> > Running iperf tcp test on 2 sparc systems with i40e connected back to
> > back, I see huge number of 'port.rx_dropped' (on iperf server). Based
> on
> > past experience with ixgbe, this could very well because of PCIe RO
> > (relaxed ordering) not enabled.
> > 
> > I am trying to confirm RO is enabled. i40e datasheet mentioned RO
> > settings in 3 different sections:
> > 
> > 1. section 10.2.2.2.38 PCIe Global Config 2 - GLPCI_CNF2 register
> > contains global status fields of PCIe configuration. The bit 0 of the
> > register is "RO_DIS". If this bit is set to 1 RO is disabled.
> > 
> > RO_DIS in my setup is 0 imply RO is not disabled.
> > 
> > 2. section 12.3.5.5 Device Control Register (0xA8; RW) has bit 4
> > that enable/disable RO. This is pcie cap register.
> > 
> > In my i40e pcie config space value at offset 0xA8 is "2110". i.e 4th
> bit
> > set to 1 imply RO is enabled.
> > 
> > 3. section 3.1.2.7.2 mentions some relaxed ordering rules
> > e.g. "The GLLAN_RCTL.RXDESCRDROEN bit (loaded from NVM) enables relaxed
> > ordering for Rx descriptor reads"
> > 
> > However, GLLAN_RCTL register definition has not bit like RXDESCRDROEN.
> > Same goes for GLLAN_TCTL.TXDESCRDROEN.
> > 
> > Am I missing anything? please advise.
> 
> I would try posting this question to the e1000 developer list over at
> Source Forge.  The Intel customer support folks used to monitor that
> list closely when I was there, hopefully they still are.

We are supposed to be monitoring both lists, but just in case I have added
e1000-devel list...

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 387 bytes --]

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

[-- Attachment #3: Type: text/plain, Size: 257 bytes --]

_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired