Re: AF_XDP busy poll receives packets in batches of 8 on i40e

[Jay Vosburgh <jay.vosburgh@canonical.com>]

Jesse Brandeburg <jesse.brandeburg@intel.com> wrote:

>On 3/17/2022 8:34 AM, Federico Parola wrote:
>>>     We observed similar "batching" behavior on i40e devices late
>>> last year in ordinary use (not XDP, but using SR-IOV VFs).  We
>>> instrumented the drivers at the send and receive sides, and determined
>>> that it appeared to be a behavior of the receiving device itself, i.e.,
>>> packets 1 - 7 would be held indefinitely (as I recall, no interrupt or
>>> update of the RX ring pointers) until packet 8 arrives, at which point
>>> all 8 were delivered simultaneously.
>>>
>>>     The issue was evidently in the firmware, and was resolved after
>>> a firmware upgrade.
>> Hi Jay,
>> I just updated the firmware to the latest version (v8.50 from v8.30) but
>> unfortunately the problem is still there.
>> However I'm experiencing the problem only when using AF_XDP in busy poll
>> mode, all other modes (standard AF_XDP and normal packet reception) work
>> just fine.
>> Maybe the two problems are correlated in some way.
>
>This sounds related to the WB_ON_ITR feature in our hardware. If
>interrupts are disabled the driver needs to set that bit (and an ITR
>value) so that packets get written back in a timely manner and don't just
>wait for a cache line edge (I bet your Cache Line Size value in PCIe space
>(lspci) is set to 128?)

	FWIW, in the case we had, driver changes (5.14 upstream, the
then-current Intel out of tree driver) didn't change the behavior.  The
PCI Cache Line Size was 64 bytes; the device was

13:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
        Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 562SFP+ Adapter

	From an lspci -vvv perspective, failing and non-failing cards
differed only in the serial numbers.

>Maybe something to chase there? We may also be able to look into it if it
>wasn't fixed already upstream.

	-J

---
	-Jay Vosburgh, jay.vosburgh@canonical.com