Re: AF_XDP not transmitting frames immediately

[Maciej Fijalkowski <maciej.fijalkowski@intel.com>]

On Tue, Dec 14, 2021 at 04:42:18PM +0100, Jesper Dangaard Brouer wrote:
> 
> On 14/12/2021 15.57, Ong, Boon Leong wrote:
> > > I suspected Boon Leong (cc) would have this hardware.
> >
> > Unfortunately, my current setup in lab does not have I225 hooked-up
> > and I am working remotely due to control access to intel facility.
> > Perhaps, Ederson may have ready system to test?
> > 
> > For ZC mode, the igc driver (also true to stmmac) depends on the XSK wakeup
> > to trigger the NAPI poll (igc_poll) to first clean-up Tx ring and eventually call
> > igc_xdp_xmit_zc() to start submitting Tx frame into DMA engine. We have
> > used busy-poll to ensure in smaller Tx frame latency/jitter.
> > 
> > There was another issue in stmmac that was patched [1] recently to ensure
> > the driver does not perform MAC reset whenever XDP program is added
> > so that between XDP socket creation, the Tx transmit does not take extra
> > 2-3s due to link down/up. Jesper, are you seeing something similar in your
> > app?
> 
> Yes, and it is quite annoying.
> 
> In my setup, if I AF_XDP transmit packets too early they are simply lost...
> that confused me a bit.
> 
> I wanted to ask AF_XDP maintainers:
>  - What is the best way to know when AF_XDP is ready to Tx packets?
> 
> E.g. what API should I call, e.g. that blocks, until XSK socket is ready to
> transmit on?

Not a maintainer, but anyway.
From a driver POV xsk_tx_peek_desc() (or batching variant) is used to make
sure that user space produced entries in the XSK Tx ring so that driver
can consume it and place it onto HW descriptors.

From the top of my head I'm not aware of any blocking calls, maybe you
could spin on xsk_tx_peek_desc.

For igc maybe it would be worth returning some status from
igc_xdp_xmit_zc(). Like, imagine that you consumed all the budget but
there are still descriptors in the XSK Tx ring. You'd like to signal to
NAPI that there is still work to be done.

> 
> 
> > If yes, then it is likely because of the implementation of igc driver in mainline
> > that is doing igc_down(), a little bit too aggressive in reseting MAC completely.
> > 
> 
> It would be good to fix igc too, like[1].
> BUT afaik it will only make the window smaller when XSK is not ready for TX
> packets.
> 
> 
> > [1]https://patchwork.kernel.org/project/netdevbpf/patch/20211111143949.2806049-1-boon.leong.ong@intel.com/
> > 
> 
> Thanks for the link
> --Jesper
>