Hi Ferenc,

On Wed Feb 14 2024, Ferenc Fejes wrote:
> Hi,
>
> We are experimenting with scheduled packet transfers using the
> AF_PACKET socket. There is the ETF disk, which is great for most cases.

For what use cases does it not work? Are you running PREEMPT_RT? Just
asking out of curiosity :-).

> When we bypassed ETF, everything seemed ok regarding the timing: our
> packet received about +/-15ns offset at the receiver (now its the same
> machine just to make sure with the timesync) compared to the timestamp
> set with SO_TXTIME CMSG.
>
> What we tried now is to bypass the ETF qdisc. We enabled the ETF qdisc
> with hardware offload and sent the exact same packets, but this time
> with PACKET_QDISC_BAYPASS enabled on the AF_PACKET socket. The codepath
> looks good, the qdisc part is not called, the packet_snd calls the
> dev_direct_xmit which calls the igc_xmit_frame. However, in this case
> the packet was sent more or less immediately.

Well, yeah the code path looks good indeed. packet_snd() copies the
transmit time which is provided by the CMSG and calls into
packet_xmit(), dev_direct_xmit()...

>
> I wonder why we do not see the delayed sending in this case? We tried
> with different offsets (e.g. 0.1, 0.01, 0.001 sec in the future) but we
> received the packet after 20-30usec every time. I cant see any code
> that touches the skb timestamp after the packet_snd, so I suspect that
> the igc_xmit_frame sees the same timestamp that it would see in the
> non-baypass case.

Maybe add some trace_printk()s to track what timestamps are actually
calculated in igc_tx_launchtime() and if it makes sense?

Second point to make sure is that the Tx queue your packet is being
transmitted to has Launch Time enabled.

>
> I happen to have the i225 user manual, but after some grep I cannot
> find any debug registers or counters to monitor the behavior of the
> scheduled transmission (scheduling errors or bad timestamps, etc.). Are
> there any?

Not that I'm aware of.

>
> I am afraid this issue might also be relevant for the AF_XDP case,
> which also hooks after the qdisc layer, so the launchtime (or whatever
> it is called) is handled directly by the igc driver.

Is that already possible with AF_XDP? There were some patches on
xdp-hints, but i don't think it has been merged yet.

Thanks,
Kurt