Re: [PATCH net-next RFC v4 2/5] net: Expose available time stamping layers to user space.

[Vladimir Oltean <vladimir.oltean@nxp.com>]

On Thu, May 11, 2023 at 03:09:02PM -0700, Jakub Kicinski wrote:
> Exactly, think Tx. For Rx hauling the TS in metadata from PHY/MAC to
> descriptor is easy. For Tx device will buffer the packet so the DMA
> completion happens before the packet actually left onto the wire.
> 
> Which is not to say that some devices may not generate the Rx timestamp
> when the packet is DMA'ed out of laziness, too.
> 
> > timestamps is an alternate solution to the same problem as DMA
> > timestamps, or whatever:
> > https://lore.kernel.org/netdev/20221018010733.4765-1-muhammad.husaini.zulkifli@intel.com/
> 
> What I was thinking was:
> 
>  - PHY - per spec, at the RS layer
>  - MAC - "close to the wire" in the MAC, specifically the pipeline
>    delay (PHY stamp vs MAC stamp) should be constant for all packets;
>    there must be no variable-time buffering and (for Tx) the time
>    stamping must be past the stage of the pipeline affected by pause
>    frames
>  - DMA - worst quality, variable delay timestamp, usually taken when
>    packets DMA descriptors (Rx or completion) are being written
> 
> With these definitions MAC and PHY timestamps are pretty similar
> from the perspective of accuracy.

So if I add a call to ptp_clock_info :: gettimex64() where the
skb_tx_timestamp() call is located in a driver, could that pass as
a DMA timestamp?

The question is how much do we want to encourage these DMA timestamps:
enough to make them a first-class citizen in the UAPI? Are users even
happy with their existence?

I mean, I can't ignore the fact that there are NICs that can provide
2-step TX timestamps at line rate for all packets (not just PTP) for
line rates exceeding 10G, in band with the descriptor in its TX
completion queue. I don't want to give any names, just to point out
that there isn't any inherent limitation in the technology. AFAIU from
igc_ptp_tx_hwtstamp(), it's just that the igc DMA controller did not
bother to transport the timestamps from the MAC back into the
descriptor, leaving it up to software to do it out of band, which of
course may cause correlation bugs and limits throughput. Surely they
can do better.