Re: [PATCH RFC net-next 0/9] net: BPF Timestamping 2.0 for TCP

[Stanislav Fomichev <sdf.kernel@gmail.com>]

On 09/19, Jason Xing wrote:
> From: Jason Xing <kernelxing@gmail.com>
> 
> Greeting,
> 
> It's BPF Timestamping 2.0 that aims to observe the packet latency
> more efficiently and simply for different protocols. The current
> series is only focused on TCP protocol.
> 
> At Netdev 0x1a/Netconf 2026, the history, background, motivation and
> rough implementation of the feature were exhaustively introduced[1].
> 
> History
> =======
> - In 2009, Patrick Ohly implemented the basic infrastructure
> - In 2014, Willem de Bruijn enhanced the TCP latency observation
> - In 2024, Jason Xing proposed its lightweight BPF version
> Detailed slides from 28 to 32 [1].
> 
> Background
> ==========
> Even though BPF Timetamping 1.0 is comparatively low-overhead,
> transparent, it's still complicated due to a few points inherited
> from the design:
> - Inflexible/fixed reporting phases (qdisc/driver/ack)
>   When we confirm the issue arises from the kernel by using attribution
>   ability of timestamping feature, we need to further minimize the scope
>   until the issue is fixed. That means, we then have to resort to write
>   a few complex BPF progs with the similar functionalities (like skb
>   level tag) which should not happen.
> - Not enough low-overhead
>   Serving the sensitive applications, an always-on latency observation
>   platform should mitigate the self-impact as much as possible. As we
>   can conclude from BPF Timestamping selftests, there are some blocking
>   and time-consuming points like where reading/writing BPF maps happen
>   in the extremely hot paths.
> - Minor flaws
>   There are a few minor flaws inherited from the initial design, like
>   missing tagging the last packet[2][3][4].
> Detailed slides from 33 to 39 [1].
> 
> Motivation
> ==========
> During the process of the large scale deployment over the last few years,
> we eventually realized timestamping feature doesn't support container
> scenario and we need a finer-grained and flexible tracing tool (packet
> basis) after a few rounds of attribution of issues.
> 
> Design
> ======
> - Start time
>   For the specific protocol, we need to accurately set the start time of
>   each packet first. For TCP, we chose the entry of tcp_sendmsg_locked
>   and the driver time as the start point, so that any BPF program is
>   capable of computing the delta between start time and current time.
> - Simplicity
>   Previous BPF program (like selftests) is too complex to implement. The
>   core idea is to make everything as simple as possible. And it should be
>   decoupled from BPF area and previous timestamping feature as much as
>   possible.
> - Flexibility
>   BPF program hooking any function with skb parameter can get the latency
>   value, which means it's no longer bound to the pre-embeded reporting
>   phases (see __skb_tstamp_tx)
> - Efficiency
>   Avoid the previous BPF operations as much as possible. Make sure the
>   feature achieves the lowest performance impact, which means only time
>   operations remain.
> Detailed slides from 40 to 62 [1].
> 
> Implementations
> ===============
> in-kernel
> - Find a suitable place to timestamp for each packet
> - Pick the right start time for TCP
> - Handle the split skb due to various reasons
> BPF prog
> - Hook any functions that carry skb parameter
> - Read out the start time from the skb
> - Generate the current time and then compute the latency
> 
> Discussion?
> ===========
> - Do we need a kfunc to allow users to reset the start time of each skb?
>   What I had in mind is if someone tries to observe the latency between
>   two specific functions (rather than tcp_sendmsg_locked).
> - Current implementation is real hardware timestamp always wins, which
>   means BPF prog possibly gets the hardware time that is not aligned
>   with bpf_ktime_get_real_ns.
> - After the series, do we need to implement the same logic for
>   SYN/FIN/PROBE... As far as I know according to numerous user reports,
>   a small handful of issues came from 3-way handshake.
> - Reusing the slot of hwtstamp might bring potential problems or make the
>   code hard to maintain. Can we add a timestamping specific field in
>   skb to deal with the latency observation?
> - netdev_data conflict in IGC driver. It seems unavoidable to pollute
>   start time when it's enabled. Should V2 feature coexist with hardware
>   timestamping?
> - Should V2 coexist with net timestamping and BPF timestamping? If not,
>   the maintenance should be easier.

After netconf discussion, I was under the impressions that no kernel
changes are needed, so what changed? Is it hard to track start_time
from tcp_sendmsg_locked on the bpf side that we need kernel support?