[PATCH bpf-next v12 00/12] net-timestamp: bpf extension to equip applications transparently

[PATCH bpf-next v12 00/12] net-timestamp: bpf extension to equip applications transparently

[Jason Xing]

"Timestamping is key to debugging network stack latency. With
SO_TIMESTAMPING, bugs that are otherwise incorrectly assumed to be
network issues can be attributed to the kernel." This is extracted
from the talk "SO_TIMESTAMPING: Powering Fleetwide RPC Monitoring"
addressed by Willem de Bruijn at netdevconf 0x17).

There are a few areas that need optimization with the consideration of
easier use and less performance impact, which I highlighted and mainly
discussed at netconf 2024 with Willem de Bruijn and John Fastabend:
uAPI compatibility, extra system call overhead, and the need for
application modification. I initially managed to solve these issues
by writing a kernel module that hooks various key functions. However,
this approach is not suitable for the next kernel release. Therefore,
a BPF extension was proposed. During recent period, Martin KaFai Lau
provides invaluable suggestions about BPF along the way. Many thanks
here!

This series adds the BPF networking timestamping infrastructure through
reusing most of the tx timestamping callback that is currently enabled
by the SO_TIMESTAMPING.. This series also adds TX timestamping support
for TCP. The RX timestamping and UDP support will be added in the future.

---
v11
Link: https://lore.kernel.org/all/20250214010038.54131-1-kerneljasonxing@gmail.com/
1. remove the unused variable 'sk'
2. correct grammar mistakes
3. move two places where using SKBTX_HW_TSTAMP_NOBPF from patch 9 to 8.

v10
Link: https://lore.kernel.org/all/20250212061855.71154-1-kerneljasonxing@gmail.com/
1. rename hwts with hwtimestamp
2. use subtest and pid filter in selftest
3. use 'tcb->txstamp_ack |= TSTAMP_ACK_SK'

v9
Link: https://lore.kernel.org/all/20250208103220.72294-1-kerneljasonxing@gmail.com/
1. set the hwtstamp to skb when the skb enters into the hw SND case
2. fix co-existence problem in patch 9 and add corresponding check in
patch 12.
3. refine some commit messages and titles

v8
Link: https://lore.kernel.org/all/20250128084620.57547-1-kerneljasonxing@gmail.com/
1. adjust some commit messages and titles
2. add sk cookie in selftests
3. handle the NULL pointer in hwstamp
4. use kfunc to do selective sampling

v7
Link: https://lore.kernel.org/all/20250121012901.87763-1-kerneljasonxing@gmail.com/
1. target bpf-next tree
2. simplely and directly stop timestamping callbacks calling a few BPF
CALLS due to safety concern.
3. add more new testcases and adjust the existing testcases
4. revise some comments of new timestamping callbacks
5. remove a few BPF CGROUP locks

RFC v6
In the meantime, any suggestions and reviews are welcome!
Link: https://lore.kernel.org/all/20250112113748.73504-1-kerneljasonxing@gmail.com/
1. handle those safety problem by using the correct method.
2. support bpf_getsockopt.
3. adjust the position of BPF_SOCK_OPS_TS_TCP_SND_CB
4. fix mishandling the hardware timestamp error
5. add more corresponding tests

v5
Link: https://lore.kernel.org/all/20241207173803.90744-1-kerneljasonxing@gmail.com/
1. handle the safety issus when someone tries to call unrelated bpf
helpers.
2. avoid adding direct function call in the hot path like
__dev_queue_xmit()
3. remove reporting the hardware timestamp and tskey since they can be
fetched through the existing helper with the help of
bpf_skops_init_skb(), please see the selftest.
4. add new sendmsg callback in tcp_sendmsg, and introduce tskey_bpf used
by bpf program to correlate tcp_sendmsg with other hook points in patch [13/15].

v4
Link: https://lore.kernel.org/all/20241028110535.82999-1-kerneljasonxing@gmail.com/
1. introduce sk->sk_bpf_cb_flags to let user use bpf_setsockopt() (Martin)
2. introduce SKBTX_BPF to enable the bpf SO_TIMESTAMPING feature (Martin)
3. introduce bpf map in tests (Martin)
4. I choose to make this series as simple as possible, so I only support
most cases in the tx path for TCP protocol.

v3
Link: https://lore.kernel.org/all/20241012040651.95616-1-kerneljasonxing@gmail.com/
1. support UDP proto by introducing a new generation point.
2. for OPT_ID, introducing sk_tskey_bpf_offset to compute the delta
between the current socket key and bpf socket key. It is desiged for
UDP, which also applies to TCP.
3. support bpf_getsockopt()
4. use cgroup static key instead.
5. add one simple bpf selftest to show how it can be used.
6. remove the rx support from v2 because the number of patches could
exceed the limit of one series.

V2
Link: https://lore.kernel.org/all/20241008095109.99918-1-kerneljasonxing@gmail.com/
1. Introduce tsflag requestors so that we are able to extend more in the
future. Besides, it enables TX flags for bpf extension feature separately
without breaking users. It is suggested by Vadim Fedorenko.
2. introduce a static key to control the whole feature. (Willem)
3. Open the gate of bpf_setsockopt for the SO_TIMESTAMPING feature in
some TX/RX cases, not all the cases.


Jason Xing (12):
  bpf: add networking timestamping support to bpf_get/setsockopt()
  bpf: prepare the sock_ops ctx and call bpf prog for TX timestamping
  bpf: prevent unsafe access to the sock fields in the BPF timestamping
    callback
  bpf: disable unsafe helpers in TX timestamping callbacks
  net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING
  bpf: add BPF_SOCK_OPS_TS_SCHED_OPT_CB callback
  bpf: add BPF_SOCK_OPS_TS_SW_OPT_CB callback
  bpf: add BPF_SOCK_OPS_TS_HW_OPT_CB callback
  bpf: add BPF_SOCK_OPS_TS_ACK_OPT_CB callback
  bpf: add BPF_SOCK_OPS_TS_SND_CB callback
  bpf: support selective sampling for bpf timestamping
  selftests/bpf: add simple bpf tests in the tx path for timestamping
    feature

 include/linux/filter.h                        |   1 +
 include/linux/skbuff.h                        |  12 +-
 include/net/sock.h                            |  10 +
 include/net/tcp.h                             |   7 +-
 include/uapi/linux/bpf.h                      |  30 +++
 kernel/bpf/btf.c                              |   1 +
 net/core/dev.c                                |   3 +-
 net/core/filter.c                             |  79 +++++-
 net/core/skbuff.c                             |  53 ++++
 net/core/sock.c                               |  14 +
 net/dsa/user.c                                |   2 +-
 net/ipv4/tcp.c                                |   6 +-
 net/ipv4/tcp_input.c                          |   2 +
 net/ipv4/tcp_output.c                         |   2 +
 net/socket.c                                  |   2 +-
 tools/include/uapi/linux/bpf.h                |  23 ++
 .../bpf/prog_tests/net_timestamping.c         | 239 +++++++++++++++++
 .../selftests/bpf/progs/net_timestamping.c    | 248 ++++++++++++++++++
 18 files changed, 720 insertions(+), 14 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/net_timestamping.c
 create mode 100644 tools/testing/selftests/bpf/progs/net_timestamping.c

-- 
2.43.5

[PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Jason Xing]

The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
added to bpf_get/setsockopt. The later patches will implement the
BPF networking timestamping. The BPF program will use
bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
enable the BPF networking timestamping on a socket.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/net/sock.h             |  3 +++
 include/uapi/linux/bpf.h       |  8 ++++++++
 net/core/filter.c              | 23 +++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 35 insertions(+)

diff --git a/include/net/sock.h b/include/net/sock.h
index 8036b3b79cd8..7916982343c6 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -303,6 +303,7 @@ struct sk_filter;
   *	@sk_stamp: time stamp of last packet received
   *	@sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
   *	@sk_tsflags: SO_TIMESTAMPING flags
+  *	@sk_bpf_cb_flags: used in bpf_setsockopt()
   *	@sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
   *			   Sockets that can be used under memory reclaim should
   *			   set this to false.
@@ -445,6 +446,8 @@ struct sock {
 	u32			sk_reserved_mem;
 	int			sk_forward_alloc;
 	u32			sk_tsflags;
+#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
+	u32			sk_bpf_cb_flags;
 	__cacheline_group_end(sock_write_rxtx);
 
 	__cacheline_group_begin(sock_write_tx);
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index fff6cdb8d11a..fa666d51dffe 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -6916,6 +6916,13 @@ enum {
 	BPF_SOCK_OPS_ALL_CB_FLAGS       = 0x7F,
 };
 
+/* Definitions for bpf_sk_cb_flags */
+enum {
+	SK_BPF_CB_TX_TIMESTAMPING	= 1<<0,
+	SK_BPF_CB_MASK			= (SK_BPF_CB_TX_TIMESTAMPING - 1) |
+					   SK_BPF_CB_TX_TIMESTAMPING
+};
+
 /* List of known BPF sock_ops operators.
  * New entries can only be added at the end
  */
@@ -7094,6 +7101,7 @@ enum {
 	TCP_BPF_SYN_IP		= 1006, /* Copy the IP[46] and TCP header */
 	TCP_BPF_SYN_MAC         = 1007, /* Copy the MAC, IP[46], and TCP header */
 	TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */
+	SK_BPF_CB_FLAGS		= 1009, /* Used to set socket bpf flags */
 };
 
 enum {
diff --git a/net/core/filter.c b/net/core/filter.c
index 2ec162dd83c4..1c6c07507a78 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -5222,6 +5222,25 @@ static const struct bpf_func_proto bpf_get_socket_uid_proto = {
 	.arg1_type      = ARG_PTR_TO_CTX,
 };
 
+static int sk_bpf_set_get_cb_flags(struct sock *sk, char *optval, bool getopt)
+{
+	u32 sk_bpf_cb_flags;
+
+	if (getopt) {
+		*(u32 *)optval = sk->sk_bpf_cb_flags;
+		return 0;
+	}
+
+	sk_bpf_cb_flags = *(u32 *)optval;
+
+	if (sk_bpf_cb_flags & ~SK_BPF_CB_MASK)
+		return -EINVAL;
+
+	sk->sk_bpf_cb_flags = sk_bpf_cb_flags;
+
+	return 0;
+}
+
 static int sol_socket_sockopt(struct sock *sk, int optname,
 			      char *optval, int *optlen,
 			      bool getopt)
@@ -5238,6 +5257,7 @@ static int sol_socket_sockopt(struct sock *sk, int optname,
 	case SO_MAX_PACING_RATE:
 	case SO_BINDTOIFINDEX:
 	case SO_TXREHASH:
+	case SK_BPF_CB_FLAGS:
 		if (*optlen != sizeof(int))
 			return -EINVAL;
 		break;
@@ -5247,6 +5267,9 @@ static int sol_socket_sockopt(struct sock *sk, int optname,
 		return -EINVAL;
 	}
 
+	if (optname == SK_BPF_CB_FLAGS)
+		return sk_bpf_set_get_cb_flags(sk, optval, getopt);
+
 	if (getopt) {
 		if (optname == SO_BINDTODEVICE)
 			return -EINVAL;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 2acf9b336371..70366f74ef4e 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7091,6 +7091,7 @@ enum {
 	TCP_BPF_SYN_IP		= 1006, /* Copy the IP[46] and TCP header */
 	TCP_BPF_SYN_MAC         = 1007, /* Copy the MAC, IP[46], and TCP header */
 	TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */
+	SK_BPF_CB_FLAGS		= 1009, /* Used to set socket bpf flags */
 };
 
 enum {
-- 
2.43.5

[PATCH bpf-next v12 02/12] bpf: prepare the sock_ops ctx and call bpf prog for TX timestamping

[Jason Xing]

This patch introduces a new bpf_skops_tx_timestamping() function
that prepares the "struct bpf_sock_ops" ctx and then executes the
sockops BPF program.

The subsequent patch will utilize bpf_skops_tx_timestamping() at
the existing TX timestamping kernel callbacks (__sk_tstamp_tx
specifically) to call the sockops BPF program. Later, four callback
points to report information to user space based on this patch will
be introduced.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/net/sock.h |  7 +++++++
 net/core/sock.c    | 14 ++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/include/net/sock.h b/include/net/sock.h
index 7916982343c6..6f4d54faba92 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2923,6 +2923,13 @@ int sock_set_timestamping(struct sock *sk, int optname,
 			  struct so_timestamping timestamping);
 
 void sock_enable_timestamps(struct sock *sk);
+#if defined(CONFIG_CGROUP_BPF)
+void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op);
+#else
+static inline void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op)
+{
+}
+#endif
 void sock_no_linger(struct sock *sk);
 void sock_set_keepalive(struct sock *sk);
 void sock_set_priority(struct sock *sk, u32 priority);
diff --git a/net/core/sock.c b/net/core/sock.c
index eae2ae70a2e0..bde45569d4da 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -948,6 +948,20 @@ int sock_set_timestamping(struct sock *sk, int optname,
 	return 0;
 }
 
+#if defined(CONFIG_CGROUP_BPF)
+void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op)
+{
+	struct bpf_sock_ops_kern sock_ops;
+
+	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
+	sock_ops.op = op;
+	sock_ops.is_fullsock = 1;
+	sock_ops.sk = sk;
+	bpf_skops_init_skb(&sock_ops, skb, 0);
+	__cgroup_bpf_run_filter_sock_ops(sk, &sock_ops, CGROUP_SOCK_OPS);
+}
+#endif
+
 void sock_set_keepalive(struct sock *sk)
 {
 	lock_sock(sk);
-- 
2.43.5

[PATCH bpf-next v12 03/12] bpf: prevent unsafe access to the sock fields in the BPF timestamping callback

[Jason Xing]

The subsequent patch will implement BPF TX timestamping. It will
call the sockops BPF program without holding the sock lock.

This breaks the current assumption that all sock ops programs will
hold the sock lock. The sock's fields of the uapi's bpf_sock_ops
requires this assumption.

To address this, a new "u8 is_locked_tcp_sock;" field is added. This
patch sets it in the current sock_ops callbacks. The "is_fullsock"
test is then replaced by the "is_locked_tcp_sock" test during
sock_ops_convert_ctx_access().

The new TX timestamping callbacks added in the subsequent patch will
not have this set. This will prevent unsafe access from the new
timestamping callbacks.

Potentially, we could allow read-only access. However, this would
require identifying which callback is read-safe-only and also requires
additional BPF instruction rewrites in the covert_ctx. Since the BPF
program can always read everything from a socket (e.g., by using
bpf_core_cast), this patch keeps it simple and disables all read
and write access to any socket fields through the bpf_sock_ops
UAPI from the new TX timestamping callback.

Moreover, note that some of the fields in bpf_sock_ops are specific
to tcp_sock, and sock_ops currently only supports tcp_sock. In
the future, UDP timestamping will be added, which will also break
this assumption. The same idea used in this patch will be reused.
Considering that the current sock_ops only supports tcp_sock, the
variable is named is_locked_"tcp"_sock.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/linux/filter.h | 1 +
 include/net/tcp.h      | 1 +
 net/core/filter.c      | 8 ++++----
 net/ipv4/tcp_input.c   | 2 ++
 net/ipv4/tcp_output.c  | 2 ++
 5 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/include/linux/filter.h b/include/linux/filter.h
index a3ea46281595..d36d5d5180b1 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1508,6 +1508,7 @@ struct bpf_sock_ops_kern {
 	void	*skb_data_end;
 	u8	op;
 	u8	is_fullsock;
+	u8	is_locked_tcp_sock;
 	u8	remaining_opt_len;
 	u64	temp;			/* temp and everything after is not
 					 * initialized to 0 before calling
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 5b2b04835688..4c4dca59352b 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2649,6 +2649,7 @@ static inline int tcp_call_bpf(struct sock *sk, int op, u32 nargs, u32 *args)
 	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
 	if (sk_fullsock(sk)) {
 		sock_ops.is_fullsock = 1;
+		sock_ops.is_locked_tcp_sock = 1;
 		sock_owned_by_me(sk);
 	}
 
diff --git a/net/core/filter.c b/net/core/filter.c
index 1c6c07507a78..8631036f6b64 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -10381,10 +10381,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,
 		}							      \
 		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(			      \
 						struct bpf_sock_ops_kern,     \
-						is_fullsock),		      \
+						is_locked_tcp_sock),	      \
 				      fullsock_reg, si->src_reg,	      \
 				      offsetof(struct bpf_sock_ops_kern,      \
-					       is_fullsock));		      \
+					       is_locked_tcp_sock));	      \
 		*insn++ = BPF_JMP_IMM(BPF_JEQ, fullsock_reg, 0, jmp);	      \
 		if (si->dst_reg == si->src_reg)				      \
 			*insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg,	      \
@@ -10469,10 +10469,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,
 					       temp));			      \
 		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(			      \
 						struct bpf_sock_ops_kern,     \
-						is_fullsock),		      \
+						is_locked_tcp_sock),	      \
 				      reg, si->dst_reg,			      \
 				      offsetof(struct bpf_sock_ops_kern,      \
-					       is_fullsock));		      \
+					       is_locked_tcp_sock));	      \
 		*insn++ = BPF_JMP_IMM(BPF_JEQ, reg, 0, 2);		      \
 		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(			      \
 						struct bpf_sock_ops_kern, sk),\
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index eb82e01da911..95733dcdfb4b 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -169,6 +169,7 @@ static void bpf_skops_parse_hdr(struct sock *sk, struct sk_buff *skb)
 	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
 	sock_ops.op = BPF_SOCK_OPS_PARSE_HDR_OPT_CB;
 	sock_ops.is_fullsock = 1;
+	sock_ops.is_locked_tcp_sock = 1;
 	sock_ops.sk = sk;
 	bpf_skops_init_skb(&sock_ops, skb, tcp_hdrlen(skb));
 
@@ -185,6 +186,7 @@ static void bpf_skops_established(struct sock *sk, int bpf_op,
 	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
 	sock_ops.op = bpf_op;
 	sock_ops.is_fullsock = 1;
+	sock_ops.is_locked_tcp_sock = 1;
 	sock_ops.sk = sk;
 	/* sk with TCP_REPAIR_ON does not have skb in tcp_finish_connect */
 	if (skb)
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index bc95d2a5924f..a0e779bdbc6b 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -525,6 +525,7 @@ static void bpf_skops_hdr_opt_len(struct sock *sk, struct sk_buff *skb,
 		sock_owned_by_me(sk);
 
 		sock_ops.is_fullsock = 1;
+		sock_ops.is_locked_tcp_sock = 1;
 		sock_ops.sk = sk;
 	}
 
@@ -570,6 +571,7 @@ static void bpf_skops_write_hdr_opt(struct sock *sk, struct sk_buff *skb,
 		sock_owned_by_me(sk);
 
 		sock_ops.is_fullsock = 1;
+		sock_ops.is_locked_tcp_sock = 1;
 		sock_ops.sk = sk;
 	}
 
-- 
2.43.5

[PATCH bpf-next v12 04/12] bpf: disable unsafe helpers in TX timestamping callbacks

[Jason Xing]

New TX timestamping sock_ops callbacks will be added in the
subsequent patch. Some of the existing BPF helpers will not
be safe to be used in the TX timestamping callbacks.

The bpf_sock_ops_setsockopt, bpf_sock_ops_getsockopt, and
bpf_sock_ops_cb_flags_set require owning the sock lock. TX
timestamping callbacks will not own the lock.

The bpf_sock_ops_load_hdr_opt needs the skb->data pointing
to the TCP header. This will not be true in the TX timestamping
callbacks.

At the beginning of these helpers, this patch checks the
bpf_sock->op to ensure these helpers are used by the existing
sock_ops callbacks only.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 net/core/filter.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/net/core/filter.c b/net/core/filter.c
index 8631036f6b64..7f56d0bbeb00 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -5523,6 +5523,11 @@ static int __bpf_setsockopt(struct sock *sk, int level, int optname,
 	return -EINVAL;
 }
 
+static bool is_locked_tcp_sock_ops(struct bpf_sock_ops_kern *bpf_sock)
+{
+	return bpf_sock->op <= BPF_SOCK_OPS_WRITE_HDR_OPT_CB;
+}
+
 static int _bpf_setsockopt(struct sock *sk, int level, int optname,
 			   char *optval, int optlen)
 {
@@ -5673,6 +5678,9 @@ static const struct bpf_func_proto bpf_sock_addr_getsockopt_proto = {
 BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,
 	   int, level, int, optname, char *, optval, int, optlen)
 {
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen);
 }
 
@@ -5758,6 +5766,9 @@ static int bpf_sock_ops_get_syn(struct bpf_sock_ops_kern *bpf_sock,
 BPF_CALL_5(bpf_sock_ops_getsockopt, struct bpf_sock_ops_kern *, bpf_sock,
 	   int, level, int, optname, char *, optval, int, optlen)
 {
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	if (IS_ENABLED(CONFIG_INET) && level == SOL_TCP &&
 	    optname >= TCP_BPF_SYN && optname <= TCP_BPF_SYN_MAC) {
 		int ret, copy_len = 0;
@@ -5800,6 +5811,9 @@ BPF_CALL_2(bpf_sock_ops_cb_flags_set, struct bpf_sock_ops_kern *, bpf_sock,
 	struct sock *sk = bpf_sock->sk;
 	int val = argval & BPF_SOCK_OPS_ALL_CB_FLAGS;
 
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	if (!IS_ENABLED(CONFIG_INET) || !sk_fullsock(sk))
 		return -EINVAL;
 
@@ -7609,6 +7623,9 @@ BPF_CALL_4(bpf_sock_ops_load_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
 	u8 search_kind, search_len, copy_len, magic_len;
 	int ret;
 
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	/* 2 byte is the minimal option len except TCPOPT_NOP and
 	 * TCPOPT_EOL which are useless for the bpf prog to learn
 	 * and this helper disallow loading them also.
-- 
2.43.5

[PATCH bpf-next v12 05/12] net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING

[Jason Xing]

No functional changes here. Only add test to see if the orig_skb
matches the usage of application SO_TIMESTAMPING.

In this series, bpf timestamping and previous socket timestamping
are implemented in the same function __skb_tstamp_tx(). To test
the socket enables socket timestamping feature, this function
skb_tstamp_tx_report_so_timestamping() is added.

In the next patch, another check for bpf timestamping feature
will be introduced just like the above report function, namely,
skb_tstamp_tx_report_bpf_timestamping(). Then users will be able
to know the socket enables either or both of features.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 net/core/skbuff.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index a441613a1e6c..341a3290e898 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5539,6 +5539,23 @@ void skb_complete_tx_timestamp(struct sk_buff *skb,
 }
 EXPORT_SYMBOL_GPL(skb_complete_tx_timestamp);
 
+static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb,
+						 struct skb_shared_hwtstamps *hwtstamps,
+						 int tstype)
+{
+	switch (tstype) {
+	case SCM_TSTAMP_SCHED:
+		return skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP;
+	case SCM_TSTAMP_SND:
+		return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP :
+						    SKBTX_SW_TSTAMP);
+	case SCM_TSTAMP_ACK:
+		return TCP_SKB_CB(skb)->txstamp_ack;
+	}
+
+	return false;
+}
+
 void __skb_tstamp_tx(struct sk_buff *orig_skb,
 		     const struct sk_buff *ack_skb,
 		     struct skb_shared_hwtstamps *hwtstamps,
@@ -5551,6 +5568,9 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
 	if (!sk)
 		return;
 
+	if (!skb_tstamp_tx_report_so_timestamping(orig_skb, hwtstamps, tstype))
+		return;
+
 	tsflags = READ_ONCE(sk->sk_tsflags);
 	if (!hwtstamps && !(tsflags & SOF_TIMESTAMPING_OPT_TX_SWHW) &&
 	    skb_shinfo(orig_skb)->tx_flags & SKBTX_IN_PROGRESS)
-- 
2.43.5

[PATCH bpf-next v12 06/12] bpf: add BPF_SOCK_OPS_TS_SCHED_OPT_CB callback

[Jason Xing]

Support SCM_TSTAMP_SCHED case for bpf timestamping.

Add a new sock_ops callback, BPF_SOCK_OPS_TS_SCHED_OPT_CB. This
callback will occur at the same timestamping point as the user
space's SCM_TSTAMP_SCHED. The BPF program can use it to get the
same SCM_TSTAMP_SCHED timestamp without modifying the user-space
application.

A new SKBTX_BPF flag is added to mark skb_shinfo(skb)->tx_flags,
ensuring that the new BPF timestamping and the current user
space's SO_TIMESTAMPING do not interfere with each other.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/linux/skbuff.h         |  6 +++++-
 include/uapi/linux/bpf.h       |  4 ++++
 net/core/dev.c                 |  3 ++-
 net/core/skbuff.c              | 20 ++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  4 ++++
 5 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index bb2b751d274a..52f6e033e704 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -489,10 +489,14 @@ enum {
 
 	/* generate software time stamp when entering packet scheduling */
 	SKBTX_SCHED_TSTAMP = 1 << 6,
+
+	/* used for bpf extension when a bpf program is loaded */
+	SKBTX_BPF = 1 << 7,
 };
 
 #define SKBTX_ANY_SW_TSTAMP	(SKBTX_SW_TSTAMP    | \
-				 SKBTX_SCHED_TSTAMP)
+				 SKBTX_SCHED_TSTAMP | \
+				 SKBTX_BPF)
 #define SKBTX_ANY_TSTAMP	(SKBTX_HW_TSTAMP | \
 				 SKBTX_HW_TSTAMP_USE_CYCLES | \
 				 SKBTX_ANY_SW_TSTAMP)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index fa666d51dffe..68664ececdc0 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7035,6 +7035,10 @@ enum {
 					 * by the kernel or the
 					 * earlier bpf-progs.
 					 */
+	BPF_SOCK_OPS_TS_SCHED_OPT_CB,	/* Called when skb is passing through
+					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
diff --git a/net/core/dev.c b/net/core/dev.c
index c0021cbd28fc..cbbde68c17cb 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4500,7 +4500,8 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev)
 	skb_reset_mac_header(skb);
 	skb_assert_len(skb);
 
-	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP))
+	if (unlikely(skb_shinfo(skb)->tx_flags &
+		     (SKBTX_SCHED_TSTAMP | SKBTX_BPF)))
 		__skb_tstamp_tx(skb, NULL, NULL, skb->sk, SCM_TSTAMP_SCHED);
 
 	/* Disable soft irqs for various locks below. Also
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 341a3290e898..0aa54d102624 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5556,6 +5556,23 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb,
 	return false;
 }
 
+static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb,
+						  struct sock *sk,
+						  int tstype)
+{
+	int op;
+
+	switch (tstype) {
+	case SCM_TSTAMP_SCHED:
+		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
+		break;
+	default:
+		return;
+	}
+
+	bpf_skops_tx_timestamping(sk, skb, op);
+}
+
 void __skb_tstamp_tx(struct sk_buff *orig_skb,
 		     const struct sk_buff *ack_skb,
 		     struct skb_shared_hwtstamps *hwtstamps,
@@ -5568,6 +5585,9 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
 	if (!sk)
 		return;
 
+	if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF)
+		skb_tstamp_tx_report_bpf_timestamping(orig_skb, sk, tstype);
+
 	if (!skb_tstamp_tx_report_so_timestamping(orig_skb, hwtstamps, tstype))
 		return;
 
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 70366f74ef4e..eed91b7296b7 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7025,6 +7025,10 @@ enum {
 					 * by the kernel or the
 					 * earlier bpf-progs.
 					 */
+	BPF_SOCK_OPS_TS_SCHED_OPT_CB,	/* Called when skb is passing through
+					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
-- 
2.43.5

[PATCH bpf-next v12 07/12] bpf: add BPF_SOCK_OPS_TS_SW_OPT_CB callback

[Jason Xing]

Support sw SCM_TSTAMP_SND case for bpf timestamping.

Add a new sock_ops callback, BPF_SOCK_OPS_TS_SW_OPT_CB. This
callback will occur at the same timestamping point as the user
space's software SCM_TSTAMP_SND. The BPF program can use it to
get the same SCM_TSTAMP_SND timestamp without modifying the
user-space application.

Based on this patch, BPF program will get the software
timestamp when the driver is ready to send the skb. In the
sebsequent patch, the hardware timestamp will be supported.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/linux/skbuff.h         | 2 +-
 include/uapi/linux/bpf.h       | 4 ++++
 net/core/skbuff.c              | 9 ++++++++-
 tools/include/uapi/linux/bpf.h | 4 ++++
 4 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 52f6e033e704..76582500c5ea 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -4568,7 +4568,7 @@ void skb_tstamp_tx(struct sk_buff *orig_skb,
 static inline void skb_tx_timestamp(struct sk_buff *skb)
 {
 	skb_clone_tx_timestamp(skb);
-	if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP)
+	if (skb_shinfo(skb)->tx_flags & (SKBTX_SW_TSTAMP | SKBTX_BPF))
 		skb_tstamp_tx(skb, NULL);
 }
 
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 68664ececdc0..b3bd92281084 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7039,6 +7039,10 @@ enum {
 					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SW_OPT_CB,	/* Called when skb is about to send
+					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 0aa54d102624..03b90f04d0b0 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5557,6 +5557,7 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb,
 }
 
 static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb,
+						  struct skb_shared_hwtstamps *hwtstamps,
 						  struct sock *sk,
 						  int tstype)
 {
@@ -5566,6 +5567,11 @@ static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb,
 	case SCM_TSTAMP_SCHED:
 		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
 		break;
+	case SCM_TSTAMP_SND:
+		if (hwtstamps)
+			return;
+		op = BPF_SOCK_OPS_TS_SW_OPT_CB;
+		break;
 	default:
 		return;
 	}
@@ -5586,7 +5592,8 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
 		return;
 
 	if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF)
-		skb_tstamp_tx_report_bpf_timestamping(orig_skb, sk, tstype);
+		skb_tstamp_tx_report_bpf_timestamping(orig_skb, hwtstamps,
+						      sk, tstype);
 
 	if (!skb_tstamp_tx_report_so_timestamping(orig_skb, hwtstamps, tstype))
 		return;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index eed91b7296b7..9bd1c7c77b17 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7029,6 +7029,10 @@ enum {
 					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SW_OPT_CB,	/* Called when skb is about to send
+					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
-- 
2.43.5

[PATCH bpf-next v12 08/12] bpf: add BPF_SOCK_OPS_TS_HW_OPT_CB callback

[Jason Xing]

Support hw SCM_TSTAMP_SND case for bpf timestamping.

Add a new sock_ops callback, BPF_SOCK_OPS_TS_HW_OPT_CB. This
callback will occur at the same timestamping point as the user
space's hardware SCM_TSTAMP_SND. The BPF program can use it to
get the same SCM_TSTAMP_SND timestamp without modifying the
user-space application.

To avoid increasing the code complexity, replace SKBTX_HW_TSTAMP
with SKBTX_HW_TSTAMP_NOBPF instead of changing numerous callers
from driver side using SKBTX_HW_TSTAMP. The new definition of
SKBTX_HW_TSTAMP means the combination tests of socket timestamping
and bpf timestamping. After this patch, drivers can work under the
bpf timestamping.

Considering some drivers don't assign the skb with hardware
timestamp, this patch does the assignment and then BPF program
can acquire the hwstamp from skb directly.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/linux/skbuff.h         |  4 +++-
 include/uapi/linux/bpf.h       |  4 ++++
 net/core/skbuff.c              | 11 +++++++----
 net/dsa/user.c                 |  2 +-
 net/socket.c                   |  2 +-
 tools/include/uapi/linux/bpf.h |  4 ++++
 6 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 76582500c5ea..0b4f1889500d 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -470,7 +470,7 @@ struct skb_shared_hwtstamps {
 /* Definitions for tx_flags in struct skb_shared_info */
 enum {
 	/* generate hardware time stamp */
-	SKBTX_HW_TSTAMP = 1 << 0,
+	SKBTX_HW_TSTAMP_NOBPF = 1 << 0,
 
 	/* generate software time stamp when queueing packet to NIC */
 	SKBTX_SW_TSTAMP = 1 << 1,
@@ -494,6 +494,8 @@ enum {
 	SKBTX_BPF = 1 << 7,
 };
 
+#define SKBTX_HW_TSTAMP		(SKBTX_HW_TSTAMP_NOBPF | SKBTX_BPF)
+
 #define SKBTX_ANY_SW_TSTAMP	(SKBTX_SW_TSTAMP    | \
 				 SKBTX_SCHED_TSTAMP | \
 				 SKBTX_BPF)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index b3bd92281084..f70edd067edf 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7043,6 +7043,10 @@ enum {
 					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_HW_OPT_CB,	/* Called in hardware phase when
+					 * SK_BPF_CB_TX_TIMESTAMPING feature
+					 * is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 03b90f04d0b0..acafa05f7f58 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5547,7 +5547,7 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb,
 	case SCM_TSTAMP_SCHED:
 		return skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP;
 	case SCM_TSTAMP_SND:
-		return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP :
+		return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP_NOBPF :
 						    SKBTX_SW_TSTAMP);
 	case SCM_TSTAMP_ACK:
 		return TCP_SKB_CB(skb)->txstamp_ack;
@@ -5568,9 +5568,12 @@ static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb,
 		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
 		break;
 	case SCM_TSTAMP_SND:
-		if (hwtstamps)
-			return;
-		op = BPF_SOCK_OPS_TS_SW_OPT_CB;
+		if (hwtstamps) {
+			op = BPF_SOCK_OPS_TS_HW_OPT_CB;
+			*skb_hwtstamps(skb) = *hwtstamps;
+		} else {
+			op = BPF_SOCK_OPS_TS_SW_OPT_CB;
+		}
 		break;
 	default:
 		return;
diff --git a/net/dsa/user.c b/net/dsa/user.c
index 291ab1b4acc4..794fe553dd77 100644
--- a/net/dsa/user.c
+++ b/net/dsa/user.c
@@ -897,7 +897,7 @@ static void dsa_skb_tx_timestamp(struct dsa_user_priv *p,
 {
 	struct dsa_switch *ds = p->dp->ds;
 
-	if (!(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
+	if (!(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP_NOBPF))
 		return;
 
 	if (!ds->ops->port_txtstamp)
diff --git a/net/socket.c b/net/socket.c
index 262a28b59c7f..517de433d4bb 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -676,7 +676,7 @@ void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags)
 	u8 flags = *tx_flags;
 
 	if (tsflags & SOF_TIMESTAMPING_TX_HARDWARE) {
-		flags |= SKBTX_HW_TSTAMP;
+		flags |= SKBTX_HW_TSTAMP_NOBPF;
 
 		/* PTP hardware clocks can provide a free running cycle counter
 		 * as a time base for virtual clocks. Tell driver to use the
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 9bd1c7c77b17..7b9652ce7e3c 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7033,6 +7033,10 @@ enum {
 					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_HW_OPT_CB,	/* Called in hardware phase when
+					 * SK_BPF_CB_TX_TIMESTAMPING feature
+					 * is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
-- 
2.43.5

[PATCH bpf-next v12 09/12] bpf: add BPF_SOCK_OPS_TS_ACK_OPT_CB callback

[Jason Xing]

Support the ACK case for bpf timestamping.

Add a new sock_ops callback, BPF_SOCK_OPS_TS_ACK_OPT_CB. This
callback will occur at the same timestamping point as the user
space's SCM_TSTAMP_ACK. The BPF program can use it to get the
same SCM_TSTAMP_ACK timestamp without modifying the user-space
application.

This patch extends txstamp_ack to two bits: 1 stands for
SO_TIMESTAMPING mode, 2 bpf extension.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/net/tcp.h              | 6 ++++--
 include/uapi/linux/bpf.h       | 5 +++++
 net/core/skbuff.c              | 5 ++++-
 net/ipv4/tcp.c                 | 2 +-
 tools/include/uapi/linux/bpf.h | 5 +++++
 5 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 4c4dca59352b..2e2fc72e115b 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -958,10 +958,12 @@ struct tcp_skb_cb {
 
 	__u8		sacked;		/* State flags for SACK.	*/
 	__u8		ip_dsfield;	/* IPv4 tos or IPv6 dsfield	*/
-	__u8		txstamp_ack:1,	/* Record TX timestamp for ack? */
+#define TSTAMP_ACK_SK	0x1
+#define TSTAMP_ACK_BPF	0x2
+	__u8		txstamp_ack:2,	/* Record TX timestamp for ack? */
 			eor:1,		/* Is skb MSG_EOR marked? */
 			has_rxtstamp:1,	/* SKB has a RX timestamp	*/
-			unused:5;
+			unused:4;
 	__u32		ack_seq;	/* Sequence number ACK'd	*/
 	union {
 		struct {
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index f70edd067edf..9355d617767f 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7047,6 +7047,11 @@ enum {
 					 * SK_BPF_CB_TX_TIMESTAMPING feature
 					 * is on.
 					 */
+	BPF_SOCK_OPS_TS_ACK_OPT_CB,	/* Called when all the skbs in the
+					 * same sendmsg call are acked
+					 * when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index acafa05f7f58..f096ca6c2ced 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5550,7 +5550,7 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb,
 		return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP_NOBPF :
 						    SKBTX_SW_TSTAMP);
 	case SCM_TSTAMP_ACK:
-		return TCP_SKB_CB(skb)->txstamp_ack;
+		return TCP_SKB_CB(skb)->txstamp_ack & TSTAMP_ACK_SK;
 	}
 
 	return false;
@@ -5575,6 +5575,9 @@ static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb,
 			op = BPF_SOCK_OPS_TS_SW_OPT_CB;
 		}
 		break;
+	case SCM_TSTAMP_ACK:
+		op = BPF_SOCK_OPS_TS_ACK_OPT_CB;
+		break;
 	default:
 		return;
 	}
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 0d704bda6c41..12b9c4f9c151 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -488,7 +488,7 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
 
 		sock_tx_timestamp(sk, sockc, &shinfo->tx_flags);
 		if (tsflags & SOF_TIMESTAMPING_TX_ACK)
-			tcb->txstamp_ack = 1;
+			tcb->txstamp_ack |= TSTAMP_ACK_SK;
 		if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
 			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
 	}
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 7b9652ce7e3c..d3e2988b3b4c 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7037,6 +7037,11 @@ enum {
 					 * SK_BPF_CB_TX_TIMESTAMPING feature
 					 * is on.
 					 */
+	BPF_SOCK_OPS_TS_ACK_OPT_CB,	/* Called when all the skbs in the
+					 * same sendmsg call are acked
+					 * when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
-- 
2.43.5

[PATCH bpf-next v12 10/12] bpf: add BPF_SOCK_OPS_TS_SND_CB callback

[Jason Xing]

This patch introduces a new callback in tcp_tx_timestamp() to correlate
tcp_sendmsg timestamp with timestamps from other tx timestamping
callbacks (e.g., SND/SW/ACK).

Without this patch, BPF program wouldn't know which timestamps belong
to which flow because of no socket lock protection. This new callback
is inserted in tcp_tx_timestamp() to address this issue because
tcp_tx_timestamp() still owns the same socket lock with
tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
the timestamping related fields for the skb, especially tskey. The
tskey is the bridge to do the correlation.

For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
then stores the sendmsg timestamp at the bpf_sk_storage, correlating
this timestamp with its tskey that are later used in other sending
timestamping callbacks.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 include/uapi/linux/bpf.h       | 5 +++++
 net/ipv4/tcp.c                 | 4 ++++
 tools/include/uapi/linux/bpf.h | 5 +++++
 3 files changed, 14 insertions(+)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 9355d617767f..86fca729fbd8 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7052,6 +7052,11 @@ enum {
 					 * when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SND_CB,		/* Called when every sendmsg syscall
+					 * is triggered. It's used to correlate
+					 * sendmsg timestamp with corresponding
+					 * tskey.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 12b9c4f9c151..4b9739cd3bc5 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -492,6 +492,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
 		if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
 			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
 	}
+
+	if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
+	    SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb)
+		bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB);
 }
 
 static bool tcp_stream_is_readable(struct sock *sk, int target)
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index d3e2988b3b4c..2739ee0154a0 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7042,6 +7042,11 @@ enum {
 					 * when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SND_CB,		/* Called when every sendmsg syscall
+					 * is triggered. It's used to correlate
+					 * sendmsg timestamp with corresponding
+					 * tskey.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
-- 
2.43.5

[PATCH bpf-next v12 11/12] bpf: support selective sampling for bpf timestamping

[Jason Xing]

Add the bpf_sock_ops_enable_tx_tstamp kfunc to allow BPF programs to
selectively enable TX timestamping on a skb during tcp_sendmsg().

For example, BPF program will limit tracking X numbers of packets
and then will stop there instead of tracing all the sendmsgs of
matched flow all along. It would be helpful for users who cannot
afford to calculate latencies from every sendmsg call probably
due to the performance or storage space consideration.

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 kernel/bpf/btf.c  |  1 +
 net/core/filter.c | 31 ++++++++++++++++++++++++++++++-
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 9433b6467bbe..740210f883dc 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -8522,6 +8522,7 @@ static int bpf_prog_type_to_kfunc_hook(enum bpf_prog_type prog_type)
 	case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
 	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
 	case BPF_PROG_TYPE_CGROUP_SYSCTL:
+	case BPF_PROG_TYPE_SOCK_OPS:
 		return BTF_KFUNC_HOOK_CGROUP;
 	case BPF_PROG_TYPE_SCHED_ACT:
 		return BTF_KFUNC_HOOK_SCHED_ACT;
diff --git a/net/core/filter.c b/net/core/filter.c
index 7f56d0bbeb00..5611d4a7cf89 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -12102,6 +12102,25 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk,
 #endif
 }
 
+__bpf_kfunc int bpf_sock_ops_enable_tx_tstamp(struct bpf_sock_ops_kern *skops,
+					      u64 flags)
+{
+	struct sk_buff *skb;
+
+	if (skops->op != BPF_SOCK_OPS_TS_SND_CB)
+		return -EOPNOTSUPP;
+
+	if (flags)
+		return -EINVAL;
+
+	skb = skops->skb;
+	skb_shinfo(skb)->tx_flags |= SKBTX_BPF;
+	TCP_SKB_CB(skb)->txstamp_ack |= TSTAMP_ACK_BPF;
+	skb_shinfo(skb)->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
+
+	return 0;
+}
+
 __bpf_kfunc_end_defs();
 
 int bpf_dynptr_from_skb_rdonly(struct __sk_buff *skb, u64 flags,
@@ -12135,6 +12154,10 @@ BTF_KFUNCS_START(bpf_kfunc_check_set_tcp_reqsk)
 BTF_ID_FLAGS(func, bpf_sk_assign_tcp_reqsk, KF_TRUSTED_ARGS)
 BTF_KFUNCS_END(bpf_kfunc_check_set_tcp_reqsk)
 
+BTF_KFUNCS_START(bpf_kfunc_check_set_sock_ops)
+BTF_ID_FLAGS(func, bpf_sock_ops_enable_tx_tstamp, KF_TRUSTED_ARGS)
+BTF_KFUNCS_END(bpf_kfunc_check_set_sock_ops)
+
 static const struct btf_kfunc_id_set bpf_kfunc_set_skb = {
 	.owner = THIS_MODULE,
 	.set = &bpf_kfunc_check_set_skb,
@@ -12155,6 +12178,11 @@ static const struct btf_kfunc_id_set bpf_kfunc_set_tcp_reqsk = {
 	.set = &bpf_kfunc_check_set_tcp_reqsk,
 };
 
+static const struct btf_kfunc_id_set bpf_kfunc_set_sock_ops = {
+	.owner = THIS_MODULE,
+	.set = &bpf_kfunc_check_set_sock_ops,
+};
+
 static int __init bpf_kfunc_init(void)
 {
 	int ret;
@@ -12173,7 +12201,8 @@ static int __init bpf_kfunc_init(void)
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &bpf_kfunc_set_xdp);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_CGROUP_SOCK_ADDR,
 					       &bpf_kfunc_set_sock_addr);
-	return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_kfunc_set_tcp_reqsk);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_kfunc_set_tcp_reqsk);
+	return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SOCK_OPS, &bpf_kfunc_set_sock_ops);
 }
 late_initcall(bpf_kfunc_init);
 
-- 
2.43.5

[PATCH bpf-next v12 12/12] selftests/bpf: add simple bpf tests in the tx path for timestamping feature

[Jason Xing]

BPF program calculates a couple of latency deltas between each tx
timestamping callbacks. It can be used in the real world to diagnose
the kernel behaviour in the tx path.

Check the safety issues by accessing a few bpf calls in
bpf_test_access_bpf_calls() which are implemented in the patch 3 and 4.

Check if the bpf timestamping can co-exist with socket timestamping.

There remains a few realistic things[1][2] to highlight:
1. in general a packet may pass through multiple qdiscs. For instance
with bonding or tunnel virtual devices in the egress path.
2. packets may be resent, in which case an ACK might precede a repeat
SCHED and SND.
3. erroneous or malicious peers may also just never send an ACK.

[1]: https://lore.kernel.org/all/67a389af981b0_14e0832949d@willemb.c.googlers.com.notmuch/
[2]: https://lore.kernel.org/all/c329a0c1-239b-4ca1-91f2-cb30b8dd2f6a@linux.dev/

Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
---
 .../bpf/prog_tests/net_timestamping.c         | 239 +++++++++++++++++
 .../selftests/bpf/progs/net_timestamping.c    | 248 ++++++++++++++++++
 2 files changed, 487 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/net_timestamping.c
 create mode 100644 tools/testing/selftests/bpf/progs/net_timestamping.c

diff --git a/tools/testing/selftests/bpf/prog_tests/net_timestamping.c b/tools/testing/selftests/bpf/prog_tests/net_timestamping.c
new file mode 100644
index 000000000000..dbfd87499b6b
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/net_timestamping.c
@@ -0,0 +1,239 @@
+#include <linux/net_tstamp.h>
+#include <sys/time.h>
+#include <linux/errqueue.h>
+#include "test_progs.h"
+#include "network_helpers.h"
+#include "net_timestamping.skel.h"
+
+#define CG_NAME "/net-timestamping-test"
+#define NSEC_PER_SEC    1000000000LL
+
+static const char addr4_str[] = "127.0.0.1";
+static const char addr6_str[] = "::1";
+static struct net_timestamping *skel;
+static const int cfg_payload_len = 30;
+static struct timespec usr_ts;
+static u64 delay_tolerance_nsec = 10000000000; /* 10 seconds */
+int SK_TS_SCHED;
+int SK_TS_TXSW;
+int SK_TS_ACK;
+
+static int64_t timespec_to_ns64(struct timespec *ts)
+{
+	return ts->tv_sec * NSEC_PER_SEC + ts->tv_nsec;
+}
+
+static void validate_key(int tskey, int tstype)
+{
+	static int expected_tskey = -1;
+
+	if (tstype == SCM_TSTAMP_SCHED)
+		expected_tskey = cfg_payload_len - 1;
+
+	ASSERT_EQ(expected_tskey, tskey, "tskey mismatch");
+
+	expected_tskey = tskey;
+}
+
+static void validate_timestamp(struct timespec *cur, struct timespec *prev)
+{
+	int64_t cur_ns, prev_ns;
+
+	cur_ns = timespec_to_ns64(cur);
+	prev_ns = timespec_to_ns64(prev);
+
+	ASSERT_LT(cur_ns - prev_ns, delay_tolerance_nsec, "latency");
+}
+
+static void test_socket_timestamp(struct scm_timestamping *tss, int tstype,
+				  int tskey)
+{
+	static struct timespec prev_ts;
+
+	validate_key(tskey, tstype);
+
+	switch (tstype) {
+	case SCM_TSTAMP_SCHED:
+		validate_timestamp(&tss->ts[0], &usr_ts);
+		SK_TS_SCHED += 1;
+		break;
+	case SCM_TSTAMP_SND:
+		validate_timestamp(&tss->ts[0], &prev_ts);
+		SK_TS_TXSW += 1;
+		break;
+	case SCM_TSTAMP_ACK:
+		validate_timestamp(&tss->ts[0], &prev_ts);
+		SK_TS_ACK += 1;
+		break;
+	}
+
+	prev_ts = tss->ts[0];
+}
+
+static void test_recv_errmsg_cmsg(struct msghdr *msg)
+{
+	struct sock_extended_err *serr = NULL;
+	struct scm_timestamping *tss = NULL;
+	struct cmsghdr *cm;
+
+	for (cm = CMSG_FIRSTHDR(msg);
+	     cm && cm->cmsg_len;
+	     cm = CMSG_NXTHDR(msg, cm)) {
+		if (cm->cmsg_level == SOL_SOCKET &&
+		    cm->cmsg_type == SCM_TIMESTAMPING) {
+			tss = (void *)CMSG_DATA(cm);
+		} else if ((cm->cmsg_level == SOL_IP &&
+			    cm->cmsg_type == IP_RECVERR) ||
+			   (cm->cmsg_level == SOL_IPV6 &&
+			    cm->cmsg_type == IPV6_RECVERR) ||
+			   (cm->cmsg_level == SOL_PACKET &&
+			    cm->cmsg_type == PACKET_TX_TIMESTAMP)) {
+			serr = (void *)CMSG_DATA(cm);
+			ASSERT_EQ(serr->ee_origin, SO_EE_ORIGIN_TIMESTAMPING,
+				  "cmsg type");
+		}
+
+		if (serr && tss)
+			test_socket_timestamp(tss, serr->ee_info,
+					      serr->ee_data);
+	}
+}
+
+static bool socket_recv_errmsg(int fd)
+{
+	static char ctrl[1024 /* overprovision*/];
+	char data[cfg_payload_len];
+	static struct msghdr msg;
+	struct iovec entry;
+	int n = 0;
+
+	memset(&msg, 0, sizeof(msg));
+	memset(&entry, 0, sizeof(entry));
+	memset(ctrl, 0, sizeof(ctrl));
+
+	entry.iov_base = data;
+	entry.iov_len = cfg_payload_len;
+	msg.msg_iov = &entry;
+	msg.msg_iovlen = 1;
+	msg.msg_name = NULL;
+	msg.msg_namelen = 0;
+	msg.msg_control = ctrl;
+	msg.msg_controllen = sizeof(ctrl);
+
+	n = recvmsg(fd, &msg, MSG_ERRQUEUE);
+	if (n == -1)
+		ASSERT_EQ(errno, EAGAIN, "recvmsg MSG_ERRQUEUE");
+
+	if (n >= 0)
+		test_recv_errmsg_cmsg(&msg);
+
+	return n == -1;
+}
+
+static void test_socket_timestamping(int fd)
+{
+	while (!socket_recv_errmsg(fd));
+
+	ASSERT_EQ(SK_TS_SCHED, 1, "SCM_TSTAMP_SCHED");
+	ASSERT_EQ(SK_TS_TXSW, 1, "SCM_TSTAMP_SND");
+	ASSERT_EQ(SK_TS_ACK, 1, "SCM_TSTAMP_ACK");
+
+	SK_TS_SCHED = 0;
+	SK_TS_TXSW = 0;
+	SK_TS_ACK = 0;
+}
+
+static void test_tcp(int family, bool enable_socket_timestamping)
+{
+	struct net_timestamping__bss *bss;
+	char buf[cfg_payload_len];
+	int sfd = -1, cfd = -1;
+	unsigned int sock_opt;
+	struct netns_obj *ns;
+	int cg_fd;
+	int ret;
+
+	cg_fd = test__join_cgroup(CG_NAME);
+	if (!ASSERT_OK_FD(cg_fd, "join cgroup"))
+		return;
+
+	ns = netns_new("net_timestamping_ns", true);
+	if (!ASSERT_OK_PTR(ns, "create ns"))
+		goto out;
+
+	skel = net_timestamping__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open and load skel"))
+		goto out;
+
+	if (!ASSERT_OK(net_timestamping__attach(skel), "attach skel"))
+		goto out;
+
+	skel->links.skops_sockopt =
+		bpf_program__attach_cgroup(skel->progs.skops_sockopt, cg_fd);
+	if (!ASSERT_OK_PTR(skel->links.skops_sockopt, "attach cgroup"))
+		goto out;
+
+	bss = skel->bss;
+	memset(bss, 0, sizeof(*bss));
+
+	skel->bss->monitored_pid = getpid();
+
+	sfd = start_server(family, SOCK_STREAM,
+			   family == AF_INET6 ? addr6_str : addr4_str, 0, 0);
+	if (!ASSERT_OK_FD(sfd, "start_server"))
+		goto out;
+
+	cfd = connect_to_fd(sfd, 0);
+	if (!ASSERT_OK_FD(cfd, "connect_to_fd_server"))
+		goto out;
+
+	if (enable_socket_timestamping) {
+		sock_opt = SOF_TIMESTAMPING_SOFTWARE |
+			   SOF_TIMESTAMPING_OPT_ID |
+			   SOF_TIMESTAMPING_TX_SCHED |
+			   SOF_TIMESTAMPING_TX_SOFTWARE |
+			   SOF_TIMESTAMPING_TX_ACK;
+		ret = setsockopt(cfd, SOL_SOCKET, SO_TIMESTAMPING,
+				 (char *) &sock_opt, sizeof(sock_opt));
+		if (!ASSERT_OK(ret, "setsockopt SO_TIMESTAMPING"))
+			goto out;
+
+		ret = clock_gettime(CLOCK_REALTIME, &usr_ts);
+		if (!ASSERT_OK(ret, "get user time"))
+			goto out;
+	}
+
+	ret = write(cfd, buf, sizeof(buf));
+	if (!ASSERT_EQ(ret, sizeof(buf), "send to server"))
+		goto out;
+
+	if (enable_socket_timestamping)
+		test_socket_timestamping(cfd);
+
+	ASSERT_EQ(bss->nr_active, 1, "nr_active");
+	ASSERT_EQ(bss->nr_snd, 2, "nr_snd");
+	ASSERT_EQ(bss->nr_sched, 1, "nr_sched");
+	ASSERT_EQ(bss->nr_txsw, 1, "nr_txsw");
+	ASSERT_EQ(bss->nr_ack, 1, "nr_ack");
+
+out:
+	if (sfd >= 0)
+		close(sfd);
+	if (cfd >= 0)
+		close(cfd);
+	net_timestamping__destroy(skel);
+	netns_free(ns);
+	close(cg_fd);
+}
+
+void test_net_timestamping(void)
+{
+	if (test__start_subtest("INET4: bpf timestamping"))
+		test_tcp(AF_INET, false);
+	if (test__start_subtest("INET4: bpf and socket timestamping"))
+		test_tcp(AF_INET, true);
+	if (test__start_subtest("INET6: bpf timestamping"))
+		test_tcp(AF_INET6, false);
+	if (test__start_subtest("INET6: bpf and socket timestamping"))
+		test_tcp(AF_INET6, true);
+}
diff --git a/tools/testing/selftests/bpf/progs/net_timestamping.c b/tools/testing/selftests/bpf/progs/net_timestamping.c
new file mode 100644
index 000000000000..ab2ed2ea5822
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/net_timestamping.c
@@ -0,0 +1,248 @@
+#include "vmlinux.h"
+#include "bpf_tracing_net.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+#include "bpf_kfuncs.h"
+#include <errno.h>
+
+__u32 monitored_pid = 0;
+
+int nr_active;
+int nr_snd;
+int nr_passive;
+int nr_sched;
+int nr_txsw;
+int nr_ack;
+
+struct sk_stg {
+	__u64 sendmsg_ns;	/* record ts when sendmsg is called */
+};
+
+struct sk_tskey {
+	u64 cookie;
+	u32 tskey;
+};
+
+struct delay_info {
+	u64 sendmsg_ns;		/* record ts when sendmsg is called */
+	u32 sched_delay;	/* SCHED_OPT_CB - sendmsg_ns */
+	u32 sw_snd_delay;	/* SW_OPT_CB - SCHED_OPT_CB */
+	u32 ack_delay;		/* ACK_OPT_CB - SW_OPT_CB */
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_SK_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, struct sk_stg);
+} sk_stg_map SEC(".maps");
+
+struct {
+	__uint(type, BPF_MAP_TYPE_HASH);
+	__type(key, struct sk_tskey);
+	__type(value, struct delay_info);
+	__uint(max_entries, 1024);
+} time_map SEC(".maps");
+
+static u64 delay_tolerance_nsec = 10000000000; /* 10 second as an example */
+
+extern int bpf_sock_ops_enable_tx_tstamp(struct bpf_sock_ops_kern *skops, u64 flags) __ksym;
+
+static int bpf_test_sockopt(void *ctx, const struct sock *sk, int expected)
+{
+	int tmp, new = SK_BPF_CB_TX_TIMESTAMPING;
+	int opt = SK_BPF_CB_FLAGS;
+	int level = SOL_SOCKET;
+
+	if (bpf_setsockopt(ctx, level, opt, &new, sizeof(new)) != expected)
+		return 1;
+
+	if (bpf_getsockopt(ctx, level, opt, &tmp, sizeof(tmp)) != expected ||
+	    (!expected && tmp != new))
+		return 1;
+
+	return 0;
+}
+
+static bool bpf_test_access_sockopt(void *ctx, const struct sock *sk)
+{
+	if (bpf_test_sockopt(ctx, sk, -EOPNOTSUPP))
+		return true;
+	return false;
+}
+
+static bool bpf_test_access_load_hdr_opt(struct bpf_sock_ops *skops)
+{
+	u8 opt[3] = {0};
+	int load_flags = 0;
+	int ret;
+
+	ret = bpf_load_hdr_opt(skops, opt, sizeof(opt), load_flags);
+	if (ret != -EOPNOTSUPP)
+		return true;
+
+	return false;
+}
+
+static bool bpf_test_access_cb_flags_set(struct bpf_sock_ops *skops)
+{
+	int ret;
+
+	ret = bpf_sock_ops_cb_flags_set(skops, 0);
+	if (ret != -EOPNOTSUPP)
+		return true;
+
+	return false;
+}
+
+/* In the timestamping callbacks, we're not allowed to call the following
+ * BPF CALLs for the safety concern. Return false if expected.
+ */
+static bool bpf_test_access_bpf_calls(struct bpf_sock_ops *skops,
+				      const struct sock *sk)
+{
+	if (bpf_test_access_sockopt(skops, sk))
+		return true;
+
+	if (bpf_test_access_load_hdr_opt(skops))
+		return true;
+
+	if (bpf_test_access_cb_flags_set(skops))
+		return true;
+
+	return false;
+}
+
+static bool bpf_test_delay(struct bpf_sock_ops *skops, const struct sock *sk)
+{
+	struct bpf_sock_ops_kern *skops_kern;
+	u64 timestamp = bpf_ktime_get_ns();
+	struct skb_shared_info *shinfo;
+	struct delay_info dinfo = {0};
+	struct sk_tskey key = {0};
+	struct delay_info *val;
+	struct sk_buff *skb;
+	struct sk_stg *stg;
+	u64 prior_ts, delay;
+
+	if (bpf_test_access_bpf_calls(skops, sk))
+		return false;
+
+	skops_kern = bpf_cast_to_kern_ctx(skops);
+	skb = skops_kern->skb;
+	shinfo = bpf_core_cast(skb->head + skb->end, struct skb_shared_info);
+
+	key.cookie = bpf_get_socket_cookie(skops);
+	if (!key.cookie)
+		return false;
+
+	if (skops->op == BPF_SOCK_OPS_TS_SND_CB) {
+		stg = bpf_sk_storage_get(&sk_stg_map, (void *)sk, 0, 0);
+		if (!stg)
+			return false;
+		dinfo.sendmsg_ns = stg->sendmsg_ns;
+		bpf_sock_ops_enable_tx_tstamp(skops_kern, 0);
+		key.tskey = shinfo->tskey;
+		if (!key.tskey)
+			return false;
+		bpf_map_update_elem(&time_map, &key, &dinfo, BPF_ANY);
+		return true;
+	}
+
+	key.tskey = shinfo->tskey;
+	if (!key.tskey)
+		return false;
+
+	val = bpf_map_lookup_elem(&time_map, &key);
+	if (!val)
+		return false;
+
+	switch (skops->op) {
+	case BPF_SOCK_OPS_TS_SCHED_OPT_CB:
+		val->sched_delay = timestamp - val->sendmsg_ns;
+		delay = val->sched_delay;
+		break;
+	case BPF_SOCK_OPS_TS_SW_OPT_CB:
+		prior_ts = val->sched_delay + val->sendmsg_ns;
+		val->sw_snd_delay = timestamp - prior_ts;
+		delay = val->sw_snd_delay;
+		break;
+	case BPF_SOCK_OPS_TS_ACK_OPT_CB:
+		prior_ts = val->sw_snd_delay + val->sched_delay + val->sendmsg_ns;
+		val->ack_delay = timestamp - prior_ts;
+		delay = val->ack_delay;
+		break;
+	}
+
+	if (delay >= delay_tolerance_nsec)
+		return false;
+
+	/* Since it's the last one, remove from the map after latency check */
+	if (skops->op == BPF_SOCK_OPS_TS_ACK_OPT_CB)
+		bpf_map_delete_elem(&time_map, &key);
+
+	return true;
+}
+
+SEC("fentry/tcp_sendmsg_locked")
+int BPF_PROG(trace_tcp_sendmsg_locked, struct sock *sk, struct msghdr *msg,
+	     size_t size)
+{
+	__u32 pid = bpf_get_current_pid_tgid() >> 32;
+	u64 timestamp = bpf_ktime_get_ns();
+	u32 flag = sk->sk_bpf_cb_flags;
+	struct sk_stg *stg;
+
+	if (pid != monitored_pid || !flag)
+		return 0;
+
+	stg = bpf_sk_storage_get(&sk_stg_map, sk, 0,
+				 BPF_SK_STORAGE_GET_F_CREATE);
+	if (!stg)
+		return 0;
+
+	stg->sendmsg_ns = timestamp;
+	nr_snd += 1;
+	return 0;
+}
+
+SEC("sockops")
+int skops_sockopt(struct bpf_sock_ops *skops)
+{
+	struct bpf_sock *bpf_sk = skops->sk;
+	const struct sock *sk;
+
+	if (!bpf_sk)
+		return 1;
+
+	sk = (struct sock *)bpf_skc_to_tcp_sock(bpf_sk);
+	if (!sk)
+		return 1;
+
+	switch (skops->op) {
+	case BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB:
+		nr_active += !bpf_test_sockopt(skops, sk, 0);
+		break;
+	case BPF_SOCK_OPS_TS_SND_CB:
+		if (bpf_test_delay(skops, sk))
+			nr_snd += 1;
+		break;
+	case BPF_SOCK_OPS_TS_SCHED_OPT_CB:
+		if (bpf_test_delay(skops, sk))
+			nr_sched += 1;
+		break;
+	case BPF_SOCK_OPS_TS_SW_OPT_CB:
+		if (bpf_test_delay(skops, sk))
+			nr_txsw += 1;
+		break;
+	case BPF_SOCK_OPS_TS_ACK_OPT_CB:
+		if (bpf_test_delay(skops, sk))
+			nr_ack += 1;
+		break;
+	}
+
+	return 1;
+}
+
+char _license[] SEC("license") = "GPL";
-- 
2.43.5

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Willem de Bruijn]

Jason Xing wrote:
> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
> added to bpf_get/setsockopt. The later patches will implement the
> BPF networking timestamping. The BPF program will use
> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
> enable the BPF networking timestamping on a socket.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> ---
>  include/net/sock.h             |  3 +++
>  include/uapi/linux/bpf.h       |  8 ++++++++
>  net/core/filter.c              | 23 +++++++++++++++++++++++
>  tools/include/uapi/linux/bpf.h |  1 +
>  4 files changed, 35 insertions(+)
> 
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 8036b3b79cd8..7916982343c6 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -303,6 +303,7 @@ struct sk_filter;
>    *	@sk_stamp: time stamp of last packet received
>    *	@sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
>    *	@sk_tsflags: SO_TIMESTAMPING flags
> +  *	@sk_bpf_cb_flags: used in bpf_setsockopt()
>    *	@sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
>    *			   Sockets that can be used under memory reclaim should
>    *			   set this to false.
> @@ -445,6 +446,8 @@ struct sock {
>  	u32			sk_reserved_mem;
>  	int			sk_forward_alloc;
>  	u32			sk_tsflags;
> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> +	u32			sk_bpf_cb_flags;
>  	__cacheline_group_end(sock_write_rxtx);

So far only one bit is defined. Does this have to be a 32-bit field in
every socket?

Sorry for the late notice. I had not followed the BPF part closely, as
that is not my expertise.

Re: [PATCH bpf-next v12 05/12] net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING

[Willem de Bruijn]

Jason Xing wrote:
> No functional changes here. Only add test to see if the orig_skb
> matches the usage of application SO_TIMESTAMPING.
> 
> In this series, bpf timestamping and previous socket timestamping
> are implemented in the same function __skb_tstamp_tx(). To test
> the socket enables socket timestamping feature, this function
> skb_tstamp_tx_report_so_timestamping() is added.
> 
> In the next patch, another check for bpf timestamping feature
> will be introduced just like the above report function, namely,
> skb_tstamp_tx_report_bpf_timestamping(). Then users will be able
> to know the socket enables either or both of features.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>

Reviewed-by: Willem de Bruijn <willemb@google.com>

Re: [PATCH bpf-next v12 06/12] bpf: add BPF_SOCK_OPS_TS_SCHED_OPT_CB callback

[Willem de Bruijn]

Jason Xing wrote:
> Support SCM_TSTAMP_SCHED case for bpf timestamping.
> 
> Add a new sock_ops callback, BPF_SOCK_OPS_TS_SCHED_OPT_CB. This
> callback will occur at the same timestamping point as the user
> space's SCM_TSTAMP_SCHED. The BPF program can use it to get the
> same SCM_TSTAMP_SCHED timestamp without modifying the user-space
> application.
> 
> A new SKBTX_BPF flag is added to mark skb_shinfo(skb)->tx_flags,
> ensuring that the new BPF timestamping and the current user
> space's SO_TIMESTAMPING do not interfere with each other.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>

Reviewed-by: Willem de Bruijn <willemb@google.com>

Re: [PATCH bpf-next v12 07/12] bpf: add BPF_SOCK_OPS_TS_SW_OPT_CB callback

[Willem de Bruijn]

Jason Xing wrote:
> Support sw SCM_TSTAMP_SND case for bpf timestamping.
> 
> Add a new sock_ops callback, BPF_SOCK_OPS_TS_SW_OPT_CB. This
> callback will occur at the same timestamping point as the user
> space's software SCM_TSTAMP_SND. The BPF program can use it to
> get the same SCM_TSTAMP_SND timestamp without modifying the
> user-space application.
> 
> Based on this patch, BPF program will get the software
> timestamp when the driver is ready to send the skb. In the
> sebsequent patch, the hardware timestamp will be supported.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>

Reviewed-by: Willem de Bruijn <willemb@google.com>

Re: [PATCH bpf-next v12 08/12] bpf: add BPF_SOCK_OPS_TS_HW_OPT_CB callback

[Willem de Bruijn]

Jason Xing wrote:
> Support hw SCM_TSTAMP_SND case for bpf timestamping.
> 
> Add a new sock_ops callback, BPF_SOCK_OPS_TS_HW_OPT_CB. This
> callback will occur at the same timestamping point as the user
> space's hardware SCM_TSTAMP_SND. The BPF program can use it to
> get the same SCM_TSTAMP_SND timestamp without modifying the
> user-space application.
> 
> To avoid increasing the code complexity, replace SKBTX_HW_TSTAMP
> with SKBTX_HW_TSTAMP_NOBPF instead of changing numerous callers
> from driver side using SKBTX_HW_TSTAMP. The new definition of
> SKBTX_HW_TSTAMP means the combination tests of socket timestamping
> and bpf timestamping. After this patch, drivers can work under the
> bpf timestamping.
> 
> Considering some drivers don't assign the skb with hardware
> timestamp, this patch does the assignment and then BPF program
> can acquire the hwstamp from skb directly.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>

Reviewed-by: Willem de Bruijn <willemb@google.com>

Re: [PATCH bpf-next v12 09/12] bpf: add BPF_SOCK_OPS_TS_ACK_OPT_CB callback

[Willem de Bruijn]

Jason Xing wrote:
> Support the ACK case for bpf timestamping.
> 
> Add a new sock_ops callback, BPF_SOCK_OPS_TS_ACK_OPT_CB. This
> callback will occur at the same timestamping point as the user
> space's SCM_TSTAMP_ACK. The BPF program can use it to get the
> same SCM_TSTAMP_ACK timestamp without modifying the user-space
> application.
> 
> This patch extends txstamp_ack to two bits: 1 stands for
> SO_TIMESTAMPING mode, 2 bpf extension.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>

Reviewed-by: Willem de Bruijn <willemb@google.com>

Re: [PATCH bpf-next v12 10/12] bpf: add BPF_SOCK_OPS_TS_SND_CB callback

[Willem de Bruijn]

Jason Xing wrote:
> This patch introduces a new callback in tcp_tx_timestamp() to correlate
> tcp_sendmsg timestamp with timestamps from other tx timestamping
> callbacks (e.g., SND/SW/ACK).
> 
> Without this patch, BPF program wouldn't know which timestamps belong
> to which flow because of no socket lock protection. This new callback
> is inserted in tcp_tx_timestamp() to address this issue because
> tcp_tx_timestamp() still owns the same socket lock with
> tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
> the timestamping related fields for the skb, especially tskey. The
> tskey is the bridge to do the correlation.
> 
> For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
> then stores the sendmsg timestamp at the bpf_sk_storage, correlating
> this timestamp with its tskey that are later used in other sending
> timestamping callbacks.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>

Reviewed-by: Willem de Bruijn <willemb@google.com>

Re: [PATCH bpf-next v12 12/12] selftests/bpf: add simple bpf tests in the tx path for timestamping feature

[Willem de Bruijn]

Jason Xing wrote:
> BPF program calculates a couple of latency deltas between each tx
> timestamping callbacks. It can be used in the real world to diagnose
> the kernel behaviour in the tx path.
> 
> Check the safety issues by accessing a few bpf calls in
> bpf_test_access_bpf_calls() which are implemented in the patch 3 and 4.
> 
> Check if the bpf timestamping can co-exist with socket timestamping.
> 
> There remains a few realistic things[1][2] to highlight:
> 1. in general a packet may pass through multiple qdiscs. For instance
> with bonding or tunnel virtual devices in the egress path.
> 2. packets may be resent, in which case an ACK might precede a repeat
> SCHED and SND.
> 3. erroneous or malicious peers may also just never send an ACK.
> 
> [1]: https://lore.kernel.org/all/67a389af981b0_14e0832949d@willemb.c.googlers.com.notmuch/
> [2]: https://lore.kernel.org/all/c329a0c1-239b-4ca1-91f2-cb30b8dd2f6a@linux.dev/
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>

Reviewed-by: Willem de Bruijn <willemb@google.com>

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Martin KaFai Lau]

On 2/18/25 6:22 AM, Willem de Bruijn wrote:
> Jason Xing wrote:
>> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
>> added to bpf_get/setsockopt. The later patches will implement the
>> BPF networking timestamping. The BPF program will use
>> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
>> enable the BPF networking timestamping on a socket.
>>
>> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
>> ---
>>   include/net/sock.h             |  3 +++
>>   include/uapi/linux/bpf.h       |  8 ++++++++
>>   net/core/filter.c              | 23 +++++++++++++++++++++++
>>   tools/include/uapi/linux/bpf.h |  1 +
>>   4 files changed, 35 insertions(+)
>>
>> diff --git a/include/net/sock.h b/include/net/sock.h
>> index 8036b3b79cd8..7916982343c6 100644
>> --- a/include/net/sock.h
>> +++ b/include/net/sock.h
>> @@ -303,6 +303,7 @@ struct sk_filter;
>>     *	@sk_stamp: time stamp of last packet received
>>     *	@sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
>>     *	@sk_tsflags: SO_TIMESTAMPING flags
>> +  *	@sk_bpf_cb_flags: used in bpf_setsockopt()
>>     *	@sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
>>     *			   Sockets that can be used under memory reclaim should
>>     *			   set this to false.
>> @@ -445,6 +446,8 @@ struct sock {
>>   	u32			sk_reserved_mem;
>>   	int			sk_forward_alloc;
>>   	u32			sk_tsflags;
>> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
>> +	u32			sk_bpf_cb_flags;
>>   	__cacheline_group_end(sock_write_rxtx);
> 
> So far only one bit is defined. Does this have to be a 32-bit field in
> every socket?

iirc, I think there were multiple callback (cb) flags/bits in the earlier 
revisions, but it had been simplified to one bit in the later revisions.

It's an internal implementation detail. We can reuse some free bits from another 
variable for now. Probably get a bit from sk_tsflags? SOCKCM_FLAG_TS_OPT_ID uses 
BIT(31). Maybe a new SK_TS_FLAG_BPF_TX that uses BIT(30)? I don't have a strong 
preference on the name.

When the BPF program calls bpf_setsockopt(SK_BPF_CB_FLAGS, 
SK_BPF_CB_TX_TIMESTAMPING), the kernel will set/test the BIT(30) of sk_tsflags.

We can wait until there are more socket-level cb flags in the future (e.g., more 
SK_BPF_CB_XXX will be needed) before adding a dedicated int field in the sock.

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Jason Xing]

On Wed, Feb 19, 2025 at 5:55 AM Martin KaFai Lau <martin.lau@linux.dev> wrote:
>
> On 2/18/25 6:22 AM, Willem de Bruijn wrote:
> > Jason Xing wrote:
> >> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
> >> added to bpf_get/setsockopt. The later patches will implement the
> >> BPF networking timestamping. The BPF program will use
> >> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
> >> enable the BPF networking timestamping on a socket.
> >>
> >> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> >> ---
> >>   include/net/sock.h             |  3 +++
> >>   include/uapi/linux/bpf.h       |  8 ++++++++
> >>   net/core/filter.c              | 23 +++++++++++++++++++++++
> >>   tools/include/uapi/linux/bpf.h |  1 +
> >>   4 files changed, 35 insertions(+)
> >>
> >> diff --git a/include/net/sock.h b/include/net/sock.h
> >> index 8036b3b79cd8..7916982343c6 100644
> >> --- a/include/net/sock.h
> >> +++ b/include/net/sock.h
> >> @@ -303,6 +303,7 @@ struct sk_filter;
> >>     *        @sk_stamp: time stamp of last packet received
> >>     *        @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
> >>     *        @sk_tsflags: SO_TIMESTAMPING flags
> >> +  * @sk_bpf_cb_flags: used in bpf_setsockopt()
> >>     *        @sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
> >>     *                           Sockets that can be used under memory reclaim should
> >>     *                           set this to false.
> >> @@ -445,6 +446,8 @@ struct sock {
> >>      u32                     sk_reserved_mem;
> >>      int                     sk_forward_alloc;
> >>      u32                     sk_tsflags;
> >> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> >> +    u32                     sk_bpf_cb_flags;
> >>      __cacheline_group_end(sock_write_rxtx);
> >
> > So far only one bit is defined. Does this have to be a 32-bit field in
> > every socket?
>
> iirc, I think there were multiple callback (cb) flags/bits in the earlier
> revisions, but it had been simplified to one bit in the later revisions.
>
> It's an internal implementation detail. We can reuse some free bits from another
> variable for now. Probably get a bit from sk_tsflags? SOCKCM_FLAG_TS_OPT_ID uses
> BIT(31). Maybe a new SK_TS_FLAG_BPF_TX that uses BIT(30)? I don't have a strong
> preference on the name.
>
> When the BPF program calls bpf_setsockopt(SK_BPF_CB_FLAGS,
> SK_BPF_CB_TX_TIMESTAMPING), the kernel will set/test the BIT(30) of sk_tsflags.
>
> We can wait until there are more socket-level cb flags in the future (e.g., more
> SK_BPF_CB_XXX will be needed) before adding a dedicated int field in the sock.

Sorry, I still preferred the way we've discussed already:
1) Introducing a new field sk_bpf_cb_flags extends the use for bpf
timestamping, more than SK_BPF_CB_TX_TIMESTAMPING one flag. I think
SK_BPF_CB_RX_TIMESTAMPING is also needed in the next move. And more
subfeatures (like bpf extension for OPT_ID) will use it. It gives us a
separate way to do more things based on this bpf timestamping.
2) sk_bpf_cb_flags provides a way to let the socket-level use new
features for bpf while now we only have a tcp_sock-level, namely,
bpf_sock_ops_cb_flags. It's obviously good for others.

It's the first move to open the gate for socket-level usage for BPF,
just like how TCP_BPF_SOCK_OPS_CB_FLAGS works in sol_tcp_sockopt(). So
I hope we will not abandon this good approach :(

Now I wonder if I should use the u8 sk_bpf_cb_flags in V13 or just
keep it as-is? Either way is fine with me :) bpf_sock_ops_cb_flags
uses u8 as an example, thus I think we prefer the former?

Thanks,
Jason

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Willem de Bruijn]

Jason Xing wrote:
> On Wed, Feb 19, 2025 at 5:55 AM Martin KaFai Lau <martin.lau@linux.dev> wrote:
> >
> > On 2/18/25 6:22 AM, Willem de Bruijn wrote:
> > > Jason Xing wrote:
> > >> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
> > >> added to bpf_get/setsockopt. The later patches will implement the
> > >> BPF networking timestamping. The BPF program will use
> > >> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
> > >> enable the BPF networking timestamping on a socket.
> > >>
> > >> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> > >> ---
> > >>   include/net/sock.h             |  3 +++
> > >>   include/uapi/linux/bpf.h       |  8 ++++++++
> > >>   net/core/filter.c              | 23 +++++++++++++++++++++++
> > >>   tools/include/uapi/linux/bpf.h |  1 +
> > >>   4 files changed, 35 insertions(+)
> > >>
> > >> diff --git a/include/net/sock.h b/include/net/sock.h
> > >> index 8036b3b79cd8..7916982343c6 100644
> > >> --- a/include/net/sock.h
> > >> +++ b/include/net/sock.h
> > >> @@ -303,6 +303,7 @@ struct sk_filter;
> > >>     *        @sk_stamp: time stamp of last packet received
> > >>     *        @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
> > >>     *        @sk_tsflags: SO_TIMESTAMPING flags
> > >> +  * @sk_bpf_cb_flags: used in bpf_setsockopt()
> > >>     *        @sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
> > >>     *                           Sockets that can be used under memory reclaim should
> > >>     *                           set this to false.
> > >> @@ -445,6 +446,8 @@ struct sock {
> > >>      u32                     sk_reserved_mem;
> > >>      int                     sk_forward_alloc;
> > >>      u32                     sk_tsflags;
> > >> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> > >> +    u32                     sk_bpf_cb_flags;
> > >>      __cacheline_group_end(sock_write_rxtx);
> > >
> > > So far only one bit is defined. Does this have to be a 32-bit field in
> > > every socket?
> >
> > iirc, I think there were multiple callback (cb) flags/bits in the earlier
> > revisions, but it had been simplified to one bit in the later revisions.
> >
> > It's an internal implementation detail. We can reuse some free bits from another
> > variable for now. Probably get a bit from sk_tsflags? SOCKCM_FLAG_TS_OPT_ID uses
> > BIT(31). Maybe a new SK_TS_FLAG_BPF_TX that uses BIT(30)? I don't have a strong
> > preference on the name.
> >
> > When the BPF program calls bpf_setsockopt(SK_BPF_CB_FLAGS,
> > SK_BPF_CB_TX_TIMESTAMPING), the kernel will set/test the BIT(30) of sk_tsflags.
> >
> > We can wait until there are more socket-level cb flags in the future (e.g., more
> > SK_BPF_CB_XXX will be needed) before adding a dedicated int field in the sock.
> 
> Sorry, I still preferred the way we've discussed already:

Adding fields to structs in the hot path is a tragedy of the commons.

Every developer focuses on their specific workload and pet feature,
while imposing a cost on everyone else.

We have a duty to be frugal and mitigate this cost where possible.
Especially for a feature that is likely to be used sparingly.

> 1) Introducing a new field sk_bpf_cb_flags extends the use for bpf
> timestamping, more than SK_BPF_CB_TX_TIMESTAMPING one flag. I think
> SK_BPF_CB_RX_TIMESTAMPING is also needed in the next move. And more
> subfeatures (like bpf extension for OPT_ID) will use it. It gives us a
> separate way to do more things based on this bpf timestamping.
> 2) sk_bpf_cb_flags provides a way to let the socket-level use new
> features for bpf while now we only have a tcp_sock-level, namely,
> bpf_sock_ops_cb_flags. It's obviously good for others.
> 
> It's the first move to open the gate for socket-level usage for BPF,

Can you give a short list of bits that you could see being used, to
get an idea of the count. In my mind this is a very short list, not
worth reserving 32 bits for. But you might have more developed plans.

> just like how TCP_BPF_SOCK_OPS_CB_FLAGS works in sol_tcp_sockopt(). So
> I hope we will not abandon this good approach :(
> 
> Now I wonder if I should use the u8 sk_bpf_cb_flags in V13 or just
> keep it as-is? Either way is fine with me :) bpf_sock_ops_cb_flags
> uses u8 as an example, thus I think we prefer the former?

If it fits in a u8 and that in practice also results in less memory
and cache pressure (i.e., does not just add a 24b hole), then it is a
net improvement.

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Jason Xing]

On Wed, Feb 19, 2025 at 10:32 AM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> Jason Xing wrote:
> > On Wed, Feb 19, 2025 at 5:55 AM Martin KaFai Lau <martin.lau@linux.dev> wrote:
> > >
> > > On 2/18/25 6:22 AM, Willem de Bruijn wrote:
> > > > Jason Xing wrote:
> > > >> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
> > > >> added to bpf_get/setsockopt. The later patches will implement the
> > > >> BPF networking timestamping. The BPF program will use
> > > >> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
> > > >> enable the BPF networking timestamping on a socket.
> > > >>
> > > >> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> > > >> ---
> > > >>   include/net/sock.h             |  3 +++
> > > >>   include/uapi/linux/bpf.h       |  8 ++++++++
> > > >>   net/core/filter.c              | 23 +++++++++++++++++++++++
> > > >>   tools/include/uapi/linux/bpf.h |  1 +
> > > >>   4 files changed, 35 insertions(+)
> > > >>
> > > >> diff --git a/include/net/sock.h b/include/net/sock.h
> > > >> index 8036b3b79cd8..7916982343c6 100644
> > > >> --- a/include/net/sock.h
> > > >> +++ b/include/net/sock.h
> > > >> @@ -303,6 +303,7 @@ struct sk_filter;
> > > >>     *        @sk_stamp: time stamp of last packet received
> > > >>     *        @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
> > > >>     *        @sk_tsflags: SO_TIMESTAMPING flags
> > > >> +  * @sk_bpf_cb_flags: used in bpf_setsockopt()
> > > >>     *        @sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
> > > >>     *                           Sockets that can be used under memory reclaim should
> > > >>     *                           set this to false.
> > > >> @@ -445,6 +446,8 @@ struct sock {
> > > >>      u32                     sk_reserved_mem;
> > > >>      int                     sk_forward_alloc;
> > > >>      u32                     sk_tsflags;
> > > >> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> > > >> +    u32                     sk_bpf_cb_flags;
> > > >>      __cacheline_group_end(sock_write_rxtx);
> > > >
> > > > So far only one bit is defined. Does this have to be a 32-bit field in
> > > > every socket?
> > >
> > > iirc, I think there were multiple callback (cb) flags/bits in the earlier
> > > revisions, but it had been simplified to one bit in the later revisions.
> > >
> > > It's an internal implementation detail. We can reuse some free bits from another
> > > variable for now. Probably get a bit from sk_tsflags? SOCKCM_FLAG_TS_OPT_ID uses
> > > BIT(31). Maybe a new SK_TS_FLAG_BPF_TX that uses BIT(30)? I don't have a strong
> > > preference on the name.
> > >
> > > When the BPF program calls bpf_setsockopt(SK_BPF_CB_FLAGS,
> > > SK_BPF_CB_TX_TIMESTAMPING), the kernel will set/test the BIT(30) of sk_tsflags.
> > >
> > > We can wait until there are more socket-level cb flags in the future (e.g., more
> > > SK_BPF_CB_XXX will be needed) before adding a dedicated int field in the sock.
> >
> > Sorry, I still preferred the way we've discussed already:
>
> Adding fields to structs in the hot path is a tragedy of the commons.
>
> Every developer focuses on their specific workload and pet feature,
> while imposing a cost on everyone else.
>
> We have a duty to be frugal and mitigate this cost where possible.
> Especially for a feature that is likely to be used sparingly.

Point taken and I agree on this point all along. Don't get me wrong. I
insisted on using sk_bpf_cb_flags only, not the size of this field.
Please see details below.

>
> > 1) Introducing a new field sk_bpf_cb_flags extends the use for bpf
> > timestamping, more than SK_BPF_CB_TX_TIMESTAMPING one flag. I think
> > SK_BPF_CB_RX_TIMESTAMPING is also needed in the next move. And more
> > subfeatures (like bpf extension for OPT_ID) will use it. It gives us a
> > separate way to do more things based on this bpf timestamping.
> > 2) sk_bpf_cb_flags provides a way to let the socket-level use new
> > features for bpf while now we only have a tcp_sock-level, namely,
> > bpf_sock_ops_cb_flags. It's obviously good for others.
> >
> > It's the first move to open the gate for socket-level usage for BPF,
>
> Can you give a short list of bits that you could see being used, to
> get an idea of the count. In my mind this is a very short list, not
> worth reserving 32 bits for. But you might have more developed plans.

RX is one flag that we're going to add. Another new bit might be used
for bpf extension of OPT_ID to handle the udp lockless case, or
tracing every skb (which needs more official discussion) 'm not that
sure.

>
> > just like how TCP_BPF_SOCK_OPS_CB_FLAGS works in sol_tcp_sockopt(). So
> > I hope we will not abandon this good approach :(
> >
> > Now I wonder if I should use the u8 sk_bpf_cb_flags in V13 or just
> > keep it as-is? Either way is fine with me :) bpf_sock_ops_cb_flags
> > uses u8 as an example, thus I think we prefer the former?
>
> If it fits in a u8 and that in practice also results in less memory
> and cache pressure (i.e., does not just add a 24b hole), then it is a
> net improvement.

Probably I didn't state it clearly. I agree with you on saving memory:)

In the previous response, I was trying to keep the sk_bpf_cb_flags
flag and use a u8 instead. I admit u32 is too large after you noticed
this.

Would the following diff on top of this series be acceptable for you?
And would it be a proper place to put the u8 sk_bpf_cb_flags in struct
sock?
diff --git a/include/net/sock.h b/include/net/sock.h
index 6f4d54faba92..e85d6fb3a2ba 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -447,7 +447,7 @@ struct sock {
        int                     sk_forward_alloc;
        u32                     sk_tsflags;
 #define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
-       u32                     sk_bpf_cb_flags;
+       u8                      sk_bpf_cb_flags;
        __cacheline_group_end(sock_write_rxtx);

        __cacheline_group_begin(sock_write_tx);

The following output is the result of running 'pahole --hex -C sock vmlinux'.
Before this series:
        u32                        sk_tsflags;           /* 0x168   0x4 */
        __u8
__cacheline_group_end__sock_write_rxtx[0]; /* 0x16c     0 */
        __u8
__cacheline_group_begin__sock_write_tx[0]; /* 0x16c     0 */
        int                        sk_write_pending;     /* 0x16c   0x4 */
        atomic_t                   sk_omem_alloc;        /* 0x170   0x4 */
        int                        sk_sndbuf;            /* 0x174   0x4 */
        int                        sk_wmem_queued;       /* 0x178   0x4 */
        refcount_t                 sk_wmem_alloc;        /* 0x17c   0x4 */
        /* --- cacheline 6 boundary (384 bytes) --- */
        long unsigned int          sk_tsq_flags;         /* 0x180   0x8 */
...
/* sum members: 773, holes: 1, sum holes: 1 */

After this diff patch:
        u32                        sk_tsflags;           /* 0x168   0x4 */
        u8                         sk_bpf_cb_flags;      /* 0x16c   0x1 */
        __u8
__cacheline_group_end__sock_write_rxtx[0]; /* 0x16d     0 */
        __u8
__cacheline_group_begin__sock_write_tx[0]; /* 0x16d     0 */

        /* XXX 3 bytes hole, try to pack */

        int                        sk_write_pending;     /* 0x170   0x4 */
        atomic_t                   sk_omem_alloc;        /* 0x174   0x4 */
        int                        sk_sndbuf;            /* 0x178   0x4 */
        int                        sk_wmem_queued;       /* 0x17c   0x4 */
        /* --- cacheline 6 boundary (384 bytes) --- */
        refcount_t                 sk_wmem_alloc;        /* 0x180   0x4 */

        /* XXX 4 bytes hole, try to pack */

        long unsigned int          sk_tsq_flags;         /* 0x188   0x8 */
...
/* sum members: 774, holes: 3, sum holes: 8 */

It will introduce 7 extra sum holes if this series with this u8 change
gets applied. I think it's a proper position because this new
sk_bpf_cb_flags will be used in the tx and rx path just like
sk_tsflags, aligned with rules introduced by the commit[1].

I'd like to know your opinion here. Hopefully we can let this series
pass in the next re-spin:)

Thanks in advance.

[1]
commit 5d4cc87414c5d11345c4b11d61377d351b5c28a2
Author: Eric Dumazet <edumazet@google.com>
Date:   Fri Feb 16 16:20:06 2024 +0000

    net: reorganize "struct sock" fields

    Last major reorg happened in commit 9115e8cd2a0c ("net: reorganize
    struct sock for better data locality")

    Since then, many changes have been done.

    Before SO_PEEK_OFF support is added to TCP, we need
    to move sk_peek_off to a better location.

    It is time to make another pass, and add six groups,
    without explicit alignment.

    - sock_write_rx (following sk_refcnt) read-write fields in rx path.
    - sock_read_rx read-mostly fields in rx path.
    - sock_read_rxtx read-mostly fields in both rx and tx paths.
    - sock_write_rxtx read-write fields in both rx and tx paths.
    - sock_write_tx read-write fields in tx paths.
    - sock_read_tx read-mostly fields in tx paths.

Thanks,
Jason

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Jason Xing]

On Tue, Feb 18, 2025 at 1:02 PM Jason Xing <kerneljasonxing@gmail.com> wrote:
>
> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
> added to bpf_get/setsockopt. The later patches will implement the
> BPF networking timestamping. The BPF program will use
> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
> enable the BPF networking timestamping on a socket.
>
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> ---
>  include/net/sock.h             |  3 +++
>  include/uapi/linux/bpf.h       |  8 ++++++++
>  net/core/filter.c              | 23 +++++++++++++++++++++++
>  tools/include/uapi/linux/bpf.h |  1 +
>  4 files changed, 35 insertions(+)
>
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 8036b3b79cd8..7916982343c6 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -303,6 +303,7 @@ struct sk_filter;
>    *    @sk_stamp: time stamp of last packet received
>    *    @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
>    *    @sk_tsflags: SO_TIMESTAMPING flags
> +  *    @sk_bpf_cb_flags: used in bpf_setsockopt()
>    *    @sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
>    *                       Sockets that can be used under memory reclaim should
>    *                       set this to false.
> @@ -445,6 +446,8 @@ struct sock {
>         u32                     sk_reserved_mem;
>         int                     sk_forward_alloc;
>         u32                     sk_tsflags;
> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> +       u32                     sk_bpf_cb_flags;
>         __cacheline_group_end(sock_write_rxtx);
>
>         __cacheline_group_begin(sock_write_tx);
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index fff6cdb8d11a..fa666d51dffe 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -6916,6 +6916,13 @@ enum {
>         BPF_SOCK_OPS_ALL_CB_FLAGS       = 0x7F,
>  };
>
> +/* Definitions for bpf_sk_cb_flags */

nit: s/bpf_sk_cb_flags/sk_bpf_cb_flags

I will correct it.

> +enum {
> +       SK_BPF_CB_TX_TIMESTAMPING       = 1<<0,
> +       SK_BPF_CB_MASK                  = (SK_BPF_CB_TX_TIMESTAMPING - 1) |
> +                                          SK_BPF_CB_TX_TIMESTAMPING
> +};

Martin, I would like to know if it's necessary to update the above new
enum in tools/include/uapi/linux/bpf.h as well?

Thanks,
Jason

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Willem de Bruijn]

> > > Now I wonder if I should use the u8 sk_bpf_cb_flags in V13 or just
> > > keep it as-is? Either way is fine with me :) bpf_sock_ops_cb_flags
> > > uses u8 as an example, thus I think we prefer the former?
> >
> > If it fits in a u8 and that in practice also results in less memory
> > and cache pressure (i.e., does not just add a 24b hole), then it is a
> > net improvement.
> 
> Probably I didn't state it clearly. I agree with you on saving memory:)
> 
> In the previous response, I was trying to keep the sk_bpf_cb_flags
> flag and use a u8 instead. I admit u32 is too large after you noticed
> this.
> 
> Would the following diff on top of this series be acceptable for you?
> And would it be a proper place to put the u8 sk_bpf_cb_flags in struct
> sock?
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 6f4d54faba92..e85d6fb3a2ba 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -447,7 +447,7 @@ struct sock {
>         int                     sk_forward_alloc;
>         u32                     sk_tsflags;
>  #define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> -       u32                     sk_bpf_cb_flags;
> +       u8                      sk_bpf_cb_flags;
>         __cacheline_group_end(sock_write_rxtx);
> 
>         __cacheline_group_begin(sock_write_tx);
> 
> The following output is the result of running 'pahole --hex -C sock vmlinux'.
> Before this series:
>         u32                        sk_tsflags;           /* 0x168   0x4 */
>         __u8
> __cacheline_group_end__sock_write_rxtx[0]; /* 0x16c     0 */
>         __u8
> __cacheline_group_begin__sock_write_tx[0]; /* 0x16c     0 */
>         int                        sk_write_pending;     /* 0x16c   0x4 */
>         atomic_t                   sk_omem_alloc;        /* 0x170   0x4 */
>         int                        sk_sndbuf;            /* 0x174   0x4 */
>         int                        sk_wmem_queued;       /* 0x178   0x4 */
>         refcount_t                 sk_wmem_alloc;        /* 0x17c   0x4 */
>         /* --- cacheline 6 boundary (384 bytes) --- */
>         long unsigned int          sk_tsq_flags;         /* 0x180   0x8 */
> ...
> /* sum members: 773, holes: 1, sum holes: 1 */
> 
> After this diff patch:
>         u32                        sk_tsflags;           /* 0x168   0x4 */
>         u8                         sk_bpf_cb_flags;      /* 0x16c   0x1 */
>         __u8
> __cacheline_group_end__sock_write_rxtx[0]; /* 0x16d     0 */
>         __u8
> __cacheline_group_begin__sock_write_tx[0]; /* 0x16d     0 */
> 
>         /* XXX 3 bytes hole, try to pack */
> 
>         int                        sk_write_pending;     /* 0x170   0x4 */
>         atomic_t                   sk_omem_alloc;        /* 0x174   0x4 */
>         int                        sk_sndbuf;            /* 0x178   0x4 */
>         int                        sk_wmem_queued;       /* 0x17c   0x4 */
>         /* --- cacheline 6 boundary (384 bytes) --- */
>         refcount_t                 sk_wmem_alloc;        /* 0x180   0x4 */
> 
>         /* XXX 4 bytes hole, try to pack */
> 
>         long unsigned int          sk_tsq_flags;         /* 0x188   0x8 */
> ...
> /* sum members: 774, holes: 3, sum holes: 8 */
> 
> It will introduce 7 extra sum holes if this series with this u8 change
> gets applied. I think it's a proper position because this new
> sk_bpf_cb_flags will be used in the tx and rx path just like
> sk_tsflags, aligned with rules introduced by the commit[1].

Reducing a u64 to u8 can leave 7b of holes, but that is not great,
of course.

Since this bitmap is only touched if a BPF program is loaded, arguably
it need not be in the hot path cacheline groups.

Can you find a hole further down to place this in, or at least a spot
that does not result in 7b of wasted space (in the hotpath cacheline
groups of all places).

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Martin KaFai Lau]

On 2/18/25 11:03 PM, Jason Xing wrote:
> On Tue, Feb 18, 2025 at 1:02 PM Jason Xing <kerneljasonxing@gmail.com> wrote:
>>
>> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
>> added to bpf_get/setsockopt. The later patches will implement the
>> BPF networking timestamping. The BPF program will use
>> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
>> enable the BPF networking timestamping on a socket.
>>
>> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
>> ---
>>   include/net/sock.h             |  3 +++
>>   include/uapi/linux/bpf.h       |  8 ++++++++
>>   net/core/filter.c              | 23 +++++++++++++++++++++++
>>   tools/include/uapi/linux/bpf.h |  1 +
>>   4 files changed, 35 insertions(+)
>>
>> diff --git a/include/net/sock.h b/include/net/sock.h
>> index 8036b3b79cd8..7916982343c6 100644
>> --- a/include/net/sock.h
>> +++ b/include/net/sock.h
>> @@ -303,6 +303,7 @@ struct sk_filter;
>>     *    @sk_stamp: time stamp of last packet received
>>     *    @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
>>     *    @sk_tsflags: SO_TIMESTAMPING flags
>> +  *    @sk_bpf_cb_flags: used in bpf_setsockopt()
>>     *    @sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
>>     *                       Sockets that can be used under memory reclaim should
>>     *                       set this to false.
>> @@ -445,6 +446,8 @@ struct sock {
>>          u32                     sk_reserved_mem;
>>          int                     sk_forward_alloc;
>>          u32                     sk_tsflags;
>> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
>> +       u32                     sk_bpf_cb_flags;
>>          __cacheline_group_end(sock_write_rxtx);
>>
>>          __cacheline_group_begin(sock_write_tx);
>> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
>> index fff6cdb8d11a..fa666d51dffe 100644
>> --- a/include/uapi/linux/bpf.h
>> +++ b/include/uapi/linux/bpf.h
>> @@ -6916,6 +6916,13 @@ enum {
>>          BPF_SOCK_OPS_ALL_CB_FLAGS       = 0x7F,
>>   };
>>
>> +/* Definitions for bpf_sk_cb_flags */
> 
> nit: s/bpf_sk_cb_flags/sk_bpf_cb_flags
> 
> I will correct it.
> 
>> +enum {
>> +       SK_BPF_CB_TX_TIMESTAMPING       = 1<<0,
>> +       SK_BPF_CB_MASK                  = (SK_BPF_CB_TX_TIMESTAMPING - 1) |
>> +                                          SK_BPF_CB_TX_TIMESTAMPING
>> +};
> 
> Martin, I would like to know if it's necessary to update the above new
> enum in tools/include/uapi/linux/bpf.h as well?

Yes, the tools/include/uapi/linux/bpf.h should be updated. If you diff them, two 
of them should be exactly the same. This patch should do the same to keep the 
tools bpf.h up-to-date.

For other headers in tools/include/uapi, I guess it depends. e.g. the tcp.h in 
your another RTO patch, the two tcp.h files are very different already and the 
selftest does not need the new macro either.

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Jason Xing]

On Wed, Feb 19, 2025 at 11:12 PM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> > > > Now I wonder if I should use the u8 sk_bpf_cb_flags in V13 or just
> > > > keep it as-is? Either way is fine with me :) bpf_sock_ops_cb_flags
> > > > uses u8 as an example, thus I think we prefer the former?
> > >
> > > If it fits in a u8 and that in practice also results in less memory
> > > and cache pressure (i.e., does not just add a 24b hole), then it is a
> > > net improvement.
> >
> > Probably I didn't state it clearly. I agree with you on saving memory:)
> >
> > In the previous response, I was trying to keep the sk_bpf_cb_flags
> > flag and use a u8 instead. I admit u32 is too large after you noticed
> > this.
> >
> > Would the following diff on top of this series be acceptable for you?
> > And would it be a proper place to put the u8 sk_bpf_cb_flags in struct
> > sock?
> > diff --git a/include/net/sock.h b/include/net/sock.h
> > index 6f4d54faba92..e85d6fb3a2ba 100644
> > --- a/include/net/sock.h
> > +++ b/include/net/sock.h
> > @@ -447,7 +447,7 @@ struct sock {
> >         int                     sk_forward_alloc;
> >         u32                     sk_tsflags;
> >  #define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> > -       u32                     sk_bpf_cb_flags;
> > +       u8                      sk_bpf_cb_flags;
> >         __cacheline_group_end(sock_write_rxtx);
> >
> >         __cacheline_group_begin(sock_write_tx);
> >
> > The following output is the result of running 'pahole --hex -C sock vmlinux'.
> > Before this series:
> >         u32                        sk_tsflags;           /* 0x168   0x4 */
> >         __u8
> > __cacheline_group_end__sock_write_rxtx[0]; /* 0x16c     0 */
> >         __u8
> > __cacheline_group_begin__sock_write_tx[0]; /* 0x16c     0 */
> >         int                        sk_write_pending;     /* 0x16c   0x4 */
> >         atomic_t                   sk_omem_alloc;        /* 0x170   0x4 */
> >         int                        sk_sndbuf;            /* 0x174   0x4 */
> >         int                        sk_wmem_queued;       /* 0x178   0x4 */
> >         refcount_t                 sk_wmem_alloc;        /* 0x17c   0x4 */
> >         /* --- cacheline 6 boundary (384 bytes) --- */
> >         long unsigned int          sk_tsq_flags;         /* 0x180   0x8 */
> > ...
> > /* sum members: 773, holes: 1, sum holes: 1 */
> >
> > After this diff patch:
> >         u32                        sk_tsflags;           /* 0x168   0x4 */
> >         u8                         sk_bpf_cb_flags;      /* 0x16c   0x1 */
> >         __u8
> > __cacheline_group_end__sock_write_rxtx[0]; /* 0x16d     0 */
> >         __u8
> > __cacheline_group_begin__sock_write_tx[0]; /* 0x16d     0 */
> >
> >         /* XXX 3 bytes hole, try to pack */
> >
> >         int                        sk_write_pending;     /* 0x170   0x4 */
> >         atomic_t                   sk_omem_alloc;        /* 0x174   0x4 */
> >         int                        sk_sndbuf;            /* 0x178   0x4 */
> >         int                        sk_wmem_queued;       /* 0x17c   0x4 */
> >         /* --- cacheline 6 boundary (384 bytes) --- */
> >         refcount_t                 sk_wmem_alloc;        /* 0x180   0x4 */
> >
> >         /* XXX 4 bytes hole, try to pack */
> >
> >         long unsigned int          sk_tsq_flags;         /* 0x188   0x8 */
> > ...
> > /* sum members: 774, holes: 3, sum holes: 8 */
> >
> > It will introduce 7 extra sum holes if this series with this u8 change
> > gets applied. I think it's a proper position because this new
> > sk_bpf_cb_flags will be used in the tx and rx path just like
> > sk_tsflags, aligned with rules introduced by the commit[1].
>
> Reducing a u64 to u8 can leave 7b of holes, but that is not great,
> of course.
>
> Since this bitmap is only touched if a BPF program is loaded, arguably
> it need not be in the hot path cacheline groups.

Point taken.

>
> Can you find a hole further down to place this in, or at least a spot
> that does not result in 7b of wasted space (in the hotpath cacheline
> groups of all places).

There is one place where I can simply insert the flag.

The diff patch on top of this series is:
diff --git a/include/net/sock.h b/include/net/sock.h
index e85d6fb3a2ba..9fa27693fb02 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -446,8 +446,6 @@ struct sock {
        u32                     sk_reserved_mem;
        int                     sk_forward_alloc;
        u32                     sk_tsflags;
-#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
-       u8                      sk_bpf_cb_flags;
        __cacheline_group_end(sock_write_rxtx);

        __cacheline_group_begin(sock_write_tx);
@@ -528,6 +526,8 @@ struct sock {
        u8                      sk_txtime_deadline_mode : 1,
                                sk_txtime_report_errors : 1,
                                sk_txtime_unused : 6;
+#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
+       u8                      sk_bpf_cb_flags;

        void                    *sk_user_data;
 #ifdef CONFIG_SECURITY


1) before applying the whole series:
...
        /* --- cacheline 10 boundary (640 bytes) --- */
        ktime_t                    sk_stamp;             /* 0x280   0x8 */
        int                        sk_disconnects;       /* 0x288   0x4 */
        u8                         sk_txrehash;          /* 0x28c   0x1 */
        u8                         sk_clockid;           /* 0x28d   0x1 */
        u8                         sk_txtime_deadline_mode:1; /* 0x28e: 0 0x1 */
        u8                         sk_txtime_report_errors:1; /*
0x28e:0x1 0x1 */
        u8                         sk_txtime_unused:6;   /* 0x28e:0x2 0x1 */

        /* XXX 1 byte hole, try to pack */

        void *                     sk_user_data;         /* 0x290   0x8 */
        void *                     sk_security;          /* 0x298   0x8 */
        struct sock_cgroup_data    sk_cgrp_data;         /* 0x2a0  0x10 */
...
/* sum members: 773, holes: 1, sum holes: 1 */


2) after applying the series with the above diff patch:
...
        /* --- cacheline 10 boundary (640 bytes) --- */
        ktime_t                    sk_stamp;             /* 0x280   0x8 */
        int                        sk_disconnects;       /* 0x288   0x4 */
        u8                         sk_txrehash;          /* 0x28c   0x1 */
        u8                         sk_clockid;           /* 0x28d   0x1 */
        u8                         sk_txtime_deadline_mode:1; /* 0x28e: 0 0x1 */
        u8                         sk_txtime_report_errors:1; /*
0x28e:0x1 0x1 */
        u8                         sk_txtime_unused:6;   /* 0x28e:0x2 0x1 */
        u8                         sk_bpf_cb_flags;      /* 0x28f   0x1 */
        void *                     sk_user_data;         /* 0x290
0x8 */
        void *                     sk_security;          /* 0x298   0x8 */
        struct sock_cgroup_data    sk_cgrp_data;         /* 0x2a0  0x10 */
...
/* sum members: 774 */

It turns out that the new sk_bpf_cb_flags fills the hole exactly. The
new field and some of its nearby fields are quite similar because they
are only/nearly written during the creation or setsockopt phase.

I think now it's a good place to insert the new flag?

Thanks,
Jason

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Jason Xing]

On Thu, Feb 20, 2025 at 3:48 AM Martin KaFai Lau <martin.lau@linux.dev> wrote:
>
> On 2/18/25 11:03 PM, Jason Xing wrote:
> > On Tue, Feb 18, 2025 at 1:02 PM Jason Xing <kerneljasonxing@gmail.com> wrote:
> >>
> >> The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are
> >> added to bpf_get/setsockopt. The later patches will implement the
> >> BPF networking timestamping. The BPF program will use
> >> bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to
> >> enable the BPF networking timestamping on a socket.
> >>
> >> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> >> ---
> >>   include/net/sock.h             |  3 +++
> >>   include/uapi/linux/bpf.h       |  8 ++++++++
> >>   net/core/filter.c              | 23 +++++++++++++++++++++++
> >>   tools/include/uapi/linux/bpf.h |  1 +
> >>   4 files changed, 35 insertions(+)
> >>
> >> diff --git a/include/net/sock.h b/include/net/sock.h
> >> index 8036b3b79cd8..7916982343c6 100644
> >> --- a/include/net/sock.h
> >> +++ b/include/net/sock.h
> >> @@ -303,6 +303,7 @@ struct sk_filter;
> >>     *    @sk_stamp: time stamp of last packet received
> >>     *    @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
> >>     *    @sk_tsflags: SO_TIMESTAMPING flags
> >> +  *    @sk_bpf_cb_flags: used in bpf_setsockopt()
> >>     *    @sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
> >>     *                       Sockets that can be used under memory reclaim should
> >>     *                       set this to false.
> >> @@ -445,6 +446,8 @@ struct sock {
> >>          u32                     sk_reserved_mem;
> >>          int                     sk_forward_alloc;
> >>          u32                     sk_tsflags;
> >> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> >> +       u32                     sk_bpf_cb_flags;
> >>          __cacheline_group_end(sock_write_rxtx);
> >>
> >>          __cacheline_group_begin(sock_write_tx);
> >> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> >> index fff6cdb8d11a..fa666d51dffe 100644
> >> --- a/include/uapi/linux/bpf.h
> >> +++ b/include/uapi/linux/bpf.h
> >> @@ -6916,6 +6916,13 @@ enum {
> >>          BPF_SOCK_OPS_ALL_CB_FLAGS       = 0x7F,
> >>   };
> >>
> >> +/* Definitions for bpf_sk_cb_flags */
> >
> > nit: s/bpf_sk_cb_flags/sk_bpf_cb_flags
> >
> > I will correct it.
> >
> >> +enum {
> >> +       SK_BPF_CB_TX_TIMESTAMPING       = 1<<0,
> >> +       SK_BPF_CB_MASK                  = (SK_BPF_CB_TX_TIMESTAMPING - 1) |
> >> +                                          SK_BPF_CB_TX_TIMESTAMPING
> >> +};
> >
> > Martin, I would like to know if it's necessary to update the above new
> > enum in tools/include/uapi/linux/bpf.h as well?
>
> Yes, the tools/include/uapi/linux/bpf.h should be updated. If you diff them, two
> of them should be exactly the same. This patch should do the same to keep the
> tools bpf.h up-to-date.
>
> For other headers in tools/include/uapi, I guess it depends. e.g. the tcp.h in
> your another RTO patch, the two tcp.h files are very different already and the
> selftest does not need the new macro either.

I learned. Thanks.

Re: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt()

[Willem de Bruijn]

> > Can you find a hole further down to place this in, or at least a spot
> > that does not result in 7b of wasted space (in the hotpath cacheline
> > groups of all places).
> 
> There is one place where I can simply insert the flag.
> 
> The diff patch on top of this series is:
> diff --git a/include/net/sock.h b/include/net/sock.h
> index e85d6fb3a2ba..9fa27693fb02 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -446,8 +446,6 @@ struct sock {
>         u32                     sk_reserved_mem;
>         int                     sk_forward_alloc;
>         u32                     sk_tsflags;
> -#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> -       u8                      sk_bpf_cb_flags;
>         __cacheline_group_end(sock_write_rxtx);
> 
>         __cacheline_group_begin(sock_write_tx);
> @@ -528,6 +526,8 @@ struct sock {
>         u8                      sk_txtime_deadline_mode : 1,
>                                 sk_txtime_report_errors : 1,
>                                 sk_txtime_unused : 6;
> +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
> +       u8                      sk_bpf_cb_flags;
> 
>         void                    *sk_user_data;
>  #ifdef CONFIG_SECURITY
> 
> 
> 1) before applying the whole series:
> ...
>         /* --- cacheline 10 boundary (640 bytes) --- */
>         ktime_t                    sk_stamp;             /* 0x280   0x8 */
>         int                        sk_disconnects;       /* 0x288   0x4 */
>         u8                         sk_txrehash;          /* 0x28c   0x1 */
>         u8                         sk_clockid;           /* 0x28d   0x1 */
>         u8                         sk_txtime_deadline_mode:1; /* 0x28e: 0 0x1 */
>         u8                         sk_txtime_report_errors:1; /*
> 0x28e:0x1 0x1 */
>         u8                         sk_txtime_unused:6;   /* 0x28e:0x2 0x1 */
> 
>         /* XXX 1 byte hole, try to pack */
> 
>         void *                     sk_user_data;         /* 0x290   0x8 */
>         void *                     sk_security;          /* 0x298   0x8 */
>         struct sock_cgroup_data    sk_cgrp_data;         /* 0x2a0  0x10 */
> ...
> /* sum members: 773, holes: 1, sum holes: 1 */
> 
> 
> 2) after applying the series with the above diff patch:
> ...
>         /* --- cacheline 10 boundary (640 bytes) --- */
>         ktime_t                    sk_stamp;             /* 0x280   0x8 */
>         int                        sk_disconnects;       /* 0x288   0x4 */
>         u8                         sk_txrehash;          /* 0x28c   0x1 */
>         u8                         sk_clockid;           /* 0x28d   0x1 */
>         u8                         sk_txtime_deadline_mode:1; /* 0x28e: 0 0x1 */
>         u8                         sk_txtime_report_errors:1; /*
> 0x28e:0x1 0x1 */
>         u8                         sk_txtime_unused:6;   /* 0x28e:0x2 0x1 */
>         u8                         sk_bpf_cb_flags;      /* 0x28f   0x1 */
>         void *                     sk_user_data;         /* 0x290
> 0x8 */
>         void *                     sk_security;          /* 0x298   0x8 */
>         struct sock_cgroup_data    sk_cgrp_data;         /* 0x2a0  0x10 */
> ...
> /* sum members: 774 */
> 
> It turns out that the new sk_bpf_cb_flags fills the hole exactly. The
> new field and some of its nearby fields are quite similar because they
> are only/nearly written during the creation or setsockopt phase.
> 
> I think now it's a good place to insert the new flag?

Thanks. This seems fine to me.
 

Re: [PATCH bpf-next v12 10/12] bpf: add BPF_SOCK_OPS_TS_SND_CB callback

[Willem de Bruijn]

Jason Xing wrote:
> This patch introduces a new callback in tcp_tx_timestamp() to correlate
> tcp_sendmsg timestamp with timestamps from other tx timestamping
> callbacks (e.g., SND/SW/ACK).
> 
> Without this patch, BPF program wouldn't know which timestamps belong
> to which flow because of no socket lock protection. This new callback
> is inserted in tcp_tx_timestamp() to address this issue because
> tcp_tx_timestamp() still owns the same socket lock with
> tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
> the timestamping related fields for the skb, especially tskey. The
> tskey is the bridge to do the correlation.
> 
> For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
> then stores the sendmsg timestamp at the bpf_sk_storage, correlating
> this timestamp with its tskey that are later used in other sending
> timestamping callbacks.
> 
> Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> ---
>  include/uapi/linux/bpf.h       | 5 +++++
>  net/ipv4/tcp.c                 | 4 ++++
>  tools/include/uapi/linux/bpf.h | 5 +++++
>  3 files changed, 14 insertions(+)
> 
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 9355d617767f..86fca729fbd8 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -7052,6 +7052,11 @@ enum {
>  					 * when SK_BPF_CB_TX_TIMESTAMPING
>  					 * feature is on.
>  					 */
> +	BPF_SOCK_OPS_TS_SND_CB,		/* Called when every sendmsg syscall
> +					 * is triggered. It's used to correlate
> +					 * sendmsg timestamp with corresponding
> +					 * tskey.
> +					 */
>  };
>  
>  /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
> diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> index 12b9c4f9c151..4b9739cd3bc5 100644
> --- a/net/ipv4/tcp.c
> +++ b/net/ipv4/tcp.c
> @@ -492,6 +492,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
>  		if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
>  			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
>  	}
> +
> +	if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
> +	    SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb)
> +		bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB);
>  }
>  
>  static bool tcp_stream_is_readable(struct sock *sk, int target)
> diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
> index d3e2988b3b4c..2739ee0154a0 100644
> --- a/tools/include/uapi/linux/bpf.h
> +++ b/tools/include/uapi/linux/bpf.h
> @@ -7042,6 +7042,11 @@ enum {
>  					 * when SK_BPF_CB_TX_TIMESTAMPING
>  					 * feature is on.
>  					 */
> +	BPF_SOCK_OPS_TS_SND_CB,		/* Called when every sendmsg syscall
> +					 * is triggered. It's used to correlate
> +					 * sendmsg timestamp with corresponding
> +					 * tskey.
> +					 */

Feel free to decline this late in the review process, but a bit more
bikeshedding..

Can we spell out TSTAMP instead of TS in these definitions? Within
the context of this series it is self-explanatory, but when reading
kernel code the meaning of a two letter acronym is not that clear.

And instead of SND can we use SENDMSG or something like that?
SND here confused me as the software timestamp is SCM_TSTAMP_SND.

For instance:

    BPF_SOCK_OPS_TSTAMP_SENDMSG_CB,
    BPF_SOCK_OPS_TSTAMP_SCHED_CB,
    BPF_SOCK_OPS_TSTAMP_SND_SW_CB,
    BPF_SOCK_OPS_TSTAMP_SND_HW_CB,
   (BPF_SOCK_OPS_TSTAMP_TX_COMPLETION_CB,)
    BPF_SOCK_OPS_TSTAMP_ACK_CB,

(not sure what the OPT in OPT_CB added).

Re: [PATCH bpf-next v12 10/12] bpf: add BPF_SOCK_OPS_TS_SND_CB callback

[Jason Xing]

On Thu, Feb 20, 2025 at 10:55 AM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> Jason Xing wrote:
> > This patch introduces a new callback in tcp_tx_timestamp() to correlate
> > tcp_sendmsg timestamp with timestamps from other tx timestamping
> > callbacks (e.g., SND/SW/ACK).
> >
> > Without this patch, BPF program wouldn't know which timestamps belong
> > to which flow because of no socket lock protection. This new callback
> > is inserted in tcp_tx_timestamp() to address this issue because
> > tcp_tx_timestamp() still owns the same socket lock with
> > tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
> > the timestamping related fields for the skb, especially tskey. The
> > tskey is the bridge to do the correlation.
> >
> > For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
> > then stores the sendmsg timestamp at the bpf_sk_storage, correlating
> > this timestamp with its tskey that are later used in other sending
> > timestamping callbacks.
> >
> > Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> > ---
> >  include/uapi/linux/bpf.h       | 5 +++++
> >  net/ipv4/tcp.c                 | 4 ++++
> >  tools/include/uapi/linux/bpf.h | 5 +++++
> >  3 files changed, 14 insertions(+)
> >
> > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > index 9355d617767f..86fca729fbd8 100644
> > --- a/include/uapi/linux/bpf.h
> > +++ b/include/uapi/linux/bpf.h
> > @@ -7052,6 +7052,11 @@ enum {
> >                                        * when SK_BPF_CB_TX_TIMESTAMPING
> >                                        * feature is on.
> >                                        */
> > +     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
> > +                                      * is triggered. It's used to correlate
> > +                                      * sendmsg timestamp with corresponding
> > +                                      * tskey.
> > +                                      */
> >  };
> >
> >  /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
> > diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> > index 12b9c4f9c151..4b9739cd3bc5 100644
> > --- a/net/ipv4/tcp.c
> > +++ b/net/ipv4/tcp.c
> > @@ -492,6 +492,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
> >               if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
> >                       shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
> >       }
> > +
> > +     if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
> > +         SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb)
> > +             bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB);
> >  }
> >
> >  static bool tcp_stream_is_readable(struct sock *sk, int target)
> > diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
> > index d3e2988b3b4c..2739ee0154a0 100644
> > --- a/tools/include/uapi/linux/bpf.h
> > +++ b/tools/include/uapi/linux/bpf.h
> > @@ -7042,6 +7042,11 @@ enum {
> >                                        * when SK_BPF_CB_TX_TIMESTAMPING
> >                                        * feature is on.
> >                                        */
> > +     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
> > +                                      * is triggered. It's used to correlate
> > +                                      * sendmsg timestamp with corresponding
> > +                                      * tskey.
> > +                                      */
>
> Feel free to decline this late in the review process, but a bit more
> bikeshedding..
>
> Can we spell out TSTAMP instead of TS in these definitions? Within
> the context of this series it is self-explanatory, but when reading
> kernel code the meaning of a two letter acronym is not that clear.

Even though I feel reluctant to change across the whole series because
if so, I will adjust in many places. Of course, you're right about the
new name being clearer :)

>
> And instead of SND can we use SENDMSG or something like that?
> SND here confused me as the software timestamp is SCM_TSTAMP_SND.

I'm not sure about this. For TCP, it's not implemented in the
tcp_sendmsg_locked but tcp_tx_timestamp. Well, I have no strong
preference.

You can make the final call :)

Thanks,
Jason

>
> For instance:
>
>     BPF_SOCK_OPS_TSTAMP_SENDMSG_CB,
>     BPF_SOCK_OPS_TSTAMP_SCHED_CB,
>     BPF_SOCK_OPS_TSTAMP_SND_SW_CB,
>     BPF_SOCK_OPS_TSTAMP_SND_HW_CB,
>    (BPF_SOCK_OPS_TSTAMP_TX_COMPLETION_CB,)
>     BPF_SOCK_OPS_TSTAMP_ACK_CB,
>
> (not sure what the OPT in OPT_CB added).
>
>

Re: [PATCH bpf-next v12 10/12] bpf: add BPF_SOCK_OPS_TS_SND_CB callback

[Jason Xing]

On Thu, Feb 20, 2025 at 11:15 AM Jason Xing <kerneljasonxing@gmail.com> wrote:
>
> On Thu, Feb 20, 2025 at 10:55 AM Willem de Bruijn
> <willemdebruijn.kernel@gmail.com> wrote:
> >
> > Jason Xing wrote:
> > > This patch introduces a new callback in tcp_tx_timestamp() to correlate
> > > tcp_sendmsg timestamp with timestamps from other tx timestamping
> > > callbacks (e.g., SND/SW/ACK).
> > >
> > > Without this patch, BPF program wouldn't know which timestamps belong
> > > to which flow because of no socket lock protection. This new callback
> > > is inserted in tcp_tx_timestamp() to address this issue because
> > > tcp_tx_timestamp() still owns the same socket lock with
> > > tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
> > > the timestamping related fields for the skb, especially tskey. The
> > > tskey is the bridge to do the correlation.
> > >
> > > For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
> > > then stores the sendmsg timestamp at the bpf_sk_storage, correlating
> > > this timestamp with its tskey that are later used in other sending
> > > timestamping callbacks.
> > >
> > > Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> > > ---
> > >  include/uapi/linux/bpf.h       | 5 +++++
> > >  net/ipv4/tcp.c                 | 4 ++++
> > >  tools/include/uapi/linux/bpf.h | 5 +++++
> > >  3 files changed, 14 insertions(+)
> > >
> > > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > > index 9355d617767f..86fca729fbd8 100644
> > > --- a/include/uapi/linux/bpf.h
> > > +++ b/include/uapi/linux/bpf.h
> > > @@ -7052,6 +7052,11 @@ enum {
> > >                                        * when SK_BPF_CB_TX_TIMESTAMPING
> > >                                        * feature is on.
> > >                                        */
> > > +     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
> > > +                                      * is triggered. It's used to correlate
> > > +                                      * sendmsg timestamp with corresponding
> > > +                                      * tskey.
> > > +                                      */
> > >  };
> > >
> > >  /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
> > > diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> > > index 12b9c4f9c151..4b9739cd3bc5 100644
> > > --- a/net/ipv4/tcp.c
> > > +++ b/net/ipv4/tcp.c
> > > @@ -492,6 +492,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
> > >               if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
> > >                       shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
> > >       }
> > > +
> > > +     if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
> > > +         SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb)
> > > +             bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB);
> > >  }
> > >
> > >  static bool tcp_stream_is_readable(struct sock *sk, int target)
> > > diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
> > > index d3e2988b3b4c..2739ee0154a0 100644
> > > --- a/tools/include/uapi/linux/bpf.h
> > > +++ b/tools/include/uapi/linux/bpf.h
> > > @@ -7042,6 +7042,11 @@ enum {
> > >                                        * when SK_BPF_CB_TX_TIMESTAMPING
> > >                                        * feature is on.
> > >                                        */
> > > +     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
> > > +                                      * is triggered. It's used to correlate
> > > +                                      * sendmsg timestamp with corresponding
> > > +                                      * tskey.
> > > +                                      */
> >
> > Feel free to decline this late in the review process, but a bit more
> > bikeshedding..
> >
> > Can we spell out TSTAMP instead of TS in these definitions? Within
> > the context of this series it is self-explanatory, but when reading
> > kernel code the meaning of a two letter acronym is not that clear.
>
> Even though I feel reluctant to change across the whole series because
> if so, I will adjust in many places. Of course, you're right about the
> new name being clearer :)
>
> >
> > And instead of SND can we use SENDMSG or something like that?
> > SND here confused me as the software timestamp is SCM_TSTAMP_SND.
>
> I'm not sure about this. For TCP, it's not implemented in the
> tcp_sendmsg_locked but tcp_tx_timestamp. Well, I have no strong
> preference.
>
> You can make the final call :)

After taking a break, I feel full of energy and I will update them all
as you requested :)

Thanks,
Jason

Re: [PATCH bpf-next v12 10/12] bpf: add BPF_SOCK_OPS_TS_SND_CB callback

[Willem de Bruijn]

Jason Xing wrote:
> On Thu, Feb 20, 2025 at 11:15 AM Jason Xing <kerneljasonxing@gmail.com> wrote:
> >
> > On Thu, Feb 20, 2025 at 10:55 AM Willem de Bruijn
> > <willemdebruijn.kernel@gmail.com> wrote:
> > >
> > > Jason Xing wrote:
> > > > This patch introduces a new callback in tcp_tx_timestamp() to correlate
> > > > tcp_sendmsg timestamp with timestamps from other tx timestamping
> > > > callbacks (e.g., SND/SW/ACK).
> > > >
> > > > Without this patch, BPF program wouldn't know which timestamps belong
> > > > to which flow because of no socket lock protection. This new callback
> > > > is inserted in tcp_tx_timestamp() to address this issue because
> > > > tcp_tx_timestamp() still owns the same socket lock with
> > > > tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
> > > > the timestamping related fields for the skb, especially tskey. The
> > > > tskey is the bridge to do the correlation.
> > > >
> > > > For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
> > > > then stores the sendmsg timestamp at the bpf_sk_storage, correlating
> > > > this timestamp with its tskey that are later used in other sending
> > > > timestamping callbacks.
> > > >
> > > > Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
> > > > ---
> > > >  include/uapi/linux/bpf.h       | 5 +++++
> > > >  net/ipv4/tcp.c                 | 4 ++++
> > > >  tools/include/uapi/linux/bpf.h | 5 +++++
> > > >  3 files changed, 14 insertions(+)
> > > >
> > > > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > > > index 9355d617767f..86fca729fbd8 100644
> > > > --- a/include/uapi/linux/bpf.h
> > > > +++ b/include/uapi/linux/bpf.h
> > > > @@ -7052,6 +7052,11 @@ enum {
> > > >                                        * when SK_BPF_CB_TX_TIMESTAMPING
> > > >                                        * feature is on.
> > > >                                        */
> > > > +     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
> > > > +                                      * is triggered. It's used to correlate
> > > > +                                      * sendmsg timestamp with corresponding
> > > > +                                      * tskey.
> > > > +                                      */
> > > >  };
> > > >
> > > >  /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
> > > > diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> > > > index 12b9c4f9c151..4b9739cd3bc5 100644
> > > > --- a/net/ipv4/tcp.c
> > > > +++ b/net/ipv4/tcp.c
> > > > @@ -492,6 +492,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
> > > >               if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
> > > >                       shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
> > > >       }
> > > > +
> > > > +     if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
> > > > +         SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb)
> > > > +             bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB);
> > > >  }
> > > >
> > > >  static bool tcp_stream_is_readable(struct sock *sk, int target)
> > > > diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
> > > > index d3e2988b3b4c..2739ee0154a0 100644
> > > > --- a/tools/include/uapi/linux/bpf.h
> > > > +++ b/tools/include/uapi/linux/bpf.h
> > > > @@ -7042,6 +7042,11 @@ enum {
> > > >                                        * when SK_BPF_CB_TX_TIMESTAMPING
> > > >                                        * feature is on.
> > > >                                        */
> > > > +     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
> > > > +                                      * is triggered. It's used to correlate
> > > > +                                      * sendmsg timestamp with corresponding
> > > > +                                      * tskey.
> > > > +                                      */
> > >
> > > Feel free to decline this late in the review process, but a bit more
> > > bikeshedding..
> > >
> > > Can we spell out TSTAMP instead of TS in these definitions? Within
> > > the context of this series it is self-explanatory, but when reading
> > > kernel code the meaning of a two letter acronym is not that clear.
> >
> > Even though I feel reluctant to change across the whole series because
> > if so, I will adjust in many places. Of course, you're right about the
> > new name being clearer :)
> >
> > >
> > > And instead of SND can we use SENDMSG or something like that?
> > > SND here confused me as the software timestamp is SCM_TSTAMP_SND.
> >
> > I'm not sure about this. For TCP, it's not implemented in the
> > tcp_sendmsg_locked but tcp_tx_timestamp. Well, I have no strong
> > preference.
> >
> > You can make the final call :)
> 
> After taking a break, I feel full of energy and I will update them all
> as you requested :)

Awesome, thank you.