[PATCH net-next 00/15] Gated tracepoints for skb lifecycle (free+clone)

[PATCH net-next 00/15] Gated tracepoints for skb lifecycle (free+clone)

[Jakub Sitnicki]

This patch set implements Alexei's idea of gated tracepoints for skb lifecycle
events, which offers an alternative to an skb extension for BPF metadata [1]. I
have kept it down to a usable minimum - patches introduce gated tracepoints only
for skb consume/drop and clone/copy.

* Implementation

... is pretty straightforward. Please see patch 1 description for details.

One note - at first I tried to use of skb extensions - the free bit left in
skb->active_extensions and the skb_ext_reset/put/copy code paths. That turned
out to be a wrong direction. A "special" skb extension that doesn't require
space allocated in skb->extensions slab makes the logic messier.

Hence, I went back to simplest possible approach of having a dedicated bit
directly in sk_buff and adding callbacks directly from core networking code.

* Performance

Early results I've posted on Friday [2] were wrong. I owe a correction. They
were heavily skewed by KVM halt polling being enabled. That led to some
nonsensical numbers sometimes - like a plain TC packet counter measuring as more
expensive than TC + skb extensions.

New, stable, reproducible results tell a different story - gated tracepoints are
~2.5x more expensive than skb extensions:

| variant           | Δ busy         | ns/pkt @146k |
|-------------------|----------------|--------------|
| gated tracepoints | +16.4 ± 2.8 pp | ≈ +1125      |
| bpf skb ext       | +6.6 ± 1.1 pp  | ≈ +450       |

Setup:
- 146k pps spread over 3 flows,
- 36B UDP msgs over loopback,
- 6 measured vCPUs on pinned cores clamped @ 2.7 GHz,
- per-packet metadata on every packet,
- KVM halt polling disabled on host.

I've published full benchmark results + setup + code on GH [3].

* Final thoughts

Even though gated tracepoints are more CPU intensive, I think they offer a
better building platform for users. Every user/tenant manages their own metadata
stash and they can select an allocation strategy that best fits their use case.

There is extra CPU overhead, but the fact that we need to pay it only for
packets that we're tracing makes it palatable. As I mentioned earlier, we attach
metadata to less than 1% of skbs today. Naturally, we would be looking to lower
the gated-tracepoints cost.

There is also potential for giving users finer-grained control over metadata
lifetime in the future by adding tracepoints on skb scrub, GRO merge, or TCP
coalesce/collapse.

While we can make either solution work because we don't have tenants with access
to BPF in production, others might not have that luxury. Now would be a good
time for folks operating K8S environments to speak up.

Looking forward to feedback,
-jkbs

PS. I'm prepping for Plumbers this week so probably won't be able to circle back
to this until next week.

[1] https://lore.kernel.org/all/20260910-bpf-meta-inside-skb-ext-v2-0-0b21e42180b0@cloudflare.com/
[2] http://msgid.link/87fqyypp67.fsf@cloudflare.com/
[3] https://github.com/jsitnicki/skb-metadata-bench/blob/main/REPORT.md

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
Jakub Sitnicki (15):
      bpf: Add gated tracepoints for skb life-cycle
      bpf: Add bpf_skb_fclone_orig() kfunc
      bpf: Accept trusted pointers to kern ctx type for kfunc ctx args
      bpf: Allow access to bpf_sock_ops_kern->skb
      selftests/bpf: Basic tests for metadata passing with gated skb tracepoints
      selftests/bpf: Test passing metadata across veth and GRE
      selftests/bpf: Test metadata passing to cgroup_skb and sk_filter hooks
      selftests/bpf: Test passing metadata to sock_ops and LSM hooks
      selftests/bpf: Test passing metadata to kfree_skb tracepoint
      selftests/bpf: Test passing metadata to TCP egress
      selftests/bpf: Test passing metadata to netfilter hook
      selftests/bpf: Test passing metadata to/from LWT hooks
      selftests/bpf: Test passing metadata to seg6local End.BPF hook
      selftests/bpf: Test passing metadata to sk_skb stream verdict hook
      selftests/bpf: Use non-trivial test payload in xdp_context tests

 include/linux/skbuff.h                             |   38 +
 include/trace/events/skb.h                         |   57 +
 kernel/bpf/verifier.c                              |   22 +
 net/Kconfig                                        |    9 +
 net/core/dev.c                                     |    1 +
 net/core/filter.c                                  |  112 ++
 net/core/gro.c                                     |    1 +
 net/core/skbuff.c                                  |   25 +
 tools/testing/selftests/bpf/config                 |    2 +
 .../selftests/bpf/prog_tests/socket_helpers.h      |    1 +
 .../bpf/prog_tests/xdp_context_test_run.c          | 1399 +++++++++++++++++++-
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  |  395 +++++-
 12 files changed, 2049 insertions(+), 13 deletions(-)

[PATCH net-next 01/15] bpf: Add gated tracepoints for skb life-cycle

[Jakub Sitnicki]

BPF programs attached at different points in the network stack have no way
to pass data between each other on a per-packet basis. xdp/skb->data_meta
works for XDP-to-TC handoff, but is not available to programs running at
later hooks like cgroup/skb, sock_ops, socket filters, tracing or LSM.

Enable programs to keep per-packet metadata in a shared BPF map keyed by
the skb address. What rides on the packet is only a marker: a new
skb->traced flag, occupying the spare bit after csum_valid, which makes the
packet's lifetime events observable. The flag lives in the sk_buff itself,
not in the extension block, so it adds no memory pressure and naturally
survives packet scrubbing (tunnel encap/decap, cross-netns forwarding).

The flag makes the packet emit gated tracepoints, which fire only while the
flag is armed, driving the lifecycle of the map-backed metadata stash:

1. skb_free fires when the skb is freed, so the program can reap the stash
   entry. It carries the skb address, the free location, and the drop
   reason (mirroring kfree_skb). It is emitted from:

   - the kfree_skb and consume_skb tracepoint call sites, and
   - skb_release_head_state() to cover TCP RX delivery, queue purges, head
     theft in TCP coalesce/collapse,
   - napi_skb_free_stolen_head() and napi_reuse_skb() to cover skb stealing
     and recycling by GRO.

2. skb_copy fires when a copy or clone of a traced skb is created, so the
   program can propagate the entry. The flag is not inherited by the copy:
   __copy_skb_header() fires the tracepoint after the headers memcpy, which
   would otherwise silently carry the flag over, and the hook clears it
   again before handing the copy to the program, so the tracepoint program
   decides whether to tag the copy with bpf_trace_skb() as well.

The flag is armed by calling bpf_trace_skb(). It must be called from a hook
that owns the skb - the flag is updated without locking.

Guard the feature behind a new CONFIG_SKB_GATED_TRACEPOINTS option.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 include/linux/skbuff.h     | 38 ++++++++++++++++++++
 include/trace/events/skb.h | 57 ++++++++++++++++++++++++++++++
 net/Kconfig                |  9 +++++
 net/core/dev.c             |  1 +
 net/core/filter.c          | 87 ++++++++++++++++++++++++++++++++++++++++++++++
 net/core/gro.c             |  1 +
 net/core/skbuff.c          | 25 +++++++++++++
 7 files changed, 218 insertions(+)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 84308498a3a8..0353df4a4637 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -842,6 +842,7 @@ enum skb_tstamp_type {
  *	@encapsulation: indicates the inner headers in the skbuff are valid
  *	@encap_hdr_csum: software checksum is needed
  *	@csum_valid: checksum is already valid
+ *	@traced: tagged for gated life-cycle tracepoints
  *	@csum_not_inet: use CRC32c to resolve CHECKSUM_PARTIAL
  *	@csum_complete_sw: checksum was completed by software
  *	@csum_level: indicates the number of consecutive checksums found in
@@ -1002,6 +1003,9 @@ struct sk_buff {
 	__u8			encapsulation:1;
 	__u8			encap_hdr_csum:1;
 	__u8			csum_valid:1;
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+	__u8			traced:1;
+#endif
 #ifdef CONFIG_IPV6_NDISC_NODETYPE
 	__u8			ndisc_nodetype:2;
 #endif
@@ -5075,6 +5079,38 @@ void *skb_ext_add(struct sk_buff *skb, enum skb_ext_id id);
 void __skb_ext_del(struct sk_buff *skb, enum skb_ext_id id);
 void __skb_ext_put(struct skb_ext *ext);
 
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+
+void __skb_trace_free(struct sk_buff *skb, enum skb_drop_reason reason,
+		      void *location);
+void __skb_trace_copy(struct sk_buff *dst, const struct sk_buff *src);
+
+static __always_inline void skb_maybe_trace_free(struct sk_buff *skb,
+						 enum skb_drop_reason reason)
+{
+	if (skb->traced) {
+		__skb_trace_free(skb, reason, __builtin_return_address(0));
+		skb->traced = 0;
+	}
+}
+
+static __always_inline void skb_maybe_trace_copy(struct sk_buff *dst,
+						 const struct sk_buff *src)
+{
+	if (src->traced) {
+		dst->traced = 0;
+		__skb_trace_copy(dst, src);
+	}
+}
+
+#else
+
+static inline void skb_maybe_trace_free(struct sk_buff *skb,
+					enum skb_drop_reason reason) {}
+static inline void skb_maybe_trace_copy(struct sk_buff *dst, const struct sk_buff *src) {}
+
+#endif /* CONFIG_SKB_GATED_TRACEPOINTS */
+
 static inline void skb_ext_put(struct sk_buff *skb)
 {
 	if (skb->active_extensions)
@@ -5147,6 +5183,8 @@ static inline void skb_ext_del(struct sk_buff *skb, int unused) {}
 static inline void __skb_ext_copy(struct sk_buff *d, const struct sk_buff *s) {}
 static inline void skb_ext_copy(struct sk_buff *dst, const struct sk_buff *s) {}
 static inline bool skb_has_extensions(struct sk_buff *skb) { return false; }
+static inline void skb_maybe_trace_free(struct sk_buff *skb,
+					enum skb_drop_reason reason) {}
 #endif /* CONFIG_SKB_EXTENSIONS */
 
 static inline void nf_reset_ct(struct sk_buff *skb)
diff --git a/include/trace/events/skb.h b/include/trace/events/skb.h
index 2945aa7fe9a7..b162a8f23f26 100644
--- a/include/trace/events/skb.h
+++ b/include/trace/events/skb.h
@@ -92,6 +92,63 @@ TRACE_EVENT(skb_copy_datagram_iovec,
 	TP_printk("skbaddr=%p len=%d", __entry->skbaddr, __entry->len)
 );
 
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+
+#undef FN
+#undef FNe
+#define FN(reason)	{ SKB_DROP_REASON_##reason, #reason },
+#define FNe(reason)	{ SKB_DROP_REASON_##reason, #reason }
+
+TRACE_EVENT(skb_free,
+
+	    TP_PROTO(struct sk_buff *skb, enum skb_drop_reason reason,
+		     void *location),
+
+	    TP_ARGS(skb, reason, location),
+
+	    TP_STRUCT__entry(
+		    __field(	void *,		skbaddr)
+		    __field(	enum skb_drop_reason,	reason)
+		    __field(	void *,		location)
+		    ),
+
+	    TP_fast_assign(
+		    __entry->skbaddr = skb;
+		    __entry->reason = reason;
+		    __entry->location = location;
+		    ),
+
+	    TP_printk("skbaddr=%p reason=%s location=%pS",
+		      __entry->skbaddr,
+		      __print_symbolic(__entry->reason,
+				       DEFINE_DROP_REASON(FN, FNe)),
+		      __entry->location)
+	);
+
+#undef FN
+#undef FNe
+
+TRACE_EVENT(skb_copy,
+
+	    TP_PROTO(struct sk_buff *dst, const struct sk_buff *src),
+
+	    TP_ARGS(dst, src),
+
+	    TP_STRUCT__entry(
+		    __field(	void *,	dst)
+		    __field(	const void *,	src)
+		    ),
+
+	    TP_fast_assign(
+		    __entry->dst = dst;
+		    __entry->src = src;
+		    ),
+
+	    TP_printk("dst=%p src=%p", __entry->dst, __entry->src)
+	);
+
+#endif /* CONFIG_SKB_GATED_TRACEPOINTS */
+
 #endif /* _TRACE_SKB_H */
 
 /* This part must be outside protection */
diff --git a/net/Kconfig b/net/Kconfig
index 76ab44aa439a..c2e57198862c 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -539,4 +539,13 @@ config NET_TEST
 
 	  If unsure, say N.
 
+config SKB_GATED_TRACEPOINTS
+	bool "gated skb life-cycle tracepoints"
+	depends on BPF_SYSCALL
+	help
+	  Enable the gated skb life-cycle tracepoints (skb_free, skb_copy).
+	  The tracepoints fire only for packets tagged with bpf_trace_skb(),
+	  which arms a flag on the skb, allowing BPF programs to pass
+	  per-packet state across the network stack.
+
 endif   # if NET
diff --git a/net/core/dev.c b/net/core/dev.c
index f660fccfc0db..6a4c1d8b8f2f 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5863,6 +5863,7 @@ static __latent_entropy void net_tx_action(void)
 			else
 				trace_kfree_skb(skb, net_tx_action,
 						get_kfree_skb_cb(skb)->reason, NULL);
+			skb_maybe_trace_free(skb, get_kfree_skb_cb(skb)->reason);
 
 			if (skb->fclone != SKB_FCLONE_UNAVAILABLE)
 				__kfree_skb(skb);
diff --git a/net/core/filter.c b/net/core/filter.c
index 70dc621672f2..33c1ee3b966b 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -12426,6 +12426,51 @@ __bpf_kfunc int bpf_dynptr_from_skb(struct __sk_buff *s, u64 flags,
 	return 0;
 }
 
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+/**
+ * bpf_trace_skb() - Opt an skb into gated lifetime tracepoints
+ * @skb_: socket buffer to tag
+ * @flags__k: future use, must be zero
+ *
+ * Tag the skb as traced so that the gated tracepoints fire for it: skb_free
+ * when the skb is freed or stolen, and skb_copy when a copy or clone of the skb
+ * is created. Programs can use this to drive the life-cycle of per-packet
+ * metadata kept in a BPF map keyed by the skb address - reaping the entry on
+ * skb_free and propagating it on skb_copy. skb_free carries the drop reason,
+ * mirroring kfree_skb. The tracepoint program decides whether the copy is also
+ * traced; the tag is not inherited.
+ *
+ * The caller must own the skb because the traced flag is updated without
+ * locking.
+ *
+ * Return:
+ * * %0         - skb tagged or already tagged
+ * * %-EINVAL   - nonzero flags
+ */
+__bpf_kfunc int bpf_trace_skb(struct sk_buff *skb, u64 flags__k)
+{
+	if (flags__k)
+		return -EINVAL;
+
+	if (!skb->traced)
+		skb->traced = 1;
+
+	return 0;
+}
+
+/**
+ * bpf_skb_is_traced() - Check whether an skb is tagged
+ * @skb_: socket buffer to query
+ *
+ * Return: true if the skb was tagged with bpf_trace_skb().
+ */
+__bpf_kfunc bool bpf_skb_is_traced(struct sk_buff *skb)
+{
+	return skb->traced;
+}
+
+#endif /* CONFIG_SKB_GATED_TRACEPOINTS */
+
 /**
  * bpf_dynptr_from_skb_meta() - Initialize a dynptr to the skb metadata area.
  * @skb_: socket buffer carrying the metadata
@@ -12807,6 +12852,13 @@ BTF_KFUNCS_START(bpf_kfunc_check_set_skb_meta)
 BTF_ID_FLAGS(func, bpf_dynptr_from_skb_meta)
 BTF_KFUNCS_END(bpf_kfunc_check_set_skb_meta)
 
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+BTF_KFUNCS_START(bpf_kfunc_check_set_skb_tracepoints)
+BTF_ID_FLAGS(func, bpf_trace_skb)
+BTF_ID_FLAGS(func, bpf_skb_is_traced)
+BTF_KFUNCS_END(bpf_kfunc_check_set_skb_tracepoints)
+#endif
+
 BTF_KFUNCS_START(bpf_kfunc_check_set_xdp)
 BTF_ID_FLAGS(func, bpf_dynptr_from_xdp)
 BTF_ID_FLAGS(func, bpf_xdp_pull_data)
@@ -12838,6 +12890,13 @@ static const struct btf_kfunc_id_set bpf_kfunc_set_skb_meta = {
 	.set = &bpf_kfunc_check_set_skb_meta,
 };
 
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+static const struct btf_kfunc_id_set bpf_kfunc_set_skb_tracepoints = {
+	.owner = THIS_MODULE,
+	.set = &bpf_kfunc_check_set_skb_tracepoints,
+};
+#endif
+
 static const struct btf_kfunc_id_set bpf_kfunc_set_xdp = {
 	.owner = THIS_MODULE,
 	.set = &bpf_kfunc_check_set_xdp,
@@ -12880,6 +12939,34 @@ static int __init bpf_kfunc_init(void)
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_kfunc_set_skb);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_kfunc_set_skb_meta);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_ACT, &bpf_kfunc_set_skb_meta);
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_ACT,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_CGROUP_SKB,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SOCK_OPS,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SK_SKB,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SOCKET_FILTER,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_LWT_OUT,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_LWT_IN,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_LWT_XMIT,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_LWT_SEG6LOCAL,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_NETFILTER,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_LSM,
+					       &bpf_kfunc_set_skb_tracepoints);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING,
+					       &bpf_kfunc_set_skb_tracepoints);
+#endif
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &bpf_kfunc_set_xdp);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_CGROUP_SOCK_ADDR,
 					       &bpf_kfunc_set_sock_addr);
diff --git a/net/core/gro.c b/net/core/gro.c
index 29b4d02bf519..51cd0b6040af 100644
--- a/net/core/gro.c
+++ b/net/core/gro.c
@@ -683,6 +683,7 @@ static void napi_reuse_skb(struct napi_struct *napi, struct sk_buff *skb)
 		nf_reset_ct(skb);
 		skb->slow_gro = 0;
 	}
+	skb_maybe_trace_free(skb, SKB_CONSUMED);
 
 	napi->skb = skb;
 }
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 8912a66cd909..038854c83701 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -1187,6 +1187,7 @@ void skb_release_head_state(struct sk_buff *skb)
 	}
 	nf_reset_ct(skb);
 	skb_ext_reset(skb);
+	skb_maybe_trace_free(skb, SKB_DROP_REASON_NOT_SPECIFIED);
 }
 
 /* Free everything but the sk_buff shell. */
@@ -1229,6 +1230,7 @@ bool __sk_skb_reason_drop(const struct sock *sk, struct sk_buff *skb,
 		trace_consume_skb(skb, __builtin_return_address(0));
 	else
 		trace_kfree_skb(skb, __builtin_return_address(0), reason, sk);
+	skb_maybe_trace_free(skb, reason);
 	return true;
 }
 
@@ -1445,6 +1447,7 @@ void consume_skb(struct sk_buff *skb)
 		return;
 
 	trace_consume_skb(skb, __builtin_return_address(0));
+	skb_maybe_trace_free(skb, SKB_CONSUMED);
 	__kfree_skb(skb);
 }
 EXPORT_SYMBOL(consume_skb);
@@ -1460,6 +1463,7 @@ EXPORT_SYMBOL(consume_skb);
 void __consume_stateless_skb(struct sk_buff *skb)
 {
 	trace_consume_skb(skb, __builtin_return_address(0));
+	skb_maybe_trace_free(skb, SKB_CONSUMED);
 	skb_release_data(skb, SKB_CONSUMED);
 	kfree_skbmem(skb);
 }
@@ -1504,6 +1508,7 @@ void napi_skb_free_stolen_head(struct sk_buff *skb)
 		skb_orphan(skb);
 		skb->slow_gro = 0;
 	}
+	skb_maybe_trace_free(skb, SKB_CONSUMED);
 	napi_skb_cache_put(skb);
 }
 
@@ -1539,6 +1544,7 @@ void napi_consume_skb(struct sk_buff *skb, int budget)
 
 	/* if reaching here SKB is ready to free */
 	trace_consume_skb(skb, __builtin_return_address(0));
+	skb_maybe_trace_free(skb, SKB_CONSUMED);
 
 	/* if SKB is a clone, don't handle this case */
 	if (skb->fclone != SKB_FCLONE_UNAVAILABLE) {
@@ -1601,6 +1607,7 @@ static void __copy_skb_header(struct sk_buff *new, const struct sk_buff *old)
 	CHECK_SKB_FIELD(tc_index);
 #endif
 
+	skb_maybe_trace_copy(new, old);
 }
 
 /*
@@ -7342,6 +7349,24 @@ void __skb_ext_put(struct skb_ext *ext)
 	kmem_cache_free(skbuff_ext_cache, ext);
 }
 EXPORT_SYMBOL(__skb_ext_put);
+
+#ifdef CONFIG_SKB_GATED_TRACEPOINTS
+
+void __skb_trace_free(struct sk_buff *skb, enum skb_drop_reason reason,
+		      void *location)
+{
+	trace_skb_free(skb, reason, location);
+}
+EXPORT_SYMBOL(__skb_trace_free);
+
+void __skb_trace_copy(struct sk_buff *dst, const struct sk_buff *src)
+{
+	trace_skb_copy(dst, src);
+}
+EXPORT_SYMBOL(__skb_trace_copy);
+
+#endif
+
 #endif /* CONFIG_SKB_EXTENSIONS */
 
 static void kfree_skb_napi_cache(struct sk_buff *skb)

-- 
2.43.0

[PATCH net-next 02/15] bpf: Add bpf_skb_fclone_orig() kfunc

[Jakub Sitnicki]

TCP write-queue skbs are allocated from the fclone cache, the packet
transmitted on the wire is a fast clone. A prog can only associate
metadata with an skb by keying on its address. On retransmission TCP
reuses the write-queue original, a different address with no packet
hook, so the retransmit carries no metadata. A prog can use this kfunc
to retrieve the address of the write-queue original and populate the
metadata stash for it, so the metadata shows up on skb retransmits.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 net/core/filter.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/net/core/filter.c b/net/core/filter.c
index 33c1ee3b966b..fdfc1016030a 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -12469,6 +12469,30 @@ __bpf_kfunc bool bpf_skb_is_traced(struct sk_buff *skb)
 	return skb->traced;
 }
 
+/**
+ * bpf_skb_fclone_orig() - Return the fast-clone original of an skb
+ * @skb_: socket buffer allocated from the fclone cache
+ *
+ * TCP write-queue skbs are allocated from the fclone cache, the packet
+ * transmitted on the wire is a fast clone. A prog can only associate
+ * metadata with an skb by keying on its address. On retransmission TCP
+ * reuses the write-queue original, a different address with no packet
+ * hook, so the retransmit carries no metadata. A prog can use this kfunc
+ * to retrieve the address of the write-queue original and populate the
+ * metadata stash for it, so the metadata shows up on skb retransmits.
+ *
+ * Return:
+ * * orig skb address  - skb is a fast clone
+ * * %NULL             - skb is not a fast clone
+ */
+__bpf_kfunc struct sk_buff *bpf_skb_fclone_orig(struct sk_buff *skb)
+{
+	if (skb->fclone == SKB_FCLONE_CLONE)
+		return &container_of(skb, struct sk_buff_fclones, skb2)->skb1;
+
+	return NULL;
+}
+
 #endif /* CONFIG_SKB_GATED_TRACEPOINTS */
 
 /**
@@ -12856,6 +12880,7 @@ BTF_KFUNCS_END(bpf_kfunc_check_set_skb_meta)
 BTF_KFUNCS_START(bpf_kfunc_check_set_skb_tracepoints)
 BTF_ID_FLAGS(func, bpf_trace_skb)
 BTF_ID_FLAGS(func, bpf_skb_is_traced)
+BTF_ID_FLAGS(func, bpf_skb_fclone_orig, KF_RET_NULL)
 BTF_KFUNCS_END(bpf_kfunc_check_set_skb_tracepoints)
 #endif
 

-- 
2.43.0

[PATCH net-next 03/15] bpf: Accept trusted pointers to kern ctx type for kfunc ctx args

[Jakub Sitnicki]

A kfunc argument declared with the kernel projection of the program
context type (struct sk_buff for __sk_buff hooks) is classified as
KF_ARG_PTR_TO_CTX and, until now, accepted only the raw program context
pointer (PTR_TO_CTX).

That is fine when calling such a kfunc on the context itself, but it
blocks passing a kernel pointer derived from it, such as the fast-clone
twin returned by bpf_skb_fclone_twin():

    SEC("tc")
    int tc_write_twins(struct __sk_buff *ctx)
    {
        struct sk_buff *twin;

        bpf_trace_skb(bpf_cast_to_kern_ctx(ctx), 0);  /* ok: PTR_TO_CTX */

        twin = bpf_skb_fclone_twin(bpf_cast_to_kern_ctx(ctx));
        if (!twin)
            return TC_ACT_UNSPEC;

        /* rejected: twin is PTR_TO_BTF_ID, not PTR_TO_CTX */
        bpf_trace_skb(twin, 0);
        return TC_ACT_UNSPEC;
    }

The last call fails to verify with "expected pointer to ctx, but got
ptr_to_btf_id", even though twin is a trusted pointer to the very type
the kfunc expects.

Accept a trusted PTR_TO_BTF_ID pointing to the kernel context type in
place of the raw ctx. Restricting the relaxation to trusted pointers
without unsafe modifiers keeps the guarantee that the argument is a
valid kernel object on the current path.

The type match is only applied when the register points into vmlinux BTF:
the type ID comparison is same-BTF, so a module-local type ID cannot alias
the vmlinux context type ID.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 kernel/bpf/verifier.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 41b49c56e123..da5d6de58650 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -12891,6 +12891,23 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
 			break;
 		case KF_ARG_PTR_TO_CTX:
 			if (reg->type != PTR_TO_CTX) {
+				/* Only accept trusted kernel-context pointers from
+				 * vmlinux BTF. reg->btf_id is scoped to reg->btf,
+				 * so a module-local ID could otherwise alias the
+				 * vmlinux context type ID.
+				 */
+				if (base_type(reg->type) == PTR_TO_BTF_ID &&
+				    reg->btf == btf_vmlinux &&
+				    is_trusted_reg(env, reg) &&
+				    !bpf_type_has_unsafe_modifiers(reg->type)) {
+					ret = get_kern_ctx_btf_id(&env->log,
+								  resolve_prog_type(env->prog));
+					if (ret < 0)
+						return -EINVAL;
+					if (reg->btf_id == ret)
+						break;
+				}
+
 				verbose(env, "%s expected pointer to ctx, but got %s\n",
 					reg_arg_name(env, argno), reg_type_str(env, reg->type));
 				bpf_diag_call_arg_fmt(env, insn_idx, argno, func_name,

-- 
2.43.0

[PATCH net-next 04/15] bpf: Allow access to bpf_sock_ops_kern->skb

[Jakub Sitnicki]

sock_ops programs receive bpf_sock_ops_kern as their kernel context, which
holds a pointer to the sk_buff being processed. Mark bpf_sock_ops_kern->skb
as BTF_TYPE_SAFE_TRUSTED_OR_NULL so that BPF programs can dereference it
and pass it to kfuncs expecting a trusted sk_buff pointer.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 kernel/bpf/verifier.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index da5d6de58650..ac7b7871c2be 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5936,6 +5936,10 @@ BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct vm_area_struct) {
 	struct file *vm_file;
 };
 
+BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct bpf_sock_ops_kern) {
+	struct sk_buff *skb;
+};
+
 static bool type_is_rcu(struct bpf_verifier_env *env,
 			struct bpf_reg_state *reg,
 			const char *field_name, u32 btf_id)
@@ -5978,6 +5982,7 @@ static bool type_is_trusted_or_null(struct bpf_verifier_env *env,
 	BTF_TYPE_EMIT(BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct socket));
 	BTF_TYPE_EMIT(BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct dentry));
 	BTF_TYPE_EMIT(BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct vm_area_struct));
+	BTF_TYPE_EMIT(BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct bpf_sock_ops_kern));
 
 	return btf_nested_type_is_trusted(&env->log, reg, field_name, btf_id,
 					  "__safe_trusted_or_null");

-- 
2.43.0

[PATCH net-next 05/15] selftests/bpf: Basic tests for metadata passing with gated skb tracepoints

[Jakub Sitnicki]

Cover metadata passing where test programs stash per-packet metadata in a
hash map (meta_map) keyed by the skb address, tag packets as traced with
bpf_trace_skb(), propagate entries from the tp_btf/skb_copy tracepoint, and
reclaim them from the raw_tp/skb_free tracepoint:

- write_read: stash metadata at tap ingress, read it back on the same hook
  after a device round trip, and verify the free tracepoint drained the
  meta_map
- clone_read: metadata stashed at tap ingress is readable from a mirred
  clone (tc mirror to a dummy device) - the copy tracepoint propagates the
  entry to the clone, and the free tracepoint reaps both

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 tools/testing/selftests/bpf/config                 |   2 +
 .../bpf/prog_tests/xdp_context_test_run.c          | 165 ++++++++++++++++++++-
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  |  83 +++++++++++
 3 files changed, 245 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index ea7044f30adc..692909b28d3a 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -7,6 +7,7 @@ CONFIG_BPF_JIT=y
 CONFIG_BPF_KPROBE_OVERRIDE=y
 CONFIG_BPF_LIRC_MODE2=y
 CONFIG_BPF_LSM=y
+CONFIG_SKB_GATED_TRACEPOINTS=y
 CONFIG_BPF_STREAM_PARSER=y
 CONFIG_BPF_SYSCALL=y
 # CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
@@ -136,3 +137,4 @@ CONFIG_SMC_HS_CTRL_BPF=y
 CONFIG_DIBS=y
 CONFIG_DIBS_LO=y
 CONFIG_PM_WAKELOCKS=y
+CONFIG_NET_SCH_NETEM=y
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index 448807676176..c9b115c14acb 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -16,6 +16,7 @@
 #define DUMMY_NAME "dum0"
 #define TAP_NETNS "xdp_context_tuntap"
 #define LWT_NETNS "xdp_context_lwt"
+#define SKB_TRACE_NETNS "skb_trace_tuntap"
 
 #define TEST_PAYLOAD_LEN 32
 static const __u8 test_payload[TEST_PAYLOAD_LEN] = {
@@ -331,10 +332,11 @@ void test_xdp_context_veth(void)
 	netns_free(tx_ns);
 }
 
-static void test_tuntap(struct bpf_program *xdp_prog,
-			struct bpf_program *tc_prio_1_prog,
-			struct bpf_program *tc_prio_2_prog,
-			bool *test_pass)
+static void __test_tuntap(const char *nsname,
+			  struct bpf_program *xdp_prog,
+			  struct bpf_program *tc_prio_1_prog,
+			  struct bpf_program *tc_prio_2_prog,
+			  bool *test_pass)
 {
 	LIBBPF_OPTS(bpf_tc_hook, tc_hook, .attach_point = BPF_TC_INGRESS);
 	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
@@ -345,7 +347,7 @@ static void test_tuntap(struct bpf_program *xdp_prog,
 
 	*test_pass = false;
 
-	ns = netns_new(TAP_NETNS, true);
+	ns = netns_new(nsname, true);
 	if (!ASSERT_OK_PTR(ns, "create and open ns"))
 		return;
 
@@ -396,6 +398,15 @@ static void test_tuntap(struct bpf_program *xdp_prog,
 	netns_free(ns);
 }
 
+static void test_tuntap(struct bpf_program *xdp_prog,
+			struct bpf_program *tc_prio_1_prog,
+			struct bpf_program *tc_prio_2_prog,
+			bool *test_pass)
+{
+	__test_tuntap(TAP_NETNS, xdp_prog, tc_prio_1_prog, tc_prio_2_prog,
+		      test_pass);
+}
+
 /* Write a packet to a tap dev and copy it to ingress of a dummy dev */
 static void test_tuntap_mirred(struct bpf_program *xdp_prog,
 			       struct bpf_program *tc_prog,
@@ -693,3 +704,147 @@ void test_xdp_context_lwt_encap(void)
 
 	test_xdp_meta__destroy(skel);
 }
+
+/* Assert that the free tracepoint prog drained the metadata stash */
+static void assert_meta_stash_empty(struct bpf_map *meta_map)
+{
+	struct bpf_map_info info = {};
+	__u32 info_len = sizeof(info);
+
+	if (!ASSERT_OK(bpf_map_get_info_by_fd(bpf_map__fd(meta_map), &info,
+					      &info_len),
+		       "get_info"))
+		return;
+
+	ASSERT_EQ(info.map_extra, 0, "meta_map not empty");
+}
+
+static void test_skb_trace__tuntap(struct test_xdp_meta *skel)
+{
+	struct bpf_link *link;
+
+	skel->bss->free_count = 0;
+
+	link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(link, "prog_attach"))
+		return;
+
+	__test_tuntap(SKB_TRACE_NETNS, NULL /* xdp */, skel->progs.tc_meta_write,
+		      skel->progs.tc_meta_read, &skel->bss->test_pass);
+
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+	bpf_link__destroy(link);
+}
+
+/*
+ * Test if stashed metadata survives skb clone (via tc mirred).
+ * tap_prog writes on the tap ingress, dummy_prog runs on the clone
+ * (dummy ingress).
+ */
+static void test_skb_trace__mirred_clone(struct test_xdp_meta *skel)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook, .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	struct bpf_link *put_link = NULL, *copy_link = NULL;
+	struct netns_obj *ns = NULL;
+	int dummy_ifindex;
+	int tap_ifindex;
+	int tap_fd = -1;
+	int ret;
+
+	skel->bss->write_done = false;
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+	skel->bss->copy_done = false;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	copy_link = bpf_program__attach(skel->progs.skb_copy);
+	if (!ASSERT_OK_PTR(copy_link, "copy_attach"))
+		goto close;
+
+	ns = netns_new("mirred_clone", true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto close;
+
+	/* Dummy dev: attach reader */
+	SYS(close, "ip link add name " DUMMY_NAME " type dummy");
+	SYS(close, "ip link set dev " DUMMY_NAME " up");
+
+	dummy_ifindex = if_nametoindex(DUMMY_NAME);
+	if (!ASSERT_GT(dummy_ifindex, 0, "dummy_ifindex"))
+		goto close;
+
+	tc_hook.ifindex = dummy_ifindex;
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "dummy_hook_create"))
+		goto close;
+
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_read);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "dummy_attach"))
+		goto close;
+
+	/* TAP dev: attach writer + mirred to dummy */
+	tap_fd = open_tuntap(TAP_NAME, true);
+	if (!ASSERT_GE(tap_fd, 0, "open_tuntap"))
+		goto close;
+
+	SYS(close, "ip link set dev " TAP_NAME " up");
+
+	tap_ifindex = if_nametoindex(TAP_NAME);
+	if (!ASSERT_GT(tap_ifindex, 0, "tap_ifindex"))
+		goto close;
+
+	tc_hook.ifindex = tap_ifindex;
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "tap_hook_create"))
+		goto close;
+
+	tc_opts.prog_id = 0;
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "tap_attach"))
+		goto close;
+
+	SYS(close, "tc filter add dev " TAP_NAME " ingress "
+		   "protocol all matchall "
+		   "action mirred ingress mirror dev " DUMMY_NAME);
+
+	ret = write_test_packet(tap_fd);
+	if (!ASSERT_OK(ret, "write_test_packet"))
+		goto close;
+
+	ASSERT_TRUE(skel->bss->write_done, "write_done");
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+	ASSERT_TRUE(skel->bss->copy_done, "copy_done");
+	ASSERT_GE(skel->bss->free_count, 2, "free_count");
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+close:
+	bpf_link__destroy(copy_link);
+	bpf_link__destroy(put_link);
+	if (tap_fd >= 0)
+		close(tap_fd);
+	netns_free(ns);
+}
+
+void test_skb_trace_basic(void)
+{
+	struct test_xdp_meta *skel = NULL;
+
+	skel = test_xdp_meta__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open and load skeleton"))
+		return;
+
+	if (test__start_subtest("write_read"))
+		test_skb_trace__tuntap(skel);
+	if (test__start_subtest("clone_read"))
+		test_skb_trace__mirred_clone(skel);
+
+	test_xdp_meta__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 08b03be0b891..35a2c0e6ccb0 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -3,9 +3,11 @@
 
 #include <bpf/bpf_endian.h>
 #include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
 #include <errno.h>
 
 #include "bpf_kfuncs.h"
+#include "bpf_misc.h"
 #include "bpf_tracing_net.h"
 
 #define META_SIZE 32
@@ -689,4 +691,85 @@ int helper_skb_change_proto(struct __sk_buff *ctx)
 	return TC_ACT_SHOT;
 }
 
+struct {
+	__uint(type, BPF_MAP_TYPE_RHASH);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__uint(max_entries, 128);
+	__type(key, struct __sk_buff *);
+	__type(value, __u8[META_SIZE]);
+} meta_map SEC(".maps");
+
+bool write_done;
+
+static void write_meta(struct __sk_buff *ctx, void *map)
+{
+	if (bpf_map_update_elem(map, &ctx, &meta_want, BPF_NOEXIST))
+		return;
+	bpf_trace_skb(bpf_cast_to_kern_ctx(ctx), 0);
+	write_done = true;
+}
+
+/* Stash test metadata in the meta_map, keyed by the skb address */
+SEC("tc")
+int tc_meta_write(struct __sk_buff *ctx)
+{
+	if (!is_test_packet_tc(ctx))
+		return TC_ACT_SHOT;
+
+	write_meta(ctx, &meta_map);
+	return TC_ACT_UNSPEC;
+}
+
+/* Read test metadata stashed in the meta_map on the same hook */
+SEC("tc")
+int tc_meta_read(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		return TC_ACT_SHOT;
+	if (!check_metadata(meta_have))
+		return TC_ACT_SHOT;
+
+	test_pass = true;
+	return TC_ACT_UNSPEC;
+}
+
+int free_count;
+
+SEC("raw_tp/skb_free")
+int BPF_PROG(skb_free, struct sk_buff *skb, enum skb_drop_reason reason,
+	     void *location)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &skb);
+	if (!meta_have)
+		return 0;
+	if (bpf_map_delete_elem(&meta_map, &skb))
+		return 0;
+
+	free_count++;
+	return 0;
+}
+
+bool copy_done;
+
+SEC("tp_btf/skb_copy")
+int BPF_PROG(skb_copy, struct sk_buff *dst, struct sk_buff *src)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &src);
+	if (!meta_have)
+		return 0;
+	if (bpf_map_update_elem(&meta_map, &dst, meta_have, BPF_NOEXIST))
+		return 0;
+
+	bpf_trace_skb(dst, 0);
+	copy_done = true;
+	return 0;
+}
+
 char _license[] SEC("license") = "GPL";

-- 
2.43.0

[PATCH net-next 06/15] selftests/bpf: Test passing metadata across veth and GRE

[Jakub Sitnicki]

Stash metadata at TC egress and read it back at TC ingress after the packet
crosses device boundaries, relying on the gated skb tracepoints to reclaim
the stash once the packet is freed:

- survives_veth: veth forwarding between two netns
- survives_gre: GRE encap/decap between two gretap endpoints over loopback

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 190 +++++++++++++++++++++
 1 file changed, 190 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index c9b115c14acb..1155e868ffbd 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -833,6 +833,192 @@ static void test_skb_trace__mirred_clone(struct test_xdp_meta *skel)
 	netns_free(ns);
 }
 
+/* Test if skb_ext survives veth cross-netns forward */
+static void test_skb_trace__veth(struct test_xdp_meta *skel)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tx_hook, .attach_point = BPF_TC_EGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tx_opts, .handle = 1, .priority = 1);
+	LIBBPF_OPTS(bpf_tc_hook, rx_hook, .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, rx_opts, .handle = 1, .priority = 1);
+	struct netns_obj *rx_ns = NULL, *tx_ns = NULL;
+	struct bpf_link *put_link = NULL;
+	struct nstoken *nstoken = NULL;
+	int rx_ifindex, tx_ifindex;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	tx_ns = netns_new(TX_NETNS, false);
+	if (!ASSERT_OK_PTR(tx_ns, "create tx_ns"))
+		goto close;
+
+	rx_ns = netns_new(RX_NETNS, false);
+	if (!ASSERT_OK_PTR(rx_ns, "create rx_ns"))
+		goto close;
+
+	SYS(close, "ip link add " RX_NAME " netns " RX_NETNS
+	    " type veth peer name " TX_NAME " netns " TX_NETNS);
+
+	/* Setup RX side: TC ingress reader */
+	nstoken = open_netns(RX_NETNS);
+	if (!ASSERT_OK_PTR(nstoken, "setns rx_ns"))
+		goto close;
+
+	SYS(close, "ip link set dev " RX_NAME " up");
+
+	rx_ifindex = if_nametoindex(RX_NAME);
+	if (!ASSERT_GT(rx_ifindex, 0, "if_nametoindex rx"))
+		goto close;
+
+	rx_hook.ifindex = rx_ifindex;
+	ret = bpf_tc_hook_create(&rx_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create rx"))
+		goto close;
+
+	rx_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_read);
+	ret = bpf_tc_attach(&rx_hook, &rx_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach rx"))
+		goto close;
+
+	close_netns(nstoken);
+
+	/* Setup TX side: TC egress writer */
+	nstoken = open_netns(TX_NETNS);
+	if (!ASSERT_OK_PTR(nstoken, "setns tx_ns"))
+		goto close;
+
+	SYS(close, "ip link set dev " TX_NAME " up");
+
+	tx_ifindex = if_nametoindex(TX_NAME);
+	if (!ASSERT_GT(tx_ifindex, 0, "if_nametoindex tx"))
+		goto close;
+
+	tx_hook.ifindex = tx_ifindex;
+	ret = bpf_tc_hook_create(&tx_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create tx"))
+		goto close;
+
+	tx_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write);
+	ret = bpf_tc_attach(&tx_hook, &tx_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach tx"))
+		goto close;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	ret = send_test_packet(tx_ifindex);
+	if (!ASSERT_OK(ret, "send_test_packet"))
+		goto close;
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+close:
+	bpf_link__destroy(put_link);
+	close_netns(nstoken);
+	netns_free(rx_ns);
+	netns_free(tx_ns);
+}
+
+/* Test if skb_ext survives GRE tunnel encap+decap */
+static void test_skb_trace__gre(struct test_xdp_meta *skel)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tx_hook, .attach_point = BPF_TC_EGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tx_opts, .handle = 1, .priority = 1);
+	LIBBPF_OPTS(bpf_tc_hook, rx_hook, .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, rx_opts, .handle = 1, .priority = 1);
+	struct bpf_link *put_link = NULL;
+	struct netns_obj *ns = NULL;
+	int tx_ifindex;
+	int rx_ifindex;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	ns = netns_new("gre_test", true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto close;
+
+	/* Setup: gre_tx -> lo -> gre_rx */
+	SYS(close, "ip link set lo up");
+	SYS(close, "ip link add gre_tx type gretap"
+	    " local 127.0.0.1 remote 127.0.0.2");
+	SYS(close, "ip link set gre_tx up");
+	SYS(close, "ip addr add 127.0.0.2/8 dev lo");
+	SYS(close, "ip link add gre_rx type gretap"
+	    " local 127.0.0.2 remote 127.0.0.1");
+	SYS(close, "ip link set gre_rx up");
+
+	/* Write skb_ext on TC egress on GRE tx */
+	tx_ifindex = if_nametoindex("gre_tx");
+	if (!ASSERT_GT(tx_ifindex, 0, "tx_ifindex"))
+		goto close;
+
+	tx_hook.ifindex = tx_ifindex;
+	ret = bpf_tc_hook_create(&tx_hook);
+	if (!ASSERT_OK(ret, "tx_hook_create"))
+		goto close;
+
+	tx_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write);
+	ret = bpf_tc_attach(&tx_hook, &tx_opts);
+	if (!ASSERT_OK(ret, "tx_attach"))
+		goto close;
+
+	/* Read skb_ext on TC ingress on GRE rx */
+	rx_ifindex = if_nametoindex("gre_rx");
+	if (!ASSERT_GT(rx_ifindex, 0, "rx_ifindex"))
+		goto close;
+
+	rx_hook.ifindex = rx_ifindex;
+	ret = bpf_tc_hook_create(&rx_hook);
+	if (!ASSERT_OK(ret, "rx_hook_create"))
+		goto close;
+
+	rx_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_read);
+	ret = bpf_tc_attach(&rx_hook, &rx_opts);
+	if (!ASSERT_OK(ret, "rx_attach"))
+		goto close;
+
+	/* Then use send_test_packet on GRE tx */
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	ret = send_test_packet(tx_ifindex);
+	if (!ASSERT_OK(ret, "send_test_packet"))
+		goto close;
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+close:
+	bpf_link__destroy(put_link);
+	netns_free(ns);
+}
+
 void test_skb_trace_basic(void)
 {
 	struct test_xdp_meta *skel = NULL;
@@ -845,6 +1031,10 @@ void test_skb_trace_basic(void)
 		test_skb_trace__tuntap(skel);
 	if (test__start_subtest("clone_read"))
 		test_skb_trace__mirred_clone(skel);
+	if (test__start_subtest("survives_veth"))
+		test_skb_trace__veth(skel);
+	if (test__start_subtest("survives_gre"))
+		test_skb_trace__gre(skel);
 
 	test_xdp_meta__destroy(skel);
 }

-- 
2.43.0

[PATCH net-next 07/15] selftests/bpf: Test metadata passing to cgroup_skb and sk_filter hooks

[Jakub Sitnicki]

Extend metadata cross-hook coverage to UDP readers: attach a TC ingress
program on loopback that tags the skb, send a UDP datagram over loopback,
and read the stashed metadata back from a cgroup_skb/ingress program and
from a socket filter attached with SO_ATTACH_BPF.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 139 +++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  |  33 +++++
 2 files changed, 172 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index 1155e868ffbd..25a16daf5fdf 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -1038,3 +1038,142 @@ void test_skb_trace_basic(void)
 
 	test_xdp_meta__destroy(skel);
 }
+
+/* Send test_payload over loopback UDP to recv_fd */
+static int send_loopback_udp(int recv_fd)
+{
+	struct sockaddr_in addr = {
+		.sin_family = AF_INET,
+		.sin_addr.s_addr = htonl(INADDR_LOOPBACK),
+	};
+	char buf[TEST_PAYLOAD_LEN];
+	int ret = -1;
+	int fd = -1;
+	int port;
+
+	port = get_socket_local_port(recv_fd);
+	if (!ASSERT_GE(port, 0, "get_port"))
+		goto out;
+
+	fd = socket(AF_INET, SOCK_DGRAM, 0);
+	if (!ASSERT_GE(fd, 0, "socket"))
+		goto out;
+
+	addr.sin_port = port;
+	sendto(fd, test_payload, TEST_PAYLOAD_LEN, 0,
+	       (void *)&addr, sizeof(addr));
+	recvfrom(recv_fd, buf, sizeof(buf), 0, NULL, NULL);
+	ret = 0;
+out:
+	if (fd >= 0)
+		close(fd);
+	return ret;
+}
+
+enum udp_reader_type {
+	READER_CGRP_SKB,
+	READER_SK_FILTER,
+};
+
+/* Test metadata passing across TC ingress -> UDP reader hook */
+static void test_skb_trace__udp(struct test_xdp_meta *skel, const char *name,
+				enum udp_reader_type reader)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook,
+		    .ifindex = 1 /* IFINDEX_LO */,
+		    .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	struct bpf_link *reader_link = NULL;
+	struct bpf_link *put_link = NULL;
+	struct netns_obj *ns = NULL;
+	int server_fd = -1;
+	int cgroup_fd = -1;
+	int filter_fd;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	ns = netns_new(name, true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto cleanup;
+
+	cgroup_fd = test__join_cgroup("/skb_ext_udp");
+	if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+		goto cleanup;
+
+	server_fd = start_server(AF_INET, SOCK_DGRAM, "127.0.0.1", 0, 0);
+	if (!ASSERT_GE(server_fd, 0, "start_server"))
+		goto cleanup;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup;
+
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach"))
+		goto cleanup;
+
+	switch (reader) {
+	case READER_CGRP_SKB:
+		reader_link = bpf_program__attach_cgroup(skel->progs.cgrp_meta_read,
+							 cgroup_fd);
+		if (!ASSERT_OK_PTR(reader_link, "attach_cgroup"))
+			goto cleanup;
+		break;
+	case READER_SK_FILTER:
+		filter_fd = bpf_program__fd(skel->progs.sk_filter_meta_read);
+		ret = setsockopt(server_fd, SOL_SOCKET, SO_ATTACH_BPF,
+				 &filter_fd, sizeof(filter_fd));
+		if (!ASSERT_OK(ret, "attach_socket_filter"))
+			goto cleanup;
+		break;
+	}
+
+	if (send_loopback_udp(server_fd))
+		goto cleanup;
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+cleanup:
+	bpf_link__destroy(reader_link);
+	bpf_link__destroy(put_link);
+	bpf_tc_hook_destroy(&tc_hook);
+	if (server_fd >= 0)
+		close(server_fd);
+	if (cgroup_fd >= 0)
+		close(cgroup_fd);
+	netns_free(ns);
+}
+
+void test_skb_trace_cross_hook(void)
+{
+	struct test_xdp_meta *skel = NULL;
+
+	skel = test_xdp_meta__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open and load skeleton"))
+		return;
+
+	if (test__start_subtest("tc_to_cgrp_ingress"))
+		test_skb_trace__udp(skel, "tc_to_cgrp_ingress", READER_CGRP_SKB);
+	if (test__start_subtest("tc_to_sk_filter"))
+		test_skb_trace__udp(skel, "tc_to_sk_filter", READER_SK_FILTER);
+
+	test_xdp_meta__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 35a2c0e6ccb0..32c39c264cd7 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -772,4 +772,37 @@ int BPF_PROG(skb_copy, struct sk_buff *dst, struct sk_buff *src)
 	return 0;
 }
 
+/* Read skb_ext from cgroup/skb ingress -- tests cross-hook survival */
+SEC("cgroup_skb/ingress")
+int cgrp_meta_read(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		return 1;
+	if (!check_metadata(meta_have))
+		return 1;
+
+	test_pass = true;
+	return 1;
+}
+
+/* Read skb_ext from socket filter -- tests TC -> sk_filter path */
+SEC("socket")
+int sk_filter_meta_read(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		goto out;
+	if (!check_metadata(meta_have))
+		goto out;
+
+	test_pass = true;
+out:
+	return ctx->len;
+}
+
 char _license[] SEC("license") = "GPL";

-- 
2.43.0

[PATCH net-next 08/15] selftests/bpf: Test passing metadata to sock_ops and LSM hooks

[Jakub Sitnicki]

Extend metadata cross-hook coverage to the TCP ingress path: attach a TC
ingress program on loopback that tags the skb with bpf_trace_skb(), and
read the stashed metadata back from a sock_ops program
(BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB, reaching the skb via
bpf_sock_ops_kern->skb) and from the LSM inet_conn_established hook.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 113 ++++++++++++++++++++-
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  |  68 +++++++++++++
 2 files changed, 178 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index 25a16daf5fdf..b90ae0a6156d 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -1162,6 +1162,112 @@ static void test_skb_trace__udp(struct test_xdp_meta *skel, const char *name,
 	netns_free(ns);
 }
 
+enum tcp_reader_type {
+	READER_SKOPS,
+	READER_LSM,
+};
+
+/* Test metadata passing across TC ingress -> TCP reader hook */
+static void test_skb_trace__tcp_ingress(struct test_xdp_meta *skel, const char *name,
+					enum tcp_reader_type reader)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook,
+		    .ifindex = 1 /* IFINDEX_LO */,
+		    .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	struct bpf_link *reader_link = NULL;
+	struct bpf_link *put_link = NULL;
+	struct netns_obj *ns = NULL;
+	int server_fd = -1;
+	int cgroup_fd = -1;
+	int client_fd = -1;
+	int conn_fd = -1;
+	int port;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	ns = netns_new(name, true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto cleanup;
+
+	cgroup_fd = test__join_cgroup("/skb_ext_tcp");
+	if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+		goto cleanup;
+
+	server_fd = start_server(AF_INET, SOCK_STREAM, "127.0.0.1", 0, 0);
+	if (!ASSERT_GE(server_fd, 0, "start_server"))
+		goto cleanup;
+
+	port = get_socket_local_port(server_fd);
+	if (!ASSERT_GE(port, 0, "get_port"))
+		goto cleanup;
+
+	skel->bss->target_port = port;
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup;
+
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write_port);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach"))
+		goto cleanup;
+
+	switch (reader) {
+	case READER_SKOPS:
+		reader_link = bpf_program__attach_cgroup(skel->progs.skops_meta_read,
+							 cgroup_fd);
+		if (!ASSERT_OK_PTR(reader_link, "attach_skops"))
+			goto cleanup;
+		break;
+	case READER_LSM:
+		reader_link = bpf_program__attach_lsm(skel->progs.lsm_meta_read);
+		if (!ASSERT_OK_PTR(reader_link, "attach_lsm"))
+			goto cleanup;
+		break;
+	}
+
+	client_fd = connect_to_fd(server_fd, 0);
+	if (!ASSERT_GE(client_fd, 0, "connect"))
+		goto cleanup;
+
+	conn_fd = accept(server_fd, NULL, NULL);
+	if (!ASSERT_GE(conn_fd, 0, "accept"))
+		goto cleanup;
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+cleanup:
+	if (conn_fd >= 0)
+		close(conn_fd);
+	if (client_fd >= 0)
+		close(client_fd);
+	bpf_link__destroy(reader_link);
+	bpf_link__destroy(put_link);
+	bpf_tc_hook_destroy(&tc_hook);
+	if (server_fd >= 0)
+		close(server_fd);
+	if (cgroup_fd >= 0)
+		close(cgroup_fd);
+	netns_free(ns);
+}
+
 void test_skb_trace_cross_hook(void)
 {
 	struct test_xdp_meta *skel = NULL;
@@ -1174,6 +1280,7 @@ void test_skb_trace_cross_hook(void)
 		test_skb_trace__udp(skel, "tc_to_cgrp_ingress", READER_CGRP_SKB);
 	if (test__start_subtest("tc_to_sk_filter"))
 		test_skb_trace__udp(skel, "tc_to_sk_filter", READER_SK_FILTER);
-
-	test_xdp_meta__destroy(skel);
-}
+	if (test__start_subtest("tc_to_lsm"))
+		test_skb_trace__tcp_ingress(skel, "tc_to_lsm", READER_LSM);
+	if (test__start_subtest("tc_to_skops"))
+		test_skb_trace__tcp_ingress(skel, "tc_to_skops", READER_SKOPS);
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 32c39c264cd7..4da50c848beb 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -788,6 +788,74 @@ int cgrp_meta_read(struct __sk_buff *ctx)
 	return 1;
 }
 
+__be16 target_port;
+
+#define TCPV4_HDR_OFF	(sizeof(struct ethhdr) + sizeof(struct iphdr))
+#define TCPV4_SPORT_OFF	(TCPV4_HDR_OFF + offsetof(struct tcphdr, source))
+#define TCPV4_DPORT_OFF	(TCPV4_HDR_OFF + offsetof(struct tcphdr, dest))
+
+/* Write metadata on TCP packets to/from target_port */
+SEC("tc")
+int tc_meta_write_port(struct __sk_buff *ctx)
+{
+	__be16 sport, dport;
+
+	if (ctx->protocol != __bpf_constant_htons(ETH_P_IP))
+		return TC_ACT_UNSPEC;
+	if (bpf_skb_load_bytes(ctx, TCPV4_SPORT_OFF, &sport, sizeof(sport)))
+		return TC_ACT_UNSPEC;
+	if (bpf_skb_load_bytes(ctx, TCPV4_DPORT_OFF, &dport, sizeof(dport)))
+		return TC_ACT_UNSPEC;
+	if (sport != target_port && dport != target_port)
+		return TC_ACT_UNSPEC;
+
+	write_meta(ctx, &meta_map);
+
+	return TC_ACT_UNSPEC;
+}
+
+/* Read stashed metadata from sock_ops passive established hook */
+SEC("sockops")
+int skops_meta_read(struct bpf_sock_ops *ctx)
+{
+	struct bpf_sock_ops_kern *kctx;
+	struct sk_buff *skb;
+	__u8 *meta_have;
+
+	if (ctx->op != BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB)
+		return 1;
+
+	kctx = bpf_cast_to_kern_ctx(ctx);
+	skb = kctx->skb;
+	if (!skb)
+		return 1;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &skb);
+	if (!meta_have)
+		return 1;
+	if (!check_metadata(meta_have))
+		return 1;
+
+	test_pass = true;
+	return 1;
+}
+
+/* Read stashed metadata from LSM inet_conn_established hook */
+SEC("lsm/inet_conn_established")
+int BPF_PROG(lsm_meta_read, struct sock *sk, struct sk_buff *skb)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &skb);
+	if (!meta_have)
+		return 0;
+	if (!check_metadata(meta_have))
+		return 0;
+
+	test_pass = true;
+	return 0;
+}
+
 /* Read skb_ext from socket filter -- tests TC -> sk_filter path */
 SEC("socket")
 int sk_filter_meta_read(struct __sk_buff *ctx)

-- 
2.43.0

[PATCH net-next 09/15] selftests/bpf: Test passing metadata to kfree_skb tracepoint

[Jakub Sitnicki]

Tag the skb at cgroup/skb egress, send UDP to a closed loopback port so the
packet is dropped, and read the stashed metadata back from a
tp_btf/kfree_skb program.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 91 ++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  | 43 ++++++++++
 2 files changed, 134 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index b90ae0a6156d..890a881a6a1f 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -1268,6 +1268,92 @@ static void test_skb_trace__tcp_ingress(struct test_xdp_meta *skel, const char *
 	netns_free(ns);
 }
 
+/*
+ * Test skb_ext survival until skb free: cgroup/skb egress -> kfree_skb.
+ * Send UDP to a closed port. The packet is dropped, triggering kfree_skb.
+ */
+static void test_cgrp_egress_to_kfree_skb(struct test_xdp_meta *skel)
+{
+	struct sockaddr_in addr = {
+		.sin_family = AF_INET,
+		.sin_port = htons(4321),
+		.sin_addr.s_addr = htonl(INADDR_LOOPBACK),
+	};
+	struct bpf_link *cg_link = NULL;
+	struct bpf_link *tp_link = NULL;
+	struct bpf_link *put_link = NULL;
+	struct netns_obj *ns = NULL;
+	int cgroup_fd = -1;
+	char buf[1];
+	int fd = -1;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	cgroup_fd = test__join_cgroup("/cgrp_to_kfree");
+	if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+		goto cleanup;
+
+	ns = netns_new("cgrp_to_kfree", true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto cleanup;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	cg_link = bpf_program__attach_cgroup(skel->progs.cgrp_meta_write,
+					     cgroup_fd);
+	if (!ASSERT_OK_PTR(cg_link, "attach_cgroup"))
+		goto cleanup;
+
+	tp_link = bpf_program__attach_trace(skel->progs.tp_kfree_meta_read);
+	if (!ASSERT_OK_PTR(tp_link, "attach_tp"))
+		goto cleanup;
+
+	fd = socket(AF_INET, SOCK_DGRAM, 0);
+	if (!ASSERT_GE(fd, 0, "socket"))
+		goto cleanup;
+
+	ret = connect(fd, (void *)&addr, sizeof(addr));
+	if (!ASSERT_OK(ret, "connect"))
+		goto cleanup;
+
+	ret = send(fd, test_payload, TEST_PAYLOAD_LEN, 0);
+	if (!ASSERT_EQ(ret, TEST_PAYLOAD_LEN, "send"))
+		goto cleanup;
+
+	/* Wait for ICMP error -- confirms the packet was freed */
+	ret = recv(fd, buf, sizeof(buf), 0);
+	if (!ASSERT_EQ(ret, -1, "recv"))
+		goto cleanup;
+	ASSERT_EQ(errno, ECONNREFUSED, "recv_econnrefused");
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+cleanup:
+	if (fd >= 0)
+		close(fd);
+	bpf_link__destroy(tp_link);
+	bpf_link__destroy(cg_link);
+	bpf_link__destroy(put_link);
+	netns_free(ns);
+	if (cgroup_fd >= 0)
+		close(cgroup_fd);
+}
+
 void test_skb_trace_cross_hook(void)
 {
 	struct test_xdp_meta *skel = NULL;
@@ -1284,3 +1370,8 @@ void test_skb_trace_cross_hook(void)
 		test_skb_trace__tcp_ingress(skel, "tc_to_lsm", READER_LSM);
 	if (test__start_subtest("tc_to_skops"))
 		test_skb_trace__tcp_ingress(skel, "tc_to_skops", READER_SKOPS);
+	if (test__start_subtest("cgrp_egress_to_kfree_skb"))
+		test_cgrp_egress_to_kfree_skb(skel);
+
+	test_xdp_meta__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 4da50c848beb..9ff6ef4b7f3c 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -873,4 +873,47 @@ int sk_filter_meta_read(struct __sk_buff *ctx)
 	return ctx->len;
 }
 
+/* Write skb_ext from cgroup/skb egress */
+SEC("cgroup_skb/egress")
+int cgrp_meta_write(struct __sk_buff *ctx)
+{
+	if (!is_test_packet_tc(ctx))
+		return 1;
+
+	write_meta(ctx, &meta_map);
+
+	return 1;
+}
+
+bool write_once_done;
+
+/* Write skb_ext from cgroup/skb egress -- only the first matching packet */
+SEC("cgroup_skb/egress")
+int cgrp_meta_write_once(struct __sk_buff *ctx)
+{
+	if (write_once_done || !is_test_packet_tc(ctx))
+		return 1;
+
+	write_meta(ctx, &meta_map);
+	write_once_done = true;
+
+	return 1;
+}
+
+/* Read skb_ext from tp_btf/kfree_skb -- tests survival until skb free */
+SEC("tp_btf/kfree_skb")
+int BPF_PROG(tp_kfree_meta_read, struct sk_buff *skb)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &skb);
+	if (!meta_have)
+		return 0;
+	if (!check_metadata(meta_have))
+		return 0;
+
+	test_pass = true;
+	return 0;
+}
+
 char _license[] SEC("license") = "GPL";

-- 
2.43.0

[PATCH net-next 10/15] selftests/bpf: Test passing metadata to TCP egress

[Jakub Sitnicki]

Tag the skb at cgroup/skb egress and read the stashed metadata back at TC
egress: for a regular TCP transmission, and for a retransmission, where
the first transmission is dropped at TC and the write-queue original,
tagged via bpf_skb_fclone_orig, passes the metadata to the retransmitted
fast clone via the copy tracepoint.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 216 +++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  |  72 +++++++
 2 files changed, 288 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index 890a881a6a1f..ac5c0f067b3f 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -1268,6 +1268,218 @@ static void test_skb_trace__tcp_ingress(struct test_xdp_meta *skel, const char *
 	netns_free(ns);
 }
 
+/*
+ * Test metadata passing across cgroup/skb egress -> TC egress with TCP
+ * traffic.  TCP fast-clones the write-queue skb at tcp_transmit_skb();
+ * both hooks run on the transmitted clone.
+ */
+static void test_skb_trace__tcp_egress(struct test_xdp_meta *skel)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook,
+		    .ifindex = 1 /* IFINDEX_LO */,
+		    .attach_point = BPF_TC_EGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	struct bpf_link *put_link = NULL;
+	struct bpf_link *cg_link = NULL;
+	struct netns_obj *ns = NULL;
+	char buf[TEST_PAYLOAD_LEN];
+	int server_fd = -1;
+	int cgroup_fd = -1;
+	int client_fd = -1;
+	int conn_fd = -1;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	cgroup_fd = test__join_cgroup("/cgrp_to_tc");
+	if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+		goto cleanup;
+
+	ns = netns_new("cgrp_to_tc", true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto cleanup;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	cg_link = bpf_program__attach_cgroup(skel->progs.cgrp_meta_write,
+					     cgroup_fd);
+	if (!ASSERT_OK_PTR(cg_link, "attach_cgroup"))
+		goto cleanup;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup;
+
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_read_pass);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach"))
+		goto cleanup;
+
+	server_fd = start_server(AF_INET, SOCK_STREAM, "127.0.0.1", 0, 0);
+	if (!ASSERT_GE(server_fd, 0, "start_server"))
+		goto cleanup;
+
+	client_fd = connect_to_fd(server_fd, 0);
+	if (!ASSERT_GE(client_fd, 0, "connect"))
+		goto cleanup;
+
+	conn_fd = accept(server_fd, NULL, NULL);
+	if (!ASSERT_GE(conn_fd, 0, "accept"))
+		goto cleanup;
+
+	ret = send(client_fd, test_payload, TEST_PAYLOAD_LEN, 0);
+	if (!ASSERT_EQ(ret, TEST_PAYLOAD_LEN, "send"))
+		goto cleanup;
+
+	/* Receive -- synchronizes with packet delivery and free */
+	ret = recv(conn_fd, buf, sizeof(buf), 0);
+	ASSERT_EQ(ret, TEST_PAYLOAD_LEN, "recv");
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+cleanup:
+	if (conn_fd >= 0)
+		close(conn_fd);
+	if (client_fd >= 0)
+		close(client_fd);
+	if (server_fd >= 0)
+		close(server_fd);
+	bpf_link__destroy(cg_link);
+	bpf_link__destroy(put_link);
+	bpf_tc_hook_destroy(&tc_hook);
+	if (cgroup_fd >= 0)
+		close(cgroup_fd);
+	netns_free(ns);
+}
+
+/*
+ * Test metadata passing across TCP retransmission with fast-clone
+ * support: cgroup/skb egress -> TC egress.  The cgroup writer tags the
+ * transmitted fast clone and, via bpf_skb_fclone_orig, the write-queue
+ * original.  The TC drop program kills the first transmission; the
+ * retransmit is a fresh fast clone of the tagged original, so the copy
+ * tracepoint propagates the entry to it and the TC reader sees the
+ * metadata.
+ */
+static void test_skb_trace__tcp_retrans(struct test_xdp_meta *skel)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook,
+		    .ifindex = 1 /* IFINDEX_LO */,
+		    .attach_point = BPF_TC_EGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, drop_opts, .handle = 1, .priority = 1);
+	LIBBPF_OPTS(bpf_tc_opts, read_opts, .handle = 2, .priority = 2);
+	struct bpf_link *put_link = NULL;
+	struct bpf_link *copy_link = NULL;
+	struct bpf_link *cg_link = NULL;
+	struct netns_obj *ns = NULL;
+	char buf[TEST_PAYLOAD_LEN];
+	int server_fd = -1;
+	int cgroup_fd = -1;
+	int client_fd = -1;
+	int conn_fd = -1;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	copy_link = bpf_program__attach(skel->progs.skb_copy);
+	if (!ASSERT_OK_PTR(copy_link, "copy_attach"))
+		goto cleanup;
+
+	cgroup_fd = test__join_cgroup("/cgrp_to_tc_twins");
+	if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+		goto cleanup;
+
+	ns = netns_new("cgrp_to_tc_twins", true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto cleanup;
+
+	skel->bss->write_once_done = false;
+	skel->bss->drop_once_done = false;
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	cg_link = bpf_program__attach_cgroup(skel->progs.cgrp_meta_write_twins,
+					     cgroup_fd);
+	if (!ASSERT_OK_PTR(cg_link, "attach_cgroup"))
+		goto cleanup;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup;
+
+	drop_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_drop_once);
+	ret = bpf_tc_attach(&tc_hook, &drop_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach drop"))
+		goto cleanup;
+
+	read_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_read_pass);
+	ret = bpf_tc_attach(&tc_hook, &read_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach read"))
+		goto cleanup;
+
+	server_fd = start_server(AF_INET, SOCK_STREAM, "127.0.0.1", 0, 0);
+	if (!ASSERT_GE(server_fd, 0, "start_server"))
+		goto cleanup;
+
+	client_fd = connect_to_fd(server_fd, 0);
+	if (!ASSERT_GE(client_fd, 0, "connect"))
+		goto cleanup;
+
+	conn_fd = accept(server_fd, NULL, NULL);
+	if (!ASSERT_GE(conn_fd, 0, "accept"))
+		goto cleanup;
+
+	ret = send(client_fd, test_payload, TEST_PAYLOAD_LEN, 0);
+	if (!ASSERT_EQ(ret, TEST_PAYLOAD_LEN, "send"))
+		goto cleanup;
+
+	/* First transmission is dropped at TC; recv unblocks on retransmit */
+	ret = recv(conn_fd, buf, sizeof(buf), 0);
+	ASSERT_EQ(ret, TEST_PAYLOAD_LEN, "recv");
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+cleanup:
+	if (conn_fd >= 0)
+		close(conn_fd);
+	if (client_fd >= 0)
+		close(client_fd);
+	if (server_fd >= 0)
+		close(server_fd);
+	bpf_link__destroy(cg_link);
+	bpf_link__destroy(copy_link);
+	bpf_link__destroy(put_link);
+	bpf_tc_hook_destroy(&tc_hook);
+	if (cgroup_fd >= 0)
+		close(cgroup_fd);
+	netns_free(ns);
+}
+
 /*
  * Test skb_ext survival until skb free: cgroup/skb egress -> kfree_skb.
  * Send UDP to a closed port. The packet is dropped, triggering kfree_skb.
@@ -1372,6 +1584,10 @@ void test_skb_trace_cross_hook(void)
 		test_skb_trace__tcp_ingress(skel, "tc_to_skops", READER_SKOPS);
 	if (test__start_subtest("cgrp_egress_to_kfree_skb"))
 		test_cgrp_egress_to_kfree_skb(skel);
+	if (test__start_subtest("cgrp_egress_to_tc_egress"))
+		test_skb_trace__tcp_egress(skel);
+	if (test__start_subtest("cgrp_egress_to_tc_egress_fclone"))
+		test_skb_trace__tcp_retrans(skel);
 
 	test_xdp_meta__destroy(skel);
 }
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 9ff6ef4b7f3c..50aa045bcc22 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -736,6 +736,44 @@ int tc_meta_read(struct __sk_buff *ctx)
 	return TC_ACT_UNSPEC;
 }
 
+/*
+ * Read skb_ext at TC, never drop -- for cross-hook tests where untagged
+ * packets (e.g. TCP handshake) must pass through the same hook.
+ */
+SEC("tc")
+int tc_meta_read_pass(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		return TC_ACT_UNSPEC;
+	if (!check_metadata(meta_have))
+		return TC_ACT_UNSPEC;
+
+	test_pass = true;
+	return TC_ACT_UNSPEC;
+}
+
+bool drop_once_done;
+
+/* Drop the first packet carrying an skb_ext entry -- force TCP retransmit */
+SEC("tc")
+int tc_meta_drop_once(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	if (drop_once_done)
+		return TC_ACT_UNSPEC;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		return TC_ACT_UNSPEC;
+
+	drop_once_done = true;
+	return TC_ACT_SHOT;
+}
+
 int free_count;
 
 SEC("raw_tp/skb_free")
@@ -900,6 +938,40 @@ int cgrp_meta_write_once(struct __sk_buff *ctx)
 	return 1;
 }
 
+/*
+ * Write skb_ext from cgroup/skb egress on both fast-clone twins: the
+ * transmitted packet is a fast clone of the write-queue original; tag
+ * both so the retransmit (a fresh clone of the tagged original) inherits
+ * the entry via the skb_copy tracepoint.
+ */
+SEC("cgroup_skb/egress")
+int cgrp_meta_write_twins(struct __sk_buff *ctx)
+{
+	struct sk_buff *twin;
+	__u8 *meta_have;
+
+	if (write_once_done || !is_test_packet_tc(ctx))
+		return 1;
+
+	write_meta(ctx, &meta_map);
+	write_once_done = true;
+
+	twin = bpf_skb_fclone_orig(bpf_cast_to_kern_ctx(ctx));
+	if (!twin)
+		return 1;
+
+	if (bpf_map_update_elem(&meta_map, &twin, &meta_want, BPF_NOEXIST))
+		return 1;
+	bpf_trace_skb(twin, 0);
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &twin);
+	if (!meta_have)
+		return 1;
+	__builtin_memcpy(meta_have, meta_want, META_SIZE);
+
+	return 1;
+}
+
 /* Read skb_ext from tp_btf/kfree_skb -- tests survival until skb free */
 SEC("tp_btf/kfree_skb")
 int BPF_PROG(tp_kfree_meta_read, struct sk_buff *skb)

-- 
2.43.0

[PATCH net-next 11/15] selftests/bpf: Test passing metadata to netfilter hook

[Jakub Sitnicki]

Add a TC -> netfilter cross-hook test for the metadata stash. A TC ingress
program tags the skb on loopback, and a netfilter program at
NF_INET_LOCAL_IN reads the stashed metadata.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 74 ++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  | 21 ++++++
 2 files changed, 95 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index ac5c0f067b3f..eb0d36e7a45d 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -2,6 +2,7 @@
 #include <test_progs.h>
 #include <network_helpers.h>
 #include <linux/ipv6.h>
+#include <linux/netfilter.h>
 #include <arpa/inet.h>
 #include "test_xdp_context_test_run.skel.h"
 #include "test_xdp_meta.skel.h"
@@ -1566,6 +1567,77 @@ static void test_cgrp_egress_to_kfree_skb(struct test_xdp_meta *skel)
 		close(cgroup_fd);
 }
 
+/* Test metadata passing across TC ingress -> netfilter hook */
+static void test_skb_trace__nf(struct test_xdp_meta *skel, const char *name)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook,
+		    .ifindex = 1 /* IFINDEX_LO */,
+		    .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	LIBBPF_OPTS(bpf_netfilter_opts, nf_opts,
+		    .pf = NFPROTO_IPV4,
+		    .hooknum = NF_INET_LOCAL_IN,
+		    .priority = 1);
+	struct bpf_link *reader_link = NULL;
+	struct bpf_link *put_link = NULL;
+	struct netns_obj *ns = NULL;
+	int server_fd = -1;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	ns = netns_new(name, true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto cleanup;
+
+	server_fd = start_server(AF_INET, SOCK_DGRAM, "127.0.0.1", 0, 0);
+	if (!ASSERT_GE(server_fd, 0, "start_server"))
+		goto cleanup;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup;
+
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach"))
+		goto cleanup;
+
+	reader_link = bpf_program__attach_netfilter(skel->progs.nf_meta_read,
+						    &nf_opts);
+	if (!ASSERT_OK_PTR(reader_link, "attach_nf"))
+		goto cleanup;
+
+	if (send_loopback_udp(server_fd))
+		goto cleanup;
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+cleanup:
+	bpf_link__destroy(reader_link);
+	bpf_link__destroy(put_link);
+	bpf_tc_hook_destroy(&tc_hook);
+	if (server_fd >= 0)
+		close(server_fd);
+	netns_free(ns);
+}
+
 void test_skb_trace_cross_hook(void)
 {
 	struct test_xdp_meta *skel = NULL;
@@ -1588,6 +1660,8 @@ void test_skb_trace_cross_hook(void)
 		test_skb_trace__tcp_egress(skel);
 	if (test__start_subtest("cgrp_egress_to_tc_egress_fclone"))
 		test_skb_trace__tcp_retrans(skel);
+	if (test__start_subtest("tc_to_nf"))
+		test_skb_trace__nf(skel, "tc_to_nf");
 
 	test_xdp_meta__destroy(skel);
 }
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 50aa045bcc22..d85b60c24f73 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -988,4 +988,25 @@ int BPF_PROG(tp_kfree_meta_read, struct sk_buff *skb)
 	return 0;
 }
 
+#define NF_ACCEPT 1
+
+SEC("netfilter")
+int nf_meta_read(struct bpf_nf_ctx *ctx)
+{
+	struct __sk_buff *skb = (struct __sk_buff *)ctx->skb;
+	__u8 *meta_have;
+
+	if (!skb)
+		return NF_ACCEPT;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &skb);
+	if (!meta_have)
+		return NF_ACCEPT;
+	if (!check_metadata(meta_have))
+		return NF_ACCEPT;
+
+	test_pass = true;
+	return NF_ACCEPT;
+}
+
 char _license[] SEC("license") = "GPL";

-- 
2.43.0

[PATCH net-next 12/15] selftests/bpf: Test passing metadata to/from LWT hooks

[Jakub Sitnicki]

Add cross-hook tests for the metadata stash covering all generic LWT
program types. For lwt_in, a TC ingress program tags the skb on loopback
and the LWT program reads the stashed metadata. For lwt_out and lwt_xmit,
the LWT programs tag the skb on the output path and a TC ingress program
reads the metadata back after loopback.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 121 +++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  |  35 ++++++
 2 files changed, 156 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index eb0d36e7a45d..aea7a1c8a755 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -1638,6 +1638,118 @@ static void test_skb_trace__nf(struct test_xdp_meta *skel, const char *name)
 	netns_free(ns);
 }
 
+#define LWT_EXT_PIN_PATH "/sys/fs/bpf/skb_trace_lwt"
+
+/*
+ * Test metadata passing across LWT hooks on loopback.
+ *
+ * @lwt_prog:  BPF program to pin and attach via ip route encap
+ * @encap_dir: "in", "out", or "xmit"
+ * @writer:    true if lwt_prog tags the skb (reader on TC ingress),
+ *             false if lwt_prog reads the metadata (writer on TC ingress)
+ */
+static void test_skb_trace__lwt(struct test_xdp_meta *skel, const char *name,
+				struct bpf_program *lwt_prog,
+				const char *encap_dir, bool writer)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook,
+		    .ifindex = 1 /* IFINDEX_LO */,
+		    .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	struct sockaddr_in addr = {
+		.sin_family = AF_INET,
+	};
+	struct bpf_program *tc_prog;
+	struct bpf_link *put_link = NULL;
+	char buf[TEST_PAYLOAD_LEN];
+	struct netns_obj *ns = NULL;
+	bool pinned = false;
+	int server_fd = -1;
+	int fd = -1;
+	int port;
+	int ret;
+	int i;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		return;
+
+	unlink(LWT_EXT_PIN_PATH);
+	ret = bpf_program__pin(lwt_prog, LWT_EXT_PIN_PATH);
+	if (!ASSERT_OK(ret, "pin lwt"))
+		goto cleanup;
+	pinned = true;
+
+	ns = netns_new(name, true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		goto cleanup_prehook;
+
+	SYS(cleanup, "ip addr add 10.0.0.1/32 dev lo");
+	SYS(cleanup, "ip route replace table local local 10.0.0.1 "
+		     "encap bpf %s pinned " LWT_EXT_PIN_PATH " dev lo",
+		     encap_dir);
+
+	server_fd = start_server(AF_INET, SOCK_DGRAM, "10.0.0.1", 0, 0);
+	if (!ASSERT_GE(server_fd, 0, "start_server"))
+		goto cleanup_prehook;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup_prehook;
+
+	/* When LWT writes, TC ingress reads; when LWT reads, TC ingress writes */
+	tc_prog = writer ? skel->progs.tc_meta_read
+			 : skel->progs.tc_meta_write;
+	tc_opts.prog_fd = bpf_program__fd(tc_prog);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach"))
+		goto cleanup;
+
+	port = get_socket_local_port(server_fd);
+	if (!ASSERT_GE(port, 0, "get_port"))
+		goto cleanup;
+
+	fd = socket(AF_INET, SOCK_DGRAM, 0);
+	if (!ASSERT_GE(fd, 0, "socket"))
+		goto cleanup;
+
+	inet_pton(AF_INET, "10.0.0.1", &addr.sin_addr);
+	addr.sin_port = port;
+	ret = sendto(fd, test_payload, TEST_PAYLOAD_LEN, 0,
+		     (void *)&addr, sizeof(addr));
+	if (!ASSERT_EQ(ret, TEST_PAYLOAD_LEN, "sendto"))
+		goto cleanup;
+	ret = recvfrom(server_fd, buf, sizeof(buf), 0, NULL, NULL);
+	ASSERT_EQ(ret, TEST_PAYLOAD_LEN, "recvfrom");
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+cleanup:
+	if (fd >= 0)
+		close(fd);
+	bpf_tc_hook_destroy(&tc_hook);
+cleanup_prehook:
+	bpf_link__destroy(put_link);
+	if (server_fd >= 0)
+		close(server_fd);
+	netns_free(ns);
+	if (pinned)
+		unlink(LWT_EXT_PIN_PATH);
+}
+
 void test_skb_trace_cross_hook(void)
 {
 	struct test_xdp_meta *skel = NULL;
@@ -1662,6 +1774,15 @@ void test_skb_trace_cross_hook(void)
 		test_skb_trace__tcp_retrans(skel);
 	if (test__start_subtest("tc_to_nf"))
 		test_skb_trace__nf(skel, "tc_to_nf");
+	if (test__start_subtest("tc_to_lwt_in"))
+		test_skb_trace__lwt(skel, "tc_to_lwt_in",
+				    skel->progs.lwt_in_meta_read, "in", false);
+	if (test__start_subtest("lwt_out_to_tc"))
+		test_skb_trace__lwt(skel, "lwt_out_to_tc",
+				    skel->progs.lwt_out_meta_write, "out", true);
+	if (test__start_subtest("lwt_xmit_to_tc"))
+		test_skb_trace__lwt(skel, "lwt_xmit_to_tc",
+				    skel->progs.lwt_xmit_meta_write, "xmit", true);
 
 	test_xdp_meta__destroy(skel);
 }
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index d85b60c24f73..adad5433fc19 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -1009,4 +1009,39 @@ int nf_meta_read(struct bpf_nf_ctx *ctx)
 	return NF_ACCEPT;
 }
 
+SEC("lwt_in")
+int lwt_in_meta_read(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		return BPF_OK;
+	if (!check_metadata(meta_have))
+		return BPF_OK;
+
+	test_pass = true;
+	return BPF_OK;
+}
+
+SEC("lwt_out")
+int lwt_out_meta_write(struct __sk_buff *ctx)
+{
+	if (!is_test_packet_tc(ctx))
+		return BPF_OK;
+	write_meta(ctx, &meta_map);
+
+	return BPF_OK;
+}
+
+SEC("lwt_xmit")
+int lwt_xmit_meta_write(struct __sk_buff *ctx)
+{
+	if (!is_test_packet_tc(ctx))
+		return BPF_OK;
+	write_meta(ctx, &meta_map);
+
+	return BPF_OK;
+}
+
 char _license[] SEC("license") = "GPL";

-- 
2.43.0

[PATCH net-next 13/15] selftests/bpf: Test passing metadata to seg6local End.BPF hook

[Jakub Sitnicki]

Test accessing the metadata stash from lwt_seg6local context:
- TC tags the encapped packet leaving NS1,
- NS2 runs End.BPF for fd01::1 which reads the stashed metadata back,
- NS3 receives the packet and decapsulates it.

Note that End.BPF cannot deliver locally since seg6_lookup_nexthop rejects
loopback routes, hence a three netns setup is necessary.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../bpf/prog_tests/xdp_context_test_run.c          | 189 +++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  |  16 ++
 2 files changed, 205 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index aea7a1c8a755..ae89eccffd2d 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -1750,6 +1750,193 @@ static void test_skb_trace__lwt(struct test_xdp_meta *skel, const char *name,
 		unlink(LWT_EXT_PIN_PATH);
 }
 
+#define SEG6_PIN_PATH "/sys/fs/bpf/skb_trace_seg6local"
+
+/*
+ * Test metadata passing across TC egress -> seg6local End.BPF hook.
+ *
+ * Topology:
+ *
+ *     NS1           NS2            NS3
+ *   lo veth1 <-> veth2 veth3 <-> veth4 lo
+ *
+ * NS1 encaps fb01::2 with SRH (segments fd01::1,fd01::2) on output and
+ * tags the skb at TC egress on veth1. NS2 runs End.BPF for fd01::1,
+ * which reads the stashed metadata, then forwards to NS3
+ * (seg6_lookup_nexthop rejects loopback routes with
+ * local_delivery=false, so End.BPF cannot deliver locally). NS3 has
+ * fd01::2 local, decapsulates, and delivers the inner UDP (dst=fb01::2)
+ * to a local socket for test synchronization.
+ *
+ * The TC program drops non-test packets, so NDP is bypassed with a
+ * static neighbor entry on the NS1->NS2 hop.
+ */
+static void test_skb_trace__seg6local(struct test_xdp_meta *skel)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook, .attach_point = BPF_TC_EGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	struct bpf_link *put_link = NULL;
+	struct nstoken *nstoken = NULL;
+	struct sockaddr_in6 dst = {};
+	char buf[TEST_PAYLOAD_LEN];
+	bool pinned = false;
+	int client_fd = -1;
+	int server_fd = -1;
+	ssize_t bytes;
+	int ret;
+	int i;
+
+	unlink(SEG6_PIN_PATH);
+	ret = bpf_program__pin(skel->progs.seg6local_meta_read,
+			       SEG6_PIN_PATH);
+	if (!ASSERT_OK(ret, "pin seg6local"))
+		return;
+	pinned = true;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		goto cleanup;
+
+	SYS(cleanup, "ip netns add seg6_1");
+	SYS(cleanup, "ip netns add seg6_2");
+	SYS(cleanup, "ip netns add seg6_3");
+
+	/* NS1 <-> NS2 veth pair; static NDP since TC drops NS/NA */
+	SYS(cleanup, "ip -n seg6_1 link add veth1 type veth peer name veth2 netns seg6_2");
+	SYS(cleanup, "ip -n seg6_1 link set dev veth1 up");
+	SYS(cleanup, "ip -n seg6_2 link set dev veth2 address 02:00:00:00:00:02");
+	SYS(cleanup, "ip -n seg6_2 link set dev veth2 up");
+	SYS(cleanup, "ip -n seg6_1 -6 addr add fb00::12/16 dev veth1 scope link nodad");
+	SYS(cleanup, "ip -n seg6_2 -6 addr add fb00::21/16 dev veth2 scope link nodad");
+	SYS(cleanup, "ip -n seg6_1 -6 neigh add fb00::21 lladdr 02:00:00:00:00:02 "
+		     "nud permanent dev veth1");
+
+	/* NS2 <-> NS3 veth pair */
+	SYS(cleanup, "ip -n seg6_2 link add veth3 type veth peer name veth4 netns seg6_3");
+	SYS(cleanup, "ip -n seg6_2 link set dev veth3 up");
+	SYS(cleanup, "ip -n seg6_3 link set dev veth4 up");
+	SYS(cleanup, "ip -n seg6_2 -6 addr add fb00::34/16 dev veth3 scope link nodad");
+	SYS(cleanup, "ip -n seg6_3 -6 addr add fb00::43/16 dev veth4 scope link nodad");
+
+	/* NS1: source address + SRH encap on output + route for fd01::/16 */
+	SYS(cleanup, "ip -n seg6_1 link set dev lo up");
+	SYS(cleanup, "ip -n seg6_1 -6 addr add fb01::1/128 dev lo nodad");
+	SYS(cleanup, "ip -n seg6_1 -6 route add fb01::2 encap seg6 mode encap "
+		     "segs fd01::1,fd01::2 dev veth1 via fb00::21");
+	SYS(cleanup, "ip -n seg6_1 -6 route add fd01::/16 dev veth1 via fb00::21");
+
+	/* NS2: seg6local End.BPF for fd01::1, then forward to NS3.
+	 * Use open_netns so the pinned prog on bpffs is accessible.
+	 */
+	nstoken = open_netns("seg6_2");
+	if (!ASSERT_OK_PTR(nstoken, "open seg6_2"))
+		goto cleanup;
+
+	SYS(cleanup, "ip link set dev lo up");
+	SYS(cleanup, "sysctl -wq net.ipv6.conf.all.forwarding=1");
+	SYS(cleanup, "ip -6 route add fd01::2/128 dev veth3 via fb00::43");
+	SYS(cleanup, "ip -6 route add fd01::1 encap seg6local "
+		     "action End.BPF endpoint pinned " SEG6_PIN_PATH " dev veth2");
+
+	close_netns(nstoken);
+	nstoken = NULL;
+
+	/* NS3: decap point + UDP server.
+	 * fd01::2 local triggers SRH decap (segments_left=0).
+	 * fb01::2 local delivers the inner UDP to the socket.
+	 */
+	nstoken = open_netns("seg6_3");
+	if (!ASSERT_OK_PTR(nstoken, "open seg6_3"))
+		goto cleanup;
+
+	SYS(cleanup, "ip link set dev lo up");
+	SYS(cleanup, "sysctl -wq net.ipv6.conf.all.seg6_enabled=1");
+	SYS(cleanup, "sysctl -wq net.ipv6.conf.lo.seg6_enabled=1");
+	SYS(cleanup, "sysctl -wq net.ipv6.conf.veth4.seg6_enabled=1");
+	SYS(cleanup, "ip -6 addr add fd01::2/128 dev lo nodad");
+	SYS(cleanup, "ip -6 addr add fb01::2/128 dev lo nodad");
+
+	server_fd = start_server(AF_INET6, SOCK_DGRAM, "fb01::2", 1234, 0);
+	if (!ASSERT_GE(server_fd, 0, "start_server"))
+		goto cleanup;
+
+	close_netns(nstoken);
+	nstoken = NULL;
+
+	/* NS1: write skb_ext at TC egress on veth1, then send.
+	 * TC egress fires after seg6 encap, on the encapped packet.
+	 */
+	nstoken = open_netns("seg6_1");
+	if (!ASSERT_OK_PTR(nstoken, "open seg6_1"))
+		goto cleanup;
+
+	tc_hook.ifindex = if_nametoindex("veth1");
+	if (!ASSERT_GT(tc_hook.ifindex, 0, "ifindex veth1"))
+		goto cleanup;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup;
+
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach"))
+		goto cleanup;
+
+	client_fd = socket(AF_INET6, SOCK_DGRAM, 0);
+	if (!ASSERT_GE(client_fd, 0, "socket"))
+		goto cleanup;
+
+	dst.sin6_family = AF_INET6;
+	dst.sin6_port = htons(1234);
+	inet_pton(AF_INET6, "fb01::2", &dst.sin6_addr);
+
+	bytes = sendto(client_fd, test_payload, TEST_PAYLOAD_LEN, 0,
+		       (void *)&dst, sizeof(dst));
+
+	close_netns(nstoken);
+	nstoken = NULL;
+
+	if (!ASSERT_EQ(bytes, TEST_PAYLOAD_LEN, "sendto"))
+		goto cleanup;
+
+	/* Receive decapsulated packet -- synchronizes with seg6local */
+	nstoken = open_netns("seg6_3");
+	if (!ASSERT_OK_PTR(nstoken, "open seg6_3"))
+		goto cleanup;
+
+	bytes = recvfrom(server_fd, buf, sizeof(buf), 0, NULL, NULL);
+	ASSERT_EQ(bytes, TEST_PAYLOAD_LEN, "recvfrom");
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+cleanup:
+	close_netns(nstoken);
+	if (server_fd >= 0)
+		close(server_fd);
+	if (client_fd >= 0)
+		close(client_fd);
+	SYS_NOFAIL("ip netns del seg6_3");
+	SYS_NOFAIL("ip netns del seg6_2");
+	SYS_NOFAIL("ip netns del seg6_1");
+	if (pinned)
+		unlink(SEG6_PIN_PATH);
+	bpf_link__destroy(put_link);
+}
+
 void test_skb_trace_cross_hook(void)
 {
 	struct test_xdp_meta *skel = NULL;
@@ -1783,6 +1970,8 @@ void test_skb_trace_cross_hook(void)
 	if (test__start_subtest("lwt_xmit_to_tc"))
 		test_skb_trace__lwt(skel, "lwt_xmit_to_tc",
 				    skel->progs.lwt_xmit_meta_write, "xmit", true);
+	if (test__start_subtest("tc_to_seg6local"))
+		test_skb_trace__seg6local(skel);
 
 	test_xdp_meta__destroy(skel);
 }
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index adad5433fc19..3e323bb7eb96 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -1044,4 +1044,20 @@ int lwt_xmit_meta_write(struct __sk_buff *ctx)
 	return BPF_OK;
 }
 
+/* Read stashed metadata from seg6local End.BPF */
+SEC("lwt_seg6local")
+int seg6local_meta_read(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		return BPF_OK;
+	if (!check_metadata(meta_have))
+		return BPF_OK;
+
+	test_pass = true;
+	return BPF_OK;
+}
+
 char _license[] SEC("license") = "GPL";

-- 
2.43.0

[PATCH net-next 14/15] selftests/bpf: Test passing metadata to sk_skb stream verdict hook

[Jakub Sitnicki]

Tag the packet with bpf_trace_skb() from TC on loopback and stash metadata
in a BPF map keyed by the skb address; the verdict program reads the entry
on skb delivery to a socket in the sockmap.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 .../selftests/bpf/prog_tests/socket_helpers.h      |  1 +
 .../bpf/prog_tests/xdp_context_test_run.c          | 99 ++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  | 16 ++++
 3 files changed, 116 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/socket_helpers.h b/tools/testing/selftests/bpf/prog_tests/socket_helpers.h
index 0d59503a0c73..eb114a6ea6f9 100644
--- a/tools/testing/selftests/bpf/prog_tests/socket_helpers.h
+++ b/tools/testing/selftests/bpf/prog_tests/socket_helpers.h
@@ -3,6 +3,7 @@
 #ifndef __SOCKET_HELPERS__
 #define __SOCKET_HELPERS__
 
+#include <error.h>
 #include <sys/un.h>
 #include <linux/vm_sockets.h>
 
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index ae89eccffd2d..0aafdb1b8feb 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -4,6 +4,7 @@
 #include <linux/ipv6.h>
 #include <linux/netfilter.h>
 #include <arpa/inet.h>
+#include "socket_helpers.h"
 #include "test_xdp_context_test_run.skel.h"
 #include "test_xdp_meta.skel.h"
 
@@ -1937,6 +1938,102 @@ static void test_skb_trace__seg6local(struct test_xdp_meta *skel)
 	bpf_link__destroy(put_link);
 }
 
+/*
+ * Test metadata passing across TC ingress -> sk_skb verdict hook.  TC ingress
+ * tags the skb on loopback; the verdict program reads the metadata stashed
+ * for it on skb delivery to a socket in the sockmap.
+ */
+static void test_skb_trace__sk_skb(struct test_xdp_meta *skel)
+{
+	LIBBPF_OPTS(bpf_tc_hook, tc_hook,
+		    .ifindex = 1 /* IFINDEX_LO */,
+		    .attach_point = BPF_TC_INGRESS);
+	LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1);
+	struct bpf_link *verdict_link = NULL;
+	struct bpf_link *put_link = NULL;
+	struct sockaddr_in addr;
+	socklen_t addr_len = sizeof(addr);
+	struct netns_obj *ns = NULL;
+	char buf[TEST_PAYLOAD_LEN];
+	int c1 = -1, p1 = -1;
+	int map = -1, zero = 0;
+	ssize_t n;
+	int ret;
+	int i;
+
+	ns = netns_new("tc_to_sk_skb", true);
+	if (!ASSERT_OK_PTR(ns, "netns_new"))
+		return;
+
+	skel->bss->test_pass = false;
+	skel->bss->free_count = 0;
+
+	put_link = bpf_program__attach(skel->progs.skb_free);
+	if (!ASSERT_OK_PTR(put_link, "put_attach"))
+		goto cleanup;
+
+	ret = bpf_tc_hook_create(&tc_hook);
+	if (!ASSERT_OK(ret, "bpf_tc_hook_create"))
+		goto cleanup;
+
+	tc_opts.prog_fd = bpf_program__fd(skel->progs.tc_meta_write_port);
+	ret = bpf_tc_attach(&tc_hook, &tc_opts);
+	if (!ASSERT_OK(ret, "bpf_tc_attach"))
+		goto cleanup;
+
+	map = bpf_map_create(BPF_MAP_TYPE_SOCKMAP, NULL, sizeof(int),
+			     sizeof(int), 1, NULL);
+	if (!ASSERT_GE(map, 0, "bpf_map_create"))
+		goto cleanup;
+
+	verdict_link = bpf_program__attach_sockmap(skel->progs.sk_skb_meta_read,
+						   map);
+	if (!ASSERT_OK_PTR(verdict_link, "attach_sockmap"))
+		goto cleanup;
+
+	if (!ASSERT_OK(create_pair(AF_INET, SOCK_STREAM, &c1, &p1), "create_pair"))
+		goto cleanup;
+
+	if (!ASSERT_OK(bpf_map_update_elem(map, &zero, &c1, BPF_NOEXIST), "map_update"))
+		goto cleanup;
+
+	if (!ASSERT_OK(getsockname(c1, (struct sockaddr *)&addr, &addr_len),
+		       "getsockname"))
+		goto cleanup;
+	skel->bss->target_port = addr.sin_port;
+
+	n = send(p1, test_payload, TEST_PAYLOAD_LEN, 0);
+	if (!ASSERT_EQ(n, TEST_PAYLOAD_LEN, "send"))
+		goto cleanup;
+
+	n = recv_timeout(c1, buf, sizeof(buf), MSG_DONTWAIT, IO_TIMEOUT_SEC);
+	ASSERT_EQ(n, TEST_PAYLOAD_LEN, "recv");
+
+	/* Poll every 10 msec up to 1 sec */
+	for (i = 0; i < 100 && !skel->bss->test_pass; i++)
+		usleep(10000);
+	ASSERT_TRUE(skel->bss->test_pass, "test_pass");
+
+	/* Skb is freed after RX delivery; wait for free tp */
+	for (i = 0; i < 100 && !skel->bss->free_count; i++)
+		usleep(10000);
+	ASSERT_GE(skel->bss->free_count, 1, "free_count");
+
+	assert_meta_stash_empty(skel->maps.meta_map);
+
+cleanup:
+	if (c1 >= 0)
+		close(c1);
+	if (p1 >= 0)
+		close(p1);
+	bpf_link__destroy(verdict_link);
+	bpf_link__destroy(put_link);
+	if (map >= 0)
+		close(map);
+	bpf_tc_hook_destroy(&tc_hook);
+	netns_free(ns);
+}
+
 void test_skb_trace_cross_hook(void)
 {
 	struct test_xdp_meta *skel = NULL;
@@ -1972,6 +2069,8 @@ void test_skb_trace_cross_hook(void)
 				    skel->progs.lwt_xmit_meta_write, "xmit", true);
 	if (test__start_subtest("tc_to_seg6local"))
 		test_skb_trace__seg6local(skel);
+	if (test__start_subtest("tc_to_sk_skb"))
+		test_skb_trace__sk_skb(skel);
 
 	test_xdp_meta__destroy(skel);
 }
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 3e323bb7eb96..913caff5a35c 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -1060,4 +1060,20 @@ int seg6local_meta_read(struct __sk_buff *ctx)
 	return BPF_OK;
 }
 
+/* Read stashed metadata from sk_skb stream verdict hook */
+SEC("sk_skb/stream_verdict")
+int sk_skb_meta_read(struct __sk_buff *ctx)
+{
+	__u8 *meta_have;
+
+	meta_have = bpf_map_lookup_elem(&meta_map, &ctx);
+	if (!meta_have)
+		return SK_PASS;
+	if (!check_metadata(meta_have))
+		return SK_PASS;
+
+	test_pass = true;
+	return SK_PASS;
+}
+
 char _license[] SEC("license") = "GPL";

-- 
2.43.0

[PATCH net-next 15/15] selftests/bpf: Use non-trivial test payload in xdp_context tests

[Jakub Sitnicki]

Replace the sequential test payload pattern (0x01..0x38) with
higher-entropy values that don't alias common protocol header fields
such as IP version/IHL or ethertype LSBs. This reduces the chance of
false positives from stray packets.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
 tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c | 8 ++++----
 tools/testing/selftests/bpf/progs/test_xdp_meta.c             | 8 ++++----
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
index 0aafdb1b8feb..e40c9bebdbb3 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_context_test_run.c
@@ -22,10 +22,10 @@
 
 #define TEST_PAYLOAD_LEN 32
 static const __u8 test_payload[TEST_PAYLOAD_LEN] = {
-	0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
-	0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
-	0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28,
-	0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38,
+	0x49, 0x20, 0x63, 0x72, 0x6f, 0x73, 0x73, 0x65,
+	0x64, 0x20, 0x68, 0x6f, 0x6f, 0x6b, 0x73, 0x20,
+	0x26, 0x20, 0x6c, 0x69, 0x76, 0x65, 0x64, 0x20,
+	0x74, 0x6f, 0x20, 0x74, 0x65, 0x6c, 0x6c, 0x21,
 };
 
 void test_xdp_context_error(int prog_fd, struct bpf_test_run_opts opts,
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_meta.c b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
index 913caff5a35c..55ad17fdbec3 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_meta.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_meta.c
@@ -24,10 +24,10 @@
 bool test_pass;
 
 static const __u8 meta_want[META_SIZE] = {
-	0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
-	0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
-	0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28,
-	0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38,
+	0x49, 0x20, 0x63, 0x72, 0x6f, 0x73, 0x73, 0x65,
+	0x64, 0x20, 0x68, 0x6f, 0x6f, 0x6b, 0x73, 0x20,
+	0x26, 0x20, 0x6c, 0x69, 0x76, 0x65, 0x64, 0x20,
+	0x74, 0x6f, 0x20, 0x74, 0x65, 0x6c, 0x6c, 0x21,
 };
 
 static bool check_metadata(const char *file, int line, __u8 *meta_have)

-- 
2.43.0

Re: [PATCH net-next 00/15] Gated tracepoints for skb lifecycle (free+clone)

[Jakub Kicinski]

On Mon, 28 Sep 2026 13:40:17 +0200 Jakub Sitnicki wrote:
> Subject: [PATCH net-next 00/15] Gated tracepoints for skb lifecycle (free+clone)

is this expected not to build on top of net-next?

Re: [PATCH net-next 00/15] Gated tracepoints for skb lifecycle (free+clone)

[Jakub Sitnicki]

On Thu, Oct 01, 2026 at 06:45 PM -07, Jakub Kicinski wrote:
> On Mon, 28 Sep 2026 13:40:17 +0200 Jakub Sitnicki wrote:
>> Subject: [PATCH net-next 00/15] Gated tracepoints for skb lifecycle (free+clone)
>
> is this expected not to build on top of net-next?

No, it should build cleanly. Base was 42a9fb3382fc ("Merge
git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net") [1].

But you can ignore v1. I will be rebasing & respinning it to address
your feedback [2]. Likely during Plumbers.

Cover letter also needs an update since benchmarks got fixed [3].

[1] https://github.com/jsitnicki/linux/commits/sent/20260924-bpf-meta-gated-tracepoints-4a731fd0ef64-v1
[2] https://lore.kernel.org/all/20260925132004.171b751e@kernel.org/
[3] http://msgid.link/87ik3kl1lr.fsf@cloudflare.com