[PATCH v3 1/3] bpf: propagate cb_access from freplace programs

[PATCH v3 1/3] bpf: propagate cb_access from freplace programs

[Weiming Shi]

bpf_prog_run_save_cb() and bpf_prog_run_clear_cb() use the target
program's cb_access flag to decide whether skb->cb must be saved or
cleared.  An extension program can introduce ctx->cb[] access behind a
target which does not access it itself, leaving protocol-owned control
block contents visible to the replacement and preventing the wrapper
from restoring them.

Make cb_access independently addressable and propagate it to the target
before activating a replacement.  Wait for wrappers which observed the
old value, and snapshot the flag once per invocation so save and restore
decisions remain paired.

Cc: stable@vger.kernel.org
Fixes: be8704ff07d2 ("bpf: Introduce dynamic program extensions")
Suggested-by: Daniel Borkmann <daniel@iogearbox.net>
Assisted-by: LLM
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
---
 include/linux/bpf.h    | 2 +-
 include/linux/filter.h | 7 ++++---
 kernel/bpf/syscall.c   | 6 ++++++
 3 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e57af902560c3..606e7cf397af2 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1864,11 +1864,11 @@ struct bpf_prog_aux {
 
 struct bpf_prog {
 	u16			pages;		/* Number of allocated pages */
+	bool			cb_access;	/* Is control block accessed? */
 	u32			jited:1,	/* Is our filter JIT'ed? */
 				jit_requested:1,/* archs need to JIT the prog */
 				jit_required:1,	/* program strictly requires JIT compiler */
 				gpl_compatible:1, /* Is filter GPL compatible? */
-				cb_access:1,	/* Is control block accessed? */
 				dst_needed:1,	/* Do we need dst entry? */
 				blinding_requested:1, /* needs constant blinding */
 				blinded:1,	/* Was blinded */
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 39decde7fc730..788c2d625db4a 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1047,16 +1047,17 @@ static inline u32 __bpf_prog_run_save_cb(const struct bpf_prog *prog,
 	const struct sk_buff *skb = ctx;
 	u8 *cb_data = bpf_skb_cb(skb);
 	u8 cb_saved[BPF_SKB_CB_LEN];
+	bool cb_access = READ_ONCE(prog->cb_access);
 	u32 res;
 
-	if (unlikely(prog->cb_access)) {
+	if (unlikely(cb_access)) {
 		memcpy(cb_saved, cb_data, sizeof(cb_saved));
 		memset(cb_data, 0, sizeof(cb_saved));
 	}
 
 	res = bpf_prog_run(prog, skb);
 
-	if (unlikely(prog->cb_access))
+	if (unlikely(cb_access))
 		memcpy(cb_data, cb_saved, sizeof(cb_saved));
 
 	return res;
@@ -1079,7 +1080,7 @@ static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
 	u8 *cb_data = bpf_skb_cb(skb);
 	u32 res;
 
-	if (unlikely(prog->cb_access))
+	if (unlikely(READ_ONCE(prog->cb_access)))
 		memset(cb_data, 0, BPF_SKB_CB_LEN);
 
 	res = bpf_prog_run_pin_on_cpu(prog, skb);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index c7bc9ba9b331f..43a29e47c8d01 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3812,6 +3812,12 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog,
 	if (err)
 		goto out_unlock;
 
+	if (prog->type == BPF_PROG_TYPE_EXT && READ_ONCE(prog->cb_access)) {
+		WRITE_ONCE(tgt_prog->cb_access, true);
+		/* Drain runs that observed cb_access=false before enabling freplace. */
+		synchronize_rcu();
+	}
+
 	err = bpf_trampoline_link_prog(&link->link.node, tr, tgt_prog);
 	if (err) {
 		bpf_link_cleanup(&link_primer);
-- 
2.55.0

Re: [PATCH v3 1/3] bpf: propagate cb_access from freplace programs

[Alexei Starovoitov]

On Mon, Sep 21, 2026 at 12:32 AM Weiming Shi <bestswngs@gmail.com> wrote:
> Cc: stable@vger.kernel.org
> Fixes: be8704ff07d2 ("bpf: Introduce dynamic program extensions")

That shouldn't go to stable. Loading an EXT prog takes CAP_BPF +
CAP_NET_ADMIN + CAP_PERFMON.

[...]

> +	if (prog->type == BPF_PROG_TYPE_EXT && READ_ONCE(prog->cb_access)) {
> +		WRITE_ONCE(tgt_prog->cb_access, true);
> +		/* Drain runs that observed cb_access=false before enabling freplace. */
> +		synchronize_rcu();
> +	}

tc doesn't look at cb_access at all, yet every tc freplace that touches
cb[] now waits for an RCU grace period in attach.

This is a separate issue from the LWT one. Patch 2 resets the cb after
the run whether the prog has cb_access or not, so it doesn't depend on
this patch. Pls drop it from this series.

Re: [PATCH v3 1/3] bpf: propagate cb_access from freplace programs

[Alexei Starovoitov]

On Sun Sep 20, 2026 at 4:32 PM UTC, Weiming Shi wrote:
>  
>  struct bpf_prog {
>  	u16			pages;		/* Number of allocated pages */
> +	bool			cb_access;	/* Is control block accessed? */
>  	u32			jited:1,	/* Is our filter JIT'ed? */
>  				jit_requested:1,/* archs need to JIT the prog */
>  				jit_required:1,	/* program strictly requires JIT compiler */
>  				gpl_compatible:1, /* Is filter GPL compatible? */
> -				cb_access:1,	/* Is control block accessed? */

No. keep the signle bit.

[PATCH v3 2/3] bpf: clear stale IPv4 options after LWT encapsulation

[Weiming Shi]

bpf_lwt_push_ip_encap() rebases the network header after prepending an IP
header, but leaves IPCB(skb)->opt describing the inner IPv4 header. An
ingress LWT route can consequently make an ICMP error use stale option
offsets when constructing its reply.

Mark each completed LWT IP encapsulation in the run's BPF network context
and reset the protocol control block after bpf_prog_run_save_cb() restores
it. This also covers an skb which was already marked encapsulated before
entering LWT. A failed SEG6 encapsulation does not set the marker, so it no
longer causes valid IPv6 metadata to be cleared.

Select the reset layout from the protocol callback which consumes the
packet: the original family for BPF_OK and unsupported redirects, or the
new family for supported reroute and redirect paths. Preserve the ingress
interface and L3-slave state from the restored original control block and
initialize the IPv6 next-header offset when needed. Save and restore the
marker around nested LWT runs.

Cc: stable@vger.kernel.org
Fixes: 52f278774e79 ("bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap")
Reported-by: Xiang Mei <xmei5@asu.edu>
Link: https://lore.kernel.org/bpf/20260915170147.3943392-2-bestswngs@gmail.com/
Suggested-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/48990076-414c-4196-99b9-86fce41b8054@iogearbox.net/
Assisted-by: LLM
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
---
 include/linux/filter.h |  1 +
 net/core/lwt_bpf.c     | 47 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/include/linux/filter.h b/include/linux/filter.h
index 788c2d625db4a..c8ca526d0661d 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -848,6 +848,7 @@ struct bpf_nh_params {
 #define BPF_RI_F_CPU_MAP_INIT	BIT(2)
 #define BPF_RI_F_DEV_MAP_INIT	BIT(3)
 #define BPF_RI_F_XSK_MAP_INIT	BIT(4)
+#define BPF_RI_F_LWT_IP_ENCAP	BIT(5)
 
 struct bpf_redirect_info {
 	u64 tgt_index;
diff --git a/net/core/lwt_bpf.c b/net/core/lwt_bpf.c
index da49364ec63de..d585484a3a766 100644
--- a/net/core/lwt_bpf.c
+++ b/net/core/lwt_bpf.c
@@ -36,10 +36,44 @@ static inline struct bpf_lwt *bpf_lwt_lwtunnel(struct lwtunnel_state *lwt)
 #define NO_REDIRECT false
 #define CAN_REDIRECT true
 
+static void bpf_lwt_reset_cb(struct sk_buff *skb, __be16 orig_proto,
+			     bool use_new_proto)
+{
+	__be16 cb_proto = use_new_proto ? skb->protocol : orig_proto;
+	int iif = skb->skb_iif;
+	bool l3slave = false;
+
+	/* VRF may have replaced skb_iif with the master device index. */
+	if (orig_proto == htons(ETH_P_IP)) {
+		iif = IPCB(skb)->iif;
+		l3slave = ipv4_l3mdev_skb(IPCB(skb)->flags);
+	} else if (orig_proto == htons(ETH_P_IPV6)) {
+		iif = IP6CB(skb)->iif;
+		l3slave = ipv6_l3mdev_skb(IP6CB(skb)->flags);
+	}
+
+	if (cb_proto == htons(ETH_P_IP)) {
+		memset(IPCB(skb), 0, sizeof(*IPCB(skb)));
+		IPCB(skb)->iif = iif;
+		if (l3slave)
+			IPCB(skb)->flags |= IPSKB_L3SLAVE;
+	} else if (cb_proto == htons(ETH_P_IPV6)) {
+		memset(IP6CB(skb), 0, sizeof(*IP6CB(skb)));
+		IP6CB(skb)->iif = iif;
+		IP6CB(skb)->nhoff = offsetof(struct ipv6hdr, nexthdr);
+		if (l3slave)
+			IP6CB(skb)->flags |= IP6SKB_L3SLAVE;
+	}
+}
+
 static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt,
 		       struct dst_entry *dst, bool can_redirect)
 {
 	struct bpf_net_context __bpf_net_ctx, *bpf_net_ctx;
+	struct bpf_redirect_info *ri;
+	bool lwt_ip_encap, nested_lwt_ip_encap;
+	__be16 orig_proto = skb->protocol;
+	bool use_new_proto;
 	int ret;
 
 	/* Disabling BH is needed to protect per-CPU bpf_redirect_info between
@@ -47,8 +81,20 @@ static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt,
 	 */
 	local_bh_disable();
 	bpf_net_ctx = bpf_net_ctx_set(&__bpf_net_ctx);
+	ri = bpf_net_ctx_get_ri();
+	nested_lwt_ip_encap = ri->kern_flags & BPF_RI_F_LWT_IP_ENCAP;
+	ri->kern_flags &= ~BPF_RI_F_LWT_IP_ENCAP;
 	bpf_compute_data_pointers(skb);
 	ret = bpf_prog_run_save_cb(lwt->prog, skb);
+	lwt_ip_encap = ri->kern_flags & BPF_RI_F_LWT_IP_ENCAP;
+	ri->kern_flags &= ~BPF_RI_F_LWT_IP_ENCAP;
+	if (nested_lwt_ip_encap)
+		ri->kern_flags |= BPF_RI_F_LWT_IP_ENCAP;
+	use_new_proto = (ret == BPF_LWT_REROUTE &&
+			 lwt->prog->type != BPF_PROG_TYPE_LWT_OUT) ||
+			(ret == BPF_REDIRECT && can_redirect);
+	if (lwt_ip_encap)
+		bpf_lwt_reset_cb(skb, orig_proto, use_new_proto);
 
 	switch (ret) {
 	case BPF_OK:
@@ -668,6 +714,7 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 	} else {
 		skb->protocol = htons(ETH_P_IPV6);
 	}
+	bpf_net_ctx_get_ri()->kern_flags |= BPF_RI_F_LWT_IP_ENCAP;
 
 	if (skb_is_gso(skb))
 		return handle_gso_encap(skb, ipv4, len);
-- 
2.55.0

Re: [PATCH v3 2/3] bpf: clear stale IPv4 options after LWT encapsulation

[Alexei Starovoitov]

On Mon, Sep 21, 2026 at 12:32 AM Weiming Shi <bestswngs@gmail.com> wrote:
> Select the reset layout from the protocol callback which consumes the
> packet: the original family for BPF_OK and unsupported redirects, or the
> new family for supported reroute and redirect paths. Preserve the ingress
> interface and L3-slave state from the restored original control block and
> initialize the IPv6 next-header offset when needed. Save and restore the
> marker around nested LWT runs.

What Daniel sketched in v2 was 20 lines. This is still an overkill.

[...]

> +static void bpf_lwt_reset_cb(struct sk_buff *skb, __be16 orig_proto,
> +			     bool use_new_proto)
> +{
> +	__be16 cb_proto = use_new_proto ? skb->protocol : orig_proto;
> +	int iif = skb->skb_iif;
> +	bool l3slave = false;
> +
> +	/* VRF may have replaced skb_iif with the master device index. */
> +	if (orig_proto == htons(ETH_P_IP)) {
> +		iif = IPCB(skb)->iif;
> +		l3slave = ipv4_l3mdev_skb(IPCB(skb)->flags);



When the family doesn't change only IPCB(skb)->opt is stale.
Clear just that like ip_tunnel_xmit() and udp_tunnel_xmit_skb() do.
iif and flags stay as they are and the VRF special casing goes away.
When the family changes do what seg6_do_srh_encap() does.

[...]

> +	use_new_proto = (ret == BPF_LWT_REROUTE &&
> +			 lwt->prog->type != BPF_PROG_TYPE_LWT_OUT) ||
> +			(ret == BPF_REDIRECT && can_redirect);
> +	if (lwt_ip_encap)
> +		bpf_lwt_reset_cb(skb, orig_proto, use_new_proto);

Not needed. BPF_OK after the prog changed the family is broken no
matter which layout the cb has. bpf_xmit() drops such skb and
bpf_input() hands a v6 packet to ip_forward().
Use skb->protocol.

pw-bot: cr