Netdev List
 help / color / mirror / Atom feed
From: Nikolay Aleksandrov <razor@blackwall.org>
To: bpf@vger.kernel.org
Cc: Daniel Borkmann <daniel@iogearbox.net>,
	Nikolay Aleksandrov <razor@blackwall.org>,
	Ayush Sawal <ayush.sawal@chelsio.com>,
	Andrew Lunn <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@kernel.org>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	Alexei Starovoitov <ast@kernel.org>,
	Andrii Nakryiko <andrii@kernel.org>,
	Eduard Zingerman <eddyz87@gmail.com>,
	Kumar Kartikeya Dwivedi <memxor@gmail.com>,
	Martin KaFai Lau <martin.lau@linux.dev>,
	Song Liu <song@kernel.org>,
	Yonghong Song <yonghong.song@linux.dev>,
	Jiri Olsa <jolsa@kernel.org>,
	Emil Tsalapatis <emil@etsalapatis.com>,
	Ihor Solodrai <ihor.solodrai@linux.dev>,
	John Fastabend <john.fastabend@gmail.com>,
	Stanislav Fomichev <sdf@fomichev.me>,
	Simon Horman <horms@kernel.org>,
	Kuniyuki Iwashima <kuniyu@google.com>,
	Willem de Bruijn <willemb@google.com>,
	Jamal Hadi Salim <jhs@mojatatu.com>,
	Jiri Pirko <jiri@resnulli.us>,
	Sabrina Dubroca <sd@queasysnail.net>,
	Paul Chaignon <paul@isovalent.com>,
	Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>,
	netdev@vger.kernel.org
Subject: [PATCH bpf-next 3/8] bpf: allow bpf_sk_assign on tc egress with refcounted sockets
Date: Thu, 19 Nov 2026 16:22:25 +0200	[thread overview]
Message-ID: <20261119142231.2612515-4-razor@blackwall.org> (raw)
In-Reply-To: <20261119142231.2612515-1-razor@blackwall.org>

From: Daniel Borkmann <daniel@iogearbox.net>

bpf_sk_assign() has been limited to tc ingress because the assigned
socket may be a SOCK_RCU_FREE listener held without a refcount, relying
on the current RCU read-side section for its lifetime. That assumption
breaks on egress where the skb can sit in a qdisc across grace periods.
However, the helper is also useful on the egress path, for example to
attribute forwarded traffic to a specific established socket for qdisc
flow classification (fq, fq_codel), cgroup accounting, or SO_PRIORITY
propagation. Established sockets are refcounted and therefore safe to
pin on an skb that may be queued.

Allow TC egress to assign full refcounted sockets, taking an skb-owned
reference after orphaning any previous send-side owner. Preserve it
across redirects, classic BPF verdicts and netem delay. Reject decrypted
egress skbs to preserve the original TLS transmit owner.

Using sock_pfree_ref as a destructor encodes the refcount state so:

  - sock_pfree_ref releases the refcount on drop (unlike sock_pfree
    which skips non-refcounted sockets).
  - skb_sk_is_prefetched recognizes both destructors so ip_rcv_core
    still skips the skb_orphan on the receive side.
  - skb_steal_sock sets refcounted=true for sock_pfree_ref so the
    transport layer calls sock_put after consuming the socket.
  - skb_do_redirect only orphans sock_pfree + !sk_is_refcounted (the
    true no-refcount case from ingress), preserving sock_pfree_ref
    sockets across redirects.

The refcount_inc_not_zero cost per bpf_sk_assign call is negligible
relative to the BPF program and the sk_lookup that precedes it, and
does not affect the UDP early demux path which continues to use
sock_pfree without a refcount.

Handle reference-only ownership in TLS software fallback and Chelsio
record reconstruction without sk_wmem_alloc charges or lost references.

Software GSO splitting after assignment is unsupported, including qdisc,
device/driver/tunnel fallback and netkit peer-side NFQUEUE segmentation.
Segments can lose the selected socket; intact GSO and assignment after
segmentation remain supported. Ordinary loopback and veth peer NAPI/GRO
preparation can also discard the assignment, even without GSO.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Nikolay Aleksandrov <razor@blackwall.org>
---
CC: Ayush Sawal <ayush.sawal@chelsio.com>
CC: Andrew Lunn <andrew+netdev@lunn.ch>
CC: David S. Miller <davem@davemloft.net>
CC: Eric Dumazet <edumazet@kernel.org>
CC: Jakub Kicinski <kuba@kernel.org>
CC: Paolo Abeni <pabeni@redhat.com>
CC: Alexei Starovoitov <ast@kernel.org>
CC: Daniel Borkmann <daniel@iogearbox.net>
CC: Andrii Nakryiko <andrii@kernel.org>
CC: Eduard Zingerman <eddyz87@gmail.com>
CC: Kumar Kartikeya Dwivedi <memxor@gmail.com>
CC: Martin KaFai Lau <martin.lau@linux.dev>
CC: Song Liu <song@kernel.org>
CC: Yonghong Song <yonghong.song@linux.dev>
CC: Jiri Olsa <jolsa@kernel.org>
CC: Emil Tsalapatis <emil@etsalapatis.com>
CC: Ihor Solodrai <ihor.solodrai@linux.dev>
CC: John Fastabend <john.fastabend@gmail.com>
CC: Stanislav Fomichev <sdf@fomichev.me>
CC: Simon Horman <horms@kernel.org>
CC: Kuniyuki Iwashima <kuniyu@google.com>
CC: Willem de Bruijn <willemb@google.com>
CC: Jamal Hadi Salim <jhs@mojatatu.com>
CC: Jiri Pirko <jiri@resnulli.us>
CC: Sabrina Dubroca <sd@queasysnail.net>
CC: Paul Chaignon <paul@isovalent.com>
CC: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
CC: netdev@vger.kernel.org

 .../chelsio/inline_crypto/ch_ktls/chcr_ktls.c |  5 ++++-
 include/uapi/linux/bpf.h                      |  8 ++++---
 net/core/filter.c                             | 22 +++++++++++++------
 net/core/sock.c                               |  4 ++++
 net/sched/act_bpf.c                           |  4 +++-
 net/tls/tls_device_fallback.c                 |  5 +++--
 tools/include/uapi/linux/bpf.h                |  8 ++++---
 7 files changed, 39 insertions(+), 17 deletions(-)

diff --git a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
index 4d645771596a..796497e9d3b9 100644
--- a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
+++ b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
@@ -1669,7 +1669,10 @@ static void chcr_ktls_copy_record_in_skb(struct sk_buff *nskb,
 	nskb->truesize += record->len;
 	nskb->sk = skb->sk;
 	nskb->destructor = skb->destructor;
-	refcount_add(nskb->truesize, &nskb->sk->sk_wmem_alloc);
+	if (nskb->destructor == sock_pfree_ref)
+		sock_hold(nskb->sk);
+	else
+		refcount_add(nskb->truesize, &nskb->sk->sk_wmem_alloc);
 }
 
 /*
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 62de0919db72..45e3018fb2ae 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -4611,7 +4611,9 @@ union bpf_attr {
  *		**bpf_clone_redirect**\ () or other methods outside of BPF may
  *		interfere with successful delivery to the socket.
  *
- *		This operation is only valid from TC ingress path.
+ *		This operation is valid from TC ingress and egress paths.
+ *		On egress, only full refcounted sockets are accepted.
+ *		Software GSO segmentation after assignment is unsupported.
  *
  *		The *flags* argument must be zero.
  *	Return
@@ -4623,8 +4625,8 @@ union bpf_attr {
  *
  *		**-ENETUNREACH** if the socket is unreachable (wrong netns).
  *
- *		**-EOPNOTSUPP** if the operation is not supported, for example
- *		a call from outside of TC ingress.
+ *		**-EOPNOTSUPP** if the socket is unhashed, or TC egress uses
+ *		a non-full or non-refcounted socket or a decrypted packet.
  *
  * long bpf_sk_assign(struct bpf_sk_lookup *ctx, struct bpf_sock *sk, u64 flags)
  *	Description
diff --git a/net/core/filter.c b/net/core/filter.c
index 1ad57a371534..feeb752f7a00 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2559,14 +2559,17 @@ int skb_do_redirect(struct sk_buff *skb)
 	if (unlikely(!dev))
 		goto out_drop;
 
+#ifdef CONFIG_INET
 	/*
-	 * Prefetched sockets may borrow a listener under RCU. Orphan them
-	 * before redirects that can outlive this section. Peer ingress
-	 * stays in the same RCU section.
+	 * Prefetched listeners may be borrowed under RCU. Drop borrowed
+	 * assignments before redirects that can outlive this section.
+	 * Peer ingress stays in the same RCU section.
 	 */
-	if (skb_sk_is_prefetched(skb) &&
+	if (skb->destructor == sock_pfree &&
+	    !sk_is_refcounted(skb->sk) &&
 	    (!(flags & BPF_F_PEER) || (flags & BPF_F_EGRESS)))
 		skb_orphan(skb);
+#endif
 	if (flags & BPF_F_PEER) {
 		dev = skb_get_peer_dev(dev);
 		if (unlikely(!dev ||
@@ -8115,21 +8118,26 @@ static const struct bpf_func_proto bpf_tcp_gen_syncookie_proto = {
 
 BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
 {
+	bool egress = !skb_at_tc_ingress(skb);
+	bool refcounted;
+
 	if (!sk || flags != 0)
 		return -EINVAL;
-	if (!skb_at_tc_ingress(skb))
+	if (egress && (!sk_fullsock(sk) || !sk_is_refcounted(sk) ||
+		       skb_is_decrypted(skb)))
 		return -EOPNOTSUPP;
+	refcounted = egress || sk_is_refcounted(sk);
 	if (unlikely(dev_net(skb->dev) != sock_net(sk)))
 		return -ENETUNREACH;
 	if (sk_unhashed(sk))
 		return -EOPNOTSUPP;
-	if (sk_is_refcounted(sk) &&
+	if (refcounted &&
 	    unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
 		return -ENOENT;
 
 	skb_orphan(skb);
 	skb->sk = sk;
-	skb->destructor = sock_pfree;
+	skb->destructor = egress ? sock_pfree_ref : sock_pfree;
 
 	return 0;
 }
diff --git a/net/core/sock.c b/net/core/sock.c
index 3b1f60eaa73e..f4ec58a86056 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -2818,6 +2818,10 @@ static bool can_skb_orphan_partial(const struct sk_buff *skb)
  */
 void skb_orphan_partial(struct sk_buff *skb)
 {
+#ifdef CONFIG_INET
+	if (skb->destructor == sock_pfree_ref)
+		return;
+#endif
 	if (skb_is_tcp_pure_ack(skb))
 		return;
 
diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c
index 06d8f78b7368..a0918fe9a71a 100644
--- a/net/sched/act_bpf.c
+++ b/net/sched/act_bpf.c
@@ -54,8 +54,10 @@ TC_INDIRECT_SCOPE int tcf_bpf_act(struct sk_buff *skb,
 	}
 	if (unlikely(!skb->tstamp && skb->tstamp_type))
 		skb->tstamp_type = SKB_CLOCK_REALTIME;
-	if (skb_sk_is_prefetched(skb) && filter_res != TC_ACT_OK)
+#ifdef CONFIG_INET
+	if (skb->destructor == sock_pfree && filter_res != TC_ACT_OK)
 		skb_orphan(skb);
+#endif
 
 	/* A BPF program may overwrite the default action opcode.
 	 * Similarly as in cls_bpf, if filter_res == -1 we use the
diff --git a/net/tls/tls_device_fallback.c b/net/tls/tls_device_fallback.c
index b547027cc879..1450335a94dc 100644
--- a/net/tls/tls_device_fallback.c
+++ b/net/tls/tls_device_fallback.c
@@ -194,8 +194,9 @@ static void complete_skb(struct sk_buff *nskb, struct sk_buff *skb, int headln)
 
 	update_chksum(nskb, headln);
 
-	/* sock_efree means skb must gone through skb_orphan_partial() */
-	if (nskb->destructor == sock_efree)
+	/* these skbs hold a socket ref but aren't charged to sk_wmem_alloc */
+	if (nskb->destructor == sock_efree ||
+	    nskb->destructor == sock_pfree_ref)
 		return;
 
 	delta = nskb->truesize - skb->truesize;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 62de0919db72..45e3018fb2ae 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -4611,7 +4611,9 @@ union bpf_attr {
  *		**bpf_clone_redirect**\ () or other methods outside of BPF may
  *		interfere with successful delivery to the socket.
  *
- *		This operation is only valid from TC ingress path.
+ *		This operation is valid from TC ingress and egress paths.
+ *		On egress, only full refcounted sockets are accepted.
+ *		Software GSO segmentation after assignment is unsupported.
  *
  *		The *flags* argument must be zero.
  *	Return
@@ -4623,8 +4625,8 @@ union bpf_attr {
  *
  *		**-ENETUNREACH** if the socket is unreachable (wrong netns).
  *
- *		**-EOPNOTSUPP** if the operation is not supported, for example
- *		a call from outside of TC ingress.
+ *		**-EOPNOTSUPP** if the socket is unhashed, or TC egress uses
+ *		a non-full or non-refcounted socket or a decrypted packet.
  *
  * long bpf_sk_assign(struct bpf_sk_lookup *ctx, struct bpf_sock *sk, u64 flags)
  *	Description
-- 
2.47.3


  parent reply	other threads:[~2026-11-19 14:28 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 14:22 [PATCH bpf-next 0/8] bpf: add bpf_sk_assign support on egress Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 1/8] bpf: Orphan prefetched sk on skb_do_redirect() Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 2/8] net: track references owned by prefetched socket assignments Nikolay Aleksandrov
2026-11-19 14:22 ` Nikolay Aleksandrov [this message]
2026-11-19 15:19   ` [PATCH bpf-next 3/8] bpf: allow bpf_sk_assign on tc egress with refcounted sockets bot+bpf-ci
2026-11-19 15:36     ` Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 4/8] bpf: support SOCK_RCU_FREE sockets in bpf_sk_assign on egress Nikolay Aleksandrov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119142231.2612515-4-razor@blackwall.org \
    --to=razor@blackwall.org \
    --cc=alasdair.mcwilliam@isovalent.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=ayush.sawal@chelsio.com \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=eddyz87@gmail.com \
    --cc=edumazet@kernel.org \
    --cc=emil@etsalapatis.com \
    --cc=horms@kernel.org \
    --cc=ihor.solodrai@linux.dev \
    --cc=jhs@mojatatu.com \
    --cc=jiri@resnulli.us \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kuba@kernel.org \
    --cc=kuniyu@google.com \
    --cc=martin.lau@linux.dev \
    --cc=memxor@gmail.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=paul@isovalent.com \
    --cc=sd@queasysnail.net \
    --cc=sdf@fomichev.me \
    --cc=song@kernel.org \
    --cc=willemb@google.com \
    --cc=yonghong.song@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox