From: Nikolay Aleksandrov <razor@blackwall.org>
To: bpf@vger.kernel.org
Cc: Daniel Borkmann <daniel@iogearbox.net>,
Nikolay Aleksandrov <razor@blackwall.org>,
Ayush Sawal <ayush.sawal@chelsio.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Alexei Starovoitov <ast@kernel.org>,
Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Song Liu <song@kernel.org>,
Yonghong Song <yonghong.song@linux.dev>,
Jiri Olsa <jolsa@kernel.org>,
Emil Tsalapatis <emil@etsalapatis.com>,
Ihor Solodrai <ihor.solodrai@linux.dev>,
John Fastabend <john.fastabend@gmail.com>,
Stanislav Fomichev <sdf@fomichev.me>,
Simon Horman <horms@kernel.org>,
Kuniyuki Iwashima <kuniyu@google.com>,
Willem de Bruijn <willemb@google.com>,
Jamal Hadi Salim <jhs@mojatatu.com>,
Jiri Pirko <jiri@resnulli.us>,
Sabrina Dubroca <sd@queasysnail.net>,
Paul Chaignon <paul@isovalent.com>,
Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>,
netdev@vger.kernel.org
Subject: [PATCH bpf-next 3/8] bpf: allow bpf_sk_assign on tc egress with refcounted sockets
Date: Thu, 19 Nov 2026 16:22:25 +0200 [thread overview]
Message-ID: <20261119142231.2612515-4-razor@blackwall.org> (raw)
In-Reply-To: <20261119142231.2612515-1-razor@blackwall.org>
From: Daniel Borkmann <daniel@iogearbox.net>
bpf_sk_assign() has been limited to tc ingress because the assigned
socket may be a SOCK_RCU_FREE listener held without a refcount, relying
on the current RCU read-side section for its lifetime. That assumption
breaks on egress where the skb can sit in a qdisc across grace periods.
However, the helper is also useful on the egress path, for example to
attribute forwarded traffic to a specific established socket for qdisc
flow classification (fq, fq_codel), cgroup accounting, or SO_PRIORITY
propagation. Established sockets are refcounted and therefore safe to
pin on an skb that may be queued.
Allow TC egress to assign full refcounted sockets, taking an skb-owned
reference after orphaning any previous send-side owner. Preserve it
across redirects, classic BPF verdicts and netem delay. Reject decrypted
egress skbs to preserve the original TLS transmit owner.
Using sock_pfree_ref as a destructor encodes the refcount state so:
- sock_pfree_ref releases the refcount on drop (unlike sock_pfree
which skips non-refcounted sockets).
- skb_sk_is_prefetched recognizes both destructors so ip_rcv_core
still skips the skb_orphan on the receive side.
- skb_steal_sock sets refcounted=true for sock_pfree_ref so the
transport layer calls sock_put after consuming the socket.
- skb_do_redirect only orphans sock_pfree + !sk_is_refcounted (the
true no-refcount case from ingress), preserving sock_pfree_ref
sockets across redirects.
The refcount_inc_not_zero cost per bpf_sk_assign call is negligible
relative to the BPF program and the sk_lookup that precedes it, and
does not affect the UDP early demux path which continues to use
sock_pfree without a refcount.
Handle reference-only ownership in TLS software fallback and Chelsio
record reconstruction without sk_wmem_alloc charges or lost references.
Software GSO splitting after assignment is unsupported, including qdisc,
device/driver/tunnel fallback and netkit peer-side NFQUEUE segmentation.
Segments can lose the selected socket; intact GSO and assignment after
segmentation remain supported. Ordinary loopback and veth peer NAPI/GRO
preparation can also discard the assignment, even without GSO.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Nikolay Aleksandrov <razor@blackwall.org>
---
CC: Ayush Sawal <ayush.sawal@chelsio.com>
CC: Andrew Lunn <andrew+netdev@lunn.ch>
CC: David S. Miller <davem@davemloft.net>
CC: Eric Dumazet <edumazet@kernel.org>
CC: Jakub Kicinski <kuba@kernel.org>
CC: Paolo Abeni <pabeni@redhat.com>
CC: Alexei Starovoitov <ast@kernel.org>
CC: Daniel Borkmann <daniel@iogearbox.net>
CC: Andrii Nakryiko <andrii@kernel.org>
CC: Eduard Zingerman <eddyz87@gmail.com>
CC: Kumar Kartikeya Dwivedi <memxor@gmail.com>
CC: Martin KaFai Lau <martin.lau@linux.dev>
CC: Song Liu <song@kernel.org>
CC: Yonghong Song <yonghong.song@linux.dev>
CC: Jiri Olsa <jolsa@kernel.org>
CC: Emil Tsalapatis <emil@etsalapatis.com>
CC: Ihor Solodrai <ihor.solodrai@linux.dev>
CC: John Fastabend <john.fastabend@gmail.com>
CC: Stanislav Fomichev <sdf@fomichev.me>
CC: Simon Horman <horms@kernel.org>
CC: Kuniyuki Iwashima <kuniyu@google.com>
CC: Willem de Bruijn <willemb@google.com>
CC: Jamal Hadi Salim <jhs@mojatatu.com>
CC: Jiri Pirko <jiri@resnulli.us>
CC: Sabrina Dubroca <sd@queasysnail.net>
CC: Paul Chaignon <paul@isovalent.com>
CC: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
CC: netdev@vger.kernel.org
.../chelsio/inline_crypto/ch_ktls/chcr_ktls.c | 5 ++++-
include/uapi/linux/bpf.h | 8 ++++---
net/core/filter.c | 22 +++++++++++++------
net/core/sock.c | 4 ++++
net/sched/act_bpf.c | 4 +++-
net/tls/tls_device_fallback.c | 5 +++--
tools/include/uapi/linux/bpf.h | 8 ++++---
7 files changed, 39 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
index 4d645771596a..796497e9d3b9 100644
--- a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
+++ b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
@@ -1669,7 +1669,10 @@ static void chcr_ktls_copy_record_in_skb(struct sk_buff *nskb,
nskb->truesize += record->len;
nskb->sk = skb->sk;
nskb->destructor = skb->destructor;
- refcount_add(nskb->truesize, &nskb->sk->sk_wmem_alloc);
+ if (nskb->destructor == sock_pfree_ref)
+ sock_hold(nskb->sk);
+ else
+ refcount_add(nskb->truesize, &nskb->sk->sk_wmem_alloc);
}
/*
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 62de0919db72..45e3018fb2ae 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -4611,7 +4611,9 @@ union bpf_attr {
* **bpf_clone_redirect**\ () or other methods outside of BPF may
* interfere with successful delivery to the socket.
*
- * This operation is only valid from TC ingress path.
+ * This operation is valid from TC ingress and egress paths.
+ * On egress, only full refcounted sockets are accepted.
+ * Software GSO segmentation after assignment is unsupported.
*
* The *flags* argument must be zero.
* Return
@@ -4623,8 +4625,8 @@ union bpf_attr {
*
* **-ENETUNREACH** if the socket is unreachable (wrong netns).
*
- * **-EOPNOTSUPP** if the operation is not supported, for example
- * a call from outside of TC ingress.
+ * **-EOPNOTSUPP** if the socket is unhashed, or TC egress uses
+ * a non-full or non-refcounted socket or a decrypted packet.
*
* long bpf_sk_assign(struct bpf_sk_lookup *ctx, struct bpf_sock *sk, u64 flags)
* Description
diff --git a/net/core/filter.c b/net/core/filter.c
index 1ad57a371534..feeb752f7a00 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2559,14 +2559,17 @@ int skb_do_redirect(struct sk_buff *skb)
if (unlikely(!dev))
goto out_drop;
+#ifdef CONFIG_INET
/*
- * Prefetched sockets may borrow a listener under RCU. Orphan them
- * before redirects that can outlive this section. Peer ingress
- * stays in the same RCU section.
+ * Prefetched listeners may be borrowed under RCU. Drop borrowed
+ * assignments before redirects that can outlive this section.
+ * Peer ingress stays in the same RCU section.
*/
- if (skb_sk_is_prefetched(skb) &&
+ if (skb->destructor == sock_pfree &&
+ !sk_is_refcounted(skb->sk) &&
(!(flags & BPF_F_PEER) || (flags & BPF_F_EGRESS)))
skb_orphan(skb);
+#endif
if (flags & BPF_F_PEER) {
dev = skb_get_peer_dev(dev);
if (unlikely(!dev ||
@@ -8115,21 +8118,26 @@ static const struct bpf_func_proto bpf_tcp_gen_syncookie_proto = {
BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
{
+ bool egress = !skb_at_tc_ingress(skb);
+ bool refcounted;
+
if (!sk || flags != 0)
return -EINVAL;
- if (!skb_at_tc_ingress(skb))
+ if (egress && (!sk_fullsock(sk) || !sk_is_refcounted(sk) ||
+ skb_is_decrypted(skb)))
return -EOPNOTSUPP;
+ refcounted = egress || sk_is_refcounted(sk);
if (unlikely(dev_net(skb->dev) != sock_net(sk)))
return -ENETUNREACH;
if (sk_unhashed(sk))
return -EOPNOTSUPP;
- if (sk_is_refcounted(sk) &&
+ if (refcounted &&
unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
return -ENOENT;
skb_orphan(skb);
skb->sk = sk;
- skb->destructor = sock_pfree;
+ skb->destructor = egress ? sock_pfree_ref : sock_pfree;
return 0;
}
diff --git a/net/core/sock.c b/net/core/sock.c
index 3b1f60eaa73e..f4ec58a86056 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -2818,6 +2818,10 @@ static bool can_skb_orphan_partial(const struct sk_buff *skb)
*/
void skb_orphan_partial(struct sk_buff *skb)
{
+#ifdef CONFIG_INET
+ if (skb->destructor == sock_pfree_ref)
+ return;
+#endif
if (skb_is_tcp_pure_ack(skb))
return;
diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c
index 06d8f78b7368..a0918fe9a71a 100644
--- a/net/sched/act_bpf.c
+++ b/net/sched/act_bpf.c
@@ -54,8 +54,10 @@ TC_INDIRECT_SCOPE int tcf_bpf_act(struct sk_buff *skb,
}
if (unlikely(!skb->tstamp && skb->tstamp_type))
skb->tstamp_type = SKB_CLOCK_REALTIME;
- if (skb_sk_is_prefetched(skb) && filter_res != TC_ACT_OK)
+#ifdef CONFIG_INET
+ if (skb->destructor == sock_pfree && filter_res != TC_ACT_OK)
skb_orphan(skb);
+#endif
/* A BPF program may overwrite the default action opcode.
* Similarly as in cls_bpf, if filter_res == -1 we use the
diff --git a/net/tls/tls_device_fallback.c b/net/tls/tls_device_fallback.c
index b547027cc879..1450335a94dc 100644
--- a/net/tls/tls_device_fallback.c
+++ b/net/tls/tls_device_fallback.c
@@ -194,8 +194,9 @@ static void complete_skb(struct sk_buff *nskb, struct sk_buff *skb, int headln)
update_chksum(nskb, headln);
- /* sock_efree means skb must gone through skb_orphan_partial() */
- if (nskb->destructor == sock_efree)
+ /* these skbs hold a socket ref but aren't charged to sk_wmem_alloc */
+ if (nskb->destructor == sock_efree ||
+ nskb->destructor == sock_pfree_ref)
return;
delta = nskb->truesize - skb->truesize;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 62de0919db72..45e3018fb2ae 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -4611,7 +4611,9 @@ union bpf_attr {
* **bpf_clone_redirect**\ () or other methods outside of BPF may
* interfere with successful delivery to the socket.
*
- * This operation is only valid from TC ingress path.
+ * This operation is valid from TC ingress and egress paths.
+ * On egress, only full refcounted sockets are accepted.
+ * Software GSO segmentation after assignment is unsupported.
*
* The *flags* argument must be zero.
* Return
@@ -4623,8 +4625,8 @@ union bpf_attr {
*
* **-ENETUNREACH** if the socket is unreachable (wrong netns).
*
- * **-EOPNOTSUPP** if the operation is not supported, for example
- * a call from outside of TC ingress.
+ * **-EOPNOTSUPP** if the socket is unhashed, or TC egress uses
+ * a non-full or non-refcounted socket or a decrypted packet.
*
* long bpf_sk_assign(struct bpf_sk_lookup *ctx, struct bpf_sock *sk, u64 flags)
* Description
--
2.47.3
next prev parent reply other threads:[~2026-11-19 14:28 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 14:22 [PATCH bpf-next 0/8] bpf: add bpf_sk_assign support on egress Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 1/8] bpf: Orphan prefetched sk on skb_do_redirect() Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 2/8] net: track references owned by prefetched socket assignments Nikolay Aleksandrov
2026-11-19 14:22 ` Nikolay Aleksandrov [this message]
2026-11-19 15:19 ` [PATCH bpf-next 3/8] bpf: allow bpf_sk_assign on tc egress with refcounted sockets bot+bpf-ci
2026-11-19 15:36 ` Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 4/8] bpf: support SOCK_RCU_FREE sockets in bpf_sk_assign on egress Nikolay Aleksandrov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119142231.2612515-4-razor@blackwall.org \
--to=razor@blackwall.org \
--cc=alasdair.mcwilliam@isovalent.com \
--cc=andrew+netdev@lunn.ch \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=ayush.sawal@chelsio.com \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=eddyz87@gmail.com \
--cc=edumazet@kernel.org \
--cc=emil@etsalapatis.com \
--cc=horms@kernel.org \
--cc=ihor.solodrai@linux.dev \
--cc=jhs@mojatatu.com \
--cc=jiri@resnulli.us \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=kuba@kernel.org \
--cc=kuniyu@google.com \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=paul@isovalent.com \
--cc=sd@queasysnail.net \
--cc=sdf@fomichev.me \
--cc=song@kernel.org \
--cc=willemb@google.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox