From: Stanislav Fomichev <sdf.kernel@gmail.com>
To: Nikolay Aleksandrov <razor@blackwall.org>
Cc: bpf@vger.kernel.org, Daniel Borkmann <daniel@iogearbox.net>,
Ayush Sawal <ayush.sawal@chelsio.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>,
Paolo Abeni <pabeni@redhat.com>,
Alexei Starovoitov <ast@kernel.org>,
Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Song Liu <song@kernel.org>,
Yonghong Song <yonghong.song@linux.dev>,
Jiri Olsa <jolsa@kernel.org>,
Emil Tsalapatis <emil@etsalapatis.com>,
Ihor Solodrai <ihor.solodrai@linux.dev>,
John Fastabend <john.fastabend@gmail.com>,
Stanislav Fomichev <sdf@fomichev.me>,
Simon Horman <horms@kernel.org>,
Kuniyuki Iwashima <kuniyu@google.com>,
Willem de Bruijn <willemb@google.com>,
Jamal Hadi Salim <jhs@mojatatu.com>,
Jiri Pirko <jiri@resnulli.us>,
Sabrina Dubroca <sd@queasysnail.net>,
Paul Chaignon <paul@isovalent.com>,
Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>,
netdev@vger.kernel.org
Subject: Re: [PATCH bpf-next 3/8] bpf: allow bpf_sk_assign on tc egress with refcounted sockets
Date: Fri, 20 Nov 2026 08:31:08 -0800 [thread overview]
Message-ID: <awB1eao9nFgiTn8A@devvm16445.scu0.facebook.com> (raw)
In-Reply-To: <20261119142231.2612515-4-razor@blackwall.org>
On 11/19, Nikolay Aleksandrov wrote:
> From: Daniel Borkmann <daniel@iogearbox.net>
>
> bpf_sk_assign() has been limited to tc ingress because the assigned
> socket may be a SOCK_RCU_FREE listener held without a refcount, relying
> on the current RCU read-side section for its lifetime. That assumption
> breaks on egress where the skb can sit in a qdisc across grace periods.
> However, the helper is also useful on the egress path, for example to
> attribute forwarded traffic to a specific established socket for qdisc
> flow classification (fq, fq_codel), cgroup accounting, or SO_PRIORITY
> propagation. Established sockets are refcounted and therefore safe to
> pin on an skb that may be queued.
>
> Allow TC egress to assign full refcounted sockets, taking an skb-owned
> reference after orphaning any previous send-side owner. Preserve it
> across redirects, classic BPF verdicts and netem delay. Reject decrypted
> egress skbs to preserve the original TLS transmit owner.
>
> Using sock_pfree_ref as a destructor encodes the refcount state so:
>
> - sock_pfree_ref releases the refcount on drop (unlike sock_pfree
> which skips non-refcounted sockets).
> - skb_sk_is_prefetched recognizes both destructors so ip_rcv_core
> still skips the skb_orphan on the receive side.
> - skb_steal_sock sets refcounted=true for sock_pfree_ref so the
> transport layer calls sock_put after consuming the socket.
> - skb_do_redirect only orphans sock_pfree + !sk_is_refcounted (the
> true no-refcount case from ingress), preserving sock_pfree_ref
> sockets across redirects.
>
> The refcount_inc_not_zero cost per bpf_sk_assign call is negligible
> relative to the BPF program and the sk_lookup that precedes it, and
> does not affect the UDP early demux path which continues to use
> sock_pfree without a refcount.
>
> Handle reference-only ownership in TLS software fallback and Chelsio
> record reconstruction without sk_wmem_alloc charges or lost references.
>
> Software GSO splitting after assignment is unsupported, including qdisc,
> device/driver/tunnel fallback and netkit peer-side NFQUEUE segmentation.
> Segments can lose the selected socket; intact GSO and assignment after
> segmentation remain supported. Ordinary loopback and veth peer NAPI/GRO
> preparation can also discard the assignment, even without GSO.
>
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Signed-off-by: Nikolay Aleksandrov <razor@blackwall.org>
> ---
> CC: Ayush Sawal <ayush.sawal@chelsio.com>
> CC: Andrew Lunn <andrew+netdev@lunn.ch>
> CC: David S. Miller <davem@davemloft.net>
> CC: Eric Dumazet <edumazet@kernel.org>
> CC: Jakub Kicinski <kuba@kernel.org>
> CC: Paolo Abeni <pabeni@redhat.com>
> CC: Alexei Starovoitov <ast@kernel.org>
> CC: Daniel Borkmann <daniel@iogearbox.net>
> CC: Andrii Nakryiko <andrii@kernel.org>
> CC: Eduard Zingerman <eddyz87@gmail.com>
> CC: Kumar Kartikeya Dwivedi <memxor@gmail.com>
> CC: Martin KaFai Lau <martin.lau@linux.dev>
> CC: Song Liu <song@kernel.org>
> CC: Yonghong Song <yonghong.song@linux.dev>
> CC: Jiri Olsa <jolsa@kernel.org>
> CC: Emil Tsalapatis <emil@etsalapatis.com>
> CC: Ihor Solodrai <ihor.solodrai@linux.dev>
> CC: John Fastabend <john.fastabend@gmail.com>
> CC: Stanislav Fomichev <sdf@fomichev.me>
> CC: Simon Horman <horms@kernel.org>
> CC: Kuniyuki Iwashima <kuniyu@google.com>
> CC: Willem de Bruijn <willemb@google.com>
> CC: Jamal Hadi Salim <jhs@mojatatu.com>
> CC: Jiri Pirko <jiri@resnulli.us>
> CC: Sabrina Dubroca <sd@queasysnail.net>
> CC: Paul Chaignon <paul@isovalent.com>
> CC: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
> CC: netdev@vger.kernel.org
>
> .../chelsio/inline_crypto/ch_ktls/chcr_ktls.c | 5 ++++-
> include/uapi/linux/bpf.h | 8 ++++---
> net/core/filter.c | 22 +++++++++++++------
> net/core/sock.c | 4 ++++
> net/sched/act_bpf.c | 4 +++-
> net/tls/tls_device_fallback.c | 5 +++--
> tools/include/uapi/linux/bpf.h | 8 ++++---
> 7 files changed, 39 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
> index 4d645771596a..796497e9d3b9 100644
> --- a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
> +++ b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c
> @@ -1669,7 +1669,10 @@ static void chcr_ktls_copy_record_in_skb(struct sk_buff *nskb,
> nskb->truesize += record->len;
> nskb->sk = skb->sk;
> nskb->destructor = skb->destructor;
> - refcount_add(nskb->truesize, &nskb->sk->sk_wmem_alloc);
> + if (nskb->destructor == sock_pfree_ref)
> + sock_hold(nskb->sk);
> + else
> + refcount_add(nskb->truesize, &nskb->sk->sk_wmem_alloc);
> }
>
> /*
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 62de0919db72..45e3018fb2ae 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -4611,7 +4611,9 @@ union bpf_attr {
> * **bpf_clone_redirect**\ () or other methods outside of BPF may
> * interfere with successful delivery to the socket.
> *
> - * This operation is only valid from TC ingress path.
> + * This operation is valid from TC ingress and egress paths.
> + * On egress, only full refcounted sockets are accepted.
> + * Software GSO segmentation after assignment is unsupported.
> *
> * The *flags* argument must be zero.
> * Return
> @@ -4623,8 +4625,8 @@ union bpf_attr {
> *
> * **-ENETUNREACH** if the socket is unreachable (wrong netns).
> *
> - * **-EOPNOTSUPP** if the operation is not supported, for example
> - * a call from outside of TC ingress.
> + * **-EOPNOTSUPP** if the socket is unhashed, or TC egress uses
> + * a non-full or non-refcounted socket or a decrypted packet.
> *
> * long bpf_sk_assign(struct bpf_sk_lookup *ctx, struct bpf_sock *sk, u64 flags)
> * Description
> diff --git a/net/core/filter.c b/net/core/filter.c
> index 1ad57a371534..feeb752f7a00 100644
> --- a/net/core/filter.c
> +++ b/net/core/filter.c
> @@ -2559,14 +2559,17 @@ int skb_do_redirect(struct sk_buff *skb)
> if (unlikely(!dev))
> goto out_drop;
>
> +#ifdef CONFIG_INET
> /*
> - * Prefetched sockets may borrow a listener under RCU. Orphan them
> - * before redirects that can outlive this section. Peer ingress
> - * stays in the same RCU section.
> + * Prefetched listeners may be borrowed under RCU. Drop borrowed
> + * assignments before redirects that can outlive this section.
> + * Peer ingress stays in the same RCU section.
> */
> - if (skb_sk_is_prefetched(skb) &&
> + if (skb->destructor == sock_pfree &&
> + !sk_is_refcounted(skb->sk) &&
> (!(flags & BPF_F_PEER) || (flags & BPF_F_EGRESS)))
> skb_orphan(skb);
> +#endif
> if (flags & BPF_F_PEER) {
> dev = skb_get_peer_dev(dev);
> if (unlikely(!dev ||
> @@ -8115,21 +8118,26 @@ static const struct bpf_func_proto bpf_tcp_gen_syncookie_proto = {
>
> BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
> {
> + bool egress = !skb_at_tc_ingress(skb);
> + bool refcounted;
> +
> if (!sk || flags != 0)
> return -EINVAL;
> - if (!skb_at_tc_ingress(skb))
> + if (egress && (!sk_fullsock(sk) || !sk_is_refcounted(sk) ||
> + skb_is_decrypted(skb)))
> return -EOPNOTSUPP;
> + refcounted = egress || sk_is_refcounted(sk);
> if (unlikely(dev_net(skb->dev) != sock_net(sk)))
> return -ENETUNREACH;
> if (sk_unhashed(sk))
> return -EOPNOTSUPP;
> - if (sk_is_refcounted(sk) &&
> + if (refcounted &&
> unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
> return -ENOENT;
>
> skb_orphan(skb);
> skb->sk = sk;
> - skb->destructor = sock_pfree;
> + skb->destructor = egress ? sock_pfree_ref : sock_pfree;
>
> return 0;
> }
[..]
> diff --git a/net/core/sock.c b/net/core/sock.c
> index 3b1f60eaa73e..f4ec58a86056 100644
> --- a/net/core/sock.c
> +++ b/net/core/sock.c
> @@ -2818,6 +2818,10 @@ static bool can_skb_orphan_partial(const struct sk_buff *skb)
> */
> void skb_orphan_partial(struct sk_buff *skb)
> {
> +#ifdef CONFIG_INET
> + if (skb->destructor == sock_pfree_ref)
> + return;
> +#endif
> if (skb_is_tcp_pure_ack(skb))
> return;
>
> diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c
> index 06d8f78b7368..a0918fe9a71a 100644
> --- a/net/sched/act_bpf.c
> +++ b/net/sched/act_bpf.c
> @@ -54,8 +54,10 @@ TC_INDIRECT_SCOPE int tcf_bpf_act(struct sk_buff *skb,
> }
> if (unlikely(!skb->tstamp && skb->tstamp_type))
> skb->tstamp_type = SKB_CLOCK_REALTIME;
> - if (skb_sk_is_prefetched(skb) && filter_res != TC_ACT_OK)
> +#ifdef CONFIG_INET
> + if (skb->destructor == sock_pfree && filter_res != TC_ACT_OK)
> skb_orphan(skb);
> +#endif
>
> /* A BPF program may overwrite the default action opcode.
> * Similarly as in cls_bpf, if filter_res == -1 we use the
Maybe it's just me, but I don't think all these unrolled ==
sock_pfree{,_ref} checks are readable. Can we at least have new
skb_sk_is_prefetched-like wrappers to do these? With clear
names/comments/intent?
next prev parent reply other threads:[~2026-11-20 16:31 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 14:22 [PATCH bpf-next 0/8] bpf: add bpf_sk_assign support on egress Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 1/8] bpf: Orphan prefetched sk on skb_do_redirect() Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 2/8] net: track references owned by prefetched socket assignments Nikolay Aleksandrov
2026-11-19 14:22 ` [PATCH bpf-next 3/8] bpf: allow bpf_sk_assign on tc egress with refcounted sockets Nikolay Aleksandrov
2026-11-19 15:19 ` bot+bpf-ci
2026-11-19 15:36 ` Nikolay Aleksandrov
2026-11-20 16:31 ` Stanislav Fomichev [this message]
2026-11-19 14:22 ` [PATCH bpf-next 4/8] bpf: support SOCK_RCU_FREE sockets in bpf_sk_assign on egress Nikolay Aleksandrov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=awB1eao9nFgiTn8A@devvm16445.scu0.facebook.com \
--to=sdf.kernel@gmail.com \
--cc=alasdair.mcwilliam@isovalent.com \
--cc=andrew+netdev@lunn.ch \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=ayush.sawal@chelsio.com \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=eddyz87@gmail.com \
--cc=edumazet@kernel.org \
--cc=emil@etsalapatis.com \
--cc=horms@kernel.org \
--cc=ihor.solodrai@linux.dev \
--cc=jhs@mojatatu.com \
--cc=jiri@resnulli.us \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=kuba@kernel.org \
--cc=kuniyu@google.com \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=paul@isovalent.com \
--cc=razor@blackwall.org \
--cc=sd@queasysnail.net \
--cc=sdf@fomichev.me \
--cc=song@kernel.org \
--cc=willemb@google.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox