* Re: [PATCH bpf-next v4 1/3] bpf: Add BPF_FIB_LOOKUP_VLAN flag to bpf_fib_lookup() helper
From: Toke Høiland-Jørgensen @ 2026-06-23 19:45 UTC (permalink / raw)
To: Avinash Duduskar, ast, daniel, andrii
Cc: eddyz87, memxor, martin.lau, song, yonghong.song, jolsa, emil,
john.fastabend, sdf, davem, edumazet, kuba, pabeni, horms, shuah,
hawk, yatsenko, leon.hwang, kpsingh, a.s.protopopov, ameryhung,
rongtao, eyal.birger, bpf, netdev, linux-kernel, linux-kselftest,
dsahern
In-Reply-To: <20260623182849.2623521-1-avinash.duduskar@gmail.com>
Avinash Duduskar <avinash.duduskar@gmail.com> writes:
> Toke Høiland-Jørgensen <toke@redhat.com> writes:
>
>> I think it's better to just move the assignment of params->ifindex
>> entirely into bpf_fib_set_fwd_params(), instead of this restore dance.
>> That way this can be simplified to:
>>
>> err = bpf_fib_set_fwd_params(dev, params, flags, mtu);
>> if (!err && fwd_dev)
>> *fwd_dev = dev;
>> return err;
>
> The caller-side restore is ungainly, agreed, but the assignment can't move
> all the way into the helper. The early params->ifindex = dev->ifindex
> sits above the neighbour lookup on purpose: that is d1c362e1dd68a
> ("bpf: Always return target ifindex in bpf_fib_lookup"), which took it
> out of bpf_fib_set_fwd_params() and put it there so a program still
> gets the target ifindex on the BPF_FIB_LKUP_RET_NO_NEIGH path and can
> bpf_redirect_neigh() on it. bpf_fib_set_fwd_params() is called only at
> the set_fwd_params label, below the NO_NEIGH return (and below the IPv6
> NO_SRC_ADDR return), so an assignment living in the helper never runs
> on those paths and params->ifindex falls back to the input. That would
> change the reported ifindex for plain bpf_fib_lookup() callers hitting
> NO_NEIGH, not only the VLAN ones.
Right. Well, seems I forgot about that patch, even though I seem to have
written it :)
> I can still get the caller down to your form by keeping the early write
> and moving just the VLAN_FAILURE rewind into the helper, with one extra
> parameter, the input ifindex saved before the egress write:
>
> err = bpf_fib_set_fwd_params(dev, params, flags, mtu, in_ifindex);
> if (!err && fwd_dev)
> *fwd_dev = dev;
> return err;
>
> and the helper owning the rewind in the unreducible branch:
>
> } else {
> params->ifindex = in_ifindex;
> return BPF_FIB_LKUP_RET_VLAN_FAILURE;
> }
OK, if we do need to restore it, I think it's better to do it there.
Also, wrt the fwd_dev parameter: Do we really have a use case from using
this from TC? In TC you can just redirect to the VLAN device; this is
meant for XDP which can't do that. So how about we just reject the flag
on the TC side, and get rid of the fwd_dev parameter entirely?
If we do that we're back to just a plain 'return bpf_fib_set_fwd_params()' :)
-Toke
^ permalink raw reply
* Re: [PATCH bpf-next v2] bpf, unix: Guard sk_msg-dependent code behind CONFIG_NET_SOCK_MSG
From: Alexei Starovoitov @ 2026-06-23 19:33 UTC (permalink / raw)
To: Kuniyuki Iwashima
Cc: Jakub Sitnicki, bpf, Alexei Starovoitov, Daniel Borkmann,
Jakub Kicinski, Jiayuan Chen, John Fastabend, Network Development,
kernel-team
In-Reply-To: <CAAVpQUBARp1qCEomgzWXVe35WatdaswujVLku+RESm_LW0dE7Q@mail.gmail.com>
On Tue, Jun 23, 2026 at 12:31 PM Kuniyuki Iwashima <kuniyu@google.com> wrote:
>
> On Tue, Jun 23, 2026 at 12:21 PM Jakub Sitnicki <jakub@cloudflare.com> wrote:
> >
> > On Tue, Jun 23, 2026 at 09:08 AM -07, Kuniyuki Iwashima wrote:
> > > On Tue, Jun 23, 2026 at 4:20 AM Jakub Sitnicki <jakub@cloudflare.com> wrote:
> > >>
> > >> Prepare to decouple BPF_SYSCALL config option from NET_SOCK_MSG. When
> > >> completed all code paths related to sockmap-based redirects should be
> > >> guarded by BPF_SYSCALL && NET_SOCK_MSG to allow users to opt out by
> > >> disabling NET_SOCK_MSG. The implementation of sockmap as a container for
> > >> socket references would remain under BPF_SYSCALL.
> > >>
> > >> Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
> > >> ---
> > >> Changes in v2:
> > >> - Handle prot->recvmsg being NULL (Sashiko)
> > >> - Elaborate on the end goal in description
> > >> - Link to v1: https://patch.msgid.link/20260622-bpf-sk_msg-split-unix-v1-1-d7e0cb7bb03b@cloudflare.com
> > >> ---
> > >> net/unix/af_unix.c | 4 ++--
> > >> net/unix/unix_bpf.c | 6 ++++++
> > >> 2 files changed, 8 insertions(+), 2 deletions(-)
> > >>
> > >> diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
> > >> index f7a9d55eee8a..84c11c60c75f 100644
> > >> --- a/net/unix/af_unix.c
> > >> +++ b/net/unix/af_unix.c
> > >> @@ -2675,7 +2675,7 @@ static int unix_dgram_recvmsg(struct socket *sock, struct msghdr *msg, size_t si
> > >> #ifdef CONFIG_BPF_SYSCALL
> > >> const struct proto *prot = READ_ONCE(sk->sk_prot);
> > >>
> > >> - if (prot != &unix_dgram_proto)
> > >> + if (prot->recvmsg)
> > >
> > > There is no reason to have this dead branch when
> > > CONFIG_BPF_SYSCALL && !NET_SOCK_MSG.
> > >
> > > Let's compile out all sockmap code when both configs
> > > are not enabled.
> > >
> > > Since AF_UNIX differs from TCP/UDP, it can take the
> > > simpler approach.
> >
> > Okay, will put the whole file behind hidden config option like so:
> >
> > --- a/net/unix/Kconfig
> > +++ b/net/unix/Kconfig
> > @@ -30,3 +30,8 @@ config UNIX_DIAG
> > help
> > Support for UNIX socket monitoring interface used by the ss tool.
> > If unsure, say Y.
> > +
> > +config UNIX_BPF
>
> Maybe UNIX_BPF_SOCKMAP or something.
> bpf_iter is supported without this config.
I don't like where it's going.
I strongly dislike new config knobs.
I'd rather remove existing knobs.
What is the motivation?
^ permalink raw reply
* Re: [PATCH bpf 2/2] selftests/bpf: Add test for UDP sock leak on sockmap lookup-bind-release
From: bot+bpf-ci @ 2026-06-23 19:32 UTC (permalink / raw)
To: mhal, john.fastabend, jakub, jiayuan.chen, davem, edumazet, kuba,
pabeni, horms, ast, cong.wang, daniel, andrii, eddyz87, memxor,
martin.lau, song, yonghong.song, jolsa, emil, shuah
Cc: netdev, bpf, linux-kernel, linux-kselftest, mhal, ast, andrii,
daniel, martin.lau, eddyz87, yonghong.song, clm, ihor.solodrai
In-Reply-To: <20260623-sockmap-lookup-udp-leak-v1-2-05804f9308e4@rbox.co>
[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]
> selftests/bpf: Add test for UDP sock leak on sockmap lookup-bind-release
>
> Setup and join a cgroup, then attach a cgroup/connect4 program that runs
>
> sk = bpf_map_lookup_elem(sockmap, 0)
> bpf_bind(ctx, sa, sizeof(sa))
> bpf_sk_release(sk)
>
> Unpatched kernel leaks the socket.
>
> Signed-off-by: Michal Luczaj <mhal@rbox.co>
This test reproduces a UDP socket leak across sockmap lookup-bind-release,
but there is no Fixes: tag.
The sibling fix commit a2510dc351c5 ("bpf, sockmap: Don't leak UDP socks on
lookup-bind-release") carries a Fixes: tag pointing at the commit that lifted
the socket-state restriction allowing unbound UDP sockets into sockmap.
Should this test carry the same tag so it stays backportable alongside the
fix?
Fixes: 0c48eefae712 ("sock_map: Lift socket state restriction for datagram sockets")
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28049771327
^ permalink raw reply
* Re: s2io: driver still in use - please reconsider removal
From: Ethan Nelson-Moore @ 2026-06-23 19:31 UTC (permalink / raw)
To: Michael Pratte
Cc: Jakub Kicinski, Paolo Abeni, Eric Dumazet, Andrew Lunn,
Simon Horman, David S . Miller, netdev
In-Reply-To: <20260623112133.752195-1-slatoncomputers@gmail.com>
Hi, Michael,
On Tue, Jun 23, 2026 at 4:21 AM Michael Pratte
<slatoncomputers@gmail.com> wrote:
> Commit aba0138eb7d7 ("net: ethernet: neterion: s2io: remove unused
> driver") removed s2io in v7.0 as "highly unlikely to still be used."
> It is still in use here: an Exar Xframe-II (PCI 17d5:5832) in a
> Supermicro X5DA8.
>
> Bringing it up, I found that no TCP can be transmitted on these
> adapters since v4.2.
[...]
> Given it is evidently still in use, would
> you consider reverting the removal?
Given that the driver has not been working for almost 11 years and you
are seemingly the first person to notice, I would like to respectfully
disagree with this assertion.
Are you using the card for actual work, or are you just testing it out
of curiosity? What kernel version were you running before you upgraded
to a current kernel?
Ethan
^ permalink raw reply
* Re: [PATCH bpf-next v2] bpf, unix: Guard sk_msg-dependent code behind CONFIG_NET_SOCK_MSG
From: Kuniyuki Iwashima @ 2026-06-23 19:31 UTC (permalink / raw)
To: Jakub Sitnicki
Cc: bpf, Alexei Starovoitov, Daniel Borkmann, Jakub Kicinski,
Jiayuan Chen, John Fastabend, netdev, kernel-team
In-Reply-To: <87v7b9ysep.fsf@cloudflare.com>
On Tue, Jun 23, 2026 at 12:21 PM Jakub Sitnicki <jakub@cloudflare.com> wrote:
>
> On Tue, Jun 23, 2026 at 09:08 AM -07, Kuniyuki Iwashima wrote:
> > On Tue, Jun 23, 2026 at 4:20 AM Jakub Sitnicki <jakub@cloudflare.com> wrote:
> >>
> >> Prepare to decouple BPF_SYSCALL config option from NET_SOCK_MSG. When
> >> completed all code paths related to sockmap-based redirects should be
> >> guarded by BPF_SYSCALL && NET_SOCK_MSG to allow users to opt out by
> >> disabling NET_SOCK_MSG. The implementation of sockmap as a container for
> >> socket references would remain under BPF_SYSCALL.
> >>
> >> Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
> >> ---
> >> Changes in v2:
> >> - Handle prot->recvmsg being NULL (Sashiko)
> >> - Elaborate on the end goal in description
> >> - Link to v1: https://patch.msgid.link/20260622-bpf-sk_msg-split-unix-v1-1-d7e0cb7bb03b@cloudflare.com
> >> ---
> >> net/unix/af_unix.c | 4 ++--
> >> net/unix/unix_bpf.c | 6 ++++++
> >> 2 files changed, 8 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
> >> index f7a9d55eee8a..84c11c60c75f 100644
> >> --- a/net/unix/af_unix.c
> >> +++ b/net/unix/af_unix.c
> >> @@ -2675,7 +2675,7 @@ static int unix_dgram_recvmsg(struct socket *sock, struct msghdr *msg, size_t si
> >> #ifdef CONFIG_BPF_SYSCALL
> >> const struct proto *prot = READ_ONCE(sk->sk_prot);
> >>
> >> - if (prot != &unix_dgram_proto)
> >> + if (prot->recvmsg)
> >
> > There is no reason to have this dead branch when
> > CONFIG_BPF_SYSCALL && !NET_SOCK_MSG.
> >
> > Let's compile out all sockmap code when both configs
> > are not enabled.
> >
> > Since AF_UNIX differs from TCP/UDP, it can take the
> > simpler approach.
>
> Okay, will put the whole file behind hidden config option like so:
>
> --- a/net/unix/Kconfig
> +++ b/net/unix/Kconfig
> @@ -30,3 +30,8 @@ config UNIX_DIAG
> help
> Support for UNIX socket monitoring interface used by the ss tool.
> If unsure, say Y.
> +
> +config UNIX_BPF
Maybe UNIX_BPF_SOCKMAP or something.
bpf_iter is supported without this config.
> + bool
> + depends on UNIX
> + default y if BPF_SYSCALL && NET_SOCK_MSG
^ permalink raw reply
* Re: [PATCH bpf-next v2] bpf, unix: Guard sk_msg-dependent code behind CONFIG_NET_SOCK_MSG
From: Jakub Sitnicki @ 2026-06-23 19:21 UTC (permalink / raw)
To: Kuniyuki Iwashima
Cc: bpf, Alexei Starovoitov, Daniel Borkmann, Jakub Kicinski,
Jiayuan Chen, John Fastabend, netdev, kernel-team
In-Reply-To: <CAAVpQUBsQFFxJFDnJzxmsER3bOjm=zqJ5P5MSeW_T9v-4639cw@mail.gmail.com>
On Tue, Jun 23, 2026 at 09:08 AM -07, Kuniyuki Iwashima wrote:
> On Tue, Jun 23, 2026 at 4:20 AM Jakub Sitnicki <jakub@cloudflare.com> wrote:
>>
>> Prepare to decouple BPF_SYSCALL config option from NET_SOCK_MSG. When
>> completed all code paths related to sockmap-based redirects should be
>> guarded by BPF_SYSCALL && NET_SOCK_MSG to allow users to opt out by
>> disabling NET_SOCK_MSG. The implementation of sockmap as a container for
>> socket references would remain under BPF_SYSCALL.
>>
>> Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
>> ---
>> Changes in v2:
>> - Handle prot->recvmsg being NULL (Sashiko)
>> - Elaborate on the end goal in description
>> - Link to v1: https://patch.msgid.link/20260622-bpf-sk_msg-split-unix-v1-1-d7e0cb7bb03b@cloudflare.com
>> ---
>> net/unix/af_unix.c | 4 ++--
>> net/unix/unix_bpf.c | 6 ++++++
>> 2 files changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
>> index f7a9d55eee8a..84c11c60c75f 100644
>> --- a/net/unix/af_unix.c
>> +++ b/net/unix/af_unix.c
>> @@ -2675,7 +2675,7 @@ static int unix_dgram_recvmsg(struct socket *sock, struct msghdr *msg, size_t si
>> #ifdef CONFIG_BPF_SYSCALL
>> const struct proto *prot = READ_ONCE(sk->sk_prot);
>>
>> - if (prot != &unix_dgram_proto)
>> + if (prot->recvmsg)
>
> There is no reason to have this dead branch when
> CONFIG_BPF_SYSCALL && !NET_SOCK_MSG.
>
> Let's compile out all sockmap code when both configs
> are not enabled.
>
> Since AF_UNIX differs from TCP/UDP, it can take the
> simpler approach.
Okay, will put the whole file behind hidden config option like so:
--- a/net/unix/Kconfig
+++ b/net/unix/Kconfig
@@ -30,3 +30,8 @@ config UNIX_DIAG
help
Support for UNIX socket monitoring interface used by the ss tool.
If unsure, say Y.
+
+config UNIX_BPF
+ bool
+ depends on UNIX
+ default y if BPF_SYSCALL && NET_SOCK_MSG
^ permalink raw reply
* Re: [PATCH] crypto: af_alg - Document the deprecation of AF_ALG
From: Eric Biggers @ 2026-06-23 19:19 UTC (permalink / raw)
To: Linus Torvalds
Cc: Bastien Nocera, linux-crypto, Herbert Xu, Marcel Holtmann,
Luiz Augusto von Dentz, linux-doc, linux-api, linux-kernel,
netdev, linux-bluetooth, ell
In-Reply-To: <CAHk-=wgNG=F3xO9PjL0RcKy3UWvq0Np9uZu+nFUQBAA8So9xdA@mail.gmail.com>
On Tue, Jun 23, 2026 at 11:56:10AM -0700, Linus Torvalds wrote:
> On Tue, 23 Jun 2026 at 09:51, Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > We're aware of that and are taking it into account in the allowlist:
>
> Note that if we can just unconditionally make it depend on
> CAP_NET_ADMIN, that would be good - independently of any allowlist.
>
> Because if iwd and abluetoothd are the main two users, and both of
> those already require CAP_NET_ADMIN anyway...
There's also cryptsetup, including unprivileged benchmarking and also
(in theory) formatting support, and pre-7.0 versions of iproute2 which
used it for computing SHA-1 hashes of BPF programs.
If we broke unprivileged 'cryptsetup benchmark', some people would
definitely notice. However, since it's just a manually-run benchmark
anyway, users could just run it with sudo.
I don't know about the iproute2 case.
It depends how aggressive we want to be. My current proposal
(https://lore.kernel.org/linux-crypto/20260622234803.6982-1-ebiggers@kernel.org/)
has the entries in the allowlist marked as either privileged or
unprivileged. There are just a few unprivileged ones, for cryptsetup
and iproute2 as mentioned. But we could try doing away with the
unprivileged ones entirely and see who complains.
- Eric
^ permalink raw reply
* Re: [PATCH] crypto: af_alg - Document the deprecation of AF_ALG
From: Linus Torvalds @ 2026-06-23 18:56 UTC (permalink / raw)
To: Eric Biggers
Cc: Bastien Nocera, linux-crypto, Herbert Xu, Marcel Holtmann,
Luiz Augusto von Dentz, linux-doc, linux-api, linux-kernel,
netdev, linux-bluetooth, ell
In-Reply-To: <20260623164932.GA1793@sol>
On Tue, 23 Jun 2026 at 09:51, Eric Biggers <ebiggers@kernel.org> wrote:
>
> We're aware of that and are taking it into account in the allowlist:
Note that if we can just unconditionally make it depend on
CAP_NET_ADMIN, that would be good - independently of any allowlist.
Because if iwd and abluetoothd are the main two users, and both of
those already require CAP_NET_ADMIN anyway...
Linus
^ permalink raw reply
* [PATCH bpf 1/2] bpf, sockmap: Don't leak UDP socks on lookup-bind-release
From: Michal Luczaj @ 2026-06-23 18:03 UTC (permalink / raw)
To: John Fastabend, Jakub Sitnicki, Jiayuan Chen, David S. Miller,
Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
Alexei Starovoitov, Cong Wang, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan
Cc: netdev, bpf, linux-kernel, linux-kselftest, Michal Luczaj
In-Reply-To: <20260623-sockmap-lookup-udp-leak-v1-0-05804f9308e4@rbox.co>
UDP sockets get SOCK_RCU_FREE set when (auto-)bound. This means
sk_is_refcounted(unbound) = true, while sk_is_refcounted(bound) = false.
Because sockmap accepts unbound UDP sockets, a BPF program can increment a
socket's refcount via lookup. If the socket is subsequently bound, the
transition from unbound to bound causes bpf_sk_release() to skip the
decrement of the refcount, causing a memory leak.
unreferenced object 0xffff88810bc2eb40 (size 1984):
comm "test_progs", pid 2451, jiffies 4295320596
hex dump (first 32 bytes):
7f 00 00 01 7f 00 00 01 d2 04 1b b7 04 d2 00 00 ................
02 00 01 40 00 00 00 00 00 00 00 00 00 00 00 00 ...@............
backtrace (crc bdee079d):
kmem_cache_alloc_noprof+0x557/0x660
sk_prot_alloc+0x69/0x240
sk_alloc+0x30/0x460
inet_create+0x2ce/0xf80
__sock_create+0x25b/0x5c0
__sys_socket+0x119/0x1d0
__x64_sys_socket+0x72/0xd0
do_syscall_64+0xa1/0x5f0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
Maintain balanced refcounts across sk lookup/release: (re-)set
SOCK_RCU_FREE on proto update to treat the socket (whether bound or
unbound) as not requiring a refcount increment on (a RCU protected) lookup.
Fixes: 0c48eefae712 ("sock_map: Lift socket state restriction for datagram sockets")
Signed-off-by: Michal Luczaj <mhal@rbox.co>
---
Note: this issue is related to commit 67312adc96b5 ("bpf: reject unhashed
sockets in bpf_sk_assign").
---
net/ipv4/udp_bpf.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/net/ipv4/udp_bpf.c b/net/ipv4/udp_bpf.c
index ad57c4c9eaab..970327b59582 100644
--- a/net/ipv4/udp_bpf.c
+++ b/net/ipv4/udp_bpf.c
@@ -173,6 +173,9 @@ int udp_bpf_update_proto(struct sock *sk, struct sk_psock *psock, bool restore)
if (sk->sk_family == AF_INET6)
udp_bpf_check_v6_needs_rebuild(psock->sk_proto);
+ /* Treat all sockets as non-refcounted, regardless of binding state. */
+ sock_set_flag(sk, SOCK_RCU_FREE);
+
sock_replace_proto(sk, &udp_bpf_prots[family]);
return 0;
}
--
2.54.0
^ permalink raw reply related
* [PATCH net 1/1] net/sched: sch_teql: Introduce slaves_lock to avoid race condition and UAF
From: Jamal Hadi Salim @ 2026-06-23 18:42 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, victor, andrew+netdev,
zdi-disclosures, security, stable, Jamal Hadi Salim
The teql master->slaves singly linked list is not protected against multiple
writes. It can be mod'ed concurently from teql_master_xmit(), teql_dequeue(),
teql_init() and teql_destroy() without holding any list lock or RCU protection.
zdi-disclosures@trendmicro.com has demonstrated that the qdisc is freed
after an RCU grace period, but teql_master_xmit() running on another
CPU can still hold a stale pointer into the list, resulting in a
slab-use-after-free:
BUG: KASAN: slab-use-after-free in teql_destroy+0x3ca/0x440 linux/net/sched/sch_teql.c:142
Read of size 8 at addr ffff88802923aa80 by task ip/10024
CPU: 1 UID: 0 PID: 10024 Comm: ip Not tainted 7.1.0-rc5 #1 PREEMPT(lazy)
Hardware name: QEMU Ubuntu 25.10 PC v2 (i440FX + PIIX, + 10.1 machine, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Call Trace:
<TASK>
__dump_stack linux/lib/dump_stack.c:94
dump_stack_lvl+0x100/0x190 linux/lib/dump_stack.c:120
print_address_description linux/mm/kasan/report.c:378
print_report+0x139/0x4ad linux/mm/kasan/report.c:482
kasan_report+0xe4/0x1d0 linux/mm/kasan/report.c:595
teql_destroy+0x3ca/0x440 linux/net/sched/sch_teql.c:142
__qdisc_destroy+0x109/0x540 linux/net/sched/sch_generic.c:1100
qdisc_put+0xad/0xf0 linux/net/sched/sch_generic.c:1128
dev_shutdown+0x1cd/0x450 linux/net/sched/sch_generic.c:1493
unregister_netdevice_many_notify+0xd30/0x24b0 linux/net/core/dev.c:12409
rtnl_delete_link linux/net/core/rtnetlink.c:3552
rtnl_dellink+0x476/0xb50 linux/net/core/rtnetlink.c:3594
rtnetlink_rcv_msg+0x954/0xe80 linux/net/core/rtnetlink.c:6997
netlink_rcv_skb+0x156/0x420 linux/net/netlink/af_netlink.c:2550
netlink_unicast_kernel linux/net/netlink/af_netlink.c:1318
netlink_unicast+0x58d/0x860 linux/net/netlink/af_netlink.c:1344
netlink_sendmsg+0x89a/0xd80 linux/net/netlink/af_netlink.c:1894
sock_sendmsg_nosec linux/net/socket.c:787
__sock_sendmsg linux/net/socket.c:802
____sys_sendmsg+0x9d9/0xb70 linux/net/socket.c:2698
___sys_sendmsg+0x194/0x1e0 linux/net/socket.c:2752
__sys_sendmsg+0x171/0x220 linux/net/socket.c:2784
do_syscall_x64 linux/arch/x86/entry/syscall_64.c:63
do_syscall_64+0xff/0x890 linux/arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f linux/arch/x86/entry/entry_64.S:121
[..]
The zdi-disclosures@trendmicro.com repro created concurrent AF_PACKET senders
on a teql device against a thread that repeatedly adds/deletes the slave qdisc,
together with a SLUB spray that reclaims the freed slot; the resulting
UAF is controllable enough to be turned into a read/write primitive against the
freed qdisc object.
The fix?
Add a per-master slaves_lock spinlock that serializes all mutations of
master->slaves and the NEXT_SLAVE() links in teql_destroy() and
teql_qdisc_init(). teql_master_xmit() also takes the same slaves_lock around
those updates.
Pair this with READ_ONCE()/WRITE_ONCE() on the shared pointers and
rcu_read_lock_bh()/rcu_read_unlock_bh() around the list traversal in
teql_master_xmit() and teql_dequeue(), so that readers either observe a fully
linked list or are deferred until the in-flight mutation completes. The two
early-return paths in teql_master_xmit() are updated to release the RCU-bh
read-side critical section before returning, since leaving it held would disable
BH on that CPU for good.
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Reported-by: zdi-disclosures@trendmicro.com
Tested-by: Victor Nogueira <victor@mojatatu.com>
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
---
net/sched/sch_teql.c | 71 +++++++++++++++++++++++++++++---------------
1 file changed, 47 insertions(+), 24 deletions(-)
diff --git a/net/sched/sch_teql.c b/net/sched/sch_teql.c
index e7bbc9e5174d..dacdc46637df 100644
--- a/net/sched/sch_teql.c
+++ b/net/sched/sch_teql.c
@@ -53,6 +53,7 @@ struct teql_master {
struct Qdisc_ops qops;
struct net_device *dev;
struct Qdisc *slaves;
+ spinlock_t slaves_lock; /* serializes writes to ->slaves */
struct list_head master_list;
unsigned long tx_bytes;
unsigned long tx_packets;
@@ -101,7 +102,9 @@ teql_dequeue(struct Qdisc *sch)
if (skb == NULL) {
struct net_device *m = qdisc_dev(q);
if (m) {
- dat->m->slaves = sch;
+ spin_lock_bh(&dat->m->slaves_lock);
+ rcu_assign_pointer(dat->m->slaves, sch);
+ spin_unlock_bh(&dat->m->slaves_lock);
netif_wake_queue(m);
}
} else {
@@ -132,34 +135,37 @@ teql_destroy(struct Qdisc *sch)
struct Qdisc *q, *prev;
struct teql_sched_data *dat = qdisc_priv(sch);
struct teql_master *master = dat->m;
+ struct netdev_queue *txq = NULL;
+ bool reset_master_queue = false;
if (!master)
return;
- prev = master->slaves;
+ spin_lock_bh(&master->slaves_lock);
+ prev = READ_ONCE(master->slaves);
if (prev) {
do {
- q = NEXT_SLAVE(prev);
+ q = READ_ONCE(NEXT_SLAVE(prev));
if (q == sch) {
- NEXT_SLAVE(prev) = NEXT_SLAVE(q);
- if (q == master->slaves) {
- master->slaves = NEXT_SLAVE(q);
- if (q == master->slaves) {
- struct netdev_queue *txq;
-
+ WRITE_ONCE(NEXT_SLAVE(prev), READ_ONCE(NEXT_SLAVE(q)));
+ if (q == READ_ONCE(master->slaves)) {
+ WRITE_ONCE(master->slaves, READ_ONCE(NEXT_SLAVE(q)));
+ if (q == READ_ONCE(master->slaves)) {
txq = netdev_get_tx_queue(master->dev, 0);
- master->slaves = NULL;
-
- dev_reset_queue(master->dev,
- txq, NULL);
+ WRITE_ONCE(master->slaves, NULL);
+ reset_master_queue = true;
}
}
skb_queue_purge(&dat->q);
break;
}
- } while ((prev = q) != master->slaves);
+ } while ((prev = q) != READ_ONCE(master->slaves));
}
+ spin_unlock_bh(&master->slaves_lock);
+
+ if (reset_master_queue)
+ dev_reset_queue(master->dev, txq, NULL);
}
static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
@@ -184,7 +190,8 @@ static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
skb_queue_head_init(&q->q);
- if (m->slaves) {
+ spin_lock_bh(&m->slaves_lock);
+ if (READ_ONCE(m->slaves)) {
if (m->dev->flags & IFF_UP) {
if ((m->dev->flags & IFF_POINTOPOINT &&
!(dev->flags & IFF_POINTOPOINT)) ||
@@ -192,8 +199,10 @@ static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
!(dev->flags & IFF_BROADCAST)) ||
(m->dev->flags & IFF_MULTICAST &&
!(dev->flags & IFF_MULTICAST)) ||
- dev->mtu < m->dev->mtu)
+ dev->mtu < m->dev->mtu) {
+ spin_unlock_bh(&m->slaves_lock);
return -EINVAL;
+ }
} else {
if (!(dev->flags&IFF_POINTOPOINT))
m->dev->flags &= ~IFF_POINTOPOINT;
@@ -204,14 +213,15 @@ static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
if (dev->mtu < m->dev->mtu)
m->dev->mtu = dev->mtu;
}
- q->next = NEXT_SLAVE(m->slaves);
- NEXT_SLAVE(m->slaves) = sch;
+ WRITE_ONCE(q->next, READ_ONCE(NEXT_SLAVE(m->slaves)));
+ rcu_assign_pointer(NEXT_SLAVE(m->slaves), sch);
} else {
- q->next = sch;
- m->slaves = sch;
+ WRITE_ONCE(q->next, sch);
+ rcu_assign_pointer(m->slaves, sch);
m->dev->mtu = dev->mtu;
m->dev->flags = (m->dev->flags&~FMASK)|(dev->flags&FMASK);
}
+ spin_unlock_bh(&m->slaves_lock);
return 0;
}
@@ -285,7 +295,9 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
int subq = skb_get_queue_mapping(skb);
struct sk_buff *skb_res = NULL;
- start = master->slaves;
+ rcu_read_lock_bh();
+
+ start = rcu_dereference_bh(master->slaves);
restart:
nores = 0;
@@ -317,10 +329,14 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
netdev_start_xmit(skb, slave, slave_txq, false) ==
NETDEV_TX_OK) {
__netif_tx_unlock(slave_txq);
- master->slaves = NEXT_SLAVE(q);
+ spin_lock_bh(&master->slaves_lock);
+ rcu_assign_pointer(master->slaves,
+ rcu_dereference_bh(NEXT_SLAVE(q)));
+ spin_unlock_bh(&master->slaves_lock);
netif_wake_queue(dev);
master->tx_packets++;
master->tx_bytes += length;
+ rcu_read_unlock_bh();
return NETDEV_TX_OK;
}
__netif_tx_unlock(slave_txq);
@@ -329,14 +345,18 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
busy = 1;
break;
case 1:
- master->slaves = NEXT_SLAVE(q);
+ spin_lock_bh(&master->slaves_lock);
+ rcu_assign_pointer(master->slaves,
+ rcu_dereference_bh(NEXT_SLAVE(q)));
+ spin_unlock_bh(&master->slaves_lock);
+ rcu_read_unlock_bh();
return NETDEV_TX_OK;
default:
nores = 1;
break;
}
__skb_pull(skb, skb_network_offset(skb));
- } while ((q = NEXT_SLAVE(q)) != start);
+ } while ((q = rcu_dereference_bh(NEXT_SLAVE(q))) != start);
if (nores && skb_res == NULL) {
skb_res = skb;
@@ -345,12 +365,14 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
if (busy) {
netif_stop_queue(dev);
+ rcu_read_unlock_bh();
return NETDEV_TX_BUSY;
}
master->tx_errors++;
drop:
master->tx_dropped++;
+ rcu_read_unlock_bh();
dev_kfree_skb(skb);
return NETDEV_TX_OK;
}
@@ -444,6 +466,7 @@ static __init void teql_master_setup(struct net_device *dev)
struct teql_master *master = netdev_priv(dev);
struct Qdisc_ops *ops = &master->qops;
+ spin_lock_init(&master->slaves_lock);
master->dev = dev;
ops->priv_size = sizeof(struct teql_sched_data);
--
2.54.0
^ permalink raw reply related
* [PATCH bpf 2/2] selftests/bpf: Add test for UDP sock leak on sockmap lookup-bind-release
From: Michal Luczaj @ 2026-06-23 18:03 UTC (permalink / raw)
To: John Fastabend, Jakub Sitnicki, Jiayuan Chen, David S. Miller,
Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
Alexei Starovoitov, Cong Wang, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan
Cc: netdev, bpf, linux-kernel, linux-kselftest, Michal Luczaj
In-Reply-To: <20260623-sockmap-lookup-udp-leak-v1-0-05804f9308e4@rbox.co>
Setup and join a cgroup, then attach a cgroup/connect4 program that runs
sk = bpf_map_lookup_elem(sockmap, 0)
bpf_bind(ctx, sa, sizeof(sa))
bpf_sk_release(sk)
Unpatched kernel leaks the socket.
Signed-off-by: Michal Luczaj <mhal@rbox.co>
---
.../selftests/bpf/prog_tests/sockmap_basic.c | 50 ++++++++++++++++++++++
.../bpf/progs/test_sockmap_lookup_bind_release.c | 37 ++++++++++++++++
2 files changed, 87 insertions(+)
diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c
index cb3229711f93..11972ffdb16e 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c
@@ -7,6 +7,7 @@
#include "test_progs.h"
#include "test_skmsg_load_helpers.skel.h"
+#include "test_sockmap_lookup_bind_release.skel.h"
#include "test_sockmap_update.skel.h"
#include "test_sockmap_invalid_update.skel.h"
#include "test_sockmap_skb_verdict_attach.skel.h"
@@ -17,6 +18,7 @@
#include "test_sockmap_msg_pop_data.skel.h"
#include "bpf_iter_sockmap.skel.h"
+#include "cgroup_helpers.h"
#include "sockmap_helpers.h"
#define TCP_REPAIR 19 /* TCP sock is under repair right now */
@@ -1373,6 +1375,52 @@ static void test_sockmap_multi_channels(int sotype)
test_sockmap_pass_prog__destroy(skel);
}
+#define LOOKUP_BIND_RELEASE_CG "/sockmap_lookup-bind-release"
+#define LOOKUP_BIND_RELEASE_REP 64
+
+static void test_sockmap_lookup_bind_release(void)
+{
+ struct test_sockmap_lookup_bind_release *skel;
+ struct sockaddr_in sa;
+ int cg, i;
+
+ cg = cgroup_setup_and_join(LOOKUP_BIND_RELEASE_CG);
+ if (!ASSERT_OK_FD(cg, "cgroup_setup_and_join"))
+ return;
+
+ skel = test_sockmap_lookup_bind_release__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto cleanup;
+
+ skel->links.connect = bpf_program__attach_cgroup(skel->progs.connect, cg);
+ if (!ASSERT_OK_PTR(skel->links.connect, "attach_cgroup"))
+ goto destroy;
+
+ sa.sin_family = AF_INET;
+ sa.sin_port = bpf_htons(1234);
+ sa.sin_addr.s_addr = bpf_htonl(INADDR_LOOPBACK);
+
+ for (i = 0; i < LOOKUP_BIND_RELEASE_REP; ++i) {
+ __close_fd int sk;
+
+ sk = xsocket(AF_INET, SOCK_DGRAM, 0);
+ if (sk < 0)
+ break;
+
+ if (xbpf_map_update_elem(bpf_map__fd(skel->maps.sockmap), &u32(0),
+ &sk, BPF_ANY))
+ break;
+
+ if (xconnect(sk, (struct sockaddr *)&sa, sizeof(sa)))
+ break;
+ }
+
+destroy:
+ test_sockmap_lookup_bind_release__destroy(skel);
+cleanup:
+ cleanup_cgroup_environment();
+}
+
void test_sockmap_basic(void)
{
if (test__start_subtest("sockmap create_update_free"))
@@ -1451,4 +1499,6 @@ void test_sockmap_basic(void)
test_sockmap_multi_channels(SOCK_STREAM);
if (test__start_subtest("sockmap udp multi channels"))
test_sockmap_multi_channels(SOCK_DGRAM);
+ if (test__start_subtest("sockmap lookup-bind-release"))
+ test_sockmap_lookup_bind_release();
}
diff --git a/tools/testing/selftests/bpf/progs/test_sockmap_lookup_bind_release.c b/tools/testing/selftests/bpf/progs/test_sockmap_lookup_bind_release.c
new file mode 100644
index 000000000000..cc77b193893b
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_sockmap_lookup_bind_release.c
@@ -0,0 +1,37 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <linux/in.h>
+#include <bpf/bpf_helpers.h>
+
+struct {
+ __uint(type, BPF_MAP_TYPE_SOCKMAP);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, int);
+} sockmap SEC(".maps");
+
+SEC("cgroup/connect4")
+int connect(struct bpf_sock_addr *ctx)
+{
+ struct bpf_sock *sk;
+ int ret = SK_DROP;
+
+ sk = bpf_map_lookup_elem(&sockmap, &(int){0});
+ if (sk) {
+ if (sk == ctx->sk) {
+ struct sockaddr_in sa = {
+ .sin_family = ctx->user_family,
+ .sin_port = ctx->user_port,
+ .sin_addr.s_addr = ctx->user_ip4
+ };
+
+ ret = !bpf_bind(ctx, (struct sockaddr *)&sa, sizeof(sa));
+ }
+
+ bpf_sk_release(sk);
+ }
+
+ return ret;
+}
+
+char _license[] SEC("license") = "GPL";
--
2.54.0
^ permalink raw reply related
* [PATCH bpf 0/2] bpf, sockmap: Fix sockmap leaking UDP socks
From: Michal Luczaj @ 2026-06-23 18:03 UTC (permalink / raw)
To: John Fastabend, Jakub Sitnicki, Jiayuan Chen, David S. Miller,
Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
Alexei Starovoitov, Cong Wang, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan
Cc: netdev, bpf, linux-kernel, linux-kselftest, Michal Luczaj
Fix for UDP sockets refcount asymmetry in sockmap lookup/release.
Accompanied by a selftest.
Signed-off-by: Michal Luczaj <mhal@rbox.co>
---
Michal Luczaj (2):
bpf, sockmap: Don't leak UDP socks on lookup-bind-release
selftests/bpf: Add test for UDP sock leak on sockmap lookup-bind-release
net/ipv4/udp_bpf.c | 3 ++
.../selftests/bpf/prog_tests/sockmap_basic.c | 50 ++++++++++++++++++++++
.../bpf/progs/test_sockmap_lookup_bind_release.c | 37 ++++++++++++++++
3 files changed, 90 insertions(+)
---
base-commit: 12091470c6b4c1c14b2de12dcbae2ada6cb6d20b
change-id: 20260617-sockmap-lookup-udp-leak-bc4e5c5481d7
Best regards,
--
Michal Luczaj <mhal@rbox.co>
^ permalink raw reply
* Re: [PATCH bpf-next v4 1/3] bpf: Add BPF_FIB_LOOKUP_VLAN flag to bpf_fib_lookup() helper
From: Avinash Duduskar @ 2026-06-23 18:28 UTC (permalink / raw)
To: Toke Høiland-Jørgensen, ast, daniel, andrii
Cc: eddyz87, memxor, martin.lau, song, yonghong.song, jolsa, emil,
john.fastabend, sdf, davem, edumazet, kuba, pabeni, horms, shuah,
hawk, yatsenko, leon.hwang, kpsingh, a.s.protopopov, ameryhung,
rongtao, eyal.birger, bpf, netdev, linux-kernel, linux-kselftest,
dsahern
In-Reply-To: <877bnpeaeq.fsf@toke.dk>
Toke Høiland-Jørgensen <toke@redhat.com> writes:
> I think it's better to just move the assignment of params->ifindex
> entirely into bpf_fib_set_fwd_params(), instead of this restore dance.
> That way this can be simplified to:
>
> err = bpf_fib_set_fwd_params(dev, params, flags, mtu);
> if (!err && fwd_dev)
> *fwd_dev = dev;
> return err;
The caller-side restore is ungainly, agreed, but the assignment can't move
all the way into the helper. The early params->ifindex = dev->ifindex
sits above the neighbour lookup on purpose: that is d1c362e1dd68a
("bpf: Always return target ifindex in bpf_fib_lookup"), which took it
out of bpf_fib_set_fwd_params() and put it there so a program still
gets the target ifindex on the BPF_FIB_LKUP_RET_NO_NEIGH path and can
bpf_redirect_neigh() on it. bpf_fib_set_fwd_params() is called only at
the set_fwd_params label, below the NO_NEIGH return (and below the IPv6
NO_SRC_ADDR return), so an assignment living in the helper never runs
on those paths and params->ifindex falls back to the input. That would
change the reported ifindex for plain bpf_fib_lookup() callers hitting
NO_NEIGH, not only the VLAN ones.
I can still get the caller down to your form by keeping the early write
and moving just the VLAN_FAILURE rewind into the helper, with one extra
parameter, the input ifindex saved before the egress write:
err = bpf_fib_set_fwd_params(dev, params, flags, mtu, in_ifindex);
if (!err && fwd_dev)
*fwd_dev = dev;
return err;
and the helper owning the rewind in the unreducible branch:
} else {
params->ifindex = in_ifindex;
return BPF_FIB_LKUP_RET_VLAN_FAILURE;
}
So the restore leaves the caller; the early egress write stays because
NO_NEIGH and NO_SRC_ADDR depend on it.
3/3 adds a NO_NEIGH arm that pins the egress ifindex (input != egress):
with the assignment moved into the helper, that case reports the input
ifindex instead, while the return code stays NO_NEIGH, only the ifindex
flips. It passes with the early write kept.
> If you move the ifdef into the if statement, the if statement can have
> an else-branch that assigns params->ifindex, so you don't need the
> restore dance (see below).
Same constraint: an else-branch inside bpf_fib_set_fwd_params() only
runs when the helper runs, which is never on the NO_NEIGH/NO_SRC_ADDR
returns, so it cannot be the sole writer of the egress ifindex.
Does the in_ifindex version look right to you? The alternative is to
route the error returns through the label so the assignment can live
fully in the helper; threading the return codes back through it works,
but it is its own kind of dance and reads worse to me.
Thanks,
Avinash
^ permalink raw reply
* Re: [PATCH net] nfc: nci: fix out-of-bounds write in nci_target_auto_activated()
From: Simon Horman @ 2026-06-23 17:21 UTC (permalink / raw)
To: sam
Cc: Simon Horman, david, davem, edumazet, kuba, pabeni, oe-linux-nfc,
netdev, linux-kernel, stable
In-Reply-To: <20260622145243.3167276-1-sam@bynar.io>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
nfc: nci: fix out-of-bounds write in nci_target_auto_activated()
This commit adds a bounds check to nci_target_auto_activated() to prevent
an out-of-bounds array write. If ndev->n_targets equals
NCI_MAX_DISCOVERED_TARGETS, it prints a debug message and returns early.
> diff --git a/net/nfc/nci/ntf.c b/net/nfc/nci/ntf.c
> index c96512bb86531..566ca839fa488 100644
> --- a/net/nfc/nci/ntf.c
> +++ b/net/nfc/nci/ntf.c
> @@ -603,6 +603,12 @@ static void nci_target_auto_activated(struct nci_dev *ndev,
> struct nfc_target *target;
> int rc;
>
> + /* This is a new target, check if we've enough room */
> + if (ndev->n_targets == NCI_MAX_DISCOVERED_TARGETS) {
> + pr_debug("not enough room, ignoring new target...\n");
> + return;
[Severity: High]
Does this early return cause the state machine to stall?
Looking at nci_rf_intf_activated_ntf_packet(), the state is transitioned
to NCI_POLL_ACTIVE right before this function is called:
net/nfc/nci/ntf.c:nci_rf_intf_activated_ntf_packet() {
...
atomic_set(&ndev->state, NCI_POLL_ACTIVE);
if (err == NCI_STATUS_OK)
nci_target_auto_activated(ndev, &ntf);
...
}
Because of the early return here, nfc_targets_found() is skipped. Does this
leave the state wedged in NCI_POLL_ACTIVE, permanently blocking future
attempts to stop polling since nci_stop_poll() requires the state to be
NCI_DISCOVERY?
> + }
> +
> target = &ndev->targets[ndev->n_targets];
[ ... ]
[Severity: Critical]
This isn't a bug introduced by this patch, but can the ntf union copy
uninitialized stack memory into persistent device structures?
In net/nfc/nci/ntf.c:nci_rf_discover_ntf_packet(), the local ntf struct
is declared on the stack:
struct nci_rf_discover_ntf ntf;
If the incoming packet has ntf.rf_tech_specific_params_len == 0, the
extraction routines are bypassed, leaving the rf_tech_specific_params union
uninitialized. The struct is then passed to nci_add_new_target():
if (add_target == true)
nci_add_new_target(ndev, &ntf);
This eventually calls nci_add_new_protocol(), which blindly reads from this
union:
nfca_poll = (struct rf_tech_specific_params_nfca_poll *)params;
target->sens_res = nfca_poll->sens_res;
target->sel_res = nfca_poll->sel_res;
target->nfcid1_len = nfca_poll->nfcid1_len;
Could this uninitialized stack garbage be copied into ndev->targets, and
later broadcasted to userspace via Netlink in nfc_targets_found()?
[Severity: Critical]
This isn't a bug introduced by this patch, but does the general bytes length
calculation underflow?
In net/nfc/nci/ntf.c:nci_store_general_bytes_nfc_dep():
ndev->remote_gb_len = min_t(__u8,
(ntf->activation_params.listen_nfc_dep.atr_req_len -
NFC_ATR_REQ_GT_OFFSET), NFC_ATR_REQ_GB_MAXSIZE);
memcpy(ndev->remote_gb,
(ntf->activation_params.listen_nfc_dep.atr_req +
NFC_ATR_REQ_GT_OFFSET), ndev->remote_gb_len);
The length is computed by subtracting NFC_ATR_REQ_GT_OFFSET (14) from
atr_req_len. If atr_req_len is less than 14, this subtraction will
underflow.
Because of the min_t cast, this large underflowed value is bounded to
NFC_ATR_REQ_GB_MAXSIZE (48). The memcpy() then copies 48 bytes starting
from atr_req + 14 into ndev->remote_gb.
Since the source buffer is part of the ntf union allocated on the stack
in nci_rf_intf_activated_ntf_packet(), could this leak up to 48 bytes of
uninitialized kernel stack memory to userspace?
[Severity: High]
This isn't a bug introduced by this patch, but does the lack of skb->len
validation in NCI response handlers allow out-of-bounds reads?
In nci_core_init_rsp_packet_v1() in net/nfc/nci/rsp.c, fields are read
and arrays are copied from the payload:
ndev->num_supported_rf_interfaces =
rsp_1->num_supported_rf_interfaces;
memcpy(ndev->supported_rf_interfaces, rsp_1->supported_rf_interfaces,
ndev->num_supported_rf_interfaces);
Similarly in nci_core_init_rsp_packet_v2():
while (rf_interface_idx < ndev->num_supported_rf_interfaces) {
ndev->supported_rf_interfaces[rf_interface_idx++] =
*supported_rf_interface++;
}
Since the NCI control header is stripped via skb_pull(), leaving the
payload length in skb->len, what happens if an NFCC sends a short response
packet (e.g. plen = 1)? Could this cause the handlers to read adjacent
kernel slab memory?
[Severity: High]
This isn't a bug introduced by this patch, but is there a missing boundary
validation when extracting activation parameters?
In net/nfc/nci/ntf.c:nci_extract_activation_params_iso_dep():
nfca_poll->rats_res_len = min_t(__u8, *data++, NFC_ATS_MAXSIZE);
if (nfca_poll->rats_res_len > 0) {
memcpy(nfca_poll->rats_res, data, nfca_poll->rats_res_len);
}
And in nci_extract_activation_params_nfc_dep():
poll->atr_res_len = min_t(__u8, *data++, NFC_ATR_RES_MAXSIZE - 2);
if (poll->atr_res_len > 0)
memcpy(poll->atr_res, data, poll->atr_res_len);
These inner lengths are bounded by a maximum size like NFC_ATS_MAXSIZE, but
they are not validated against the remaining bytes in the SKB (skb->len).
If a malformed packet provides a large inner length but a short payload,
could this memcpy() read out of bounds into the device structure,
potentially leaking adjacent slab memory to userspace later?
[Severity: High]
This isn't a bug introduced by this patch, but is there a potential
priority inversion or timeout deadlock involving the device lock?
Functions like nfc_stop_poll() in net/nfc/core.c acquire the device lock:
device_lock(&dev->dev);
...
if (dev->ops->stop_poll)
dev->ops->stop_poll(dev);
This eventually calls nci_request(), which synchronously waits for a
completion signaled by the nci_rx_work thread. However, if an NTF packet
is received first, nci_rx_work processes it and invokes
nfc_targets_found(), which also attempts to acquire the device lock:
device_lock(&dev->dev);
Since the calling thread already holds the device lock, nci_rx_work blocks
indefinitely. Because the RX worker is blocked, it cannot process the
pending RSP, causing nci_request() to time out and fail. Could this
deadlock the RX thread?
^ permalink raw reply
* Re: [PATCH v12 11/12] x86/vmscape: Resolve conflict between attack-vectors and vmscape=force
From: bot+bpf-ci @ 2026-06-23 18:13 UTC (permalink / raw)
To: pawan.kumar.gupta, x86, jon, nik.borisov, hpa, jpoimboe,
david.kaplan, seanjc, bp, dave.hansen, peterz, ast, daniel,
andrii, kpsingh, jolsa, davem, david.laight.linux, luto, tglx,
mingo, dsahern, martin.lau, eddyz87, song, yonghong.song,
john.fastabend, sdf, haoluo, pbonzini, corbet, jbaron, aliceryhl,
rostedt, ardb, skhan
Cc: linux-kernel, kvm, asit.k.mallick, tao1.zhang, bpf, netdev,
linux-doc, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, clm, ihor.solodrai
In-Reply-To: <20260622-vmscape-bhb-v12-11-76cbda0ae3e5@linux.intel.com>
[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]
> x86/vmscape: Resolve conflict between attack-vectors and vmscape=force
>
> vmscape=force option currently defaults to AUTO mitigation. This lets
> attack-vector controls to override the vmscape mitigation. Preventing the
> user from being able to force VMSCAPE mitigation.
>
> When vmscape mitigation is forced, allow it be deployed irrespective of
> attack vectors. Introduce VMSCAPE_MITIGATION_ON that wins over
> attack-vector controls.
>
> Tested-by: Jon Kohler <jon@nutanix.com>
> Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
This appears to fix a behavioral bug where vmscape=force mapped to
VMSCAPE_MITIGATION_AUTO, which is gated by should_mitigate_vuln(), so
attack-vector controls could override a user-forced mitigation.
Should this carry a Fixes: tag?
The force -> VMSCAPE_MITIGATION_AUTO mapping that allowed the override
predates this series. The earlier refactor 68d94c61c09a ("x86/vmscape:
Move mitigation selection to a switch()") only moved the logic into a
switch() and did not introduce the conflict, so it does not look like the
right Fixes: target. A precise commit to point at could not be confirmed
from this tree.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28046221590
^ permalink raw reply
* Re: [PATCH v12 02/12] x86/bhi: Make clear_bhb_loop() effective on newer CPUs
From: bot+bpf-ci @ 2026-06-23 18:13 UTC (permalink / raw)
To: pawan.kumar.gupta, x86, jon, nik.borisov, hpa, jpoimboe,
david.kaplan, seanjc, bp, dave.hansen, peterz, ast, daniel,
andrii, kpsingh, jolsa, davem, david.laight.linux, luto, tglx,
mingo, dsahern, martin.lau, eddyz87, song, yonghong.song,
john.fastabend, sdf, haoluo, pbonzini, corbet, jbaron, aliceryhl,
rostedt, ardb, skhan
Cc: linux-kernel, kvm, asit.k.mallick, tao1.zhang, bpf, netdev,
linux-doc, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, clm, ihor.solodrai
In-Reply-To: <20260622-vmscape-bhb-v12-2-76cbda0ae3e5@linux.intel.com>
[-- Attachment #1: Type: text/plain, Size: 3513 bytes --]
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index 83f51cab0b1e..2cb4a96247d8 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
[ ... ]
> @@ -3242,6 +3246,15 @@ void __init cpu_select_mitigations(void)
> x86_spec_ctrl_base &= ~SPEC_CTRL_MITIGATIONS_MASK;
> }
>
> + /*
> + * Switch to long BHB clear sequence on newer CPUs (with BHI_CTRL
> + * support), see Intel's BHI guidance.
> + */
> + if (cpu_feature_enabled(X86_FEATURE_BHI_CTRL)) {
> + bhb_seq_outer_loop = 12;
> + bhb_seq_inner_loop = 7;
> + }
Should this also consider the case of a guest running under a hypervisor
that does not expose BHI_CTRL?
When BHI_CTRL is absent the loop counts keep their short defaults (5/5)
and the guest declares itself BHI-mitigated. In a heterogeneous
migration pool a guest may present as Ice Lake (no BHI_CTRL) while
actually running on Alder Lake or newer hardware, where the short
sequence is not sufficient.
A default-safe approach would be to also select the long sequence when
running under a hypervisor without BHI_CTRL, for example:
if (cpu_feature_enabled(X86_FEATURE_BHI_CTRL) ||
cpu_feature_enabled(X86_FEATURE_HYPERVISOR)) {
bhb_seq_outer_loop = 12;
bhb_seq_inner_loop = 7;
}
There is precedent for assuming the worst under a hypervisor, such as
vulnerable_to_its(), cpu_set_bug_bits() and check_null_seg_clears_base().
This came up during review and the resolution was to let the series move
forward and revisit the hypervisor case afterwards. Is that still the
intent, or should the hypervisor fallback land with this patch?
For the missing Fixes: tag, quoting the full commit message:
commit a2a0812335ee19ae1df81f88e736da5210748c2b
Author: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
> x86/bhi: Make clear_bhb_loop() effective on newer CPUs
>
> As a mitigation for BHI, clear_bhb_loop() executes branches that overwrite
> the Branch History Buffer (BHB). On Alder Lake and newer parts this
> sequence is not sufficient because it doesn't clear enough entries. This
> was not an issue because these CPUs use the BHI_DIS_S hardware mitigation
> in the kernel.
> Now with VMSCAPE (BHI variant) it is also required to isolate branch
> history between guests and userspace. Since BHI_DIS_S only protects the
> kernel, the newer CPUs also use IBPB.
> A cheaper alternative to the current IBPB mitigation is clear_bhb_loop().
> But it currently does not clear enough BHB entries to be effective on newer
> CPUs with larger BHB. At boot, dynamically set the loop count of
> clear_bhb_loop() such that it is effective on newer CPUs too.
> Introduce global loop counts, initializing them with appropriate value
> based on the hardware feature X86_FEATURE_BHI_CTRL.
> Suggested-by: Dave Hansen <dave.hansen@linux.intel.com>
> Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
The short BHB clear sequence with the hardcoded loop count of 5 was
added by clear_bhb_loop() in entry_64.S, which is the code this patch
makes configurable. Should this carry:
Fixes: 7390db8aea0d ("x86/bhi: Add support for clearing branch history at syscall entry")
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28046221590
^ permalink raw reply
* Re: [PATCH v12 08/12] KVM: Define EXPORT_STATIC_CALL_FOR_KVM()
From: bot+bpf-ci @ 2026-06-23 18:13 UTC (permalink / raw)
To: pawan.kumar.gupta, x86, jon, nik.borisov, hpa, jpoimboe,
david.kaplan, seanjc, bp, dave.hansen, peterz, ast, daniel,
andrii, kpsingh, jolsa, davem, david.laight.linux, luto, tglx,
mingo, dsahern, martin.lau, eddyz87, song, yonghong.song,
john.fastabend, sdf, haoluo, pbonzini, corbet, jbaron, aliceryhl,
rostedt, ardb, skhan
Cc: linux-kernel, kvm, asit.k.mallick, tao1.zhang, bpf, netdev,
linux-doc, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, clm, ihor.solodrai
In-Reply-To: <20260622-vmscape-bhb-v12-8-76cbda0ae3e5@linux.intel.com>
[-- Attachment #1: Type: text/plain, Size: 2597 bytes --]
> EXPORT_SYMBOL_FOR_KVM() exists to export symbols to KVM modules. Static
> calls need the same treatment when the core kernel defines a static_call
> that KVM needs access to (e.g. from a VM-exit path).
>
> Define EXPORT_STATIC_CALL_FOR_KVM() as the static_call analogue of
> EXPORT_SYMBOL_FOR_KVM(). The same three-way logic applies:
>
> - KVM_SUB_MODULES defined: export to "kvm," plus all sub-modules
> - KVM=m, no sub-modules: export to "kvm" only
> - KVM built-in: no export needed (noop)
>
> As with EXPORT_SYMBOL_FOR_KVM(), allow architectures to override both
> macros (e.g. to suppress the export when kvm.ko itself will not be
> built despite CONFIG_KVM=m). Add the x86 no-op overrides in
> arch/x86/include/asm/kvm_types.h for that case. To keep the pair in
> sync, EXPORT_STATIC_CALL_FOR_KVM() is defined inside the
> EXPORT_SYMBOL_FOR_KVM #ifndef block; an arch that defines
> EXPORT_SYMBOL_FOR_KVM must also define EXPORT_STATIC_CALL_FOR_KVM or the
> build will fail with a compile-time error.
>
> As with EXPORT_SYMBOL_FOR_KVM(), allow architectures to override
> EXPORT_STATIC_CALL_FOR_KVM definition (e.g. to suppress the export when
> kvm.ko itself will not be built despite CONFIG_KVM=m). Add the x86 no-op
> override in arch/x86/include/asm/kvm_types.h for that case.
>
> Architectures must also define EXPORT_STATIC_CALL_FOR_KVM when they define
> EXPORT_SYMBOL_FOR_KVM.
This isn't a bug, but the arch-override rationale appears twice in nearly
identical form. The third paragraph (starting "As with
EXPORT_SYMBOL_FOR_KVM(), allow architectures to override both macros") and
the fourth paragraph (starting "As with EXPORT_SYMBOL_FOR_KVM(), allow
architectures to override EXPORT_STATIC_CALL_FOR_KVM definition") both
explain suppressing the export when kvm.ko will not be built and both
mention adding the x86 no-op override.
The requirement that an arch defining EXPORT_SYMBOL_FOR_KVM must also
define EXPORT_STATIC_CALL_FOR_KVM also shows up twice: once inside the
third paragraph ("an arch that defines EXPORT_SYMBOL_FOR_KVM must also
define EXPORT_STATIC_CALL_FOR_KVM or the build will fail") and again as the
standalone final sentence ("Architectures must also define
EXPORT_STATIC_CALL_FOR_KVM when they define EXPORT_SYMBOL_FOR_KVM").
Could the duplicate paragraphs be folded into a single statement?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28046221590
^ permalink raw reply
* Re: [PATCH net 1/1] net/sched: cls_api: Handle TC_ACT_CONSUMED in tcf_qevent_handle
From: Jamal Hadi Salim @ 2026-06-23 18:00 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, jiri, victor, security,
Zero Day Initiative
In-Reply-To: <20260620130749.226642-1-jhs@mojatatu.com>
On Sat, Jun 20, 2026 at 9:07 AM Jamal Hadi Salim <jhs@mojatatu.com> wrote:
>
> tcf_classify() can return TC_ACT_CONSUMED while the skb is held by the
> defragmentation engine (e.g. act_ct on out-of-order fragments). When
> that happens the skb is no longer owned by the caller and must not be
> touched again.
>
> tcf_qevent_handle() did not handle TC_ACT_CONSUMED: it fell through the
> switch and returned the skb to the caller as if classification had
> passed. The only qdisc that wires up qevents today is RED, via three call sites
> (qe_mark on RED_PROB_MARK/HARD_MARK, qe_early_drop on congestion_drop)
> red_enqueue() was continuing to operate on an skb it no longer owns in this
> case -- enqueueing it, dropping it, or updating statistics. Resulting in a UAF.
>
> tc qdisc add dev eth0 root handle 1: red ... qevent early_drop block 10
> tc filter add block 10 ... action ct
>
> (with ct defrag enabled and traffic that produces out-of-order
> fragments, e.g. a fragmented UDP stream)
>
> Handle TC_ACT_CONSUMED in tcf_qevent_handle() the same way the ingress
> and egress fast paths do: treat it as stolen and return NULL without
> touching the skb. Unlike the TC_ACT_STOLEN case, the skb must not be
> dropped/freed here, as it is no longer owned by us.
>
I just looked at sashiko claims - one of them (on ebpf) is legit but
the one on qdiscs is some BS it is making up. I will address the ebpf
one this week.
cheers,
jamal
> Fixes: 3f14b377d01d ("net/sched: act_ct: fix skb leak and crash on ooo frags")
> Reported-by: Zero Day Initiative <zdi-disclosures@trendmicro.com>
> Tested-by: Victor Nogueira <victor@mojatatu.com>
> Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
> ---
> net/sched/cls_api.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
> index 20f7f9ee0b353..3e67600a4a1a1 100644
> --- a/net/sched/cls_api.c
> +++ b/net/sched/cls_api.c
> @@ -4049,6 +4049,9 @@ struct sk_buff *tcf_qevent_handle(struct tcf_qevent *qe, struct Qdisc *sch, stru
> skb_do_redirect(skb);
> *ret = __NET_XMIT_STOLEN;
> return NULL;
> + case TC_ACT_CONSUMED:
> + *ret = __NET_XMIT_STOLEN;
> + return NULL;
> }
>
> return skb;
> --
> 2.34.1
>
^ permalink raw reply
* Re: [PATCH v14 0/9] tls: Add TLS 1.3 hardware offload support
From: Nils Juenemann @ 2026-06-23 17:53 UTC (permalink / raw)
To: rjethwani, netdev
Cc: borisp, davem, edumazet, john.fastabend, kuba, leon, mbloch,
saeedm, sd, tariqt
Hi Rishikesh, all,
we have been testing the v14 TLS 1.3 HW offload series on a ConnectX-6
DX and hit a sendfile() final-record loss on the device TX path. We
reduced it to a self-contained C reproducer and characterized it;
reporting it here with the analysis and a question on where a fix belongs.
Setup:
NIC: ConnectX-6 DX (crypto enabled), FW 22.47.1026, SR-IOV VF,
TX offload only
Kernel: net-next + this v14 series
TLS 1.3, AES-128-GCM, kTLS installed via setsockopt(TLS_TX) on the
sending side with fixed test crypto material and no handshake, like
tools/testing/selftests/net/tls
a server sends a file with the raw sendfile(2) syscall; a client on
another host reads the decrypted stream and counts the bytes
Trigger: sendfile(2) with a count larger than the bytes remaining in
the file (count > EOF). This is what a generic copy loop / Go's
net.TCPConn.ReadFrom passes for a file of unknown length (~2 GiB). The
kernel sends up to EOF, but the connection's final TLS record then
appears not to be put on the wire unless a subsequent write flushes it.
An abrupt close() appears to drop it, and the peer receives the whole
body except the last record's bytes.
Reproducer results (two hosts over the ConnectX - a loopback/same-host
connection stays on TLS_SW and does not show it). Same file, 226965
bytes (= 13*16384 + 13973):
TLS_HW count>EOF close() -> 212992 short
TLS_HW count>EOF close(), no zerocopy -> 212992 same
TLS_HW count==exact close() -> 226965 full
TLS_HW count>EOF close_notify, then close() -> 226965 full
TLS_SW count>EOF close(), hw-tx-offload off -> 226965 full
So it is specific to the device-offload TX path: the final record of a
count > EOF sendfile() appears not to be finalized/flushed at EOF, only
by a following write. A bounded count, a trailing write (close_notify),
or software kTLS all avoid it. TLS_TX_ZEROCOPY_RO makes no difference.
We are currently using the exact-count workaround in a preview environment.
We may be misreading the code, so this is only a pointer: with
count > EOF tls_push_data() fills the last record without reaching the
size==0 case; on the device path tls_device_record_close() for that
pending record appears to run only on the next push, and an abrupt
teardown appears to discard it. The software path seems to flush
pending TX records on close (tls_sw_release_resources_tx), which would
explain why it is unaffected.
Reproducer:
https://gist.github.com/totallyunknown/a8f0ad3c54e40befde2f5a8d360fa6be
It installs kTLS with fixed test crypto material via
setsockopt(TLS_TX/TLS_RX), sends a file using the raw sendfile(2)
syscall, and compares count > EOF against exact-count and close_notify.
The v14 selftest (patch 9/9) sends via send() only and ends cleanly, so
it misses this; a sendfile() + count > EOF case reproduces it
deterministically for us.
Question: should the device offload finalize and flush the connection's
final record at EOF / on close, the way software kTLS does, or is a
trailing write required by contract? And should a fix live in net/tls
(device record close on the final partial record / the close path) or
on the mlx5 side?
Thanks,
Nils Juenemann
^ permalink raw reply
* Re: [PATCH net] net/mlx5e: Use sender devcom for MPV master-up
From: manjunath.b.patil @ 2026-06-23 17:51 UTC (permalink / raw)
To: Tariq Toukan, Saeed Mahameed, Mark Bloch, Leon Romanovsky, netdev
Cc: Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Patrisious Haddad, linux-rdma, linux-kernel, stable
In-Reply-To: <293db0b4-f308-469e-99c1-ef1b57d41451@nvidia.com>
On 6/22/26 2:01 AM, Tariq Toukan wrote:
>
>
> On 10/06/2026 20:39, Manjunath Patil wrote:
>> After PCIe DPC recovery, mlx5 reloads the affected functions and
>> replays multiport affiliation events. In the reported failure, the
>> first relevant device error was:
>>
>> pcieport 0000:10:01.1: DPC: containment event
>> pcieport 0000:10:01.1: PCIe Bus Error: severity=Uncorrected (Fatal)
>> pcieport 0000:10:01.1: [ 5] SDES (First)
>>
>> mlx5 recovered the PCI functions and resumed 0000:11:00.1. During
>> that resume, RDMA multiport binding replayed
>> MLX5_DRIVER_EVENT_AFFILIATION_DONE and mlx5e sent
>> MPV_DEVCOM_MASTER_UP. The host then panicked with:
>>
>> BUG: kernel NULL pointer dereference, address: 0000000000000010
>> RIP: mlx5_devcom_comp_set_ready+0x5/0x40 [mlx5_core]
>> RDI: 0000000000000000
>>
>> Call trace included:
>>
>> mlx5_devcom_comp_set_ready
>> mlx5e_devcom_event_mpv
>> mlx5_devcom_send_event
>> mlx5_ib_bind_slave_port
>> mlx5r_mp_probe
>> mlx5_pci_resume
>>
>> MPV devcom registration publishes mlx5e private data to the component
>> peer list before mlx5e_devcom_init_mpv() stores the returned component
>> device in priv->devcom. A concurrent master-up event can therefore
>> reach a peer whose private data is visible but whose priv->devcom
>> backpointer is still NULL.
>>
>> MPV_DEVCOM_MASTER_UP already carries the sender/master mlx5e private
>> data as event_data. The ready bit is stored on the shared devcom
>> component, not on an individual peer. Use the sender devcom when
>> marking the MPV component ready.
>>
>> This preserves the readiness transition while avoiding a NULL
>> dereference of the peer devcom pointer during affiliation replay after
>> PCI error recovery.
>>
>> Fixes: bf11485f8419 ("net/mlx5: Register mlx5e priv to devcom in MPV
>> mode")
>> Assisted-by: Codex:gpt-5
>> Signed-off-by: Manjunath Patil <manjunath.b.patil@oracle.com>
>> Cc: stable@vger.kernel.org # 6.7+
>> ---
>
> Thanks for your patch and sorry for the late response.
>
>> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 +++++--
>> 1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/
>> drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> index 8f2b3abe0092..f7ff20b97e8c 100644
>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> @@ -211,11 +211,14 @@ static void mlx5e_disable_async_events(struct
>> mlx5e_priv *priv)
>> static int mlx5e_devcom_event_mpv(int event, void *my_data, void
>> *event_data)
>> {
>> - struct mlx5e_priv *slave_priv = my_data;
>> + struct mlx5e_priv *master_priv = event_data;
>
> makes sense.
>
>> switch (event) {
>> case MPV_DEVCOM_MASTER_UP:
>> - mlx5_devcom_comp_set_ready(slave_priv->devcom, true);
>> + if (!master_priv || !master_priv->devcom)
>> + return -EINVAL;
>
> is this currently possible? or just being defensive?
> if this return is unreachable I'd drop it.
Yes, the check is only defensive. For MPV_DEVCOM_MASTER_UP, event_data
is passed from mlx5e_devcom_init_mpv() after priv->devcom has been
assigned, so it should not be reachable in the valid path.
Please feel free to drop the check while applying. If you prefer a v2,
let me know and I will send one.
Thanks,
Manjunath
>
>> +
>> + mlx5_devcom_comp_set_ready(master_priv->devcom, true);
>> break;
>> case MPV_DEVCOM_MASTER_DOWN:
>> /* no need for comp set ready false since we unregister after
>
^ permalink raw reply
* [PATCH bpf-next v2 15/15] selftests/bpf: Add test for bpf_tcp_ops header option hooks
From: Amery Hung @ 2026-06-23 17:50 UTC (permalink / raw)
To: bpf
Cc: netdev, alexei.starovoitov, andrii, daniel, eddyz87, memxor,
martin.lau, shakeel.butt, roman.gushchin, kuniyu, kerneljasonxing,
ameryhung, kernel-team
In-Reply-To: <20260623175006.3136053-1-ameryhung@gmail.com>
Add a test exercising the bpf_tcp_ops parse_hdr, hdr_opt_len and
write_hdr_opt members together with the header option helpers.
The struct_ops program (progs/bpf_tcp_ops_hdr.c) reserves space in
hdr_opt_len via bpf_reserve_hdr_opt(), writes an experimental option in
write_hdr_opt via bpf_store_hdr_opt(), and recovers it in parse_hdr via
bpf_load_hdr_opt() on the incoming skb. Each hook bumps a counter and the
parse hook records the option payload, so the three callbacks and all
three overloaded helpers are covered.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
---
.../bpf/prog_tests/bpf_tcp_ops_hdr.c | 97 +++++++++++++++++++
.../selftests/bpf/progs/bpf_tcp_ops_hdr.c | 83 ++++++++++++++++
2 files changed, 180 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops_hdr.c
create mode 100644 tools/testing/selftests/bpf/progs/bpf_tcp_ops_hdr.c
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops_hdr.c b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops_hdr.c
new file mode 100644
index 000000000000..73e34d2be9a4
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops_hdr.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <test_progs.h>
+#include <network_helpers.h>
+#include "cgroup_helpers.h"
+#include "bpf_tcp_ops_hdr.skel.h"
+
+#define CGROUP_PATH "/bpf_tcp_ops_hdr"
+#define TEST_NETNS "bpf_tcp_ops_hdr"
+
+#define TEST_OPT_D0 0xAB
+#define TEST_OPT_D1 0xCD
+
+static void send_recv(void)
+{
+ char buf[64] = {};
+ int server_fd, client_fd, accept_fd;
+ ssize_t n;
+
+ server_fd = start_server(AF_INET6, SOCK_STREAM, "::1", 0, 0);
+ if (!ASSERT_GE(server_fd, 0, "start_server"))
+ return;
+
+ client_fd = connect_to_fd(server_fd, 0);
+ if (!ASSERT_OK_FD(client_fd, "connect_to_fd"))
+ goto close_server;
+
+ accept_fd = accept(server_fd, NULL, NULL);
+ if (!ASSERT_OK_FD(accept_fd, "accept"))
+ goto close_client;
+
+ /* Exchange data both directions so option-bearing data packets
+ * are sent and parsed on each side.
+ */
+ n = send(client_fd, buf, sizeof(buf), 0);
+ ASSERT_EQ(n, sizeof(buf), "client_send");
+ n = recv(accept_fd, buf, sizeof(buf), 0);
+ ASSERT_EQ(n, sizeof(buf), "server_recv");
+
+ n = send(accept_fd, buf, sizeof(buf), 0);
+ ASSERT_EQ(n, sizeof(buf), "server_send");
+ n = recv(client_fd, buf, sizeof(buf), 0);
+ ASSERT_EQ(n, sizeof(buf), "client_recv");
+
+ close(accept_fd);
+close_client:
+ close(client_fd);
+close_server:
+ close(server_fd);
+}
+
+static void run_hdr_opt(void)
+{
+ struct bpf_tcp_ops_hdr *skel = NULL;
+ struct bpf_link *link = NULL;
+ struct netns_obj *ns = NULL;
+ int cgroup_fd;
+
+ cgroup_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+ return;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops_hdr__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ link = bpf_map__attach_cgroup_opts(skel->maps.test_hdr_ops, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link, "attach_cgroup"))
+ goto done;
+
+ send_recv();
+
+ /* Reserve + write hooks ran while sending. */
+ ASSERT_GT(skel->bss->hdr_opt_len_cnt, 0, "hdr_opt_len_cnt");
+ ASSERT_GT(skel->bss->write_cnt, 0, "write_cnt");
+ /* Parse hook ran and recovered our option on the receive side. */
+ ASSERT_GT(skel->bss->parse_cnt, 0, "parse_cnt");
+ ASSERT_GT(skel->bss->found_cnt, 0, "found_cnt");
+ ASSERT_EQ(skel->bss->found_d0, TEST_OPT_D0, "found_d0");
+ ASSERT_EQ(skel->bss->found_d1, TEST_OPT_D1, "found_d1");
+
+done:
+ bpf_link__destroy(link);
+ bpf_tcp_ops_hdr__destroy(skel);
+ netns_free(ns);
+ close(cgroup_fd);
+}
+
+void test_bpf_tcp_ops_hdr(void)
+{
+ run_hdr_opt();
+}
diff --git a/tools/testing/selftests/bpf/progs/bpf_tcp_ops_hdr.c b/tools/testing/selftests/bpf/progs/bpf_tcp_ops_hdr.c
new file mode 100644
index 000000000000..46618a604d96
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_tcp_ops_hdr.c
@@ -0,0 +1,83 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+/* Experimental option kind and payload written/parsed by this test. */
+#define TEST_OPT_KIND 0xFD
+#define TEST_OPT_LEN 4
+#define TEST_OPT_D0 0xAB
+#define TEST_OPT_D1 0xCD
+
+int hdr_opt_len_cnt;
+int write_cnt;
+int parse_cnt;
+int found_cnt;
+__u8 found_d0;
+__u8 found_d1;
+
+SEC("struct_ops")
+void BPF_PROG(test_hdr_opt_len, struct sock *sk, struct sk_buff *skb,
+ struct request_sock *req, struct sk_buff *syn_skb,
+ enum tcp_synack_type synack_type, unsigned int *remaining)
+{
+ hdr_opt_len_cnt++;
+
+ /* Reserve TEST_OPT_LEN bytes; the helper decrements *remaining. Stacks
+ * with other progs in the cgroup hierarchy.
+ */
+ bpf_reserve_hdr_opt(ctx, TEST_OPT_LEN, 0);
+}
+
+SEC("struct_ops")
+void BPF_PROG(test_write_hdr_opt, struct sock *sk, struct sk_buff *skb,
+ struct request_sock *req, struct sk_buff *syn_skb,
+ enum tcp_synack_type synack_type, __u32 opt_off)
+{
+ __u8 opt[TEST_OPT_LEN] = {
+ TEST_OPT_KIND, TEST_OPT_LEN, TEST_OPT_D0, TEST_OPT_D1,
+ };
+
+ /* bpf_store_hdr_opt() takes the program ctx (the kernel reads the
+ * outgoing skb from it); it appends after any options already written
+ * in the reserved window, rejects duplicates, and confines the write to
+ * the header option scratch. Stacks across progs in the cgroup hierarchy.
+ */
+ if (bpf_store_hdr_opt(ctx, opt, sizeof(opt), 0))
+ return;
+
+ write_cnt++;
+}
+
+SEC("struct_ops")
+void BPF_PROG(test_parse_hdr, struct sock *sk, struct sk_buff *skb)
+{
+ __u8 opt[TEST_OPT_LEN] = {
+ TEST_OPT_KIND, TEST_OPT_LEN, TEST_OPT_D0, TEST_OPT_D1,
+ };
+
+ parse_cnt++;
+
+ /* Look up the experimental option written by test_write_hdr_opt() in
+ * the incoming skb. For an experimental kind the search matches on the
+ * 2-byte magic in opt[2..3]; on a match the found option is copied back
+ * into opt[].
+ */
+ if (bpf_load_hdr_opt(ctx, opt, sizeof(opt), 0) < 0)
+ return;
+
+ found_d0 = opt[2];
+ found_d1 = opt[3];
+ found_cnt++;
+}
+
+SEC(".struct_ops.link")
+struct bpf_tcp_ops test_hdr_ops = {
+ .hdr_opt_len = (void *)test_hdr_opt_len,
+ .write_hdr_opt = (void *)test_write_hdr_opt,
+ .parse_hdr = (void *)test_parse_hdr,
+};
+
+char _license[] SEC("license") = "GPL";
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH bpf-next v2 14/15] selftests/bpf: Test attaching struct_ops to a cgroup
From: Amery Hung @ 2026-06-23 17:50 UTC (permalink / raw)
To: bpf
Cc: netdev, alexei.starovoitov, andrii, daniel, eddyz87, memxor,
martin.lau, shakeel.butt, roman.gushchin, kuniyu, kerneljasonxing,
ameryhung, kernel-team
In-Reply-To: <20260623175006.3136053-1-ameryhung@gmail.com>
From: Martin KaFai Lau <martin.lau@kernel.org>
Exercise attaching the bpf_tcp_ops struct_ops to cgroups via the generic
cgroup link infrastructure. The struct_ops instances record their
execution order and the previous return value to validate correctness.
Subtests:
- query: BPF_F_QUERY_EFFECTIVE and attached query return the maps
- order: BPF_F_PREORDER vs attach order within a cgroup
- before_after: BPF_F_BEFORE/BPF_F_AFTER relative positioning
- update: bpf_link__update_map swaps a link's map, keeping its slot
- retval: int return value chained across timeout_init progs of
multiple bpf_tcp_ops attached to a cgroup
- hierarchy: parent and child attachments merge in the child's
effective array (descendant before ancestor)
- inherit: a child created after the attach inherits the parent's
prog
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
---
.../selftests/bpf/prog_tests/bpf_tcp_ops.c | 554 ++++++++++++++++++
.../testing/selftests/bpf/progs/bpf_tcp_ops.c | 141 +++++
2 files changed, 695 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops.c
create mode 100644 tools/testing/selftests/bpf/progs/bpf_tcp_ops.c
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops.c b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops.c
new file mode 100644
index 000000000000..4d087bdc4613
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ops.c
@@ -0,0 +1,554 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <test_progs.h>
+#include <network_helpers.h>
+#include <bpf/btf.h>
+#include "cgroup_helpers.h"
+#include "bpf_tcp_ops.skel.h"
+
+#define CGROUP_PATH "/bpf_tcp_ops"
+#define TEST_NETNS "bpf_tcp_ops"
+
+static __s32 get_bpf_tcp_ops_type_id(void)
+{
+ struct btf *vmlinux_btf;
+ __s32 type_id;
+
+ vmlinux_btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(vmlinux_btf, "load_vmlinux_btf"))
+ return -1;
+
+ type_id = btf__find_by_name_kind(vmlinux_btf, "bpf_tcp_ops", BTF_KIND_STRUCT);
+ btf__free(vmlinux_btf);
+
+ ASSERT_GT(type_id, 0, "find_bpf_tcp_ops");
+ return type_id;
+}
+
+static void reset_order(struct bpf_tcp_ops *skel)
+{
+ memset(skel->bss->listen_order, 0, sizeof(skel->bss->listen_order));
+ memset(skel->bss->connect_order, 0, sizeof(skel->bss->connect_order));
+ skel->bss->listen_cnt = 0;
+ skel->bss->connect_cnt = 0;
+}
+
+static void do_listen_connect(int family)
+{
+ const char *addr = family == AF_INET ? "127.0.0.1" : "::1";
+ int server_fd, client_fd;
+
+ server_fd = start_server(family, SOCK_STREAM, addr, 0, 0);
+ if (!ASSERT_GE(server_fd, 0, "start_server"))
+ return;
+
+ client_fd = connect_to_fd(server_fd, 0);
+ if (ASSERT_OK_FD(client_fd, "connect_to_fd"))
+ close(client_fd);
+
+ close(server_fd);
+}
+
+/*
+ * Attach ops1 and ops2 normally (in that order), then ops3 with
+ * BPF_F_PREORDER. Expected execution order: [3, 1, 2] — ops3 runs
+ * first despite being attached last, ops1 before ops2 by attach order.
+ */
+static void test_order(int cgroup_fd, struct bpf_tcp_ops *skel, int family)
+{
+ LIBBPF_OPTS(bpf_cgroup_opts, preorder_opts, .flags = BPF_F_PREORDER);
+ struct bpf_link *link1 = NULL, *link2 = NULL, *link3 = NULL;
+
+ link1 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops1, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link1, "attach_ops1"))
+ goto done;
+
+ link2 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops2, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link2, "attach_ops2"))
+ goto done;
+
+ link3 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops3, cgroup_fd,
+ &preorder_opts);
+ if (!ASSERT_OK_PTR(link3, "attach_ops3_preorder"))
+ goto done;
+
+ reset_order(skel);
+ do_listen_connect(family);
+
+ ASSERT_EQ(skel->bss->listen_cnt, 3, "listen_cnt");
+ ASSERT_EQ(skel->bss->listen_order[0], 3, "listen_order[0]");
+ ASSERT_EQ(skel->bss->listen_order[1], 1, "listen_order[1]");
+ ASSERT_EQ(skel->bss->listen_order[2], 2, "listen_order[2]");
+
+ ASSERT_EQ(skel->bss->connect_cnt, 3, "connect_cnt");
+ ASSERT_EQ(skel->bss->connect_order[0], 3, "connect_order[0]");
+ ASSERT_EQ(skel->bss->connect_order[1], 1, "connect_order[1]");
+ ASSERT_EQ(skel->bss->connect_order[2], 2, "connect_order[2]");
+
+done:
+ bpf_link__destroy(link3);
+ bpf_link__destroy(link2);
+ bpf_link__destroy(link1);
+}
+
+static void run_order_subtest(void)
+{
+ struct bpf_tcp_ops *skel = NULL;
+ struct netns_obj *ns = NULL;
+ int cgroup_fd;
+
+ cgroup_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+ return;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ test_order(cgroup_fd, skel, AF_INET);
+ test_order(cgroup_fd, skel, AF_INET6);
+
+done:
+ bpf_tcp_ops__destroy(skel);
+ netns_free(ns);
+ close(cgroup_fd);
+}
+
+/*
+ * Position a new attachment relative to an existing one. Attach ops1, then
+ * ops2 with BPF_F_BEFORE ops1, then ops3 with BPF_F_AFTER ops2. Expected
+ * execution order: [2, 3, 1]. For struct_ops, relative_fd refers to a link
+ * fd, so BPF_F_LINK must be set.
+ */
+static void test_before_after(int cgroup_fd, struct bpf_tcp_ops *skel)
+{
+ struct bpf_link *link1 = NULL, *link2 = NULL, *link3 = NULL;
+ LIBBPF_OPTS(bpf_cgroup_opts, opts);
+
+ link1 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops1, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link1, "attach_ops1"))
+ goto done;
+
+ opts.flags = BPF_F_BEFORE | BPF_F_LINK;
+ opts.relative_fd = bpf_link__fd(link1);
+ link2 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops2, cgroup_fd, &opts);
+ if (!ASSERT_OK_PTR(link2, "attach_ops2_before"))
+ goto done;
+
+ opts.flags = BPF_F_AFTER | BPF_F_LINK;
+ opts.relative_fd = bpf_link__fd(link2);
+ link3 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops3, cgroup_fd, &opts);
+ if (!ASSERT_OK_PTR(link3, "attach_ops3_after"))
+ goto done;
+
+ reset_order(skel);
+ do_listen_connect(AF_INET6);
+
+ ASSERT_EQ(skel->bss->listen_cnt, 3, "listen_cnt");
+ ASSERT_EQ(skel->bss->listen_order[0], 2, "listen_order[0]");
+ ASSERT_EQ(skel->bss->listen_order[1], 3, "listen_order[1]");
+ ASSERT_EQ(skel->bss->listen_order[2], 1, "listen_order[2]");
+
+done:
+ bpf_link__destroy(link3);
+ bpf_link__destroy(link2);
+ bpf_link__destroy(link1);
+}
+
+static void run_before_after_subtest(void)
+{
+ struct bpf_tcp_ops *skel = NULL;
+ struct netns_obj *ns = NULL;
+ int cgroup_fd;
+
+ cgroup_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+ return;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ test_before_after(cgroup_fd, skel);
+
+done:
+ bpf_tcp_ops__destroy(skel);
+ netns_free(ns);
+ close(cgroup_fd);
+}
+
+static void test_query(int cgroup_fd, struct bpf_tcp_ops *skel)
+{
+ struct bpf_map_info info = {};
+ __u32 info_len = sizeof(info);
+ LIBBPF_OPTS(bpf_prog_query_opts, query_opts);
+ struct bpf_link *link1 = NULL, *link2 = NULL;
+ __u32 map1_id, map2_id, map_ids[2] = {};
+ __s32 type_id;
+
+ type_id = get_bpf_tcp_ops_type_id();
+ if (type_id <= 0)
+ return;
+
+ bpf_map_get_info_by_fd(bpf_map__fd(skel->maps.tcp_ops1), &info, &info_len);
+ map1_id = info.id;
+
+ bpf_map_get_info_by_fd(bpf_map__fd(skel->maps.tcp_ops2), &info, &info_len);
+ map2_id = info.id;
+
+ link1 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops1, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link1, "attach_ops1"))
+ goto done;
+
+ link2 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops2, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link2, "attach_ops2"))
+ goto done;
+
+ /* query effective: expect 2 entries in attachment order */
+ query_opts.type_id = type_id;
+ query_opts.prog_ids = map_ids;
+ query_opts.count = ARRAY_SIZE(map_ids);
+ query_opts.query_flags = BPF_F_QUERY_EFFECTIVE;
+ ASSERT_OK(bpf_prog_query_opts(cgroup_fd, BPF_STRUCT_OPS, &query_opts),
+ "query_effective");
+ ASSERT_EQ(query_opts.count, 2, "query_effective_count");
+ ASSERT_EQ(map_ids[0], map1_id, "map_ids[0]");
+ ASSERT_EQ(map_ids[1], map2_id, "map_ids[1]");
+
+ /* query attached (non-effective): expect 2 entries */
+ memset(map_ids, 0, sizeof(map_ids));
+ query_opts.query_flags = 0;
+ query_opts.count = ARRAY_SIZE(map_ids);
+ ASSERT_OK(bpf_prog_query_opts(cgroup_fd, BPF_STRUCT_OPS, &query_opts),
+ "query_attached");
+ ASSERT_EQ(query_opts.count, 2, "query_attached_count");
+ ASSERT_EQ(map_ids[0], map1_id, "attached_map_ids[0]");
+ ASSERT_EQ(map_ids[1], map2_id, "attached_map_ids[1]");
+
+done:
+ bpf_link__destroy(link2);
+ bpf_link__destroy(link1);
+}
+
+static void run_query_subtest(void)
+{
+ struct bpf_tcp_ops *skel = NULL;
+ struct netns_obj *ns = NULL;
+ int cgroup_fd;
+
+ cgroup_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+ return;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ test_query(cgroup_fd, skel);
+
+done:
+ bpf_tcp_ops__destroy(skel);
+ netns_free(ns);
+ close(cgroup_fd);
+}
+
+/* Must match progs/bpf_tcp_ops.c */
+#define OPS_RETVAL1 11
+#define OPS_RETVAL2 22
+
+/*
+ * Attach three struct_ops implementing timeout_init to the same cgroup; they
+ * run in attach order [retval1, retval2, retval3]. timeout_init's return value
+ * is chained: the first prog reads the kernel seed via bpf_get_retval() (0,
+ * since no legacy sockops prog is attached) and returns OPS_RETVAL1; each
+ * subsequent prog must then observe the previous prog's return value. This
+ * proves the trampoline inherits the retval across an array of struct_ops.
+ */
+static void test_retval(int cgroup_fd, struct bpf_tcp_ops *skel)
+{
+ struct bpf_link *link1 = NULL, *link2 = NULL, *link3 = NULL;
+
+ skel->bss->retval_saw1 = -1;
+ skel->bss->retval_saw2 = -1;
+ skel->bss->retval_saw3 = -1;
+
+ link1 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops_retval1, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link1, "attach_retval1"))
+ goto done;
+
+ link2 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops_retval2, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link2, "attach_retval2"))
+ goto done;
+
+ link3 = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops_retval3, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link3, "attach_retval3"))
+ goto done;
+
+ do_listen_connect(AF_INET6);
+
+ /* First prog inherits the kernel seed (no legacy sockops -> 0). */
+ ASSERT_EQ(skel->bss->retval_saw1, 0, "retval_saw1");
+ /* Each subsequent prog inherits the previous prog's return value. */
+ ASSERT_EQ(skel->bss->retval_saw2, OPS_RETVAL1, "retval_saw2");
+ ASSERT_EQ(skel->bss->retval_saw3, OPS_RETVAL2, "retval_saw3");
+
+done:
+ bpf_link__destroy(link3);
+ bpf_link__destroy(link2);
+ bpf_link__destroy(link1);
+}
+
+static void run_retval_subtest(void)
+{
+ struct bpf_tcp_ops *skel = NULL;
+ struct netns_obj *ns = NULL;
+ int cgroup_fd;
+
+ cgroup_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+ return;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ test_retval(cgroup_fd, skel);
+
+done:
+ bpf_tcp_ops__destroy(skel);
+ netns_free(ns);
+ close(cgroup_fd);
+}
+
+/*
+ * bpf_link__update_map() swaps the struct_ops map backing an attached link.
+ * The link keeps its position, including BPF_F_PREORDER, across the update.
+ * Attach ops1 (normal) and ops2 (preorder): order [2, 1]. Update the normal
+ * link to ops3 -> [2, 3]; update the preorder link to ops1 -> [1, 3].
+ */
+static void test_update(int cgroup_fd, struct bpf_tcp_ops *skel)
+{
+ LIBBPF_OPTS(bpf_cgroup_opts, preorder_opts, .flags = BPF_F_PREORDER);
+ struct bpf_link *link = NULL, *link_pre = NULL;
+
+ link = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops1, cgroup_fd, NULL);
+ if (!ASSERT_OK_PTR(link, "attach_ops1"))
+ goto done;
+
+ link_pre = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops2, cgroup_fd,
+ &preorder_opts);
+ if (!ASSERT_OK_PTR(link_pre, "attach_ops2_preorder"))
+ goto done;
+
+ reset_order(skel);
+ do_listen_connect(AF_INET6);
+ ASSERT_EQ(skel->bss->listen_cnt, 2, "cnt_initial");
+ ASSERT_EQ(skel->bss->listen_order[0], 2, "order0_initial");
+ ASSERT_EQ(skel->bss->listen_order[1], 1, "order1_initial");
+
+ /* Update the normal link's map (ops1 -> ops3); position is unchanged. */
+ if (!ASSERT_OK(bpf_link__update_map(link, skel->maps.tcp_ops3), "update_normal"))
+ goto done;
+
+ reset_order(skel);
+ do_listen_connect(AF_INET6);
+ ASSERT_EQ(skel->bss->listen_order[0], 2, "order0_after_normal");
+ ASSERT_EQ(skel->bss->listen_order[1], 3, "order1_after_normal");
+
+ /* Update the preorder link's map (ops2 -> ops1); it stays first. */
+ if (!ASSERT_OK(bpf_link__update_map(link_pre, skel->maps.tcp_ops1), "update_preorder"))
+ goto done;
+
+ reset_order(skel);
+ do_listen_connect(AF_INET6);
+ ASSERT_EQ(skel->bss->listen_order[0], 1, "order0_after_preorder");
+ ASSERT_EQ(skel->bss->listen_order[1], 3, "order1_after_preorder");
+
+done:
+ bpf_link__destroy(link_pre);
+ bpf_link__destroy(link);
+}
+
+static void run_update_subtest(void)
+{
+ struct bpf_tcp_ops *skel = NULL;
+ struct netns_obj *ns = NULL;
+ int cgroup_fd;
+
+ cgroup_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup"))
+ return;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ test_update(cgroup_fd, skel);
+
+done:
+ bpf_tcp_ops__destroy(skel);
+ netns_free(ns);
+ close(cgroup_fd);
+}
+
+/*
+ * Two-level hierarchy. Attach ops1 to the parent and ops2 to the child, then
+ * trigger from a socket in the child. Descendant progs run before ancestor
+ * progs, so the order is [2 (child), 1 (parent)].
+ */
+static void test_hierarchy(int parent_fd, int child_fd, struct bpf_tcp_ops *skel)
+{
+ struct bpf_link *plink = NULL, *clink = NULL;
+
+ plink = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops1, parent_fd, NULL);
+ if (!ASSERT_OK_PTR(plink, "attach_parent"))
+ goto done;
+
+ clink = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops2, child_fd, NULL);
+ if (!ASSERT_OK_PTR(clink, "attach_child"))
+ goto done;
+
+ reset_order(skel);
+ do_listen_connect(AF_INET6);
+
+ ASSERT_EQ(skel->bss->listen_cnt, 2, "listen_cnt");
+ ASSERT_EQ(skel->bss->listen_order[0], 2, "listen_order[0]");
+ ASSERT_EQ(skel->bss->listen_order[1], 1, "listen_order[1]");
+
+done:
+ bpf_link__destroy(clink);
+ bpf_link__destroy(plink);
+}
+
+static void run_hierarchy_subtest(void)
+{
+ struct bpf_tcp_ops *skel = NULL;
+ struct netns_obj *ns = NULL;
+ int parent_fd, child_fd = -1;
+
+ parent_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(parent_fd, 0, "join_parent_cgroup"))
+ return;
+
+ child_fd = create_and_get_cgroup(CGROUP_PATH "/child");
+ if (!ASSERT_GE(child_fd, 0, "create_child_cgroup"))
+ goto done;
+
+ if (!ASSERT_OK(join_cgroup(CGROUP_PATH "/child"), "join_child_cgroup"))
+ goto done;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ test_hierarchy(parent_fd, child_fd, skel);
+
+done:
+ bpf_tcp_ops__destroy(skel);
+ netns_free(ns);
+ if (child_fd >= 0)
+ close(child_fd);
+ close(parent_fd);
+}
+
+/*
+ * Attach ops1 to the parent, then create and join the child cgroup. The child
+ * is created after the attach, so it must inherit the parent's effective progs
+ * via cgroup_bpf_inherit(). A socket in the child runs the parent's prog.
+ */
+static void test_inherit(int parent_fd, struct bpf_tcp_ops *skel)
+{
+ struct bpf_link *plink = NULL;
+ int child_fd = -1;
+
+ plink = bpf_map__attach_cgroup_opts(skel->maps.tcp_ops1, parent_fd, NULL);
+ if (!ASSERT_OK_PTR(plink, "attach_parent"))
+ goto done;
+
+ child_fd = create_and_get_cgroup(CGROUP_PATH "/child");
+ if (!ASSERT_GE(child_fd, 0, "create_child_cgroup"))
+ goto done;
+
+ if (!ASSERT_OK(join_cgroup(CGROUP_PATH "/child"), "join_child_cgroup"))
+ goto done;
+
+ reset_order(skel);
+ do_listen_connect(AF_INET6);
+
+ ASSERT_EQ(skel->bss->listen_cnt, 1, "listen_cnt");
+ ASSERT_EQ(skel->bss->listen_order[0], 1, "listen_order[0]");
+
+done:
+ if (child_fd >= 0)
+ close(child_fd);
+ bpf_link__destroy(plink);
+}
+
+static void run_inherit_subtest(void)
+{
+ struct bpf_tcp_ops *skel = NULL;
+ struct netns_obj *ns = NULL;
+ int parent_fd;
+
+ parent_fd = test__join_cgroup(CGROUP_PATH);
+ if (!ASSERT_GE(parent_fd, 0, "join_parent_cgroup"))
+ return;
+
+ ns = netns_new(TEST_NETNS, true);
+ if (!ASSERT_OK_PTR(ns, "netns_new"))
+ goto done;
+
+ skel = bpf_tcp_ops__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ goto done;
+
+ test_inherit(parent_fd, skel);
+
+done:
+ bpf_tcp_ops__destroy(skel);
+ netns_free(ns);
+ close(parent_fd);
+}
+
+void test_bpf_tcp_ops(void)
+{
+ if (test__start_subtest("order"))
+ run_order_subtest();
+ if (test__start_subtest("before_after"))
+ run_before_after_subtest();
+ if (test__start_subtest("query"))
+ run_query_subtest();
+ if (test__start_subtest("retval"))
+ run_retval_subtest();
+ if (test__start_subtest("update"))
+ run_update_subtest();
+ if (test__start_subtest("hierarchy"))
+ run_hierarchy_subtest();
+ if (test__start_subtest("inherit"))
+ run_inherit_subtest();
+}
diff --git a/tools/testing/selftests/bpf/progs/bpf_tcp_ops.c b/tools/testing/selftests/bpf/progs/bpf_tcp_ops.c
new file mode 100644
index 000000000000..94a7f52573d5
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_tcp_ops.c
@@ -0,0 +1,141 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+#define MAX_CGROUP_OPS 8
+
+/* Call order for listen and connect, indexed by call sequence */
+u32 listen_order[MAX_CGROUP_OPS];
+u32 listen_cnt;
+
+u32 connect_order[MAX_CGROUP_OPS];
+u32 connect_cnt;
+
+static void record_listen(int id)
+{
+ u32 idx = listen_cnt;
+
+ if (idx < MAX_CGROUP_OPS) {
+ listen_order[idx] = id;
+ listen_cnt = idx + 1;
+ }
+}
+
+static void record_connect(int id)
+{
+ u32 idx = connect_cnt;
+
+ if (idx < MAX_CGROUP_OPS) {
+ connect_order[idx] = id;
+ connect_cnt = idx + 1;
+ }
+}
+
+/* struct_ops instance 1 */
+
+SEC("struct_ops")
+void BPF_PROG(tcp_ops1_listen, struct sock *sk)
+{
+ record_listen(1);
+}
+
+SEC("struct_ops")
+void BPF_PROG(tcp_ops1_connect, struct sock *sk)
+{
+ record_connect(1);
+}
+
+SEC(".struct_ops.link")
+struct bpf_tcp_ops tcp_ops1 = {
+ .listen = (void *)tcp_ops1_listen,
+ .connect = (void *)tcp_ops1_connect,
+};
+
+/* struct_ops instance 2 */
+
+SEC("struct_ops")
+void BPF_PROG(tcp_ops2_listen, struct sock *sk)
+{
+ record_listen(2);
+}
+
+SEC("struct_ops")
+void BPF_PROG(tcp_ops2_connect, struct sock *sk)
+{
+ record_connect(2);
+}
+
+SEC(".struct_ops.link")
+struct bpf_tcp_ops tcp_ops2 = {
+ .listen = (void *)tcp_ops2_listen,
+ .connect = (void *)tcp_ops2_connect,
+};
+
+/* struct_ops instance 3 */
+
+SEC("struct_ops")
+void BPF_PROG(tcp_ops3_listen, struct sock *sk)
+{
+ record_listen(3);
+}
+
+SEC("struct_ops")
+void BPF_PROG(tcp_ops3_connect, struct sock *sk)
+{
+ record_connect(3);
+}
+
+SEC(".struct_ops.link")
+struct bpf_tcp_ops tcp_ops3 = {
+ .listen = (void *)tcp_ops3_listen,
+ .connect = (void *)tcp_ops3_connect,
+};
+
+#define OPS_RETVAL1 11
+#define OPS_RETVAL2 22
+#define OPS_RETVAL3 33
+
+int retval_saw1;
+int retval_saw2;
+int retval_saw3;
+
+SEC("struct_ops")
+int BPF_PROG(tcp_ops_retval1_timeout_init, struct sock *sk, struct request_sock *req)
+{
+ retval_saw1 = bpf_get_retval();
+ return OPS_RETVAL1;
+}
+
+SEC(".struct_ops.link")
+struct bpf_tcp_ops tcp_ops_retval1 = {
+ .timeout_init = (void *)tcp_ops_retval1_timeout_init,
+};
+
+SEC("struct_ops")
+int BPF_PROG(tcp_ops_retval2_timeout_init, struct sock *sk, struct request_sock *req)
+{
+ retval_saw2 = bpf_get_retval();
+ return OPS_RETVAL2;
+}
+
+SEC(".struct_ops.link")
+struct bpf_tcp_ops tcp_ops_retval2 = {
+ .timeout_init = (void *)tcp_ops_retval2_timeout_init,
+};
+
+SEC("struct_ops")
+int BPF_PROG(tcp_ops_retval3_timeout_init, struct sock *sk, struct request_sock *req)
+{
+ retval_saw3 = bpf_get_retval();
+ return OPS_RETVAL3;
+}
+
+SEC(".struct_ops.link")
+struct bpf_tcp_ops tcp_ops_retval3 = {
+ .timeout_init = (void *)tcp_ops_retval3_timeout_init,
+};
+
+char _license[] SEC("license") = "GPL";
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH bpf-next v2 13/15] libbpf: Support attaching struct_ops to a cgroup
From: Amery Hung @ 2026-06-23 17:50 UTC (permalink / raw)
To: bpf
Cc: netdev, alexei.starovoitov, andrii, daniel, eddyz87, memxor,
martin.lau, shakeel.butt, roman.gushchin, kuniyu, kerneljasonxing,
ameryhung, kernel-team
In-Reply-To: <20260623175006.3136053-1-ameryhung@gmail.com>
From: Martin KaFai Lau <martin.lau@kernel.org>
Add bpf_map__attach_cgroup_opts() to attach a struct_ops map to a cgroup
through a BPF link.
Also extend struct bpf_prog_query_opts with a type_id field so a
BPF_STRUCT_OPS query on a cgroup can select the struct_ops type to
enumerate.
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
---
tools/lib/bpf/bpf.c | 2 ++
tools/lib/bpf/bpf.h | 3 +-
tools/lib/bpf/libbpf.c | 59 ++++++++++++++++++++++++++++++++++
tools/lib/bpf/libbpf.h | 3 ++
tools/lib/bpf/libbpf.map | 5 +++
tools/lib/bpf/libbpf_version.h | 2 +-
6 files changed, 72 insertions(+), 2 deletions(-)
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index 96819c082c77..a9de7f107cf7 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -934,6 +934,7 @@ int bpf_link_create(int prog_fd, int target_fd,
case BPF_CGROUP_GETSOCKOPT:
case BPF_CGROUP_SETSOCKOPT:
case BPF_LSM_CGROUP:
+ case BPF_STRUCT_OPS:
relative_fd = OPTS_GET(opts, cgroup.relative_fd, 0);
relative_id = OPTS_GET(opts, cgroup.relative_id, 0);
if (relative_fd && relative_id)
@@ -1056,6 +1057,7 @@ int bpf_prog_query_opts(int target, enum bpf_attach_type type,
attr.query.attach_type = type;
attr.query.query_flags = OPTS_GET(opts, query_flags, 0);
attr.query.count = OPTS_GET(opts, count, 0);
+ attr.query.type_id = OPTS_GET(opts, type_id, 0);
attr.query.prog_ids = ptr_to_u64(OPTS_GET(opts, prog_ids, NULL));
attr.query.link_ids = ptr_to_u64(OPTS_GET(opts, link_ids, NULL));
attr.query.prog_attach_flags = ptr_to_u64(OPTS_GET(opts, prog_attach_flags, NULL));
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 7534a593edae..490e8cb4ba53 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -637,9 +637,10 @@ struct bpf_prog_query_opts {
__u32 *link_ids;
__u32 *link_attach_flags;
__u64 revision;
+ __u32 type_id;
size_t :0;
};
-#define bpf_prog_query_opts__last_field revision
+#define bpf_prog_query_opts__last_field type_id
/**
* @brief **bpf_prog_query_opts()** queries the BPF programs and BPF links
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 1368752aa13c..17f8466e33fa 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -14104,6 +14104,65 @@ struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map)
return &link->link;
}
+struct bpf_link *bpf_map__attach_cgroup_opts(const struct bpf_map *map, int cgroup_fd,
+ const struct bpf_cgroup_opts *opts)
+{
+ LIBBPF_OPTS(bpf_link_create_opts, link_create_opts);
+ struct bpf_link_struct_ops *link;
+ __u32 relative_id, zero = 0;
+ int err, fd, relative_fd;
+
+ if (!OPTS_VALID(opts, bpf_cgroup_opts))
+ return libbpf_err_ptr(-EINVAL);
+
+ if (!bpf_map__is_struct_ops(map)) {
+ pr_warn("map '%s': can't attach non-struct_ops map\n", map->name);
+ return libbpf_err_ptr(-EINVAL);
+ }
+
+ if (map->fd < 0) {
+ pr_warn("map '%s': can't attach BPF map without FD (was it created?)\n", map->name);
+ return libbpf_err_ptr(-EINVAL);
+ }
+
+ relative_id = OPTS_GET(opts, relative_id, 0);
+ relative_fd = OPTS_GET(opts, relative_fd, 0);
+
+ if (relative_fd && relative_id) {
+ pr_warn("map '%s': relative_fd and relative_id cannot be set at the same time\n",
+ map->name);
+ return libbpf_err_ptr(-EINVAL);
+ }
+
+ link_create_opts.cgroup.expected_revision = OPTS_GET(opts, expected_revision, 0);
+ link_create_opts.cgroup.relative_fd = relative_fd;
+ link_create_opts.cgroup.relative_id = relative_id;
+ link_create_opts.flags = OPTS_GET(opts, flags, 0);
+
+ link = calloc(1, sizeof(*link));
+ if (!link)
+ return libbpf_err_ptr(-ENOMEM);
+
+ err = bpf_map_update_elem(map->fd, &zero, map->st_ops->kern_vdata, 0);
+ if (err && err != -EBUSY) {
+ free(link);
+ return libbpf_err_ptr(err);
+ }
+
+ link->link.detach = bpf_link__detach_struct_ops;
+
+ fd = bpf_link_create(map->fd, cgroup_fd, BPF_STRUCT_OPS, &link_create_opts);
+ if (fd < 0) {
+ free(link);
+ return libbpf_err_ptr(fd);
+ }
+
+ link->link.fd = fd;
+ link->map_fd = map->fd;
+
+ return &link->link;
+}
+
/*
* Swap the back struct_ops of a link with a new struct_ops map.
*/
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index b965ad571540..0e5f4e9bba41 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -960,6 +960,9 @@ bpf_program__attach_cgroup_opts(const struct bpf_program *prog, int cgroup_fd,
struct bpf_map;
LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map);
+LIBBPF_API struct bpf_link *bpf_map__attach_cgroup_opts(const struct bpf_map *map,
+ int cgroup_fd,
+ const struct bpf_cgroup_opts *opts);
LIBBPF_API int bpf_link__update_map(struct bpf_link *link, const struct bpf_map *map);
struct bpf_iter_attach_opts {
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index b731df19ae69..1b01d49e58eb 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -462,3 +462,8 @@ LIBBPF_1.8.0 {
bpf_program__clone;
btf__new_empty_opts;
} LIBBPF_1.7.0;
+
+LIBBPF_1.9.0 {
+ global:
+ bpf_map__attach_cgroup_opts;
+} LIBBPF_1.8.0;
diff --git a/tools/lib/bpf/libbpf_version.h b/tools/lib/bpf/libbpf_version.h
index c446c0cd8cf9..57b74ef3618c 100644
--- a/tools/lib/bpf/libbpf_version.h
+++ b/tools/lib/bpf/libbpf_version.h
@@ -4,6 +4,6 @@
#define __LIBBPF_VERSION_H
#define LIBBPF_MAJOR_VERSION 1
-#define LIBBPF_MINOR_VERSION 8
+#define LIBBPF_MINOR_VERSION 9
#endif /* __LIBBPF_VERSION_H */
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH bpf-next v2 12/15] bpf: tcp: Support parse/len/write header option hooks in bpf_tcp_ops
From: Amery Hung @ 2026-06-23 17:50 UTC (permalink / raw)
To: bpf
Cc: netdev, alexei.starovoitov, andrii, daniel, eddyz87, memxor,
martin.lau, shakeel.butt, roman.gushchin, kuniyu, kerneljasonxing,
ameryhung, kernel-team
In-Reply-To: <20260623175006.3136053-1-ameryhung@gmail.com>
Add the TCP header option callbacks to the bpf_tcp_ops struct_ops type:
parse_hdr - parse the options of an incoming skb on an established
connection
hdr_opt_len - reserve space in the TCP header for bpf options
write_hdr_opt - write the reserved bpf options
These mirror the BPF_SOCK_OPS_PARSE_HDR_OPT_CB, _HDR_OPT_LEN_CB and
_WRITE_HDR_OPT_CB legacy sockops callbacks, but are exposed as struct_ops
members so a program can implement them with normal function signatures
and per-member helper sets.
The reserved header window is shared between the legacy sockops and
bpf_tcp_ops paths. tcp_{syn,synack,established}_options() first run the
legacy BPF_SOCK_OPS_HDR_OPT_LEN_CB and then call hdr_opt_len, so both
sources accumulate into opts->bpf_opt_len; at write time the legacy
options are emitted first and bpf_tcp_ops writes after them.
API design
bpf_tcp_ops overloads the sock_ops header-option helpers rather than
introducing a new API: bpf_reserve_hdr_opt(), bpf_store_hdr_opt() and
bpf_load_hdr_opt() are exposed per-member (reserve for hdr_opt_len,
store/load for write_hdr_opt, load for parse_hdr) and share the existing
kernel option-walking core via _bpf_sock_ops{store,load}hdr_opt(), with
the bpf_tcp_ops wrappers synthesizing a temporary bpf_sock_ops_kern from
the program ctx. This keeps a port from the legacy
BPF_SOCK_OPS*_HDR_OPT_CB callbacks mechanical (same helper calls) and
adds no new UAPI helper/kfunc surface.
An alternative considered was to drop the option helpers entirely: have
hdr_opt_len reserve space purely through its return value, and introduce
a dedicated TCP-header-option dynptr used for both reading and writing.
That is a cleaner, more self-contained interface, but it is a larger
change and does not reuse the legacy helpers, making a port from sockops
less mechanical. It can be pursued as a follow-up; the helper-based
interface here keeps this series focused on moving the hooks to
struct_ops.
The hdr_opt_len fast path in tcp_established_options() is gated by
cgroup_bpf_enabled(CGROUP_TCP_SOCK_OPS). Note this is a global,
per-attach-type static branch: it is enabled whenever any bpf_tcp_ops is
attached, even one that does not implement hdr_opt_len or that is attached
to a different cgroup. In those cases the block still runs but
bpf_tcp_ops_hdr_opt_len() no-ops via the per-member check in the dispatch
macro. A per-member/per-cgroup gate could be added later if the extra
fast-path work proves measurable.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
---
include/linux/filter.h | 5 ++
include/net/tcp.h | 40 ++++++++++
include/uapi/linux/bpf.h | 35 ++++++---
net/core/filter.c | 32 +++++---
net/ipv4/bpf_tcp_ops.c | 139 ++++++++++++++++++++++++++++++++-
net/ipv4/tcp_input.c | 13 +++
net/ipv4/tcp_output.c | 46 +++++++++++
tools/include/uapi/linux/bpf.h | 35 ++++++---
8 files changed, 306 insertions(+), 39 deletions(-)
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 67d337ede91b..fe28db65fb6a 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1843,6 +1843,11 @@ static __always_inline long __bpf_xdp_redirect_map(struct bpf_map *map, u64 inde
return XDP_REDIRECT;
}
+int __bpf_sock_ops_load_hdr_opt(struct bpf_sock_ops_kern *bpf_sock,
+ void *search_res, u32 len, u64 flags);
+int __bpf_sock_ops_store_hdr_opt(struct bpf_sock_ops_kern *bpf_sock,
+ const void *from, u32 len, u64 flags);
+
#ifdef CONFIG_NET
int __bpf_skb_load_bytes(const struct sk_buff *skb, u32 offset, void *to, u32 len);
int __bpf_skb_store_bytes(struct sk_buff *skb, u32 offset, const void *from,
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 2102f9f2afd6..7bf702117602 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -3005,6 +3005,45 @@ struct bpf_tcp_ops {
/* Called on listen(2), right after the socket enters TCP_LISTEN. */
void (*listen)(struct sock *sk);
+
+ /* Parse the TCP header options of an incoming skb received on an
+ * established connection. Use bpf_dynptr_from_skb()/bpf_skb_load_bytes()
+ * to access the options.
+ */
+ void (*parse_hdr)(struct sock *sk, struct sk_buff *skb);
+
+ /* Reserve space in the outgoing TCP header for options to be written
+ * later by write_hdr_opt(). Call bpf_reserve_hdr_opt() to reserve bytes.
+ *
+ * @skb: outgoing packet. NULL when called from tcp_current_mss()
+ * (MSS sizing).
+ * @req: request_sock on the synack path; NULL otherwise.
+ * @syn_skb: incoming SYN on the synack path; NULL otherwise.
+ * @synack_type: TCP_SYNACK_COOKIE indicates a stateless syncookie.
+ * @remaining: pointer to the size of space still available; cast it
+ * using bpf_rdonly_cast() before dereferencing.
+ */
+ void (*hdr_opt_len)(struct sock *sk, struct sk_buff *skb,
+ struct request_sock *req, struct sk_buff *syn_skb,
+ enum tcp_synack_type synack_type,
+ unsigned int *remaining);
+
+ /* Write header options into the space reserved earlier by hdr_opt_len().
+ * Use bpf_store_hdr_opt() to write; it appends within the reserved window
+ * shared with legacy SOCKOPS.
+ *
+ * @skb: outgoing packet.
+ * @req: request_sock on the synack path; NULL otherwise.
+ * @syn_skb: incoming SYN on the synack path; NULL otherwise.
+ * @synack_type: TCP_SYNACK_COOKIE indicates a stateless syncookie.
+ * @opt_off: offset in the outgoing @skb's TCP header where the
+ * bpf_tcp_ops portion of the reserved window begins, i.e. after
+ * the kernel and legacy options.
+ */
+ void (*write_hdr_opt)(struct sock *sk, struct sk_buff *skb,
+ struct request_sock *req, struct sk_buff *syn_skb,
+ enum tcp_synack_type synack_type,
+ u32 opt_off);
};
#define bpf_tcp_ops_call(op, sk, ...) \
@@ -3056,6 +3095,7 @@ do { \
} \
__retval; \
})
+
#else
#define bpf_tcp_ops_call(op, sk, ...) do { } while (0)
#define bpf_tcp_ops_call_int(op, init_retval, sk, ...) (init_retval)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 2b84c69eb814..45b9ee29e461 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -4799,15 +4799,18 @@ union bpf_attr {
* The non-negative copied *buf* length equal to or less than
* *size* on success, or a negative error in case of failure.
*
- * long bpf_load_hdr_opt(struct bpf_sock_ops *skops, void *searchby_res, u32 len, u64 flags)
+ * long bpf_load_hdr_opt(void *ctx, void *searchby_res, u32 len, u64 flags)
* Description
* Load header option. Support reading a particular TCP header
- * option for bpf program (**BPF_PROG_TYPE_SOCK_OPS**).
+ * option for bpf program (**BPF_PROG_TYPE_SOCK_OPS**). For the
+ * **bpf_tcp_ops** struct_ops, this helper can be called from the
+ * **parse_hdr**\ () and **write_hdr_opt**\ () operators.
*
- * If *flags* is 0, it will search the option from the
- * *skops*\ **->skb_data**. The comment in **struct bpf_sock_ops**
- * has details on what skb_data contains under different
- * *skops*\ **->op**.
+ * If *flags* is 0, it will search the option from the packet
+ * associated with the current operation. For
+ * **BPF_PROG_TYPE_SOCK_OPS**, the comment in
+ * **struct bpf_sock_ops** has details on what skb_data
+ * contains under different *op*.
*
* The first byte of the *searchby_res* specifies the
* kind that it wants to search.
@@ -4840,6 +4843,8 @@ union bpf_attr {
*
* * **BPF_LOAD_HDR_OPT_TCP_SYN** to search from the
* saved_syn packet or the just-received syn packet.
+ * Not supported by the **bpf_tcp_ops** struct_ops, which
+ * rejects all flags.
*
* Return
* > 0 when found, the header option is copied to *searchby_res*.
@@ -4860,9 +4865,9 @@ union bpf_attr {
* packet.
*
* **-EPERM** if the helper cannot be used under the current
- * *skops*\ **->op**.
+ * operation.
*
- * long bpf_store_hdr_opt(struct bpf_sock_ops *skops, const void *from, u32 len, u64 flags)
+ * long bpf_store_hdr_opt(void *ctx, const void *from, u32 len, u64 flags)
* Description
* Store header option. The data will be copied
* from buffer *from* with length *len* to the TCP header.
@@ -4878,7 +4883,9 @@ union bpf_attr {
* by searching the same option in the outgoing skb.
*
* This helper can only be called during
- * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**.
+ * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**, or from the
+ * **write_hdr_opt**\ () operator of the **bpf_tcp_ops**
+ * struct_ops.
*
* Return
* 0 on success, or negative error in case of failure:
@@ -4893,9 +4900,9 @@ union bpf_attr {
* **-EFAULT** on failure to parse the existing header options.
*
* **-EPERM** if the helper cannot be used under the current
- * *skops*\ **->op**.
+ * operation.
*
- * long bpf_reserve_hdr_opt(struct bpf_sock_ops *skops, u32 len, u64 flags)
+ * long bpf_reserve_hdr_opt(void *ctx, u32 len, u64 flags)
* Description
* Reserve *len* bytes for the bpf header option. The
* space will be used by **bpf_store_hdr_opt**\ () later in
@@ -4905,7 +4912,9 @@ union bpf_attr {
* the total number of bytes will be reserved.
*
* This helper can only be called during
- * **BPF_SOCK_OPS_HDR_OPT_LEN_CB**.
+ * **BPF_SOCK_OPS_HDR_OPT_LEN_CB**, or from the
+ * **hdr_opt_len**\ () operator of the **bpf_tcp_ops**
+ * struct_ops.
*
* Return
* 0 on success, or negative error in case of failure:
@@ -4915,7 +4924,7 @@ union bpf_attr {
* **-ENOSPC** if there is not enough space in the header.
*
* **-EPERM** if the helper cannot be used under the current
- * *skops*\ **->op**.
+ * operation.
*
* void *bpf_inode_storage_get(struct bpf_map *map, void *inode, void *value, u64 flags)
* Description
diff --git a/net/core/filter.c b/net/core/filter.c
index f85578772930..dc44ffb7a380 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -7885,17 +7885,14 @@ static const u8 *bpf_search_tcp_opt(const u8 *op, const u8 *opend,
return ERR_PTR(-ENOMSG);
}
-BPF_CALL_4(bpf_sock_ops_load_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
- void *, search_res, u32, len, u64, flags)
+int __bpf_sock_ops_load_hdr_opt(struct bpf_sock_ops_kern *bpf_sock,
+ void *search_res, u32 len, u64 flags)
{
bool eol, load_syn = flags & BPF_LOAD_HDR_OPT_TCP_SYN;
const u8 *op, *opend, *magic, *search = search_res;
u8 search_kind, search_len, copy_len, magic_len;
int ret;
- if (!is_locked_tcp_sock_ops(bpf_sock))
- return -EOPNOTSUPP;
-
/* 2 byte is the minimal option len except TCPOPT_NOP and
* TCPOPT_EOL which are useless for the bpf prog to learn
* and this helper disallow loading them also.
@@ -7956,6 +7953,15 @@ BPF_CALL_4(bpf_sock_ops_load_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
return ret;
}
+BPF_CALL_4(bpf_sock_ops_load_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
+ void *, search_res, u32, len, u64, flags)
+{
+ if (!is_locked_tcp_sock_ops(bpf_sock))
+ return -EOPNOTSUPP;
+
+ return __bpf_sock_ops_load_hdr_opt(bpf_sock, search_res, len, flags);
+}
+
static const struct bpf_func_proto bpf_sock_ops_load_hdr_opt_proto = {
.func = bpf_sock_ops_load_hdr_opt,
.gpl_only = false,
@@ -7966,17 +7972,14 @@ static const struct bpf_func_proto bpf_sock_ops_load_hdr_opt_proto = {
.arg4_type = ARG_ANYTHING,
};
-BPF_CALL_4(bpf_sock_ops_store_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
- const void *, from, u32, len, u64, flags)
+int __bpf_sock_ops_store_hdr_opt(struct bpf_sock_ops_kern *bpf_sock,
+ const void *from, u32 len, u64 flags)
{
u8 new_kind, new_kind_len, magic_len = 0, *opend;
const u8 *op, *new_op, *magic = NULL;
struct sk_buff *skb;
bool eol;
- if (bpf_sock->op != BPF_SOCK_OPS_WRITE_HDR_OPT_CB)
- return -EPERM;
-
if (len < 2 || flags)
return -EINVAL;
@@ -8034,6 +8037,15 @@ BPF_CALL_4(bpf_sock_ops_store_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
return 0;
}
+BPF_CALL_4(bpf_sock_ops_store_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
+ const void *, from, u32, len, u64, flags)
+{
+ if (bpf_sock->op != BPF_SOCK_OPS_WRITE_HDR_OPT_CB)
+ return -EPERM;
+
+ return __bpf_sock_ops_store_hdr_opt(bpf_sock, from, len, flags);
+}
+
static const struct bpf_func_proto bpf_sock_ops_store_hdr_opt_proto = {
.func = bpf_sock_ops_store_hdr_opt,
.gpl_only = false,
diff --git a/net/ipv4/bpf_tcp_ops.c b/net/ipv4/bpf_tcp_ops.c
index cf53c95a0dbc..0c7352517ac3 100644
--- a/net/ipv4/bpf_tcp_ops.c
+++ b/net/ipv4/bpf_tcp_ops.c
@@ -4,6 +4,7 @@
#include <linux/bpf.h>
#include <linux/btf_ids.h>
#include <linux/bpf_verifier.h>
+#include <linux/filter.h>
#include <net/bpf_sk_storage.h>
#include <net/tcp.h>
@@ -55,6 +56,26 @@ static void listen_stub(struct sock *sk)
{
}
+static void parse_hdr_stub(struct sock *sk, struct sk_buff *skb)
+{
+}
+
+static void hdr_opt_len_stub(struct sock *sk, struct sk_buff *skb__nullable,
+ struct request_sock *req__nullable,
+ struct sk_buff *syn_skb__nullable,
+ enum tcp_synack_type synack_type,
+ unsigned int *remaining)
+{
+}
+
+static void write_hdr_opt_stub(struct sock *sk, struct sk_buff *skb,
+ struct request_sock *req__nullable,
+ struct sk_buff *syn_skb__nullable,
+ enum tcp_synack_type synack_type,
+ u32 opt_off)
+{
+}
+
static struct bpf_tcp_ops __bpf_tcp_ops = {
.timeout_init = timeout_init_stub,
.rwnd_init = rwnd_init_stub,
@@ -66,6 +87,99 @@ static struct bpf_tcp_ops __bpf_tcp_ops = {
.retrans = retrans_stub,
.connect = connect_stub,
.listen = listen_stub,
+ .parse_hdr = parse_hdr_stub,
+ .hdr_opt_len = hdr_opt_len_stub,
+ .write_hdr_opt = write_hdr_opt_stub,
+};
+
+BPF_CALL_4(bpf_tcp_ops_store_hdr_opt, void *, ctx, const void *, from,
+ u32, len, u64, flags)
+{
+ struct sk_buff *skb = ((struct sk_buff **)ctx)[1];
+ struct bpf_sock_ops_kern sock_ops = {};
+ u32 opt_off = ((u64 *)ctx)[5];
+ u8 *op, *opend;
+
+ /* bpf_tcp_ops does not keep track of the end of the written TCP header
+ * options, so search for it every time the helper is called. The free
+ * space is NOP-filled, so a TCPOPT_NOP ends the search rather than being
+ * skipped as in a normal option walk in sockops.
+ */
+ op = skb->data + opt_off;
+ opend = skb->data + tcp_hdrlen(skb);
+ while (op < opend && *op != TCPOPT_NOP) {
+ if (*op == TCPOPT_EOL || op + 1 >= opend || op[1] < 2)
+ break;
+ op += op[1];
+ }
+
+ sock_ops.skb = skb;
+ sock_ops.skb_data_end = op;
+ sock_ops.remaining_opt_len = opend - op;
+
+ return __bpf_sock_ops_store_hdr_opt(&sock_ops, from, len, flags);
+}
+
+static const struct bpf_func_proto bpf_tcp_ops_store_hdr_opt_proto = {
+ .func = bpf_tcp_ops_store_hdr_opt,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
+ .arg3_type = ARG_CONST_SIZE,
+ .arg4_type = ARG_ANYTHING,
+};
+
+BPF_CALL_4(bpf_tcp_ops_load_hdr_opt, void *, ctx, void *, search_res,
+ u32, len, u64, flags)
+{
+ struct sk_buff *skb = ((struct sk_buff **)ctx)[1];
+ struct bpf_sock_ops_kern sock_ops = {};
+
+ /* No flags supported. In particular BPF_LOAD_HDR_OPT_TCP_SYN, which
+ * loads from the saved SYN, is not available because bpf_tcp_ops has no
+ * carrier to track the SYN source across the hooks.
+ */
+ if (flags)
+ return -EINVAL;
+
+ sock_ops.skb = skb;
+ sock_ops.skb_data_end = skb->data + tcp_hdrlen(skb);
+
+ return __bpf_sock_ops_load_hdr_opt(&sock_ops, search_res, len, flags);
+}
+
+static const struct bpf_func_proto bpf_tcp_ops_load_hdr_opt_proto = {
+ .func = bpf_tcp_ops_load_hdr_opt,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_WRITE,
+ .arg3_type = ARG_CONST_SIZE,
+ .arg4_type = ARG_ANYTHING,
+};
+
+BPF_CALL_3(bpf_tcp_ops_reserve_hdr_opt, void *, ctx, u32, len, u64, flags)
+{
+ unsigned int *remaining = ((unsigned int **)ctx)[5];
+
+ if (flags || len < 2)
+ return -EINVAL;
+
+ if (len > *remaining)
+ return -ENOSPC;
+
+ *remaining -= len;
+ return 0;
+}
+
+static const struct bpf_func_proto bpf_tcp_ops_reserve_hdr_opt_proto = {
+ .func = bpf_tcp_ops_reserve_hdr_opt,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_ANYTHING,
+ .arg3_type = ARG_ANYTHING,
};
BPF_CALL_0(bpf_tcp_ops_get_retval)
@@ -102,14 +216,20 @@ get_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
case BPF_FUNC_sk_storage_delete:
return &bpf_sk_storage_delete_proto;
case BPF_FUNC_setsockopt:
- /* The listener is not locked. */
+ /* The sk may be an unlocked listener (synack path) or NULL
+ * fullsock; disable for members that can run unlocked.
+ */
if (moff == offsetof(struct bpf_tcp_ops, rwnd_init) ||
- moff == offsetof(struct bpf_tcp_ops, timeout_init))
+ moff == offsetof(struct bpf_tcp_ops, timeout_init) ||
+ moff == offsetof(struct bpf_tcp_ops, hdr_opt_len) ||
+ moff == offsetof(struct bpf_tcp_ops, write_hdr_opt))
return NULL;
return &bpf_sk_setsockopt_proto;
case BPF_FUNC_getsockopt:
if (moff == offsetof(struct bpf_tcp_ops, rwnd_init) ||
- moff == offsetof(struct bpf_tcp_ops, timeout_init))
+ moff == offsetof(struct bpf_tcp_ops, timeout_init) ||
+ moff == offsetof(struct bpf_tcp_ops, hdr_opt_len) ||
+ moff == offsetof(struct bpf_tcp_ops, write_hdr_opt))
return NULL;
return &bpf_sk_getsockopt_proto;
case BPF_FUNC_get_retval:
@@ -117,6 +237,19 @@ get_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
moff == offsetof(struct bpf_tcp_ops, rwnd_init))
return &bpf_tcp_ops_get_retval_proto;
return NULL;
+ case BPF_FUNC_reserve_hdr_opt:
+ if (moff == offsetof(struct bpf_tcp_ops, hdr_opt_len))
+ return &bpf_tcp_ops_reserve_hdr_opt_proto;
+ return NULL;
+ case BPF_FUNC_load_hdr_opt:
+ if (moff == offsetof(struct bpf_tcp_ops, parse_hdr) ||
+ moff == offsetof(struct bpf_tcp_ops, write_hdr_opt))
+ return &bpf_tcp_ops_load_hdr_opt_proto;
+ return NULL;
+ case BPF_FUNC_store_hdr_opt:
+ if (moff == offsetof(struct bpf_tcp_ops, write_hdr_opt))
+ return &bpf_tcp_ops_store_hdr_opt_proto;
+ return NULL;
default:
return bpf_base_func_proto(func_id, prog);
}
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 12fb690d21c4..a36146789138 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -208,6 +208,18 @@ static void bpf_skops_established(struct sock *sk, int bpf_op,
}
#endif
+static void bpf_tcp_ops_parse_hdr(struct sock *sk, struct sk_buff *skb)
+{
+ switch (sk->sk_state) {
+ case TCP_SYN_RECV:
+ case TCP_SYN_SENT:
+ case TCP_LISTEN:
+ return;
+ }
+
+ bpf_tcp_ops_call(parse_hdr, sk, skb);
+}
+
static __cold void tcp_gro_dev_warn(const struct sock *sk, const struct sk_buff *skb,
unsigned int len)
{
@@ -6431,6 +6443,7 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
pass:
bpf_skops_parse_hdr(sk, skb);
+ bpf_tcp_ops_parse_hdr(sk, skb);
return true;
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 93f4a95399ea..580652d0a135 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -573,6 +573,13 @@ static void bpf_skops_write_hdr_opt(struct sock *sk, struct sk_buff *skb,
if (nr_written < max_opt_len)
memset(skb->data + first_opt_off + nr_written, TCPOPT_NOP,
max_opt_len - nr_written);
+
+ /* bpf_tcp_ops portion is NOP-filled (everything past the sockops
+ * writer's bytes). The writer find the append point by scanning from
+ * first_opt_off + nr_written to the first NOP.
+ */
+ bpf_tcp_ops_call(write_hdr_opt, sk, skb, req, syn_skb, synack_type,
+ first_opt_off + nr_written);
}
#else
static u32 bpf_skops_hdr_opt_len(struct sock *sk, struct sk_buff *skb,
@@ -594,6 +601,32 @@ static void bpf_skops_write_hdr_opt(struct sock *sk, struct sk_buff *skb,
}
#endif
+static u32 bpf_tcp_ops_hdr_opt_len(struct sock *sk, struct sk_buff *skb,
+ struct request_sock *req,
+ struct sk_buff *syn_skb,
+ enum tcp_synack_type synack_type,
+ struct tcp_out_options *opts,
+ u32 remaining)
+{
+ unsigned int remaining_out = remaining, reserved;
+
+ if (!remaining)
+ return 0;
+
+ /* bpf_tcp_ops_reserve_hdr_opt() reserves space via remaining_out */
+ bpf_tcp_ops_call(hdr_opt_len, sk, skb, req, syn_skb, synack_type, &remaining_out);
+
+ reserved = remaining - remaining_out;
+ if (!reserved)
+ return remaining;
+
+ /* round up to 4 bytes */
+ reserved = (reserved + 3) & ~3;
+
+ opts->bpf_opt_len += reserved;
+ return remaining - reserved;
+}
+
static __be32 *process_tcp_ao_options(struct tcp_sock *tp,
const struct tcp_request_sock *tcprsk,
struct tcp_out_options *opts,
@@ -1053,6 +1086,8 @@ static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
remaining = bpf_skops_hdr_opt_len(sk, skb, NULL, NULL, 0, opts,
remaining);
+ remaining = bpf_tcp_ops_hdr_opt_len(sk, skb, NULL, NULL, 0, opts,
+ remaining);
return MAX_TCP_OPTION_SPACE - remaining;
}
@@ -1141,6 +1176,8 @@ static unsigned int tcp_synack_options(const struct sock *sk,
remaining = bpf_skops_hdr_opt_len((struct sock *)sk, skb, req, syn_skb,
synack_type, opts, remaining);
+ remaining = bpf_tcp_ops_hdr_opt_len((struct sock *)sk, skb, req, syn_skb,
+ synack_type, opts, remaining);
return MAX_TCP_OPTION_SPACE - remaining;
}
@@ -1244,6 +1281,15 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
size = MAX_TCP_OPTION_SPACE - remaining;
}
+ if (cgroup_bpf_enabled(CGROUP_TCP_SOCK_OPS)) {
+ unsigned int remaining = MAX_TCP_OPTION_SPACE - size;
+
+ remaining = bpf_tcp_ops_hdr_opt_len(sk, skb, NULL, NULL, 0, opts,
+ remaining);
+
+ size = MAX_TCP_OPTION_SPACE - remaining;
+ }
+
return size;
}
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 2b84c69eb814..45b9ee29e461 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -4799,15 +4799,18 @@ union bpf_attr {
* The non-negative copied *buf* length equal to or less than
* *size* on success, or a negative error in case of failure.
*
- * long bpf_load_hdr_opt(struct bpf_sock_ops *skops, void *searchby_res, u32 len, u64 flags)
+ * long bpf_load_hdr_opt(void *ctx, void *searchby_res, u32 len, u64 flags)
* Description
* Load header option. Support reading a particular TCP header
- * option for bpf program (**BPF_PROG_TYPE_SOCK_OPS**).
+ * option for bpf program (**BPF_PROG_TYPE_SOCK_OPS**). For the
+ * **bpf_tcp_ops** struct_ops, this helper can be called from the
+ * **parse_hdr**\ () and **write_hdr_opt**\ () operators.
*
- * If *flags* is 0, it will search the option from the
- * *skops*\ **->skb_data**. The comment in **struct bpf_sock_ops**
- * has details on what skb_data contains under different
- * *skops*\ **->op**.
+ * If *flags* is 0, it will search the option from the packet
+ * associated with the current operation. For
+ * **BPF_PROG_TYPE_SOCK_OPS**, the comment in
+ * **struct bpf_sock_ops** has details on what skb_data
+ * contains under different *op*.
*
* The first byte of the *searchby_res* specifies the
* kind that it wants to search.
@@ -4840,6 +4843,8 @@ union bpf_attr {
*
* * **BPF_LOAD_HDR_OPT_TCP_SYN** to search from the
* saved_syn packet or the just-received syn packet.
+ * Not supported by the **bpf_tcp_ops** struct_ops, which
+ * rejects all flags.
*
* Return
* > 0 when found, the header option is copied to *searchby_res*.
@@ -4860,9 +4865,9 @@ union bpf_attr {
* packet.
*
* **-EPERM** if the helper cannot be used under the current
- * *skops*\ **->op**.
+ * operation.
*
- * long bpf_store_hdr_opt(struct bpf_sock_ops *skops, const void *from, u32 len, u64 flags)
+ * long bpf_store_hdr_opt(void *ctx, const void *from, u32 len, u64 flags)
* Description
* Store header option. The data will be copied
* from buffer *from* with length *len* to the TCP header.
@@ -4878,7 +4883,9 @@ union bpf_attr {
* by searching the same option in the outgoing skb.
*
* This helper can only be called during
- * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**.
+ * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**, or from the
+ * **write_hdr_opt**\ () operator of the **bpf_tcp_ops**
+ * struct_ops.
*
* Return
* 0 on success, or negative error in case of failure:
@@ -4893,9 +4900,9 @@ union bpf_attr {
* **-EFAULT** on failure to parse the existing header options.
*
* **-EPERM** if the helper cannot be used under the current
- * *skops*\ **->op**.
+ * operation.
*
- * long bpf_reserve_hdr_opt(struct bpf_sock_ops *skops, u32 len, u64 flags)
+ * long bpf_reserve_hdr_opt(void *ctx, u32 len, u64 flags)
* Description
* Reserve *len* bytes for the bpf header option. The
* space will be used by **bpf_store_hdr_opt**\ () later in
@@ -4905,7 +4912,9 @@ union bpf_attr {
* the total number of bytes will be reserved.
*
* This helper can only be called during
- * **BPF_SOCK_OPS_HDR_OPT_LEN_CB**.
+ * **BPF_SOCK_OPS_HDR_OPT_LEN_CB**, or from the
+ * **hdr_opt_len**\ () operator of the **bpf_tcp_ops**
+ * struct_ops.
*
* Return
* 0 on success, or negative error in case of failure:
@@ -4915,7 +4924,7 @@ union bpf_attr {
* **-ENOSPC** if there is not enough space in the header.
*
* **-EPERM** if the helper cannot be used under the current
- * *skops*\ **->op**.
+ * operation.
*
* void *bpf_inode_storage_get(struct bpf_map *map, void *inode, void *value, u64 flags)
* Description
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH bpf-next v2 11/15] bpf: tcp: Support selected sock_ops callbacks as struct_ops
From: Amery Hung @ 2026-06-23 17:49 UTC (permalink / raw)
To: bpf
Cc: netdev, alexei.starovoitov, andrii, daniel, eddyz87, memxor,
martin.lau, shakeel.butt, roman.gushchin, kuniyu, kerneljasonxing,
ameryhung, kernel-team
In-Reply-To: <20260623175006.3136053-1-ameryhung@gmail.com>
In LSFMMBPF 2025, I have talked about moving the BPF_PROG_TYPE_SOCK_OPS
to a struct_ops interface [1].
The BPF_SOCK_OPS_*_CB enum interface has grown over time as new TCP
callback points were added. A BPF_PROG_TYPE_SOCK_OPS program now
commonly needs a large switch on sock_ops->op, and the shared
bpf_sock_ops_kern context has become harder to extend because different
callbacks have different locking, argument, skb, and helper
requirements. The existing 'union { u32 args[4]; u32 replylong[4]; }' is
also not reliable in passing args to bpf prog when there are multiple
progs attached to a cgroup.
The above has already been solved in struct_ops. Add a TCP-specific
struct_ops type, bpf_tcp_ops, and support attaching it to cgroups.
This allows each callback have its own func signature and allows
the verifier to select kfuncs/helpers based on the specific
struct_ops member being implemented.
This patch wires up the following existing sock_ops callbacks:
- BPF_SOCK_OPS_TIMEOUT_INIT
- BPF_SOCK_OPS_RWND_INIT
- BPF_SOCK_OPS_RTT_CB
- BPF_SOCK_OPS_STATE_CB
- BPF_SOCK_OPS_RETRANS_CB
- BPF_SOCK_OPS_TCP_CONNECT_CB
- BPF_SOCK_OPS_TCP_LISTEN_CB
- BPF_SOCK_OPS_RTO_CB
- BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB
- BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB
BASE_RTT is ignored as it is not particularly useful. NEEDS_ECN should
be done in bpf-tcp-cc instead. The tstamp ones should be a separate
struct_ops (e.g. "bpf_sock_ops") that can work in both TCP and UDP.
timeout_init and rwnd_init could have a request_sock pointer. This patch
tries a different API and direclty passes the request_sock pointer as
an arg.
Two other approaches were considered before settling on having
bpf_get_retval() read the dispatcher's run_ctx via saved_run_ctx. The
first was to inherit the retval in the trampoline itself: add a helper
in the four __bpf_prog_enter*() paths that, for struct_ops programs,
copies the chained value from the caller's run_ctx (now saved_run_ctx)
into the program's own run_ctx. It works but puts a per-enter
program-type check on the generic trampoline fast path, taxing all
fentry/fexit/lsm callers for a cgroup-struct_ops-only feature. The
second was to do that same inherit only for the int-returning members
via a gen_prologue that emits a hidden kfunc at the start of
timeout_init/rwnd_init; this keeps the cost off the generic path and
scoped to bpf_tcp_ops, but needs a kfunc + BTF_ID + prologue-emission
machinery. The chosen approach avoids both: it touches neither the
trampoline nor the program, since saved_run_ctx already points at the
dispatcher's run_ctx that carries the value.
[1], page 13: https://drive.google.com/file/d/1wjKZth6T0llLJ_ONPAL_6Q_jbxbAjByp/view?usp=sharing
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
---
include/linux/bpf.h | 1 +
include/net/tcp.h | 113 +++++++++++++++++++++++++-
net/ipv4/Makefile | 1 +
net/ipv4/af_inet.c | 1 +
net/ipv4/bpf_tcp_ops.c | 177 +++++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp.c | 1 +
net/ipv4/tcp_input.c | 4 +
net/ipv4/tcp_output.c | 2 +
net/ipv4/tcp_timer.c | 1 +
9 files changed, 299 insertions(+), 2 deletions(-)
create mode 100644 net/ipv4/bpf_tcp_ops.c
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index df95ae690da5..91024d2da4ea 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -2597,6 +2597,7 @@ struct bpf_trace_run_ctx {
struct bpf_tramp_run_ctx {
struct bpf_run_ctx run_ctx;
u64 bpf_cookie;
+ int retval;
struct bpf_run_ctx *saved_run_ctx;
};
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 6d376ea4d1c0..2102f9f2afd6 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2953,12 +2953,120 @@ static inline void tcp_clear_sock_ops_cb_flags(struct sock *sk)
#endif
+#if defined(CONFIG_BPF_JIT) && defined(CONFIG_CGROUP_BPF)
+
+struct bpf_tcp_ops {
+ /* Should return the initial SYN (active open) or SYN-ACK (passive open)
+ * retransmission timeout. Return the timeout in jiffies, or <= 0 for
+ * the kernel default.
+ *
+ * @req: request_sock on the passive (synack) path; NULL otherwise.
+ */
+ int (*timeout_init)(struct sock *sk, struct request_sock *req);
+
+ /* Should return the initial advertised receive window, in packets,
+ * or < 0 for the kernel default. @req as in timeout_init().
+ */
+ int (*rwnd_init)(struct sock *sk, struct request_sock *req);
+
+ /* Called when an active connection becomes established.
+ * @skb is the SYNACK that completed the 3WHS, or NULL for a
+ * TCP_REPAIR socket (tcp_finish_connect() with no skb).
+ */
+ void (*active_established)(struct sock *sk, struct sk_buff *skb__nullable);
+
+ /* Called when a passive connection becomes established.
+ * @skb is the ACK that completed the 3WHS.
+ */
+ void (*passive_established)(struct sock *sk, struct sk_buff *skb);
+
+ /* Called when the retransmission timer fires. */
+ void (*rto)(struct sock *sk);
+
+ /* Called on every RTT sample.
+ * @mrtt: the measured RTT, in microseconds.
+ * @srtt: the updated smoothed RTT.
+ */
+ void (*rtt)(struct sock *sk, long mrtt, u32 srtt);
+
+ /* Called when the connection changes TCP state.
+ * @state: the new state (one of the TCP_* states).
+ */
+ void (*set_state)(struct sock *sk, int state);
+
+ /* Called when an skb is retransmitted.
+ * @skb: the retransmitted skb.
+ * @err: tcp_transmit_skb() return value (0 on success).
+ */
+ void (*retrans)(struct sock *sk, struct sk_buff *skb, int err);
+
+ /* Called right before an active connection is initialized. */
+ void (*connect)(struct sock *sk);
+
+ /* Called on listen(2), right after the socket enters TCP_LISTEN. */
+ void (*listen)(struct sock *sk);
+};
+
+#define bpf_tcp_ops_call(op, sk, ...) \
+do { \
+ if (cgroup_bpf_enabled(CGROUP_TCP_SOCK_OPS)) { \
+ const struct bpf_prog_array_item *item; \
+ const struct bpf_tcp_ops *tcp_ops; \
+ struct cgroup *cgrp; \
+ \
+ cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data); \
+ rcu_read_lock_dont_migrate(); \
+ bpf_cgroup_struct_ops_foreach(tcp_ops, item, cgrp, \
+ CGROUP_TCP_SOCK_OPS) { \
+ if (tcp_ops->op) \
+ tcp_ops->op(sk, ##__VA_ARGS__); \
+ } \
+ rcu_read_unlock_migrate(); \
+ } \
+} while (0)
+
+#define bpf_tcp_ops_call_int(op, init_retval, sk, ...) \
+({ \
+ int __retval = (init_retval); \
+ if (cgroup_bpf_enabled(CGROUP_TCP_SOCK_OPS)) { \
+ const struct bpf_prog_array_item *item; \
+ const struct bpf_tcp_ops *tcp_ops; \
+ struct bpf_tramp_run_ctx run_ctx; \
+ struct bpf_run_ctx *old_run_ctx; \
+ struct sock *__sk = sk_to_full_sk(sk); \
+ struct request_sock *req = NULL; \
+ struct cgroup *cgrp; \
+ \
+ if (__sk) { \
+ run_ctx.retval = (init_retval); \
+ cgrp = sock_cgroup_ptr(&__sk->sk_cgrp_data); \
+ if (!sk_fullsock(sk)) \
+ req = (struct request_sock *)sk; \
+ rcu_read_lock_dont_migrate(); \
+ old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx);\
+ bpf_cgroup_struct_ops_foreach(tcp_ops, item, cgrp, \
+ CGROUP_TCP_SOCK_OPS) { \
+ if (tcp_ops->op) \
+ run_ctx.retval = tcp_ops->op(__sk, req, ##__VA_ARGS__); \
+ } \
+ bpf_reset_run_ctx(old_run_ctx); \
+ rcu_read_unlock_migrate(); \
+ __retval = run_ctx.retval; \
+ } \
+ } \
+ __retval; \
+})
+#else
+#define bpf_tcp_ops_call(op, sk, ...) do { } while (0)
+#define bpf_tcp_ops_call_int(op, init_retval, sk, ...) (init_retval)
+#endif
+
static inline u32 tcp_timeout_init(struct sock *sk)
{
int timeout;
timeout = tcp_call_bpf(sk, BPF_SOCK_OPS_TIMEOUT_INIT, 0, NULL);
-
+ timeout = bpf_tcp_ops_call_int(timeout_init, timeout, sk);
if (timeout <= 0)
timeout = TCP_TIMEOUT_INIT;
return min_t(int, timeout, TCP_RTO_MAX);
@@ -2969,7 +3077,7 @@ static inline u32 tcp_rwnd_init_bpf(struct sock *sk)
int rwnd;
rwnd = tcp_call_bpf(sk, BPF_SOCK_OPS_RWND_INIT, 0, NULL);
-
+ rwnd = bpf_tcp_ops_call_int(rwnd_init, rwnd, sk);
if (rwnd < 0)
rwnd = 0;
return rwnd;
@@ -2984,6 +3092,7 @@ static inline void tcp_bpf_rtt(struct sock *sk, long mrtt, u32 srtt)
{
if (BPF_SOCK_OPS_TEST_FLAG(tcp_sk(sk), BPF_SOCK_OPS_RTT_CB_FLAG))
tcp_call_bpf_2arg(sk, BPF_SOCK_OPS_RTT_CB, mrtt, srtt);
+ bpf_tcp_ops_call(rtt, sk, mrtt, srtt);
}
#if IS_ENABLED(CONFIG_SMC)
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index 06e21c26b76f..afbac63d1cb4 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -70,6 +70,7 @@ obj-$(CONFIG_TCP_AO) += tcp_ao.o
ifeq ($(CONFIG_BPF_JIT),y)
obj-$(CONFIG_BPF_SYSCALL) += bpf_tcp_ca.o
+obj-$(CONFIG_CGROUP_BPF) += bpf_tcp_ops.o
endif
ifdef CONFIG_GCOV_PROFILE_NETFILTER
diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c
index 32d006c1a8ee..ac8431da67f4 100644
--- a/net/ipv4/af_inet.c
+++ b/net/ipv4/af_inet.c
@@ -227,6 +227,7 @@ int __inet_listen_sk(struct sock *sk, int backlog)
return err;
tcp_call_bpf(sk, BPF_SOCK_OPS_TCP_LISTEN_CB, 0, NULL);
+ bpf_tcp_ops_call(listen, sk);
}
return 0;
}
diff --git a/net/ipv4/bpf_tcp_ops.c b/net/ipv4/bpf_tcp_ops.c
new file mode 100644
index 000000000000..cf53c95a0dbc
--- /dev/null
+++ b/net/ipv4/bpf_tcp_ops.c
@@ -0,0 +1,177 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <linux/bpf.h>
+#include <linux/btf_ids.h>
+#include <linux/bpf_verifier.h>
+#include <net/bpf_sk_storage.h>
+#include <net/tcp.h>
+
+static int timeout_init_stub(struct sock *sk, struct request_sock *req__nullable)
+{
+ struct bpf_tramp_run_ctx *ctx =
+ container_of(current->bpf_ctx, struct bpf_tramp_run_ctx, run_ctx);
+
+ return ctx->retval;
+}
+
+static int rwnd_init_stub(struct sock *sk, struct request_sock *req__nullable)
+{
+ struct bpf_tramp_run_ctx *ctx =
+ container_of(current->bpf_ctx, struct bpf_tramp_run_ctx, run_ctx);
+
+ return ctx->retval;
+}
+
+static void active_established_stub(struct sock *sk, struct sk_buff *skb__nullable)
+{
+}
+
+static void passive_established_stub(struct sock *sk, struct sk_buff *skb)
+{
+}
+
+static void rto_stub(struct sock *sk)
+{
+}
+
+static void rtt_stub(struct sock *sk, long mrtt, u32 srtt)
+{
+}
+
+static void set_state_stub(struct sock *sk, int state)
+{
+}
+
+static void retrans_stub(struct sock *sk, struct sk_buff *skb, int err)
+{
+}
+
+static void connect_stub(struct sock *sk)
+{
+}
+
+static void listen_stub(struct sock *sk)
+{
+}
+
+static struct bpf_tcp_ops __bpf_tcp_ops = {
+ .timeout_init = timeout_init_stub,
+ .rwnd_init = rwnd_init_stub,
+ .active_established = active_established_stub,
+ .passive_established = passive_established_stub,
+ .rto = rto_stub,
+ .rtt = rtt_stub,
+ .set_state = set_state_stub,
+ .retrans = retrans_stub,
+ .connect = connect_stub,
+ .listen = listen_stub,
+};
+
+BPF_CALL_0(bpf_tcp_ops_get_retval)
+{
+ struct bpf_tramp_run_ctx *ctx =
+ container_of(current->bpf_ctx, struct bpf_tramp_run_ctx, run_ctx);
+
+ /* bpf_get_retval() is only exposed to timeout_init/rwnd_init, which
+ * always run via bpf_tcp_ops_call_int(). Its run_ctx carries the int
+ * return value chained across the bpf_tcp_ops attached to the cgroup
+ * and is this program's saved_run_ctx.
+ */
+ if (WARN_ON_ONCE(!ctx->saved_run_ctx))
+ return 0;
+
+ return container_of(ctx->saved_run_ctx, struct bpf_tramp_run_ctx,
+ run_ctx)->retval;
+}
+
+const struct bpf_func_proto bpf_tcp_ops_get_retval_proto = {
+ .func = bpf_tcp_ops_get_retval,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+};
+
+static const struct bpf_func_proto *
+get_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
+{
+ u32 moff = prog->aux->attach_st_ops_member_off;
+
+ switch (func_id) {
+ case BPF_FUNC_sk_storage_get:
+ return &bpf_sk_storage_get_proto;
+ case BPF_FUNC_sk_storage_delete:
+ return &bpf_sk_storage_delete_proto;
+ case BPF_FUNC_setsockopt:
+ /* The listener is not locked. */
+ if (moff == offsetof(struct bpf_tcp_ops, rwnd_init) ||
+ moff == offsetof(struct bpf_tcp_ops, timeout_init))
+ return NULL;
+ return &bpf_sk_setsockopt_proto;
+ case BPF_FUNC_getsockopt:
+ if (moff == offsetof(struct bpf_tcp_ops, rwnd_init) ||
+ moff == offsetof(struct bpf_tcp_ops, timeout_init))
+ return NULL;
+ return &bpf_sk_getsockopt_proto;
+ case BPF_FUNC_get_retval:
+ if (moff == offsetof(struct bpf_tcp_ops, timeout_init) ||
+ moff == offsetof(struct bpf_tcp_ops, rwnd_init))
+ return &bpf_tcp_ops_get_retval_proto;
+ return NULL;
+ default:
+ return bpf_base_func_proto(func_id, prog);
+ }
+}
+
+static bool is_valid_access(int off, int size, enum bpf_access_type type,
+ const struct bpf_prog *prog, struct bpf_insn_access_aux *info)
+{
+ if (!bpf_tracing_btf_ctx_access(off, size, type, prog, info))
+ return false;
+
+ if (base_type(info->reg_type) == PTR_TO_BTF_ID &&
+ !bpf_type_has_unsafe_modifiers(info->reg_type) &&
+ info->btf_id == btf_sock_ids[BTF_SOCK_TYPE_SOCK])
+ /* promote it to tcp_sock */
+ info->btf_id = btf_sock_ids[BTF_SOCK_TYPE_TCP];
+
+ return true;
+}
+
+static int bpf_tcp_ops_init_member(const struct btf_type *t,
+ const struct btf_member *member,
+ void *kdata, const void *udata)
+{
+ return 0;
+}
+
+static int bpf_tcp_ops_init(struct btf *btf)
+{
+ return 0;
+}
+
+static int bpf_tcp_ops_validate(void *kdata)
+{
+ return 0;
+}
+
+static const struct bpf_verifier_ops bpf_tcp_ops_verifier = {
+ .get_func_proto = get_func_proto,
+ .is_valid_access = is_valid_access,
+};
+
+static struct bpf_struct_ops bpf_tcp_ops = {
+ .verifier_ops = &bpf_tcp_ops_verifier,
+ .init_member = bpf_tcp_ops_init_member,
+ .init = bpf_tcp_ops_init,
+ .validate = bpf_tcp_ops_validate,
+ .name = "bpf_tcp_ops",
+ .cgroup_atype = CGROUP_TCP_SOCK_OPS,
+ .cfi_stubs = &__bpf_tcp_ops,
+ .owner = THIS_MODULE,
+};
+
+static int __init __bpf_tcp_ops_init(void)
+{
+ return register_bpf_struct_ops(&bpf_tcp_ops, bpf_tcp_ops);
+}
+late_initcall(__bpf_tcp_ops_init);
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 455441f1b694..94ed1ac2abc1 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2996,6 +2996,7 @@ void tcp_set_state(struct sock *sk, int state)
if (BPF_SOCK_OPS_TEST_FLAG(tcp_sk(sk), BPF_SOCK_OPS_STATE_CB_FLAG))
tcp_call_bpf_2arg(sk, BPF_SOCK_OPS_STATE_CB, oldstate, state);
+ bpf_tcp_ops_call(set_state, sk, state);
switch (state) {
case TCP_ESTABLISHED:
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 61045a8886e4..12fb690d21c4 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -6694,6 +6694,10 @@ void tcp_init_transfer(struct sock *sk, int bpf_op, struct sk_buff *skb)
tp->snd_cwnd_stamp = tcp_jiffies32;
bpf_skops_established(sk, bpf_op, skb);
+ if (bpf_op == BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB)
+ bpf_tcp_ops_call(active_established, sk, skb);
+ else
+ bpf_tcp_ops_call(passive_established, sk, skb);
/* Initialize congestion control unless BPF initialized it already: */
if (!icsk->icsk_ca_initialized)
tcp_init_congestion_control(sk);
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 26dd751ec72a..93f4a95399ea 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -3678,6 +3678,7 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs)
if (BPF_SOCK_OPS_TEST_FLAG(tp, BPF_SOCK_OPS_RETRANS_CB_FLAG))
tcp_call_bpf_3arg(sk, BPF_SOCK_OPS_RETRANS_CB,
TCP_SKB_CB(skb)->seq, segs, err);
+ bpf_tcp_ops_call(retrans, sk, skb, err);
if (unlikely(err) && err != -EBUSY)
NET_ADD_STATS(sock_net(sk), LINUX_MIB_TCPRETRANSFAIL, segs);
@@ -4298,6 +4299,7 @@ int tcp_connect(struct sock *sk)
int err;
tcp_call_bpf(sk, BPF_SOCK_OPS_TCP_CONNECT_CB, 0, NULL);
+ bpf_tcp_ops_call(connect, sk);
#if defined(CONFIG_TCP_MD5SIG) && defined(CONFIG_TCP_AO)
/* Has to be checked late, after setting daddr/saddr/ops.
diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
index bf171b5e1eb3..4337627ee0ea 100644
--- a/net/ipv4/tcp_timer.c
+++ b/net/ipv4/tcp_timer.c
@@ -290,6 +290,7 @@ static int tcp_write_timeout(struct sock *sk)
tcp_call_bpf_3arg(sk, BPF_SOCK_OPS_RTO_CB,
icsk->icsk_retransmits,
icsk->icsk_rto, (int)expired);
+ bpf_tcp_ops_call(rto, sk);
if (expired) {
/* Has it gone just too far? */
--
2.53.0-Meta
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox