* Re: [PATCH 1/1] ovpn: tcp - defer TX from softirq to workqueue
From: Antonio Quartulli @ 2026-05-03 21:14 UTC (permalink / raw)
To: Dao Zhong Ma, linux-kernel, netdev
Cc: sd, andrew+netdev, davem, edumazet, kuba, pabeni, Ralf Lici
In-Reply-To: <20260501145425.757147-2-cz1346219@gmail.com>
Hi,
On 01/05/2026 16:54, Dao Zhong Ma wrote:
> ovpn_tcp_send_skb() holds sk->sk_lock.slock while performing the full TCP
> send in softirq context. This can hold the spinlock for a long time
> (large skb), blocking lock_sock() users. This can starve the RCU GP
Are you sure that the stall is triggered by just processing 1 large skb?
> kthread and trigger RCU stalls warnings and hung tasks.
Can you trigger the stall/warning easily?
Any steps to reproduce?
>
> Defer the TCP send operation to process context:
> - In interrupt context, only enqueue the skb under the spinlock
> schedule tcp_tx_work.
> - In process context, dequeue and flush the send queue under lock_sock()
>
> This reduces the softirq critical section to a short duration, allowing
> lock_sock() users to make progress and preventing RCU stalls.
>
> Signed-off-by: Dao Zhong Ma <cz1346219@gmail.com>
As Jakub pointed out, this is breaking the selftests, therefore you
should first double check what's going wrong.
On top of that, could you please run some performance tests? I think
this is going to hit quite a penalty.
Especially if you have multiple peers and you defer all their traffic to
the same workqueue.
Regards,
--
Antonio Quartulli
OpenVPN Inc.
^ permalink raw reply
* Re: [PATCH iproute2-next 0/6] seg6: SRv6 Mobile User Plane (RFC 9433)
From: Stephen Hemminger @ 2026-05-03 21:05 UTC (permalink / raw)
To: Yuya Kusakabe; +Cc: dsahern, netdev
In-Reply-To: <20260503153006.900533-1-y-kusakabe@bbsakura.net>
On Mon, 4 May 2026 00:30:00 +0900
Yuya Kusakabe <yuya.kusakabe@gmail.com> wrote:
> From: Yuya Kusakabe <yuya.kusakabe@gmail.com>
>
> This series adds the iproute2 frontend for the SRv6 Mobile User Plane
> (MUP) endpoint behaviors of RFC 9433. It is sent in parallel with the
> matching kernel net-next series; each commit here is self-contained
> and brings in the seg6local UAPI bits it needs from the in-progress
> kernel UAPI header (include/uapi/linux/seg6_local.h):
>
> Section 6.2 End.MAP
> Section 6.3 End.M.GTP6.D
> Section 6.4 End.M.GTP6.D.Di
> Section 6.5 End.M.GTP6.E
> Section 6.6 End.M.GTP4.E
> Section 6.7 H.M.GTP4.D
>
> The series adds these seg6local CLI keywords:
>
> src IPv6 source-address template
> v4_mask_len length of the IPv4 DA portion of the SID, in
> bits (1..32)
> sr_prefix_len locator length of the egress End.M.GTP*.E SID,
> in bits (1..88, leaving 40 bits for the
> Args.Mob.Session field)
> v6_src_prefix_len Source UPF Prefix length P in the IPv6 SA
> template (1..127, defaults to 64); requires
> P + v4_mask_len <= 128
> pdu_type GTP-U PDU Session Container PDU Type (3GPP
> TS 38.415 Section 5.5.2): downlink|dl|uplink|ul
> or 0..15. When omitted, the egress emits a
> short GTPv1-U header (no PDU Session Container)
> regardless of the QFI in the SID; 5G N3
> deployments must set pdu_type explicitly.
>
> A small per-action attribute validator (introduced in patch 1 and
> extended by each subsequent behavior) rejects obvious typos in the
> seg6local block at the userspace layer instead of leaving the
> operator with an opaque kernel EINVAL.
>
> Link: https://datatracker.ietf.org/doc/html/rfc9433
>
> Yuya Kusakabe (6):
> seg6: add support for the End.MAP behavior
> seg6: add support for the End.M.GTP4.E behavior
> seg6: add support for the End.M.GTP6.E behavior
> seg6: add support for the End.M.GTP6.D behavior
> seg6: add support for the End.M.GTP6.D.Di behavior
> seg6: add support for the H.M.GTP4.D behavior
>
> include/uapi/linux/seg6_local.h | 17 +++
> ip/ip_common.h | 2 +-
> ip/ipnexthop.c | 2 +-
> ip/iproute.c | 14 +-
> ip/iproute_lwtunnel.c | 263 +++++++++++++++++++++++++++++++-
> man/man8/ip-route.8.in | 154 +++++++++++++++++++
> 6 files changed, 442 insertions(+), 10 deletions(-)
>
>
> base-commit: 4f5de57e2ff11a5925dacdf3deeeabee7ba9502a
Automated AI review of iproute2 is not setup yet, but manual run showed:
Subject: Re: [PATCH iproute2-next v1 RESEND 1/6] seg6: add support for the End.MAP behavior
On Mon, 4 May 2026, Yuya Kusakabe wrote:
> +static void seg6local_action_check_attrs(int action, int nh6_ok)
> +{
> + switch (action) {
> + case SEG6_LOCAL_ACTION_END_MAP:
> + if (!nh6_ok)
> + invarg("End.MAP requires \"nh6\"\n", "");
> + break;
> + }
> +}
The invarg() helper expects the invalid argument string as its second parameter,
not an empty string. This should be:
invarg("End.MAP requires \"nh6\"", argv_value);
However, since this is a validation function called after parsing and doesn't have
access to argv, a different error reporting approach is needed. Consider using
fprintf(stderr, ...) directly or restructuring to keep the argv value available.
This same issue appears in all patches in this series whenever invarg() is called
from seg6local_action_check_attrs():
- Patch 1: End.MAP validation
- Patch 2: End.M.GTP4.E validation (multiple instances)
- Patch 3: End.M.GTP6.E validation
- Patch 4: End.M.GTP6.D validation
- Patch 5: End.M.GTP6.D.Di validation
- Patch 6: H.M.GTP4.D validation
The rest of the implementation looks good:
- Correctly uses strcmp() instead of matches() for new argument parsing
- Properly uses print_XXX() helpers with PRINT_ANY for JSON output
- Includes proper documentation updates
Please fix the invarg() usage throughout the series.
^ permalink raw reply
* Re: [PATCH net-next v2 5/7] selftests: rds: Fix gcov and pcap collection
From: Allison Henderson @ 2026-05-03 20:49 UTC (permalink / raw)
To: Jakub Kicinski
Cc: netdev, pabeni, edumazet, horms, linux-rdma, linux-kselftest,
shuah
In-Reply-To: <20260502093858.35b27793@kernel.org>
On Sat, 2026-05-02 at 09:38 -0700, Jakub Kicinski wrote:
> On Fri, 01 May 2026 22:43:04 -0700 Allison Henderson wrote:
> > > point #2 my vng does have an overlayfs mounted over /tmp so the
> > > mountpoint check doesn't trigger IDK if this is what you meant
> > > or I have a different version.
> >
> > I did try using --overlay-rwdir, and I think that gives rw to the
> > guest, but it's ro to the host, so we dont get to keep the pcaps post
> > mortem.
> >
> > What we can do, if it sounds ok to you, is set up a temp scratch area
> > in the rds_logs folder so it's not mounting over /tmp, and then
> > run.sh can handle the mount with a cleanup trap addressing Sashiko's
> > concern
> >
> > Let me know what you think?
>
> Oh, you need these files for a post mortem analysis? I missed that
> point. IDK if there's a well established way to save extra debug info
> from the tests. runner has per_test_log_dir but I don't think it's
> exposed to tests? Until ksft has such a thing I'd probably go with
> an extra env variable to point the test to a specific dir.
> Which dir will depend on the CI harness. If var is not set - don't
> output the logs or use /tmp and clean up when test exits.
Ok, I will see if I can do some refactoring and pull the log dir out
of the test scripts and into an environment variable. I think that
should work if I adopt the $SUDO_USER pattern the other tests use.
Thank you for the reviews!
Allison
^ permalink raw reply
* Re: [PATCH net v6] ipv6: flowlabel: enforce per-netns limit for unprivileged callers
From: Willem de Bruijn @ 2026-05-03 20:43 UTC (permalink / raw)
To: Maoyi Xie, davem, kuba, pabeni, edumazet
Cc: dsahern, kuznet, willemb, willemdebruijn.kernel, netdev,
linux-kernel, stable
In-Reply-To: <20260502150918.4171847-1-maoyi.xie@ntu.edu.sg>
Maoyi Xie wrote:
> fl_size, fl_ht and ip6_fl_lock in net/ipv6/ip6_flowlabel.c are file
> scope and shared across netns. mem_check() reads fl_size to decide
> whether to deny non-CAP_NET_ADMIN callers; capable() runs against
> init_user_ns, so an unprivileged user in any non-init userns can
> push fl_size past FL_MAX_SIZE - FL_MAX_SIZE/4 and starve every
> other unprivileged userns on the host.
>
> Add struct netns_ipv6::flowlabel_count, bumped and decremented next
> to fl_size in fl_intern, ip6_fl_gc and ip6_fl_purge. The new field
> is placed in the existing 4-byte hole after ipmr_seq, so struct
> netns_ipv6 stays the same size on 64-bit builds.
>
> Bump FL_MAX_SIZE from 4096 to 8192. It has been 4096 since the file
> was added; machines and connection counts have grown.
>
> mem_check() folds an extra per-netns ceiling into the existing
> non-CAP_NET_ADMIN conditional. The ceiling is half of the total
> budget that unprivileged callers have ever been able to use, i.e.
> (FL_MAX_SIZE - FL_MAX_SIZE/4) / 2 = 3072 entries. With FL_MAX_SIZE
> doubled, this preserves the original per-user reach (~3K, what an
> unprivileged caller could already obtain before this change) while
> forcing an attacker to spread allocations across at least two
> netns to exhaust the global non-CAP_NET_ADMIN budget.
>
> CAP_NET_ADMIN against init_user_ns still bypasses both caps.
>
> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> Suggested-by: Willem de Bruijn <willemb@google.com>
> Cc: stable@vger.kernel.org # v5.15+
> Signed-off-by: Maoyi Xie <maoyi.xie@ntu.edu.sg>
> ---
> v6 (this submission, addressing v5 review by Willem):
> - Rebased onto current net (resolves the conflict on
> include/net/netns/ipv6.h that v5 hit. ipmr_seq is now
> atomic_t but remains 4 bytes, so flowlabel_count still
> fills the 4-byte hole after it).
> - Restored fl_free() to its original position in both
> ip6_fl_gc() and ip6_fl_purge(). v5 had moved fl_free()
> after the new atomic_dec() to avoid the use-after-free
> on fl->fl_net. v6 instead caches fl->fl_net into a
> local before fl_free() in ip6_fl_gc(), and uses the
> net argument already in scope in ip6_fl_purge().
> v5: replaced the per-netns ceiling FL_MAX_SIZE/8 with the
> computed unpriv_user_limit = (FL_MAX_SIZE - FL_MAX_SIZE/4)/2,
> which evaluates to 3072. v4's FL_MAX_SIZE/8 = 1024 would
> have reduced the per-user budget below the ~3K an
> unprivileged caller could already obtain before any of
> this work, defeating the reason FL_MAX_SIZE was doubled
> in the first place.
> v4: addressed Willem's v3 review on netdev. Dropped the
> flowlabel_has_excl cacheline argument in favour of "fills
> the existing 4-byte hole after ipmr_seq", and reordered
> atomic_dec(&...flowlabel_count) to sit immediately after
> atomic_dec(&fl_size) in ip6_fl_gc and ip6_fl_purge.
> v3: addressed Willem's review on the private security@ thread.
> Merged FL_MAX_SIZE doubling, dropped test data, moved
> flowlabel_count near ipmr_seq, inlined fl->fl_net in
> ip6_fl_gc.
> v2: per-netns counter + cap, sent to security@ as a 2-patch
> series.
> v1: fix-shape sketch in original disclosure.
>
> include/net/netns/ipv6.h | 1 +
> net/ipv6/ip6_flowlabel.c | 14 ++++++++++++--
> 2 files changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/include/net/netns/ipv6.h b/include/net/netns/ipv6.h
> index 499e42881..ef698f5fa 100644
> --- a/include/net/netns/ipv6.h
> +++ b/include/net/netns/ipv6.h
> @@ -119,6 +119,7 @@ struct netns_ipv6 {
> struct fib_notifier_ops *notifier_ops;
> struct fib_notifier_ops *ip6mr_notifier_ops;
> atomic_t ipmr_seq;
> + atomic_t flowlabel_count;
> struct {
> struct hlist_head head;
> spinlock_t lock;
> diff --git a/net/ipv6/ip6_flowlabel.c b/net/ipv6/ip6_flowlabel.c
> index c92f98c6f..28e43718d 100644
> --- a/net/ipv6/ip6_flowlabel.c
> +++ b/net/ipv6/ip6_flowlabel.c
> @@ -36,7 +36,7 @@
> /* FL hash table */
>
> #define FL_MAX_PER_SOCK 32
> -#define FL_MAX_SIZE 4096
> +#define FL_MAX_SIZE 8192
> #define FL_HASH_MASK 255
> #define FL_HASH(l) (ntohl(l)&FL_HASH_MASK)
>
> @@ -161,9 +161,12 @@ static void ip6_fl_gc(struct timer_list *unused)
> fl->expires = ttd;
> ttd = fl->expires;
> if (time_after_eq(now, ttd)) {
> + struct net *net = fl->fl_net;
> +
> *flp = fl->next;
> fl_free(fl);
> atomic_dec(&fl_size);
> + atomic_dec(&net->ipv6.flowlabel_count);
If resubmitting, moving fl_free here makes sense (only the second case
was entirely unnecessary).
> continue;
> }
> if (!sched || time_before(ttd, sched))
> @@ -197,6 +200,7 @@ static void __net_exit ip6_fl_purge(struct net *net)
> *flp = fl->next;
> fl_free(fl);
> atomic_dec(&fl_size);
> + atomic_dec(&net->ipv6.flowlabel_count);
> continue;
> }
> flp = &fl->next;
> @@ -245,6 +249,7 @@ static struct ip6_flowlabel *fl_intern(struct net *net,
> fl->next = fl_ht[FL_HASH(fl->label)];
> rcu_assign_pointer(fl_ht[FL_HASH(fl->label)], fl);
> atomic_inc(&fl_size);
> + atomic_inc(&net->ipv6.flowlabel_count);
> spin_unlock_bh(&ip6_fl_lock);
> rcu_read_unlock();
> return NULL;
> @@ -464,6 +469,9 @@ fl_create(struct net *net, struct sock *sk, struct in6_flowlabel_req *freq,
>
> static int mem_check(struct sock *sk)
> {
> + const int unpriv_total_limit = FL_MAX_SIZE - (FL_MAX_SIZE / 4);
> + const int unpriv_user_limit = unpriv_total_limit / 2;
> + struct net *net = sock_net(sk);
> int room = FL_MAX_SIZE - atomic_read(&fl_size);
> struct ipv6_fl_socklist *sfl;
> int count = 0;
> @@ -478,7 +486,9 @@ static int mem_check(struct sock *sk)
>
> if (room <= 0 ||
> ((count >= FL_MAX_PER_SOCK ||
> - (count > 0 && room < FL_MAX_SIZE/2) || room < FL_MAX_SIZE/4) &&
> + (count > 0 && room < FL_MAX_SIZE/2) ||
> + room < FL_MAX_SIZE/4 ||
And here make checkpatch happy and add spaces around the division
operator.
> + atomic_read(&net->ipv6.flowlabel_count) >= unpriv_user_limit) &&
> !capable(CAP_NET_ADMIN)))
> return -ENOBUFS;
>
> --
> 2.34.1
>
^ permalink raw reply
* Re: [PATCH net v6] ipv6: flowlabel: enforce per-netns limit for unprivileged callers
From: Willem de Bruijn @ 2026-05-03 20:40 UTC (permalink / raw)
To: Maoyi Xie, davem, kuba, pabeni, edumazet
Cc: dsahern, kuznet, willemb, willemdebruijn.kernel, netdev,
linux-kernel, stable
In-Reply-To: <20260502150918.4171847-1-maoyi.xie@ntu.edu.sg>
Maoyi Xie wrote:
> fl_size, fl_ht and ip6_fl_lock in net/ipv6/ip6_flowlabel.c are file
> scope and shared across netns. mem_check() reads fl_size to decide
> whether to deny non-CAP_NET_ADMIN callers; capable() runs against
> init_user_ns, so an unprivileged user in any non-init userns can
> push fl_size past FL_MAX_SIZE - FL_MAX_SIZE/4 and starve every
> other unprivileged userns on the host.
>
> Add struct netns_ipv6::flowlabel_count, bumped and decremented next
> to fl_size in fl_intern, ip6_fl_gc and ip6_fl_purge. The new field
> is placed in the existing 4-byte hole after ipmr_seq, so struct
> netns_ipv6 stays the same size on 64-bit builds.
>
> Bump FL_MAX_SIZE from 4096 to 8192. It has been 4096 since the file
> was added; machines and connection counts have grown.
>
> mem_check() folds an extra per-netns ceiling into the existing
> non-CAP_NET_ADMIN conditional. The ceiling is half of the total
> budget that unprivileged callers have ever been able to use, i.e.
> (FL_MAX_SIZE - FL_MAX_SIZE/4) / 2 = 3072 entries. With FL_MAX_SIZE
> doubled, this preserves the original per-user reach (~3K, what an
> unprivileged caller could already obtain before this change) while
> forcing an attacker to spread allocations across at least two
> netns to exhaust the global non-CAP_NET_ADMIN budget.
>
> CAP_NET_ADMIN against init_user_ns still bypasses both caps.
>
> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> Suggested-by: Willem de Bruijn <willemb@google.com>
> Cc: stable@vger.kernel.org # v5.15+
> Signed-off-by: Maoyi Xie <maoyi.xie@ntu.edu.sg>
Reviewed-by: Willem de Bruijn <willemb@google.com>
> ---
> v6 (this submission, addressing v5 review by Willem):
> - Rebased onto current net (resolves the conflict on
> include/net/netns/ipv6.h that v5 hit. ipmr_seq is now
> atomic_t but remains 4 bytes, so flowlabel_count still
> fills the 4-byte hole after it).
> - Restored fl_free() to its original position in both
> ip6_fl_gc() and ip6_fl_purge(). v5 had moved fl_free()
> after the new atomic_dec() to avoid the use-after-free
> on fl->fl_net. v6 instead caches fl->fl_net into a
> local before fl_free() in ip6_fl_gc(), and uses the
> net argument already in scope in ip6_fl_purge().
> v5: replaced the per-netns ceiling FL_MAX_SIZE/8 with the
> computed unpriv_user_limit = (FL_MAX_SIZE - FL_MAX_SIZE/4)/2,
> which evaluates to 3072. v4's FL_MAX_SIZE/8 = 1024 would
> have reduced the per-user budget below the ~3K an
> unprivileged caller could already obtain before any of
> this work, defeating the reason FL_MAX_SIZE was doubled
> in the first place.
> v4: addressed Willem's v3 review on netdev. Dropped the
> flowlabel_has_excl cacheline argument in favour of "fills
> the existing 4-byte hole after ipmr_seq", and reordered
> atomic_dec(&...flowlabel_count) to sit immediately after
> atomic_dec(&fl_size) in ip6_fl_gc and ip6_fl_purge.
> v3: addressed Willem's review on the private security@ thread.
> Merged FL_MAX_SIZE doubling, dropped test data, moved
> flowlabel_count near ipmr_seq, inlined fl->fl_net in
> ip6_fl_gc.
> v2: per-netns counter + cap, sent to security@ as a 2-patch
> series.
> v1: fix-shape sketch in original disclosure.
>
> include/net/netns/ipv6.h | 1 +
> net/ipv6/ip6_flowlabel.c | 14 ++++++++++++--
> 2 files changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/include/net/netns/ipv6.h b/include/net/netns/ipv6.h
> index 499e42881..ef698f5fa 100644
> --- a/include/net/netns/ipv6.h
> +++ b/include/net/netns/ipv6.h
> @@ -119,6 +119,7 @@ struct netns_ipv6 {
> struct fib_notifier_ops *notifier_ops;
> struct fib_notifier_ops *ip6mr_notifier_ops;
> atomic_t ipmr_seq;
> + atomic_t flowlabel_count;
> struct {
> struct hlist_head head;
> spinlock_t lock;
> diff --git a/net/ipv6/ip6_flowlabel.c b/net/ipv6/ip6_flowlabel.c
> index c92f98c6f..28e43718d 100644
> --- a/net/ipv6/ip6_flowlabel.c
> +++ b/net/ipv6/ip6_flowlabel.c
> @@ -36,7 +36,7 @@
> /* FL hash table */
>
> #define FL_MAX_PER_SOCK 32
> -#define FL_MAX_SIZE 4096
> +#define FL_MAX_SIZE 8192
> #define FL_HASH_MASK 255
> #define FL_HASH(l) (ntohl(l)&FL_HASH_MASK)
>
> @@ -161,9 +161,12 @@ static void ip6_fl_gc(struct timer_list *unused)
> fl->expires = ttd;
> ttd = fl->expires;
> if (time_after_eq(now, ttd)) {
> + struct net *net = fl->fl_net;
> +
> *flp = fl->next;
> fl_free(fl);
> atomic_dec(&fl_size);
> + atomic_dec(&net->ipv6.flowlabel_count);
> continue;
> }
> if (!sched || time_before(ttd, sched))
> @@ -197,6 +200,7 @@ static void __net_exit ip6_fl_purge(struct net *net)
> *flp = fl->next;
> fl_free(fl);
> atomic_dec(&fl_size);
> + atomic_dec(&net->ipv6.flowlabel_count);
> continue;
> }
> flp = &fl->next;
> @@ -245,6 +249,7 @@ static struct ip6_flowlabel *fl_intern(struct net *net,
> fl->next = fl_ht[FL_HASH(fl->label)];
> rcu_assign_pointer(fl_ht[FL_HASH(fl->label)], fl);
> atomic_inc(&fl_size);
> + atomic_inc(&net->ipv6.flowlabel_count);
> spin_unlock_bh(&ip6_fl_lock);
> rcu_read_unlock();
> return NULL;
> @@ -464,6 +469,9 @@ fl_create(struct net *net, struct sock *sk, struct in6_flowlabel_req *freq,
>
> static int mem_check(struct sock *sk)
> {
> + const int unpriv_total_limit = FL_MAX_SIZE - (FL_MAX_SIZE / 4);
> + const int unpriv_user_limit = unpriv_total_limit / 2;
> + struct net *net = sock_net(sk);
> int room = FL_MAX_SIZE - atomic_read(&fl_size);
Sashiko correctly points out that the existing fl_size and this new
test are racy.
fl_intern takes ip6_fl_lock not much later. Moving the tests inside
that critical section also avoids the need for atomic ops.
That can be a separate patch. Ideally that conversion happens before
adding this new field, so that it can be backported without conflicts.
Basically, move the spin_lock_bh(&ip6_sk_fl_lock) out of fl_intern
into its only caller ipv6_flowlabel_get, to also cover mem_check, and
converting fl_size to a regular int.
Let me know if you're up for that and adding it to this series, else I
can prepare it.
Does not look as impactful in practice, but it's still a small fix.
> struct ipv6_fl_socklist *sfl;
> int count = 0;
> @@ -478,7 +486,9 @@ static int mem_check(struct sock *sk)
>
> if (room <= 0 ||
> ((count >= FL_MAX_PER_SOCK ||
> - (count > 0 && room < FL_MAX_SIZE/2) || room < FL_MAX_SIZE/4) &&
> + (count > 0 && room < FL_MAX_SIZE/2) ||
> + room < FL_MAX_SIZE/4 ||
> + atomic_read(&net->ipv6.flowlabel_count) >= unpriv_user_limit) &&
> !capable(CAP_NET_ADMIN)))
> return -ENOBUFS;
>
> --
> 2.34.1
>
^ permalink raw reply
* Re: [PATCH] MAINTAINERS: Update address for Michael Grzeschik
From: Michael Grzeschik @ 2026-05-03 20:35 UTC (permalink / raw)
To: Michael Grzeschik; +Cc: linux-kernel, netdev
In-Reply-To: <20260426-maintainer-v1-1-d74d2f4343f7@kernel.org>
Adding netdev to Cc:
On Sun, Apr 26, 2026 at 12:20:59AM +0200, Michael Grzeschik wrote:
> Since I am moving from Pengutronix update my email address for the
> ARCNET subsystems to point to my kernel.org address.
>
> Also update .mailmap.
>
> Acked-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
> Signed-off-by: Michael Grzeschik <mgr@kernel.org>
> ---
> .mailmap | 2 ++
> MAINTAINERS | 2 +-
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/.mailmap b/.mailmap
> index 34acd34bbf9bf..7b01246851705 100644
> --- a/.mailmap
> +++ b/.mailmap
> @@ -579,6 +579,8 @@ Mayuresh Janorkar <mayur@ti.com>
> Md Sadre Alam <quic_mdalam@quicinc.com> <mdalam@codeaurora.org>
> Miaoqing Pan <quic_miaoqing@quicinc.com> <miaoqing@codeaurora.org>
> Michael Buesch <m@bues.ch>
> +Michal Grzeschik <mgr@kernel.org> <m.grzeschik@pengutronix.de>
> +Michal Grzeschik <mgr@kernel.org> <mgr@pengutronix.de>
> Michael Riesch <michael.riesch@collabora.com> <michael.riesch@wolfvision.net>
> Michal Simek <michal.simek@amd.com> <michal.simek@xilinx.com>
> Michel Dänzer <michel@tungstengraphics.com>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 2fb1c75afd163..48b601d36b846 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2058,7 +2058,7 @@ F: Documentation/devicetree/bindings/display/snps,arcpgu.txt
> F: drivers/gpu/drm/tiny/arcpgu.c
>
> ARCNET NETWORK LAYER
> -M: Michael Grzeschik <m.grzeschik@pengutronix.de>
> +M: Michael Grzeschik <mgr@kernel.org>
> L: netdev@vger.kernel.org
> S: Maintained
> F: drivers/net/arcnet/
>
> ---
> base-commit: 897d54018cc9aa97fd1529ca08a53b429d05a566
> change-id: 20260426-maintainer-eecd046c4a87
>
> Best regards,
> --
> Michael Grzeschik <mgr@kernel.org>
^ permalink raw reply
* [PATCH net-next V3 7/7] net/mlx5: E-Switch, load reps via work queue after registration
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
In-Reply-To: <20260503202726.266415-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
mlx5_eswitch_register_vport_reps() only installs representor callbacks and
marks the rep type as registered. If the E-Switch is already in switchdev
mode, the newly registered rep type must then be loaded for already enabled
vports.
That load path needs to run under the devlink lock, which is not held by
the auxiliary driver registration context. Queue the reload to the E-Switch
workqueue, whose handler acquires the devlink lock, and load the relevant
representors from there.
Since representor registration runs from sleepable auxiliary-driver
context, queue the late reload with GFP_KERNEL. The functions-change
notifier path remains the GFP_ATOMIC user of mlx5_esw_add_work().
The unregister path is unchanged and still unloads representors
synchronously while tearing down the registered callbacks.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../mellanox/mlx5/core/eswitch_offloads.c | 34 +++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 8a7491e9f13d..dea5647de548 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -4565,6 +4565,38 @@ mlx5_eswitch_register_vport_reps_blocked(struct mlx5_eswitch *esw,
}
}
+static void mlx5_eswitch_reload_reps_blocked(struct mlx5_eswitch *esw)
+{
+ struct mlx5_vport *vport;
+ unsigned long i;
+
+ if (esw->mode != MLX5_ESWITCH_OFFLOADS)
+ return;
+
+ if (mlx5_esw_offloads_rep_load(esw, MLX5_VPORT_UPLINK))
+ return;
+
+ mlx5_esw_for_each_vport(esw, i, vport) {
+ if (!vport)
+ continue;
+ if (!vport->enabled)
+ continue;
+ if (vport->vport == MLX5_VPORT_UPLINK)
+ continue;
+ if (!mlx5_eswitch_vport_has_rep(esw, vport->vport))
+ continue;
+
+ mlx5_esw_offloads_rep_load(esw, vport->vport);
+ }
+}
+
+static void mlx5_eswitch_reload_reps(struct mlx5_eswitch *esw)
+{
+ mlx5_esw_reps_block(esw);
+ mlx5_eswitch_reload_reps_blocked(esw);
+ mlx5_esw_reps_unblock(esw);
+}
+
static void
mlx5_eswitch_register_vport_reps_locked(struct mlx5_eswitch *esw,
const struct mlx5_eswitch_rep_ops *ops,
@@ -4576,6 +4608,8 @@ mlx5_eswitch_register_vport_reps_locked(struct mlx5_eswitch *esw,
mlx5_esw_reps_block(esw);
mlx5_eswitch_register_vport_reps_blocked(esw, ops, rep_type);
mlx5_esw_reps_unblock(esw);
+
+ mlx5_esw_add_work(esw, mlx5_eswitch_reload_reps, GFP_KERNEL);
}
void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw,
--
2.44.0
^ permalink raw reply related
* [PATCH net-next V3 6/7] net/mlx5: E-Switch, unwind only newly loaded representor types
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
In-Reply-To: <20260503202726.266415-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
__esw_offloads_load_rep() may return success without invoking the
representor load callback when the representor type is already loaded.
On a later load failure, mlx5_esw_offloads_rep_load() unconditionally
unloaded all previously iterated representor types. This could unload
representor types that were already loaded before this load attempt.
Track which representor types were actually loaded by the current call and
unwind only those on error. Also restore the representor state back to
REP_REGISTERED when the load callback itself fails.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../mellanox/mlx5/core/eswitch_offloads.c | 38 ++++++++++++++-----
1 file changed, 29 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index a393efaa2fd7..8a7491e9f13d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -2786,13 +2786,28 @@ void esw_offloads_cleanup(struct mlx5_eswitch *esw)
}
static int __esw_offloads_load_rep(struct mlx5_eswitch *esw,
- struct mlx5_eswitch_rep *rep, u8 rep_type)
+ struct mlx5_eswitch_rep *rep,
+ u8 rep_type, bool *newly_loaded)
{
+ int err;
+
mlx5_esw_assert_reps_locked(esw);
+ if (newly_loaded)
+ *newly_loaded = false;
+
if (atomic_cmpxchg(&rep->rep_data[rep_type].state,
- REP_REGISTERED, REP_LOADED) == REP_REGISTERED)
- return esw->offloads.rep_ops[rep_type]->load(esw->dev, rep);
+ REP_REGISTERED, REP_LOADED) != REP_REGISTERED)
+ return 0;
+
+ err = esw->offloads.rep_ops[rep_type]->load(esw->dev, rep);
+ if (err) {
+ atomic_set(&rep->rep_data[rep_type].state, REP_REGISTERED);
+ return err;
+ }
+
+ if (newly_loaded)
+ *newly_loaded = true;
return 0;
}
@@ -2822,22 +2837,27 @@ static void __unload_reps_all_vport(struct mlx5_eswitch *esw, u8 rep_type)
static int mlx5_esw_offloads_rep_load(struct mlx5_eswitch *esw, u16 vport_num)
{
struct mlx5_eswitch_rep *rep;
+ unsigned long loaded = 0;
+ bool newly_loaded;
int rep_type;
int err;
rep = mlx5_eswitch_get_rep(esw, vport_num);
for (rep_type = 0; rep_type < NUM_REP_TYPES; rep_type++) {
- err = __esw_offloads_load_rep(esw, rep, rep_type);
+ err = __esw_offloads_load_rep(esw, rep, rep_type,
+ &newly_loaded);
if (err)
goto err_reps;
+ if (newly_loaded)
+ loaded |= BIT(rep_type);
}
return 0;
err_reps:
- atomic_set(&rep->rep_data[rep_type].state, REP_REGISTERED);
- for (--rep_type; rep_type >= 0; rep_type--)
- __esw_offloads_unload_rep(esw, rep, rep_type);
+ while (--rep_type >= 0)
+ if (test_bit(rep_type, &loaded))
+ __esw_offloads_unload_rep(esw, rep, rep_type);
return err;
}
@@ -3591,13 +3611,13 @@ int mlx5_eswitch_reload_ib_reps(struct mlx5_eswitch *esw)
if (atomic_read(&rep->rep_data[REP_ETH].state) != REP_LOADED)
return 0;
- ret = __esw_offloads_load_rep(esw, rep, REP_IB);
+ ret = __esw_offloads_load_rep(esw, rep, REP_IB, NULL);
if (ret)
return ret;
mlx5_esw_for_each_rep(esw, i, rep) {
if (atomic_read(&rep->rep_data[REP_ETH].state) == REP_LOADED)
- __esw_offloads_load_rep(esw, rep, REP_IB);
+ __esw_offloads_load_rep(esw, rep, REP_IB, NULL);
}
return 0;
--
2.44.0
^ permalink raw reply related
* [PATCH net-next V3 5/7] net/mlx5: E-Switch, serialize representor lifecycle
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
In-Reply-To: <20260503202726.266415-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
Representor callbacks can be registered and unregistered while the
E-Switch is already in switchdev mode, and the same E-Switch may also be
reconfigured by devlink, VF changes and SF changes. Serialize these paths
with the per-E-Switch representor mutex instead of relying on ad-hoc bit
state and wait queues.
Take the representor lock around the mode transition, VF/SF representor
changes and representor ops registration. Keep mode_lock and the
representor lock unnested by using the operation flag while the mode lock
is dropped. During mode changes, drop the representor lock around the
auxiliary bus rescan because driver bind/unbind may register or unregister
representor ops.
Split representor ops registration into locked public wrappers and blocked
internal helpers, clear the ops pointer on unregister, and add nested
wrappers for the shared-FDB master IB path that registers peer
representor ops while another E-Switch representor lock is already held.
On unregister, always call __unload_reps_all_vport() before marking reps
unregistered and clearing rep_ops. The per-representor state check makes
this a no-op for types that were not loaded, so unregister no longer has
to infer load state from esw->mode.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
drivers/infiniband/hw/mlx5/ib_rep.c | 6 +-
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 10 ++
.../mellanox/mlx5/core/eswitch_offloads.c | 105 ++++++++++++++++--
.../ethernet/mellanox/mlx5/core/sf/devlink.c | 5 +
include/linux/mlx5/eswitch.h | 6 +
5 files changed, 120 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c
index 1709b628702e..65d8767d1830 100644
--- a/drivers/infiniband/hw/mlx5/ib_rep.c
+++ b/drivers/infiniband/hw/mlx5/ib_rep.c
@@ -262,9 +262,10 @@ mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep)
struct mlx5_core_dev *peer_mdev;
struct mlx5_eswitch *esw;
+ /* Called while the master E-Switch reps_lock is held. */
mlx5_lag_for_each_peer_mdev(mdev, peer_mdev, i) {
esw = peer_mdev->priv.eswitch;
- mlx5_eswitch_unregister_vport_reps(esw, REP_IB);
+ mlx5_eswitch_unregister_vport_reps_nested(esw, REP_IB);
}
mlx5_ib_release_transport(mdev);
}
@@ -284,9 +285,10 @@ static void mlx5_ib_register_peer_vport_reps(struct mlx5_core_dev *mdev)
struct mlx5_eswitch *esw;
int i;
+ /* Called while the master E-Switch reps_lock is held. */
mlx5_lag_for_each_peer_mdev(mdev, peer_mdev, i) {
esw = peer_mdev->priv.eswitch;
- mlx5_eswitch_register_vport_reps(esw, &rep_ops, REP_IB);
+ mlx5_eswitch_register_vport_reps_nested(esw, &rep_ops, REP_IB);
}
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 66a773a99876..f70737437954 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1712,6 +1712,7 @@ int mlx5_eswitch_enable(struct mlx5_eswitch *esw, int num_vfs)
mlx5_lag_disable_change(esw->dev);
mlx5_eswitch_invalidate_wq(esw);
+ mlx5_esw_reps_block(esw);
if (!mlx5_esw_is_fdb_created(esw)) {
ret = mlx5_eswitch_enable_locked(esw, num_vfs);
@@ -1735,6 +1736,8 @@ int mlx5_eswitch_enable(struct mlx5_eswitch *esw, int num_vfs)
}
}
+ mlx5_esw_reps_unblock(esw);
+
if (toggle_lag)
mlx5_lag_enable_change(esw->dev);
@@ -1759,6 +1762,7 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw, bool clear_vf)
esw->esw_funcs.num_vfs, esw->esw_funcs.num_ec_vfs, esw->enabled_vports);
mlx5_eswitch_invalidate_wq(esw);
+ mlx5_esw_reps_block(esw);
if (!mlx5_core_is_ecpf(esw->dev)) {
mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs);
@@ -1770,6 +1774,8 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw, bool clear_vf)
mlx5_eswitch_clear_ec_vf_vports_info(esw);
}
+ mlx5_esw_reps_unblock(esw);
+
if (esw->mode == MLX5_ESWITCH_OFFLOADS) {
struct devlink *devlink = priv_to_devlink(esw->dev);
@@ -1825,7 +1831,11 @@ void mlx5_eswitch_disable(struct mlx5_eswitch *esw)
devl_assert_locked(priv_to_devlink(esw->dev));
mlx5_lag_disable_change(esw->dev);
+
+ mlx5_esw_reps_block(esw);
mlx5_eswitch_disable_locked(esw);
+ mlx5_esw_reps_unblock(esw);
+
esw->mode = MLX5_ESWITCH_LEGACY;
mlx5_lag_enable_change(esw->dev);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index af7d0d58c048..a393efaa2fd7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -36,6 +36,7 @@
#include <linux/mlx5/mlx5_ifc.h>
#include <linux/mlx5/vport.h>
#include <linux/mlx5/fs.h>
+#include <linux/lockdep.h>
#include "mlx5_core.h"
#include "eswitch.h"
#include "esw/indir_table.h"
@@ -2413,11 +2414,21 @@ static int esw_create_restore_table(struct mlx5_eswitch *esw)
return err;
}
+static void mlx5_esw_assert_reps_locked(struct mlx5_eswitch *esw)
+{
+ lockdep_assert_held(&esw->offloads.reps_lock);
+}
+
void mlx5_esw_reps_block(struct mlx5_eswitch *esw)
{
mutex_lock(&esw->offloads.reps_lock);
}
+static void mlx5_esw_reps_block_nested(struct mlx5_eswitch *esw)
+{
+ mutex_lock_nested(&esw->offloads.reps_lock, SINGLE_DEPTH_NESTING);
+}
+
void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw)
{
mutex_unlock(&esw->offloads.reps_lock);
@@ -2425,21 +2436,22 @@ void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw)
static void esw_mode_change(struct mlx5_eswitch *esw, u16 mode)
{
+ mlx5_esw_reps_unblock(esw);
mlx5_devcom_comp_lock(esw->dev->priv.hca_devcom_comp);
if (esw->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_IB_ADEV ||
mlx5_core_mp_enabled(esw->dev)) {
esw->mode = mode;
- mlx5_rescan_drivers_locked(esw->dev);
- mlx5_devcom_comp_unlock(esw->dev->priv.hca_devcom_comp);
- return;
+ goto out;
}
esw->dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
mlx5_rescan_drivers_locked(esw->dev);
esw->mode = mode;
esw->dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
+out:
mlx5_rescan_drivers_locked(esw->dev);
mlx5_devcom_comp_unlock(esw->dev->priv.hca_devcom_comp);
+ mlx5_esw_reps_block(esw);
}
static void mlx5_esw_fdb_drop_destroy(struct mlx5_eswitch *esw)
@@ -2776,6 +2788,8 @@ void esw_offloads_cleanup(struct mlx5_eswitch *esw)
static int __esw_offloads_load_rep(struct mlx5_eswitch *esw,
struct mlx5_eswitch_rep *rep, u8 rep_type)
{
+ mlx5_esw_assert_reps_locked(esw);
+
if (atomic_cmpxchg(&rep->rep_data[rep_type].state,
REP_REGISTERED, REP_LOADED) == REP_REGISTERED)
return esw->offloads.rep_ops[rep_type]->load(esw->dev, rep);
@@ -2786,6 +2800,8 @@ static int __esw_offloads_load_rep(struct mlx5_eswitch *esw,
static void __esw_offloads_unload_rep(struct mlx5_eswitch *esw,
struct mlx5_eswitch_rep *rep, u8 rep_type)
{
+ mlx5_esw_assert_reps_locked(esw);
+
if (atomic_cmpxchg(&rep->rep_data[rep_type].state,
REP_LOADED, REP_REGISTERED) == REP_LOADED) {
if (rep_type == REP_ETH)
@@ -3691,6 +3707,7 @@ static void esw_vfs_changed_event_handler(struct mlx5_eswitch *esw)
if (new_num_vfs == esw->esw_funcs.num_vfs || host_pf_disabled)
goto free;
+ mlx5_esw_reps_block(esw);
/* Number of VFs can only change from "0 to x" or "x to 0". */
if (esw->esw_funcs.num_vfs > 0) {
mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs);
@@ -3700,9 +3717,11 @@ static void esw_vfs_changed_event_handler(struct mlx5_eswitch *esw)
err = mlx5_eswitch_load_vf_vports(esw, new_num_vfs,
MLX5_VPORT_UC_ADDR_CHANGE);
if (err)
- goto free;
+ goto unblock;
}
esw->esw_funcs.num_vfs = new_num_vfs;
+unblock:
+ mlx5_esw_reps_unblock(esw);
free:
kvfree(out);
}
@@ -4190,9 +4209,14 @@ int mlx5_devlink_eswitch_mode_set(struct devlink *devlink, u16 mode,
goto unlock;
}
+ /* Keep mode_lock and reps_lock unnested. The operation flag excludes
+ * mode users while mode_lock is dropped before taking reps_lock.
+ */
esw->eswitch_operation_in_progress = true;
up_write(&esw->mode_lock);
+ mlx5_esw_reps_block(esw);
+
if (mlx5_mode == MLX5_ESWITCH_OFFLOADS &&
!mlx5_devlink_netdev_netns_immutable_set(devlink, true)) {
NL_SET_ERR_MSG_MOD(extack,
@@ -4225,6 +4249,10 @@ int mlx5_devlink_eswitch_mode_set(struct devlink *devlink, u16 mode,
skip:
if (mlx5_mode == MLX5_ESWITCH_OFFLOADS && err)
mlx5_devlink_netdev_netns_immutable_set(devlink, false);
+ /* Reconfiguration is done; drop reps_lock before taking mode_lock again
+ * to clear the operation flag.
+ */
+ mlx5_esw_reps_unblock(esw);
down_write(&esw->mode_lock);
esw->eswitch_operation_in_progress = false;
unlock:
@@ -4498,9 +4526,10 @@ mlx5_eswitch_vport_has_rep(const struct mlx5_eswitch *esw, u16 vport_num)
return true;
}
-void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw,
- const struct mlx5_eswitch_rep_ops *ops,
- u8 rep_type)
+static void
+mlx5_eswitch_register_vport_reps_blocked(struct mlx5_eswitch *esw,
+ const struct mlx5_eswitch_rep_ops *ops,
+ u8 rep_type)
{
struct mlx5_eswitch_rep_data *rep_data;
struct mlx5_eswitch_rep *rep;
@@ -4515,21 +4544,77 @@ void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw,
}
}
}
+
+static void
+mlx5_eswitch_register_vport_reps_locked(struct mlx5_eswitch *esw,
+ const struct mlx5_eswitch_rep_ops *ops,
+ u8 rep_type, bool nested)
+{
+ if (nested)
+ mlx5_esw_reps_block_nested(esw);
+ else
+ mlx5_esw_reps_block(esw);
+ mlx5_eswitch_register_vport_reps_blocked(esw, ops, rep_type);
+ mlx5_esw_reps_unblock(esw);
+}
+
+void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw,
+ const struct mlx5_eswitch_rep_ops *ops,
+ u8 rep_type)
+{
+ mlx5_eswitch_register_vport_reps_locked(esw, ops, rep_type, false);
+}
EXPORT_SYMBOL(mlx5_eswitch_register_vport_reps);
-void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_type)
+void
+mlx5_eswitch_register_vport_reps_nested(struct mlx5_eswitch *esw,
+ const struct mlx5_eswitch_rep_ops *ops,
+ u8 rep_type)
+{
+ mlx5_eswitch_register_vport_reps_locked(esw, ops, rep_type, true);
+}
+EXPORT_SYMBOL(mlx5_eswitch_register_vport_reps_nested);
+
+static void
+mlx5_eswitch_unregister_vport_reps_blocked(struct mlx5_eswitch *esw,
+ u8 rep_type)
{
struct mlx5_eswitch_rep *rep;
unsigned long i;
- if (esw->mode == MLX5_ESWITCH_OFFLOADS)
- __unload_reps_all_vport(esw, rep_type);
+ __unload_reps_all_vport(esw, rep_type);
mlx5_esw_for_each_rep(esw, i, rep)
atomic_set(&rep->rep_data[rep_type].state, REP_UNREGISTERED);
+
+ esw->offloads.rep_ops[rep_type] = NULL;
+}
+
+static void
+mlx5_eswitch_unregister_vport_reps_locked(struct mlx5_eswitch *esw,
+ u8 rep_type, bool nested)
+{
+ if (nested)
+ mlx5_esw_reps_block_nested(esw);
+ else
+ mlx5_esw_reps_block(esw);
+ mlx5_eswitch_unregister_vport_reps_blocked(esw, rep_type);
+ mlx5_esw_reps_unblock(esw);
+}
+
+void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_type)
+{
+ mlx5_eswitch_unregister_vport_reps_locked(esw, rep_type, false);
}
EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_reps);
+void mlx5_eswitch_unregister_vport_reps_nested(struct mlx5_eswitch *esw,
+ u8 rep_type)
+{
+ mlx5_eswitch_unregister_vport_reps_locked(esw, rep_type, true);
+}
+EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_reps_nested);
+
void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type)
{
struct mlx5_eswitch_rep *rep;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c
index 8503e532f423..2fc69897e35b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c
@@ -245,8 +245,10 @@ static int mlx5_sf_add(struct mlx5_core_dev *dev, struct mlx5_sf_table *table,
if (IS_ERR(sf))
return PTR_ERR(sf);
+ mlx5_esw_reps_block(esw);
err = mlx5_eswitch_load_sf_vport(esw, sf->hw_fn_id, MLX5_VPORT_UC_ADDR_CHANGE,
&sf->dl_port, new_attr->controller, new_attr->sfnum);
+ mlx5_esw_reps_unblock(esw);
if (err)
goto esw_err;
*dl_port = &sf->dl_port.dl_port;
@@ -367,7 +369,10 @@ int mlx5_devlink_sf_port_del(struct devlink *devlink,
struct mlx5_sf_table *table = dev->priv.sf_table;
struct mlx5_sf *sf = mlx5_sf_by_dl_port(dl_port);
+ mlx5_esw_reps_block(dev->priv.eswitch);
mlx5_sf_del(table, sf);
+ mlx5_esw_reps_unblock(dev->priv.eswitch);
+
return 0;
}
diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h
index 3b29a3c6794d..a0dd162baa78 100644
--- a/include/linux/mlx5/eswitch.h
+++ b/include/linux/mlx5/eswitch.h
@@ -63,7 +63,13 @@ struct mlx5_eswitch_rep {
void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw,
const struct mlx5_eswitch_rep_ops *ops,
u8 rep_type);
+void
+mlx5_eswitch_register_vport_reps_nested(struct mlx5_eswitch *esw,
+ const struct mlx5_eswitch_rep_ops *ops,
+ u8 rep_type);
void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_type);
+void mlx5_eswitch_unregister_vport_reps_nested(struct mlx5_eswitch *esw,
+ u8 rep_type);
void *mlx5_eswitch_get_proto_dev(struct mlx5_eswitch *esw,
u16 vport_num,
u8 rep_type);
--
2.44.0
^ permalink raw reply related
* [PATCH net-next V3 4/7] net/mlx5: Lag, avoid LAG and representor lock cycles
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
In-Reply-To: <20260503202726.266415-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
The LAG shared-FDB and multiport E-Switch transitions rescan auxiliary
devices and reload IB representors while holding ldev->lock. Driver
bind/unbind paths may register or unregister E-Switch representor ops, and
representor load paths may enter LAG code, so holding ldev->lock across
those calls creates lock-order cycles with the E-Switch representor lock.
Keep the devcom component locked for the transition, but drop ldev->lock
before rescanning auxiliary devices or reloading IB representors. Mark the
LAG transition as in progress while the lock is dropped and assert the
devcom lock where the helper relies on it. This preserves LAG serialization
while avoiding ldev->lock nesting under E-Switch representor registration.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/core/lag/lag.c | 142 ++++++++++++++----
.../net/ethernet/mellanox/mlx5/core/lag/lag.h | 7 +-
.../ethernet/mellanox/mlx5/core/lag/mpesw.c | 10 +-
.../ethernet/mellanox/mlx5/core/lib/devcom.c | 8 +
.../ethernet/mellanox/mlx5/core/lib/devcom.h | 1 +
5 files changed, 134 insertions(+), 34 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
index a474f970e056..e77f9931c39c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
@@ -1063,37 +1063,99 @@ bool mlx5_lag_check_prereq(struct mlx5_lag *ldev)
return true;
}
-void mlx5_lag_add_devices(struct mlx5_lag *ldev)
+static void mlx5_lag_assert_locked_transition(struct mlx5_lag *ldev)
{
+ struct mlx5_devcom_comp_dev *devcom = NULL;
struct lag_func *pf;
int i;
- mlx5_ldev_for_each(i, 0, ldev) {
- pf = mlx5_lag_pf(ldev, i);
- if (pf->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)
- continue;
+ lockdep_assert_held(&ldev->lock);
- pf->dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(pf->dev);
+ i = mlx5_get_next_ldev_func(ldev, 0);
+ if (i < MLX5_MAX_PORTS) {
+ pf = mlx5_lag_pf(ldev, i);
+ devcom = pf->dev->priv.hca_devcom_comp;
}
+ mlx5_devcom_comp_assert_locked(devcom);
}
-void mlx5_lag_remove_devices(struct mlx5_lag *ldev)
+static void mlx5_lag_drop_lock_for_reps(struct mlx5_lag *ldev)
+{
+ mlx5_lag_assert_locked_transition(ldev);
+
+ /* Keep PF membership stable while ldev->lock is dropped. Device add
+ * and remove paths observe mode_changes_in_progress and retry.
+ */
+ ldev->mode_changes_in_progress++;
+ mutex_unlock(&ldev->lock);
+}
+
+static void mlx5_lag_retake_lock_after_reps(struct mlx5_lag *ldev)
{
+ mutex_lock(&ldev->lock);
+ ldev->mode_changes_in_progress--;
+}
+
+void mlx5_lag_rescan_dev_locked(struct mlx5_lag *ldev,
+ struct mlx5_core_dev *dev,
+ bool enable)
+{
+ if (dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)
+ return;
+
+ if (enable)
+ dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
+ else
+ dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
+
+ /* Auxiliary bus probe/remove can register or unregister representor
+ * callbacks and take reps_lock. Drop ldev->lock so the only ordering
+ * remains reps_lock -> ldev->lock from representor callbacks.
+ */
+ mlx5_lag_drop_lock_for_reps(ldev);
+ mlx5_rescan_drivers_locked(dev);
+ mlx5_lag_retake_lock_after_reps(ldev);
+}
+
+static void mlx5_lag_rescan_devices_locked(struct mlx5_lag *ldev, bool enable)
+{
+ struct mlx5_core_dev *devs[MLX5_MAX_PORTS];
struct lag_func *pf;
+ int num_devs = 0;
int i;
+ mlx5_lag_assert_locked_transition(ldev);
+
mlx5_ldev_for_each(i, 0, ldev) {
pf = mlx5_lag_pf(ldev, i);
if (pf->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)
continue;
- pf->dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(pf->dev);
+ if (enable)
+ pf->dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
+ else
+ pf->dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
+ devs[num_devs++] = pf->dev;
}
+
+ mlx5_lag_drop_lock_for_reps(ldev);
+ for (i = 0; i < num_devs; i++)
+ mlx5_rescan_drivers_locked(devs[i]);
+ mlx5_lag_retake_lock_after_reps(ldev);
}
-int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, bool cont_on_fail)
+void mlx5_lag_add_devices(struct mlx5_lag *ldev)
+{
+ mlx5_lag_rescan_devices_locked(ldev, true);
+}
+
+void mlx5_lag_remove_devices(struct mlx5_lag *ldev)
+{
+ mlx5_lag_rescan_devices_locked(ldev, false);
+}
+
+static int mlx5_lag_reload_ib_reps_unlocked(struct mlx5_lag *ldev, u32 flags,
+ bool cont_on_fail)
{
struct lag_func *pf;
int ret;
@@ -1105,7 +1167,9 @@ int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, bool cont_on_fail)
struct mlx5_eswitch *esw;
esw = pf->dev->priv.eswitch;
+ mlx5_esw_reps_block(esw);
ret = mlx5_eswitch_reload_ib_reps(esw);
+ mlx5_esw_reps_unblock(esw);
if (ret && !cont_on_fail)
return ret;
}
@@ -1114,6 +1178,34 @@ int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, bool cont_on_fail)
return 0;
}
+static int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags,
+ bool cont_on_fail)
+{
+ int ret;
+
+ /* The HCA devcom component lock serializes LAG mode transitions while
+ * ldev->lock is dropped here. Dropping ldev->lock is required because
+ * the reload takes the per-E-Switch reps_lock, and representor
+ * load/unload callbacks can re-enter LAG netdev add/remove and take
+ * ldev->lock. Keep the ordering reps_lock -> ldev->lock.
+ */
+ mlx5_lag_drop_lock_for_reps(ldev);
+ ret = mlx5_lag_reload_ib_reps_unlocked(ldev, flags, cont_on_fail);
+ mlx5_lag_retake_lock_after_reps(ldev);
+
+ return ret;
+}
+
+int mlx5_lag_reload_ib_reps_from_locked(struct mlx5_lag *ldev, u32 flags,
+ bool cont_on_fail)
+{
+ int ret;
+
+ ret = mlx5_lag_reload_ib_reps(ldev, flags, cont_on_fail);
+
+ return ret;
+}
+
void mlx5_disable_lag(struct mlx5_lag *ldev)
{
bool shared_fdb = test_bit(MLX5_LAG_MODE_FLAG_SHARED_FDB, &ldev->mode_flags);
@@ -1132,10 +1224,7 @@ void mlx5_disable_lag(struct mlx5_lag *ldev)
if (shared_fdb) {
mlx5_lag_remove_devices(ldev);
} else if (roce_lag) {
- if (!(dev0->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)) {
- dev0->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(dev0);
- }
+ mlx5_lag_rescan_dev_locked(ldev, dev0, false);
mlx5_ldev_for_each(i, 0, ldev) {
if (i == idx)
continue;
@@ -1151,8 +1240,9 @@ void mlx5_disable_lag(struct mlx5_lag *ldev)
mlx5_lag_add_devices(ldev);
if (shared_fdb)
- mlx5_lag_reload_ib_reps(ldev, MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV,
- true);
+ mlx5_lag_reload_ib_reps_from_locked(ldev,
+ MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV,
+ true);
}
bool mlx5_lag_shared_fdb_supported(struct mlx5_lag *ldev)
@@ -1409,7 +1499,8 @@ static void mlx5_do_bond(struct mlx5_lag *ldev)
if (shared_fdb || roce_lag)
mlx5_lag_add_devices(ldev);
if (shared_fdb)
- mlx5_lag_reload_ib_reps(ldev, 0, true);
+ mlx5_lag_reload_ib_reps_from_locked(ldev, 0,
+ true);
return;
}
@@ -1417,8 +1508,7 @@ static void mlx5_do_bond(struct mlx5_lag *ldev)
if (roce_lag) {
struct mlx5_core_dev *dev;
- dev0->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(dev0);
+ mlx5_lag_rescan_dev_locked(ldev, dev0, true);
mlx5_ldev_for_each(i, 0, ldev) {
if (i == idx)
continue;
@@ -1427,15 +1517,15 @@ static void mlx5_do_bond(struct mlx5_lag *ldev)
mlx5_nic_vport_enable_roce(dev);
}
} else if (shared_fdb) {
- dev0->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(dev0);
- err = mlx5_lag_reload_ib_reps(ldev, 0, false);
+ mlx5_lag_rescan_dev_locked(ldev, dev0, true);
+ err = mlx5_lag_reload_ib_reps_from_locked(ldev, 0,
+ false);
if (err) {
- dev0->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(dev0);
+ mlx5_lag_rescan_dev_locked(ldev, dev0, false);
mlx5_deactivate_lag(ldev);
mlx5_lag_add_devices(ldev);
- mlx5_lag_reload_ib_reps(ldev, 0, true);
+ mlx5_lag_reload_ib_reps_from_locked(ldev, 0,
+ true);
mlx5_core_err(dev0, "Failed to enable lag\n");
return;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
index daca8ebd5256..6afe7707d076 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
@@ -164,6 +164,9 @@ void mlx5_disable_lag(struct mlx5_lag *ldev);
void mlx5_lag_remove_devices(struct mlx5_lag *ldev);
int mlx5_deactivate_lag(struct mlx5_lag *ldev);
void mlx5_lag_add_devices(struct mlx5_lag *ldev);
+void mlx5_lag_rescan_dev_locked(struct mlx5_lag *ldev,
+ struct mlx5_core_dev *dev,
+ bool enable);
struct mlx5_devcom_comp_dev *mlx5_lag_get_devcom_comp(struct mlx5_lag *ldev);
#ifdef CONFIG_MLX5_ESWITCH
@@ -199,6 +202,6 @@ int mlx5_get_next_ldev_func(struct mlx5_lag *ldev, int start_idx);
int mlx5_lag_get_dev_index_by_seq(struct mlx5_lag *ldev, int seq);
int mlx5_lag_num_devs(struct mlx5_lag *ldev);
int mlx5_lag_num_netdevs(struct mlx5_lag *ldev);
-int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags,
- bool cont_on_fail);
+int mlx5_lag_reload_ib_reps_from_locked(struct mlx5_lag *ldev, u32 flags,
+ bool cont_on_fail);
#endif /* __MLX5_LAG_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
index edcd06f3be7a..8a349f8fd823 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
@@ -100,9 +100,8 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev)
goto err_add_devices;
}
- dev0->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(dev0);
- err = mlx5_lag_reload_ib_reps(ldev, 0, false);
+ mlx5_lag_rescan_dev_locked(ldev, dev0, true);
+ err = mlx5_lag_reload_ib_reps_from_locked(ldev, 0, false);
if (err)
goto err_rescan_drivers;
@@ -111,12 +110,11 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev)
return 0;
err_rescan_drivers:
- dev0->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
- mlx5_rescan_drivers_locked(dev0);
+ mlx5_lag_rescan_dev_locked(ldev, dev0, false);
mlx5_deactivate_lag(ldev);
err_add_devices:
mlx5_lag_add_devices(ldev);
- mlx5_lag_reload_ib_reps(ldev, 0, true);
+ mlx5_lag_reload_ib_reps_from_locked(ldev, 0, true);
mlx5_mpesw_metadata_cleanup(ldev);
return err;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c
index 4b5ac2db55ce..d40c53193ea8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c
@@ -3,6 +3,7 @@
#include <linux/mlx5/vport.h>
#include <linux/list.h>
+#include <linux/lockdep.h>
#include "lib/devcom.h"
#include "lib/mlx5.h"
#include "mlx5_core.h"
@@ -438,3 +439,10 @@ int mlx5_devcom_comp_trylock(struct mlx5_devcom_comp_dev *devcom)
return 0;
return down_write_trylock(&devcom->comp->sem);
}
+
+void mlx5_devcom_comp_assert_locked(struct mlx5_devcom_comp_dev *devcom)
+{
+ if (!devcom)
+ return;
+ lockdep_assert_held_write(&devcom->comp->sem);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h
index 91e5ae529d5c..316052a85ca5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h
@@ -75,5 +75,6 @@ void *mlx5_devcom_get_next_peer_data_rcu(struct mlx5_devcom_comp_dev *devcom,
void mlx5_devcom_comp_lock(struct mlx5_devcom_comp_dev *devcom);
void mlx5_devcom_comp_unlock(struct mlx5_devcom_comp_dev *devcom);
int mlx5_devcom_comp_trylock(struct mlx5_devcom_comp_dev *devcom);
+void mlx5_devcom_comp_assert_locked(struct mlx5_devcom_comp_dev *devcom);
#endif /* __LIB_MLX5_DEVCOM_H__ */
--
2.44.0
^ permalink raw reply related
* [PATCH net-next V3 3/7] net/mlx5: E-Switch, add representor lifecycle lock
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
In-Reply-To: <20260503202726.266415-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
Add a per-E-Switch mutex for serializing representor lifecycle work and
provide small helpers for taking and dropping it. Initialize and destroy
the mutex with the E-Switch offloads state.
Add the lock and helper API first. Follow-up patches will take the lock in
the individual representor lifecycle components. This keeps the functional
changes split by component and leaves this patch without intended behavior
change, making the series easier to review and bisectable.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/eswitch.h | 6 ++++++
.../ethernet/mellanox/mlx5/core/eswitch_offloads.c | 12 ++++++++++++
2 files changed, 18 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 2fd601bd102f..3858690e09b4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -316,6 +316,7 @@ struct mlx5_esw_offload {
DECLARE_HASHTABLE(termtbl_tbl, 8);
struct mutex termtbl_mutex; /* protects termtbl hash */
struct xarray vhca_map;
+ struct mutex reps_lock; /* protects representor load/unload/register */
const struct mlx5_eswitch_rep_ops *rep_ops[NUM_REP_TYPES];
u8 inline_mode;
atomic64_t num_flows;
@@ -951,6 +952,8 @@ mlx5_esw_lag_demux_fg_create(struct mlx5_eswitch *esw,
struct mlx5_flow_handle *
mlx5_esw_lag_demux_rule_create(struct mlx5_eswitch *esw, u16 vport_num,
struct mlx5_flow_table *lag_ft);
+void mlx5_esw_reps_block(struct mlx5_eswitch *esw);
+void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw);
#else /* CONFIG_MLX5_ESWITCH */
/* eswitch API stubs */
static inline int mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; }
@@ -1028,6 +1031,9 @@ mlx5_esw_host_functions_enabled(const struct mlx5_core_dev *dev)
return true;
}
+static inline void mlx5_esw_reps_block(struct mlx5_eswitch *esw) {}
+static inline void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw) {}
+
static inline bool
mlx5_esw_vport_vhca_id(struct mlx5_eswitch *esw, u16 vportn, u16 *vhca_id)
{
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 69134ce2a908..af7d0d58c048 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -2413,6 +2413,16 @@ static int esw_create_restore_table(struct mlx5_eswitch *esw)
return err;
}
+void mlx5_esw_reps_block(struct mlx5_eswitch *esw)
+{
+ mutex_lock(&esw->offloads.reps_lock);
+}
+
+void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw)
+{
+ mutex_unlock(&esw->offloads.reps_lock);
+}
+
static void esw_mode_change(struct mlx5_eswitch *esw, u16 mode)
{
mlx5_devcom_comp_lock(esw->dev->priv.hca_devcom_comp);
@@ -2645,6 +2655,7 @@ static void esw_offloads_cleanup_reps(struct mlx5_eswitch *esw)
mlx5_esw_for_each_rep(esw, i, rep)
mlx5_esw_offloads_rep_cleanup(esw, rep);
xa_destroy(&esw->offloads.vport_reps);
+ mutex_destroy(&esw->offloads.reps_lock);
}
static int esw_offloads_init_reps(struct mlx5_eswitch *esw)
@@ -2654,6 +2665,7 @@ static int esw_offloads_init_reps(struct mlx5_eswitch *esw)
int err;
xa_init(&esw->offloads.vport_reps);
+ mutex_init(&esw->offloads.reps_lock);
mlx5_esw_for_each_vport(esw, i, vport) {
err = mlx5_esw_offloads_rep_add(esw, vport);
--
2.44.0
^ permalink raw reply related
* [PATCH net-next V3 2/7] net/mlx5: E-Switch, let esw work callers choose GFP flags
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
In-Reply-To: <20260503202726.266415-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
mlx5_esw_add_work() always allocates the queued work item with
GFP_ATOMIC. That is required for the E-Switch functions-change notifier,
but not every caller of this helper will run from atomic context.
Pass an allocation flag to mlx5_esw_add_work() and keep the notifier
caller using GFP_ATOMIC. This allows sleepable callers to use GFP_KERNEL
instead of unnecessarily relying on atomic reserves.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 69ddf56e2fc9..69134ce2a908 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -3736,11 +3736,12 @@ static void esw_wq_handler(struct work_struct *work)
}
static int mlx5_esw_add_work(struct mlx5_eswitch *esw,
- void (*func)(struct mlx5_eswitch *esw))
+ void (*func)(struct mlx5_eswitch *esw),
+ gfp_t gfp)
{
struct mlx5_host_work *host_work;
- host_work = kzalloc_obj(*host_work, GFP_ATOMIC);
+ host_work = kzalloc_obj(*host_work, gfp);
if (!host_work)
return -ENOMEM;
@@ -3764,7 +3765,8 @@ int mlx5_esw_funcs_changed_handler(struct notifier_block *nb,
esw_funcs = mlx5_nb_cof(nb, struct mlx5_esw_functions, nb);
esw = container_of(esw_funcs, struct mlx5_eswitch, esw_funcs);
- ret = mlx5_esw_add_work(esw, esw_vfs_changed_event_handler);
+ ret = mlx5_esw_add_work(esw, esw_vfs_changed_event_handler,
+ GFP_ATOMIC);
if (ret)
return NOTIFY_DONE;
--
2.44.0
^ permalink raw reply related
* [PATCH net-next V3 1/7] net/mlx5: Lag: refactor representor reload handling
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
In-Reply-To: <20260503202726.266415-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
Representor reload during LAG/MPESW transitions has to be repeated in
several flows, and each open-coded loop was easy to get out of sync
when adding new flags or tweaking error handling. Move the sequencing
into a single helper so that all call sites share the same ordering
and checks.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Reviewed-by: Shay Drori <shayd@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/core/lag/lag.c | 45 +++++++++++--------
.../net/ethernet/mellanox/mlx5/core/lag/lag.h | 2 +
.../ethernet/mellanox/mlx5/core/lag/mpesw.c | 12 ++---
3 files changed, 33 insertions(+), 26 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
index 449e4bd86c06..a474f970e056 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
@@ -1093,6 +1093,27 @@ void mlx5_lag_remove_devices(struct mlx5_lag *ldev)
}
}
+int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, bool cont_on_fail)
+{
+ struct lag_func *pf;
+ int ret;
+ int i;
+
+ mlx5_ldev_for_each(i, 0, ldev) {
+ pf = mlx5_lag_pf(ldev, i);
+ if (!(pf->dev->priv.flags & flags)) {
+ struct mlx5_eswitch *esw;
+
+ esw = pf->dev->priv.eswitch;
+ ret = mlx5_eswitch_reload_ib_reps(esw);
+ if (ret && !cont_on_fail)
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
void mlx5_disable_lag(struct mlx5_lag *ldev)
{
bool shared_fdb = test_bit(MLX5_LAG_MODE_FLAG_SHARED_FDB, &ldev->mode_flags);
@@ -1130,9 +1151,8 @@ void mlx5_disable_lag(struct mlx5_lag *ldev)
mlx5_lag_add_devices(ldev);
if (shared_fdb)
- mlx5_ldev_for_each(i, 0, ldev)
- if (!(mlx5_lag_pf(ldev, i)->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV))
- mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch);
+ mlx5_lag_reload_ib_reps(ldev, MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV,
+ true);
}
bool mlx5_lag_shared_fdb_supported(struct mlx5_lag *ldev)
@@ -1388,10 +1408,8 @@ static void mlx5_do_bond(struct mlx5_lag *ldev)
if (err) {
if (shared_fdb || roce_lag)
mlx5_lag_add_devices(ldev);
- if (shared_fdb) {
- mlx5_ldev_for_each(i, 0, ldev)
- mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch);
- }
+ if (shared_fdb)
+ mlx5_lag_reload_ib_reps(ldev, 0, true);
return;
}
@@ -1409,24 +1427,15 @@ static void mlx5_do_bond(struct mlx5_lag *ldev)
mlx5_nic_vport_enable_roce(dev);
}
} else if (shared_fdb) {
- int i;
-
dev0->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
mlx5_rescan_drivers_locked(dev0);
-
- mlx5_ldev_for_each(i, 0, ldev) {
- err = mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch);
- if (err)
- break;
- }
-
+ err = mlx5_lag_reload_ib_reps(ldev, 0, false);
if (err) {
dev0->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
mlx5_rescan_drivers_locked(dev0);
mlx5_deactivate_lag(ldev);
mlx5_lag_add_devices(ldev);
- mlx5_ldev_for_each(i, 0, ldev)
- mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch);
+ mlx5_lag_reload_ib_reps(ldev, 0, true);
mlx5_core_err(dev0, "Failed to enable lag\n");
return;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
index 6c911374f409..daca8ebd5256 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
@@ -199,4 +199,6 @@ int mlx5_get_next_ldev_func(struct mlx5_lag *ldev, int start_idx);
int mlx5_lag_get_dev_index_by_seq(struct mlx5_lag *ldev, int seq);
int mlx5_lag_num_devs(struct mlx5_lag *ldev);
int mlx5_lag_num_netdevs(struct mlx5_lag *ldev);
+int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags,
+ bool cont_on_fail);
#endif /* __MLX5_LAG_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
index 5eea12a6887a..edcd06f3be7a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
@@ -70,7 +70,6 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev)
int idx = mlx5_lag_get_dev_index_by_seq(ldev, MLX5_LAG_P1);
struct mlx5_core_dev *dev0;
int err;
- int i;
if (ldev->mode == MLX5_LAG_MODE_MPESW)
return 0;
@@ -103,11 +102,9 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev)
dev0->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
mlx5_rescan_drivers_locked(dev0);
- mlx5_ldev_for_each(i, 0, ldev) {
- err = mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch);
- if (err)
- goto err_rescan_drivers;
- }
+ err = mlx5_lag_reload_ib_reps(ldev, 0, false);
+ if (err)
+ goto err_rescan_drivers;
mlx5_lag_set_vports_agg_speed(ldev);
@@ -119,8 +116,7 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev)
mlx5_deactivate_lag(ldev);
err_add_devices:
mlx5_lag_add_devices(ldev);
- mlx5_ldev_for_each(i, 0, ldev)
- mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch);
+ mlx5_lag_reload_ib_reps(ldev, 0, true);
mlx5_mpesw_metadata_cleanup(ldev);
return err;
}
--
2.44.0
^ permalink raw reply related
* [PATCH net-next V3 0/7] net/mlx5: Improve representor lifecycle and late IB representor loading
From: Tariq Toukan @ 2026-05-03 20:27 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Leon Romanovsky, Jason Gunthorpe, Saeed Mahameed, Tariq Toukan,
Mark Bloch, Shay Drory, Or Har-Toov, Edward Srouji, Simon Horman,
Maher Sanalla, Parav Pandit, Patrisious Haddad, Kees Cook,
Gerd Bayer, Moshe Shemesh, Carolina Jubran, Cosmin Ratiu,
linux-rdma, linux-kernel, netdev, Gal Pressman, Dragos Tatulea
Hi,
Find detailed description by Mark below.
Regards,
Tariq
This series addresses two problems that have been present for years, and
fixes one representor reload error-unwind case exposed while making the
reload path reusable.
First, there is no coordination between E-Switch reconfiguration and
representor registration. The E-Switch can be mid-way through a mode
change or VF count update while mlx5_ib walks in and registers or
unregisters representors. Nothing stops them. The race window is small
and there is no field report, but it is clearly wrong.
Second, loading mlx5_ib while the device is already in switchdev mode
does not bring up the IB representors. mlx5_eswitch_register_vport_reps()
only stores callbacks; nobody triggers the actual load after registration.
The series fixes the registration race with a per-E-Switch representor
mutex. The lock is introduced first, then LAG shared-FDB and multiport
E-Switch transitions are adjusted so auxiliary device rescans and IB
representor reloads do not hold ldev->lock while taking the representor
lock. This keeps the intermediate commits bisectable before the stricter
E-Switch serialization and lock assertions are enabled.
After the LAG ordering is fixed, all E-Switch reconfiguration paths that
create, destroy, load, or unload representors take the representor mutex.
esw_mode_change() deliberately drops the mutex around
mlx5_rescan_drivers_locked(), because auxiliary probe and remove paths
re-enter mlx5_eswitch_register_vport_reps() and
mlx5_eswitch_unregister_vport_reps() on the same thread.
The shared-FDB peer IB registration path can hold one E-Switch
representor mutex and then register peer representor ops on another
E-Switch. The series annotates that case as nested locking so lockdep can
distinguish it from recursive locking on the same E-Switch.
For the missing IB representors, mlx5_eswitch_register_vport_reps() queues
a work item that acquires the devlink lock and loads all relevant
representors. This is the change that actually fixes the long-standing
bug.
The reload path also learns to track which representor types were loaded by
the current attempt, so an error does not unload representors that were
already active before the retry.
Patch 1 is cleanup. LAG and MPESW had the same representor reload
sequence duplicated in several places and the copies had started to
drift. This consolidates them into one helper.
Patch 2 lets E-Switch workqueue callers choose GFP allocation flags.
Patch 3 adds the per-E-Switch representor lifecycle lock and helper APIs.
Patch 4 adjusts the LAG shared-FDB and multiport E-Switch transitions so
auxiliary device rescans and IB representor reloads run without
ldev->lock held while taking the representor lock.
Patch 5 protects the E-Switch reconfiguration, representor registration
and peer IB representor paths with the representor lock.
Patch 6 fixes representor load error unwind so only representor types
loaded by the current attempt are unloaded on failure.
Patch 7 moves the representor load triggered by
mlx5_eswitch_register_vport_reps() onto the work queue. This is the patch
that fixes IB representors not coming up when mlx5_ib is loaded while the
device is already in switchdev mode.
Changes:
v2 -> v3:
Drop the default switchdev module parameter patch. The proper user facing
interface is still under discussion, and this may be better handled by
devlink core infrastructure.
Patch 2: Add a new patch, per Sashiko's feedback, that lets E-Switch
workqueue callers pass GFP allocation flags to mlx5_esw_add_work(). The
functions-change notifier keeps using GFP_ATOMIC, while sleepable callers
can use GFP_KERNEL.
Patch 5: The unregister path now always unloads the selected representor
type before marking it unregistered and clearing rep_ops. It no longer
depends on esw->mode == MLX5_ESWITCH_OFFLOADS.
Patch 7: The queued late representor reload now calls mlx5_esw_add_work()
with GFP_KERNEL instead of relying on the helper's previous hardcoded
GFP_ATOMIC allocation.
v1 -> v2:
Split v1 into two parts: the E-Switch workqueue deadlock fix and the
representor lifecycle changes. This is the second part; the first part
has already been accepted [1].
Patch 1: Add a cont_on_fail flag so callers can decide whether reload
should continue after a failure.
Patches 2, 3, 4: Replace the atomic-variable based scheme with a mutex,
per Jakub's feedback.
Patch 5: New patch that fixes the unwind on representor load failure.
Patch 7: Switch from profile 4 to profile 8. Since the profile mainly
targets E-Switch handling, keep it separate from the NIC profiles.
V2:
https://lore.kernel.org/all/20260501041633.231662-1-tariqt@nvidia.com/
V1:
https://lore.kernel.org/all/20260409115550.156419-1-tariqt@nvidia.com/
[1] https://lore.kernel.org/all/20260428051018.219093-1-tariqt@nvidia.com/
Mark Bloch (7):
net/mlx5: Lag: refactor representor reload handling
net/mlx5: E-Switch, let esw work callers choose GFP flags
net/mlx5: E-Switch, add representor lifecycle lock
net/mlx5: Lag, avoid LAG and representor lock cycles
net/mlx5: E-Switch, serialize representor lifecycle
net/mlx5: E-Switch, unwind only newly loaded representor types
net/mlx5: E-Switch, load reps via work queue after registration
drivers/infiniband/hw/mlx5/ib_rep.c | 6 +-
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 10 +
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 6 +
.../mellanox/mlx5/core/eswitch_offloads.c | 197 ++++++++++++++++--
.../net/ethernet/mellanox/mlx5/core/lag/lag.c | 171 +++++++++++----
.../net/ethernet/mellanox/mlx5/core/lag/lag.h | 5 +
.../ethernet/mellanox/mlx5/core/lag/mpesw.c | 18 +-
.../ethernet/mellanox/mlx5/core/lib/devcom.c | 8 +
.../ethernet/mellanox/mlx5/core/lib/devcom.h | 1 +
.../ethernet/mellanox/mlx5/core/sf/devlink.c | 5 +
include/linux/mlx5/eswitch.h | 6 +
11 files changed, 361 insertions(+), 72 deletions(-)
base-commit: 98878ed91b68a3150126fccef125ee7b1bb86ab2
--
2.44.0
^ permalink raw reply
* [PATCH net-next v2 5/5] net/sched: netem: handle multi-segment skb in corruption
From: Stephen Hemminger @ 2026-05-03 19:52 UTC (permalink / raw)
To: netdev; +Cc: jhs, jiri, Stephen Hemminger
In-Reply-To: <20260503195348.521225-1-stephen@networkplumber.org>
The packet corruption code only flipped bits in the linear
header portion of the skb, skipping corruption when
skb_headlen() was zero.
Use skb_header_pointer() and skb_store_bits() to access the
full packet data, allowing any bit in the packet to be
corrupted regardless of how the skb is laid out.
Replaces d64cb81dcbd5 ("net/sched: sch_netem: fix out-of-bounds access
in packet corruption") with a more general solution.
Only count the number of packets that were actually corrupted.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
net/sched/sch_netem.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index e710898ce96e..5cbd1a0dbfda 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -509,7 +509,6 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
* do it now in software before we mangle it.
*/
if (q->corrupt && q->corrupt >= get_crandom(&q->corrupt_cor, &q->prng)) {
- WRITE_ONCE(q->corrupted, q->corrupted + 1);
if (skb_is_gso(skb)) {
skb = netem_segment(skb, sch, to_free);
if (!skb)
@@ -532,9 +531,18 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
goto finish_segs;
}
- if (skb_headlen(skb))
- skb->data[get_random_u32_below(skb_headlen(skb))] ^=
- 1 << get_random_u32_below(8);
+ if (skb->len > 0) {
+ unsigned int offset = get_random_u32_below(skb->len);
+ u8 *ptr, val;
+
+ /* handle multi-segment skb's */
+ ptr = skb_header_pointer(skb, offset, 1, &val);
+ if (ptr) {
+ val = *ptr ^ (1 << get_random_u32_below(8));
+ skb_store_bits(skb, offset, &val, 1);
+ WRITE_ONCE(q->corrupted, q->corrupted + 1);
+ }
+ }
}
if (unlikely(sch->q.qlen >= sch->limit)) {
--
2.53.0
^ permalink raw reply related
* [PATCH net-next v2 4/5] net/sched: netem: add per-impairment extended statistics
From: Stephen Hemminger @ 2026-05-03 19:52 UTC (permalink / raw)
To: netdev; +Cc: jhs, jiri, Stephen Hemminger
In-Reply-To: <20260503195348.521225-1-stephen@networkplumber.org>
Adds new counters that keep track of when netem applied
impairments (delay, loss, corruption, duplication, reordering).
Add a struct tc_netem_xstats reported via TCA_STATS_APP so that
userspace (tc -s qdisc show) can display per-impairment counters.
Use the WRITE_ONCE/READ_ONCE pattern to allow for lockless
qdisc usage.
Accompanying iproute2 change is submitted separately.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
include/uapi/linux/pkt_sched.h | 9 ++++++
net/sched/sch_netem.c | 55 ++++++++++++++++++++++++++--------
2 files changed, 52 insertions(+), 12 deletions(-)
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index 66e8072f44df..1c84c8076e22 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -569,6 +569,15 @@ struct tc_netem_gemodel {
#define NETEM_DIST_SCALE 8192
#define NETEM_DIST_MAX 16384
+struct tc_netem_xstats {
+ __u64 delayed; /* packets delayed */
+ __u64 dropped; /* packets dropped by loss model */
+ __u64 corrupted; /* packets with bit errors injected */
+ __u64 duplicated; /* duplicate packets generated */
+ __u64 reordered; /* packets sent out of order */
+ __u64 ecn_marked; /* packets ECN CE-marked (not dropped)*/
+};
+
/* DRR */
enum {
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 53961d1e70d7..e710898ce96e 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -100,8 +100,7 @@ struct netem_sched_data {
s64 latency;
s64 jitter;
u64 rate;
- u32 gap;
- u32 loss;
+ u64 delayed;
/* Cacheline 1: zero-check scalars and correlation states. */
u32 duplicate;
@@ -112,7 +111,8 @@ struct netem_sched_data {
u32 last;
u32 rho;
} delay_cor, loss_cor, dup_cor, reorder_cor, corrupt_cor;
- u8 loss_model;
+ u32 gap;
+ u32 loss;
/* Cacheline 2: PRNG, distribution tables, slot dequeue state etc. */
struct prng {
@@ -125,21 +125,27 @@ struct netem_sched_data {
s32 packets_left;
s32 bytes_left;
} slot;
- struct disttable *slot_dist;
struct Qdisc *qdisc;
+ u8 loss_model;
/*
- * Warm: rate-shaping parameters (only read when rate != 0) and
- * configuration-only fields. The fast path reads sch->limit, not
- * q->limit.
+ * Rare-write impairment counters (read together by netem_dump) and
+ * rate-shaping parameters (only consulted when rate != 0). The
+ * fast path reads sch->limit, not q->limit.
*/
+ u64 dropped;
+ u64 corrupted;
+ u64 duplicated;
+ u64 ecn_marked;
+ u64 reordered;
s32 packet_overhead;
u32 cell_size;
struct reciprocal_value cell_size_reciprocal;
s32 cell_overhead;
u32 limit;
- /* Correlated Loss Generation models */
+ /* Cold tail: slot reschedule config and the watchdog timer. */
+ struct disttable *slot_dist;
struct clgstate {
/* 4-states and Gilbert-Elliot models */
u32 a1; /* p13 for 4-states or p for GE */
@@ -152,7 +158,6 @@ struct netem_sched_data {
u8 state;
} clg;
- /* Cold tail: slot reschedule config and the watchdog timer. */
struct tc_netem_slot slot_config;
struct qdisc_watchdog watchdog;
};
@@ -462,17 +467,23 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
skb->prev = NULL;
/* Random duplication */
- if (q->duplicate && q->duplicate >= get_crandom(&q->dup_cor, &q->prng))
+ if (q->duplicate && q->duplicate >= get_crandom(&q->dup_cor, &q->prng)) {
++count;
+ WRITE_ONCE(q->duplicated, q->duplicated + 1);
+ }
/* Drop packet? */
if (loss_event(q)) {
- if (q->ecn && INET_ECN_set_ce(skb))
+ if (q->ecn && INET_ECN_set_ce(skb)) {
qdisc_qstats_drop(sch); /* mark packet */
- else
+ WRITE_ONCE(q->ecn_marked, q->ecn_marked + 1);
+ } else {
--count;
+ }
}
+
if (count == 0) {
+ WRITE_ONCE(q->dropped, q->dropped + 1);
qdisc_qstats_drop(sch);
__qdisc_drop(skb, to_free);
return NET_XMIT_SUCCESS | __NET_XMIT_BYPASS;
@@ -498,6 +509,7 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
* do it now in software before we mangle it.
*/
if (q->corrupt && q->corrupt >= get_crandom(&q->corrupt_cor, &q->prng)) {
+ WRITE_ONCE(q->corrupted, q->corrupted + 1);
if (skb_is_gso(skb)) {
skb = netem_segment(skb, sch, to_free);
if (!skb)
@@ -603,12 +615,15 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
cb->time_to_send = now + delay;
++q->counter;
+ WRITE_ONCE(q->delayed, q->delayed + 1);
+
tfifo_enqueue(skb, sch);
} else {
/*
* Do re-ordering by putting one out of N packets at the front
* of the queue.
*/
+ WRITE_ONCE(q->reordered, q->reordered + 1);
cb->time_to_send = ktime_get_ns();
q->counter = 0;
@@ -1348,6 +1363,21 @@ static int netem_dump(struct Qdisc *sch, struct sk_buff *skb)
return -1;
}
+static int netem_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
+{
+ struct netem_sched_data *q = qdisc_priv(sch);
+ struct tc_netem_xstats st = {
+ .delayed = READ_ONCE(q->delayed),
+ .dropped = READ_ONCE(q->dropped),
+ .corrupted = READ_ONCE(q->corrupted),
+ .duplicated = READ_ONCE(q->duplicated),
+ .reordered = READ_ONCE(q->reordered),
+ .ecn_marked = READ_ONCE(q->ecn_marked),
+ };
+
+ return gnet_stats_copy_app(d, &st, sizeof(st));
+}
+
static int netem_dump_class(struct Qdisc *sch, unsigned long cl,
struct sk_buff *skb, struct tcmsg *tcm)
{
@@ -1410,6 +1440,7 @@ static struct Qdisc_ops netem_qdisc_ops __read_mostly = {
.destroy = netem_destroy,
.change = netem_change,
.dump = netem_dump,
+ .dump_stats = netem_dump_stats,
.owner = THIS_MODULE,
};
MODULE_ALIAS_NET_SCH("netem");
--
2.53.0
^ permalink raw reply related
* [PATCH net-next v2 3/5] net/sched: netem: replace pr_info with netlink extack error messages
From: Stephen Hemminger @ 2026-05-03 19:52 UTC (permalink / raw)
To: netdev; +Cc: jhs, jiri, Stephen Hemminger
In-Reply-To: <20260503195348.521225-1-stephen@networkplumber.org>
Use netlink extack to report errors instead of sending them
to the kernel log with pr_info(). The error message can them be seen
with tc commands; and avoids log spam.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
net/sched/sch_netem.c | 29 ++++++++++++++++-------------
1 file changed, 16 insertions(+), 13 deletions(-)
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 2b0b5c032e70..53961d1e70d7 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -921,7 +921,8 @@ static void get_rate(struct netem_sched_data *q, const struct nlattr *attr)
q->cell_size_reciprocal = (struct reciprocal_value) { 0 };
}
-static int get_loss_clg(struct netem_sched_data *q, const struct nlattr *attr)
+static int get_loss_clg(struct netem_sched_data *q, const struct nlattr *attr,
+ struct netlink_ext_ack *extack)
{
const struct nlattr *la;
int rem;
@@ -934,7 +935,8 @@ static int get_loss_clg(struct netem_sched_data *q, const struct nlattr *attr)
const struct tc_netem_gimodel *gi = nla_data(la);
if (nla_len(la) < sizeof(struct tc_netem_gimodel)) {
- pr_info("netem: incorrect gi model size\n");
+ NL_SET_ERR_MSG_ATTR(extack, la,
+ "netem: incorrect gi model size");
return -EINVAL;
}
@@ -953,7 +955,8 @@ static int get_loss_clg(struct netem_sched_data *q, const struct nlattr *attr)
const struct tc_netem_gemodel *ge = nla_data(la);
if (nla_len(la) < sizeof(struct tc_netem_gemodel)) {
- pr_info("netem: incorrect ge model size\n");
+ NL_SET_ERR_MSG_ATTR(extack, la,
+ "netem: incorrect ge model size");
return -EINVAL;
}
@@ -967,7 +970,8 @@ static int get_loss_clg(struct netem_sched_data *q, const struct nlattr *attr)
}
default:
- pr_info("netem: unknown loss type %u\n", type);
+ NL_SET_ERR_MSG_ATTR_FMT(extack, la,
+ "netem: unknown loss type %u", type);
return -EINVAL;
}
}
@@ -990,19 +994,21 @@ static const struct nla_policy netem_policy[TCA_NETEM_MAX + 1] = {
};
static int parse_attr(struct nlattr *tb[], int maxtype, struct nlattr *nla,
- const struct nla_policy *policy, int len)
+ const struct nla_policy *policy, int len,
+ struct netlink_ext_ack *extack)
{
int nested_len = nla_len(nla) - NLA_ALIGN(len);
if (nested_len < 0) {
- pr_info("netem: invalid attributes len %d\n", nested_len);
+ NL_SET_ERR_MSG_FMT(extack, "netem: invalid attributes len %u < %d",
+ nla_len(nla), NLA_ALIGN(len));
return -EINVAL;
}
if (nested_len >= nla_attr_size(0))
return nla_parse_deprecated(tb, maxtype,
nla_data(nla) + NLA_ALIGN(len),
- nested_len, policy, NULL);
+ nested_len, policy, extack);
memset(tb, 0, sizeof(struct nlattr *) * (maxtype + 1));
return 0;
@@ -1044,8 +1050,7 @@ static int check_netem_in_tree(struct Qdisc *sch, bool duplicates,
}
/* Parse netlink message to set options */
-static int netem_change(struct Qdisc *sch, struct nlattr *opt,
- struct netlink_ext_ack *extack)
+static int netem_change(struct Qdisc *sch, struct nlattr *opt, struct netlink_ext_ack *extack)
{
struct netem_sched_data *q = qdisc_priv(sch);
struct nlattr *tb[TCA_NETEM_MAX + 1];
@@ -1057,7 +1062,7 @@ static int netem_change(struct Qdisc *sch, struct nlattr *opt,
int ret;
qopt = nla_data(opt);
- ret = parse_attr(tb, TCA_NETEM_MAX, opt, netem_policy, sizeof(*qopt));
+ ret = parse_attr(tb, TCA_NETEM_MAX, opt, netem_policy, sizeof(*qopt), extack);
if (ret < 0)
return ret;
@@ -1097,7 +1102,7 @@ static int netem_change(struct Qdisc *sch, struct nlattr *opt,
old_loss_model = q->loss_model;
if (tb[TCA_NETEM_LOSS]) {
- ret = get_loss_clg(q, tb[TCA_NETEM_LOSS]);
+ ret = get_loss_clg(q, tb[TCA_NETEM_LOSS], extack);
if (ret) {
q->loss_model = old_loss_model;
q->clg = old_clg;
@@ -1193,8 +1198,6 @@ static int netem_init(struct Qdisc *sch, struct nlattr *opt,
prandom_seed_state(&q->prng.prng_state, q->prng.seed);
ret = netem_change(sch, opt, extack);
- if (ret)
- pr_info("netem: change failed\n");
return ret;
}
--
2.53.0
^ permalink raw reply related
* [PATCH net-next v2 2/5] net/sched: netem: remove useless VERSION
From: Stephen Hemminger @ 2026-05-03 19:52 UTC (permalink / raw)
To: netdev; +Cc: jhs, jiri, Stephen Hemminger
In-Reply-To: <20260503195348.521225-1-stephen@networkplumber.org>
The version printed was never updated and kernel version is
better indication of what is fixed or not.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
net/sched/sch_netem.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 616d33879fdc..2b0b5c032e70 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -27,8 +27,6 @@
#include <net/pkt_sched.h>
#include <net/inet_ecn.h>
-#define VERSION "1.3"
-
/* Network Emulation Queuing algorithm.
====================================
@@ -1413,16 +1411,15 @@ static struct Qdisc_ops netem_qdisc_ops __read_mostly = {
};
MODULE_ALIAS_NET_SCH("netem");
-
static int __init netem_module_init(void)
{
- pr_info("netem: version " VERSION "\n");
return register_qdisc(&netem_qdisc_ops);
}
static void __exit netem_module_exit(void)
{
unregister_qdisc(&netem_qdisc_ops);
}
+
module_init(netem_module_init)
module_exit(netem_module_exit)
MODULE_LICENSE("GPL");
--
2.53.0
^ permalink raw reply related
* [PATCH net-next v2 1/5] net/sched: netem: reorder struct netem_sched_data
From: Stephen Hemminger @ 2026-05-03 19:51 UTC (permalink / raw)
To: netdev; +Cc: jhs, jiri, Stephen Hemminger
In-Reply-To: <20260503195348.521225-1-stephen@networkplumber.org>
The current layout of struct netem_sched_data can be improved
by optimizing cache locality, compacting data types (use u8
for enum) and eliminating unused elements.
Reorganize the struct as follows:
- Cacheline 0 holds the tfifo state (t_root/t_head/t_tail/t_len),
counter, and the unconditional enqueue scalars
latency/jitter/rate/gap/loss.
- Cacheline 1 holds the remaining zero-check scalars
(duplicate/reorder/corrupt/ecn), all five crndstate correlation
structures, and loss_model.
- Cacheline 2 holds prng, delay_dist, the slot dequeue state,
slot_dist, and the inner classful qdisc pointer.
- Rate-shaping fields, q->limit (config-only; the fast path reads
sch->limit), and the CLG Markov state move to the warm tail.
- tc_netem_slot slot_config and qdisc_watchdog (only consulted on
slot reschedule and watchdog wake) move to the cold tail.
Also reorder struct clgstate to place the u8 state member after the
u32 transition probabilities. This removes the 3-byte interior hole
without changing the struct's size.
Should have no functional change.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
net/sched/sch_netem.c | 123 +++++++++++++++++++++---------------------
1 file changed, 63 insertions(+), 60 deletions(-)
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index bc18e1976b6e..616d33879fdc 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -71,89 +71,92 @@ struct disttable {
s16 table[] __counted_by(size);
};
-struct netem_sched_data {
- /* internal t(ime)fifo qdisc uses t_root and sch->limit */
- struct rb_root t_root;
-
- /* a linear queue; reduces rbtree rebalancing when jitter is low */
- struct sk_buff *t_head;
- struct sk_buff *t_tail;
-
- u32 t_len;
-
- /* optional qdisc for classful handling (NULL at netem init) */
- struct Qdisc *qdisc;
-
- struct qdisc_watchdog watchdog;
+/* Loss models */
+enum {
+ CLG_RANDOM,
+ CLG_4_STATES,
+ CLG_GILB_ELL,
+};
- s64 latency;
- s64 jitter;
+/* States in GE model */
+enum {
+ GOOD_STATE = 1,
+ BAD_STATE,
+};
- u32 loss;
- u32 ecn;
- u32 limit;
- u32 counter;
- u32 gap;
- u32 duplicate;
- u32 reorder;
- u32 corrupt;
- u64 rate;
- s32 packet_overhead;
- u32 cell_size;
- struct reciprocal_value cell_size_reciprocal;
- s32 cell_overhead;
+/* States in 4 state model */
+enum {
+ TX_IN_GAP_PERIOD = 1,
+ TX_IN_BURST_PERIOD,
+ LOST_IN_GAP_PERIOD,
+ LOST_IN_BURST_PERIOD,
+};
+struct netem_sched_data {
+ /* Cacheline 0: tfifo state and per-packet enqueue/dequeue scalars. */
+ struct rb_root t_root;
+ struct sk_buff *t_head;
+ struct sk_buff *t_tail;
+ u32 t_len;
+ u32 counter;
+ s64 latency;
+ s64 jitter;
+ u64 rate;
+ u32 gap;
+ u32 loss;
+
+ /* Cacheline 1: zero-check scalars and correlation states. */
+ u32 duplicate;
+ u32 reorder;
+ u32 corrupt;
+ u32 ecn;
struct crndstate {
u32 last;
u32 rho;
} delay_cor, loss_cor, dup_cor, reorder_cor, corrupt_cor;
+ u8 loss_model;
- struct prng {
+ /* Cacheline 2: PRNG, distribution tables, slot dequeue state etc. */
+ struct prng {
u64 seed;
struct rnd_state prng_state;
} prng;
+ struct disttable *delay_dist;
+ struct slotstate {
+ u64 slot_next;
+ s32 packets_left;
+ s32 bytes_left;
+ } slot;
+ struct disttable *slot_dist;
+ struct Qdisc *qdisc;
- struct disttable *delay_dist;
-
- enum {
- CLG_RANDOM,
- CLG_4_STATES,
- CLG_GILB_ELL,
- } loss_model;
-
- enum {
- TX_IN_GAP_PERIOD = 1,
- TX_IN_BURST_PERIOD,
- LOST_IN_GAP_PERIOD,
- LOST_IN_BURST_PERIOD,
- } _4_state_model;
-
- enum {
- GOOD_STATE = 1,
- BAD_STATE,
- } GE_state_model;
+ /*
+ * Warm: rate-shaping parameters (only read when rate != 0) and
+ * configuration-only fields. The fast path reads sch->limit, not
+ * q->limit.
+ */
+ s32 packet_overhead;
+ u32 cell_size;
+ struct reciprocal_value cell_size_reciprocal;
+ s32 cell_overhead;
+ u32 limit;
/* Correlated Loss Generation models */
struct clgstate {
- /* state of the Markov chain */
- u8 state;
-
/* 4-states and Gilbert-Elliot models */
u32 a1; /* p13 for 4-states or p for GE */
u32 a2; /* p31 for 4-states or r for GE */
u32 a3; /* p32 for 4-states or h for GE */
u32 a4; /* p14 for 4-states or 1-k for GE */
u32 a5; /* p23 used only in 4-states */
- } clg;
- struct tc_netem_slot slot_config;
- struct slotstate {
- u64 slot_next;
- s32 packets_left;
- s32 bytes_left;
- } slot;
+ /* state of the Markov chain */
+ u8 state;
+ } clg;
- struct disttable *slot_dist;
+ /* Cold tail: slot reschedule config and the watchdog timer. */
+ struct tc_netem_slot slot_config;
+ struct qdisc_watchdog watchdog;
};
/* Time stamp put into socket buffer control block
--
2.53.0
^ permalink raw reply related
* [PATCH net-next v2 0/5] net/sched: netem: fixes and improvements
From: Stephen Hemminger @ 2026-05-03 19:51 UTC (permalink / raw)
To: netdev; +Cc: jhs, jiri, Stephen Hemminger
This is a collection of improvements to netem found while
investigating the fixes now in net tree.
v2 - incorporate AI suggestions
- rearrange netem_sched_data
- add multi-segment corruption improvement
Stephen Hemminger (5):
net/sched: netem: reorder struct netem_sched_data
net/sched: netem: remove useless VERSION
net/sched: netem: replace pr_info with netlink extack error messages
net/sched: netem: add per-impairment extended statistics
net/sched: netem: handle multi-segment skb in corruption
include/uapi/linux/pkt_sched.h | 9 ++
net/sched/sch_netem.c | 212 ++++++++++++++++++++-------------
2 files changed, 136 insertions(+), 85 deletions(-)
--
2.53.0
^ permalink raw reply
* [RFC PATCH] Possible use-after-free bug in mTLS connect
From: Michael Nemanov @ 2026-05-03 19:53 UTC (permalink / raw)
To: linux-nfs, netdev, linux-kernel
Cc: Chuck Lever, Jeff Layton, NeilBrown, Olga Kornievskaia, Dai Ngo,
Tom Talpey, Trond Myklebust, Anna Schumaker, David S. Miller,
Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
Dan Aloni, roi.azarzar, sagi.grimberg
The NFS-over-TLS implementation seems to have a use-after-free bug where
a raw unrefcounted pointer to an rpc_clnt is stored in xs_connect() and
accessed by a delayed workqueue item after the client has been freed.
The issue manifests when an NFS mount uses incorrect credentials (client
cert is valid but does not match the server's) during TLS setup,
leading to the client being freed while a delayed work item still
holds a pointer to it.
The patch contains several debug traces that hopefully illustrate the problem
and a key msleep(100) that helps (though not guarantees) reproduction.
I had to use pr_debug as the bug often involves a kernel crash and logs can
only be collected from vmcore-dmesg.
Used with kernel v7.0-rc5.
Traces from vmcore-dmesg.txt showing the lifecycle of the RPC
client from creation to use-after-free:
[ 38.611952] New mount #5
[ 38.630156] @ rpc_new_client: New: 00000000f636d223, parent: 0000000000000000 << Client created
[ 38.630179] @ xs_connect: Queue connect work in 0 for clnt 00000000f636d223
[ 38.630209] @ xs_tcp_tls_setup_socket: Using clnt 00000000f636d223
[ 38.630258] @ rpc_new_client: New: 0000000004f8c0fe, parent: 0000000000000000
[ 38.630265] @ xs_connect: Queue connect work in 0 for clnt 0000000004f8c0fe
[ 38.752404] @ rpc_shutdown_client: 0000000004f8c0fe
[ 38.752474] @ rpc_free_client: put_cred 0000000004f8c0fe
[ 38.752489] @ xs_tcp_tls_setup_socket: Done with clnt 00000000f636d223. status=0
[ 38.752573] @ rpc_free_client_work: xprt_put done for 0000000004f8c0fe
[ 38.752955] @ xs_sock_process_cmsg: TLS alert -13, level 2
[ 38.753069] @ xs_tcp_state_change: sk_state=8
[ 38.857474] @ xs_reset_transport: Null transport->sock for xprt 0000000066fa6fda
[ 38.857558] @ xs_connect: Queue connect work in 0 for clnt 00000000f636d223 << Client used
[ 38.857574] @ xs_tcp_tls_setup_socket: Using clnt 00000000f636d223
[ 38.857677] @ rpc_new_client: New: 00000000d1455d7f, parent: 0000000000000000
[ 38.857684] @ xs_connect: Queue connect work in 0 for clnt 00000000d1455d7f
[ 38.975657] @ rpc_shutdown_client: 00000000d1455d7f
[ 38.975728] @ rpc_free_client: put_cred 00000000d1455d7f
[ 38.975742] @ xs_tcp_tls_setup_socket: Done with clnt 00000000f636d223. status=0 << Client used
[ 38.975836] @ rpc_free_client_work: xprt_put done for 00000000d1455d7f
[ 38.976220] @ xs_sock_process_cmsg: TLS alert -13, level 2
[ 38.976269] @ xs_tcp_state_change: sk_state=8
[ 38.976303] @ xs_connect: Queue connect work in 3000 for clnt 00000000f636d223 << Client used
[ 39.065470] @ rpc_shutdown_client: 00000000f636d223
[ 39.065580] @ rpc_free_client: put_cred 00000000f636d223 << Client being destroyed
[ 42.033267] @ xs_tcp_tls_setup_socket: Using clnt 00000000f636d223 << Client used
[ 42.033481] @ rpc_new_client: New: 00000000762dc139, parent: 0000000000000000
[ 42.033505] @ xs_connect: Queue connect work in 0 for clnt 00000000762dc139
[ 42.153240] @ rpc_shutdown_client: 00000000762dc139
[ 42.153274] @ rpc_free_client: put_cred 00000000762dc139
[ 42.153283] @ xs_tcp_tls_setup_socket: Done with clnt 00000000f636d223. status=0
[ 42.153297] @ xs_reset_transport: Null transport->sock for xprt 0000000066fa6fda
[ 42.153355] @ rpc_free_client_work: xprt_put done for 00000000f636d223
[ 42.153373] @ rpc_free_client_work: xprt_put done for 00000000762dc139
[ 42.153419] @ xs_reset_transport: Null transport->sock for xprt 0000000093fd6749
[ 42.164197] ------------[ cut here ]------------
[ 42.165779] refcount_t: underflow; use-after-free.
[ 42.166596] WARNING: lib/refcount.c:28 at refcount_warn_saturate+0x5e/0x90, CPU#5: swapper/5/0
And for reference, a non-crashing attempt:
[ 16.756822] New mount #1
[ 17.122657] NFS: Registering the id_resolver key type
[ 17.123591] Key type id_resolver registered
[ 17.124329] Key type id_legacy registered
[ 17.131019] @ rpc_new_client: New: 0000000074283ee9, parent: 0000000000000000
[ 17.131035] @ xs_connect: Queue connect work in 0 for clnt 0000000074283ee9
[ 17.131042] @ xs_tcp_tls_setup_socket: Using clnt 0000000074283ee9
[ 17.131108] @ rpc_new_client: New: 00000000b25a12a5, parent: 0000000000000000
[ 17.131114] @ xs_connect: Queue connect work in 0 for clnt 00000000b25a12a5
[ 17.254743] @ xs_``tcp_state_change: sk_state=8
[ 17.262363] @ rpc_shutdown_client: 00000000b25a12a5
[ 17.262398] @ rpc_free_client: put_cred 00000000b25a12a5
[ 17.262404] @ xs_tcp_tls_setup_socket: Done with clnt 0000000074283ee9. status=0
[ 17.262448] @ rpc_free_client_work: xprt_put done for 00000000b25a12a5
[ 17.473543] @ xs_reset_transport: Null transport->sock for xprt 000000004c2083b6
[ 17.473597] @ xs_connect: Queue connect work in 0 for clnt 0000000074283ee9
[ 17.473606] @ xs_tcp_tls_setup_socket: Using clnt 0000000074283ee9
[ 17.473709] @ rpc_new_client: New: 00000000c549945c, parent: 0000000000000000
[ 17.473717] @ xs_connect: Queue connect work in 0 for clnt 00000000c549945c
[ 17.591937] @ rpc_shutdown_client: 00000000c549945c
[ 17.591977] @ rpc_free_client: put_cred 00000000c549945c
[ 17.591984] @ xs_tcp_tls_setup_socket: Done with clnt 0000000074283ee9. status=0
[ 17.592030] @ rpc_free_client_work: xprt_put done for 00000000c549945c
[ 17.592534] @ xs_sock_process_cmsg: TLS alert -13, level 2
[ 17.592564] @ xs_tcp_state_change: sk_state=8
[ 17.592569] @ rpc_shutdown_client: 0000000074283ee9
[ 17.592608] @ rpc_free_client: put_cred 0000000074283ee9
[ 17.593105] @ rpc_free_client_work: xprt_put done for 0000000074283ee9
[ 17.593148] @ xs_reset_transport: Null transport->sock for xprt 000000004c2083b6
Note that on some iterations, the queued work runs successfully
despite using a freed client. Not sure if this is of interest.
Reproduction script:
```bash
sudo echo 'file net/sunrpc/clnt.c +p' > /sys/kernel/debug/dynamic_debug/control
sudo echo 'file net/sunrpc/xprtsock.c +p' >> /sys/kernel/debug/dynamic_debug/control
sudo mkdir -p /mnt/export
for i in {1..1000}; do
echo "Iteration $i"
echo "New mount #$i" | sudo tee /dev/kmsg
sudo mount -o vers=4.2,proto=tcp,xprtsec=mtls remote_addr:/opt/export /mnt/export
sleep 5
done
```
What I understand:
The connecting task terminates, presumably due to the fatal TLS error as
expected. The upper client is being destroyed: `rpc_shutdown_client`
down to `rpc_free_client`, which *puts the creds* that would be used
by queued connect work.
Where I need help:
I don't understand if the use-after-free is the root cause and must be
fixed or a symptom of a problem elsewhere.
I am also not sure what causes the bug to manifest. It *might* be related
to the timing of server's FIN (sk_state=8 in the log) and the
flags it changes.
Any guidance or pointers on where to look next would be much appreciated.
Thank you for reading,
Michael.
Signed-off-by: Michael Nemanov <michael.nemanov@vastdata.com>
---
net/sunrpc/clnt.c | 4 ++++
net/sunrpc/xprtsock.c | 8 ++++++++
2 files changed, 12 insertions(+)
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index bc8ca470718b..dbc7d51f073d 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -429,6 +429,7 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args,
refcount_inc(&parent->cl_count);
trace_rpc_clnt_new(clnt, xprt, args);
+ pr_debug("@ %s: New: %p, parent: %p\n", __func__, clnt, parent);
return clnt;
out_no_path:
@@ -949,6 +950,7 @@ void rpc_shutdown_client(struct rpc_clnt *clnt)
might_sleep();
trace_rpc_clnt_shutdown(clnt);
+ pr_debug("@ %s: %p\n", __func__, clnt);
clnt->cl_shutdown = 1;
while (!list_empty(&clnt->cl_tasks)) {
@@ -983,6 +985,7 @@ static void rpc_free_client_work(struct work_struct *work)
rpc_free_clid(clnt);
rpc_clnt_remove_pipedir(clnt);
xprt_put(rcu_dereference_raw(clnt->cl_xprt));
+ pr_debug("@ %s: xprt_put done for %p\n", __func__, clnt);
kfree(clnt);
rpciod_down();
@@ -1000,6 +1003,7 @@ rpc_free_client(struct rpc_clnt *clnt)
clnt->cl_metrics = NULL;
xprt_iter_destroy(&clnt->cl_xpi);
put_cred(clnt->cl_cred);
+ pr_debug("@ %s: put_cred %p\n", __func__, clnt);
INIT_WORK(&clnt->cl_work, rpc_free_client_work);
schedule_work(&clnt->cl_work);
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 2e1fe6013361..cc4275e0b276 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -377,6 +377,7 @@ xs_sock_process_cmsg(struct socket *sock, struct msghdr *msg,
tls_alert_recv(sock->sk, msg, &level, &description);
ret = (level == TLS_ALERT_LEVEL_FATAL) ?
-EACCES : -EAGAIN;
+ pr_debug("@ %s: TLS alert %d, level %d\n", __func__, ret, level);
break;
default:
/* discard this record type */
@@ -1297,6 +1298,7 @@ static void xs_reset_transport(struct sock_xprt *transport)
transport->inet = NULL;
transport->sock = NULL;
transport->file = NULL;
+ pr_debug("@ %s: Null transport->sock for xprt %p\n", __func__, xprt);
sk->sk_user_data = NULL;
sk->sk_sndtimeo = 0;
@@ -1589,6 +1591,7 @@ static void xs_tcp_state_change(struct sock *sk)
*/
if (xprt->reestablish_timeout < XS_TCP_INIT_REEST_TO)
xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
+ pr_debug("@ %s: sk_state=%d\n", __func__, sk->sk_state);
break;
case TCP_LAST_ACK:
set_bit(XPRT_CLOSING, &xprt->state);
@@ -2688,6 +2691,7 @@ static void xs_tcp_tls_setup_socket(struct work_struct *work)
struct sock_xprt *upper_transport =
container_of(work, struct sock_xprt, connect_worker.work);
struct rpc_clnt *upper_clnt = upper_transport->clnt;
+ pr_debug("@ %s: Using clnt %p\n", __func__, upper_clnt);
struct rpc_xprt *upper_xprt = &upper_transport->xprt;
struct rpc_create_args args = {
.net = upper_xprt->xprt_net,
@@ -2759,6 +2763,7 @@ static void xs_tcp_tls_setup_socket(struct work_struct *work)
current_restore_flags(pflags, PF_MEMALLOC);
upper_transport->clnt = NULL;
xprt_unlock_connect(upper_xprt, upper_transport);
+ pr_debug("@ %s: Done with clnt %p. status=%d\n", __func__, upper_clnt, status);
return;
out_close:
@@ -2806,6 +2811,7 @@ static void xs_connect(struct rpc_xprt *xprt, struct rpc_task *task)
dprintk("RPC: xs_connect scheduled xprt %p\n", xprt);
transport->clnt = task->tk_client;
+ pr_debug("@ %s: Queue connect work in %lu for clnt %p\n", __func__, delay, transport->clnt);
queue_delayed_work(xprtiod_workqueue,
&transport->connect_worker,
delay);
@@ -2847,6 +2853,8 @@ static void xs_error_handle(struct work_struct *work)
struct sock_xprt *transport = container_of(work,
struct sock_xprt, error_worker);
+ msleep(100); // Improves reproducibility
+
xs_wake_disconnect(transport);
xs_wake_write(transport);
xs_wake_error(transport);
base-commit: c369299895a591d96745d6492d4888259b004a9e
--
2.43.7
^ permalink raw reply related
* Re: [PATCH net-next v9 0/5] TLS read_sock performance scalability
From: Chuck Lever @ 2026-05-03 19:34 UTC (permalink / raw)
To: Jakub Kicinski
Cc: John Fastabend, Sabrina Dubroca, Eric Dumazet, Simon Horman,
Paolo Abeni, netdev, kernel-tls-handshake, Chuck Lever,
Hannes Reinecke, Alistair Francis
In-Reply-To: <20260502180415.0b0bf12b@kernel.org>
On 5/3/26 3:04 AM, Jakub Kicinski wrote:
> On Wed, 29 Apr 2026 17:48:07 -0400 Chuck Lever wrote:
>> I'd like to encourage in-kernel kTLS consumers (i.e., NFS and
>> NVMe/TCP) to coalesce on the use of read_sock. When I suggested
>> this to Hannes, he reported a few performance scalability issues
>> with read_sock.
>
> Meaning, this series achieves.. what right now?
> I mean - the headline is "performance scalability" and there's no
> performance testing result in any of the messages :S
> Patch 5 for instance "seems logical" but how much difference does
> it make?
The cover Subject: line has not been changed so all the revisions of
this series can be located easily.
The cover letter makes it clear that the series is now only a clean-up
series. Since async_capable is set to false for TLSv1.3, there is no
performance benefit to these changes, so I don't intend to post a
motivation for it based on performance.
>> However, batch async decryption and its
>> submit/deliver scaffolding were dropped from this series because
>> async_capable is always false for TLS 1.3, the TLS version that
>> NFS and NVMe/TCP both require. Async crypto support for TLS 1.3
>> is a prerequisite for revisiting that work.
>>
>> This series is now only a set of clean-ups. Support for async
>> has been deferred until after TLS KeyUpdate has been merged.
>
> What does "after TLS KeyUpdate has been merged" mean?
> KeyUpdate is supported.. You mean in NFS? Or in async?
We want to support TLS KeyUpdate in the in-kernel TLS consumers, which
include NFSD, the NFS client, the NVMe/TCP host, and the NVMe/TCP
target. There are two pre-requisites:
1. The in-kernel TLS consumers need to reliably and securely handle TLS
Alerts. That is coming in the next series I plan to post.
2. The TLS handshake upcall needs to handle KeyUpdate operations. That
is the series Alistair has been posting since forever, and is waiting
on getting this series and support for TLS Alerts merged into the
four in-kernel TLS consumers listed above.
> FTR async support is a major pain and we'd rather get rid of it
> (and switch away from cryto API) than extend it.
That would have been nice to know three months ago when I started work
on this series.
Is there nothing left to do here but drop this series? We'd really like
to get TLS KeyUpdate working for in-kernel TLS consumers, so anything
that can move this process forward is welcome.
--
Chuck Lever
^ permalink raw reply
* [PATCH 1/5] perf trace: Sync linux/socket.h with the kernel source
From: Namhyung Kim @ 2026-05-03 19:31 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Ian Rogers, James Clark, Linus Torvalds
Cc: Jiri Olsa, Adrian Hunter, Peter Zijlstra, Ingo Molnar, LKML,
linux-perf-users, Ravi Bangoria, netdev
In-Reply-To: <20260503193137.27465-1-namhyung@kernel.org>
To pick up changes from:
c66e0f453d1afa82 ("net: use ktime_t in struct scm_timestamping_internal")
This would be used to beautify networking syscall arguments and not to
affect builds of other tools (e.g. objtool).
Please see tools/include/uapi/README.
Cc: netdev@vger.kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/trace/beauty/include/linux/socket.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/perf/trace/beauty/include/linux/socket.h b/tools/perf/trace/beauty/include/linux/socket.h
index ec715ad4bf25f5f7..ec4a0a0257939a53 100644
--- a/tools/perf/trace/beauty/include/linux/socket.h
+++ b/tools/perf/trace/beauty/include/linux/socket.h
@@ -415,7 +415,7 @@ struct __kernel_timespec;
struct old_timespec32;
struct scm_timestamping_internal {
- struct timespec64 ts[3];
+ ktime_t ts[3];
};
extern void put_cmsg_scm_timestamping64(struct msghdr *msg, struct scm_timestamping_internal *tss);
--
2.53.0
^ permalink raw reply related
* Re: [PATCH v2] tty: synclink_gt: remove broken driver
From: Ethan Nelson-Moore @ 2026-05-03 19:25 UTC (permalink / raw)
To: Andrew Lunn
Cc: Greg Kroah-Hartman, linux-doc, netdev, linux-serial,
rust-for-linux, Jonathan Corbet, Shuah Khan, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy (CS GROUP),
Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Jiri Slaby, Miguel Ojeda, Boqun Feng, Gary Guo,
Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl,
Trevor Gross, Danilo Krummrich, Bagas Sanjaya, Haren Myneni,
Eric Biggers, Qingfang Deng, Julian Braha
In-Reply-To: <e12da6e2-5e50-4819-a5a8-2bc675da4c14@lunn.ch>
Hi, Andrew,
On Sun, May 3, 2026 at 7:23 AM Andrew Lunn <andrew@lunn.ch> wrote:
> Sounds like a whack a mole problem. I assume the recent removal of ATM
> broke it as well? Maybe __has_include() could be used?
That's a great idea. I will investigate that.
Ethan
^ permalink raw reply
* [PATCH v2] ipv6: default IPV6_SIT to m
From: Alyssa Ross @ 2026-05-03 19:25 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman
Cc: netdev, linux-kernel, Fernando Fernandez Mancera,
Ricardo B . Marlière, Krzysztof Kozlowski
This basically defaulted to m until recently, since IPV6 defaulted to
m. Since IPV6 was changed to a boolean with a default of y, IPV6_SIT
started defaulting to built-in as well. This results in a surprise
sit0 device by default for defconfig (and defconfig-derived config)
users at boot. For me, this broke an (admittedly non-robust) script.
Preserve the behaviour of most configs by avoiding building this
module, that's probably overall seldom used compared to IPv6 as a
whole, into the kernel.
Fixes: 309b905deee59 ("ipv6: convert CONFIG_IPV6 to built-in only and clean up Kconfigs")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: update help text as well
net/ipv6/Kconfig | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/ipv6/Kconfig b/net/ipv6/Kconfig
index c024aa77f25ba..c3806c6ac96f9 100644
--- a/net/ipv6/Kconfig
+++ b/net/ipv6/Kconfig
@@ -164,7 +164,7 @@ config IPV6_SIT
select INET_TUNNEL
select NET_IP_TUNNEL
select IPV6_NDISC_NODETYPE
- default y
+ default m
help
Tunneling means encapsulating data of one protocol type within
another protocol and sending it over a channel that understands the
@@ -172,7 +172,7 @@ config IPV6_SIT
into IPv4 packets. This is useful if you want to connect two IPv6
networks over an IPv4-only path.
- Saying M here will produce a module called sit. If unsure, say Y.
+ Saying M here will produce a module called sit. If unsure, say M.
config IPV6_SIT_6RD
bool "IPv6: IPv6 Rapid Deployment (6RD)"
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox