From: Ralf Lici <ralf@mandelbit.com>
To: netdev-bot+sashiko@kernel.org
Cc: netdev@vger.kernel.org, dsahern@kernel.org, idosch@nvidia.com,
davem@davemloft.net, edumazet@kernel.org, kuba@kernel.org,
pabeni@redhat.com, horms@kernel.org, ncardwell@google.com,
kuniyu@google.com, steffen.klassert@secunet.com,
herbert@gondor.apana.org.au, fw@strlen.de, sd@queasysnail.net
Subject: Re: [PATCH net-next v2 2/3] ipv6: add ip6_dst_lookup_flow_cookie
Date: Mon, 16 Nov 2026 09:58:05 +0100 [thread overview]
Message-ID: <20261116085806.88363-1-ralf@mandelbit.com> (raw)
In-Reply-To: <179473299212.2199510.11039200949372183485@kernel.org>
On Sun, 15 Nov 2026 08:56:32 +0000, netdev-bot+sashiko@kernel.org wrote:
> > diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
> > index 25fe0ba98b1a6..1b273714f7cc3 100644
> > --- a/net/ipv6/ip6_output.c
> > +++ b/net/ipv6/ip6_output.c
>
> [ ... ]
>
> > @@ -1300,6 +1299,54 @@ struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struct sock *sk, st
> > }
> > EXPORT_SYMBOL_GPL(ip6_dst_lookup_flow);
> >
> > +/**
> > + * ip6_dst_lookup_flow_cookie - perform route lookup on flow with ipsec
> > + * and return the matching dst cookie
>
> [ ... ]
>
> > + * Same as ip6_dst_lookup_flow, but also returns the cookie describing
> > + * the routing state the dst was obtained from. A caller caching the dst
> > + * should use this cookie when validating it with dst_check. If the
> > + * IPv6 FIB changed during the lookup, a route dst no longer validates
> > + * with it. An xfrm bundle is validated by xfrm and ignores the cookie.
>
> [Severity: High]
> The commit message describes the stale cookie as a problem for "Callers
> that keep a dst in a persistent cache", and names an xfrm policy
> insertion as one trigger. This patch converts no callers.
>
> At the end of the series, only tcp_v6_connect(), ip6_datagram_dst_update()
> and ip6_sk_dst_lookup_flow() are converted, by "ipv6: use the lookup
> cookie when caching the dst in sockets".
>
> Are the remaining persistent caches still exposed to the same race? The
> TCP re-route path, for example, still does this:
>
> net/ipv6/inet6_connection_sock.c:inet6_csk_route_socket() {
> ...
> dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);
>
> if (!IS_ERR(dst))
> ip6_dst_store(sk, dst, false, false);
> ...
> }
>
> ip6_dst_store() reads rt6_get_cookie() only after the lookup has
> returned. This path runs from inet6_csk_xmit() whenever __sk_dst_check()
> fails, and that happens right after a rt_genid_bump_ipv6().
>
> tcp_v6_syn_recv_sock() has the same pattern for accepted sockets. Its
> window runs from inet6_csk_route_req() to ip6_dst_store().
> inet6_sk_rebuild_header() has the same pattern too.
>
> dst_cache_set_ip6() computes the cookie itself:
>
> net/core/dst_cache.c:dst_cache_set_ip6() {
> ...
> dst_cache_per_cpu_dst_set(idst, dst,
> rt6_get_cookie(dst_rt6_info(dst)));
> ...
> }
>
> It has no parameter for a cookie from the caller. Without a new dst_cache
> API, these users can't adopt the new helper:
>
> - udp_tunnel6_dst_lookup() (vxlan, geneve, bareudp)
> - ip6_tunnel
> - wireguard
> - ovpn
> - the seg6, rpl, ioam6 and ila lwtunnels
> - tipc
>
> Suppose this sequence lands between ip6_dst_lookup_flow() and
> rt6_get_cookie() in any of these paths:
>
> xfrm_policy_insert()->rt_genid_bump_ipv6()->__fib6_flush_trees()
>
> The plain route would then be cached with the post-insertion serial and
> keep passing rt6_check().
>
> Wouldn't that socket or tunnel keep sending without the new IPsec policy
> until the next FIB change? That is the plaintext leak this series sets
> out to close.
>
This is the deliberate scope of the series, as described in the cover
letter. The remaining TCP paths and other persistent caches are left for
follow-up conversions.
The goal here is to introduce the lookup-cookie API with a small set of
users and get feedback on the approach before extending the conversion
across the networking core and drivers. Is there a reason that this
incremental introduction would be unsuitable?
> > + *
> > + * Return: a valid dst pointer on success or a pointer encoded error code.
> > + */
> > +struct dst_entry *ip6_dst_lookup_flow_cookie(struct net *net,
> > + const struct sock *sk,
> > + struct flowi6 *fl6,
> > + const struct in6_addr *final_dst,
> > + u32 *cookie)
> > +{
> > + const struct rt6_info *rt;
> > + struct dst_entry *dst;
> > + int gen0, gen1;
> > +
> > + gen0 = rt_genid_ipv6(net);
> > + /* read the generation before looking at the FIB or xfrm policies */
> > + smp_rmb();
> > +
> > + dst = ip6_dst_lookup_flow(net, sk, fl6, final_dst);
> > + if (IS_ERR(dst))
> > + return dst;
> > +
> > + rt = dst_rt6_info(dst);
> > + *cookie = rt->sernum ? gen0 : rt6_get_cookie(rt);
> > +
> > + /* keep the FIB and cookie reads before the final generation read */
> > + smp_rmb();
> > + gen1 = rt_genid_ipv6(net);
> > + if (unlikely(gen0 != gen1))
> > + *cookie = 0;
>
> [Severity: Medium]
> Can the gen0/gen1 check catch writers that bump the serial before they
> finish updating the routing state?
>
> fib6_ifup() clears the nh flags, bumps the serial, and only then
> rebalances:
>
> net/ipv6/route.c:fib6_ifup() {
> ...
> rt->fib6_nh->fib_nh_flags &= ~arg->nh_flags;
> fib6_update_sernum_upto_root(net, rt);
> rt6_multipath_rebalance(rt);
> ...
> }
>
> fib6_ifdown() uses the same order for NETDEV_DOWN on multipath routes,
> and also for NETDEV_CHANGE:
>
> net/ipv6/route.c:fib6_ifdown() {
> ...
> rt6_multipath_nh_flags_set(rt, dev, RTNH_F_DEAD |
> RTNH_F_LINKDOWN);
> fib6_update_sernum(net, rt);
> rt6_multipath_rebalance(rt);
> }
> return -2;
> ...
> }
>
> The -2 return makes the walker skip the other siblings, so no second
> bump follows the rebalance.
>
> Suppose gen0 reads the new serial N+1 and fib6_select_path() runs before
> rt6_multipath_rebalance(). The lookup then sees the old
> fib_nh_upper_bound values. rt6_get_cookie() returns fn_sernum N+1 and
> gen1 is also N+1, so the returned cookie is valid. rt6_check() keeps
> accepting the dst after that.
>
> rt6_score_route() doesn't check RTNH_F_DEAD or RTNH_F_LINKDOWN. Could
> the cached dst then stay pinned to a nexthop that was just marked dead
> or link-down, or to a stale weight split, until some unrelated FIB
> change?
>
> The writer ordering predates this patch. Still, the kdoc above says "If
> the IPv6 FIB changed during the lookup, a route dst no longer validates
> with it", and that doesn't seem to hold for these writers.
>
Right, this appears to expose a pre-existing writer-side ordering issue.
Moving the serial update after rt6_multipath_rebalance, while preserving
the existing node/ancestor invalidation scope, looks like the
appropriate fix: the new serial would then be published after the
routing state it describes.
That needs validation, but it can be addressed as a separate writer-side
fix without changing the proposed lookup-cookie API.
--
Ralf Lici
Mandelbit Srl
next prev parent reply other threads:[~2026-11-16 8:58 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-11 8:22 [PATCH net-next v2 0/3] ipv6: tie the dst cookie to the lookup that returned the dst Ralf Lici
2026-11-11 8:22 ` [PATCH net-next v2 1/3] ipv6: stamp the default route node with a fresh serial on link failure Ralf Lici
2026-11-15 8:56 ` netdev-bot+sashiko
2026-11-16 8:49 ` Ralf Lici
2026-11-11 8:22 ` [PATCH net-next v2 2/3] ipv6: add ip6_dst_lookup_flow_cookie Ralf Lici
2026-11-15 8:56 ` netdev-bot+sashiko
2026-11-16 8:58 ` Ralf Lici [this message]
2026-11-11 8:22 ` [PATCH net-next v2 3/3] ipv6: use the lookup cookie when caching the dst in sockets Ralf Lici
2026-11-15 8:56 ` netdev-bot+sashiko
2026-11-16 9:02 ` Ralf Lici
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116085806.88363-1-ralf@mandelbit.com \
--to=ralf@mandelbit.com \
--cc=davem@davemloft.net \
--cc=dsahern@kernel.org \
--cc=edumazet@kernel.org \
--cc=fw@strlen.de \
--cc=herbert@gondor.apana.org.au \
--cc=horms@kernel.org \
--cc=idosch@nvidia.com \
--cc=kuba@kernel.org \
--cc=kuniyu@google.com \
--cc=ncardwell@google.com \
--cc=netdev-bot+sashiko@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=sd@queasysnail.net \
--cc=steffen.klassert@secunet.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox