[PATCH net 0/4] Mitigate a side channel in routing exception caches

[PATCH net 0/4] Mitigate a side channel in routing exception caches

[Ido Schimmel]

When an ICMP error that quotes a UDP packet is locally delivered, the
kernel only creates a routing exception if the quoted packet matches a
socket. This allows an off-path attacker to conduct a side-channel
attack on the routing exception caches in order to discover the
ephemeral ports used by connected UDP sockets.

Previous mitigations tried to make it harder for attackers to find hash
collisions in these caches and make the eviction of exceptions less
predictable. Amit Klein and Noam Caspi demonstrated that both of these
mitigations can be bypassed.

This patchset tries to mitigate such attacks by always creating an
exception, even if UDP socket matching failed. The exception is created
by the same helpers that are used when the quoted packet did not
originate from a socket, so that neither the occupancy of the cache nor
the contents of the exception are indicative of the success or failure
of UDP socket matching.

Note that this mitigation does not make it easier for attackers to fill
these caches, since they can already create exceptions with little to no
validation. For example, by sending an ICMP error that quotes an ICMP
Echo Reply or one that quotes a UDP source port that matches a wildcard
socket.

Patch #1 fixes a pre-existing bug in the handling of ICMPv6 Redirect
Message packets. Discovered while writing the selftest.

Patch #2 creates an exception from the IPv4 UDP code when socket
matching failed. Other socket types do not need this: raw sockets have
no ports, and for TCP the ICMP error is discarded unless the quoted
sequence number is in window.

Patch #3 does the same for IPv6.

Patch #4 adds a selftest.

Ido Schimmel (4):
  ipv6: Fix redirect exception creation for UDP/RAW sockets
  ipv4: udp: Create exceptions when socket matching failed
  ipv6: udp: Create exceptions when socket matching failed
  selftests: net: Add exception cache tests

 net/ipv4/route.c                              |   8 +-
 net/ipv4/udp.c                                |  10 +
 net/ipv6/route.c                              |   2 +-
 net/ipv6/udp.c                                |  17 +-
 tools/testing/selftests/net/Makefile          |   1 +
 .../testing/selftests/net/exception_cache.sh  | 501 ++++++++++++++++++
 6 files changed, 534 insertions(+), 5 deletions(-)
 create mode 100755 tools/testing/selftests/net/exception_cache.sh

-- 
2.55.0

[PATCH net 1/4] ipv6: Fix redirect exception creation for UDP/RAW sockets

[Ido Schimmel]

When an ICMP Redirect Message is matched to a socket, both IPv4 and IPv6
verify that the source IP of the ICMP packet is the current gateway for
the quoted packet. Both also pass the socket's bound device as the
expected nexthop device.

The difference is that IPv4 treats "oif=0" as "any", whereas IPv6 always
requires an exact match (see ip6_redirect_nh_match()), since the gateway
address is usually a link-local address.

Therefore, when an IPv6 UDP/RAW socket is not bound to a device, the
above verification fails and an exception is not created. This also
happens when the socket is bound to a VRF, as l3mdev_update_flow()
resets the oif to 0.

Fix this by passing the ifindex of the ingress device as the expected
nexthop device. This is consistent with the existing callers of
ip6_redirect(). Note that for ICMPv6 Redirect Message packets the VRF
driver does not reset skb->dev to the VRF device, so skb->dev is always
correct, even when it is a VRF port.

Fixes: b55b76b22144 ("ipv6:introduce function to find route for redirect")
Cc: stable@vger.kernel.org
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: David Ahern <dsahern@kernel.org>
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
---
 net/ipv6/route.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 6a40c5074543..9658939511e0 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -3255,7 +3255,7 @@ void ip6_redirect_no_header(struct sk_buff *skb, struct net *net, int oif)
 
 void ip6_sk_redirect(struct sk_buff *skb, struct sock *sk)
 {
-	ip6_redirect(skb, sock_net(sk), sk->sk_bound_dev_if,
+	ip6_redirect(skb, sock_net(sk), skb->dev->ifindex,
 		     READ_ONCE(sk->sk_mark), sk_uid(sk));
 }
 
-- 
2.55.0

[PATCH net 2/4] ipv4: udp: Create exceptions when socket matching failed

[Ido Schimmel]

Currently, when ICMP Fragmentation Needed and Redirect Message packets
are locally delivered and quote a UDP packet, a FIB nexthop exception
(FNHE) is created only if the kernel can match the UDP packet to an
existing socket.

This behavior allows off-path attackers to conduct a side-channel attack
on the FNHE cache in order to discover the ephemeral port used by a
connected UDP socket.

Commit 6457378fe796 ("ipv4: use siphash instead of Jenkins in
fnhe_hashfun()") and commit 67d6d681e15b ("ipv4: make exception cache
less predictible") tried to mitigate such attacks by making it harder
for attackers to discover hash collisions in the FNHE cache and by
randomizing the number of exceptions a hash bucket can hold,
respectively. Unfortunately, both of the mitigations can be bypassed.

Instead, mitigate such attacks by always creating a FNHE, even if socket
matching failed. Do that by calling ipv4_update_pmtu() and
ipv4_redirect(), the helpers used when the quoted packet did not
originate from a socket. The resulting FNHE is indistinguishable from
the one created when socket matching succeeded, both in terms of cache
occupancy and in terms of its contents.

Pass an oif of 0, in a similar fashion to icmp_err().

Note that this does not allow attackers to create FNHEs that they could
not create before, as both helpers can already be reached with little to
no validation. For example, by sending an ICMP error that quotes an ICMP
Echo Reply or one that quotes a UDP source port that matches a wildcard
socket.

Also create a FNHE when a socket does not wish to accept PMTU updates
(e.g., by setting 'IP_PMTUDISC_OMIT'). Otherwise, the fact that a FNHE
was not created can indicate to an off-path attacker that a socket
exists. This applies to all ipv4_sk_update_pmtu() callers, so ping and
raw sockets that decline PMTU updates will now create a FNHE as well.
Such sockets are not affected by it, as ip_skb_dst_mtu() uses the MTU of
the device for them.

Fixes: 4895c771c7f0 ("ipv4: Add FIB nexthop exceptions.")
Cc: stable@vger.kernel.org
Reported-by: Amit Klein <aksecurity@gmail.com>
Reported-by: Noam Caspi <noam.caspi@mail.huji.ac.il>
Reviewed-by: David Ahern <dsahern@kernel.org>
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
---
 net/ipv4/route.c |  8 +++++---
 net/ipv4/udp.c   | 10 ++++++++++
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 37674d76f90f..692298078700 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -1157,10 +1157,12 @@ void ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
 	bool new = false;
 	struct net *net = sock_net(sk);
 
-	bh_lock_sock(sk);
+	if (!ip_sk_accept_pmtu(sk)) {
+		ipv4_update_pmtu(skb, net, mtu, 0, sk->sk_protocol);
+		return;
+	}
 
-	if (!ip_sk_accept_pmtu(sk))
-		goto out;
+	bh_lock_sock(sk);
 
 	odst = sk_dst_get(sk);
 
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index af9603217444..6981526bd59c 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -900,6 +900,15 @@ static struct sock *__udp4_lib_err_encap(struct net *net,
 	return sk;
 }
 
+static void udp_err_no_sk(struct net *net, struct sk_buff *skb, int type,
+			  int code, u32 info)
+{
+	if (type == ICMP_DEST_UNREACH && code == ICMP_FRAG_NEEDED)
+		ipv4_update_pmtu(skb, net, info, 0, IPPROTO_UDP);
+	else if (type == ICMP_REDIRECT)
+		ipv4_redirect(skb, net, 0, IPPROTO_UDP);
+}
+
 /*
  * This routine is called by the ICMP module when it gets some
  * sort of error condition.  If err < 0 then the socket should
@@ -938,6 +947,7 @@ int udp_err(struct sk_buff *skb, u32 info)
 			sk = ERR_PTR(-ENOENT);
 
 		if (IS_ERR(sk)) {
+			udp_err_no_sk(net, skb, type, code, info);
 			__ICMP_INC_STATS(net, ICMP_MIB_INERRORS);
 			return PTR_ERR(sk);
 		}
-- 
2.55.0

[PATCH net 3/4] ipv6: udp: Create exceptions when socket matching failed

[Ido Schimmel]

Currently, when ICMPv6 Packet Too Big and Redirect Message packets are
locally delivered and quote a UDP packet, an exception is created in the
IPv6 exception cache only if the kernel can match the UDP packet to an
existing socket.

This behavior allows off-path attackers to conduct a side-channel attack
on the exception cache in order to discover the ephemeral port used by a
connected UDP socket.

Commit 4785305c05b2 ("ipv6: use siphash in rt6_exception_hash()") and
commit a00df2caffed ("ipv6: make exception cache less predictible") tried
to mitigate such attacks by making it harder for attackers to discover
hash collisions in the exception cache and by randomizing the number of
exceptions a hash bucket can hold, respectively. Unfortunately, both of
the mitigations can be bypassed.

Instead, mitigate such attacks by always creating an exception, even if
socket matching failed. Do that by calling ip6_update_pmtu() and
ip6_redirect(), the helpers used when the quoted packet did not
originate from a socket. The resulting exception is indistinguishable
from the one created when socket matching succeeded, both in terms of
cache occupancy and in terms of its contents.

Pass the ifindex of the ingress device and the default uid, in a similar
fashion to icmpv6_err(). Unlike IPv4, an oif of 0 would not match any
nexthop in ip6_redirect_nh_match() and no exception would be created in
response to a Redirect Message.

Note that this does not allow attackers to create exceptions that they
could not create before, as both helpers can already be reached with
little to no validation. For example, by sending an ICMPv6 error that
quotes an ICMPv6 Echo Reply or one that quotes a UDP source port that
matches a wildcard socket.

Also create an exception when a socket does not wish to accept PMTU
updates (e.g., by setting 'IPV6_PMTUDISC_OMIT'). Otherwise, the fact
that an exception was not created can indicate to an off-path attacker
that a socket exists. Unlike IPv4, the check is performed in udpv6_err()
and not in ip6_sk_update_pmtu(), as its only other caller, rawv6_err(),
does not consult ip6_sk_accept_pmtu() and therefore already creates an
exception unconditionally.

Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Cc: stable@vger.kernel.org
Reported-by: Amit Klein <aksecurity@gmail.com>
Reported-by: Noam Caspi <noam.caspi@mail.huji.ac.il>
Reviewed-by: David Ahern <dsahern@kernel.org>
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
---
 net/ipv6/udp.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index fd875908ac0c..df14fc2afe8a 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -690,6 +690,17 @@ static struct sock *__udp6_lib_err_encap(struct net *net,
 	return sk;
 }
 
+static void udpv6_err_no_sk(struct net *net, struct sk_buff *skb, u8 type,
+			    __be32 info)
+{
+	if (type == ICMPV6_PKT_TOOBIG)
+		ip6_update_pmtu(skb, net, info, skb->dev->ifindex, 0,
+				sock_net_uid(net, NULL));
+	else if (type == NDISC_REDIRECT)
+		ip6_redirect(skb, net, skb->dev->ifindex, 0,
+			     sock_net_uid(net, NULL));
+}
+
 static int udpv6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
 		     u8 type, u8 code, int offset, __be32 info)
 {
@@ -719,6 +730,7 @@ static int udpv6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
 			sk = ERR_PTR(-ENOENT);
 
 		if (IS_ERR(sk)) {
+			udpv6_err_no_sk(net, skb, type, info);
 			__ICMP6_INC_STATS(net, __in6_dev_get(skb->dev),
 					  ICMP6_MIB_INERRORS);
 			return PTR_ERR(sk);
@@ -731,8 +743,11 @@ static int udpv6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
 	np = inet6_sk(sk);
 
 	if (type == ICMPV6_PKT_TOOBIG) {
-		if (!ip6_sk_accept_pmtu(sk))
+		if (!ip6_sk_accept_pmtu(sk)) {
+			ip6_update_pmtu(skb, net, info, skb->dev->ifindex, 0,
+					sock_net_uid(net, NULL));
 			goto out;
+		}
 		ip6_sk_update_pmtu(skb, sk, info);
 		if (READ_ONCE(np->pmtudisc) != IPV6_PMTUDISC_DONT)
 			harderr = 1;
-- 
2.55.0

[PATCH net 4/4] selftests: net: Add exception cache tests

[Ido Schimmel]

Add a test for the IPv4 and IPv6 exception caches, covering the
exceptions that are created in response to ICMP errors quoting a UDP
packet.

The topology consists of a host (h1) that reaches a remote host (h2)
via a router (r1), with a second router (r2) attached to the segment
shared by h1 and r1. UDP packets are injected using a packet socket, so
that an ICMP error quoting them is only matched to a socket when one was
opened separately with the same source port. PMTU errors are provoked by
lowering the MTU of the far end of the path and redirects by pointing
r1's route towards h2 back over the segment it received the packet from.

The following is tested for both address families and for both PMTU and
redirect exceptions:

* An error that is not matched to a socket creates an exception that
  carries the new MTU or gateway.
* An error that is matched to a socket creates the same exception.

The PMTU tests further verify that a lower PMTU replaces the one stored
in the exception whereas a higher one does not, and that a socket which
disabled PMTU discovery using IP{,V6}_PMTUDISC_OMIT gets the same
exception as the other cases.

Without "ipv4: udp: Create exceptions when socket matching failed" and
"ipv6: udp: Create exceptions when socket matching failed", the tests
that do not open a socket fail:

 # ./exception_cache.sh
 TEST: IPv4: PMTU: exception without a matching socket               [FAIL]
         No socket: exception does not carry an MTU of 1400
 TEST: IPv6: PMTU: exception without a matching socket               [FAIL]
         No socket: exception does not carry an MTU of 1400
 TEST: IPv4: PMTU: exception with a matching socket                  [ OK ]
 TEST: IPv6: PMTU: exception with a matching socket                  [ OK ]
 TEST: IPv4: PMTU: exception with a socket ignoring it               [FAIL]
         PMTU discovery disabled: exception does not carry an MTU of 1400
 TEST: IPv6: PMTU: exception with a socket ignoring it               [FAIL]
         PMTU discovery disabled: exception does not carry an MTU of 1400
 TEST: IPv4: Redirect: exception without a matching socket           [FAIL]
         No socket: exception does not carry the new gateway
 TEST: IPv6: Redirect: exception without a matching socket           [FAIL]
         No socket: exception does not carry the new gateway
 TEST: IPv4: Redirect: exception with a matching socket              [ OK ]
 TEST: IPv6: Redirect: exception with a matching socket              [ OK ]

Assisted-by: Claude-Code:claude-opus-5
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
---
 tools/testing/selftests/net/Makefile          |   1 +
 .../testing/selftests/net/exception_cache.sh  | 501 ++++++++++++++++++
 2 files changed, 502 insertions(+)
 create mode 100755 tools/testing/selftests/net/exception_cache.sh

diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 0f5c178bc224..517c09d60bef 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -28,6 +28,7 @@ TEST_PROGS := \
 	double_udp_encap.sh \
 	drop_monitor_tests.sh \
 	ecmp_rehash.sh \
+	exception_cache.sh \
 	fcnal-ipv4.sh \
 	fcnal-ipv6.sh \
 	fcnal-other.sh \
diff --git a/tools/testing/selftests/net/exception_cache.sh b/tools/testing/selftests/net/exception_cache.sh
new file mode 100755
index 000000000000..2e73872332f2
--- /dev/null
+++ b/tools/testing/selftests/net/exception_cache.sh
@@ -0,0 +1,501 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Test that the state of the route exception cache after an ICMP error is
+# processed does not depend on whether the quoted packet was matched to a
+# socket. Otherwise, an off-path attacker can probe the cache to discover the
+# ephemeral port used by a connected UDP socket.
+#
+# When the quoted packet is not matched to a socket, the same exception is
+# created as when it is matched, so that neither its presence nor its contents
+# reveal the result of socket matching.
+#
+#                              +----+
+#                    +---------| r1 |
+#                    |         +----+
+#  +----+   +--------+            | .1
+#  | h1 |---| bridge |            |    198.51.100.0/30
+#  +----+   +--------+            |    2001:db8:2::/64
+#      .1            |            | .2
+#                    |         +----+           +----+
+#                    +---------| r2 |-----------| h2 |
+#                    .2   .3   +----+ .1     .2 +----+
+#                                   203.0.113.0/24
+#                                   2001:db8:3::/64
+#         192.0.2.0/24
+#         2001:db8:1::/64
+#
+# Traffic from h1 to h2 is routed via r1, which reaches h2's network via r2
+# over the point-to-point link. The MTU of the r2 - h2 link is lowered so that
+# r2 emits ICMP errors towards h1.
+#
+# For the redirect tests r1's route to h2's network is replaced with one via r2
+# on the shared segment, so that r1 forwards the packet back to the segment it
+# arrived from and emits a redirect towards h1.
+#
+# The packets that provoke the ICMP errors are injected with a packet socket so
+# that no socket is ever associated with them. A socket is created separately,
+# with socat, when a test needs the ICMP error to be matched.
+
+# shellcheck disable=SC1091,SC2034,SC2154,SC2329
+source lib.sh
+
+require_command jq
+require_command mausezahn
+require_command socat
+
+ALL_TESTS="
+	pmtu_no_socket_ipv4
+	pmtu_no_socket_ipv6
+	pmtu_socket_ipv4
+	pmtu_socket_ipv6
+	pmtu_omit_ipv4
+	pmtu_omit_ipv6
+	redirect_no_socket_ipv4
+	redirect_no_socket_ipv6
+	redirect_socket_ipv4
+	redirect_socket_ipv6
+"
+
+# Shared segment.
+H1_ADDR4=192.0.2.1
+R1_ADDR4=192.0.2.2
+R2_ADDR4=192.0.2.3
+H1_ADDR6=2001:db8:1::1
+R1_ADDR6=2001:db8:1::2
+R2_ADDR6=2001:db8:1::3
+
+# r1 - r2 link.
+R2_R1_ADDR4=198.51.100.2
+R2_R1_ADDR6=2001:db8:2::2
+
+# r2 - h2 link.
+H2_ADDR4=203.0.113.2
+H2_NET4=203.0.113.0/24
+H2_ADDR6=2001:db8:3::2
+H2_NET6=2001:db8:3::/64
+
+SPORT=12345
+DPORT=54321
+
+# The MTU of the shared segment and of the r1 - r2 link. Large enough for the
+# injected packets to reach r2 intact.
+SEGMENT_MTU=2000
+# Size of the injected packets. The PMTU tests need a size that exceeds every
+# MTU used for the r2 - h2 link, so that r2 responds with an ICMP error. The
+# redirect tests need a size that does not, otherwise r2 would respond with an
+# ICMP error in addition to the redirect emitted by r1.
+PMTU_PACKET_SIZE=1800
+REDIRECT_PACKET_SIZE=100
+
+# The MTUs used for the r2 - h2 link. All of them must be at least
+# IPV6_MIN_MTU, otherwise IPv6 silently ignores the error instead of creating
+# an exception.
+MTU_MID=1400
+MTU_LOW=1300
+
+# Values for the IP{,V6}_MTU_DISCOVER socket option.
+PMTUDISC_DONT=0
+PMTUDISC_OMIT=5
+
+SOCAT_PID=
+
+linklocal_get()
+{
+	local ns=$1; shift
+	local dev=$1; shift
+
+	ip -n "$ns" -j -6 addr show dev "$dev" | \
+		jq -r '.[]["addr_info"][] | select(.scope == "link") | .local'
+}
+
+linklocal_exists()
+{
+	local ns=$1; shift
+	local dev=$1; shift
+
+	[ -n "$(linklocal_get "$ns" "$dev")" ]
+}
+
+family_vars_set()
+{
+	local family=$1; shift
+
+	FAMILY=$family
+
+	if [ "$family" -eq 4 ]; then
+		H1_ADDR=$H1_ADDR4
+		H2_ADDR=$H2_ADDR4
+		MZ_FAMILY_OPT=()
+		# Without the Don't Fragment bit set r2 fragments the packet
+		# instead of reporting the MTU of the next hop.
+		MZ_IP_OPTS="df,"
+		SOCAT_DST="UDP4-CONNECT:$H2_ADDR4:$DPORT"
+		SOCAT_BIND="bind=$H1_ADDR4:$SPORT"
+		SOCAT_PMTUDISC="ip-mtu-discover"
+	else
+		H1_ADDR=$H1_ADDR6
+		H2_ADDR=$H2_ADDR6
+		MZ_FAMILY_OPT=(-6)
+		MZ_IP_OPTS=
+		SOCAT_DST="UDP6-CONNECT:[$H2_ADDR6]:$DPORT"
+		SOCAT_BIND="bind=[$H1_ADDR6]:$SPORT"
+		SOCAT_PMTUDISC="ipv6-mtu-discover"
+	fi
+}
+
+topology_setup()
+{
+	local ns
+
+	setup_ns h1 r1 r2 h2 sw
+	defer cleanup_all_ns
+
+	# Link-local addresses are generated from the MAC address and read
+	# back during setup, so request that generation mode explicitly and
+	# make the addresses available as soon as the devices are brought up.
+	for ns in "$h1" "$r1" "$r2" "$h2" "$sw"; do
+		ip netns exec "$ns" sysctl -qw \
+			net.ipv6.conf.default.addr_gen_mode=0 \
+			net.ipv6.conf.default.accept_dad=0 \
+			net.ipv6.conf.all.accept_dad=0
+	done
+
+	ip -n "$sw" link add name br0 type bridge
+	ip -n "$sw" link set dev br0 mtu "$SEGMENT_MTU" up
+
+	ip -n "$h1" link add name eth0 mtu "$SEGMENT_MTU" type veth \
+		peer name swp1 mtu "$SEGMENT_MTU" netns "$sw"
+	ip -n "$r1" link add name eth0 mtu "$SEGMENT_MTU" type veth \
+		peer name swp2 mtu "$SEGMENT_MTU" netns "$sw"
+	ip -n "$r2" link add name eth0 mtu "$SEGMENT_MTU" type veth \
+		peer name swp3 mtu "$SEGMENT_MTU" netns "$sw"
+	ip -n "$r1" link add name eth1 mtu "$SEGMENT_MTU" type veth \
+		peer name eth1 mtu "$SEGMENT_MTU" netns "$r2"
+	ip -n "$r2" link add name eth2 type veth peer name eth0 netns "$h2"
+
+	ip -n "$sw" link set dev swp1 master br0 up
+	ip -n "$sw" link set dev swp2 master br0 up
+	ip -n "$sw" link set dev swp3 master br0 up
+
+	ip -n "$h1" link set dev eth0 up
+	ip -n "$r1" link set dev eth0 up
+	ip -n "$r1" link set dev eth1 up
+	ip -n "$r2" link set dev eth0 up
+	ip -n "$r2" link set dev eth1 up
+	ip -n "$r2" link set dev eth2 up
+	ip -n "$h2" link set dev eth0 up
+
+	ip -n "$h1" address add "$H1_ADDR4/24" dev eth0
+	ip -n "$r1" address add "$R1_ADDR4/24" dev eth0
+	ip -n "$r2" address add "$R2_ADDR4/24" dev eth0
+	ip -n "$r1" address add 198.51.100.1/30 dev eth1
+	ip -n "$r2" address add "$R2_R1_ADDR4/30" dev eth1
+	ip -n "$r2" address add 203.0.113.1/24 dev eth2
+	ip -n "$h2" address add "$H2_ADDR4/24" dev eth0
+
+	ip -n "$h1" -6 address add "$H1_ADDR6/64" dev eth0 nodad
+	ip -n "$r1" -6 address add "$R1_ADDR6/64" dev eth0 nodad
+	ip -n "$r2" -6 address add "$R2_ADDR6/64" dev eth0 nodad
+	ip -n "$r1" -6 address add 2001:db8:2::1/64 dev eth1 nodad
+	ip -n "$r2" -6 address add "$R2_R1_ADDR6/64" dev eth1 nodad
+	ip -n "$r2" -6 address add 2001:db8:3::1/64 dev eth2 nodad
+	ip -n "$h2" -6 address add "$H2_ADDR6/64" dev eth0 nodad
+
+	ip netns exec "$r1" sysctl -qw net.ipv4.ip_forward=1
+	ip netns exec "$r1" sysctl -qw net.ipv4.conf.all.send_redirects=1
+	ip netns exec "$r1" sysctl -qw net.ipv6.conf.all.forwarding=1
+	ip netns exec "$r2" sysctl -qw net.ipv4.ip_forward=1
+	ip netns exec "$r2" sysctl -qw net.ipv6.conf.all.forwarding=1
+
+	ip netns exec "$h1" sysctl -qw net.ipv4.conf.all.accept_redirects=1
+	ip netns exec "$h1" sysctl -qw net.ipv4.conf.eth0.accept_redirects=1
+	ip netns exec "$h1" sysctl -qw net.ipv6.conf.all.accept_redirects=1
+	ip netns exec "$h1" sysctl -qw net.ipv6.conf.eth0.accept_redirects=1
+
+	slowwait 5 linklocal_exists "$r1" eth0
+	slowwait 5 linklocal_exists "$r2" eth0
+
+	R1_LLADDR=$(linklocal_get "$r1" eth0)
+	R2_LLADDR=$(linklocal_get "$r2" eth0)
+	R1_MAC=$(ip -n "$r1" -j link show dev eth0 | jq -r '.[]["address"]')
+	R2_MAC=$(ip -n "$r2" -j link show dev eth0 | jq -r '.[]["address"]')
+
+	ip -n "$h1" route add "$H2_NET4" via "$R1_ADDR4" dev eth0
+	ip -n "$h1" -6 route add "$H2_NET6" via "$R1_LLADDR" dev eth0
+	ip -n "$r1" route add "$H2_NET4" via "$R2_R1_ADDR4" dev eth1
+	ip -n "$r1" -6 route add "$H2_NET6" via "$R2_R1_ADDR6" dev eth1
+	ip -n "$h2" route add default via 203.0.113.1 dev eth0
+	ip -n "$h2" -6 route add default via 2001:db8:3::1 dev eth0
+
+	far_mtu_set "$MTU_MID"
+}
+
+# Make r1 forward towards h2's network over the segment it receives the packet
+# from, so that it emits a redirect towards h1.
+redirect_route_set()
+{
+	ip -n "$r1" route replace "$H2_NET4" via "$R2_ADDR4" dev eth0
+	ip -n "$r1" -6 route replace "$H2_NET6" via "$R2_LLADDR" dev eth0
+
+	# __ip_do_redirect() only creates an exception if the new gateway is
+	# already a valid neighbour. Otherwise it merely triggers address
+	# resolution. IPv6 resolves the target itself, in rt6_do_redirect().
+	ip -n "$h1" neigh replace "$R2_ADDR4" lladdr "$R2_MAC" dev eth0 \
+		nud permanent
+}
+
+far_mtu_set()
+{
+	local mtu=$1; shift
+
+	ip -n "$r2" link set dev eth2 mtu "$mtu"
+	ip -n "$h2" link set dev eth0 mtu "$mtu"
+}
+
+socket_is_open()
+{
+	ip netns exec "$h1" ss -uHn "sport = :$SPORT" | grep -q .
+}
+
+socket_start()
+{
+	# Disable PMTU discovery by default so that ICMP errors are not
+	# reported to the socket. Otherwise socat would exit when the first one
+	# arrives and later packets in the same test would not be matched to a
+	# socket. The exception is still created, as ip{,6}_sk_accept_pmtu()
+	# only rejects IP{,V6}_PMTUDISC_{INTERFACE,OMIT}.
+	local pmtudisc=${1:-$PMTUDISC_DONT}
+
+	# Send socat's diagnostics to /dev/null. It reports the ICMP errors
+	# that reach the socket, which is exactly what the tests provoke.
+	ip netns exec "$h1" socat -u -lf/dev/null \
+		"$SOCAT_DST,$SOCAT_BIND,$SOCAT_PMTUDISC=$pmtudisc" \
+		OPEN:/dev/null,wronly=1 &
+	SOCAT_PID=$!
+	defer socket_stop
+
+	slowwait 5 socket_is_open
+	check_err $? "socket did not open"
+}
+
+socket_stop()
+{
+	[ -z "$SOCAT_PID" ] && return 0
+
+	kill "$SOCAT_PID" &> /dev/null
+	wait "$SOCAT_PID" 2> /dev/null
+	SOCAT_PID=
+}
+
+# Inject a packet towards h2 with a packet socket. No socket is associated with
+# it, so an ICMP error quoting it is matched to a socket only if one was
+# created separately with the same source port.
+packet_send()
+{
+	local size=$1; shift
+
+	ip netns exec "$h1" mausezahn "${MZ_FAMILY_OPT[@]}" eth0 \
+		-a own -b "$R1_MAC" -A "$H1_ADDR" -B "$H2_ADDR" \
+		-t udp "${MZ_IP_OPTS}sp=$SPORT,dp=$DPORT" \
+		-p "$size" -c 1 -q
+}
+
+exception_show()
+{
+	if [ "$FAMILY" -eq 4 ]; then
+		# IPv4 exceptions without a bound route are not dumped, but
+		# "route get" reports the exception and binds a route to it.
+		ip -n "$h1" route get "$H2_ADDR"
+	else
+		# IPv6 does not report a cache indication in "route get"
+		# output, so dump the exceptions instead.
+		ip -n "$h1" -6 route show cache | grep -F "$H2_ADDR" || true
+	fi
+}
+
+exception_mtu_get()
+{
+	exception_show | grep -o "mtu [0-9]*" | cut -d ' ' -f 2
+}
+
+exception_gw_get()
+{
+	exception_show | grep -o "via [0-9a-f.:]*" | cut -d ' ' -f 2
+}
+
+exception_mtu_check()
+{
+	local expected=$1; shift
+
+	[ "$(exception_mtu_get)" = "$expected" ]
+}
+
+exception_pmtu_check()
+{
+	local mtu=$1; shift
+	local desc=$1; shift
+
+	busywait "$BUSYWAIT_TIMEOUT" exception_mtu_check "$mtu"
+	check_err $? "$desc: exception does not carry an MTU of $mtu"
+}
+
+pmtu_no_socket()
+{
+	local family=$1; shift
+
+	RET=0
+	family_vars_set "$family"
+	topology_setup
+
+	packet_send "$PMTU_PACKET_SIZE"
+	exception_pmtu_check "$MTU_MID" "No socket"
+
+	log_test "IPv$family: PMTU: exception without a matching socket"
+}
+
+pmtu_no_socket_ipv4()
+{
+	pmtu_no_socket 4
+}
+
+pmtu_no_socket_ipv6()
+{
+	pmtu_no_socket 6
+}
+
+pmtu_socket()
+{
+	local family=$1; shift
+
+	RET=0
+	family_vars_set "$family"
+	topology_setup
+	socket_start
+
+	packet_send "$PMTU_PACKET_SIZE"
+	exception_pmtu_check "$MTU_MID" "Matching socket"
+
+	# A lower PMTU replaces the one currently stored in the exception.
+	far_mtu_set "$MTU_LOW"
+	packet_send "$PMTU_PACKET_SIZE"
+	exception_pmtu_check "$MTU_LOW" "Lower PMTU"
+
+	# A higher PMTU is ignored, so the exception is left as it is.
+	far_mtu_set "$MTU_MID"
+	packet_send "$PMTU_PACKET_SIZE"
+	sleep 1
+	exception_mtu_check "$MTU_LOW"
+	check_err $? "Higher PMTU: exception does not carry an MTU of $MTU_LOW"
+
+	log_test "IPv$family: PMTU: exception with a matching socket"
+}
+
+pmtu_socket_ipv4()
+{
+	pmtu_socket 4
+}
+
+pmtu_socket_ipv6()
+{
+	pmtu_socket 6
+}
+
+pmtu_omit()
+{
+	local family=$1; shift
+
+	RET=0
+	family_vars_set "$family"
+	topology_setup
+	socket_start "$PMTUDISC_OMIT"
+
+	packet_send "$PMTU_PACKET_SIZE"
+	exception_pmtu_check "$MTU_MID" "PMTU discovery disabled"
+
+	log_test "IPv$family: PMTU: exception with a socket ignoring it"
+}
+
+pmtu_omit_ipv4()
+{
+	pmtu_omit 4
+}
+
+pmtu_omit_ipv6()
+{
+	pmtu_omit 6
+}
+
+exception_gw_check()
+{
+	local expected=$1; shift
+
+	[ "$(exception_gw_get)" = "$expected" ]
+}
+
+redirect_gw_new()
+{
+	if [ "$FAMILY" -eq 4 ]; then
+		echo "$R2_ADDR4"
+	else
+		echo "$R2_LLADDR"
+	fi
+}
+
+redirect_no_socket()
+{
+	local family=$1; shift
+
+	RET=0
+	family_vars_set "$family"
+	topology_setup
+	redirect_route_set
+
+	packet_send "$REDIRECT_PACKET_SIZE"
+	busywait "$BUSYWAIT_TIMEOUT" exception_gw_check "$(redirect_gw_new)"
+	check_err $? "No socket: exception does not carry the new gateway"
+
+	log_test "IPv$family: Redirect: exception without a matching socket"
+}
+
+redirect_no_socket_ipv4()
+{
+	redirect_no_socket 4
+}
+
+redirect_no_socket_ipv6()
+{
+	redirect_no_socket 6
+}
+
+redirect_socket()
+{
+	local family=$1; shift
+
+	RET=0
+	family_vars_set "$family"
+	topology_setup
+	redirect_route_set
+	socket_start
+
+	packet_send "$REDIRECT_PACKET_SIZE"
+	busywait "$BUSYWAIT_TIMEOUT" exception_gw_check "$(redirect_gw_new)"
+	check_err $? "Matching socket: exception does not carry the new gateway"
+
+	log_test "IPv$family: Redirect: exception with a matching socket"
+}
+
+redirect_socket_ipv4()
+{
+	redirect_socket 4
+}
+
+redirect_socket_ipv6()
+{
+	redirect_socket 6
+}
+
+trap defer_scopes_cleanup EXIT
+tests_run
+
+exit "$EXIT_STATUS"
-- 
2.55.0

Re: [PATCH net 2/4] ipv4: udp: Create exceptions when socket matching failed

[Eric Dumazet]

On Wed, Aug 26, 2026 at 4:39 PM Ido Schimmel <idosch@nvidia.com> wrote:
>
> Currently, when ICMP Fragmentation Needed and Redirect Message packets
> are locally delivered and quote a UDP packet, a FIB nexthop exception
> (FNHE) is created only if the kernel can match the UDP packet to an
> existing socket.
>
> This behavior allows off-path attackers to conduct a side-channel attack
> on the FNHE cache in order to discover the ephemeral port used by a
> connected UDP socket.
>
> Commit 6457378fe796 ("ipv4: use siphash instead of Jenkins in
> fnhe_hashfun()") and commit 67d6d681e15b ("ipv4: make exception cache
> less predictible") tried to mitigate such attacks by making it harder
> for attackers to discover hash collisions in the FNHE cache and by
> randomizing the number of exceptions a hash bucket can hold,
> respectively. Unfortunately, both of the mitigations can be bypassed.
>
> Instead, mitigate such attacks by always creating a FNHE, even if socket
> matching failed. Do that by calling ipv4_update_pmtu() and
> ipv4_redirect(), the helpers used when the quoted packet did not
> originate from a socket. The resulting FNHE is indistinguishable from
> the one created when socket matching succeeded, both in terms of cache
> occupancy and in terms of its contents.
>
> Pass an oif of 0, in a similar fashion to icmp_err().
>
> Note that this does not allow attackers to create FNHEs that they could
> not create before, as both helpers can already be reached with little to
> no validation. For example, by sending an ICMP error that quotes an ICMP
> Echo Reply or one that quotes a UDP source port that matches a wildcard
> socket.
>
> Also create a FNHE when a socket does not wish to accept PMTU updates
> (e.g., by setting 'IP_PMTUDISC_OMIT'). Otherwise, the fact that a FNHE
> was not created can indicate to an off-path attacker that a socket
> exists. This applies to all ipv4_sk_update_pmtu() callers, so ping and
> raw sockets that decline PMTU updates will now create a FNHE as well.
> Such sockets are not affected by it, as ip_skb_dst_mtu() uses the MTU of
> the device for them.
>
> Fixes: 4895c771c7f0 ("ipv4: Add FIB nexthop exceptions.")
> Cc: stable@vger.kernel.org
> Reported-by: Amit Klein <aksecurity@gmail.com>
> Reported-by: Noam Caspi <noam.caspi@mail.huji.ac.il>
> Reviewed-by: David Ahern <dsahern@kernel.org>
> Signed-off-by: Ido Schimmel <idosch@nvidia.com>

Reviewed-by: Eric Dumazet <edumazet@google.com>

Re: [PATCH net 3/4] ipv6: udp: Create exceptions when socket matching failed

[Eric Dumazet]

On Wed, Aug 26, 2026 at 4:39 PM Ido Schimmel <idosch@nvidia.com> wrote:
>
> Currently, when ICMPv6 Packet Too Big and Redirect Message packets are
> locally delivered and quote a UDP packet, an exception is created in the
> IPv6 exception cache only if the kernel can match the UDP packet to an
> existing socket.
>
> This behavior allows off-path attackers to conduct a side-channel attack
> on the exception cache in order to discover the ephemeral port used by a
> connected UDP socket.
>
> Commit 4785305c05b2 ("ipv6: use siphash in rt6_exception_hash()") and
> commit a00df2caffed ("ipv6: make exception cache less predictible") tried
> to mitigate such attacks by making it harder for attackers to discover
> hash collisions in the exception cache and by randomizing the number of
> exceptions a hash bucket can hold, respectively. Unfortunately, both of
> the mitigations can be bypassed.
>
> Instead, mitigate such attacks by always creating an exception, even if
> socket matching failed. Do that by calling ip6_update_pmtu() and
> ip6_redirect(), the helpers used when the quoted packet did not
> originate from a socket. The resulting exception is indistinguishable
> from the one created when socket matching succeeded, both in terms of
> cache occupancy and in terms of its contents.
>
> Pass the ifindex of the ingress device and the default uid, in a similar
> fashion to icmpv6_err(). Unlike IPv4, an oif of 0 would not match any
> nexthop in ip6_redirect_nh_match() and no exception would be created in
> response to a Redirect Message.
>
> Note that this does not allow attackers to create exceptions that they
> could not create before, as both helpers can already be reached with
> little to no validation. For example, by sending an ICMPv6 error that
> quotes an ICMPv6 Echo Reply or one that quotes a UDP source port that
> matches a wildcard socket.
>
> Also create an exception when a socket does not wish to accept PMTU
> updates (e.g., by setting 'IPV6_PMTUDISC_OMIT'). Otherwise, the fact
> that an exception was not created can indicate to an off-path attacker
> that a socket exists. Unlike IPv4, the check is performed in udpv6_err()
> and not in ip6_sk_update_pmtu(), as its only other caller, rawv6_err(),
> does not consult ip6_sk_accept_pmtu() and therefore already creates an
> exception unconditionally.
>
> Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
> Cc: stable@vger.kernel.org
> Reported-by: Amit Klein <aksecurity@gmail.com>
> Reported-by: Noam Caspi <noam.caspi@mail.huji.ac.il>
> Reviewed-by: David Ahern <dsahern@kernel.org>
> Signed-off-by: Ido Schimmel <idosch@nvidia.com>

Reviewed-by: Eric Dumazet <edumazet@google.com>

Re: [PATCH net 2/4] ipv4: udp: Create exceptions when socket matching failed

[Ido Schimmel]

On Wed, Aug 26, 2026 at 05:37:33PM +0300, Ido Schimmel wrote:
> Currently, when ICMP Fragmentation Needed and Redirect Message packets
> are locally delivered and quote a UDP packet, a FIB nexthop exception
> (FNHE) is created only if the kernel can match the UDP packet to an
> existing socket.
> 
> This behavior allows off-path attackers to conduct a side-channel attack
> on the FNHE cache in order to discover the ephemeral port used by a
> connected UDP socket.
> 
> Commit 6457378fe796 ("ipv4: use siphash instead of Jenkins in
> fnhe_hashfun()") and commit 67d6d681e15b ("ipv4: make exception cache
> less predictible") tried to mitigate such attacks by making it harder
> for attackers to discover hash collisions in the FNHE cache and by
> randomizing the number of exceptions a hash bucket can hold,
> respectively. Unfortunately, both of the mitigations can be bypassed.
> 
> Instead, mitigate such attacks by always creating a FNHE, even if socket
> matching failed. Do that by calling ipv4_update_pmtu() and
> ipv4_redirect(), the helpers used when the quoted packet did not
> originate from a socket. The resulting FNHE is indistinguishable from
> the one created when socket matching succeeded, both in terms of cache
> occupancy and in terms of its contents.
> 
> Pass an oif of 0, in a similar fashion to icmp_err().
> 
> Note that this does not allow attackers to create FNHEs that they could
> not create before, as both helpers can already be reached with little to
> no validation. For example, by sending an ICMP error that quotes an ICMP
> Echo Reply or one that quotes a UDP source port that matches a wildcard
> socket.
> 
> Also create a FNHE when a socket does not wish to accept PMTU updates
> (e.g., by setting 'IP_PMTUDISC_OMIT'). Otherwise, the fact that a FNHE
> was not created can indicate to an off-path attacker that a socket
> exists. This applies to all ipv4_sk_update_pmtu() callers, so ping and
> raw sockets that decline PMTU updates will now create a FNHE as well.
> Such sockets are not affected by it, as ip_skb_dst_mtu() uses the MTU of
> the device for them.

tl;dr - I will send v2 that calls udp_err_no_sk() unconditionally and
remove the net/ipv4/route.c hunk. Same for patch 3. It means two route
lookups in the good case (matched socket), but I think we can live with
that.

Two comments from Sashiko [1]:

1. "Inverted signal". This is correct and it's something I considered,
but it's impractical:

a. Without the patch, a wrong guess is cheap and an attacker can keep
scanning. With the patch, each wrong guess requires the attacker to
repeat the setup phase which brings the cache to a state where a wrong /
correct guess is indicative of the presence of a socket. This is
time-consuming and therefore impractical given that the sockets of
interest are short lived.

b. It requires "non-default routing attributes" which is uncommon.

We can completely eliminate the "inverted signal" by calling
udp_err_no_sk() unconditionally.

2. The comment regarding fou/gue seems valid, but it requires fou to be
loaded which is again uncommon. Can be fixed by [2] or by simply calling
udp_err_no_sk() unconditionally.

[1] https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260826143735.1819315-1-idosch%40nvidia.com

[2]
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 6981526bd59c..c32c416601da 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -941,8 +941,10 @@ int udp_err(struct sk_buff *skb, u32 info)
 		/* No socket for error: try tunnels before discarding */
 		if (static_branch_unlikely(&udp_encap_needed_key)) {
 			sk = __udp4_lib_err_encap(net, iph, uh, sk, skb, info);
-			if (!sk)
+			if (!sk) {
+				udp_err_no_sk(net, skb, type, code, info);
 				return 0;
+			}
 		} else
 			sk = ERR_PTR(-ENOENT);