* [PATCH iproute2 v2] ip: flush nexthop-object routes without compatibility attributes @ 2026-10-26 10:24 Yuen Su 2026-11-16 23:16 ` Stephen Hemminger 0 siblings, 1 reply; 4+ messages in thread From: Yuen Su @ 2026-10-26 10:24 UTC (permalink / raw) To: netdev; +Cc: Stephen Hemminger Route dumps can contain RTA_NH_ID plus expanded nexthop attributes. The kernel rejects that combination on route deletion. Strip the expansion from the flush copy, keeping the ID and route selectors. Use the compacted length for batching and retain the dump for display. IPv4 fib_dump_info() also reports blackhole nexthop objects as RTN_BLACKHOLE instead of the alias type, independently of nexthop_compat_mode. Work around this with RTN_UNSPEC only when no type filter was requested. The kernel then deletes the first alias matching the remaining selectors; alias order and changes between dump and deletion can affect that choice. Preserve explicit types. Add namespace regressions covering ID-based flushes, ordinary routes, batching and type selection, with a nexthop capability probe. Signed-off-by: Yuen Su <zypulsecore@gmail.com> --- v2: - Document the IPv4 dump workaround and its alias-matching tradeoff. - Remove the unused namespace guard and probe nexthop availability. - Rename the delete-request pointer and shorten the submission notes. - Use git send-email for delivery. AI assistance was used. Link: https://lore.kernel.org/netdev/CAC9WPdwmM4=64DH5nY85srnLhRBS7PkjR9vrsALpob-GutbmZw@mail.gmail.com/ ip/iproute.c | 39 ++++++- testsuite/tests/ip/route/flush_nexthop.t | 140 +++++++++++++++++++++++ 2 files changed, 178 insertions(+), 1 deletion(-) create mode 100755 testsuite/tests/ip/route/flush_nexthop.t diff --git a/ip/iproute.c b/ip/iproute.c index 1ce71f7..bd436c1 100644 --- a/ip/iproute.c +++ b/ip/iproute.c @@ -766,6 +766,38 @@ static void print_rta_multipath(FILE *fp, const struct rtmsg *r, close_json_array(PRINT_JSON, NULL); } +static void flush_nexthop_compat(struct nlmsghdr *n, + const struct nlmsghdr *source) +{ + const struct rtmsg *r = NLMSG_DATA(source); + struct rtmsg *dr = NLMSG_DATA(n); + const struct rtattr *rta = RTM_RTA(r); + int len = RTM_PAYLOAD(source); + char *dst = (char *)n + NLMSG_LENGTH(sizeof(*r)); + + for (; RTA_OK(rta, len); rta = RTA_NEXT(rta, len)) { + switch (rta->rta_type & NLA_TYPE_MASK) { + case RTA_OIF: + case RTA_GATEWAY: + case RTA_MULTIPATH: + case RTA_VIA: + case RTA_ENCAP_TYPE: + case RTA_ENCAP: + continue; + } + memcpy(dst, rta, RTA_ALIGN(rta->rta_len)); + dst += RTA_ALIGN(rta->rta_len); + } + n->nlmsg_len = dst - (char *)n; + /* Work around fib_dump_info() replacing the IPv4 alias type with + * RTN_BLACKHOLE for blackhole nexthop objects, independently of + * nexthop_compat_mode. Keep explicit type constraints. + */ + if (!filter.typemask && + r->rtm_family == AF_INET && r->rtm_type == RTN_BLACKHOLE) + dr->rtm_type = RTN_UNSPEC; +} + int print_route(struct nlmsghdr *n, void *arg) { FILE *fp = (FILE *)arg; @@ -810,10 +842,15 @@ int print_route(struct nlmsghdr *n, void *arg) } fn = (struct nlmsghdr *)(filter.flushb + NLMSG_ALIGN(filter.flushp)); memcpy(fn, n, n->nlmsg_len); + /* Keep the exact nexthop ID; its compatibility expansion conflicts + * with that ID in the delete request. + */ + if (tb[RTA_NH_ID]) + flush_nexthop_compat(fn, n); fn->nlmsg_type = RTM_DELROUTE; fn->nlmsg_flags = NLM_F_REQUEST; fn->nlmsg_seq = ++rth.seq; - filter.flushp = (((char *)fn) + n->nlmsg_len) - filter.flushb; + filter.flushp = (((char *)fn) + fn->nlmsg_len) - filter.flushb; filter.flushed++; if (show_stats < 2) return 0; diff --git a/testsuite/tests/ip/route/flush_nexthop.t b/testsuite/tests/ip/route/flush_nexthop.t new file mode 100755 index 0000000..70d09f8 --- /dev/null +++ b/testsuite/tests/ip/route/flush_nexthop.t @@ -0,0 +1,140 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-only + +. lib/generic.sh + +# The testsuite normally supplies IP and runs each test under unshare -n. +[ -n "$IP" ] || ts_skip +if ! LC_ALL=C "$IP" nexthop list > "$STD_OUT" 2> "$STD_ERR"; then + if [ "$(cat "$STD_ERR")" = "$(printf '%s\n' \ + 'RTNETLINK answers: Operation not supported' 'Dump terminated')" ]; then + ts_skip + fi + ts_err "$0: nexthop feature probe failed:" + ts_err_cat "$STD_ERR" + exit 1 +fi + +ts_log "[Testing route flush with nexthop objects and compatibility attributes]" +DEV=nhflush0 +ts_ip "$0" "Bring namespace loopback up" link set lo up +ts_ip "$0" "Create private dummy device" link add "$DEV" type dummy +ts_ip "$0" "Bring device up" link set "$DEV" up + +for family in 4 6; do + if [ "$family" = 4 ]; then + addr=192.0.2.2/24 + gw=192.0.2.1 + gw2=192.0.2.3 + prefix=203.0.113.0/24 + group_prefix=198.51.100.0/24 + plain_prefix=198.18.0.0/24 + nh=101 + nh2=102 + group=103 + else + addr=2001:db8:1::2/64 + gw=2001:db8:1::1 + gw2=2001:db8:1::3 + prefix=2001:db8:2::/64 + group_prefix=2001:db8:3::/64 + plain_prefix=2001:db8:4::/64 + nh=201 + nh2=202 + group=203 + fi + if [ "$family" = 6 ]; then + ts_ip "$0" "Assign IPv6 address" -6 addr add "$addr" dev "$DEV" nodad + else + ts_ip "$0" "Assign IPv4 address" -4 addr add "$addr" dev "$DEV" + fi + ts_ip "$0" "Create first nexthop" -"$family" nexthop add id "$nh" via "$gw" dev "$DEV" onlink + ts_ip "$0" "Create second nexthop" -"$family" nexthop add id "$nh2" via "$gw2" dev "$DEV" + ts_ip "$0" "Create nexthop group" nexthop add id "$group" group "$nh/$nh2" + ts_ip "$0" "Add selected single-nexthop route" -"$family" route add table 111 "$prefix" nhid "$nh" metric 10 mtu 1400 advmss 1360 + ts_ip "$0" "Add another metric on same prefix" -"$family" route add table 111 "$prefix" nhid "$nh2" metric 20 + ts_ip "$0" "Add same route in another table" -"$family" route add table 112 "$prefix" nhid "$nh" metric 10 + ts_ip "$0" "Flush selected metric only" -"$family" route flush table 111 metric 10 + ts_ip "$0" "Check other metric survived" -"$family" route show table 111 "$prefix" + test_on "nhid $nh2" + test_on "metric 20" + test_lines_count 1 + ts_ip "$0" "Check other table survived" -"$family" route show table 112 "$prefix" + test_on "nhid $nh" + test_lines_count 1 + ts_ip "$0" "Add multipath nexthop-object route" -"$family" route add table 111 "$group_prefix" nhid "$group" metric 30 + ts_ip "$0" "Flush group with verbose display of original dump" -"$family" -s -s route flush table 111 to "$group_prefix" + test_on "nhid $group" + ts_ip "$0" "Check group route was removed" -"$family" route show table 111 "$group_prefix" + test_lines_count 0 + ts_ip "$0" "Check nexthop group itself survived" nexthop show id "$group" + test_on "id $group group" + ts_ip "$0" "Add ordinary multipath route without ID" -"$family" route add table 111 "$plain_prefix" nexthop via "$gw" dev "$DEV" weight 1 nexthop via "$gw2" dev "$DEV" weight 2 + ts_ip "$0" "Flush ordinary multipath route" -"$family" route flush table 111 to "$plain_prefix" + ts_ip "$0" "Check ordinary route removed" -"$family" route show table 111 "$plain_prefix" + test_lines_count 0 + # One flush batch mixes compacted single/group messages with an unchanged + # ordinary route. The next message must start at the new compacted length. + ts_ip "$0" "Add group to mixed flush batch" -"$family" route add table 111 "$group_prefix" nhid "$group" metric 30 + ts_ip "$0" "Add ordinary route to mixed flush batch" -"$family" route add table 111 "$plain_prefix" dev "$DEV" metric 40 + ts_ip "$0" "Flush mixed batch with original dump display" -"$family" -s -s route flush table 111 + test_on "nhid $nh2" + test_on "nhid $group" + ts_ip "$0" "Check no selected routes remain" -"$family" route show table 111 + test_lines_count 0 + ts_ip "$0" "Flush second table" -"$family" route flush table 112 +done + +# A blackhole object has no compatibility gateway/device fields to strip. +ts_ip "$0" "Create IPv4 blackhole nexthop" -4 nexthop add id 301 blackhole +ts_ip "$0" "Add blackhole-object route" -4 route add table 111 192.0.2.0/24 nhid 301 +ts_ip "$0" "Flush ID-only blackhole route" -4 route flush table 111 +ts_ip "$0" "Check blackhole object retained" nexthop show id 301 +test_on "id 301 blackhole" +for type in blackhole unreachable prohibit; do + ts_ip "$0" "Add $type alias using blackhole object" -4 route add "$type" 192.0.2.0/24 table 111 nhid 301 + ts_ip "$0" "Flush $type alias without relying on synthesized dump type" -4 route flush table 111 + ts_ip "$0" "Check $type alias removed" -4 route show table 111 + test_lines_count 0 +done +ts_ip "$0" "Create IPv6 blackhole nexthop" -6 nexthop add id 302 blackhole +ts_ip "$0" "Add IPv6 blackhole-object route" -6 route add table 111 2001:db8:5::/64 nhid 302 +ts_ip "$0" "Flush IPv6 ID-only blackhole route" -6 route flush table 111 +ts_ip "$0" "Check IPv6 blackhole object retained" nexthop show id 302 +test_on "id 302 blackhole" +ts_ip "$0" "Add ordinary blackhole route without ID" -4 route add blackhole 192.0.2.0/24 table 111 +ts_ip "$0" "Flush ordinary blackhole route" -4 route flush table 111 +ts_ip "$0" "Check ordinary blackhole route removed" -4 route show table 111 +test_lines_count 0 + +# Type filtering must not delete another alias sharing the prefix, metric and ID. +ts_ip "$0" "Add unicast type-filter sentinel" -4 route add 203.0.113.0/24 table 111 nhid 101 metric 50 +ts_ip "$0" "Append explicit blackhole alias with same nexthop" -4 route append blackhole 203.0.113.0/24 table 111 nhid 101 metric 50 +ts_ip "$0" "Flush only explicit blackhole type" -4 route flush table 111 type blackhole +ts_ip "$0" "Check unicast alias survived type filtering" -4 route show table 111 +test_on "nhid 101" +test_lines_count 1 +ts_ip "$0" "Delete remaining unicast sentinel by exact type" -4 route del unicast 203.0.113.0/24 table 111 nhid 101 metric 50 + +# A blackhole object hides the original IPv4 alias type in dumps. An explicit +# type request must fail conservatively, rather than delete the unicast alias. +ts_ip "$0" "Add alias with synthesized dump type" -4 route add 192.0.2.0/24 table 111 nhid 301 +if $IP -4 route flush table 111 type blackhole > "$STD_OUT" 2> "$STD_ERR"; then + ts_err "Ambiguous blackhole-object type filter unexpectedly succeeded" +else + ts_log "Ambiguous explicit type failed conservatively:" + ts_cat "$STD_ERR" +fi +ts_ip "$0" "Check ambiguous unicast alias retained" -4 route show table 111 +test_lines_count 1 +ts_ip "$0" "Delete retained alias by its actual unicast type" -4 route del unicast 192.0.2.0/24 table 111 nhid 301 + +# IPv4 via an IPv6 nexthop expands to RTA_VIA rather than RTA_GATEWAY. +ts_ip "$0" "Add cross-family nexthop route" -4 route add table 111 203.0.113.0/24 nhid 201 +ts_ip "$0" "Flush cross-family route" -4 route flush table 111 +ts_ip "$0" "Check cross-family route removed" -4 route show table 111 +test_lines_count 0 + +ts_ip "$0" "Delete test device and its nexthops" link del "$DEV" +ts_ip "$0" "Delete blackhole nexthop" nexthop del id 301 +ts_ip "$0" "Delete IPv6 blackhole nexthop" nexthop del id 302 base-commit: fb6f0637f16b974767004f50d5e9905c8401815a -- 2.55.0 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH iproute2 v2] ip: flush nexthop-object routes without compatibility attributes 2026-10-26 10:24 [PATCH iproute2 v2] ip: flush nexthop-object routes without compatibility attributes Yuen Su @ 2026-11-16 23:16 ` Stephen Hemminger 2026-11-17 5:25 ` [PATCH iproute2 v3] " Yuen Su 0 siblings, 1 reply; 4+ messages in thread From: Stephen Hemminger @ 2026-11-16 23:16 UTC (permalink / raw) To: Yuen Su; +Cc: netdev On Mon, 26 Oct 2026 18:24:20 +0800 Yuen Su <zypulsecore@gmail.com> wrote: > Route dumps can contain RTA_NH_ID plus expanded nexthop attributes. > The kernel rejects that combination on route deletion. Strip the > expansion from the flush copy, keeping the ID and route selectors. > Use the compacted length for batching and retain the dump for display. > > IPv4 fib_dump_info() also reports blackhole nexthop objects as > RTN_BLACKHOLE instead of the alias type, independently of > nexthop_compat_mode. Work around this with RTN_UNSPEC only when no > type filter was requested. The kernel then deletes the first alias > matching the remaining selectors; alias order and changes between > dump and deletion can affect that choice. Preserve explicit types. > > Add namespace regressions covering ID-based flushes, ordinary routes, > batching and type selection, with a nexthop capability probe. > > Signed-off-by: Yuen Su <zypulsecore@gmail.com> > --- In general looked OK, but AI review with review-prompts showed some issues. On Mon, 26 Oct 2026 18:24:20 +0800, Yuen Su wrote: The mechanism here is right, and I checked the premise against the kernel: fib_frontend.c rtm_to_fib_config() rejects fc_nh_id combined with fc_oif/fc_gw_family/fc_encap/fc_mp, and your strip list covers exactly those four (RTA_OIF, RTA_GATEWAY/RTA_VIA, RTA_ENCAP/ RTA_ENCAP_TYPE, RTA_MULTIPATH). fib_nh_match() returns 0 on an ID-only request, so the compacted message matches. The AF_INET-only restriction on the type workaround is also correct: ip6_route_del() never compares fc_type against rt->fib6_type, so IPv6 needs no equivalent even though rt6_fill_node() synthesizes RTN_BLACKHOLE the same way. One case the workaround does not cover. > + /* Work around fib_dump_info() replacing the IPv4 alias type with > + * RTN_BLACKHOLE for blackhole nexthop objects, independently of > + * nexthop_compat_mode. Keep explicit type constraints. > + */ > + if (!filter.typemask && > + r->rtm_family == AF_INET && r->rtm_type == RTN_BLACKHOLE) > + dr->rtm_type = RTN_UNSPEC; filter.typemask is not only set by an explicit "type" argument. The vrf keyword sets it too, in iproute_list_flush_or_save(): } else if (matches(*argv, "vrf") == 0) { ... filter.tb = tid; filter.typemask = ~(1 << RTN_LOCAL | 1<<RTN_BROADCAST); That parser is shared between list, flush and save, so ip route flush vrf blue leaves typemask nonzero with no type requested by the user, the workaround is skipped, and the delete request carries the synthesized RTN_BLACKHOLE. For an IPv4 unicast route using a blackhole nexthop object in that VRF table, fib_table_delete() requires an exact match: if ((!cfg->fc_type || fa->fa_type == cfg->fc_type) && fa_type is RTN_UNICAST, fc_type is RTN_BLACKHOLE, no alias matches, and the delete returns ESRCH. iproute_flush() keeps redumping and resending until the 30 second timeout, then reports entries remaining. To be clear about severity: this is not a regression. Those routes fail to flush before your patch as well, for the same reason the patch exists. It is a gap in the fix, and worth deciding on rather than leaving implicit, because flush vrf is a common way to tear down a VRF. The awkward part is that typemask cannot distinguish the two cases by inspection - the vrf mask has the RTN_BLACKHOLE bit set, so you cannot test for it. How would you prefer to handle it? A separate flag recording that the user actually passed "type" would make the guard say what the comment says, and would leave the vrf mask alone: } else if (matches(*argv, "type") == 0) { ... filter.typemask = (1<<type); filter.type_explicit = 1; then test !filter.type_explicit in flush_nexthop_compat(). I have no strong preference on the spelling; the point is that the current guard is testing a proxy that has a second writer. Worth extending the new test to cover it, since it is the one flush form where the workaround silently does not apply. Everything else I looked at checked out. The buffer arithmetic is sound: the space check against the uncompacted n->nlmsg_len is conservative because compaction only shrinks, and nlmsg_len stays NLMSG-aligned since it starts at NLMSG_LENGTH(sizeof(*r)) and advances by RTA_ALIGN, so the NLMSG_ALIGN(filter.flushp) arithmetic for the next message in the batch still holds. Display still uses the untouched dump. The testsuite helpers all exist in lib/generic.sh and the build is clean. ^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH iproute2 v3] ip: flush nexthop-object routes without compatibility attributes 2026-11-16 23:16 ` Stephen Hemminger @ 2026-11-17 5:25 ` Yuen Su 2026-11-17 21:25 ` Stephen Hemminger 0 siblings, 1 reply; 4+ messages in thread From: Yuen Su @ 2026-11-17 5:25 UTC (permalink / raw) To: netdev; +Cc: Stephen Hemminger Route dumps can contain RTA_NH_ID plus expanded nexthop attributes. The kernel rejects that combination on deletion. Strip the expansion from the flush copy, preserving the ID and other route selectors. Use the compacted length for batching and retain the original for display. Blackhole nexthops also hide the IPv4 alias type. Track explicit type selectors separately from the VRF default. Without an explicit type, clear the synthetic type for unfiltered requests and for VRF requests with scope below LINK. Local and broadcast aliases cannot have that scope, and deletion still matches it. Ambiguous link/host-scope aliases retain the existing failure. Leave IPv6 blackhole dumps unchanged: they can hide a local type, and deletion does not check the type. Stripping their expansion would allow a VRF flush to delete a local route that previously survived an error. Extend the namespace tests with the VRF case, explicit types in either argument order, excluded local/broadcast aliases and IPv6 retention. Signed-off-by: Yuen Su <zypulsecore@gmail.com> --- v3: - Record explicit type selectors separately from the VRF default, and keep them explicit in either argument order. - Apply the IPv4 workaround to VRF requests only below LINK scope. A flag alone also deletes hidden local/broadcast aliases. Link/host-scope ambiguity retains the existing failure. - Preserve the original IPv6 blackhole delete request: stripping its expansion enables deletion of a hidden local route on Linux 5.4 and 7.3. - Extend the existing tests with VRF and type-retention cases. Tested with compatibility expansion on/off, strict dump unavailable via seccomp, and a Linux 5.4 VM. Linux 4.19 only covers ordinary routes; it has no nexthop objects. No new strict-dump dependency is introduced. AI assistance was used. Link: https://lists.openwall.net/netdev/2026/10/26/186 ip/iproute.c | 47 +++++- testsuite/tests/ip/route/flush_nexthop.t | 176 +++++++++++++++++++++++ 2 files changed, 221 insertions(+), 2 deletions(-) create mode 100755 testsuite/tests/ip/route/flush_nexthop.t diff --git a/ip/iproute.c b/ip/iproute.c index 067569c..5f776ee 100644 --- a/ip/iproute.c +++ b/ip/iproute.c @@ -131,6 +131,7 @@ static struct int protocol, protocolmask; int scope, scopemask; __u64 typemask; + bool type_explicit; int tos, tosmask; int iif, iifmask; int oif, oifmask; @@ -766,6 +767,39 @@ static void print_rta_multipath(FILE *fp, const struct rtmsg *r, close_json_array(PRINT_JSON, NULL); } +static void flush_nexthop_compat(struct nlmsghdr *n, + const struct nlmsghdr *source) +{ + const struct rtmsg *r = NLMSG_DATA(source); + struct rtmsg *dr = NLMSG_DATA(n); + const struct rtattr *rta = RTM_RTA(r); + int len = RTM_PAYLOAD(source); + char *dst = (char *)n + NLMSG_LENGTH(sizeof(*r)); + + for (; RTA_OK(rta, len); rta = RTA_NEXT(rta, len)) { + switch (rta->rta_type & NLA_TYPE_MASK) { + case RTA_OIF: + case RTA_GATEWAY: + case RTA_MULTIPATH: + case RTA_VIA: + case RTA_ENCAP_TYPE: + case RTA_ENCAP: + continue; + } + memcpy(dst, rta, RTA_ALIGN(rta->rta_len)); + dst += RTA_ALIGN(rta->rta_len); + } + n->nlmsg_len = dst - (char *)n; + /* Blackhole nexthops hide the IPv4 alias type. With a VRF mask, + * only scopes below LINK exclude both local and broadcast aliases. + * Keep the scope in the delete request and never relax explicit types. + */ + if (!filter.type_explicit && + (!filter.typemask || r->rtm_scope < RT_SCOPE_LINK) && + r->rtm_family == AF_INET && r->rtm_type == RTN_BLACKHOLE) + dr->rtm_type = RTN_UNSPEC; +} + int print_route(struct nlmsghdr *n, void *arg) { FILE *fp = (FILE *)arg; @@ -810,10 +844,17 @@ int print_route(struct nlmsghdr *n, void *arg) } fn = (struct nlmsghdr *)(filter.flushb + NLMSG_ALIGN(filter.flushp)); memcpy(fn, n, n->nlmsg_len); + /* Keep the exact nexthop ID without its conflicting expansion. + * Leave IPv6 blackhole dumps unchanged: they can hide a local + * type, and deletion does not check the type. + */ + if (tb[RTA_NH_ID] && + !(r->rtm_family == AF_INET6 && r->rtm_type == RTN_BLACKHOLE)) + flush_nexthop_compat(fn, n); fn->nlmsg_type = RTM_DELROUTE; fn->nlmsg_flags = NLM_F_REQUEST; fn->nlmsg_seq = ++rth.seq; - filter.flushp = (((char *)fn) + n->nlmsg_len) - filter.flushb; + filter.flushp = (((char *)fn) + fn->nlmsg_len) - filter.flushb; filter.flushed++; if (show_stats < 2) return 0; @@ -1866,7 +1907,8 @@ static int iproute_list_flush_or_save(int argc, char **argv, int action) if (tid == 0) invarg("Invalid VRF\n", *argv); filter.tb = tid; - filter.typemask = ~(1 << RTN_LOCAL | 1<<RTN_BROADCAST); + if (!filter.type_explicit) + filter.typemask = ~(1 << RTN_LOCAL | 1<<RTN_BROADCAST); } else if (matches(*argv, "cached") == 0 || matches(*argv, "cloned") == 0) { filter.cloned = 1; @@ -1910,6 +1952,7 @@ static int iproute_list_flush_or_save(int argc, char **argv, int action) if (rtnl_rtntype_a2n(&type, *argv)) invarg("node type value is invalid\n", *argv); filter.typemask = (1<<type); + filter.type_explicit = true; } else if (strcmp(*argv, "dev") == 0 || strcmp(*argv, "oif") == 0) { NEXT_ARG(); diff --git a/testsuite/tests/ip/route/flush_nexthop.t b/testsuite/tests/ip/route/flush_nexthop.t new file mode 100755 index 0000000..6820ff4 --- /dev/null +++ b/testsuite/tests/ip/route/flush_nexthop.t @@ -0,0 +1,176 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-only + +. lib/generic.sh + +[ -n "$IP" ] || ts_skip +if ! LC_ALL=C "$IP" nexthop list > "$STD_OUT" 2> "$STD_ERR"; then + if [ "$(cat "$STD_ERR")" = "$(printf '%s\n' \ + 'RTNETLINK answers: Operation not supported' 'Dump terminated')" ]; then + ts_skip + fi + ts_err "$0: nexthop feature probe failed:" + ts_err_cat "$STD_ERR" + exit 1 +fi + +ts_log "[Testing route flush with nexthop compatibility attributes]" +DEV=nhflush0 +ts_ip "$0" "Bring loopback up" link set lo up +ts_ip "$0" "Create dummy device" link add "$DEV" type dummy +ts_ip "$0" "Bring device up" link set "$DEV" up + +for family in 4 6; do + if [ "$family" = 4 ]; then + addr=192.0.2.2/24 + gw=192.0.2.1 + gw2=192.0.2.3 + prefix=203.0.113.0/24 + group_prefix=198.51.100.0/24 + plain_prefix=198.18.0.0/24 + nh=101 + nh2=102 + group=103 + else + addr=2001:db8:1::2/64 + gw=2001:db8:1::1 + gw2=2001:db8:1::3 + prefix=2001:db8:2::/64 + group_prefix=2001:db8:3::/64 + plain_prefix=2001:db8:4::/64 + nh=201 + nh2=202 + group=203 + fi + if [ "$family" = 6 ]; then + ts_ip "$0" "Assign IPv6 address" -6 addr add "$addr" dev "$DEV" nodad + else + ts_ip "$0" "Assign IPv4 address" -4 addr add "$addr" dev "$DEV" + fi + ts_ip "$0" "Create first nexthop" -"$family" nexthop add id "$nh" via "$gw" dev "$DEV" onlink + ts_ip "$0" "Create second nexthop" -"$family" nexthop add id "$nh2" via "$gw2" dev "$DEV" + ts_ip "$0" "Create nexthop group" nexthop add id "$group" group "$nh/$nh2" + ts_ip "$0" "Add selected route with metrics" -"$family" route add table 111 "$prefix" nhid "$nh" metric 10 mtu 1400 advmss 1360 + ts_ip "$0" "Add another metric on same prefix" -"$family" route add table 111 "$prefix" nhid "$nh2" metric 20 + ts_ip "$0" "Add same route in another table" -"$family" route add table 112 "$prefix" nhid "$nh" metric 10 + ts_ip "$0" "Flush selected metric" -"$family" route flush table 111 metric 10 + ts_ip "$0" "Check other metric survived" -"$family" route show table 111 "$prefix" + test_on "nhid $nh2" + test_on "metric 20" + test_lines_count 1 + ts_ip "$0" "Check other table survived" -"$family" route show table 112 "$prefix" + test_on "nhid $nh" + test_lines_count 1 + ts_ip "$0" "Add group route" -"$family" route add table 111 "$group_prefix" nhid "$group" metric 30 + ts_ip "$0" "Flush group with original dump display" -"$family" -s -s route flush table 111 to "$group_prefix" + test_on "nhid $group" + ts_ip "$0" "Check group route removed" -"$family" route show table 111 "$group_prefix" + test_lines_count 0 + ts_ip "$0" "Check nexthop group retained" nexthop show id "$group" + test_on "id $group group" + ts_ip "$0" "Add ordinary multipath route" -"$family" route add table 111 "$plain_prefix" nexthop via "$gw" dev "$DEV" weight 1 nexthop via "$gw2" dev "$DEV" weight 2 + ts_ip "$0" "Flush ordinary multipath route" -"$family" route flush table 111 to "$plain_prefix" + ts_ip "$0" "Check ordinary route removed" -"$family" route show table 111 "$plain_prefix" + test_lines_count 0 + # Mix compacted and unchanged messages to check flush buffer offsets. + ts_ip "$0" "Add group to mixed batch" -"$family" route add table 111 "$group_prefix" nhid "$group" metric 30 + ts_ip "$0" "Add ordinary route to mixed batch" -"$family" route add table 111 "$plain_prefix" dev "$DEV" metric 40 + ts_ip "$0" "Flush mixed batch with original dump display" -"$family" -s -s route flush table 111 + test_on "nhid $nh2" + test_on "nhid $group" + ts_ip "$0" "Check selected table empty" -"$family" route show table 111 + test_lines_count 0 + ts_ip "$0" "Flush second table" -"$family" route flush table 112 +done + +# An IPv4 route with an IPv6 nexthop expands to RTA_VIA. +ts_ip "$0" "Add cross-family route" -4 route add table 111 203.0.113.0/24 nhid 201 +ts_ip "$0" "Flush cross-family route" -4 route flush table 111 +ts_ip "$0" "Check cross-family route removed" -4 route show table 111 +test_lines_count 0 + +ts_ip "$0" "Create blackhole nexthop" -4 nexthop add id 301 blackhole +for type in unicast blackhole unreachable prohibit; do + ts_ip "$0" "Add $type alias" -4 route add "$type" 192.0.2.0/24 table 111 nhid 301 + ts_ip "$0" "Flush alias without type constraint" -4 route flush table 111 + ts_ip "$0" "Check alias removed" -4 route show table 111 + test_lines_count 0 +done + +VRF=nhflushvrf +ts_ip "$0" "Create VRF" link add "$VRF" type vrf table 111 +ts_ip "$0" "Bring VRF up" link set "$VRF" up +ts_ip "$0" "Add global unicast with blackhole nexthop" -4 route add 192.0.2.0/24 table 111 nhid 301 +ts_ip "$0" "Flush blackhole nexthop through VRF selector" -4 route flush vrf "$VRF" +ts_ip "$0" "Check VRF route removed" -4 route show table 111 +test_lines_count 0 + +# An explicit type must remain explicit on either side of the VRF argument. +for order in type-first vrf-first; do + if [ "$order" = type-first ]; then + set -- type unicast vrf "$VRF" + else + set -- vrf "$VRF" type unicast + fi + ts_ip "$0" "Add blackhole alias for explicit type ($order)" -4 route add blackhole 192.0.2.0/24 table 111 nhid 301 + ts_ip "$0" "Flush only explicit unicast ($order)" -4 route flush "$@" + ts_ip "$0" "Check blackhole alias retained" -4 route show table 111 + test_lines_count 1 + ts_ip "$0" "Delete blackhole alias" -4 route del blackhole 192.0.2.0/24 table 111 nhid 301 + + if [ "$order" = type-first ]; then + set -- type blackhole vrf "$VRF" + else + set -- vrf "$VRF" type blackhole + fi + ts_ip "$0" "Add hidden unicast for explicit type ($order)" -4 route add 192.0.2.0/24 table 111 nhid 301 + if "$IP" -4 route flush "$@" > "$STD_OUT" 2> "$STD_ERR"; then + ts_err "Explicit blackhole flush unexpectedly deleted a unicast alias" + fi + ts_ip "$0" "Check hidden unicast retained" -4 route show table 111 + test_lines_count 1 + ts_ip "$0" "Delete hidden unicast" -4 route del unicast 192.0.2.0/24 table 111 nhid 301 +done + +# The VRF mask excludes local/broadcast. Their synthesized type must not +# turn the wildcard workaround into permission to delete them. +for type in local broadcast; do + ts_ip "$0" "Add hidden $type alias" -4 route add "$type" 192.0.2.0/24 table 111 nhid 301 + if "$IP" -4 route flush vrf "$VRF" > "$STD_OUT" 2> "$STD_ERR"; then + ts_err "VRF flush unexpectedly succeeded on a hidden $type alias" + fi + ts_ip "$0" "Check hidden $type retained" -4 route show table 111 + test_lines_count 1 + ts_ip "$0" "Delete retained $type by its actual type" -4 route del "$type" 192.0.2.0/24 table 111 nhid 301 +done + +# At these scopes unicast cannot be distinguished from the excluded types. +for scope in link host; do + ts_ip "$0" "Add ambiguous $scope unicast" -4 route add 192.0.2.0/24 table 111 nhid 301 scope "$scope" + if "$IP" -4 route flush vrf "$VRF" > "$STD_OUT" 2> "$STD_ERR"; then + ts_err "VRF flush unexpectedly relaxed an ambiguous $scope type" + fi + ts_ip "$0" "Check ambiguous $scope unicast retained" -4 route show table 111 + test_lines_count 1 + ts_ip "$0" "Delete retained $scope unicast" -4 route del unicast 192.0.2.0/24 table 111 nhid 301 scope "$scope" +done + +# With expansion, stripping IPv6 attributes would enable deletion of a +# hidden local type. Keep the baseline failure until its type is known. +if [ "$(cat /proc/sys/net/ipv4/nexthop_compat_mode 2>/dev/null)" != 0 ]; then + ts_ip "$0" "Create IPv6 blackhole nexthop" -6 nexthop add id 302 blackhole + ts_ip "$0" "Add hidden IPv6 local route" -6 route add local 2001:db8:5::/64 table 111 nhid 302 + if "$IP" -6 route flush vrf "$VRF" > "$STD_OUT" 2> "$STD_ERR"; then + ts_err "VRF flush unexpectedly deleted the hidden IPv6 local route" + fi + ts_ip "$0" "Check hidden IPv6 local retained" -6 route show table 111 + test_lines_count 1 + ts_ip "$0" "Delete hidden IPv6 local by actual type" -6 route del local 2001:db8:5::/64 table 111 nhid 302 + ts_ip "$0" "Delete IPv6 blackhole nexthop" nexthop del id 302 +fi + +ts_ip "$0" "Check blackhole nexthop retained" nexthop show id 301 +test_on "id 301 blackhole" +ts_ip "$0" "Delete blackhole nexthop" nexthop del id 301 +ts_ip "$0" "Delete VRF" link del "$VRF" +ts_ip "$0" "Delete test device and its nexthops" link del "$DEV" base-commit: a73d93d00752fe07494c830c32c58b07fb850e84 -- 2.55.0 On Mon, 16 Nov 2026 15:16:18 -0800, Stephen Hemminger <stephen@networkplumber.org> wrote: > On Mon, 26 Oct 2026 18:24:20 +0800 > Yuen Su <zypulsecore@gmail.com> wrote: > > > Route dumps can contain RTA_NH_ID plus expanded nexthop attributes. > > The kernel rejects that combination on route deletion. Strip the > > expansion from the flush copy, keeping the ID and route selectors. > > Use the compacted length for batching and retain the dump for display. > > > > IPv4 fib_dump_info() also reports blackhole nexthop objects as > > RTN_BLACKHOLE instead of the alias type, independently of > > nexthop_compat_mode. Work around this with RTN_UNSPEC only when no > > type filter was requested. The kernel then deletes the first alias > > matching the remaining selectors; alias order and changes between > > dump and deletion can affect that choice. Preserve explicit types. > > > > Add namespace regressions covering ID-based flushes, ordinary routes, > > batching and type selection, with a nexthop capability probe. > > > > Signed-off-by: Yuen Su <zypulsecore@gmail.com> > > --- > > In general looked OK, but AI review with review-prompts showed > some issues. > > On Mon, 26 Oct 2026 18:24:20 +0800, Yuen Su wrote: > > The mechanism here is right, and I checked the premise against the > kernel: fib_frontend.c rtm_to_fib_config() rejects fc_nh_id combined > with fc_oif/fc_gw_family/fc_encap/fc_mp, and your strip list covers > exactly those four (RTA_OIF, RTA_GATEWAY/RTA_VIA, RTA_ENCAP/ > RTA_ENCAP_TYPE, RTA_MULTIPATH). fib_nh_match() returns 0 on an > ID-only request, so the compacted message matches. The AF_INET-only > restriction on the type workaround is also correct: ip6_route_del() > never compares fc_type against rt->fib6_type, so IPv6 needs no > equivalent even though rt6_fill_node() synthesizes RTN_BLACKHOLE the > same way. > > One case the workaround does not cover. > > > + /* Work around fib_dump_info() replacing the IPv4 alias type with > > + * RTN_BLACKHOLE for blackhole nexthop objects, independently of > > + * nexthop_compat_mode. Keep explicit type constraints. > > + */ > > + if (!filter.typemask && > > + r->rtm_family == AF_INET && r->rtm_type == RTN_BLACKHOLE) > > + dr->rtm_type = RTN_UNSPEC; > > filter.typemask is not only set by an explicit "type" argument. The > vrf keyword sets it too, in iproute_list_flush_or_save(): > > } else if (matches(*argv, "vrf") == 0) { > ... > filter.tb = tid; > filter.typemask = ~(1 << RTN_LOCAL | 1<<RTN_BROADCAST); > > That parser is shared between list, flush and save, so > > ip route flush vrf blue > > leaves typemask nonzero with no type requested by the user, the > workaround is skipped, and the delete request carries the synthesized > RTN_BLACKHOLE. For an IPv4 unicast route using a blackhole nexthop > object in that VRF table, fib_table_delete() requires an exact match: > > if ((!cfg->fc_type || fa->fa_type == cfg->fc_type) && > > fa_type is RTN_UNICAST, fc_type is RTN_BLACKHOLE, no alias matches, > and the delete returns ESRCH. iproute_flush() keeps redumping and > resending until the 30 second timeout, then reports entries remaining. > > To be clear about severity: this is not a regression. Those routes > fail to flush before your patch as well, for the same reason the > patch exists. It is a gap in the fix, and worth deciding on rather > than leaving implicit, because flush vrf is a common way to tear > down a VRF. > > The awkward part is that typemask cannot distinguish the two cases by > inspection - the vrf mask has the RTN_BLACKHOLE bit set, so you > cannot test for it. How would you prefer to handle it? A separate > flag recording that the user actually passed "type" would make the > guard say what the comment says, and would leave the vrf mask alone: > > } else if (matches(*argv, "type") == 0) { > ... > filter.typemask = (1<<type); > filter.type_explicit = 1; > > then test !filter.type_explicit in flush_nexthop_compat(). I have no > strong preference on the spelling; the point is that the current > guard is testing a proxy that has a second writer. > > Worth extending the new test to cover it, since it is the one flush > form where the workaround silently does not apply. > > Everything else I looked at checked out. The buffer arithmetic is > sound: the space check against the uncompacted n->nlmsg_len is > conservative because compaction only shrinks, and nlmsg_len stays > NLMSG-aligned since it starts at NLMSG_LENGTH(sizeof(*r)) and > advances by RTA_ALIGN, so the NLMSG_ALIGN(filter.flushp) arithmetic > for the next message in the batch still holds. Display still uses the > untouched dump. The testsuite helpers all exist in lib/generic.sh and > the build is clean. ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH iproute2 v3] ip: flush nexthop-object routes without compatibility attributes 2026-11-17 5:25 ` [PATCH iproute2 v3] " Yuen Su @ 2026-11-17 21:25 ` Stephen Hemminger 0 siblings, 0 replies; 4+ messages in thread From: Stephen Hemminger @ 2026-11-17 21:25 UTC (permalink / raw) To: Yuen Su; +Cc: netdev On Mon, 16 Nov 2026 21:25:37 -0800 Yuen Su <zypulsecore@gmail.com> wrote: > Route dumps can contain RTA_NH_ID plus expanded nexthop attributes. > The kernel rejects that combination on deletion. Strip the expansion > from the flush copy, preserving the ID and other route selectors. Use > the compacted length for batching and retain the original for display. > > Blackhole nexthops also hide the IPv4 alias type. Track explicit type > selectors separately from the VRF default. Without an explicit type, > clear the synthetic type for unfiltered requests and for VRF requests > with scope below LINK. Local and broadcast aliases cannot have that > scope, and deletion still matches it. Ambiguous link/host-scope aliases > retain the existing failure. > > Leave IPv6 blackhole dumps unchanged: they can hide a local type, and > deletion does not check the type. Stripping their expansion would allow > a VRF flush to delete a local route that previously survived an error. > > Extend the namespace tests with the VRF case, explicit types in either > argument order, excluded local/broadcast aliases and IPv6 retention. > > Signed-off-by: Yuen Su <zypulsecore@gmail.com> > --- The version you sent is corrupted by mail system but there is a bigger issue. After looking at this in more detail and at a higher level, the root cause of the problem is that kernel is buggy here, and working around it in ip command is the wrong way to handle it. Pre-nexthop the ip command could just do flush by getting all the routes with show and then turning around and feeding those back as delete operations. Somewhere along the way, this property was lost. That is the root cause here. I am not taking this kind of workaround in ip, fix the kernel. Yes it maybe really hard to unwind broken route API. Long winded AI prompted analysis below: On Mon, 16 Nov 2026 21:25:37 -0800, Yuen Su wrote: > Subject: [PATCH iproute2 v3] ip: flush nexthop-object routes without > compatibility attributes > > Route dumps can contain RTA_NH_ID plus expanded nexthop attributes. > The kernel rejects that combination on deletion. Right, and that is the bug -- in the kernel, not in ip. Flush has always worked by taking the dump message, flipping nlmsg_type to RTM_DELROUTE and sending it back. That round-trip is the design, which is why print_route() does a bare memcpy and nothing else. A dump the kernel will not accept back is a kernel bug, and I would rather see it fixed there than worked around here. Both halves of this are in fib_dump_info(): if (fi->nh) { if (nla_put_u32(skb, RTA_NH_ID, fi->nh->id)) goto nla_put_failure; if (nexthop_is_blackhole(fi->nh)) rtm->rtm_type = RTN_BLACKHOLE; if (!READ_ONCE(...sysctl_nexthop_compat_mode)) goto offload; } 1) The compat expansion is emitted alongside RTA_NH_ID, and rtm_to_fib_config() rejects exactly that pair: if (cfg->fc_nh_id) { if (cfg->fc_oif || cfg->fc_gw_family || cfg->fc_encap || cfg->fc_mp) { NL_SET_ERR_MSG(extack, "Nexthop specification and nexthop id are mutually exclusive"); So the dump violates the kernel's own input validation, and nexthop_compat_mode defaults to 1, so this is the default path. 2) rtm_type is overwritten with a synthetic RTN_BLACKHOLE. The route's real type is destroyed on the way out. The second one is why I do not think the type-guessing half of this patch can be made to work. It is trying to recover a field that is not on the wire. That is also what the changelog is admitting to: > Ambiguous link/host-scope aliases retain the existing failure. and the IPv4-only/IPv6-excluded asymmetry falls out of the same thing -- the clobber is in the IPv4 path only, which is why ip6_route_del() never needing fc_type is not a coincidence. The clobber came in with 4c7e8084fd46 ("ipv4: Plumb support for nexthop object in a fib_info") David Ahern, Jun 2019, v5.3 which added RTA_NH_ID and the rtm_type assignment together. The sysctl arrived later in 4f80116d3df3 ("net: ipv4: add sysctl for nexthop api compatibility mode") Roopa Prabhu, Apr 2020, v5.12 defaulting to 1, so the unacceptable-dump combination became the normal case rather than an opt-in. Either kernel fix makes this patch unnecessary. Having rtm_to_fib_config() ignore the expansion when fc_nh_id is set looks safest to me: RTA_NH_ID is the authoritative specification anyway, it keeps whatever display value compat mode has, and it makes every dump already in the field round-trippable -- including from older ip binaries, which this patch cannot help. Dropping or relocating the rtm_type assignment handles the second half. dsahern is on Cc. On the iproute2 side, if you want to carry something for kernels already deployed, please split it and send only the stripping: case RTA_OIF: case RTA_GATEWAY: case RTA_MULTIPATH: case RTA_VIA: case RTA_ENCAP_TYPE: case RTA_ENCAP: That set matches the fc_oif/fc_gw_family/fc_encap/fc_mp check exactly, drops nothing else, and needs no argument about types. It is a self-contained fix for the subject line. I checked the mechanics and they are sound: flush_nexthop_compat() reads the original n and writes the already-copied fn so there is no overlap, and the buffer bound still tests the original n->nlmsg_len, which is >= the compacted length. Drop type_explicit, the RTN_UNSPEC relaxation and the scope threshold. That machinery only exists to paper over the rtm_type clobber, it is the only part of the patch that is hard to review, it needs fib_props[] and fib_table_delete() to justify itself, and it still leaves documented failures. Most of the 176-line test is testing that logic rather than the fix. Separately: this v3 does not apply. The mail body was reflowed, hunk headers included -- > @@ -766,6 +767,39 @@ static void print_rta_multipath(FILE *fp, const > struct rtmsg *r, and the last 27 lines of the new test are missing from the diff; they appear earlier in the message, between the diffstat and base-commit. The hunk claims 176 added lines and only 149 are where the diff expects them, so un-wrapping alone will not recover it. The Received header shows gmailapi.google.com with HTTPREST, which is the usual cause. git send-email avoids it. I repaired the reflow locally to review the code; the C hunks apply cleanly once unwrapped and build warning-free. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-17 21:38 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-10-26 10:24 [PATCH iproute2 v2] ip: flush nexthop-object routes without compatibility attributes Yuen Su 2026-11-16 23:16 ` Stephen Hemminger 2026-11-17 5:25 ` [PATCH iproute2 v3] " Yuen Su 2026-11-17 21:25 ` Stephen Hemminger
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox