Maybe a bug with adding default routes?

Maybe a bug with adding default routes?

[Robert Landers]

Hello netdev,

I believe I either found a bug, or I'm doing something wrong (probably
the latter, or both!). I was experimenting with getting a "floating
IP" for my home lab, and eventually, I got it to work, but it requires
some voodoo, which intrigued me and I think I found some strange
behavior that smells like a bug. I'm on Ubuntu 22.04 (and Pop OS! on
my desktop), so it is also possible that this is fixed upstream (in
which case, I'll email that list next).

To reproduce is quite simple:

echo "1234 test" >> /etc/iproute2/rt_tables
ip route add default via 167.235.212.73 dev enp9s0 table test

This will fail with the error:

Error: Nexthop has invalid gateway.

Now, I think this makes sense, however, the routing table shouldn't
need to know about hops, Right? Maybe I'm wrong, but this voodoo
results in a correct routing table:

ip addr add 167.235.212.72/29 dev enp9s0
ip route add default via 167.235.212.73 dev enp9s0 table test
ip addr del 167.235.212.72/29 dev enp9s0

I'm not sure if this is a bug or working as designed. It smells like a
bug, but I could just as easily be doing something wrong. I grew up in
"simpler" times and am not nearly as familiar with iproute2 as I was
with the old stuff.

PS. I've removed other commands (like rules, etc) to reduce the
example down to it's most basic reproduction.

Robert Landers
Software Engineer
Utrecht NL

Re: Maybe a bug with adding default routes?

[Toke Høiland-Jørgensen]

Robert Landers <landers.robert@gmail.com> writes:

> Hello netdev,
>
> I believe I either found a bug, or I'm doing something wrong (probably
> the latter, or both!). I was experimenting with getting a "floating
> IP" for my home lab, and eventually, I got it to work, but it requires
> some voodoo, which intrigued me and I think I found some strange
> behavior that smells like a bug. I'm on Ubuntu 22.04 (and Pop OS! on
> my desktop), so it is also possible that this is fixed upstream (in
> which case, I'll email that list next).
>
> To reproduce is quite simple:
>
> echo "1234 test" >> /etc/iproute2/rt_tables
> ip route add default via 167.235.212.73 dev enp9s0 table test
>
> This will fail with the error:
>
> Error: Nexthop has invalid gateway.
>
> Now, I think this makes sense, however, the routing table shouldn't
> need to know about hops, Right? Maybe I'm wrong, but this voodoo
> results in a correct routing table:
>
> ip addr add 167.235.212.72/29 dev enp9s0
> ip route add default via 167.235.212.73 dev enp9s0 table test
> ip addr del 167.235.212.72/29 dev enp9s0
>
> I'm not sure if this is a bug or working as designed. It smells like a
> bug, but I could just as easily be doing something wrong. I grew up in
> "simpler" times and am not nearly as familiar with iproute2 as I was
> with the old stuff.

Try the 'onlink' flag:

ip route add default via 167.235.212.73 dev enp9s0 onlink table test

-Toke

Re: Maybe a bug with adding default routes?

[Robert Landers]

On Wed, Apr 19, 2023 at 11:51 AM Toke Høiland-Jørgensen <toke@kernel.org> wrote:
>
> Robert Landers <landers.robert@gmail.com> writes:
>
> > Hello netdev,
> >
> > I believe I either found a bug, or I'm doing something wrong (probably
> > the latter, or both!). I was experimenting with getting a "floating
> > IP" for my home lab, and eventually, I got it to work, but it requires
> > some voodoo, which intrigued me and I think I found some strange
> > behavior that smells like a bug. I'm on Ubuntu 22.04 (and Pop OS! on
> > my desktop), so it is also possible that this is fixed upstream (in
> > which case, I'll email that list next).
> >
> > To reproduce is quite simple:
> >
> > echo "1234 test" >> /etc/iproute2/rt_tables
> > ip route add default via 167.235.212.73 dev enp9s0 table test
> >
> > This will fail with the error:
> >
> > Error: Nexthop has invalid gateway.
> >
> > Now, I think this makes sense, however, the routing table shouldn't
> > need to know about hops, Right? Maybe I'm wrong, but this voodoo
> > results in a correct routing table:
> >
> > ip addr add 167.235.212.72/29 dev enp9s0
> > ip route add default via 167.235.212.73 dev enp9s0 table test
> > ip addr del 167.235.212.72/29 dev enp9s0
> >
> > I'm not sure if this is a bug or working as designed. It smells like a
> > bug, but I could just as easily be doing something wrong. I grew up in
> > "simpler" times and am not nearly as familiar with iproute2 as I was
> > with the old stuff.
>
> Try the 'onlink' flag:
>
> ip route add default via 167.235.212.73 dev enp9s0 onlink table test
>
> -Toke

Hello Toke,

I tried onlink but it did not work; packets appeared to never leave the device.

Re: Maybe a bug with adding default routes?

[Toke Høiland-Jørgensen]

Robert Landers <landers.robert@gmail.com> writes:

> On Wed, Apr 19, 2023 at 11:51 AM Toke Høiland-Jørgensen <toke@kernel.org> wrote:
>>
>> Robert Landers <landers.robert@gmail.com> writes:
>>
>> > Hello netdev,
>> >
>> > I believe I either found a bug, or I'm doing something wrong (probably
>> > the latter, or both!). I was experimenting with getting a "floating
>> > IP" for my home lab, and eventually, I got it to work, but it requires
>> > some voodoo, which intrigued me and I think I found some strange
>> > behavior that smells like a bug. I'm on Ubuntu 22.04 (and Pop OS! on
>> > my desktop), so it is also possible that this is fixed upstream (in
>> > which case, I'll email that list next).
>> >
>> > To reproduce is quite simple:
>> >
>> > echo "1234 test" >> /etc/iproute2/rt_tables
>> > ip route add default via 167.235.212.73 dev enp9s0 table test
>> >
>> > This will fail with the error:
>> >
>> > Error: Nexthop has invalid gateway.
>> >
>> > Now, I think this makes sense, however, the routing table shouldn't
>> > need to know about hops, Right? Maybe I'm wrong, but this voodoo
>> > results in a correct routing table:
>> >
>> > ip addr add 167.235.212.72/29 dev enp9s0
>> > ip route add default via 167.235.212.73 dev enp9s0 table test
>> > ip addr del 167.235.212.72/29 dev enp9s0
>> >
>> > I'm not sure if this is a bug or working as designed. It smells like a
>> > bug, but I could just as easily be doing something wrong. I grew up in
>> > "simpler" times and am not nearly as familiar with iproute2 as I was
>> > with the old stuff.
>>
>> Try the 'onlink' flag:
>>
>> ip route add default via 167.235.212.73 dev enp9s0 onlink table test
>>
>> -Toke
>
> Hello Toke,
>
> I tried onlink but it did not work; packets appeared to never leave the device.

Well, *is* the device on the link? When you tell the kernel that the
gateway device is 'onlink' on a device it means that the kernel will
send out neighbour advertisements to try to find it, so it has to be
answering those. You should be able to see ARP packets going out if you
run tcpdump on the interface...

-Toke