[BUG] Dependence of routing cache entries on the ignore-df flag

[BUG] Dependence of routing cache entries on the ignore-df flag

[Aleksey Shumnik]

Dear maintainers,

I found such a dependency, if the ignore-df flag is set on the
interface, then entries appear in the routing cache.
You can reproduce it as follows:

# cat /etc/network/interfaces.d/mgre0
auto mgre0
iface mgre0 inet static
address 10.10.10.1
netmask 255.255.255.0
pre-up ip tunnel add mgre0 mode gre key 1 ttl 64 tos inherit
pre-up ethtool -K mgre0 tx off > /dev/null
pre-up ip link set mgre0 mtu 1400
pre-up ip link set mgre0 multicast on
pre-up ip link set mgre0 type gre nopmtudisc
pre-up ip link set mgre0 type gre ignore-df
post-down ip link del mgre0

# ifup mgre0
# ip link set mgre0 arp on
# ip neigh add 10.10.10.2 dev mgre0 lladdr <some ipv4 addr>
# ping 10.10.10.2
# ip route show cache
10.10.10.2 dev mgre0
    cache expires 598sec mtu 1400

Creating mgre interface using the following config. Set arp on (this
is necessary).
Add an entry to the neighbor table for arp routing, specify the
address from the same subnet as the address on the mgre interface.
Ping this address. After that, an entry is added to the routing cache.
It seems that after it was not possible to find a suitable entry in
the routing table for the address 10.10.10.2, before looking at the
arp table, an entry is added to the routing cache.

But the most incomprehensible thing for me is if you remove the
ignore-df flag, restart (ifdown, ifup) the interface, then the problem
is not observed, that is, entries are not added to the routing cache.
The answer to the question of how the ignore-df flag  and adding
entries to the routing cache is connected, I have not found.

On the one hand, the routing cache has not been supported in linux for
a long time, and there should not be such entries. On the other hand,
when some services are started, they request the routing table via
netlink and the kernel responds by sending not only the entries of the
main routing table, but also the routing cache.
As a result, incorrect routing entries with 0.0.0.0 nexthop address
appear in the peer list of these services.
Of course, it's not difficult to fix it in the services themselves,
but it seems to me that this will be a correction of the consequences
of the problem, and not the cause itself. It seems to me that this is
the wrong behavior.

Might you answer the questions:
1. How the ignore-df flag and adding entries to the routing cache is
connected? In which kernel files may I look to find this connection?
2. Is this behavior wrong?
3. Is there any way to completely disable the use of the routing
cache? (as far as I understand, it used to be possible to set the
rhash_entries parameter to 0, but now there is no such parameter)
4. Why is an entry added to the routing cache if a suitable entry was
eventually found in the arp table (it is added directly, without being
temporarily added to the routing table)?


Regards,
Aleksey

Re: [BUG] Dependence of routing cache entries on the ignore-df flag

[Stephen Hemminger]

On Wed, 3 May 2023 18:01:03 +0300
Aleksey Shumnik <ashumnik9@gmail.com> wrote:

> Might you answer the questions:
> 1. How the ignore-df flag and adding entries to the routing cache is
> connected? In which kernel files may I look to find this connection?
> 2. Is this behavior wrong?
> 3. Is there any way to completely disable the use of the routing
> cache? (as far as I understand, it used to be possible to set the
> rhash_entries parameter to 0, but now there is no such parameter)
> 4. Why is an entry added to the routing cache if a suitable entry was
> eventually found in the arp table (it is added directly, without being
> temporarily added to the routing table)?

What kernel version. The route cache has been completely removed from
the kernel for a long time.

Re: [BUG] Dependence of routing cache entries on the ignore-df flag

[Aleksey Shumnik]

On Wed, May 3, 2023 at 9:35 PM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Wed, 3 May 2023 18:01:03 +0300
> Aleksey Shumnik <ashumnik9@gmail.com> wrote:
>
> > Might you answer the questions:
> > 1. How the ignore-df flag and adding entries to the routing cache is
> > connected? In which kernel files may I look to find this connection?
> > 2. Is this behavior wrong?
> > 3. Is there any way to completely disable the use of the routing
> > cache? (as far as I understand, it used to be possible to set the
> > rhash_entries parameter to 0, but now there is no such parameter)
> > 4. Why is an entry added to the routing cache if a suitable entry was
> > eventually found in the arp table (it is added directly, without being
> > temporarily added to the routing table)?
>
> What kernel version.

Sorry, I didn't specify the kernel version.
I'm using kernel version 6.1.15

> The route cache has been completely removed from the kernel for a long time.

Yes, I even found a commit in which this was done
89aef8921bfbac22f00e04f8450f6e447db13e42.
This is my question, it is strange that the routing cache is still
used in version 6 of the kernel. In my first letter I wrote how to
reproduce such behavior, which seems to be erroneous.
I also ask you to pay attention to the fact that this error is
reproduced if the ignore-df flag is set on the interface.

Re: [BUG] Dependence of routing cache entries on the ignore-df flag

[David Ahern]

On Wed, May 03, 2023 at 11:35:28AM -0700, Stephen Hemminger wrote:
> On Wed, 3 May 2023 18:01:03 +0300
> Aleksey Shumnik <ashumnik9@gmail.com> wrote:
> 
> > Might you answer the questions:
> > 1. How the ignore-df flag and adding entries to the routing cache is
> > connected? In which kernel files may I look to find this connection?
> > 2. Is this behavior wrong?
> > 3. Is there any way to completely disable the use of the routing
> > cache? (as far as I understand, it used to be possible to set the
> > rhash_entries parameter to 0, but now there is no such parameter)
> > 4. Why is an entry added to the routing cache if a suitable entry was
> > eventually found in the arp table (it is added directly, without being
> > temporarily added to the routing table)?
> 
> What kernel version. The route cache has been completely removed from
> the kernel for a long time.

These are exceptions (fib_nh_exception), not the legacy routing cache.

Re: [BUG] Dependence of routing cache entries on the ignore-df flag

[Stephen Hemminger]

On Thu, 4 May 2023 21:10:43 -0600
David Ahern <dsahern@kernel.org> wrote:

> On Wed, May 03, 2023 at 11:35:28AM -0700, Stephen Hemminger wrote:
> > On Wed, 3 May 2023 18:01:03 +0300
> > Aleksey Shumnik <ashumnik9@gmail.com> wrote:
> >   
> > > Might you answer the questions:
> > > 1. How the ignore-df flag and adding entries to the routing cache is
> > > connected? In which kernel files may I look to find this connection?
> > > 2. Is this behavior wrong?
> > > 3. Is there any way to completely disable the use of the routing
> > > cache? (as far as I understand, it used to be possible to set the
> > > rhash_entries parameter to 0, but now there is no such parameter)
> > > 4. Why is an entry added to the routing cache if a suitable entry was
> > > eventually found in the arp table (it is added directly, without being
> > > temporarily added to the routing table)?  
> > 
> > What kernel version. The route cache has been completely removed from
> > the kernel for a long time.  
> 
> These are exceptions (fib_nh_exception), not the legacy routing cache.

I tried to reproduce your example, and did not see anything.
Could it be the multicast routing daemon (pimd) is watching and adding
the entry?

Re: [BUG] Dependence of routing cache entries on the ignore-df flag

[David Ahern]

On 5/5/23 9:00 AM, Stephen Hemminger wrote:
> On Thu, 4 May 2023 21:10:43 -0600
> David Ahern <dsahern@kernel.org> wrote:
> 
>> On Wed, May 03, 2023 at 11:35:28AM -0700, Stephen Hemminger wrote:
>>> On Wed, 3 May 2023 18:01:03 +0300
>>> Aleksey Shumnik <ashumnik9@gmail.com> wrote:
>>>   
>>>> Might you answer the questions:
>>>> 1. How the ignore-df flag and adding entries to the routing cache is
>>>> connected? In which kernel files may I look to find this connection?
>>>> 2. Is this behavior wrong?
>>>> 3. Is there any way to completely disable the use of the routing
>>>> cache? (as far as I understand, it used to be possible to set the
>>>> rhash_entries parameter to 0, but now there is no such parameter)
>>>> 4. Why is an entry added to the routing cache if a suitable entry was
>>>> eventually found in the arp table (it is added directly, without being
>>>> temporarily added to the routing table)?  
>>>
>>> What kernel version. The route cache has been completely removed from
>>> the kernel for a long time.  
>>
>> These are exceptions (fib_nh_exception), not the legacy routing cache.
> 
> I tried to reproduce your example, and did not see anything.
> Could it be the multicast routing daemon (pimd) is watching and adding
> the entry?

That looks like a pmtu based entry - added by the kernel processing an ICMP.