[PATCH net] ipv4: ip_tunnel: Fix suspicious RCU usage warning in ip_tunnel_find()

[PATCH net] ipv4: ip_tunnel: Fix suspicious RCU usage warning in ip_tunnel_find()

[Ido Schimmel]

The per-netns IP tunnel hash table is protected by the RTNL mutex and
ip_tunnel_find() is only called from the control path where the mutex is
taken.

Convert hlist_for_each_entry_rcu() in ip_tunnel_find() to
hlist_for_each_entry() to avoid the suspicious RCU usage warning [1] and
add an assertion to make sure the RTNL mutex is held when the function
is called.

[1]
WARNING: suspicious RCU usage
6.12.0-rc3-custom-gd95d9a31aceb #139 Not tainted
-----------------------------
net/ipv4/ip_tunnel.c:221 RCU-list traversed in non-reader section!!

other info that might help us debug this:

rcu_scheduler_active = 2, debug_locks = 1
1 lock held by ip/362:
 #0: ffffffff86fc7cb0 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x377/0xf60

stack backtrace:
CPU: 12 UID: 0 PID: 362 Comm: ip Not tainted 6.12.0-rc3-custom-gd95d9a31aceb #139
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
Call Trace:
 <TASK>
 dump_stack_lvl+0xba/0x110
 lockdep_rcu_suspicious.cold+0x4f/0xd6
 ip_tunnel_find+0x435/0x4d0
 ip_tunnel_newlink+0x517/0x7a0
 ipgre_newlink+0x14c/0x170
 __rtnl_newlink+0x1173/0x19c0
 rtnl_newlink+0x6c/0xa0
 rtnetlink_rcv_msg+0x3cc/0xf60
 netlink_rcv_skb+0x171/0x450
 netlink_unicast+0x539/0x7f0
 netlink_sendmsg+0x8c1/0xd80
 ____sys_sendmsg+0x8f9/0xc20
 ___sys_sendmsg+0x197/0x1e0
 __sys_sendmsg+0x122/0x1f0
 do_syscall_64+0xbb/0x1d0
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

Fixes: c54419321455 ("GRE: Refactor GRE tunneling code.")
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
---
 net/ipv4/ip_tunnel.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/ip_tunnel.c b/net/ipv4/ip_tunnel.c
index d591c73e2c0e..a93c402f573e 100644
--- a/net/ipv4/ip_tunnel.c
+++ b/net/ipv4/ip_tunnel.c
@@ -218,7 +218,9 @@ static struct ip_tunnel *ip_tunnel_find(struct ip_tunnel_net *itn,
 
 	ip_tunnel_flags_copy(flags, parms->i_flags);
 
-	hlist_for_each_entry_rcu(t, head, hash_node) {
+	ASSERT_RTNL();
+
+	hlist_for_each_entry(t, head, hash_node) {
 		if (local == t->parms.iph.saddr &&
 		    remote == t->parms.iph.daddr &&
 		    link == READ_ONCE(t->parms.link) &&
-- 
2.47.0

Re: [PATCH net] ipv4: ip_tunnel: Fix suspicious RCU usage warning in ip_tunnel_find()

[Eric Dumazet]

On Tue, Oct 22, 2024 at 9:10 AM Ido Schimmel <idosch@nvidia.com> wrote:
>
> The per-netns IP tunnel hash table is protected by the RTNL mutex and
> ip_tunnel_find() is only called from the control path where the mutex is
> taken.
>
> Convert hlist_for_each_entry_rcu() in ip_tunnel_find() to
> hlist_for_each_entry() to avoid the suspicious RCU usage warning [1] and
> add an assertion to make sure the RTNL mutex is held when the function
> is called.
>
> [1]
> WARNING: suspicious RCU usage
> 6.12.0-rc3-custom-gd95d9a31aceb #139 Not tainted
> -----------------------------
> net/ipv4/ip_tunnel.c:221 RCU-list traversed in non-reader section!!
>
> other info that might help us debug this:
>
> rcu_scheduler_active = 2, debug_locks = 1
> 1 lock held by ip/362:
>  #0: ffffffff86fc7cb0 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x377/0xf60
>
> stack backtrace:
> CPU: 12 UID: 0 PID: 362 Comm: ip Not tainted 6.12.0-rc3-custom-gd95d9a31aceb #139
> Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> Call Trace:
>  <TASK>
>  dump_stack_lvl+0xba/0x110
>  lockdep_rcu_suspicious.cold+0x4f/0xd6
>  ip_tunnel_find+0x435/0x4d0
>  ip_tunnel_newlink+0x517/0x7a0
>  ipgre_newlink+0x14c/0x170
>  __rtnl_newlink+0x1173/0x19c0
>  rtnl_newlink+0x6c/0xa0
>  rtnetlink_rcv_msg+0x3cc/0xf60
>  netlink_rcv_skb+0x171/0x450
>  netlink_unicast+0x539/0x7f0
>  netlink_sendmsg+0x8c1/0xd80
>  ____sys_sendmsg+0x8f9/0xc20
>  ___sys_sendmsg+0x197/0x1e0
>  __sys_sendmsg+0x122/0x1f0
>  do_syscall_64+0xbb/0x1d0
>  entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> Fixes: c54419321455 ("GRE: Refactor GRE tunneling code.")
> Signed-off-by: Ido Schimmel <idosch@nvidia.com>
> ---
>  net/ipv4/ip_tunnel.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/net/ipv4/ip_tunnel.c b/net/ipv4/ip_tunnel.c
> index d591c73e2c0e..a93c402f573e 100644
> --- a/net/ipv4/ip_tunnel.c
> +++ b/net/ipv4/ip_tunnel.c
> @@ -218,7 +218,9 @@ static struct ip_tunnel *ip_tunnel_find(struct ip_tunnel_net *itn,
>
>         ip_tunnel_flags_copy(flags, parms->i_flags);
>
> -       hlist_for_each_entry_rcu(t, head, hash_node) {
> +       ASSERT_RTNL();
> +
> +       hlist_for_each_entry(t, head, hash_node) {
>                 if (local == t->parms.iph.saddr &&
>                     remote == t->parms.iph.daddr &&
>                     link == READ_ONCE(t->parms.link) &&
> --
> 2.47.0
>

I was looking at this recently, and my thinking is the following :

1) ASSERT_RTNL() is adding code even on non debug kernels.

2) It does not check if the current thread is owning the RTNL mutex,
only that _some_ thread is owning it.

I would think that using lockdep_rtnl_is_held() would be better ?

diff --git a/net/ipv4/ip_tunnel.c b/net/ipv4/ip_tunnel.c
index d591c73e2c0e53efefb8fb9262610cbbd1dd71ea..25505f9b724c33d2c3ec2fca5355d7fdd4e01c14
100644
--- a/net/ipv4/ip_tunnel.c
+++ b/net/ipv4/ip_tunnel.c
@@ -218,7 +218,7 @@ static struct ip_tunnel *ip_tunnel_find(struct
ip_tunnel_net *itn,

        ip_tunnel_flags_copy(flags, parms->i_flags);

-       hlist_for_each_entry_rcu(t, head, hash_node) {
+       hlist_for_each_entry_rcu(t, head, hash_node, lockdep_rtnl_is_held()) {
                if (local == t->parms.iph.saddr &&
                    remote == t->parms.iph.daddr &&
                    link == READ_ONCE(t->parms.link) &&

Re: [PATCH net] ipv4: ip_tunnel: Fix suspicious RCU usage warning in ip_tunnel_find()

[Ido Schimmel]

On Tue, Oct 22, 2024 at 09:26:11AM +0200, Eric Dumazet wrote:
> I was looking at this recently, and my thinking is the following :
> 
> 1) ASSERT_RTNL() is adding code even on non debug kernels.
> 
> 2) It does not check if the current thread is owning the RTNL mutex,
> only that _some_ thread is owning it.
> 
> I would think that using lockdep_rtnl_is_held() would be better ?

Yes, agree. I see I did the same thing in 7f6f32bb7d335. Will post v2
tomorrow unless you prefer to submit it yourself (I don't mind).

Thanks

> 
> diff --git a/net/ipv4/ip_tunnel.c b/net/ipv4/ip_tunnel.c
> index d591c73e2c0e53efefb8fb9262610cbbd1dd71ea..25505f9b724c33d2c3ec2fca5355d7fdd4e01c14
> 100644
> --- a/net/ipv4/ip_tunnel.c
> +++ b/net/ipv4/ip_tunnel.c
> @@ -218,7 +218,7 @@ static struct ip_tunnel *ip_tunnel_find(struct
> ip_tunnel_net *itn,
> 
>         ip_tunnel_flags_copy(flags, parms->i_flags);
> 
> -       hlist_for_each_entry_rcu(t, head, hash_node) {
> +       hlist_for_each_entry_rcu(t, head, hash_node, lockdep_rtnl_is_held()) {
>                 if (local == t->parms.iph.saddr &&
>                     remote == t->parms.iph.daddr &&
>                     link == READ_ONCE(t->parms.link) &&

Re: [PATCH net] ipv4: ip_tunnel: Fix suspicious RCU usage warning in ip_tunnel_find()

[Eric Dumazet]

On Tue, Oct 22, 2024 at 10:55 AM Ido Schimmel <idosch@nvidia.com> wrote:
>
> On Tue, Oct 22, 2024 at 09:26:11AM +0200, Eric Dumazet wrote:
> > I was looking at this recently, and my thinking is the following :
> >
> > 1) ASSERT_RTNL() is adding code even on non debug kernels.
> >
> > 2) It does not check if the current thread is owning the RTNL mutex,
> > only that _some_ thread is owning it.
> >
> > I would think that using lockdep_rtnl_is_held() would be better ?
>
> Yes, agree. I see I did the same thing in 7f6f32bb7d335. Will post v2
> tomorrow unless you prefer to submit it yourself (I don't mind).

Please send your v2, thanks Ido !