Netfilter: suspicious RCU usage in __nft_rule_lookup

[Matthieu Baerts <matttbe@kernel.org>]

Hello,

First, thank you for all the work you did and are still doing around
Netfilter!

I'm writing you this email, because when I run the MPTCP test suite with
a VM running a kernel built with a debug config including
CONFIG_PROVE_RCU_LIST=y (and CONFIG_RCU_EXPERT=y), I get the following
warning:


> =============================
> WARNING: suspicious RCU usage
> 6.12.0-rc3+ #7 Not tainted
> -----------------------------
> net/netfilter/nf_tables_api.c:3420 RCU-list traversed in non-reader section!!
> 
> other info that might help us debug this:
> 
> 
> rcu_scheduler_active = 2, debug_locks = 1
> 1 lock held by iptables/134:
>   #0: ffff888008c4fcc8 (&nft_net->commit_mutex){+.+.}-{3:3}, at: nf_tables_valid_genid (include/linux/jiffies.h:101) nf_tables
> 
> stack backtrace:
> CPU: 1 UID: 0 PID: 134 Comm: iptables Not tainted 6.12.0-rc3+ #7
> Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> Call Trace:
>  <TASK>
>  dump_stack_lvl (lib/dump_stack.c:123)
>  lockdep_rcu_suspicious (kernel/locking/lockdep.c:6822)
>  __nft_rule_lookup (net/netfilter/nf_tables_api.c:3420 (discriminator 7)) nf_tables
>  nf_tables_delrule (net/netfilter/nf_tables_api.c:4300 (discriminator 1)) nf_tables
>  ? __pfx_nf_tables_delrule (net/netfilter/nf_tables_api.c:4262) nf_tables
>  ? __mutex_unlock_slowpath (arch/x86/include/asm/atomic64_64.h:101 (discriminator 1))
>  ? __nla_validate_parse (lib/nlattr.c:638)
>  nfnetlink_rcv_batch (net/netfilter/nfnetlink.c:524)
>  ? __pfx_nfnetlink_rcv_batch (net/netfilter/nfnetlink.c:373)
>  ? rcu_read_lock_any_held (kernel/rcu/update.c:386 (discriminator 1))
>  ? validate_chain (kernel/locking/lockdep.c:3797 (discriminator 1))
>  ? rcu_read_lock_any_held (kernel/rcu/update.c:386 (discriminator 1))
>  ? validate_chain (kernel/locking/lockdep.c:3797 (discriminator 1))
>  ? __pfx_validate_chain (kernel/locking/lockdep.c:3860)
>  ? __nla_validate_parse (lib/nlattr.c:638)
>  nfnetlink_rcv (net/netfilter/nfnetlink.c:647)
>  ? __pfx___netlink_lookup (net/netlink/af_netlink.c:512)
>  ? __pfx_nfnetlink_rcv (net/netfilter/nfnetlink.c:651)
>  netlink_unicast (net/netlink/af_netlink.c:1331)
>  ? __pfx_netlink_unicast (net/netlink/af_netlink.c:1342)
>  ? find_held_lock (kernel/locking/lockdep.c:5315 (discriminator 1))
>  ? __might_fault (mm/memory.c:6700 (discriminator 5))
>  netlink_sendmsg (net/netlink/af_netlink.c:1901)
>  ? __pfx_netlink_sendmsg (net/netlink/af_netlink.c:1820)
>  ? __import_iovec (lib/iov_iter.c:1433 (discriminator 1))
>  ____sys_sendmsg (net/socket.c:729 (discriminator 1))
>  ? __pfx_____sys_sendmsg (net/socket.c:2553)
>  ? __pfx_copy_msghdr_from_user (net/socket.c:2533)
>  ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4347)
>  ? sk_setsockopt (net/core/sock.c:1129)
>  ? __local_bh_enable_ip (arch/x86/include/asm/irqflags.h:42)
>  ___sys_sendmsg (net/socket.c:2663)
>  ? __pfx_sk_setsockopt (net/core/sock.c:1163)
>  ? __pfx____sys_sendmsg (net/socket.c:2650)
>  ? mark_lock (kernel/locking/lockdep.c:4703 (discriminator 1))
>  ? fdget (include/linux/atomic/atomic-arch-fallback.h:479 (discriminator 2))
>  ? find_held_lock (kernel/locking/lockdep.c:5315 (discriminator 1))
>  __sys_sendmsg (net/socket.c:2690 (discriminator 1))
>  ? __pfx___sys_sendmsg (net/socket.c:2678)
>  ? __sys_setsockopt (include/linux/file.h:35)
>  do_syscall_64 (arch/x86/entry/common.c:52 (discriminator 1))
>  entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
> RIP: 0033:0x7ff6a026e004


It is very easy for me to reproduce it: simply by adding and removing an
IPTables rule. Just in case, here are the steps that can be used to have
the same behaviour:

  $ cd [kernel source code]
  $ echo "iptables -A OUTPUT -j REJECT; iptables -D OUTPUT -j REJECT" \
        > .virtme-exec-run
  $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug -e RCU_EXPERT -e PROVE_RCU_LIST


I looked a bit at the code in net/netfilter/nf_tables_api.c, and I can
see that rcu_read_(un)lock() are probably missing, but I'm a bit
confused by how the chain->rules list is protected, and modified using
or not the RCU helpers.

Do you mind looking at this issue please?
(No urgency on my side.)

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.