[PATCH net v2 0/2] netlink: fix a dump livelock with a notification queued

[Jhonata Poma-Hansen <jhonata.poma@gmail.com>]

A netlink dump that starts while a notification is still charged to a
socket with a small receive buffer fails with ENOBUFS, and every retry
from recvmsg() fails the same way. A reader that peeks before reading,
as libnl does, never consumes the message at the head of the queue and
spins forever. I hit this with "iw dev wlan0 scan" on an arm64 board.

Patch 1 goes back to checking sk_rcvbuf before charging the dump skb,
as before commit ae8f160e7eb2. Patch 2 adds a selftest; it fails
without patch 1 and is skipped without the privileges to create a
netns.

---
Changes in v2:
- Patch 1: check sk_rcvbuf before charging instead of retrying with
  the minimum skb size, which could still be rejected (Sashiko), and
  refuse a skb that would take sk_rmem_alloc past INT_MAX.
- Patch 1: say how the bug was found and how the fix was tested.
- Patch 2: use SO_RCVBUF 2048, which also fails with v1, check the
  NLMSG_DONE status and that a link was dumped, and skip without
  privileges (Sashiko).
- Link to v1: https://patch.msgid.link/20261023-netlink-dump-livelock-v1-0-478214a635d2@gmail.com

To: "David S. Miller" <davem@davemloft.net>
To: Eric Dumazet <edumazet@kernel.org>
To: Jakub Kicinski <kuba@kernel.org>
To: Paolo Abeni <pabeni@redhat.com>
To: Simon Horman <horms@kernel.org>
To: Kuniyuki Iwashima <kuniyu@google.com>
To: Shuah Khan <shuah@kernel.org>
Cc: netdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org

---
Jhonata Poma-Hansen (2):
      netlink: check sk_rcvbuf before charging the dump skb
      selftests: net: netlink-dumps: start a dump with a notification queued

 net/netlink/af_netlink.c                    |   6 +-
 tools/testing/selftests/net/netlink-dumps.c | 102 ++++++++++++++++++++++++++++
 2 files changed, 107 insertions(+), 1 deletion(-)
---
base-commit: 734bef548a96e8774d6eef5dfa2026c6edc4e80b
change-id: 20261023-netlink-dump-livelock-e83cf13f977a

Best regards,
--  
Jhonata Poma-Hansen <jhonata.poma@gmail.com>