From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: stable@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
patches@lists.linux.dev, syzkaller <syzkaller@googlegroups.com>,
Kuniyuki Iwashima <kuniyu@amazon.com>,
Jozsef Kadlecsik <kadlec@netfilter.org>,
Pablo Neira Ayuso <pablo@netfilter.org>,
Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.4 12/45] netfilter: ipset: Add schedule point in call_ad().
Date: Mon, 12 Jun 2023 12:26:06 +0200 [thread overview]
Message-ID: <20230612101655.134991274@linuxfoundation.org> (raw)
In-Reply-To: <20230612101654.644983109@linuxfoundation.org>
From: Kuniyuki Iwashima <kuniyu@amazon.com>
[ Upstream commit 24e227896bbf003165e006732dccb3516f87f88e ]
syzkaller found a repro that causes Hung Task [0] with ipset. The repro
first creates an ipset and then tries to delete a large number of IPs
from the ipset concurrently:
IPSET_ATTR_IPADDR_IPV4 : 172.20.20.187
IPSET_ATTR_CIDR : 2
The first deleting thread hogs a CPU with nfnl_lock(NFNL_SUBSYS_IPSET)
held, and other threads wait for it to be released.
Previously, the same issue existed in set->variant->uadt() that could run
so long under ip_set_lock(set). Commit 5e29dc36bd5e ("netfilter: ipset:
Rework long task execution when adding/deleting entries") tried to fix it,
but the issue still exists in the caller with another mutex.
While adding/deleting many IPs, we should release the CPU periodically to
prevent someone from abusing ipset to hang the system.
Note we need to increment the ipset's refcnt to prevent the ipset from
being destroyed while rescheduling.
[0]:
INFO: task syz-executor174:268 blocked for more than 143 seconds.
Not tainted 6.4.0-rc1-00145-gba79e9a73284 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor174 state:D stack:0 pid:268 ppid:260 flags:0x0000000d
Call trace:
__switch_to+0x308/0x714 arch/arm64/kernel/process.c:556
context_switch kernel/sched/core.c:5343 [inline]
__schedule+0xd84/0x1648 kernel/sched/core.c:6669
schedule+0xf0/0x214 kernel/sched/core.c:6745
schedule_preempt_disabled+0x58/0xf0 kernel/sched/core.c:6804
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x6fc/0xdb0 kernel/locking/mutex.c:747
__mutex_lock_slowpath+0x14/0x20 kernel/locking/mutex.c:1035
mutex_lock+0x98/0xf0 kernel/locking/mutex.c:286
nfnl_lock net/netfilter/nfnetlink.c:98 [inline]
nfnetlink_rcv_msg+0x480/0x70c net/netfilter/nfnetlink.c:295
netlink_rcv_skb+0x1c0/0x350 net/netlink/af_netlink.c:2546
nfnetlink_rcv+0x18c/0x199c net/netfilter/nfnetlink.c:658
netlink_unicast_kernel net/netlink/af_netlink.c:1339 [inline]
netlink_unicast+0x664/0x8cc net/netlink/af_netlink.c:1365
netlink_sendmsg+0x6d0/0xa4c net/netlink/af_netlink.c:1913
sock_sendmsg_nosec net/socket.c:724 [inline]
sock_sendmsg net/socket.c:747 [inline]
____sys_sendmsg+0x4b8/0x810 net/socket.c:2503
___sys_sendmsg net/socket.c:2557 [inline]
__sys_sendmsg+0x1f8/0x2a4 net/socket.c:2586
__do_sys_sendmsg net/socket.c:2595 [inline]
__se_sys_sendmsg net/socket.c:2593 [inline]
__arm64_sys_sendmsg+0x80/0x94 net/socket.c:2593
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall+0x84/0x270 arch/arm64/kernel/syscall.c:52
el0_svc_common+0x134/0x24c arch/arm64/kernel/syscall.c:142
do_el0_svc+0x64/0x198 arch/arm64/kernel/syscall.c:193
el0_svc+0x2c/0x7c arch/arm64/kernel/entry-common.c:637
el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:591
Reported-by: syzkaller <syzkaller@googlegroups.com>
Fixes: a7b4f989a629 ("netfilter: ipset: IP set core support")
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Acked-by: Jozsef Kadlecsik <kadlec@netfilter.org>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
net/netfilter/ipset/ip_set_core.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/net/netfilter/ipset/ip_set_core.c b/net/netfilter/ipset/ip_set_core.c
index 16ae770f049dd..e3c14f8890a89 100644
--- a/net/netfilter/ipset/ip_set_core.c
+++ b/net/netfilter/ipset/ip_set_core.c
@@ -1539,6 +1539,14 @@ call_ad(struct sock *ctnl, struct sk_buff *skb, struct ip_set *set,
bool eexist = flags & IPSET_FLAG_EXIST, retried = false;
do {
+ if (retried) {
+ __ip_set_get(set);
+ nfnl_unlock(NFNL_SUBSYS_IPSET);
+ cond_resched();
+ nfnl_lock(NFNL_SUBSYS_IPSET);
+ __ip_set_put(set);
+ }
+
ip_set_lock(set);
ret = set->variant->uadt(set, tb, adt, &lineno, flags, retried);
ip_set_unlock(set);
--
2.39.2
next prev parent reply other threads:[~2023-06-12 10:48 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-12 10:25 [PATCH 5.4 00/45] 5.4.247-rc1 review Greg Kroah-Hartman
2023-06-12 10:25 ` [PATCH 5.4 01/45] blk-iocost: avoid 64-bit division in ioc_timer_fn Greg Kroah-Hartman
2023-06-12 10:25 ` [PATCH 5.4 02/45] block/blk-iocost (gcc13): keep large values in a new enum Greg Kroah-Hartman
2023-06-12 10:25 ` [PATCH 5.4 03/45] i40iw: fix build warning in i40iw_manage_apbvt() Greg Kroah-Hartman
2023-06-12 10:25 ` [PATCH 5.4 04/45] i40e: fix build warnings in i40e_alloc.h Greg Kroah-Hartman
2023-06-12 10:25 ` [PATCH 5.4 05/45] spi: qup: Request DMA before enabling clocks Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 06/45] neighbour: Replace zero-length array with flexible-array member Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 07/45] neighbour: fix unaligned access to pneigh_entry Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 08/45] net: dsa: lan9303: allow vid != 0 in port_fdb_{add|del} methods Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 09/45] Bluetooth: Fix l2cap_disconnect_req deadlock Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 10/45] Bluetooth: L2CAP: Add missing checks for invalid DCID Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 11/45] netfilter: conntrack: fix NULL pointer dereference in nf_confirm_cthelper Greg Kroah-Hartman
2023-06-12 10:26 ` Greg Kroah-Hartman [this message]
2023-06-12 10:26 ` [PATCH 5.4 13/45] rfs: annotate lockless accesses to sk->sk_rxhash Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 14/45] rfs: annotate lockless accesses to RFS sock flow table Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 15/45] net: sched: move rtm_tca_policy declaration to include file Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 16/45] net: sched: fix possible refcount leak in tc_chain_tmplt_add() Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 17/45] lib: cpu_rmap: Fix potential use-after-free in irq_cpu_rmap_release() Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 18/45] bnxt_en: Query default VLAN before VNIC setup on a VF Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 19/45] batman-adv: Broken sync while rescheduling delayed work Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 20/45] Input: xpad - delete a Razer DeathAdder mouse VID/PID entry Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 21/45] Input: psmouse - fix OOB access in Elantech protocol Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 22/45] ALSA: hda/realtek: Add a quirk for HP Slim Desktop S01 Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 23/45] ALSA: hda/realtek: Add Lenovo P3 Tower platform Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 24/45] drm/amdgpu: fix xclk freq on CHIP_STONEY Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 25/45] can: j1939: j1939_sk_send_loop_abort(): improved error queue handling in J1939 Socket Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 26/45] can: j1939: change j1939_netdev_lock type to mutex Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 27/45] can: j1939: avoid possible use-after-free when j1939_can_rx_register fails Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 28/45] ceph: fix use-after-free bug for inodes when flushing capsnaps Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 29/45] Bluetooth: Fix use-after-free in hci_remove_ltk/hci_remove_irk Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 30/45] rbd: move RBD_OBJ_FLAG_COPYUP_ENABLED flag setting Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 31/45] pinctrl: meson-axg: add missing GPIOA_18 gpio group Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 32/45] usb: usbfs: Enforce page requirements for mmap Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 33/45] usb: usbfs: Use consistent mmap functions Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 34/45] bonding (gcc13): synchronize bond_{a,t}lb_xmit() types Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 35/45] i2c: sprd: Delete i2c adapter in .removes error path Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 36/45] eeprom: at24: also select REGMAP Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 37/45] ext4: only check dquot_initialize_needed() when debugging Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 38/45] cifs: get rid of unused parameter in reconn_setup_dfs_targets() Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 39/45] cifs: handle empty list of targets in cifs_reconnect() Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 40/45] drm/atomic: Dont pollute crtc_state->mode_blob with error pointers Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 41/45] rbd: get snapshot context after exclusive lock is ensured to be held Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 42/45] btrfs: check return value of btrfs_commit_transaction in relocation Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 43/45] btrfs: unset reloc control if transaction commit fails in prepare_to_relocate() Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 44/45] mtd: spinand: macronix: Add support for MX35LFxGE4AD Greg Kroah-Hartman
2023-06-12 10:26 ` [PATCH 5.4 45/45] Revert "staging: rtl8192e: Replace macro RTL_PCI_DEVICE with PCI_DEVICE" Greg Kroah-Hartman
2023-06-12 21:54 ` [PATCH 5.4 00/45] 5.4.247-rc1 review Chris Paterson
2023-06-12 22:20 ` Shuah Khan
2023-06-13 8:38 ` Jon Hunter
2023-06-13 9:06 ` Naresh Kamboju
2023-06-13 12:16 ` Harshit Mogalapalli
2023-06-13 12:18 ` Sudip Mukherjee (Codethink)
2023-06-13 23:09 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230612101655.134991274@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=kadlec@netfilter.org \
--cc=kuniyu@amazon.com \
--cc=pablo@netfilter.org \
--cc=patches@lists.linux.dev \
--cc=sashal@kernel.org \
--cc=stable@vger.kernel.org \
--cc=syzkaller@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).