From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
Xunlei Pang <xlpang@linux.alibaba.com>,
Andrew Morton <akpm@linux-foundation.org>,
Chris Down <chris@chrisdown.name>,
Johannes Weiner <hannes@cmpxchg.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Julius Hemanth Pitti <jpitti@cisco.com>
Subject: [PATCH 5.4 42/43] mm: memcg: fix memcg reclaim soft lockup
Date: Fri, 25 Sep 2020 14:48:54 +0200 [thread overview]
Message-ID: <20200925124729.884757815@linuxfoundation.org> (raw)
In-Reply-To: <20200925124723.575329814@linuxfoundation.org>
From: Xunlei Pang <xlpang@linux.alibaba.com>
commit e3336cab2579012b1e72b5265adf98e2d6e244ad upstream.
We've met softlockup with "CONFIG_PREEMPT_NONE=y", when the target memcg
doesn't have any reclaimable memory.
It can be easily reproduced as below:
watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204]
CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12
Call Trace:
shrink_lruvec+0x49f/0x640
shrink_node+0x2a6/0x6f0
do_try_to_free_pages+0xe9/0x3e0
try_to_free_mem_cgroup_pages+0xef/0x1f0
try_charge+0x2c1/0x750
mem_cgroup_charge+0xd7/0x240
__add_to_page_cache_locked+0x2fd/0x370
add_to_page_cache_lru+0x4a/0xc0
pagecache_get_page+0x10b/0x2f0
filemap_fault+0x661/0xad0
ext4_filemap_fault+0x2c/0x40
__do_fault+0x4d/0xf9
handle_mm_fault+0x1080/0x1790
It only happens on our 1-vcpu instances, because there's no chance for
oom reaper to run to reclaim the to-be-killed process.
Add a cond_resched() at the upper shrink_node_memcgs() to solve this
issue, this will mean that we will get a scheduling point for each memcg
in the reclaimed hierarchy without any dependency on the reclaimable
memory in that memcg thus making it more predictable.
Suggested-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Xunlei Pang <xlpang@linux.alibaba.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Chris Down <chris@chrisdown.name>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Link: http://lkml.kernel.org/r/1598495549-67324-1-git-send-email-xlpang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Julius Hemanth Pitti <jpitti@cisco.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
mm/vmscan.c | 8 ++++++++
1 file changed, 8 insertions(+)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2775,6 +2775,14 @@ static bool shrink_node(pg_data_t *pgdat
unsigned long reclaimed;
unsigned long scanned;
+ /*
+ * This loop can become CPU-bound when target memcgs
+ * aren't eligible for reclaim - either because they
+ * don't have any reclaimable pages, or because their
+ * memory is explicitly protected. Avoid soft lockups.
+ */
+ cond_resched();
+
switch (mem_cgroup_protected(root, memcg)) {
case MEMCG_PROT_MIN:
/*
next prev parent reply other threads:[~2020-09-25 12:53 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-25 12:48 [PATCH 5.4 00/43] 5.4.68-rc1 review Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 01/43] af_key: pfkey_dump needs parameter validation Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 02/43] ibmvnic fix NULL tx_pools and rx_tools issue at do_reset Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 03/43] ibmvnic: add missing parenthesis in do_reset() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 04/43] kprobes: fix kill kprobe which has been marked as gone Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 05/43] mm/thp: fix __split_huge_pmd_locked() for migration PMD Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 06/43] act_ife: load meta modules before tcf_idr_check_alloc() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 07/43] bnxt_en: Avoid sending firmware messages when AER error is detected Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 08/43] bnxt_en: Fix NULL ptr dereference crash in bnxt_fw_reset_task() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 09/43] cxgb4: fix memory leak during module unload Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 10/43] cxgb4: Fix offset when clearing filter byte counters Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 11/43] geneve: add transport ports in route lookup for geneve Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 12/43] hdlc_ppp: add range checks in ppp_cp_parse_cr() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 13/43] ip: fix tos reflection in ack and reset packets Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 14/43] ipv4: Initialize flowi4_multipath_hash in data path Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 15/43] ipv4: Update exception handling for multipath routes via same device Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 16/43] ipv6: avoid lockdep issue in fib6_del() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 17/43] net: bridge: br_vlan_get_pvid_rcu() should dereference the VLAN group under RCU Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 18/43] net: DCB: Validate DCB_ATTR_DCB_BUFFER argument Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 19/43] net: dsa: rtl8366: Properly clear member config Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 20/43] net: Fix bridge enslavement failure Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 21/43] net: ipv6: fix kconfig dependency warning for IPV6_SEG6_HMAC Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 22/43] net/mlx5: Fix FTE cleanup Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 23/43] net: sch_generic: aviod concurrent reset and enqueue op for lockless qdisc Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 24/43] net: sctp: Fix IPv6 ancestor_size calc in sctp_copy_descendant Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 25/43] nfp: use correct define to return NONE fec Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 26/43] taprio: Fix allowing too small intervals Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 27/43] tipc: Fix memory leak in tipc_group_create_member() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 28/43] tipc: fix shutdown() of connection oriented socket Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 29/43] tipc: use skb_unshare() instead in tipc_buf_append() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 30/43] net/mlx5e: Enable adding peer miss rules only if merged eswitch is supported Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 31/43] net/mlx5e: TLS, Do not expose FPGA TLS counter if not supported Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 32/43] bnxt_en: return proper error codes in bnxt_show_temp Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 33/43] bnxt_en: Protect bnxt_set_eee() and bnxt_set_pauseparam() with mutex Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 34/43] net: lantiq: Wake TX queue again Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 35/43] net: lantiq: use netif_tx_napi_add() for TX NAPI Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 36/43] net: lantiq: Use napi_complete_done() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 37/43] net: lantiq: Disable IRQs only if NAPI gets scheduled Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 38/43] net: phy: Avoid NPD upon phy_detach() when driver is unbound Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 39/43] net: phy: Do not warn in phy_stop() on PHY_DOWN Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 40/43] net: qrtr: check skb_put_padto() return value Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 41/43] net: add __must_check to skb_put_padto() Greg Kroah-Hartman
2020-09-25 12:48 ` Greg Kroah-Hartman [this message]
2020-09-25 12:48 ` [PATCH 5.4 43/43] iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE Greg Kroah-Hartman
2020-09-25 18:01 ` [PATCH 5.4 00/43] 5.4.68-rc1 review Jon Hunter
2020-09-25 20:02 ` Shuah Khan
2020-09-26 12:39 ` Naresh Kamboju
2020-09-26 15:43 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200925124729.884757815@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=chris@chrisdown.name \
--cc=hannes@cmpxchg.org \
--cc=jpitti@cisco.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=xlpang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.