From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
Xunlei Pang <xlpang@linux.alibaba.com>,
Andrew Morton <akpm@linux-foundation.org>,
Chris Down <chris@chrisdown.name>,
Johannes Weiner <hannes@cmpxchg.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Julius Hemanth Pitti <jpitti@cisco.com>
Subject: [PATCH 4.19 34/37] mm: memcg: fix memcg reclaim soft lockup
Date: Fri, 25 Sep 2020 14:49:02 +0200 [thread overview]
Message-ID: <20200925124726.085108517@linuxfoundation.org> (raw)
In-Reply-To: <20200925124720.972208530@linuxfoundation.org>
From: Xunlei Pang <xlpang@linux.alibaba.com>
commit e3336cab2579012b1e72b5265adf98e2d6e244ad upstream.
We've met softlockup with "CONFIG_PREEMPT_NONE=y", when the target memcg
doesn't have any reclaimable memory.
It can be easily reproduced as below:
watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204]
CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12
Call Trace:
shrink_lruvec+0x49f/0x640
shrink_node+0x2a6/0x6f0
do_try_to_free_pages+0xe9/0x3e0
try_to_free_mem_cgroup_pages+0xef/0x1f0
try_charge+0x2c1/0x750
mem_cgroup_charge+0xd7/0x240
__add_to_page_cache_locked+0x2fd/0x370
add_to_page_cache_lru+0x4a/0xc0
pagecache_get_page+0x10b/0x2f0
filemap_fault+0x661/0xad0
ext4_filemap_fault+0x2c/0x40
__do_fault+0x4d/0xf9
handle_mm_fault+0x1080/0x1790
It only happens on our 1-vcpu instances, because there's no chance for
oom reaper to run to reclaim the to-be-killed process.
Add a cond_resched() at the upper shrink_node_memcgs() to solve this
issue, this will mean that we will get a scheduling point for each memcg
in the reclaimed hierarchy without any dependency on the reclaimable
memory in that memcg thus making it more predictable.
Suggested-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Xunlei Pang <xlpang@linux.alibaba.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Chris Down <chris@chrisdown.name>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Link: http://lkml.kernel.org/r/1598495549-67324-1-git-send-email-xlpang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Julius Hemanth Pitti <jpitti@cisco.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
mm/vmscan.c | 8 ++++++++
1 file changed, 8 insertions(+)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2708,6 +2708,14 @@ static bool shrink_node(pg_data_t *pgdat
unsigned long reclaimed;
unsigned long scanned;
+ /*
+ * This loop can become CPU-bound when target memcgs
+ * aren't eligible for reclaim - either because they
+ * don't have any reclaimable pages, or because their
+ * memory is explicitly protected. Avoid soft lockups.
+ */
+ cond_resched();
+
switch (mem_cgroup_protected(root, memcg)) {
case MEMCG_PROT_MIN:
/*
next prev parent reply other threads:[~2020-09-25 12:54 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-25 12:48 [PATCH 4.19 00/37] 4.19.148-rc1 review Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 01/37] af_key: pfkey_dump needs parameter validation Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 02/37] KVM: fix memory leak in kvm_io_bus_unregister_dev() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 03/37] kprobes: fix kill kprobe which has been marked as gone Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 04/37] mm/thp: fix __split_huge_pmd_locked() for migration PMD Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 05/37] cxgb4: Fix offset when clearing filter byte counters Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 06/37] geneve: add transport ports in route lookup for geneve Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 07/37] hdlc_ppp: add range checks in ppp_cp_parse_cr() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 08/37] ip: fix tos reflection in ack and reset packets Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 09/37] ipv6: avoid lockdep issue in fib6_del() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 10/37] net: DCB: Validate DCB_ATTR_DCB_BUFFER argument Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 11/37] net: dsa: rtl8366: Properly clear member config Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 12/37] net: ipv6: fix kconfig dependency warning for IPV6_SEG6_HMAC Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 13/37] net: sch_generic: aviod concurrent reset and enqueue op for lockless qdisc Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 14/37] nfp: use correct define to return NONE fec Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 15/37] tipc: Fix memory leak in tipc_group_create_member() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 16/37] tipc: fix shutdown() of connection oriented socket Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 17/37] tipc: use skb_unshare() instead in tipc_buf_append() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 18/37] bnxt_en: return proper error codes in bnxt_show_temp Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 19/37] bnxt_en: Protect bnxt_set_eee() and bnxt_set_pauseparam() with mutex Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 20/37] net: phy: Avoid NPD upon phy_detach() when driver is unbound Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 21/37] net: qrtr: check skb_put_padto() return value Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 22/37] net: add __must_check to skb_put_padto() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 23/37] ipv4: Update exception handling for multipath routes via same device Greg Kroah-Hartman
2020-09-25 16:51 ` Pavel Machek
2020-09-26 15:46 ` Greg Kroah-Hartman
2020-09-29 12:26 ` Pavel Machek
2020-09-25 12:48 ` [PATCH 4.19 24/37] MAINTAINERS: add CLANG/LLVM BUILD SUPPORT info Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 25/37] kbuild: add OBJSIZE variable for the size tool Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 26/37] Documentation/llvm: add documentation on building w/ Clang/LLVM Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 27/37] Documentation/llvm: fix the name of llvm-size Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 28/37] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 29/37] net: wan: wanxl: use $(M68KCC) instead of $(M68KAS) " Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 30/37] x86/boot: kbuild: allow readelf executable to be specified Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 31/37] kbuild: remove AS variable Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 32/37] kbuild: replace AS=clang with LLVM_IAS=1 Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 33/37] kbuild: support LLVM=1 to switch the default tools to Clang/LLVM Greg Kroah-Hartman
2020-09-25 12:49 ` Greg Kroah-Hartman [this message]
2020-09-25 12:49 ` [PATCH 4.19 35/37] tcp_bbr: refactor bbr_target_cwnd() for general inflight provisioning Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 36/37] tcp_bbr: adapt cwnd based on ack aggregation estimation Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 37/37] serial: 8250: Avoid error message on reprobe Greg Kroah-Hartman
2020-09-25 17:39 ` [PATCH 4.19 00/37] 4.19.148-rc1 review Pavel Machek
2020-09-26 15:50 ` Greg Kroah-Hartman
2020-09-25 18:01 ` Jon Hunter
2020-09-25 20:02 ` Shuah Khan
2020-09-26 12:35 ` Naresh Kamboju
2020-09-26 15:42 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200925124726.085108517@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=chris@chrisdown.name \
--cc=hannes@cmpxchg.org \
--cc=jpitti@cisco.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=xlpang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).