From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
Xunlei Pang <xlpang@linux.alibaba.com>,
Andrew Morton <akpm@linux-foundation.org>,
Chris Down <chris@chrisdown.name>,
Johannes Weiner <hannes@cmpxchg.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Julius Hemanth Pitti <jpitti@cisco.com>
Subject: [PATCH 5.4 42/43] mm: memcg: fix memcg reclaim soft lockup
Date: Fri, 25 Sep 2020 14:48:54 +0200 [thread overview]
Message-ID: <20200925124729.884757815@linuxfoundation.org> (raw)
In-Reply-To: <20200925124723.575329814@linuxfoundation.org>
From: Xunlei Pang <xlpang@linux.alibaba.com>
commit e3336cab2579012b1e72b5265adf98e2d6e244ad upstream.
We've met softlockup with "CONFIG_PREEMPT_NONE=y", when the target memcg
doesn't have any reclaimable memory.
It can be easily reproduced as below:
watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204]
CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12
Call Trace:
shrink_lruvec+0x49f/0x640
shrink_node+0x2a6/0x6f0
do_try_to_free_pages+0xe9/0x3e0
try_to_free_mem_cgroup_pages+0xef/0x1f0
try_charge+0x2c1/0x750
mem_cgroup_charge+0xd7/0x240
__add_to_page_cache_locked+0x2fd/0x370
add_to_page_cache_lru+0x4a/0xc0
pagecache_get_page+0x10b/0x2f0
filemap_fault+0x661/0xad0
ext4_filemap_fault+0x2c/0x40
__do_fault+0x4d/0xf9
handle_mm_fault+0x1080/0x1790
It only happens on our 1-vcpu instances, because there's no chance for
oom reaper to run to reclaim the to-be-killed process.
Add a cond_resched() at the upper shrink_node_memcgs() to solve this
issue, this will mean that we will get a scheduling point for each memcg
in the reclaimed hierarchy without any dependency on the reclaimable
memory in that memcg thus making it more predictable.
Suggested-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Xunlei Pang <xlpang@linux.alibaba.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Chris Down <chris@chrisdown.name>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Link: http://lkml.kernel.org/r/1598495549-67324-1-git-send-email-xlpang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Julius Hemanth Pitti <jpitti@cisco.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
mm/vmscan.c | 8 ++++++++
1 file changed, 8 insertions(+)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2775,6 +2775,14 @@ static bool shrink_node(pg_data_t *pgdat
unsigned long reclaimed;
unsigned long scanned;
+ /*
+ * This loop can become CPU-bound when target memcgs
+ * aren't eligible for reclaim - either because they
+ * don't have any reclaimable pages, or because their
+ * memory is explicitly protected. Avoid soft lockups.
+ */
+ cond_resched();
+
switch (mem_cgroup_protected(root, memcg)) {
case MEMCG_PROT_MIN:
/*
next prev parent reply other threads:[~2020-09-25 12:53 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-25 12:48 [PATCH 5.4 00/43] 5.4.68-rc1 review Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 01/43] af_key: pfkey_dump needs parameter validation Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 02/43] ibmvnic fix NULL tx_pools and rx_tools issue at do_reset Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 03/43] ibmvnic: add missing parenthesis in do_reset() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 04/43] kprobes: fix kill kprobe which has been marked as gone Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 05/43] mm/thp: fix __split_huge_pmd_locked() for migration PMD Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 06/43] act_ife: load meta modules before tcf_idr_check_alloc() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 07/43] bnxt_en: Avoid sending firmware messages when AER error is detected Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 08/43] bnxt_en: Fix NULL ptr dereference crash in bnxt_fw_reset_task() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 09/43] cxgb4: fix memory leak during module unload Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 10/43] cxgb4: Fix offset when clearing filter byte counters Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 11/43] geneve: add transport ports in route lookup for geneve Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 12/43] hdlc_ppp: add range checks in ppp_cp_parse_cr() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 13/43] ip: fix tos reflection in ack and reset packets Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 14/43] ipv4: Initialize flowi4_multipath_hash in data path Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 15/43] ipv4: Update exception handling for multipath routes via same device Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 16/43] ipv6: avoid lockdep issue in fib6_del() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 17/43] net: bridge: br_vlan_get_pvid_rcu() should dereference the VLAN group under RCU Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 18/43] net: DCB: Validate DCB_ATTR_DCB_BUFFER argument Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 19/43] net: dsa: rtl8366: Properly clear member config Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 20/43] net: Fix bridge enslavement failure Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 21/43] net: ipv6: fix kconfig dependency warning for IPV6_SEG6_HMAC Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 22/43] net/mlx5: Fix FTE cleanup Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 23/43] net: sch_generic: aviod concurrent reset and enqueue op for lockless qdisc Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 24/43] net: sctp: Fix IPv6 ancestor_size calc in sctp_copy_descendant Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 25/43] nfp: use correct define to return NONE fec Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 26/43] taprio: Fix allowing too small intervals Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 27/43] tipc: Fix memory leak in tipc_group_create_member() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 28/43] tipc: fix shutdown() of connection oriented socket Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 29/43] tipc: use skb_unshare() instead in tipc_buf_append() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 30/43] net/mlx5e: Enable adding peer miss rules only if merged eswitch is supported Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 31/43] net/mlx5e: TLS, Do not expose FPGA TLS counter if not supported Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 32/43] bnxt_en: return proper error codes in bnxt_show_temp Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 33/43] bnxt_en: Protect bnxt_set_eee() and bnxt_set_pauseparam() with mutex Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 34/43] net: lantiq: Wake TX queue again Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 35/43] net: lantiq: use netif_tx_napi_add() for TX NAPI Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 36/43] net: lantiq: Use napi_complete_done() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 37/43] net: lantiq: Disable IRQs only if NAPI gets scheduled Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 38/43] net: phy: Avoid NPD upon phy_detach() when driver is unbound Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 39/43] net: phy: Do not warn in phy_stop() on PHY_DOWN Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 40/43] net: qrtr: check skb_put_padto() return value Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 5.4 41/43] net: add __must_check to skb_put_padto() Greg Kroah-Hartman
2020-09-25 12:48 ` Greg Kroah-Hartman [this message]
2020-09-25 12:48 ` [PATCH 5.4 43/43] iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE Greg Kroah-Hartman
2020-09-25 18:01 ` [PATCH 5.4 00/43] 5.4.68-rc1 review Jon Hunter
2020-09-25 20:02 ` Shuah Khan
2020-09-26 12:39 ` Naresh Kamboju
2020-09-26 15:43 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200925124729.884757815@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=chris@chrisdown.name \
--cc=hannes@cmpxchg.org \
--cc=jpitti@cisco.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=xlpang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).