All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
	Xunlei Pang <xlpang@linux.alibaba.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Chris Down <chris@chrisdown.name>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Julius Hemanth Pitti <jpitti@cisco.com>
Subject: [PATCH 4.19 34/37] mm: memcg: fix memcg reclaim soft lockup
Date: Fri, 25 Sep 2020 14:49:02 +0200	[thread overview]
Message-ID: <20200925124726.085108517@linuxfoundation.org> (raw)
In-Reply-To: <20200925124720.972208530@linuxfoundation.org>

From: Xunlei Pang <xlpang@linux.alibaba.com>

commit e3336cab2579012b1e72b5265adf98e2d6e244ad upstream.

We've met softlockup with "CONFIG_PREEMPT_NONE=y", when the target memcg
doesn't have any reclaimable memory.

It can be easily reproduced as below:

  watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204]
  CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12
  Call Trace:
    shrink_lruvec+0x49f/0x640
    shrink_node+0x2a6/0x6f0
    do_try_to_free_pages+0xe9/0x3e0
    try_to_free_mem_cgroup_pages+0xef/0x1f0
    try_charge+0x2c1/0x750
    mem_cgroup_charge+0xd7/0x240
    __add_to_page_cache_locked+0x2fd/0x370
    add_to_page_cache_lru+0x4a/0xc0
    pagecache_get_page+0x10b/0x2f0
    filemap_fault+0x661/0xad0
    ext4_filemap_fault+0x2c/0x40
    __do_fault+0x4d/0xf9
    handle_mm_fault+0x1080/0x1790

It only happens on our 1-vcpu instances, because there's no chance for
oom reaper to run to reclaim the to-be-killed process.

Add a cond_resched() at the upper shrink_node_memcgs() to solve this
issue, this will mean that we will get a scheduling point for each memcg
in the reclaimed hierarchy without any dependency on the reclaimable
memory in that memcg thus making it more predictable.

Suggested-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Xunlei Pang <xlpang@linux.alibaba.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Chris Down <chris@chrisdown.name>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Link: http://lkml.kernel.org/r/1598495549-67324-1-git-send-email-xlpang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Julius Hemanth Pitti <jpitti@cisco.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 mm/vmscan.c |    8 ++++++++
 1 file changed, 8 insertions(+)

--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2708,6 +2708,14 @@ static bool shrink_node(pg_data_t *pgdat
 			unsigned long reclaimed;
 			unsigned long scanned;
 
+			/*
+			 * This loop can become CPU-bound when target memcgs
+			 * aren't eligible for reclaim - either because they
+			 * don't have any reclaimable pages, or because their
+			 * memory is explicitly protected. Avoid soft lockups.
+			 */
+			cond_resched();
+
 			switch (mem_cgroup_protected(root, memcg)) {
 			case MEMCG_PROT_MIN:
 				/*



  parent reply	other threads:[~2020-09-25 12:54 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-25 12:48 [PATCH 4.19 00/37] 4.19.148-rc1 review Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 01/37] af_key: pfkey_dump needs parameter validation Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 02/37] KVM: fix memory leak in kvm_io_bus_unregister_dev() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 03/37] kprobes: fix kill kprobe which has been marked as gone Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 04/37] mm/thp: fix __split_huge_pmd_locked() for migration PMD Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 05/37] cxgb4: Fix offset when clearing filter byte counters Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 06/37] geneve: add transport ports in route lookup for geneve Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 07/37] hdlc_ppp: add range checks in ppp_cp_parse_cr() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 08/37] ip: fix tos reflection in ack and reset packets Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 09/37] ipv6: avoid lockdep issue in fib6_del() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 10/37] net: DCB: Validate DCB_ATTR_DCB_BUFFER argument Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 11/37] net: dsa: rtl8366: Properly clear member config Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 12/37] net: ipv6: fix kconfig dependency warning for IPV6_SEG6_HMAC Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 13/37] net: sch_generic: aviod concurrent reset and enqueue op for lockless qdisc Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 14/37] nfp: use correct define to return NONE fec Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 15/37] tipc: Fix memory leak in tipc_group_create_member() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 16/37] tipc: fix shutdown() of connection oriented socket Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 17/37] tipc: use skb_unshare() instead in tipc_buf_append() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 18/37] bnxt_en: return proper error codes in bnxt_show_temp Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 19/37] bnxt_en: Protect bnxt_set_eee() and bnxt_set_pauseparam() with mutex Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 20/37] net: phy: Avoid NPD upon phy_detach() when driver is unbound Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 21/37] net: qrtr: check skb_put_padto() return value Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 22/37] net: add __must_check to skb_put_padto() Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 23/37] ipv4: Update exception handling for multipath routes via same device Greg Kroah-Hartman
2020-09-25 16:51   ` Pavel Machek
2020-09-26 15:46     ` Greg Kroah-Hartman
2020-09-29 12:26       ` Pavel Machek
2020-09-25 12:48 ` [PATCH 4.19 24/37] MAINTAINERS: add CLANG/LLVM BUILD SUPPORT info Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 25/37] kbuild: add OBJSIZE variable for the size tool Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 26/37] Documentation/llvm: add documentation on building w/ Clang/LLVM Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 27/37] Documentation/llvm: fix the name of llvm-size Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 28/37] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 29/37] net: wan: wanxl: use $(M68KCC) instead of $(M68KAS) " Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 30/37] x86/boot: kbuild: allow readelf executable to be specified Greg Kroah-Hartman
2020-09-25 12:48 ` [PATCH 4.19 31/37] kbuild: remove AS variable Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 32/37] kbuild: replace AS=clang with LLVM_IAS=1 Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 33/37] kbuild: support LLVM=1 to switch the default tools to Clang/LLVM Greg Kroah-Hartman
2020-09-25 12:49 ` Greg Kroah-Hartman [this message]
2020-09-25 12:49 ` [PATCH 4.19 35/37] tcp_bbr: refactor bbr_target_cwnd() for general inflight provisioning Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 36/37] tcp_bbr: adapt cwnd based on ack aggregation estimation Greg Kroah-Hartman
2020-09-25 12:49 ` [PATCH 4.19 37/37] serial: 8250: Avoid error message on reprobe Greg Kroah-Hartman
2020-09-25 17:39 ` [PATCH 4.19 00/37] 4.19.148-rc1 review Pavel Machek
2020-09-26 15:50   ` Greg Kroah-Hartman
2020-09-25 18:01 ` Jon Hunter
2020-09-25 20:02 ` Shuah Khan
2020-09-26 12:35 ` Naresh Kamboju
2020-09-26 15:42 ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200925124726.085108517@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=akpm@linux-foundation.org \
    --cc=chris@chrisdown.name \
    --cc=hannes@cmpxchg.org \
    --cc=jpitti@cisco.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mhocko@suse.com \
    --cc=stable@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=xlpang@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.