From: Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>
To: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
Cc: Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
kernel-team-b10kYP2dOMg@public.gmane.org
Subject: Re: [PATCH] mm: memcontrol: prevent starvation when writing memory.high
Date: Wed, 13 Jan 2021 10:06:44 -0800 [thread overview]
Message-ID: <20210113180620.GA353910@carbon.lan> (raw)
In-Reply-To: <20210112163011.127833-1-hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
On Tue, Jan 12, 2021 at 11:30:11AM -0500, Johannes Weiner wrote:
> When a value is written to a cgroup's memory.high control file, the
> write() context first tries to reclaim the cgroup to size before
> putting the limit in place for the workload. Concurrent charges from
> the workload can keep such a write() looping in reclaim indefinitely.
>
> In the past, a write to memory.high would first put the limit in place
> for the workload, then do targeted reclaim until the new limit has
> been met - similar to how we do it for memory.max. This wasn't prone
> to the described starvation issue. However, this sequence could cause
> excessive latencies in the workload, when allocating threads could be
> put into long penalty sleeps on the sudden memory.high overage created
> by the write(), before that had a chance to work it off.
>
> Now that memory_high_write() performs reclaim before enforcing the new
> limit, reflect that the cgroup may well fail to converge due to
> concurrent workload activity. Bail out of the loop after a few tries.
>
> Fixes: 536d3bf261a2 ("mm: memcontrol: avoid workload stalls when lowering memory.high")
> Cc: <stable-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> # 5.8+
> Reported-by: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> Signed-off-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
> ---
> mm/memcontrol.c | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 605f671203ef..63a8d47c1cd3 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -6275,7 +6275,6 @@ static ssize_t memory_high_write(struct kernfs_open_file *of,
>
> for (;;) {
> unsigned long nr_pages = page_counter_read(&memcg->memory);
> - unsigned long reclaimed;
>
> if (nr_pages <= high)
> break;
> @@ -6289,10 +6288,10 @@ static ssize_t memory_high_write(struct kernfs_open_file *of,
> continue;
> }
>
> - reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_pages - high,
> - GFP_KERNEL, true);
> + try_to_free_mem_cgroup_pages(memcg, nr_pages - high,
> + GFP_KERNEL, true);
>
> - if (!reclaimed && !nr_retries--)
> + if (!nr_retries--)
> break;
> }
>
> --
> 2.30.0
>
Acked-by: Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>
Thanks!
WARNING: multiple messages have this Message-ID (diff)
From: Roman Gushchin <guro@fb.com>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Tejun Heo <tj@kernel.org>, Michal Hocko <mhocko@suse.com>,
<linux-mm@kvack.org>, <cgroups@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <kernel-team@fb.com>
Subject: Re: [PATCH] mm: memcontrol: prevent starvation when writing memory.high
Date: Wed, 13 Jan 2021 10:06:44 -0800 [thread overview]
Message-ID: <20210113180620.GA353910@carbon.lan> (raw)
In-Reply-To: <20210112163011.127833-1-hannes@cmpxchg.org>
On Tue, Jan 12, 2021 at 11:30:11AM -0500, Johannes Weiner wrote:
> When a value is written to a cgroup's memory.high control file, the
> write() context first tries to reclaim the cgroup to size before
> putting the limit in place for the workload. Concurrent charges from
> the workload can keep such a write() looping in reclaim indefinitely.
>
> In the past, a write to memory.high would first put the limit in place
> for the workload, then do targeted reclaim until the new limit has
> been met - similar to how we do it for memory.max. This wasn't prone
> to the described starvation issue. However, this sequence could cause
> excessive latencies in the workload, when allocating threads could be
> put into long penalty sleeps on the sudden memory.high overage created
> by the write(), before that had a chance to work it off.
>
> Now that memory_high_write() performs reclaim before enforcing the new
> limit, reflect that the cgroup may well fail to converge due to
> concurrent workload activity. Bail out of the loop after a few tries.
>
> Fixes: 536d3bf261a2 ("mm: memcontrol: avoid workload stalls when lowering memory.high")
> Cc: <stable@vger.kernel.org> # 5.8+
> Reported-by: Tejun Heo <tj@kernel.org>
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> ---
> mm/memcontrol.c | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 605f671203ef..63a8d47c1cd3 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -6275,7 +6275,6 @@ static ssize_t memory_high_write(struct kernfs_open_file *of,
>
> for (;;) {
> unsigned long nr_pages = page_counter_read(&memcg->memory);
> - unsigned long reclaimed;
>
> if (nr_pages <= high)
> break;
> @@ -6289,10 +6288,10 @@ static ssize_t memory_high_write(struct kernfs_open_file *of,
> continue;
> }
>
> - reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_pages - high,
> - GFP_KERNEL, true);
> + try_to_free_mem_cgroup_pages(memcg, nr_pages - high,
> + GFP_KERNEL, true);
>
> - if (!reclaimed && !nr_retries--)
> + if (!nr_retries--)
> break;
> }
>
> --
> 2.30.0
>
Acked-by: Roman Gushchin <guro@fb.com>
Thanks!
next prev parent reply other threads:[~2021-01-13 18:06 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-12 16:30 [PATCH] mm: memcontrol: prevent starvation when writing memory.high Johannes Weiner
2021-01-12 16:30 ` Johannes Weiner
2021-01-13 14:46 ` Michal Hocko
[not found] ` <20210113144654.GD22493-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2021-01-15 16:20 ` Johannes Weiner
2021-01-15 16:20 ` Johannes Weiner
[not found] ` <YAHA4uBSLlnxxAbu-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2021-01-15 17:03 ` Roman Gushchin
2021-01-15 17:03 ` Roman Gushchin
[not found] ` <20210115170341.GA631549-cx5fftMpWqeCjSd+JxjunQ2O0Ztt9esIQQ4Iyu8u01E@public.gmane.org>
2021-01-15 20:55 ` Johannes Weiner
2021-01-15 20:55 ` Johannes Weiner
[not found] ` <YAIBSJg3btQ+2CNZ-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2021-01-15 21:27 ` Roman Gushchin
2021-01-15 21:27 ` Roman Gushchin
2021-01-19 16:47 ` Johannes Weiner
2021-01-18 13:12 ` Michal Hocko
2021-01-18 13:12 ` Michal Hocko
[not found] ` <20210112163011.127833-1-hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2021-01-12 17:03 ` Roman Gushchin
2021-01-12 17:03 ` Roman Gushchin
[not found] ` <20210112170322.GA99586-cx5fftMpWqeCjSd+JxjunQ2O0Ztt9esIQQ4Iyu8u01E@public.gmane.org>
2021-01-12 19:45 ` Johannes Weiner
2021-01-12 19:45 ` Johannes Weiner
[not found] ` <X/38ZwyOE96SAfa9-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2021-01-12 20:12 ` Roman Gushchin
2021-01-12 20:12 ` Roman Gushchin
2021-01-12 21:11 ` Johannes Weiner
2021-01-12 21:45 ` Roman Gushchin
2021-01-12 21:45 ` Roman Gushchin
[not found] ` <20210112214512.GC99586-cx5fftMpWqeCjSd+JxjunQ2O0Ztt9esIQQ4Iyu8u01E@public.gmane.org>
2021-01-15 15:34 ` Johannes Weiner
2021-01-15 15:34 ` Johannes Weiner
2021-01-12 18:59 ` Shakeel Butt
2021-01-12 18:59 ` Shakeel Butt
2021-01-12 19:53 ` Johannes Weiner
[not found] ` <X/3+IgDVb+Jn4XfQ-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2021-01-12 20:28 ` Shakeel Butt
2021-01-12 20:28 ` Shakeel Butt
2021-01-13 17:25 ` Michal Koutný
2021-01-13 17:25 ` Michal Koutný
2021-01-13 18:06 ` Roman Gushchin [this message]
2021-01-13 18:06 ` Roman Gushchin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210113180620.GA353910@carbon.lan \
--to=guro-b10kyp2domg@public.gmane.org \
--cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
--cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org \
--cc=kernel-team-b10kYP2dOMg@public.gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
--cc=mhocko-IBi9RG/b67k@public.gmane.org \
--cc=tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.