All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
To: akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org
Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
	kernel-team-b10kYP2dOMg@public.gmane.org,
	tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org,
	chris-6Bi1550iOqEnzZ6mRAm98g@public.gmane.org,
	cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	Jakub Kicinski <kuba-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Subject: [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted
Date: Thu, 16 Apr 2020 18:06:14 -0700	[thread overview]
Message-ID: <20200417010617.927266-1-kuba@kernel.org> (raw)

Tejun describes the problem as follows:

When swap runs out, there's an abrupt change in system behavior -
the anonymous memory suddenly becomes unmanageable which readily
breaks any sort of memory isolation and can bring down the whole
system. To avoid that, oomd [1] monitors free swap space and triggers
kills when it drops below the specific threshold (e.g. 15%).

While this works, it's far from ideal:
 - Depending on IO performance and total swap size, a given
   headroom might not be enough or too much.
 - oomd has to monitor swap depletion in addition to the usual
   pressure metrics and it currently doesn't consider memory.swap.max.

Solve this by adapting the same approach that memory.high uses -
slow down allocation as the resource gets depleted turning the
depletion behavior from abrupt cliff one to gradual degradation
observable through memory pressure metric.

[1] https://github.com/facebookincubator/oomd

Jakub Kicinski (3):
  mm: prepare for swap over-high accounting and penalty calculation
  mm: move penalty delay clamping out of calculate_high_delay()
  mm: automatically penalize tasks with high swap use

 include/linux/memcontrol.h |   4 +
 mm/memcontrol.c            | 166 ++++++++++++++++++++++++++++---------
 2 files changed, 131 insertions(+), 39 deletions(-)

-- 
2.25.2


WARNING: multiple messages have this Message-ID (diff)
From: Jakub Kicinski <kuba@kernel.org>
To: akpm@linux-foundation.org
Cc: linux-mm@kvack.org, kernel-team@fb.com, tj@kernel.org,
	hannes@cmpxchg.org, chris@chrisdown.name,
	cgroups@vger.kernel.org, Jakub Kicinski <kuba@kernel.org>
Subject: [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted
Date: Thu, 16 Apr 2020 18:06:14 -0700	[thread overview]
Message-ID: <20200417010617.927266-1-kuba@kernel.org> (raw)

Tejun describes the problem as follows:

When swap runs out, there's an abrupt change in system behavior -
the anonymous memory suddenly becomes unmanageable which readily
breaks any sort of memory isolation and can bring down the whole
system. To avoid that, oomd [1] monitors free swap space and triggers
kills when it drops below the specific threshold (e.g. 15%).

While this works, it's far from ideal:
 - Depending on IO performance and total swap size, a given
   headroom might not be enough or too much.
 - oomd has to monitor swap depletion in addition to the usual
   pressure metrics and it currently doesn't consider memory.swap.max.

Solve this by adapting the same approach that memory.high uses -
slow down allocation as the resource gets depleted turning the
depletion behavior from abrupt cliff one to gradual degradation
observable through memory pressure metric.

[1] https://github.com/facebookincubator/oomd

Jakub Kicinski (3):
  mm: prepare for swap over-high accounting and penalty calculation
  mm: move penalty delay clamping out of calculate_high_delay()
  mm: automatically penalize tasks with high swap use

 include/linux/memcontrol.h |   4 +
 mm/memcontrol.c            | 166 ++++++++++++++++++++++++++++---------
 2 files changed, 131 insertions(+), 39 deletions(-)

-- 
2.25.2



             reply	other threads:[~2020-04-17  1:06 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-17  1:06 Jakub Kicinski [this message]
2020-04-17  1:06 ` [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted Jakub Kicinski
     [not found] ` <20200417010617.927266-1-kuba-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-04-17  1:06   ` [PATCH 1/3] mm: prepare for swap over-high accounting and penalty calculation Jakub Kicinski
2020-04-17  1:06     ` Jakub Kicinski
2020-04-17  1:06   ` [PATCH 2/3] mm: move penalty delay clamping out of calculate_high_delay() Jakub Kicinski
2020-04-17  1:06     ` Jakub Kicinski
2020-04-17  1:06   ` [PATCH 3/3] mm: automatically penalize tasks with high swap use Jakub Kicinski
2020-04-17  1:06     ` Jakub Kicinski
     [not found]     ` <20200417010617.927266-4-kuba-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-04-17  7:37       ` Michal Hocko
2020-04-17  7:37         ` Michal Hocko
     [not found]         ` <20200417073716.GG26707-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-17 23:22           ` Jakub Kicinski
2020-04-17 23:22             ` Jakub Kicinski
2020-04-17 16:11   ` [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted Shakeel Butt
2020-04-17 16:11     ` Shakeel Butt
     [not found]     ` <CALvZod78ZUhU+yr2x1h_gv+VgVGTPnSSGKh_+fd+MeiAKreJvg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-04-17 16:23       ` Tejun Heo
2020-04-17 16:23         ` Tejun Heo
     [not found]         ` <20200417162355.GA43469-146+VewaZzwNjtGbbfXrCEEOCMrvLtNR@public.gmane.org>
2020-04-17 17:18           ` Shakeel Butt
2020-04-17 17:18             ` Shakeel Butt
     [not found]             ` <CALvZod4ftvXCu8SbQUXwTGVvx5K2+at9h30r28chZLXEB1JdfQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-04-17 17:36               ` Tejun Heo
2020-04-17 17:36                 ` Tejun Heo
     [not found]                 ` <20200417173615.GB43469-146+VewaZzwNjtGbbfXrCEEOCMrvLtNR@public.gmane.org>
2020-04-17 17:51                   ` Shakeel Butt
2020-04-17 17:51                     ` Shakeel Butt
2020-04-17 19:35                     ` Tejun Heo
2020-04-17 19:35                       ` Tejun Heo
     [not found]                       ` <20200417193539.GC43469-146+VewaZzwNjtGbbfXrCEEOCMrvLtNR@public.gmane.org>
2020-04-17 21:51                         ` Shakeel Butt
2020-04-17 21:51                           ` Shakeel Butt
     [not found]                           ` <CALvZod6LT25t9aAA1KHmf1U4-L8zSjUXQ4VQvX4cMT1A+R_g+w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-04-17 22:59                             ` Tejun Heo
2020-04-17 22:59                               ` Tejun Heo
     [not found]                               ` <20200417225941.GE43469-146+VewaZzwNjtGbbfXrCEEOCMrvLtNR@public.gmane.org>
2020-04-20 16:12                                 ` Shakeel Butt
2020-04-20 16:12                                   ` Shakeel Butt
     [not found]                                   ` <CALvZod6M4OsM-t8m_KX9wCkEutdwUMgbP9682eHGQor9JvO_BQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-04-20 16:47                                     ` Tejun Heo
2020-04-20 16:47                                       ` Tejun Heo
     [not found]                                       ` <20200420164740.GF43469-146+VewaZzwNjtGbbfXrCEEOCMrvLtNR@public.gmane.org>
2020-04-20 17:03                                         ` Michal Hocko
2020-04-20 17:03                                           ` Michal Hocko
     [not found]                                           ` <20200420170318.GV27314-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-20 17:06                                             ` Tejun Heo
2020-04-20 17:06                                               ` Tejun Heo
     [not found]                                               ` <20200420170650.GA169746-146+VewaZzwNjtGbbfXrCEEOCMrvLtNR@public.gmane.org>
2020-04-21 11:06                                                 ` Michal Hocko
2020-04-21 11:06                                                   ` Michal Hocko
     [not found]                                                   ` <20200421110612.GD27314-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-21 14:27                                                     ` Johannes Weiner
2020-04-21 14:27                                                       ` Johannes Weiner
     [not found]                                                       ` <20200421142746.GA341682-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-21 16:11                                                         ` Michal Hocko
2020-04-21 16:11                                                           ` Michal Hocko
     [not found]                                                           ` <20200421161138.GL27314-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-21 16:56                                                             ` Johannes Weiner
2020-04-21 16:56                                                               ` Johannes Weiner
     [not found]                                                               ` <20200421165601.GA345998-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-22 13:26                                                                 ` Michal Hocko
2020-04-22 13:26                                                                   ` Michal Hocko
     [not found]                                                                   ` <20200422132632.GG30312-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-22 14:15                                                                     ` Johannes Weiner
2020-04-22 14:15                                                                       ` Johannes Weiner
     [not found]                                                                       ` <20200422141514.GA362484-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-22 15:43                                                                         ` Michal Hocko
2020-04-22 15:43                                                                           ` Michal Hocko
     [not found]                                                                           ` <20200422154318.GK30312-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-22 17:13                                                                             ` Johannes Weiner
2020-04-22 17:13                                                                               ` Johannes Weiner
     [not found]                                                                               ` <20200422171328.GC362484-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-22 18:49                                                                                 ` Michal Hocko
2020-04-22 18:49                                                                                   ` Michal Hocko
     [not found]                                                                                   ` <20200422184921.GB4206-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-23 15:00                                                                                     ` Johannes Weiner
2020-04-23 15:00                                                                                       ` Johannes Weiner
     [not found]                                                                                       ` <20200423150015.GE362484-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-24 15:05                                                                                         ` Michal Hocko
2020-04-24 15:05                                                                                           ` Michal Hocko
     [not found]                                                                                           ` <20200424150510.GH11591-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-04-28 14:24                                                                                             ` Johannes Weiner
2020-04-28 14:24                                                                                               ` Johannes Weiner
     [not found]                                                                                               ` <20200428142432.GA78561-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-29  9:55                                                                                                 ` Michal Hocko
2020-04-29  9:55                                                                                                   ` Michal Hocko
2020-04-21 19:09                                                         ` Shakeel Butt
2020-04-21 19:09                                                           ` Shakeel Butt
     [not found]                                                           ` <CALvZod650M1_46R4OiS1mug+LKbjD=1s_xqckh9T6V8fPjct2g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-04-21 21:59                                                             ` Johannes Weiner
2020-04-21 21:59                                                               ` Johannes Weiner
     [not found]                                                               ` <20200421215946.GA347151-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-21 22:39                                                                 ` Shakeel Butt
2020-04-21 22:39                                                                   ` Shakeel Butt
2020-04-21 15:20                                                     ` Tejun Heo
2020-04-21 15:20                                                       ` Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200417010617.927266-1-kuba@kernel.org \
    --to=kuba-dgejt+ai2ygdnm+yrofe0a@public.gmane.org \
    --cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
    --cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=chris-6Bi1550iOqEnzZ6mRAm98g@public.gmane.org \
    --cc=hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org \
    --cc=kernel-team-b10kYP2dOMg@public.gmane.org \
    --cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
    --cc=tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.