All of lore.kernel.org
 help / color / mirror / Atom feed
From: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
To: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Cc: Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>,
	Greg Thelen <gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Andrew Morton
	<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
	Linux MM <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>,
	Cgroups <cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	LKML <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: [PATCH] memcg: oom: ignore oom warnings from memory.max
Date: Tue, 5 May 2020 12:57:34 -0400	[thread overview]
Message-ID: <20200505165734.GD58018@cmpxchg.org> (raw)
In-Reply-To: <CALvZod5H-fDbvu73=hkzN0Man_+03ZW0d5zc1N0YObVHSiy0Tw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On Tue, May 05, 2020 at 08:03:17AM -0700, Shakeel Butt wrote:
> On Tue, May 5, 2020 at 12:13 AM Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> > So instead of potentially removing a useful information can we focus on
> > the remote charging side of the problem and deal with it in a sensible
> > way? That would make memory.high usable for your usecase and I still
> > believe that this is what you should be using in the first place.
> 
> We talked about this at LSFMM'19 and I think the decision was to not
> fix high reclaim for remote memcg until it will be an actual issue. I
> suppose now we can treat it as an actual issue.
> 
> There are a couple of open questions:
> 1) Should the remote chargers be throttled and do the high reclaim?

I would say it depends on the caller. We may need both: a centralized
charger like a kthread that does work on behalf of many cgroups should
not be throttled, or we risk priority inversions - maybe not even held
up for reclaim, although that would need some thinking about
memory.max as well - whether remote charges should be allowed to
overrun memory.max temporarily, and we punt all reclaim from such
contexts to a worker. (Preferrably, in the future, with all the cpu
cycles of that worker accounted to the cgroup. We want that for
kswapd-style background reclaim as well).

We do similar things with IO for example, when offloading IO to a
central kthread. The kthread is allowed to proceed past a cgroup's
allowance in order to make forward progress, and the cgroup will be
forced to pay up retroactively when it comes back for more of the
resource.

WARNING: multiple messages have this Message-ID (diff)
From: Johannes Weiner <hannes@cmpxchg.org>
To: Shakeel Butt <shakeelb@google.com>
Cc: Michal Hocko <mhocko@kernel.org>, Roman Gushchin <guro@fb.com>,
	Greg Thelen <gthelen@google.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linux MM <linux-mm@kvack.org>, Cgroups <cgroups@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] memcg: oom: ignore oom warnings from memory.max
Date: Tue, 5 May 2020 12:57:34 -0400	[thread overview]
Message-ID: <20200505165734.GD58018@cmpxchg.org> (raw)
In-Reply-To: <CALvZod5H-fDbvu73=hkzN0Man_+03ZW0d5zc1N0YObVHSiy0Tw@mail.gmail.com>

On Tue, May 05, 2020 at 08:03:17AM -0700, Shakeel Butt wrote:
> On Tue, May 5, 2020 at 12:13 AM Michal Hocko <mhocko@kernel.org> wrote:
> > So instead of potentially removing a useful information can we focus on
> > the remote charging side of the problem and deal with it in a sensible
> > way? That would make memory.high usable for your usecase and I still
> > believe that this is what you should be using in the first place.
> 
> We talked about this at LSFMM'19 and I think the decision was to not
> fix high reclaim for remote memcg until it will be an actual issue. I
> suppose now we can treat it as an actual issue.
> 
> There are a couple of open questions:
> 1) Should the remote chargers be throttled and do the high reclaim?

I would say it depends on the caller. We may need both: a centralized
charger like a kthread that does work on behalf of many cgroups should
not be throttled, or we risk priority inversions - maybe not even held
up for reclaim, although that would need some thinking about
memory.max as well - whether remote charges should be allowed to
overrun memory.max temporarily, and we punt all reclaim from such
contexts to a worker. (Preferrably, in the future, with all the cpu
cycles of that worker accounted to the cgroup. We want that for
kswapd-style background reclaim as well).

We do similar things with IO for example, when offloading IO to a
central kthread. The kthread is allowed to proceed past a cgroup's
allowance in order to make forward progress, and the cgroup will be
forced to pay up retroactively when it comes back for more of the
resource.


  parent reply	other threads:[~2020-05-05 16:57 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-30 18:27 [PATCH] memcg: oom: ignore oom warnings from memory.max Shakeel Butt
2020-04-30 19:06 ` Roman Gushchin
2020-04-30 19:06   ` Roman Gushchin
     [not found]   ` <20200430190610.GD339283-cx5fftMpWqeCjSd+JxjunQ2O0Ztt9esIQQ4Iyu8u01E@public.gmane.org>
2020-04-30 19:30     ` Johannes Weiner
2020-04-30 19:30       ` Johannes Weiner
     [not found]       ` <20200430193049.GB2436-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-30 20:23         ` Roman Gushchin
2020-04-30 20:23           ` Roman Gushchin
2020-04-30 19:31     ` Shakeel Butt
2020-04-30 19:31       ` Shakeel Butt
2020-04-30 19:29 ` Johannes Weiner
     [not found]   ` <20200430192907.GA2436-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-04-30 20:20     ` Shakeel Butt
2020-04-30 20:20       ` Shakeel Butt
     [not found]       ` <CALvZod6Gatw+sX7_hsVPB-O2yMt-ygXUSweQbHwxZHgwdEth4Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-04  6:57         ` Michal Hocko
2020-05-04  6:57           ` Michal Hocko
     [not found]           ` <20200504065701.GB22838-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-05-04 13:54             ` Shakeel Butt
2020-05-04 13:54               ` Shakeel Butt
2020-05-01  1:39 ` Yafang Shao
2020-05-01  2:04   ` Shakeel Butt
     [not found]     ` <CALvZod4gZ_Q=Kuh-Fx-EsKzxmtPKy5xu+gpPpjV6MYW4Ku=JbA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-01  2:12       ` Yafang Shao
2020-05-01  2:12         ` Yafang Shao
     [not found]   ` <CALOAHbC4WY00yQ46b8CFqVQ3S=JSJxE2HR00TtMqXOWLRPRZ8w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-04  7:03     ` Michal Hocko
2020-05-04  7:03       ` Michal Hocko
     [not found]       ` <20200504070301.GC22838-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-05-04  7:26         ` Yafang Shao
2020-05-04  7:26           ` Yafang Shao
2020-05-04  7:35           ` Michal Hocko
     [not found]             ` <20200504073549.GE22838-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-05-04  7:40               ` Yafang Shao
2020-05-04  7:40                 ` Yafang Shao
     [not found]                 ` <CALOAHbDd8hjURSi9jAdjQTbSLDvu9vkOkjS1hZNn04G8Yj7WdQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-04  8:03                   ` Michal Hocko
2020-05-04  8:03                     ` Michal Hocko
     [not found] ` <20200430182712.237526-1-shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2020-05-04  6:56   ` Michal Hocko
2020-05-04  6:56     ` Michal Hocko
2020-05-04 13:54     ` Shakeel Butt
2020-05-04 14:11       ` Michal Hocko
     [not found]         ` <20200504141136.GR22838-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-05-04 14:53           ` Shakeel Butt
2020-05-04 14:53             ` Shakeel Butt
2020-05-04 15:00             ` Michal Hocko
     [not found]               ` <20200504150052.GT22838-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2020-05-04 15:35                 ` Shakeel Butt
2020-05-04 15:35                   ` Shakeel Butt
     [not found]                   ` <CALvZod7EeQm-T4dsBddfMY_szYw3m8gRh5R5GfjQiuQAtCocug-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-04 15:39                     ` Yafang Shao
2020-05-04 15:39                       ` Yafang Shao
2020-05-04 16:06                     ` Michal Hocko
2020-05-04 16:06                       ` Michal Hocko
2020-05-04 19:23                       ` Shakeel Butt
     [not found]                         ` <CALvZod79hWns9366B+8ZK2Roz8c+vkdA80HqFNMep56_pumdRQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-05  7:13                           ` Michal Hocko
2020-05-05  7:13                             ` Michal Hocko
2020-05-05 15:03                             ` Shakeel Butt
     [not found]                               ` <CALvZod5H-fDbvu73=hkzN0Man_+03ZW0d5zc1N0YObVHSiy0Tw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-05 16:57                                 ` Johannes Weiner [this message]
2020-05-05 16:57                                   ` Johannes Weiner
2020-05-05 15:27                           ` Johannes Weiner
2020-05-05 15:27                             ` Johannes Weiner
     [not found]                             ` <20200505152712.GB58018-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-05-05 15:35                               ` Shakeel Butt
2020-05-05 15:35                                 ` Shakeel Butt
     [not found]                                 ` <CALvZod48mu1w=fjpD=GXqCMdNq_8rExQ177_EP+Lx+TvR6fw+w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-05 15:49                                   ` Michal Hocko
2020-05-05 15:49                                     ` Michal Hocko
2020-05-05 16:40                                   ` Johannes Weiner
2020-05-05 16:40                                     ` Johannes Weiner
     [not found]       ` <CALvZod5Ao2PEFPEOckW6URBfxisp9nNpNeon1GuctuHehqk_6Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-04 14:20         ` Tetsuo Handa
2020-05-04 14:20           ` Tetsuo Handa
2020-05-04 14:57           ` Shakeel Butt
     [not found]             ` <CALvZod5T9pYG1xVHqNM=c68jgKPVXtKjuvV0DSAR+Ld_Mm1c4A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-05-04 15:44               ` Tetsuo Handa
2020-05-04 15:44                 ` Tetsuo Handa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200505165734.GD58018@cmpxchg.org \
    --to=hannes-druugvl0lcnafugrpc6u6w@public.gmane.org \
    --cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
    --cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=guro-b10kYP2dOMg@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
    --cc=mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.