All of lore.kernel.org
 help / color / mirror / Atom feed
From: Alistair Popple <apopple-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
To: Jason Gunthorpe <jgg-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
Cc: Chris Li <chrisl-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"T.J. Mercier"
	<tjmercier-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	lsf-pc-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
	linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
	cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	Yosry Ahmed <yosryahmed-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Muchun Song <muchun.song-fxUVXftIFDnyG1zEObXtfA@public.gmane.org>,
	Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
	Roman Gushchin
	<roman.gushchin-fxUVXftIFDnyG1zEObXtfA@public.gmane.org>,
	Kalesh Singh
	<kaleshsingh-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Yu Zhao <yuzhao-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Subject: Re: [LSF/MM/BPF TOPIC] Reducing zombie memcgs
Date: Tue, 16 May 2023 22:21:10 +1000	[thread overview]
Message-ID: <87y1lo8nwp.fsf@nvidia.com> (raw)
In-Reply-To: <ZF6rACJzilA06oe+-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>


Jason Gunthorpe <jgg-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org> writes:

> On Fri, May 12, 2023 at 06:45:13PM +1000, Alistair Popple wrote:
>
>> However review comments suggested it needed to be added as part of
>> memcg. As soon as we do that we have to address how we deal with shared
>> memory. If we stick with the original RLIMIT proposal this discussion
>> goes away, but based on feedback I think I need to at least investigate
>> integrating it into memcg to get anything merged.
>
> Personally I don't see how we can effectively solve the per-page
> problem without also tracking all the owning memcgs for every
> page. This means giving each struct page an array of memcgs
>
> I suspect this will be too expensive to be realistically
> implementable.

Yep, agree with that. Tracking the list of memcgs was the main problem
that prevented this.

> If it is done then we may not even need a pin controller on its own as
> the main memcg should capture most of it. (althought it doesn't
> distinguish between movable/swappable and non-swappable memory)
>
> But this is all being done for the libvirt people, so it would be good
> to involve them

Do you know of anyone specifically there that is interested in this?
I've rebased my series on latest upstream and am about to resend it so
would be good to get some feedback from them.

Thanks.

> Jason


WARNING: multiple messages have this Message-ID (diff)
From: Alistair Popple <apopple@nvidia.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Chris Li <chrisl@kernel.org>,
	"T.J. Mercier" <tjmercier@google.com>,
	lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org,
	cgroups@vger.kernel.org, Yosry Ahmed <yosryahmed@google.com>,
	Tejun Heo <tj@kernel.org>, Shakeel Butt <shakeelb@google.com>,
	Muchun Song <muchun.song@linux.dev>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	Kalesh Singh <kaleshsingh@google.com>,
	Yu Zhao <yuzhao@google.com>
Subject: Re: [LSF/MM/BPF TOPIC] Reducing zombie memcgs
Date: Tue, 16 May 2023 22:21:10 +1000	[thread overview]
Message-ID: <87y1lo8nwp.fsf@nvidia.com> (raw)
In-Reply-To: <ZF6rACJzilA06oe+@nvidia.com>


Jason Gunthorpe <jgg@nvidia.com> writes:

> On Fri, May 12, 2023 at 06:45:13PM +1000, Alistair Popple wrote:
>
>> However review comments suggested it needed to be added as part of
>> memcg. As soon as we do that we have to address how we deal with shared
>> memory. If we stick with the original RLIMIT proposal this discussion
>> goes away, but based on feedback I think I need to at least investigate
>> integrating it into memcg to get anything merged.
>
> Personally I don't see how we can effectively solve the per-page
> problem without also tracking all the owning memcgs for every
> page. This means giving each struct page an array of memcgs
>
> I suspect this will be too expensive to be realistically
> implementable.

Yep, agree with that. Tracking the list of memcgs was the main problem
that prevented this.

> If it is done then we may not even need a pin controller on its own as
> the main memcg should capture most of it. (althought it doesn't
> distinguish between movable/swappable and non-swappable memory)
>
> But this is all being done for the libvirt people, so it would be good
> to involve them

Do you know of anyone specifically there that is interested in this?
I've rebased my series on latest upstream and am about to resend it so
would be good to get some feedback from them.

Thanks.

> Jason



  parent reply	other threads:[~2023-05-16 12:21 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-11 23:36 [LSF/MM/BPF TOPIC] Reducing zombie memcgs T.J. Mercier
2023-04-11 23:36 ` T.J. Mercier
     [not found] ` <CABdmKX2M6koq4Q0Cmp_-=wbP0Qa190HdEGGaHfxNS05gAkUtPA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2023-04-11 23:48   ` Yosry Ahmed
2023-04-11 23:48     ` Yosry Ahmed
     [not found]     ` <CAJD7tkZw9uVPe5KH2xrihsv5nDmExJmkmsUPYP6Npvv6Q0NcVw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2023-04-25 11:36       ` Yosry Ahmed
2023-04-25 11:36         ` Yosry Ahmed
     [not found]         ` <CAJD7tkb56gR0X5v3VHfmk3az3bOz=wF2jhEi+7Eek0J8XXBeWQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2023-04-25 18:42           ` Waiman Long
2023-04-25 18:42             ` Waiman Long
     [not found]             ` <27e15be8-d0eb-ed32-a0ec-5ec9b59f1f27-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2023-04-25 18:53               ` Yosry Ahmed
2023-04-25 18:53                 ` Yosry Ahmed
     [not found]                 ` <CAJD7tkb1W0bP3AU9KepOYPx-AD-fMKSfUhj_Cmth63RS9umMsg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2023-04-26 20:15                   ` Waiman Long
2023-04-26 20:15                     ` Waiman Long
2023-05-01 16:38           ` Roman Gushchin
2023-05-01 16:38             ` Roman Gushchin
2023-05-02  7:18             ` Yosry Ahmed
2023-05-02  7:18               ` Yosry Ahmed
2023-05-02 20:02             ` Yosry Ahmed
2023-05-02 20:02               ` Yosry Ahmed
2023-05-03 22:15   ` Chris Li
2023-05-03 22:15     ` Chris Li
     [not found]     ` <ZFLdDyHoIdJSXJt+-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2023-05-04 11:58       ` Alistair Popple
2023-05-04 11:58         ` Alistair Popple
     [not found]         ` <874josz4rd.fsf-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
2023-05-04 15:31           ` Chris Li
2023-05-04 15:31             ` Chris Li
     [not found]             ` <ZFPP71czDDxMPLQK-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2023-05-05 13:53               ` Alistair Popple
2023-05-05 13:53                 ` Alistair Popple
     [not found]                 ` <877ctm518f.fsf-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
2023-05-06 22:49                   ` Chris Li
2023-05-06 22:49                     ` Chris Li
     [not found]                     ` <ZFbZZPkSpsKMe8iR-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2023-05-08  8:17                       ` Alistair Popple
2023-05-08  8:17                         ` Alistair Popple
     [not found]                         ` <87ttwnkzap.fsf-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
2023-05-10 14:51                           ` Chris Li
2023-05-10 14:51                             ` Chris Li
     [not found]                             ` <ZFuvhP5qGPivokc0-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2023-05-12  8:45                               ` Alistair Popple
2023-05-12  8:45                                 ` Alistair Popple
     [not found]                                 ` <87jzxe9baj.fsf-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
2023-05-12 21:09                                   ` Jason Gunthorpe
2023-05-12 21:09                                     ` Jason Gunthorpe
     [not found]                                     ` <ZF6rACJzilA06oe+-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
2023-05-16 12:21                                       ` Alistair Popple [this message]
2023-05-16 12:21                                         ` Alistair Popple
     [not found]                                         ` <87y1lo8nwp.fsf-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
2023-05-19 15:47                                           ` Jason Gunthorpe
2023-05-19 15:47                                             ` Jason Gunthorpe
2023-05-20 15:09                                       ` Chris Li
2023-05-20 15:09                                         ` Chris Li
2023-05-20 15:31                                   ` Chris Li
2023-05-20 15:31                                     ` Chris Li
     [not found]                                     ` <ZGjntWoAfgyT0doo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2023-05-29 19:31                                       ` Jason Gunthorpe
2023-05-29 19:31                                         ` Jason Gunthorpe
2023-05-04 17:02       ` Shakeel Butt
2023-05-04 17:02         ` Shakeel Butt
     [not found]         ` <CALvZod4=+ANT6UR5h7Cp+0hKkVx6tPAaRa5iqBF=L2VBdMKERQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2023-05-04 17:36           ` Chris Li
2023-05-04 17:36             ` Chris Li
2023-05-12  3:08 ` Yosry Ahmed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87y1lo8nwp.fsf@nvidia.com \
    --to=apopple-ddmlm1+adcrqt0dzr+alfa@public.gmane.org \
    --cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=chrisl-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org \
    --cc=jgg-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org \
    --cc=kaleshsingh-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
    --cc=lsf-pc-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
    --cc=muchun.song-fxUVXftIFDnyG1zEObXtfA@public.gmane.org \
    --cc=roman.gushchin-fxUVXftIFDnyG1zEObXtfA@public.gmane.org \
    --cc=shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=tjmercier-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=yosryahmed-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=yuzhao-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.