All of lore.kernel.org
 help / color / mirror / Atom feed
From: Roman Gushchin <roman.gushchin-fxUVXftIFDnyG1zEObXtfA@public.gmane.org>
To: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Cc: "Johannes Weiner"
	<hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
	"Michal Hocko" <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"Muchun Song"
	<songmuchun-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org>,
	"Michal Koutný" <mkoutny-IBi9RG/b67k@public.gmane.org>,
	"Eric Dumazet" <edumazet-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	"Soheil Hassas Yeganeh"
	<soheil-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	"Feng Tang" <feng.tang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
	"Oliver Sang"
	<oliver.sang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
	"Andrew Morton"
	<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
	lkp-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org,
	cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
	netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [PATCH 3/3] memcg: increase MEMCG_CHARGE_BATCH to 64
Date: Mon, 22 Aug 2022 11:37:30 -0700	[thread overview]
Message-ID: <YwPM6o1+pZ2kRyy3@P9FQF9L96D> (raw)
In-Reply-To: <20220822001737.4120417-4-shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>

On Mon, Aug 22, 2022 at 12:17:37AM +0000, Shakeel Butt wrote:
> For several years, MEMCG_CHARGE_BATCH was kept at 32 but with bigger
> machines and the network intensive workloads requiring througput in
> Gbps, 32 is too small and makes the memcg charging path a bottleneck.
> For now, increase it to 64 for easy acceptance to 6.0. We will need to
> revisit this in future for ever increasing demand of higher performance.
> 
> Please note that the memcg charge path drain the per-cpu memcg charge
> stock, so there should not be any oom behavior change.
> 
> To evaluate the impact of this optimization, on a 72 CPUs machine, we
> ran the following workload in a three level of cgroup hierarchy with top
> level having min and low setup appropriately. More specifically
> memory.min equal to size of netperf binary and memory.low double of
> that.
> 
>  $ netserver -6
>  # 36 instances of netperf with following params
>  $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K
> 
> Results (average throughput of netperf):
> Without (6.0-rc1)       10482.7 Mbps
> With patch              17064.7 Mbps (62.7% improvement)
> 
> With the patch, the throughput improved by 62.7%.

This is pretty significant!

Acked-by: Roman Gushchin <roman.gushchin-fxUVXftIFDnyG1zEObXtfA@public.gmane.org>

I wonder only if we want to make it configurable (Idk a sysctl or maybe
a config option) and close the topic.

Thanks!

WARNING: multiple messages have this Message-ID (diff)
From: Roman Gushchin <roman.gushchin@linux.dev>
To: lkp@lists.01.org
Subject: Re: [PATCH 3/3] memcg: increase MEMCG_CHARGE_BATCH to 64
Date: Mon, 22 Aug 2022 11:37:30 -0700	[thread overview]
Message-ID: <YwPM6o1+pZ2kRyy3@P9FQF9L96D> (raw)
In-Reply-To: <20220822001737.4120417-4-shakeelb@google.com>

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

On Mon, Aug 22, 2022 at 12:17:37AM +0000, Shakeel Butt wrote:
> For several years, MEMCG_CHARGE_BATCH was kept at 32 but with bigger
> machines and the network intensive workloads requiring througput in
> Gbps, 32 is too small and makes the memcg charging path a bottleneck.
> For now, increase it to 64 for easy acceptance to 6.0. We will need to
> revisit this in future for ever increasing demand of higher performance.
> 
> Please note that the memcg charge path drain the per-cpu memcg charge
> stock, so there should not be any oom behavior change.
> 
> To evaluate the impact of this optimization, on a 72 CPUs machine, we
> ran the following workload in a three level of cgroup hierarchy with top
> level having min and low setup appropriately. More specifically
> memory.min equal to size of netperf binary and memory.low double of
> that.
> 
>  $ netserver -6
>  # 36 instances of netperf with following params
>  $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K
> 
> Results (average throughput of netperf):
> Without (6.0-rc1)       10482.7 Mbps
> With patch              17064.7 Mbps (62.7% improvement)
> 
> With the patch, the throughput improved by 62.7%.

This is pretty significant!

Acked-by: Roman Gushchin <roman.gushchin@linux.dev>

I wonder only if we want to make it configurable (Idk a sysctl or maybe
a config option) and close the topic.

Thanks!

WARNING: multiple messages have this Message-ID (diff)
From: Roman Gushchin <roman.gushchin@linux.dev>
To: Shakeel Butt <shakeelb@google.com>
Cc: "Johannes Weiner" <hannes@cmpxchg.org>,
	"Michal Hocko" <mhocko@kernel.org>,
	"Muchun Song" <songmuchun@bytedance.com>,
	"Michal Koutný" <mkoutny@suse.com>,
	"Eric Dumazet" <edumazet@google.com>,
	"Soheil Hassas Yeganeh" <soheil@google.com>,
	"Feng Tang" <feng.tang@intel.com>,
	"Oliver Sang" <oliver.sang@intel.com>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	lkp@lists.01.org, cgroups@vger.kernel.org, linux-mm@kvack.org,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 3/3] memcg: increase MEMCG_CHARGE_BATCH to 64
Date: Mon, 22 Aug 2022 11:37:30 -0700	[thread overview]
Message-ID: <YwPM6o1+pZ2kRyy3@P9FQF9L96D> (raw)
In-Reply-To: <20220822001737.4120417-4-shakeelb@google.com>

On Mon, Aug 22, 2022 at 12:17:37AM +0000, Shakeel Butt wrote:
> For several years, MEMCG_CHARGE_BATCH was kept at 32 but with bigger
> machines and the network intensive workloads requiring througput in
> Gbps, 32 is too small and makes the memcg charging path a bottleneck.
> For now, increase it to 64 for easy acceptance to 6.0. We will need to
> revisit this in future for ever increasing demand of higher performance.
> 
> Please note that the memcg charge path drain the per-cpu memcg charge
> stock, so there should not be any oom behavior change.
> 
> To evaluate the impact of this optimization, on a 72 CPUs machine, we
> ran the following workload in a three level of cgroup hierarchy with top
> level having min and low setup appropriately. More specifically
> memory.min equal to size of netperf binary and memory.low double of
> that.
> 
>  $ netserver -6
>  # 36 instances of netperf with following params
>  $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K
> 
> Results (average throughput of netperf):
> Without (6.0-rc1)       10482.7 Mbps
> With patch              17064.7 Mbps (62.7% improvement)
> 
> With the patch, the throughput improved by 62.7%.

This is pretty significant!

Acked-by: Roman Gushchin <roman.gushchin@linux.dev>

I wonder only if we want to make it configurable (Idk a sysctl or maybe
a config option) and close the topic.

Thanks!


  parent reply	other threads:[~2022-08-22 18:37 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-22  0:17 [PATCH 0/3] memcg: optimizatize charge codepath Shakeel Butt
2022-08-22  0:17 ` Shakeel Butt
2022-08-22  0:17 ` Shakeel Butt
     [not found] ` <20220822001737.4120417-1-shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2022-08-22  0:17   ` [PATCH 1/3] mm: page_counter: remove unneeded atomic ops for low/min Shakeel Butt
2022-08-22  0:17     ` Shakeel Butt
2022-08-22  0:17     ` Shakeel Butt
     [not found]     ` <20220822001737.4120417-2-shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2022-08-22  0:20       ` Soheil Hassas Yeganeh
2022-08-22  0:20         ` Soheil Hassas Yeganeh
2022-08-22  0:20         ` Soheil Hassas Yeganeh
2022-08-22  2:39       ` Feng Tang
2022-08-22  2:39         ` Feng Tang
2022-08-22  2:39         ` Feng Tang
2022-08-22 18:23       ` Roman Gushchin
2022-08-22 18:23         ` Roman Gushchin
2022-08-22 18:23         ` Roman Gushchin
2022-08-22  9:55     ` Michal Hocko
2022-08-22  9:55       ` Michal Hocko
     [not found]       ` <YwNSlZFPMgclrSCz-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2022-08-22 10:18         ` Michal Hocko
2022-08-22 10:18           ` Michal Hocko
2022-08-22 10:18           ` Michal Hocko
2022-08-22 14:55           ` Shakeel Butt
2022-08-22 14:55             ` Shakeel Butt
2022-08-22 15:20             ` Michal Hocko
2022-08-22 15:20               ` Michal Hocko
     [not found]               ` <YwOeocdkF/lacpKn-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2022-08-22 16:06                 ` Shakeel Butt
2022-08-22 16:06                   ` Shakeel Butt
2022-08-22 16:06                   ` Shakeel Butt
2022-08-23  9:42                 ` Michal Hocko
2022-08-23  9:42                   ` Michal Hocko
2022-08-23  9:42                   ` Michal Hocko
2022-08-22  0:17 ` [PATCH 2/3] mm: page_counter: rearrange struct page_counter fields Shakeel Butt
2022-08-22  0:17   ` Shakeel Butt
     [not found]   ` <20220822001737.4120417-3-shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2022-08-22  0:24     ` Soheil Hassas Yeganeh
2022-08-22  0:24       ` Soheil Hassas Yeganeh
2022-08-22  0:24       ` Soheil Hassas Yeganeh
2022-08-22  4:55       ` Shakeel Butt
2022-08-22  4:55         ` Shakeel Butt
2022-08-22 13:06         ` Soheil Hassas Yeganeh
2022-08-22 13:06           ` Soheil Hassas Yeganeh
2022-08-22  2:10     ` Feng Tang
2022-08-22  2:10       ` Feng Tang
2022-08-22  2:10       ` Feng Tang
2022-08-22  4:59       ` Shakeel Butt
2022-08-22  4:59         ` Shakeel Butt
2022-08-22  4:59         ` Shakeel Butt
2022-08-22 10:23     ` Michal Hocko
2022-08-22 10:23       ` Michal Hocko
2022-08-22 10:23       ` Michal Hocko
2022-08-22 15:06       ` Shakeel Butt
2022-08-22 15:06         ` Shakeel Butt
     [not found]         ` <CALvZod5pw_7hnH44hdC3rDGQxQB2XATrViNNGosG3FnUoWo-4A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2022-08-22 15:15           ` Michal Hocko
2022-08-22 15:15             ` Michal Hocko
2022-08-22 15:15             ` Michal Hocko
2022-08-22 16:04             ` Shakeel Butt
2022-08-22 16:04               ` Shakeel Butt
     [not found]               ` <CALvZod4whYX+0ZuCGgyKuG-Q_9d0g7N_x+=WXOeB_1TM=3Q7vg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2022-08-22 18:27                 ` Roman Gushchin
2022-08-22 18:27                   ` Roman Gushchin
2022-08-22 18:27                   ` Roman Gushchin
2022-08-22  0:17 ` [PATCH 3/3] memcg: increase MEMCG_CHARGE_BATCH to 64 Shakeel Butt
2022-08-22  0:17   ` Shakeel Butt
2022-08-22  2:30   ` Feng Tang
2022-08-22  2:30     ` Feng Tang
     [not found]   ` <20220822001737.4120417-4-shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2022-08-22  0:24     ` Soheil Hassas Yeganeh
2022-08-22  0:24       ` Soheil Hassas Yeganeh
2022-08-22  0:24       ` Soheil Hassas Yeganeh
2022-08-22 10:47     ` Michal Hocko
2022-08-22 10:47       ` Michal Hocko
2022-08-22 10:47       ` Michal Hocko
2022-08-22 15:09       ` Shakeel Butt
2022-08-22 15:09         ` Shakeel Butt
2022-08-22 15:22         ` Michal Hocko
2022-08-22 15:22           ` Michal Hocko
     [not found]           ` <YwOfP/6PtS8BxNhz-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2022-08-22 16:07             ` Shakeel Butt
2022-08-22 16:07               ` Shakeel Butt
2022-08-22 16:07               ` Shakeel Butt
2022-08-22 18:37     ` Roman Gushchin [this message]
2022-08-22 18:37       ` Roman Gushchin
2022-08-22 18:37       ` Roman Gushchin
2022-08-22 19:34       ` Michal Hocko
2022-08-22 19:34         ` Michal Hocko
2022-08-22 19:34         ` Michal Hocko
     [not found]         ` <YwPZ1lpJ98pZSLmw-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
2022-08-23  2:22           ` Roman Gushchin
2022-08-23  2:22             ` Roman Gushchin
2022-08-23  2:22             ` Roman Gushchin
2022-08-23  4:49             ` Michal Hocko
2022-08-23  4:49               ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YwPM6o1+pZ2kRyy3@P9FQF9L96D \
    --to=roman.gushchin-fxuvxftifdnyg1zeobxtfa@public.gmane.org \
    --cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
    --cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=edumazet-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=feng.tang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    --cc=hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
    --cc=lkp-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org \
    --cc=mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=mkoutny-IBi9RG/b67k@public.gmane.org \
    --cc=netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=oliver.sang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    --cc=shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=soheil-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=songmuchun-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.