From: Glauber Costa <glommer-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org>
To: Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>
Cc: Glauber Costa <glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>,
Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
Greg Thelen <gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org,
Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>,
Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org
Subject: Re: [PATCH v7 00/34] kmemcg shrinkers
Date: Tue, 21 May 2013 11:27:17 +0400 [thread overview]
Message-ID: <519B21D5.9090109@parallels.com> (raw)
In-Reply-To: <20130521071800.GN24543@dastard>
On 05/21/2013 11:18 AM, Dave Chinner wrote:
> On Tue, May 21, 2013 at 11:03:33AM +0400, Glauber Costa wrote:
>> On 05/20/2013 12:06 AM, Glauber Costa wrote:
>>> Initial notes:
>>> ==============
>>>
>>> Please pay attention to new patches that are debuting in this series. Patch1
>>> changes our unused countries for int to long, since Dave noticed that it wasn't
>>> being enough in some cases. Aside from that, the major change is that we now
>>> compute and keep deferred work per-node (Patch13). The biggest effect of this,
>>> is that to avoid storing a new nodemask in the stack, I am passing only the
>>> node id down to the API. This means that the lru API *does not* take a nodemask
>>> any longer, which in turn, makes it simpler.
>>>
>>> I deeply considered this matter, and decided this would be the best way to go.
>>> It is not different from what I have already done for memcgs: Only a single one
>>> is passed down, and the complexity of scanning them is moved upwards to the
>>> caller, where all the scanning logic should belong anyway.
>>>
>>> If you want, you can also grab from branch "kmemcg-lru-shrinker" at:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/glommer/memcg.git
>>>
>>> I hope the performance problems are all gone. My testing now shows a smoother
>>> and steady state for the objects during the lifetime of the workload, and
>>> postmark numbers are closer to base, although we do deviate a bit.
>>>
>>
>> Mel, Dave, et. al.
>>
>> I have applied some more fixes for things I have found here and there as
>> a result of a new round of testing. I won't post the result here until
>> Thursday or Friday, to avoid patchbombing you guys. In the meantime I
>> will be merging comments I receive from this version.
>>
>> My git tree is up to date, so if you want to test it further, please
>> pick that up.
>
> Will do. I hope to do some testing of it tommorrow.
>
>> I am attaching the result of my postmark run. I think the results look
>> really good now.
>
> What's version and command line you are using - I'll see if i can
> reproduce the same results on my test system....
>
I am using Mel's mmtest. So I cloned it, changed to config to run the
postmark benchmark set TEST_PARTITION to my disk, TEST_FILESYSTEM to
ext3 (specially that fsmark was already running xfs with your script),
and then ./run-mmtests.sh <name_of_test>
WARNING: multiple messages have this Message-ID (diff)
From: Glauber Costa <glommer-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org>
To: Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>
Cc: Glauber Costa <glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>,
Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>,
<linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>,
<cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
Greg Thelen <gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
<kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org>,
Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>,
Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
<linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
<hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH v7 00/34] kmemcg shrinkers
Date: Tue, 21 May 2013 11:27:17 +0400 [thread overview]
Message-ID: <519B21D5.9090109@parallels.com> (raw)
In-Reply-To: <20130521071800.GN24543@dastard>
On 05/21/2013 11:18 AM, Dave Chinner wrote:
> On Tue, May 21, 2013 at 11:03:33AM +0400, Glauber Costa wrote:
>> On 05/20/2013 12:06 AM, Glauber Costa wrote:
>>> Initial notes:
>>> ==============
>>>
>>> Please pay attention to new patches that are debuting in this series. Patch1
>>> changes our unused countries for int to long, since Dave noticed that it wasn't
>>> being enough in some cases. Aside from that, the major change is that we now
>>> compute and keep deferred work per-node (Patch13). The biggest effect of this,
>>> is that to avoid storing a new nodemask in the stack, I am passing only the
>>> node id down to the API. This means that the lru API *does not* take a nodemask
>>> any longer, which in turn, makes it simpler.
>>>
>>> I deeply considered this matter, and decided this would be the best way to go.
>>> It is not different from what I have already done for memcgs: Only a single one
>>> is passed down, and the complexity of scanning them is moved upwards to the
>>> caller, where all the scanning logic should belong anyway.
>>>
>>> If you want, you can also grab from branch "kmemcg-lru-shrinker" at:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/glommer/memcg.git
>>>
>>> I hope the performance problems are all gone. My testing now shows a smoother
>>> and steady state for the objects during the lifetime of the workload, and
>>> postmark numbers are closer to base, although we do deviate a bit.
>>>
>>
>> Mel, Dave, et. al.
>>
>> I have applied some more fixes for things I have found here and there as
>> a result of a new round of testing. I won't post the result here until
>> Thursday or Friday, to avoid patchbombing you guys. In the meantime I
>> will be merging comments I receive from this version.
>>
>> My git tree is up to date, so if you want to test it further, please
>> pick that up.
>
> Will do. I hope to do some testing of it tommorrow.
>
>> I am attaching the result of my postmark run. I think the results look
>> really good now.
>
> What's version and command line you are using - I'll see if i can
> reproduce the same results on my test system....
>
I am using Mel's mmtest. So I cloned it, changed to config to run the
postmark benchmark set TEST_PARTITION to my disk, TEST_FILESYSTEM to
ext3 (specially that fsmark was already running xfs with your script),
and then ./run-mmtests.sh <name_of_test>
WARNING: multiple messages have this Message-ID (diff)
From: Glauber Costa <glommer@parallels.com>
To: Dave Chinner <david@fromorbit.com>
Cc: Glauber Costa <glommer@openvz.org>, Mel Gorman <mgorman@suse.de>,
linux-mm@kvack.org, cgroups@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Greg Thelen <gthelen@google.com>,
kamezawa.hiroyu@jp.fujitsu.com, Michal Hocko <mhocko@suse.cz>,
Johannes Weiner <hannes@cmpxchg.org>,
linux-fsdevel@vger.kernel.org, hughd@google.com
Subject: Re: [PATCH v7 00/34] kmemcg shrinkers
Date: Tue, 21 May 2013 11:27:17 +0400 [thread overview]
Message-ID: <519B21D5.9090109@parallels.com> (raw)
In-Reply-To: <20130521071800.GN24543@dastard>
On 05/21/2013 11:18 AM, Dave Chinner wrote:
> On Tue, May 21, 2013 at 11:03:33AM +0400, Glauber Costa wrote:
>> On 05/20/2013 12:06 AM, Glauber Costa wrote:
>>> Initial notes:
>>> ==============
>>>
>>> Please pay attention to new patches that are debuting in this series. Patch1
>>> changes our unused countries for int to long, since Dave noticed that it wasn't
>>> being enough in some cases. Aside from that, the major change is that we now
>>> compute and keep deferred work per-node (Patch13). The biggest effect of this,
>>> is that to avoid storing a new nodemask in the stack, I am passing only the
>>> node id down to the API. This means that the lru API *does not* take a nodemask
>>> any longer, which in turn, makes it simpler.
>>>
>>> I deeply considered this matter, and decided this would be the best way to go.
>>> It is not different from what I have already done for memcgs: Only a single one
>>> is passed down, and the complexity of scanning them is moved upwards to the
>>> caller, where all the scanning logic should belong anyway.
>>>
>>> If you want, you can also grab from branch "kmemcg-lru-shrinker" at:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/glommer/memcg.git
>>>
>>> I hope the performance problems are all gone. My testing now shows a smoother
>>> and steady state for the objects during the lifetime of the workload, and
>>> postmark numbers are closer to base, although we do deviate a bit.
>>>
>>
>> Mel, Dave, et. al.
>>
>> I have applied some more fixes for things I have found here and there as
>> a result of a new round of testing. I won't post the result here until
>> Thursday or Friday, to avoid patchbombing you guys. In the meantime I
>> will be merging comments I receive from this version.
>>
>> My git tree is up to date, so if you want to test it further, please
>> pick that up.
>
> Will do. I hope to do some testing of it tommorrow.
>
>> I am attaching the result of my postmark run. I think the results look
>> really good now.
>
> What's version and command line you are using - I'll see if i can
> reproduce the same results on my test system....
>
I am using Mel's mmtest. So I cloned it, changed to config to run the
postmark benchmark set TEST_PARTITION to my disk, TEST_FILESYSTEM to
ext3 (specially that fsmark was already running xfs with your script),
and then ./run-mmtests.sh <name_of_test>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2013-05-21 7:27 UTC|newest]
Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-19 20:06 [PATCH v7 00/34] kmemcg shrinkers Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` [PATCH v7 01/34] fs: bump inode and dentry counters to long Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` [PATCH v7 02/34] super: fix calculation of shrinkable objects for small numbers Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` [PATCH v7 03/34] dcache: convert dentry_stat.nr_unused to per-cpu counters Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` [PATCH v7 04/34] dentry: move to per-sb LRU locks Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` [PATCH v7 05/34] dcache: remove dentries from LRU before putting on dispose list Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` [PATCH v7 06/34] mm: new shrinker API Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:06 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 07/34] shrinker: convert superblock shrinkers to new API Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-20 16:39 ` Glauber Costa
2013-05-20 16:39 ` Glauber Costa
2013-05-20 23:40 ` Dave Chinner
2013-05-19 20:07 ` [PATCH v7 08/34] list: add a new LRU list type Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 09/34] inode: convert inode lru list to generic lru list code Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 10/34] dcache: convert to use new lru list infrastructure Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 11/34] list_lru: per-node " Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 12/34] shrinker: add node awareness Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 13/34] vmscan: per-node deferred work Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 14/34] list_lru: per-node API Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 15/34] fs: convert inode and dentry shrinking to be node aware Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 16/34] xfs: convert buftarg LRU to generic code Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 17/34] xfs: convert dquot cache lru to list_lru Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 18/34] fs: convert fs shrinkers to new scan/count API Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-20 8:25 ` Steven Whitehouse
2013-05-20 13:46 ` Glauber Costa
2013-05-20 13:46 ` Glauber Costa
2013-05-20 13:46 ` Glauber Costa
2013-05-20 15:25 ` Glauber Costa
2013-05-20 15:25 ` Glauber Costa
2013-05-20 15:25 ` Glauber Costa
2013-05-20 23:38 ` Dave Chinner
2013-05-20 23:42 ` Glauber Costa
2013-05-20 23:42 ` Glauber Costa
2013-05-20 23:42 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 19/34] drivers: convert shrinkers to new count/scan API Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-06-03 20:03 ` Kent Overstreet
2013-06-04 9:06 ` Glauber Costa
2013-06-04 9:10 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 20/34] i915: bail out earlier when shrinker cannot acquire mutex Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 21/34] shrinker: convert remaining shrinkers to count/scan API Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 22/34] hugepage: convert huge zero page shrinker to new shrinker API Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 23/34] shrinker: Kill old ->shrink API Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 24/34] vmscan: also shrink slab in memcg pressure Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 25/34] memcg,list_lru: duplicate LRUs upon kmemcg creation Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 26/34] lru: add an element to a memcg list Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 27/34] list_lru: per-memcg walks Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 28/34] memcg: per-memcg kmem shrinking Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 29/34] memcg: scan cache objects hierarchically Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 30/34] vmscan: take at least one pass with shrinkers Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 31/34] super: targeted memcg reclaim Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 32/34] memcg: move initialization to memcg creation Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 33/34] vmpressure: in-kernel notifications Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` [PATCH v7 34/34] memcg: reap dead memcgs upon global memory pressure Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-19 20:07 ` Glauber Costa
2013-05-21 7:03 ` [PATCH v7 00/34] kmemcg shrinkers Glauber Costa
2013-05-21 7:03 ` Glauber Costa
2013-05-21 7:18 ` Dave Chinner
2013-05-21 7:27 ` Glauber Costa [this message]
2013-05-21 7:27 ` Glauber Costa
2013-05-21 7:27 ` Glauber Costa
2013-05-22 6:26 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=519B21D5.9090109@parallels.com \
--to=glommer-bzqdu9zft3wakbo8gow8eq@public.gmane.org \
--cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
--cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org \
--cc=glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org \
--cc=gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
--cc=hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org \
--cc=hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
--cc=kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org \
--cc=linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
--cc=mgorman-l3A5Bk7waGM@public.gmane.org \
--cc=mhocko-AlSwsSmVLrQ@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.