From: Glauber Costa <glommer@parallels.com>
To: Dave Chinner <david@fromorbit.com>
Cc: <linux-kernel@vger.kernel.org>, <linux-fsdevel@vger.kernel.org>,
<linux-mm@kvack.org>, <xfs@oss.sgi.com>,
Greg Thelen <gthelen@google.com>, Ying Han <yinghan@google.com>,
Suleiman Souhlal <suleiman@google.com>
Subject: Re: [PATCH 09/19] list_lru: per-node list infrastructure
Date: Fri, 18 Jan 2013 16:13:51 -0800 [thread overview]
Message-ID: <50F9E53F.4090902@parallels.com> (raw)
In-Reply-To: <20130119001042.GR2498@dastard>
On 01/18/2013 04:10 PM, Dave Chinner wrote:
> On Fri, Jan 18, 2013 at 11:10:00AM -0800, Glauber Costa wrote:
>> On 01/18/2013 12:11 AM, Dave Chinner wrote:
>>> On Thu, Jan 17, 2013 at 04:14:10PM -0800, Glauber Costa wrote:
>>>> On 01/17/2013 04:10 PM, Dave Chinner wrote:
>>>>> And then each object uses:
>>>>>
>>>>> struct lru_item {
>>>>> struct list_head global_list;
>>>>> struct list_head memcg_list;
>>>>> }
>>>> by objects you mean dentries, inodes, and the such, right?
>>>
>>> Yup.
>>>
>>>> Would it be acceptable to you?
>>>
>>> If it works the way I think it should, then yes.
>>>
>>>> We've been of course doing our best to avoid increasing the size of the
>>>> objects, therefore this is something we've never mentioned. However, if
>>>> it would be acceptable from the fs POV, this would undoubtedly make our
>>>> life extremely easier.
>>>
>>> I've been trying hard to work out how to avoid increasing the size
>>> of structures as well. But if we can't work out how to implement
>>> something sanely with only a single list head per object to work
>>> from, then increasing the size of objects is something that we need
>>> to consider if it solves all the problems we are trying to solve.
>>>
>>> i.e. if adding a second list head makes the code dumb, simple,
>>> obviously correct and hard to break then IMO it's a no-brainer.
>>> But we have to tick all the right boxes first...
>>>
>>
>> One of our main efforts recently has been trying to reduce memcg impact
>> when it is not in use, even if its compiled in. So what really bothers
>> me here is the fact that we are increasing the size of dentries and
>> inodes no matter what.
>>
>> Still within the idea of exploring the playing field, would an
>> indirection be worth it ?
>> We would increase the total per-object memory usage by 8 bytes instead
>> of 16: the dentry gets a pointer, and a separate allocation for the
>> list_lru.
>
> A separate allocation is really not an option. We can't do an
> allocation in where dentries/inodes/other objects are added to the
> LRU because they are under object state spinlocks, and adding a
> potential memory allocation failure to the "add to lru" case is
> pretty nasty, IMO.
>
That would of course happen on dentry creation time, not lru add time.
It is totally possible since at creation time, we already know if memcg
is enabled or not.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Glauber Costa <glommer@parallels.com>
To: Dave Chinner <david@fromorbit.com>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-mm@kvack.org, xfs@oss.sgi.com,
Greg Thelen <gthelen@google.com>, Ying Han <yinghan@google.com>,
Suleiman Souhlal <suleiman@google.com>
Subject: Re: [PATCH 09/19] list_lru: per-node list infrastructure
Date: Fri, 18 Jan 2013 16:13:51 -0800 [thread overview]
Message-ID: <50F9E53F.4090902@parallels.com> (raw)
In-Reply-To: <20130119001042.GR2498@dastard>
On 01/18/2013 04:10 PM, Dave Chinner wrote:
> On Fri, Jan 18, 2013 at 11:10:00AM -0800, Glauber Costa wrote:
>> On 01/18/2013 12:11 AM, Dave Chinner wrote:
>>> On Thu, Jan 17, 2013 at 04:14:10PM -0800, Glauber Costa wrote:
>>>> On 01/17/2013 04:10 PM, Dave Chinner wrote:
>>>>> And then each object uses:
>>>>>
>>>>> struct lru_item {
>>>>> struct list_head global_list;
>>>>> struct list_head memcg_list;
>>>>> }
>>>> by objects you mean dentries, inodes, and the such, right?
>>>
>>> Yup.
>>>
>>>> Would it be acceptable to you?
>>>
>>> If it works the way I think it should, then yes.
>>>
>>>> We've been of course doing our best to avoid increasing the size of the
>>>> objects, therefore this is something we've never mentioned. However, if
>>>> it would be acceptable from the fs POV, this would undoubtedly make our
>>>> life extremely easier.
>>>
>>> I've been trying hard to work out how to avoid increasing the size
>>> of structures as well. But if we can't work out how to implement
>>> something sanely with only a single list head per object to work
>>> from, then increasing the size of objects is something that we need
>>> to consider if it solves all the problems we are trying to solve.
>>>
>>> i.e. if adding a second list head makes the code dumb, simple,
>>> obviously correct and hard to break then IMO it's a no-brainer.
>>> But we have to tick all the right boxes first...
>>>
>>
>> One of our main efforts recently has been trying to reduce memcg impact
>> when it is not in use, even if its compiled in. So what really bothers
>> me here is the fact that we are increasing the size of dentries and
>> inodes no matter what.
>>
>> Still within the idea of exploring the playing field, would an
>> indirection be worth it ?
>> We would increase the total per-object memory usage by 8 bytes instead
>> of 16: the dentry gets a pointer, and a separate allocation for the
>> list_lru.
>
> A separate allocation is really not an option. We can't do an
> allocation in where dentries/inodes/other objects are added to the
> LRU because they are under object state spinlocks, and adding a
> potential memory allocation failure to the "add to lru" case is
> pretty nasty, IMO.
>
That would of course happen on dentry creation time, not lru add time.
It is totally possible since at creation time, we already know if memcg
is enabled or not.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Glauber Costa <glommer@parallels.com>
To: Dave Chinner <david@fromorbit.com>
Cc: <linux-kernel@vger.kernel.org>, <linux-fsdevel@vger.kernel.org>,
<linux-mm@kvack.org>, <xfs@oss.sgi.com>,
Greg Thelen <gthelen@google.com>, Ying Han <yinghan@google.com>,
Suleiman Souhlal <suleiman@google.com>
Subject: Re: [PATCH 09/19] list_lru: per-node list infrastructure
Date: Fri, 18 Jan 2013 16:13:51 -0800 [thread overview]
Message-ID: <50F9E53F.4090902@parallels.com> (raw)
In-Reply-To: <20130119001042.GR2498@dastard>
On 01/18/2013 04:10 PM, Dave Chinner wrote:
> On Fri, Jan 18, 2013 at 11:10:00AM -0800, Glauber Costa wrote:
>> On 01/18/2013 12:11 AM, Dave Chinner wrote:
>>> On Thu, Jan 17, 2013 at 04:14:10PM -0800, Glauber Costa wrote:
>>>> On 01/17/2013 04:10 PM, Dave Chinner wrote:
>>>>> And then each object uses:
>>>>>
>>>>> struct lru_item {
>>>>> struct list_head global_list;
>>>>> struct list_head memcg_list;
>>>>> }
>>>> by objects you mean dentries, inodes, and the such, right?
>>>
>>> Yup.
>>>
>>>> Would it be acceptable to you?
>>>
>>> If it works the way I think it should, then yes.
>>>
>>>> We've been of course doing our best to avoid increasing the size of the
>>>> objects, therefore this is something we've never mentioned. However, if
>>>> it would be acceptable from the fs POV, this would undoubtedly make our
>>>> life extremely easier.
>>>
>>> I've been trying hard to work out how to avoid increasing the size
>>> of structures as well. But if we can't work out how to implement
>>> something sanely with only a single list head per object to work
>>> from, then increasing the size of objects is something that we need
>>> to consider if it solves all the problems we are trying to solve.
>>>
>>> i.e. if adding a second list head makes the code dumb, simple,
>>> obviously correct and hard to break then IMO it's a no-brainer.
>>> But we have to tick all the right boxes first...
>>>
>>
>> One of our main efforts recently has been trying to reduce memcg impact
>> when it is not in use, even if its compiled in. So what really bothers
>> me here is the fact that we are increasing the size of dentries and
>> inodes no matter what.
>>
>> Still within the idea of exploring the playing field, would an
>> indirection be worth it ?
>> We would increase the total per-object memory usage by 8 bytes instead
>> of 16: the dentry gets a pointer, and a separate allocation for the
>> list_lru.
>
> A separate allocation is really not an option. We can't do an
> allocation in where dentries/inodes/other objects are added to the
> LRU because they are under object state spinlocks, and adding a
> potential memory allocation failure to the "add to lru" case is
> pretty nasty, IMO.
>
That would of course happen on dentry creation time, not lru add time.
It is totally possible since at creation time, we already know if memcg
is enabled or not.
next prev parent reply other threads:[~2013-01-19 0:13 UTC|newest]
Thread overview: 163+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-27 23:14 [RFC, PATCH 00/19] Numa aware LRU lists and shrinkers Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 01/19] dcache: convert dentry_stat.nr_unused to per-cpu counters Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 02/19] dentry: move to per-sb LRU locks Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 03/19] dcache: remove dentries from LRU before putting on dispose list Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 04/19] mm: new shrinker API Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 05/19] shrinker: convert superblock shrinkers to new API Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-12-20 11:06 ` Glauber Costa
2012-12-20 11:06 ` Glauber Costa
2012-12-20 11:06 ` Glauber Costa
2012-12-21 1:46 ` Dave Chinner
2012-12-21 1:46 ` Dave Chinner
2012-12-21 10:17 ` Glauber Costa
2012-12-21 10:17 ` Glauber Costa
2012-12-21 10:17 ` Glauber Costa
2012-11-27 23:14 ` [PATCH 06/19] list: add a new LRU list type Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-28 16:10 ` Christoph Hellwig
2012-11-28 16:10 ` Christoph Hellwig
2012-11-28 16:10 ` Christoph Hellwig
2012-11-27 23:14 ` [PATCH 07/19] inode: convert inode lru list to generic lru list code Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 08/19] dcache: convert to use new lru list infrastructure Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 09/19] list_lru: per-node " Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-12-20 11:21 ` Glauber Costa
2012-12-20 11:21 ` Glauber Costa
2012-12-20 11:21 ` Glauber Costa
2012-12-21 1:54 ` Dave Chinner
2012-12-21 1:54 ` Dave Chinner
2013-01-16 19:21 ` Glauber Costa
2013-01-16 19:21 ` Glauber Costa
2013-01-16 19:21 ` Glauber Costa
2013-01-16 22:55 ` Dave Chinner
2013-01-16 22:55 ` Dave Chinner
2013-01-17 0:35 ` Glauber Costa
2013-01-17 0:35 ` Glauber Costa
2013-01-17 0:35 ` Glauber Costa
2013-01-17 4:22 ` Dave Chinner
2013-01-17 4:22 ` Dave Chinner
2013-01-17 18:21 ` Glauber Costa
2013-01-17 18:21 ` Glauber Costa
2013-01-17 18:21 ` Glauber Costa
2013-01-18 0:10 ` Dave Chinner
2013-01-18 0:10 ` Dave Chinner
2013-01-18 0:14 ` Glauber Costa
2013-01-18 0:14 ` Glauber Costa
2013-01-18 0:14 ` Glauber Costa
2013-01-18 8:11 ` Dave Chinner
2013-01-18 8:11 ` Dave Chinner
2013-01-18 19:10 ` Glauber Costa
2013-01-18 19:10 ` Glauber Costa
2013-01-18 19:10 ` Glauber Costa
2013-01-19 0:10 ` Dave Chinner
2013-01-19 0:10 ` Dave Chinner
2013-01-19 0:13 ` Glauber Costa [this message]
2013-01-19 0:13 ` Glauber Costa
2013-01-19 0:13 ` Glauber Costa
2013-01-18 0:51 ` Glauber Costa
2013-01-18 0:51 ` Glauber Costa
2013-01-18 0:51 ` Glauber Costa
2013-01-18 8:08 ` Dave Chinner
2013-01-18 8:08 ` Dave Chinner
2013-01-18 19:01 ` Glauber Costa
2013-01-18 19:01 ` Glauber Costa
2013-01-18 19:01 ` Glauber Costa
2012-11-27 23:14 ` [PATCH 10/19] shrinker: add node awareness Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 11/19] fs: convert inode and dentry shrinking to be node aware Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 12/19] xfs: convert buftarg LRU to generic code Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 13/19] xfs: Node aware direct inode reclaim Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 14/19] xfs: use generic AG walk for background " Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 15/19] xfs: convert dquot cache lru to list_lru Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-28 16:17 ` Christoph Hellwig
2012-11-28 16:17 ` Christoph Hellwig
2012-11-28 16:17 ` Christoph Hellwig
2012-11-27 23:14 ` [PATCH 16/19] fs: convert fs shrinkers to new scan/count API Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 17/19] drivers: convert shrinkers to new count/scan API Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-28 1:13 ` Chris Wilson
2012-11-28 1:13 ` Chris Wilson
2012-11-28 1:13 ` Chris Wilson
2012-11-28 3:17 ` Dave Chinner
2012-11-28 3:17 ` Dave Chinner
2012-11-28 3:17 ` Dave Chinner
2012-11-28 8:21 ` Glauber Costa
2012-11-28 8:21 ` Glauber Costa
2012-11-28 8:21 ` Glauber Costa
2012-11-28 8:21 ` Glauber Costa
2012-11-28 21:28 ` Dave Chinner
2012-11-28 21:28 ` Dave Chinner
2012-11-28 21:28 ` Dave Chinner
2012-11-29 10:29 ` Glauber Costa
2012-11-29 10:29 ` Glauber Costa
2012-11-29 10:29 ` Glauber Costa
2012-11-29 10:29 ` Glauber Costa
2012-11-29 22:02 ` Dave Chinner
2012-11-29 22:02 ` Dave Chinner
2012-11-29 22:02 ` Dave Chinner
2013-06-07 13:37 ` Konrad Rzeszutek Wilk
2013-06-07 13:37 ` Konrad Rzeszutek Wilk
2013-06-07 13:37 ` Konrad Rzeszutek Wilk
2012-11-27 23:14 ` [PATCH 18/19] shrinker: convert remaining shrinkers to " Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` [PATCH 19/19] shrinker: Kill old ->shrink API Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-27 23:14 ` Dave Chinner
2012-11-29 19:02 ` [RFC, PATCH 00/19] Numa aware LRU lists and shrinkers Andi Kleen
2012-11-29 19:02 ` Andi Kleen
2012-11-29 19:02 ` Andi Kleen
2012-11-29 22:09 ` Dave Chinner
2012-11-29 22:09 ` Dave Chinner
2012-11-29 22:09 ` Dave Chinner
2012-12-20 11:45 ` Glauber Costa
2012-12-20 11:45 ` Glauber Costa
2012-12-20 11:45 ` Glauber Costa
2012-12-21 2:50 ` Dave Chinner
2012-12-21 2:50 ` Dave Chinner
2012-12-21 10:41 ` Glauber Costa
2012-12-21 10:41 ` Glauber Costa
2012-12-21 10:41 ` Glauber Costa
2013-01-21 16:08 ` Glauber Costa
2013-01-21 16:08 ` Glauber Costa
2013-01-21 16:08 ` Glauber Costa
2013-01-21 23:21 ` Dave Chinner
2013-01-21 23:21 ` Dave Chinner
2013-01-21 23:21 ` Dave Chinner
2013-01-23 14:36 ` Glauber Costa
2013-01-23 14:36 ` Glauber Costa
2013-01-23 14:36 ` Glauber Costa
2013-01-23 23:46 ` Dave Chinner
2013-01-23 23:46 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50F9E53F.4090902@parallels.com \
--to=glommer@parallels.com \
--cc=david@fromorbit.com \
--cc=gthelen@google.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=suleiman@google.com \
--cc=xfs@oss.sgi.com \
--cc=yinghan@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.