All of lore.kernel.org
 help / color / mirror / Atom feed
From: Balbir Singh <balbir@linux.vnet.ibm.com>
To: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: linux-mm@kvack.org, YAMAMOTO Takashi <yamamoto@valinux.co.jp>,
	Paul Menage <menage@google.com>,
	lizf@cn.fujitsu.com, linux-kernel@vger.kernel.org,
	Nick Piggin <nickpiggin@yahoo.com.au>,
	David Rientjes <rientjes@google.com>,
	Pavel Emelianov <xemul@openvz.org>,
	Dhaval Giani <dhaval@linux.vnet.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [mm][PATCH 0/4] Memory cgroup hierarchy introduction
Date: Wed, 05 Nov 2008 23:22:36 +0530	[thread overview]
Message-ID: <4911DD64.7010508@linux.vnet.ibm.com> (raw)
In-Reply-To: <50093.10.75.179.62.1225902786.squirrel@webmail-b.css.fujitsu.com>

KAMEZAWA Hiroyuki wrote:
> Balbir Singh said:
>> KAMEZAWA Hiroyuki wrote:
>>> On Sun, 02 Nov 2008 00:18:12 +0530
>>> Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
>>> As first impression, I think hierarchical LRU management is not
>>> good...means
>>> not fair from viewpoint of memory management.
>> Could you elaborate on this further? Is scanning of children during
>> reclaim the
>> issue? Do you want weighted reclaim for each of the children?
>>
> No. Consider follwing case
>    /root/group_root/group_A
>                    /group_B
>                    /group_C
> 
>   sum of group A, B, C is limited by group_root's limit.
> 
>   Now,
>         /group_root limit=1G, usage=990M
>                     /group_A  usage=600M , no limit, no tasks for a while
>                     /group_B  usage=10M  , no limit, no tasks
>                     /group_C  usage=380M , no limit, 2 tasks
> 
>   A user run a new task in group_B.
>   In your algorithm, group_A and B and C's memory are reclaimed
>   to the same extent becasue there is no information to show
>   "group A's memory are not accessed recently rather than B or C".
> 
>   This information is what we want for managing memory.
> 

For that sort of implementation, we'll need a common LRU. I actually thought of
implementing it by sharing a common LRU, but then we would end up with just one
common LRU at the root :)

The reclaim algorithm is smart in that it knows what pages are commonly
accessed. group A will get reclaimed more since those pages are not actively
referenced. reclaim on group_C will be harder. Simple experiments seem to show that.


>>> I'd like to show some other possible implementation of
>>> try_to_free_mem_cgroup_pages() if I can.
>>>
>> Elaborate please!
>>
> ok. but, at least, please add
>   - per-subtree hierarchy flag.
>   - cgroup_lock to walk list of cgroups somewhere.
> 
> I already sent my version "shared LRU" just as a hint for you.
> It is something extreme but contains something good, I think.
> 
>>> Anyway, I have to merge this with mem+swap controller.
>> Cool! I'll send you an updated version.
>>
> 
> Synchronized LRU patch may help you.

Let me get a good working version against current -mm and then we'll integrate
our patches.

-- 
	Balbir

WARNING: multiple messages have this Message-ID (diff)
From: Balbir Singh <balbir@linux.vnet.ibm.com>
To: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: linux-mm@kvack.org, YAMAMOTO Takashi <yamamoto@valinux.co.jp>,
	Paul Menage <menage@google.com>,
	lizf@cn.fujitsu.com, linux-kernel@vger.kernel.org,
	Nick Piggin <nickpiggin@yahoo.com.au>,
	David Rientjes <rientjes@google.com>,
	Pavel Emelianov <xemul@openvz.org>,
	Dhaval Giani <dhaval@linux.vnet.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [mm][PATCH 0/4] Memory cgroup hierarchy introduction
Date: Wed, 05 Nov 2008 23:22:36 +0530	[thread overview]
Message-ID: <4911DD64.7010508@linux.vnet.ibm.com> (raw)
In-Reply-To: <50093.10.75.179.62.1225902786.squirrel@webmail-b.css.fujitsu.com>

KAMEZAWA Hiroyuki wrote:
> Balbir Singh said:
>> KAMEZAWA Hiroyuki wrote:
>>> On Sun, 02 Nov 2008 00:18:12 +0530
>>> Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
>>> As first impression, I think hierarchical LRU management is not
>>> good...means
>>> not fair from viewpoint of memory management.
>> Could you elaborate on this further? Is scanning of children during
>> reclaim the
>> issue? Do you want weighted reclaim for each of the children?
>>
> No. Consider follwing case
>    /root/group_root/group_A
>                    /group_B
>                    /group_C
> 
>   sum of group A, B, C is limited by group_root's limit.
> 
>   Now,
>         /group_root limit=1G, usage=990M
>                     /group_A  usage=600M , no limit, no tasks for a while
>                     /group_B  usage=10M  , no limit, no tasks
>                     /group_C  usage=380M , no limit, 2 tasks
> 
>   A user run a new task in group_B.
>   In your algorithm, group_A and B and C's memory are reclaimed
>   to the same extent becasue there is no information to show
>   "group A's memory are not accessed recently rather than B or C".
> 
>   This information is what we want for managing memory.
> 

For that sort of implementation, we'll need a common LRU. I actually thought of
implementing it by sharing a common LRU, but then we would end up with just one
common LRU at the root :)

The reclaim algorithm is smart in that it knows what pages are commonly
accessed. group A will get reclaimed more since those pages are not actively
referenced. reclaim on group_C will be harder. Simple experiments seem to show that.


>>> I'd like to show some other possible implementation of
>>> try_to_free_mem_cgroup_pages() if I can.
>>>
>> Elaborate please!
>>
> ok. but, at least, please add
>   - per-subtree hierarchy flag.
>   - cgroup_lock to walk list of cgroups somewhere.
> 
> I already sent my version "shared LRU" just as a hint for you.
> It is something extreme but contains something good, I think.
> 
>>> Anyway, I have to merge this with mem+swap controller.
>> Cool! I'll send you an updated version.
>>
> 
> Synchronized LRU patch may help you.

Let me get a good working version against current -mm and then we'll integrate
our patches.

-- 
	Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2008-11-05 17:54 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-11-01 18:48 [mm][PATCH 0/4] Memory cgroup hierarchy introduction Balbir Singh
2008-11-01 18:48 ` Balbir Singh
2008-11-01 18:48 ` [mm] [PATCH 1/4] Memory cgroup hierarchy documentation Balbir Singh
2008-11-01 18:48   ` Balbir Singh
2008-11-04  6:25   ` Paul Menage
2008-11-04  6:25     ` Paul Menage
2008-11-04  6:26     ` Paul Menage
2008-11-04  6:26       ` Paul Menage
2008-11-05 13:55       ` Balbir Singh
2008-11-05 13:55         ` Balbir Singh
2008-11-01 18:48 ` [mm] [PATCH 2/4] Memory cgroup resource counters for hierarchy Balbir Singh
2008-11-01 18:48   ` Balbir Singh
2008-11-02  5:42   ` KAMEZAWA Hiroyuki
2008-11-02  5:42     ` KAMEZAWA Hiroyuki
2008-11-02  5:49     ` Balbir Singh
2008-11-02  5:49       ` Balbir Singh
2008-11-02  5:56       ` KAMEZAWA Hiroyuki
2008-11-02  5:56         ` KAMEZAWA Hiroyuki
2008-11-02 11:46         ` Balbir Singh
2008-11-02 11:46           ` Balbir Singh
2008-11-01 18:48 ` [mm] [PATCH 3/4] Memory cgroup hierarchical reclaim Balbir Singh
2008-11-01 18:48   ` Balbir Singh
2008-11-02  5:37   ` KAMEZAWA Hiroyuki
2008-11-02  5:37     ` KAMEZAWA Hiroyuki
2008-11-02  5:44     ` Balbir Singh
2008-11-02  5:44       ` Balbir Singh
2008-11-04  2:17       ` KAMEZAWA Hiroyuki
2008-11-04  2:17         ` KAMEZAWA Hiroyuki
2008-11-05 13:34         ` Balbir Singh
2008-11-05 13:34           ` Balbir Singh
2008-11-05 16:20           ` KAMEZAWA Hiroyuki
2008-11-05 16:20             ` KAMEZAWA Hiroyuki
2008-11-06 14:00             ` Balbir Singh
2008-11-06 14:00               ` Balbir Singh
2008-11-01 18:49 ` [mm] [PATCH 4/4] Memory cgroup hierarchy feature selector Balbir Singh
2008-11-01 18:49   ` Balbir Singh
2008-11-02  5:38   ` KAMEZAWA Hiroyuki
2008-11-02  5:38     ` KAMEZAWA Hiroyuki
2008-11-02  6:03     ` Balbir Singh
2008-11-02  6:03       ` Balbir Singh
2008-11-02  6:24       ` KAMEZAWA Hiroyuki
2008-11-02  6:24         ` KAMEZAWA Hiroyuki
2008-11-02 15:52         ` Balbir Singh
2008-11-02 15:52           ` Balbir Singh
2008-11-04  6:37           ` Paul Menage
2008-11-04  6:37             ` Paul Menage
2008-11-06  7:00             ` Balbir Singh
2008-11-06  7:00               ` Balbir Singh
2008-11-06  7:01               ` Balbir Singh
2008-11-06  7:01                 ` Balbir Singh
2008-11-06  6:56         ` Balbir Singh
2008-11-06  6:56           ` Balbir Singh
2008-11-06  7:30           ` KAMEZAWA Hiroyuki
2008-11-06  7:30             ` KAMEZAWA Hiroyuki
2008-11-04  0:15 ` [mm][PATCH 0/4] Memory cgroup hierarchy introduction KAMEZAWA Hiroyuki
2008-11-04  0:15   ` KAMEZAWA Hiroyuki
2008-11-05 13:51   ` Balbir Singh
2008-11-05 13:51     ` Balbir Singh
2008-11-05 16:33     ` KAMEZAWA Hiroyuki
2008-11-05 16:33       ` KAMEZAWA Hiroyuki
2008-11-05 17:52       ` Balbir Singh [this message]
2008-11-05 17:52         ` Balbir Singh
2008-11-06  0:22         ` KAMEZAWA Hiroyuki
2008-11-06  0:22           ` KAMEZAWA Hiroyuki
2008-11-04  9:21 ` [patch 1/2] memcg: hierarchy, yet another one KAMEZAWA Hiroyuki
2008-11-04  9:21   ` KAMEZAWA Hiroyuki
2008-11-04  9:25 ` KAMEZAWA Hiroyuki
2008-11-04  9:25   ` KAMEZAWA Hiroyuki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4911DD64.7010508@linux.vnet.ibm.com \
    --to=balbir@linux.vnet.ibm.com \
    --cc=akpm@linux-foundation.org \
    --cc=dhaval@linux.vnet.ibm.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lizf@cn.fujitsu.com \
    --cc=menage@google.com \
    --cc=nickpiggin@yahoo.com.au \
    --cc=rientjes@google.com \
    --cc=xemul@openvz.org \
    --cc=yamamoto@valinux.co.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.