[LSF][MM] rough agenda for memcg.

[LSF][MM] rough agenda for memcg.

[KAMEZAWA Hiroyuki]

Hi,

In this LSF/MM, we have some memcg topics in the 1st day.

>From schedule,

1. Memory cgroup : Where next ? 1hour (Balbir Singh/Kamezawa) 
2. Memcg Dirty Limit and writeback 30min(Greg Thelen)
3. Memcg LRU management 30min (Ying Han, Michal Hocko)
4. Page cgroup on a diet (Johannes Weiner)

2.5 hours. This seems long...or short ? ;)

I'd like to sort out topics before going. Please fix if I don't catch enough.

mentiont to 1. later...

Main topics on 2. Memcg Dirty Limit and writeback ....is

 a) How to implement per-memcg dirty inode finding method (list) and
    how flusher threads handle memcg.

 b) Hot to interact with IO-Less dirty page reclaim.
    IIUC, if memcg doesn't handle this correctly, OOM happens.

 Greg, do we need to have a shared session with I/O guys ?
 If needed, current schedule is O.K. ?

Main topics on 3. Memcg LRU management

 a) Isolation/Gurantee for memcg.
    Current memcg doesn't have enough isolation when globarl reclaim runs.
    .....Because it's designed not to affect global reclaim.
    But from user's point of view, it's nonsense and we should have some hints
    for isolate set of memory or implement a guarantee.
    
    One way to go is updating softlimit better. To do this, we should know what
    is problem now. I'm sorry I can't prepare data on this until LSF/MM.
    Another way is implementing a guarantee. But this will require some interaction
    with page allocator and pgscan mechanism. This will be a big work.

 b) single LRU and per memcg zone->lru_lock.
    I hear zone->lru_lock contention caused by memcg is a problem on Google servers.
    Okay, please show data. (I've never seen it.)
    Then, we need to discuss Pros. and Cons. of current design and need to consinder
    how to improve it. I think Google and Michal have their own implementation.

    Current design of double-LRU is from the 1st inclusion of memcg to the kernel.
    But I don't know that discussion was there. Balbir, could you explain the reason
    of this design ? Then, we can go ahead, somewhere.


Main topics on 4. Page cgroup on diet is...

  a) page_cgroup is too big!, we need diet....
     I think Johannes removes -> page pointer already. Ok, what's the next to
     be removed ?

  I guess the next candidate is ->lru which is related to 3-b).
  
Main topics on 1.Memory control groups: where next? is..

To be honest, I just do bug fixes in these days. And hot topics are on above..
I don't have concrete topics. What I can think of from recent linux-mm emails are...

  a) Kernel memory accounting.
  b) Need some work with Cleancache ?
  c) Should we provide a auto memory cgroup for file caches ?
     (Then we can implement a file-cache-limit.)
  d) Do we have a problem with current OOM-disable+notifier design ?
  e) ROOT cgroup should have a limit/softlimit, again ?
  f) vm_overcommit_memory should be supproted with memcg ?
     (I remember there was a trial. But I think it should be done in other cgroup
      as vmemory cgroup.)
...

I think
  a) discussing about this is too early. There is no patch.
     I think we'll just waste time.

  b) enable/disable cleancache per memcg or some share/limit ??
     But we can discuss this kind of things after cleancache is in production use...

  c) AFAIK, some other OSs have this kind of feature, a box for file-cache.
     Because file-cache is a shared object between all cgroups, it's difficult
     to handle. It may be better to have a auto cgroup for file caches and add knobs
     for memcg.

  d) I think it works well. 

  e) It seems Michal wants this for lazy users. Hmm, should we have a knob ?
     It's helpful that some guy have a performance number on the latest kernel
     with and without memcg (in limitless case).
     IIUC, with THP enabled as 'always', the number of page fault dramatically reduced and
     memcg's accounting cost gets down...

  f) I think someone mention about this...

Maybe c) and d) _can_ be a topic but seems not very important.

So, for this slot, I'd like to discuss

  I) Softlimit/Isolation (was 3-A) for 1hour
     If we have extra time, kernel memory accounting or file-cache handling
     will be good.
   
  II) Dirty page handling. (for 30min)
     Maybe we'll discuss about per-memcg inode queueing issue.

  III) Discussing the current and future design of LRU.(for 30+min)

  IV) Diet of page_cgroup (for 30-min)
      Maybe this can be combined with III.

Thanks,
-Kame

















--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF][MM] rough agenda for memcg.

[Greg Thelen]

On Wed, Mar 30, 2011 at 7:01 PM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@jp.fujitsu.com> wrote:
>
> Hi,
>
> In this LSF/MM, we have some memcg topics in the 1st day.
>
> From schedule,
>
> 1. Memory cgroup : Where next ? 1hour (Balbir Singh/Kamezawa)
> 2. Memcg Dirty Limit and writeback 30min(Greg Thelen)
> 3. Memcg LRU management 30min (Ying Han, Michal Hocko)
> 4. Page cgroup on a diet (Johannes Weiner)
>
> 2.5 hours. This seems long...or short ? ;)

I think it is a good starting plan.

> I'd like to sort out topics before going. Please fix if I don't catch enough.
>
> mentiont to 1. later...
>
> Main topics on 2. Memcg Dirty Limit and writeback ....is
>
>  a) How to implement per-memcg dirty inode finding method (list) and
>    how flusher threads handle memcg.

I have some very rough code implementing the ideas discussed in
http://thread.gmane.org/gmane.linux.kernel.mm/59707
Unfortunately, I do not yet have good patches, but maybe an RFC series
soon.  I can provide update on the direction I am thinking.

>  b) Hot to interact with IO-Less dirty page reclaim.
>    IIUC, if memcg doesn't handle this correctly, OOM happens.

The last posted memcg dirty writeback patches were based on -mm at the
time, which did not have IO-less balance_dirty_pages.  I have an
approach which I _think_ will be compatible with IO-less
balance_dirty_pages(), but I need to talk with some writeback guys to
confirm.  Seeing the Writeback talk Mon 9:30am should be very useful
for me.

>  Greg, do we need to have a shared session with I/O guys ?
>  If needed, current schedule is O.K. ?

We can contact any interested writeback guys to see if they want to
attend memcg-writeback discussion.  We might be able to defer this
detail until Mon morning.

> Main topics on 3. Memcg LRU management
>
>  a) Isolation/Gurantee for memcg.
>    Current memcg doesn't have enough isolation when globarl reclaim runs.
>    .....Because it's designed not to affect global reclaim.
>    But from user's point of view, it's nonsense and we should have some hints
>    for isolate set of memory or implement a guarantee.
>
>    One way to go is updating softlimit better. To do this, we should know what
>    is problem now. I'm sorry I can't prepare data on this until LSF/MM.
>    Another way is implementing a guarantee. But this will require some interaction
>    with page allocator and pgscan mechanism. This will be a big work.
>
>  b) single LRU and per memcg zone->lru_lock.
>    I hear zone->lru_lock contention caused by memcg is a problem on Google servers.
>    Okay, please show data. (I've never seen it.)
>    Then, we need to discuss Pros. and Cons. of current design and need to consinder
>    how to improve it. I think Google and Michal have their own implementation.
>
>    Current design of double-LRU is from the 1st inclusion of memcg to the kernel.
>    But I don't know that discussion was there. Balbir, could you explain the reason
>    of this design ? Then, we can go ahead, somewhere.
>
>
> Main topics on 4. Page cgroup on diet is...
>
>  a) page_cgroup is too big!, we need diet....
>     I think Johannes removes -> page pointer already. Ok, what's the next to
>     be removed ?
>
>  I guess the next candidate is ->lru which is related to 3-b).
>
> Main topics on 1.Memory control groups: where next? is..
>
> To be honest, I just do bug fixes in these days. And hot topics are on above..
> I don't have concrete topics. What I can think of from recent linux-mm emails are...
>
>  a) Kernel memory accounting.
>  b) Need some work with Cleancache ?
>  c) Should we provide a auto memory cgroup for file caches ?
>     (Then we can implement a file-cache-limit.)
>  d) Do we have a problem with current OOM-disable+notifier design ?
>  e) ROOT cgroup should have a limit/softlimit, again ?
>  f) vm_overcommit_memory should be supproted with memcg ?
>     (I remember there was a trial. But I think it should be done in other cgroup
>      as vmemory cgroup.)
> ...
>
> I think
>  a) discussing about this is too early. There is no patch.
>     I think we'll just waste time.
>
>  b) enable/disable cleancache per memcg or some share/limit ??
>     But we can discuss this kind of things after cleancache is in production use...
>
>  c) AFAIK, some other OSs have this kind of feature, a box for file-cache.
>     Because file-cache is a shared object between all cgroups, it's difficult
>     to handle. It may be better to have a auto cgroup for file caches and add knobs
>     for memcg.
>
>  d) I think it works well.
>
>  e) It seems Michal wants this for lazy users. Hmm, should we have a knob ?
>     It's helpful that some guy have a performance number on the latest kernel
>     with and without memcg (in limitless case).
>     IIUC, with THP enabled as 'always', the number of page fault dramatically reduced and
>     memcg's accounting cost gets down...
>
>  f) I think someone mention about this...
>
> Maybe c) and d) _can_ be a topic but seems not very important.
>
> So, for this slot, I'd like to discuss
>
>  I) Softlimit/Isolation (was 3-A) for 1hour
>     If we have extra time, kernel memory accounting or file-cache handling
>     will be good.
>
>  II) Dirty page handling. (for 30min)
>     Maybe we'll discuss about per-memcg inode queueing issue.
>
>  III) Discussing the current and future design of LRU.(for 30+min)
>
>  IV) Diet of page_cgroup (for 30-min)
>      Maybe this can be combined with III.
>
> Thanks,
> -Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF][MM] rough agenda for memcg.

[Balbir Singh]

* KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2011-03-31 11:01:13]:

> 
> Hi,
> 
> In this LSF/MM, we have some memcg topics in the 1st day.
> 
> From schedule,
> 
> 1. Memory cgroup : Where next ? 1hour (Balbir Singh/Kamezawa) 
> 2. Memcg Dirty Limit and writeback 30min(Greg Thelen)
> 3. Memcg LRU management 30min (Ying Han, Michal Hocko)
> 4. Page cgroup on a diet (Johannes Weiner)
>

Hi, Kamezawa-San,

I am afraid I am unable to come to LSF MM summit. There have
been some changes, I will however continue to monitor the discussions
over email.

-- 
	Three Cheers,
	Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF][MM] rough agenda for memcg.

[Ying Han]

[-- Attachment #1: Type: text/plain, Size: 5156 bytes --]

On Wed, Mar 30, 2011 at 7:01 PM, KAMEZAWA Hiroyuki <
kamezawa.hiroyu@jp.fujitsu.com> wrote:

>
> Hi,
>
> In this LSF/MM, we have some memcg topics in the 1st day.
>
> From schedule,
>
> 1. Memory cgroup : Where next ? 1hour (Balbir Singh/Kamezawa)
> 2. Memcg Dirty Limit and writeback 30min(Greg Thelen)
> 3. Memcg LRU management 30min (Ying Han, Michal Hocko)
> 4. Page cgroup on a diet (Johannes Weiner)
>
> 2.5 hours. This seems long...or short ? ;)
>
> I'd like to sort out topics before going. Please fix if I don't catch
> enough.
>
> mentiont to 1. later...
>
> Main topics on 2. Memcg Dirty Limit and writeback ....is
>
>  a) How to implement per-memcg dirty inode finding method (list) and
>    how flusher threads handle memcg.
>
>  b) Hot to interact with IO-Less dirty page reclaim.
>    IIUC, if memcg doesn't handle this correctly, OOM happens.
>
>  Greg, do we need to have a shared session with I/O guys ?
>  If needed, current schedule is O.K. ?
>
> Main topics on 3. Memcg LRU management
>
>  a) Isolation/Gurantee for memcg.
>    Current memcg doesn't have enough isolation when globarl reclaim runs.
>    .....Because it's designed not to affect global reclaim.
>    But from user's point of view, it's nonsense and we should have some
> hints
>    for isolate set of memory or implement a guarantee.
>
>    One way to go is updating softlimit better. To do this, we should know
> what
>    is problem now. I'm sorry I can't prepare data on this until LSF/MM.
>
I generated example which shows the inefficiency of soft_limit reclaim,
which is so far based on the code
inspection. I am not sure if I can get some data before LSF.


>    Another way is implementing a guarantee. But this will require some
> interaction
>    with page allocator and pgscan mechanism. This will be a big work.
>
Not sure about this..

>
>  b) single LRU and per memcg zone->lru_lock.
>    I hear zone->lru_lock contention caused by memcg is a problem on Google
> servers.
>    Okay, please show data. (I've never seen it.)
>

To clarify, the lock contention is bad after per-memcg background reclaim
patch. The worst case we have #-of-cpu per-memcg kswapd
reclaiming on per-memcg lru and all competing the zone->lru_lock.

--Ying

   Then, we need to discuss Pros. and Cons. of current design and need to
> consinder
>    how to improve it. I think Google and Michal have their own
> implementation.
>
>    Current design of double-LRU is from the 1st inclusion of memcg to the
> kernel.
>    But I don't know that discussion was there. Balbir, could you explain
> the reason
>    of this design ? Then, we can go ahead, somewhere.
>
>
> Main topics on 4. Page cgroup on diet is...
>
>  a) page_cgroup is too big!, we need diet....
>     I think Johannes removes -> page pointer already. Ok, what's the next
> to
>     be removed ?
>
>  I guess the next candidate is ->lru which is related to 3-b).
>
> Main topics on 1.Memory control groups: where next? is..
>
> To be honest, I just do bug fixes in these days. And hot topics are on
> above..
> I don't have concrete topics. What I can think of from recent linux-mm
> emails are...
>
>  a) Kernel memory accounting.
>  b) Need some work with Cleancache ?
>  c) Should we provide a auto memory cgroup for file caches ?
>     (Then we can implement a file-cache-limit.)
>  d) Do we have a problem with current OOM-disable+notifier design ?
>  e) ROOT cgroup should have a limit/softlimit, again ?
>  f) vm_overcommit_memory should be supproted with memcg ?
>     (I remember there was a trial. But I think it should be done in other
> cgroup
>      as vmemory cgroup.)
> ...
>
> I think
>  a) discussing about this is too early. There is no patch.
>     I think we'll just waste time.


>  b) enable/disable cleancache per memcg or some share/limit ??
>     But we can discuss this kind of things after cleancache is in
> production use...
>
>  c) AFAIK, some other OSs have this kind of feature, a box for file-cache.
>     Because file-cache is a shared object between all cgroups, it's
> difficult
>     to handle. It may be better to have a auto cgroup for file caches and
> add knobs
>     for memcg.
>
>  d) I think it works well.
>
>  e) It seems Michal wants this for lazy users. Hmm, should we have a knob ?
>     It's helpful that some guy have a performance number on the latest
> kernel
>     with and without memcg (in limitless case).
>     IIUC, with THP enabled as 'always', the number of page fault
> dramatically reduced and
>     memcg's accounting cost gets down...
>
>  f) I think someone mention about this...
>
> Maybe c) and d) _can_ be a topic but seems not very important.
>
> So, for this slot, I'd like to discuss
>
>  I) Softlimit/Isolation (was 3-A) for 1hour
>     If we have extra time, kernel memory accounting or file-cache handling
>     will be good.
>
>  II) Dirty page handling. (for 30min)
>     Maybe we'll discuss about per-memcg inode queueing issue.
>
>  III) Discussing the current and future design of LRU.(for 30+min)
>
>  IV) Diet of page_cgroup (for 30-min)
>      Maybe this can be combined with III.
>
> Thanks,
> -Kame
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 6545 bytes --]

Re: [LSF][MM] rough agenda for memcg.

[Zhu Yanhai]

Hi Kame,

2011/3/31 KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>:
>  c) Should we provide a auto memory cgroup for file caches ?
>     (Then we can implement a file-cache-limit.)
>  c) AFAIK, some other OSs have this kind of feature, a box for file-cache.
>     Because file-cache is a shared object between all cgroups, it's difficult
>     to handle. It may be better to have a auto cgroup for file caches and add knobs
>     for memcg.

I have been thinking about this idea. It seems the root cause of
current difficult is
the whole cgroup infrastructure is based on process groups, so its counters
naturally center on process. However, this is not nature for counters
of file caches,
which center on inodes/devs actually. This brought many confusing
problems - e.g.
who should be charged for a (dirty)file page?  I think the answer is
no process but
the filesystem/block device it sits on.
How about we change the view, from centering on process to centering
on filesystem/device.
Let's call it 'pcgroup'. When it enables, we can set limits for each
filesystem/device,
and charge the filesystem/device for each page seat on it. This
'pcgroup' would be
orthogonal to cgroup.memcontroler. We could make cgroup.memcontroler only
account/control the anon pages they have, leave all file-backend pages
controlled
by the 'pcgroup'.
For the implementation, 'pcgroup' can reuse the struct page_cgroup -
and res_counter
maybe even the hierarchy reclaim code, and so on - it looks like a
cgroup very much -
which might make things easier.
One problem is this 'pcgroup' may not be able to be implemented inside
cgroup frame,
as the very core code of cgroup assumes that all its
controls/res_counters are per-process(group).

Is this doable?

Thanks,
Zhu Yanhai

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Pavel Emelyanov]

>  b) single LRU and per memcg zone->lru_lock.
>     I hear zone->lru_lock contention caused by memcg is a problem on Google servers.
>     Okay, please show data. (I've never seen it.)
>     Then, we need to discuss Pros. and Cons. of current design and need to consinder
>     how to improve it. I think Google and Michal have their own implementation.
> 
>     Current design of double-LRU is from the 1st inclusion of memcg to the kernel.
>     But I don't know that discussion was there. Balbir, could you explain the reason
>     of this design ? Then, we can go ahead, somewhere.

I would like to take part in that and describe what we've done with LRU
in OpenVZ in details.

>   a) Kernel memory accounting.

This one is very interesting to me.

>   f) vm_overcommit_memory should be supproted with memcg ?
>      (I remember there was a trial. But I think it should be done in other cgroup
>       as vmemory cgroup.)

And this one too - I have an implementation of overcommit management
in OpenVZ, I can describe one and discuss pros-n-cons.

Thanks,
Pavel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Jan Kara]

On Wed 30-03-11 22:52:49, Greg Thelen wrote:
> On Wed, Mar 30, 2011 at 7:01 PM, KAMEZAWA Hiroyuki
> <kamezawa.hiroyu@jp.fujitsu.com> wrote:
> > I'd like to sort out topics before going. Please fix if I don't catch enough.
> >
> > mentiont to 1. later...
> >
> > Main topics on 2. Memcg Dirty Limit and writeback ....is
> >
> >  a) How to implement per-memcg dirty inode finding method (list) and
> >    how flusher threads handle memcg.
> 
> I have some very rough code implementing the ideas discussed in
> http://thread.gmane.org/gmane.linux.kernel.mm/59707
> Unfortunately, I do not yet have good patches, but maybe an RFC series
> soon.  I can provide update on the direction I am thinking.
> 
> >  b) Hot to interact with IO-Less dirty page reclaim.
> >    IIUC, if memcg doesn't handle this correctly, OOM happens.
> 
> The last posted memcg dirty writeback patches were based on -mm at the
> time, which did not have IO-less balance_dirty_pages.  I have an
> approach which I _think_ will be compatible with IO-less
> balance_dirty_pages(), but I need to talk with some writeback guys to
> confirm.  Seeing the Writeback talk Mon 9:30am should be very useful
> for me.
> 
> >  Greg, do we need to have a shared session with I/O guys ?
> >  If needed, current schedule is O.K. ?
> 
> We can contact any interested writeback guys to see if they want to
> attend memcg-writeback discussion.  We might be able to defer this
> detail until Mon morning.
  Yes, I plan to take part in this discussion. If this would be joint
session with fs people (which kind of makes sense) it's simpler for
me but I can surely handle if it isn't :).

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Andrea Arcangeli]

Hi KAMEZAWA,

On Thu, Mar 31, 2011 at 11:01:13AM +0900, KAMEZAWA Hiroyuki wrote:
> 1. Memory cgroup : Where next ? 1hour (Balbir Singh/Kamezawa) 

Originally it was 30min and then there was a topic "Working set
estimation" for another 30 min. That has been converted to "what's
next continued", so I assume that you can add the Working set
estimation as a subtopic.

> 2. Memcg Dirty Limit and writeback 30min(Greg Thelen)
> 3. Memcg LRU management 30min (Ying Han, Michal Hocko)
> 4. Page cgroup on a diet (Johannes Weiner)
> 2.5 hours. This seems long...or short ? ;)

Overall we've been seeing plenty of memcg emails, so I guess 2.5 hours
are ok. And I wouldn't say we're not in the short side.

I thought it was better to keep "what's next as last memcg topic" but
Hugh suggested it as first topic, no problem with the order on my
side. Now the first topic doubled in time ;).

> I'd like to sort out topics before going. Please fix if I don't catch enough.

I agree that's good practice, to prepare some material and have a
plan.

> Main topics on 1.Memory control groups: where next? is..
> 
> To be honest, I just do bug fixes in these days. And hot topics are on above..
> I don't have concrete topics. What I can think of from recent linux-mm emails are...
> 
>   a) Kernel memory accounting.
>   b) Need some work with Cleancache ?
>   c) Should we provide a auto memory cgroup for file caches ?
>      (Then we can implement a file-cache-limit.)
>   d) Do we have a problem with current OOM-disable+notifier design ?
>   e) ROOT cgroup should have a limit/softlimit, again ?
>   f) vm_overcommit_memory should be supproted with memcg ?
>      (I remember there was a trial. But I think it should be done in other cgroup
>       as vmemory cgroup.)
> ...
> 
> I think
>   a) discussing about this is too early. There is no patch.
>      I think we'll just waste time.

Hugh was thinking about this as a subtopic, I'm not intrigued by it
personally so I don't mind to skip it. I can imagine some people
liking it though. Maybe you can mention it and see if somebody has
contact with customers who needs this and if it's worth the pain for
them.

>   b) enable/disable cleancache per memcg or some share/limit ??
>      But we can discuss this kind of things after cleancache is in production use...

Kind of agreed we can skip this on my side, but I may be biased
because cleancache is not really useful to anything like KVM at least
(we already have writeback default and wirththrough by just using
O_SYNC without requiring a flood of hypercalls and vmexits even with
light I/O activity). We'll hear about the new novirt users of
transcendent memory, in Dan's talk before memcg starts.

>   c) AFAIK, some other OSs have this kind of feature, a box for file-cache.
>      Because file-cache is a shared object between all cgroups, it's difficult
>      to handle. It may be better to have a auto cgroup for file caches and add knobs
>      for memcg.
> 
>   d) I think it works well. 
> 
>   e) It seems Michal wants this for lazy users. Hmm, should we have a knob ?
>      It's helpful that some guy have a performance number on the latest kernel
>      with and without memcg (in limitless case).
>      IIUC, with THP enabled as 'always', the number of page fault dramatically reduced and
>      memcg's accounting cost gets down...
> 
>   f) I think someone mention about this...
> 
> Maybe c) and d) _can_ be a topic but seems not very important.
>
> So, for this slot, I'd like to discuss
> 
>   I) Softlimit/Isolation (was 3-A) for 1hour
>      If we have extra time, kernel memory accounting or file-cache handling
>      will be good.

Sounds good but for the what's next 1 hour slot I would keep more
subtopics like some of the ones you mentioned (if nothing else for the
second half hour) and then we see how things evolve. Maybe Michel
L. still want to talk about Working set estimation too in the second
half hour.

>   II) Dirty page handling. (for 30min)
>      Maybe we'll discuss about per-memcg inode queueing issue.
> 
>   III) Discussing the current and future design of LRU.(for 30+min)
> 
>   IV) Diet of page_cgroup (for 30-min)
>       Maybe this can be combined with III.

Looks a good plan to me, but others are more directly involved in
memcg than me so feel free to decide! About the diet topic it was
suggested by Johannes so I'll let him comment on it if he wants.

Thanks,
Andrea

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Andrea Arcangeli]

On Thu, Mar 31, 2011 at 01:23:13PM +0400, Pavel Emelyanov wrote:
> >  b) single LRU and per memcg zone->lru_lock.
> >     I hear zone->lru_lock contention caused by memcg is a problem on Google servers.
> >     Okay, please show data. (I've never seen it.)
> >     Then, we need to discuss Pros. and Cons. of current design and need to consinder
> >     how to improve it. I think Google and Michal have their own implementation.
> > 
> >     Current design of double-LRU is from the 1st inclusion of memcg to the kernel.
> >     But I don't know that discussion was there. Balbir, could you explain the reason
> >     of this design ? Then, we can go ahead, somewhere.
> 
> I would like to take part in that and describe what we've done with LRU
> in OpenVZ in details.

Sounds good.

>
> >   a) Kernel memory accounting.
> 
> This one is very interesting to me.

I expected someone would have been interested into that...

> >   f) vm_overcommit_memory should be supproted with memcg ?
> >      (I remember there was a trial. But I think it should be done in other cgroup
> >       as vmemory cgroup.)
> 
> And this one too - I have an implementation of overcommit management
> in OpenVZ, I can describe one and discuss pros-n-cons.

Ok, so I've added you to the second half of "what's next".

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Ying Han]

On Thu, Mar 31, 2011 at 9:20 AM, Andrea Arcangeli <aarcange@redhat.com> wrote:
> On Thu, Mar 31, 2011 at 01:23:13PM +0400, Pavel Emelyanov wrote:
>> >  b) single LRU and per memcg zone->lru_lock.
>> >     I hear zone->lru_lock contention caused by memcg is a problem on Google servers.
>> >     Okay, please show data. (I've never seen it.)
>> >     Then, we need to discuss Pros. and Cons. of current design and need to consinder
>> >     how to improve it. I think Google and Michal have their own implementation.
>> >
>> >     Current design of double-LRU is from the 1st inclusion of memcg to the kernel.
>> >     But I don't know that discussion was there. Balbir, could you explain the reason
>> >     of this design ? Then, we can go ahead, somewhere.
>>
>> I would like to take part in that and describe what we've done with LRU
>> in OpenVZ in details.
>
> Sounds good.
>
>>
>> >   a) Kernel memory accounting.
>>
>> This one is very interesting to me.
>
> I expected someone would have been interested into that...

We are definitely interested in that. We are testing the patch
internally now on kernel slab accounting. The author won't be in LSF,
but I can help to present our approach in the session w/ Pavel. Also,
I would like to talk a bit on the kernel memory reclaim for few
minutes.

--Ying

>
>> >   f) vm_overcommit_memory should be supproted with memcg ?
>> >      (I remember there was a trial. But I think it should be done in other cgroup
>> >       as vmemory cgroup.)
>>
>> And this one too - I have an implementation of overcommit management
>> in OpenVZ, I can describe one and discuss pros-n-cons.
>
> Ok, so I've added you to the second half of "what's next".
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Pavel Emelyanov]

>>>>   a) Kernel memory accounting.
>>>
>>> This one is very interesting to me.
>>
>> I expected someone would have been interested into that...
> 
> We are definitely interested in that. We are testing the patch
> internally now on kernel slab accounting. 

I guess this patch sent to linux-mm, so can you give me an url?

> The author won't be in LSF,
> but I can help to present our approach in the session w/ Pavel. Also,
> I would like to talk a bit on the kernel memory reclaim for few
> minutes.
> 
> --Ying

Thanks,
Pavel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Ying Han]

On Thu, Mar 31, 2011 at 12:00 PM, Pavel Emelyanov <xemul@parallels.com> wrote:
>>>>>   a) Kernel memory accounting.
>>>>
>>>> This one is very interesting to me.
>>>
>>> I expected someone would have been interested into that...
>>
>> We are definitely interested in that. We are testing the patch
>> internally now on kernel slab accounting.
>
> I guess this patch sent to linux-mm, so can you give me an url?

Pavel, I don't think we have patch sent out yet (under testing). But
Suleiman (the author) has the
proposal at http://permalink.gmane.org/gmane.linux.kernel.mm/58173

Thanks

--Ying
>
>> The author won't be in LSF,
>> but I can help to present our approach in the session w/ Pavel. Also,
>> I would like to talk a bit on the kernel memory reclaim for few
>> minutes.
>>
>> --Ying
>
> Thanks,
> Pavel
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Ying Han]

On Thu, Mar 31, 2011 at 12:16 PM, Ying Han <yinghan@google.com> wrote:
> On Thu, Mar 31, 2011 at 12:00 PM, Pavel Emelyanov <xemul@parallels.com> wrote:
>>>>>>   a) Kernel memory accounting.
>>>>>
>>>>> This one is very interesting to me.
>>>>
>>>> I expected someone would have been interested into that...
>>>
>>> We are definitely interested in that. We are testing the patch
>>> internally now on kernel slab accounting.
>>
>> I guess this patch sent to linux-mm, so can you give me an url?
>
> Pavel, I don't think we have patch sent out yet (under testing). But
> Suleiman (the author) has the
> proposal at http://permalink.gmane.org/gmane.linux.kernel.mm/58173
>
> Thanks
>
> --Ying
>>
>>> The author won't be in LSF,
>>> but I can help to present our approach in the session w/ Pavel. Also,
>>> I would like to talk a bit on the kernel memory reclaim for few
>>> minutes.
>>>
>>> --Ying
>>
>> Thanks,
>> Pavel
>>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Johannes Weiner]

On Thu, Mar 31, 2011 at 05:59:31PM +0200, Andrea Arcangeli wrote:
> On Thu, Mar 31, 2011 at 11:01:13AM +0900, KAMEZAWA Hiroyuki wrote:
> >   III) Discussing the current and future design of LRU.(for 30+min)
> > 
> >   IV) Diet of page_cgroup (for 30-min)
> >       Maybe this can be combined with III.
> 
> Looks a good plan to me, but others are more directly involved in
> memcg than me so feel free to decide! About the diet topic it was
> suggested by Johannes so I'll let him comment on it if he wants.

I suspect that we will discuss the removal of the global LRU in III,
so that would cover a major part of the diet, removing the list head.

That leaves the ideas of integrating the remaining flags field and the
mem_cgroup pointer into struct page (without increasing its size) as a
separate topic that does not fit into III, but which I would like to
discuss.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[KAMEZAWA Hiroyuki]

On Thu, 31 Mar 2011 11:01:13 +0900
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote:

> So, for this slot, I'd like to discuss
> 
>   I) Softlimit/Isolation (was 3-A) for 1hour
>      If we have extra time, kernel memory accounting or file-cache handling
>      will be good.
>    
>   II) Dirty page handling. (for 30min)
>      Maybe we'll discuss about per-memcg inode queueing issue.
> 
>   III) Discussing the current and future design of LRU.(for 30+min)
> 
>   IV) Diet of page_cgroup (for 30-min)
>       Maybe this can be combined with III.
> 

Thank you for feedbacks. I think I don't have enough time to reply all..
So, I'm sorry I make a reply in this style.

Hearing your replies and other private feedbacks, I think following schedule
will be good. And I want to spend time for topics for which patches are
posted and someone measured the cost and benefits.

1) How memcg users uses it and what's wanted ?  (for 1st session/30min)

   At start, we should sort out what's wanted as functions of memcg.
   We need to hear use cases. And see what is the problem, now. 
   Maybe we can discuss requirements for kernel memory limit, here.
   (I think we have no time to discuss implemenation in official slot.)

2) What's next ? : 30min.
   At first, Pavel will explain what OpenVZ does.
   Then, we can discuss about softlimit/isolation and LRU design.
   I think it will be a hot topic in the next half year with dirty page handling.
   
3) Dirty Page Handling.

4) Discussing LRU, costs and fairness.
   At first, need to discuss effect of per-memcg background reclaim.
   Then, discuss about new LRU design including removing page_cgroup->lru.
 
5) Diet of page_cgroup and others.
   I don't think diet topic requires full 30min. So, some extra topic is....
   It seems some guys want to overcommit vmem size with cgroup.


A topic from me was adding a memcg only-for-file-cache...but I have no patches.
Postphone until I have a concrete idea and patches.

Thanks,
-Kame






--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF][MM] rough agenda for memcg.

[Michel Lespinasse]

On Thu, Mar 31, 2011 at 2:15 AM, Zhu Yanhai <zhu.yanhai@gmail.com> wrote:
> Hi Kame,
>
> 2011/3/31 KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>:
>>  c) Should we provide a auto memory cgroup for file caches ?
>>     (Then we can implement a file-cache-limit.)
>>  c) AFAIK, some other OSs have this kind of feature, a box for file-cache.
>>     Because file-cache is a shared object between all cgroups, it's difficult
>>     to handle. It may be better to have a auto cgroup for file caches and add knobs
>>     for memcg.
>
> I have been thinking about this idea. It seems the root cause of
> current difficult is
> the whole cgroup infrastructure is based on process groups, so its counters
> naturally center on process. However, this is not nature for counters
> of file caches,
> which center on inodes/devs actually. This brought many confusing
> problems - e.g.
> who should be charged for a (dirty)file page?  I think the answer is
> no process but
> the filesystem/block device it sits on.

This has been an open issue for Google as well. Greg Thelen gave this
some though last year and had a proposal around the idea of forcing
files within certain directories to be accounted to a given cgroup.
We're not actively implementing this right now, but if there is
outside interest this might be worth discussing (might just be as an
informal conversation rather than a session though).

-- 
Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Lsf] [LSF][MM] rough agenda for memcg.

[Michel Lespinasse]

On Thu, Mar 31, 2011 at 8:59 AM, Andrea Arcangeli <aarcange@redhat.com> wrote:
> Hi KAMEZAWA,
>
> On Thu, Mar 31, 2011 at 11:01:13AM +0900, KAMEZAWA Hiroyuki wrote:
>> 1. Memory cgroup : Where next ? 1hour (Balbir Singh/Kamezawa)
>
> Originally it was 30min and then there was a topic "Working set
> estimation" for another 30 min. That has been converted to "what's
> next continued", so I assume that you can add the Working set
> estimation as a subtopic.

I see this session on the second day of the agenda. I think I'd like
to keep it a separate session; however I probably don't need the full
30 minutes; I should be able to give out the extra time to the virtual
machine memory sizing discussion that is slotted afterwards.

>> 2. Memcg Dirty Limit and writeback 30min(Greg Thelen)
>> 3. Memcg LRU management 30min (Ying Han, Michal Hocko)
>> 4. Page cgroup on a diet (Johannes Weiner)
>> 2.5 hours. This seems long...or short ? ;)
>
> Overall we've been seeing plenty of memcg emails, so I guess 2.5 hours
> are ok. And I wouldn't say we're not in the short side.

I am happy to see many parties interested in discussing memcg. I don't
think 2.5 hours are too much either.

One issue I would like to hear about is the way memcg is kept well
separated from the rest of the VM - as seen by the fact that only one
C file knows about the insides of struct mem_cgroup, that memcg has
been careful not to modify global reclaim, etc... I think this was
necessary early on when few people were interested in memcg, but now
that every major linux shop works on it it seems to me there is no
justification for that strong separation anymore. I'd like to see that
addressed explicitly because the concensus on that will affect what we
can do about memcg LRU management, page cgroup diet and a few other
topics.

>>   IV) Diet of page_cgroup (for 30-min)
>>       Maybe this can be combined with III.
>
> Looks a good plan to me, but others are more directly involved in
> memcg than me so feel free to decide! About the diet topic it was
> suggested by Johannes so I'll let him comment on it if he wants.

If the diet is successful enough, I think we could even consider
merging struct page_cgroup into struct page.

-- 
Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>