About cgroup memory limits

About cgroup memory limits

[Andre Nathan]

Hello

I'm doing some tests with LXC and how it interacts with the memory
cgroup limits, more specifically the memory.limit_in_bytes control file.

Am I correct in my understanding of the memory cgroup documentation[1]
that the limit set in memory.limit_in_bytes is applied to the sum of the
fields 'cache', 'rss' and 'mapped_file' in the memory.stat file?

I am also trying to understand the values reported in memory.stat when
compared to the statistics in /proc/$PID/statm.

Below is the sum of each field in /proc/$PID/statm for every process
running inside a test container, converted to bytes:

       size  resident     share     text  lib       data  dt
  897208320  28741632  20500480  1171456    0  170676224   0

Compare this with the usage reports from memory.stat (fields total_*,
hierarchical_* and pg* omitted):

cache                     16834560
rss                       8192000
mapped_file               3743744
swap                      0
inactive_anon             0
active_anon               8192000
inactive_file             13996032
active_file               2838528
unevictable               0

Is there a way to reconcile these numbers somehow? I understand that the
fields from the two files represent different things. What I'm trying to
do is to combine, for example, the fields from memory.stat to
approximately reach what is displayed by statm.

Thank you in advance,
Andre

[1] http://www.kernel.org/doc/Documentation/cgroups/memory.txt

Re: About cgroup memory limits

[Johannes Weiner]

On Thu, May 10, 2012 at 06:38:43PM -0300, Andre Nathan wrote:
> Hello
> 
> I'm doing some tests with LXC and how it interacts with the memory
> cgroup limits, more specifically the memory.limit_in_bytes control file.
> 
> Am I correct in my understanding of the memory cgroup documentation[1]
> that the limit set in memory.limit_in_bytes is applied to the sum of the
> fields 'cache', 'rss' and 'mapped_file' in the memory.stat file?

mapped_file is the subset of cache that is mapped into virtual memory.

cache (= inactive_file + active_file) + rss (= inactive_anon +
active_anon) is what the limit applies to.

> I am also trying to understand the values reported in memory.stat when
> compared to the statistics in /proc/$PID/statm.
> 
> Below is the sum of each field in /proc/$PID/statm for every process
> running inside a test container, converted to bytes:
> 
>        size  resident     share     text  lib       data  dt
>   897208320  28741632  20500480  1171456    0  170676224   0

statms accounts based on virtual memory, not physical memory like
memcg does.  If you have the same page mapped into two tasks, both
their "share" counters will show a page, while the memcg will only
account the single physical page in mapped_file.

> Compare this with the usage reports from memory.stat (fields total_*,
> hierarchical_* and pg* omitted):
> 
> cache                     16834560
> rss                       8192000
> mapped_file               3743744
> swap                      0
> inactive_anon             0
> active_anon               8192000
> inactive_file             13996032
> active_file               2838528
> unevictable               0
> 
> Is there a way to reconcile these numbers somehow? I understand that the
> fields from the two files represent different things. What I'm trying to
> do is to combine, for example, the fields from memory.stat to
> approximately reach what is displayed by statm.

Excluding the memory shared between tasks in the same group from the
"shared" counter gets you to "mapped_file" etc.

Re: About cgroup memory limits

[Michel Machado]

Hi Johannes,

   Thank you very much for your reply, it does help us to understand the
numbers we have at hand.

   Could you clarify your following statement further:

> > Below is the sum of each field in /proc/$PID/statm for every process
> > running inside a test container, converted to bytes:
> > 
> >        size  resident     share     text  lib       data  dt
> >   897208320  28741632  20500480  1171456    0  170676224   0
> 
> statms accounts based on virtual memory, not physical memory like
> memcg does.  If you have the same page mapped into two tasks, both
> their "share" counters will show a page, while the memcg will only
> account the single physical page in mapped_file.

   You mean when those two tasks are in the same cgroup, don't you? Is
there a case in which a page is shared by two tasks that are in
different cgroups but that page is accounted only for one of the two
cgroups? If so, how's this case triggered?

-- 
[ ]'s
Michel Machado

Re: About cgroup memory limits

[Johannes Weiner]

On Wed, May 16, 2012 at 09:15:28AM -0400, Michel Machado wrote:
> Hi Johannes,
> 
>    Thank you very much for your reply, it does help us to understand the
> numbers we have at hand.
> 
>    Could you clarify your following statement further:
> 
> > > Below is the sum of each field in /proc/$PID/statm for every process
> > > running inside a test container, converted to bytes:
> > > 
> > >        size  resident     share     text  lib       data  dt
> > >   897208320  28741632  20500480  1171456    0  170676224   0
> > 
> > statms accounts based on virtual memory, not physical memory like
> > memcg does.  If you have the same page mapped into two tasks, both
> > their "share" counters will show a page, while the memcg will only
> > account the single physical page in mapped_file.
> 
>    You mean when those two tasks are in the same cgroup, don't you? Is
> there a case in which a page is shared by two tasks that are in
> different cgroups but that page is accounted only for one of the two
> cgroups? If so, how's this case triggered?

It doesn't matter whether it's the same cgroup or separate cgroups.

When two tasks are in separate memory cgroups, the shared page will be
accounted to the group of the task that is responsible for bringing it
into memory, the one touching it for the first time.