[PATCH] proc/smaps: add proportional size of anonymous page

[PATCH] proc/smaps: add proportional size of anonymous page

[Xiaokang Qin]

Anonymous page could be shared if allocated before process fork. On Android, all
applications are forked from Zygote and it makes shared anonymous page common in
Android. Currently, the "Anonymous" in smaps doesn't reflect the shared count.
A proportional anonymous page size is better to understand the anonymous page
that the applications really using.
The "proportional anonymous page size" (PropAnonymous) of a process is the count of
anonymous pages it has in memory, where each anonymous page is devided by the number
of processes sharing it.

Signed-off-by: Xiaokang Qin <xiaokang.qin@intel.com>
---
 fs/proc/task_mmu.c |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index cfa63ee..74a4f42 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -442,6 +442,7 @@ struct mem_size_stats {
 	unsigned long swap;
 	unsigned long nonlinear;
 	u64 pss;
+	u64 panon;
 };
 
 
@@ -488,12 +489,16 @@ static void smaps_pte_entry(pte_t ptent, unsigned long addr,
 		else
 			mss->shared_clean += ptent_size;
 		mss->pss += (ptent_size << PSS_SHIFT) / mapcount;
+		if (PageAnon(page))
+			mss->panon += (ptent_size << PSS_SHIFT) / mapcount;
 	} else {
 		if (pte_dirty(ptent) || PageDirty(page))
 			mss->private_dirty += ptent_size;
 		else
 			mss->private_clean += ptent_size;
 		mss->pss += (ptent_size << PSS_SHIFT);
+		if (PageAnon(page))
+			mss->panon += (ptent_size << PSS_SHIFT);
 	}
 }
 
@@ -611,6 +616,7 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
 		   "Private_Dirty:  %8lu kB\n"
 		   "Referenced:     %8lu kB\n"
 		   "Anonymous:      %8lu kB\n"
+		   "PropAnonymous:  %8lu kB\n"
 		   "AnonHugePages:  %8lu kB\n"
 		   "Swap:           %8lu kB\n"
 		   "KernelPageSize: %8lu kB\n"
@@ -625,6 +631,7 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
 		   mss.private_dirty >> 10,
 		   mss.referenced >> 10,
 		   mss.anonymous >> 10,
+		   (unsigned long)(mss.panon >> (10 + PSS_SHIFT)),
 		   mss.anonymous_thp >> 10,
 		   mss.swap >> 10,
 		   vma_kernel_pagesize(vma) >> 10,
-- 
1.7.9.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] proc/smaps: add proportional size of anonymous page

[Dave Hansen]

On 11/07/2014 12:31 AM, Xiaokang Qin wrote:
> The "proportional anonymous page size" (PropAnonymous) of a process is the count of
> anonymous pages it has in memory, where each anonymous page is devided by the number
> of processes sharing it.

This seems like the kind of thing that should just be accounted for in
the existing pss metric.  Why do we need a new, separate one?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [PATCH] proc/smaps: add proportional size of anonymous page

[Qin, Xiaokang]

Hi, Dave

For some case especially under Android, anonymous page sharing is common, for example:
70323000-70e41000 rw-p 00000000 fd:00 120004                             /data/dalvik-cache/x86/system@framework@boot.art
Size:              11384 kB
Rss:                8840 kB
Pss:                 927 kB
Shared_Clean:       5720 kB
Shared_Dirty:       2492 kB
Private_Clean:        16 kB
Private_Dirty:       612 kB
Referenced:         7896 kB
Anonymous:          3104 kB
PropAnonymous:       697 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
The only Anonymous here is confusing to me. What I really want to know is how many anonymous page is there in Pss. After exposing PropAnonymous, we could know 697/927 is anonymous in Pss.
I suppose the Pss - PropAnonymous = Proportional Page cache size for file based memory and we want to break down the page cache into process level, how much page cache each process consumes.

Regards,
Xiaokang


-----Original Message-----
From: Hansen, Dave 
Sent: Saturday, November 08, 2014 5:35 AM
To: Qin, Xiaokang; linux-mm@kvack.org
Cc: Yin, Fengwei
Subject: Re: [PATCH] proc/smaps: add proportional size of anonymous page

On 11/07/2014 12:31 AM, Xiaokang Qin wrote:
> The "proportional anonymous page size" (PropAnonymous) of a process is 
> the count of anonymous pages it has in memory, where each anonymous 
> page is devided by the number of processes sharing it.

This seems like the kind of thing that should just be accounted for in the existing pss metric.  Why do we need a new, separate one?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] proc/smaps: add proportional size of anonymous page

[Xiaokang]

On 11/08/2014 05:34 AM, Dave Hansen wrote:
> On 11/07/2014 12:31 AM, Xiaokang Qin wrote:
>> The "proportional anonymous page size" (PropAnonymous) of a process is the count of
>> anonymous pages it has in memory, where each anonymous page is devided by the number
>> of processes sharing it.
>
> This seems like the kind of thing that should just be accounted for in
> the existing pss metric.  Why do we need a new, separate one?
>
Hi, Dave

For some case especially under Android, anonymous page sharing is 
common, for example:
70323000-70e41000 rw-p 00000000 fd:00 120004 
  /data/dalvik-cache/x86/system@framework@boot.art
Size:              11384 kB
Rss:                8840 kB
Pss:                 927 kB
Shared_Clean:       5720 kB
Shared_Dirty:       2492 kB
Private_Clean:        16 kB
Private_Dirty:       612 kB
Referenced:         7896 kB
Anonymous:          3104 kB
PropAnonymous:       697 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
The only Anonymous here is confusing to me. What I really want to know 
is how many anonymous page is there in Pss. After exposing 
PropAnonymous, we could know 697/927 is anonymous in Pss.
I suppose the Pss - PropAnonymous = Proportional Page cache size for 
file based memory and we want to break down the page cache into process 
level, how much page cache each process consumes.

Regards,
Xiaokang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] proc/smaps: add proportional size of anonymous page

[Dave Hansen]

On 11/10/2014 12:48 AM, Qin, Xiaokang wrote:
> For some case especially under Android, anonymous page sharing is common, for example:
> 70323000-70e41000 rw-p 00000000 fd:00 120004                             /data/dalvik-cache/x86/system@framework@boot.art
> Size:              11384 kB
> Rss:                8840 kB
> Pss:                 927 kB
> Shared_Clean:       5720 kB
> Shared_Dirty:       2492 kB
> Private_Clean:        16 kB
> Private_Dirty:       612 kB
> Referenced:         7896 kB
> Anonymous:          3104 kB
> PropAnonymous:       697 kB

Please don't top post.

> The only Anonymous here is confusing to me. What I really want to
> know is how many anonymous page is there in Pss. After exposing
> PropAnonymous, we could know 697/927 is anonymous in Pss.
> I suppose the Pss - PropAnonymous = Proportional Page cache size for
> file based memory and we want to break down the page cache into
> process level, how much page cache each process consumes.

Ahh, so you're talking about the anonymous pages that result from
copy-on-write copies of private file mappings?  That wasn't very clear
from the description at all.

I'll agree that this definitely provides a bit of data that we didn't
have before, albeit a fairly obscure one.

But, what's the goal of this patch?  Why are you doing this?  Was there
some application whose behavior you were not able to explain before, but
can after this patch?  If the goal is providing a "Proportional Page
cache size", why do that in an indirect way?  Have you explored doing
the same measurement with /proc/$pid/pagemap?  Is it possible with that
interface?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] proc/smaps: add proportional size of anonymous page

[Xiaokang]

On 11/11/2014 01:03 AM, Dave Hansen wrote:
> On 11/10/2014 12:48 AM, Qin, Xiaokang wrote:
>> For some case especially under Android, anonymous page sharing is common, for example:
>> 70323000-70e41000 rw-p 00000000 fd:00 120004                             /data/dalvik-cache/x86/system@framework@boot.art
>> Size:              11384 kB
>> Rss:                8840 kB
>> Pss:                 927 kB
>> Shared_Clean:       5720 kB
>> Shared_Dirty:       2492 kB
>> Private_Clean:        16 kB
>> Private_Dirty:       612 kB
>> Referenced:         7896 kB
>> Anonymous:          3104 kB
>> PropAnonymous:       697 kB
>
> Please don't top post.
>
>> The only Anonymous here is confusing to me. What I really want to
>> know is how many anonymous page is there in Pss. After exposing
>> PropAnonymous, we could know 697/927 is anonymous in Pss.
>> I suppose the Pss - PropAnonymous = Proportional Page cache size for
>> file based memory and we want to break down the page cache into
>> process level, how much page cache each process consumes.
>
> Ahh, so you're talking about the anonymous pages that result from
> copy-on-write copies of private file mappings?  That wasn't very clear
> from the description at all.

Yes, sorry for the unclear description.

>
> I'll agree that this definitely provides a bit of data that we didn't
> have before, albeit a fairly obscure one.
>
> But, what's the goal of this patch?  Why are you doing this?  Was there
> some application whose behavior you were not able to explain before, but
> can after this patch?

Under Android, when user switches the application the previous 
application will not exit but switch to background so that next time 
this application could be resumed to foreground very quickly.
So there may be many applications running at background and Android 
introduces lowmemory killer to kill processes to free memory according 
to oom_adj when memory is short. Some processes with high value of 
oom_adj will be killed first like the background running non-critical 
application. The memory used by these processes could be treated as 
"free" because it could be freed by killing. Hence under Android the 
"free" RAM is composed of: 1) memory using by non-critical application 
that could be killed easily, 2) page cache, 3) physical free page. We 
are using sum of Pss in smaps to measure the process consumed memory for 
1) and get the info for 2) 3) from /proc/meminfo. Then we have the 
problem that file based memory will be double accounted in 1) and 2). To 
mitigate the double accounting issue here, we need to know the double 
accounting part - proportional page cache size, and do the deduction.

If the goal is providing a "Proportional Page
> cache size", why do that in an indirect way?  Have you explored doing
> the same measurement with /proc/$pid/pagemap?  Is it possible with that
> interface?
>
I checked the flags in /proc/kpageflags but have no idea about what 
flags should be used to identify the pagecache. It will be appreciated 
if you could provide some advice.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] proc/smaps: add proportional size of anonymous page

[Dave Hansen]

On 11/11/2014 06:40 AM, Xiaokang wrote:
> On 11/11/2014 01:03 AM, Dave Hansen wrote:
>> I'll agree that this definitely provides a bit of data that we didn't
>> have before, albeit a fairly obscure one.
>>
>> But, what's the goal of this patch?  Why are you doing this?  Was there
>> some application whose behavior you were not able to explain before, but
>> can after this patch?
> 
> Under Android, when user switches the application the previous
> application will not exit but switch to background so that next time
> this application could be resumed to foreground very quickly.
> So there may be many applications running at background and Android
> introduces lowmemory killer to kill processes to free memory according
> to oom_adj when memory is short. Some processes with high value of
> oom_adj will be killed first like the background running non-critical
> application. The memory used by these processes could be treated as
> "free" because it could be freed by killing. Hence under Android the
> "free" RAM is composed of: 1) memory using by non-critical application
> that could be killed easily, 2) page cache, 3) physical free page. We
> are using sum of Pss in smaps to measure the process consumed memory for
> 1) and get the info for 2) 3) from /proc/meminfo. Then we have the
> problem that file based memory will be double accounted in 1) and 2). To
> mitigate the double accounting issue here, we need to know the double
> accounting part - proportional page cache size, and do the deduction.

I see what you're trying to do, but I'm not convinced your approach is
effective.  Here's why:

The end goal is to say, for a given process, "If I kill $PID, I get back
X kB of memory."  You get back nothing for page cache, so you want to
subtract it out of the measurement.  You want to account for Anonymous
memory which you *do* get back, but you unfortunately *ALSO* get nothing
back for a shared anonymous page when you kill a single process.  You
need to kill *all* of the things sharing it.  If one of the things
sharing it is one of those critical applications, then you really can't
free it no matter how many non-critical apps you kill.

Let's also see some data.  How much of the memory on a system is
consumed by these pages?  How imprecise *is* the current method and how
much more precise does this make the Android OOM kills?

I think there also needs to be some level of root-cause analysis done
here.  These pages can only happen when you do:

	1. mmap(/some/file, MAP_PRIVATE);
	2. write to mmap()
	3. fork()
	4. run forever in child and never write again, and never exec()

Maybe the zygote thingy needs to be more aggressive about unmapping
things a child would never use.  Or, maybe it needs to set MADV_DONTFORK
on some things.

> If the goal is providing a "Proportional Page
>> cache size", why do that in an indirect way?  Have you explored doing
>> the same measurement with /proc/$pid/pagemap?  Is it possible with that
>> interface?
>>
> I checked the flags in /proc/kpageflags but have no idea about what
> flags should be used to identify the pagecache. It will be appreciated
> if you could provide some advice.

See Documentation/vm/pagemap.txt

If it is !ANON the it is page cache.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>