Re: [PATCH] proc/smaps: add proportional size of anonymous page

[Dave Hansen <dave.hansen@intel.com>]

On 11/11/2014 06:40 AM, Xiaokang wrote:
> On 11/11/2014 01:03 AM, Dave Hansen wrote:
>> I'll agree that this definitely provides a bit of data that we didn't
>> have before, albeit a fairly obscure one.
>>
>> But, what's the goal of this patch?  Why are you doing this?  Was there
>> some application whose behavior you were not able to explain before, but
>> can after this patch?
> 
> Under Android, when user switches the application the previous
> application will not exit but switch to background so that next time
> this application could be resumed to foreground very quickly.
> So there may be many applications running at background and Android
> introduces lowmemory killer to kill processes to free memory according
> to oom_adj when memory is short. Some processes with high value of
> oom_adj will be killed first like the background running non-critical
> application. The memory used by these processes could be treated as
> "free" because it could be freed by killing. Hence under Android the
> "free" RAM is composed of: 1) memory using by non-critical application
> that could be killed easily, 2) page cache, 3) physical free page. We
> are using sum of Pss in smaps to measure the process consumed memory for
> 1) and get the info for 2) 3) from /proc/meminfo. Then we have the
> problem that file based memory will be double accounted in 1) and 2). To
> mitigate the double accounting issue here, we need to know the double
> accounting part - proportional page cache size, and do the deduction.

I see what you're trying to do, but I'm not convinced your approach is
effective.  Here's why:

The end goal is to say, for a given process, "If I kill $PID, I get back
X kB of memory."  You get back nothing for page cache, so you want to
subtract it out of the measurement.  You want to account for Anonymous
memory which you *do* get back, but you unfortunately *ALSO* get nothing
back for a shared anonymous page when you kill a single process.  You
need to kill *all* of the things sharing it.  If one of the things
sharing it is one of those critical applications, then you really can't
free it no matter how many non-critical apps you kill.

Let's also see some data.  How much of the memory on a system is
consumed by these pages?  How imprecise *is* the current method and how
much more precise does this make the Android OOM kills?

I think there also needs to be some level of root-cause analysis done
here.  These pages can only happen when you do:

	1. mmap(/some/file, MAP_PRIVATE);
	2. write to mmap()
	3. fork()
	4. run forever in child and never write again, and never exec()

Maybe the zygote thingy needs to be more aggressive about unmapping
things a child would never use.  Or, maybe it needs to set MADV_DONTFORK
on some things.

> If the goal is providing a "Proportional Page
>> cache size", why do that in an indirect way?  Have you explored doing
>> the same measurement with /proc/$pid/pagemap?  Is it possible with that
>> interface?
>>
> I checked the flags in /proc/kpageflags but have no idea about what
> flags should be used to identify the pagecache. It will be appreciated
> if you could provide some advice.

See Documentation/vm/pagemap.txt

If it is !ANON the it is page cache.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>