Re: How do you accurately determine a process' RAM usage?

[P@draigBrady.com]

Mauricio Lin wrote:
> Hi,
> 
> On 7/12/05, P@draigbrady.com <P@draigbrady.com> wrote:
> 
>>Andrew Morton wrote:
>>
>>>OK, please let us know how it goes.
>>
>>It went very well. I could find no problems at all.
>>I've updated my script to use the new method, so please merge smaps :)
>>http://www.pixelbeat.org/scripts/ps_mem.py
>>
>>Usually the shared mem reported by /proc/$$/statm
>>is the same as summing all the shared values in in /proc/$$/smaps
>>but there can be large discrepancies.
> 
> 
> Have you checked how the statm shared is calculated? I guess it does
> something like:
> shared = mm->rss - mm->anon_rss

yes

> But in smaps output you can have anonymous area like:
> 
> b6e0e000-b6e13000 rw-p
> Size:                20 KB
> Rss:                  4 KB
> Shared_Clean:         0 KB
> Shared_Dirty:         4 KB
> Private_Clean:        0 KB
> Private_Dirty:        0 KB
> 
> Look that it presents 4 KB of shared value in area considered anonymous.
> 
> ANDREW: anon_rss is the rss for anonymous area, right?

I see your point and I'm not sure.
The following shell gets the shared values for the
first httpd process:

FIRST_HTTPD=`ps -C httpd -o pid= | head -1 | tr -d ' '`
HTTPD_STATM_SHARED=$(expr 4 '*' `cut -f3 -d' ' /proc/$FIRST_HTTPD/statm`)
HTTPD_SMAPS_SHARED=$(grep Shared /proc/$FIRST_HTTPD/smaps | tr -s ' ' 
| cut -f2 -d' ' | ( tr '\n' +; echo 0 ) | bc)
 

This shows that "smaps" reports 3060 KB more shared mem than "statm".
However adding up all the anon sections in smaps only gives 2456 KB?

When doing this I also noticed that there are duplicate
entries in smaps. Any ideas why?

grep -F - /proc/$FIRST_HTTPD/smaps | sort | uniq -d -c

       2 b7f7d000-b7f7e000 r-xp 00000000 03:05 246646 
/usr/lib/httpd/modules/mod_auth_anon.so
       2 b7f7e000-b7f7f000 rwxp 00000000 03:05 246646 
/usr/lib/httpd/modules/mod_auth_anon.so
       2 b7f7f000-b7f81000 r-xp 00000000 03:05 246645 
/usr/lib/httpd/modules/mod_auth.so
       2 b7f81000-b7f82000 rwxp 00001000 03:05 246645 
/usr/lib/httpd/modules/mod_auth.so
       2 b7f82000-b7f84000 r-xp 00000000 03:05 246641 
/usr/lib/httpd/modules/mod_access.so
       2 b7f84000-b7f85000 rwxp 00001000 03:05 246641 
/usr/lib/httpd/modules/mod_access.so
       2 b7f85000-b7f9a000 r-xp 00000000 03:05 361234     /lib/ld-2.3.3.so
       2 b7f9a000-b7f9b000 r-xp 00014000 03:05 361234     /lib/ld-2.3.3.so
       2 b7f9b000-b7f9c000 rwxp 00015000 03:05 361234     /lib/ld-2.3.3.so
       2 bfb85000-bfb9a000 rw-p bfb85000 00:00 0          [stack]
       2 ffffe000-fffff000 ---p 00000000 00:00 0          [vdso]

If you factor that in, it means that smaps will report 156 KB
too much shared mem in this example.

>>In the real world you can see this with a newly started apache.
>>On my system statm reported that apache was using 35MB,
>>whereas smaps reported the correct amount of 11MB.
> 
> 
> How dou you know that 11MB is the correct shared value  and the 35MB
> is the wrong value?

Well I'm quite sure that COW pages ar not accounted for
in the statm shared value, which can be easily seen with
my previously posted test program. Also putting the machine
into swap, and then using httpd again causes the value reported
to be very close to 11MB (i.e. the unused pages are not swapped
back in).

-- 
Pádraig Brady - http://www.pixelbeat.org
--