Re: How do you accurately determine a process' RAM usage?

[P@draigBrady.com]

[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]

Andrew Morton wrote:
> OK, please let us know how it goes.

It went very well. I could find no problems at all.
I've updated my script to use the new method, so please merge smaps :)
http://www.pixelbeat.org/scripts/ps_mem.py

Usually the shared mem reported by /proc/$$/statm
is the same as summing all the shared values in in /proc/$$/smaps
but there can be large discrepancies.
In the real world you can see this with a newly started apache.
On my system statm reported that apache was using 35MB,
whereas smaps reported the correct amount of 11MB.
These numbers will converge over time as the unused pages
of the apache processes were swapped out.
Attached is a mem.py script that triggers and reports
the memory discrepancy on an smaps enabled system.

As for the overhead of calling smaps, well comparing to
/proc/$$/maps which does essentially the same thing only
not walking mem, on my 1.6GHz P4, 384MB laptop, gives:

time read_10K_smaps

real    0m3.323s
user    0m0.636s
sys     0m2.660s

time read_10K_maps

real    0m1.742s
user    0m0.300s
sys     0m1.428s

I've also attached a patch to make smaps output match the maps output,
as smaps was not showing info like which map was heap and stack etc.
The code is smaller also, as show_smap() now just calls show_map().
Note this patch is to be applied _after_ Hugh's changes.

-- 
Pádraig Brady - http://www.pixelbeat.org
--

[-- Attachment #2: mem.py --]
[-- Type: text/x-python, Size: 894 bytes --]

#!/usr/bin/python
import os, sys, time
PAGESIZE=os.sysconf("SC_PAGE_SIZE")/1024 #KiB

#stat = {Shared,Rss}
#file = {smaps,statm}
def getMemStat(pid,stat,file):
    if file=="smaps":
        shared_lines=[line
                      for line in open("/proc/"+str(pid)+"/smaps").readlines()
                      if line.find(stat)!=-1]
        return sum([int(line.split()[1]) for line in shared_lines])
    else:
        if stat == "Shared": stat_index=2
        elif stat == "Rss": stat_index=1
        return int(open("/proc/"+str(pid)+"/statm").readline().split()[stat_index])*PAGESIZE

a="\x00"*(32*1024*1024) #alloc 32MiB
pid=os.fork()
if(not pid):
     time.sleep(1)
     sys.exit()

for file in ["statm", "smaps"]:
    rss=float(getMemStat(pid,"Rss",file))
    shr=float(getMemStat(pid,"Shared",file))
    print "/proc/%d/%s reports %.1f%% shared" % (pid,file,((shr/rss)*100))

os.wait()

[-- Attachment #3: linux-2.6.13-rc2-mm1-hd-smaps.diff --]
[-- Type: application/x-texinfo, Size: 1468 bytes --]