smap output - unnamed entries and heap

smap output - unnamed entries and heap

[Aneesh Bhasin]

Hi All,

My apologies if this is not the right place for these type of
questions - I do not know of a better place to ask this.

I am trying to take out some memory figures from the /proc/<pid>/smaps
interface (linux v2.6.35). However, I could not find what do the
entries that have no name associated to them (no library path or
binary path with these entries) mean. So, I wrote a test C program
that allocates 100MB of memory (using malloc) and then writes to it in
chunks of 10MB. I could see that the 'Size' and 'RSS' of one of the
unnamed entries was increasing in accordance with the code, but there
were several other such unnamed sections too - is there a way to know
what and where do they come from.

Also, I could see that for some process the smap interface was showing
a '[heap]' entry but not for all - e.g. for my above test case, there
was no '[heap]' entry, although I assumed the malloc should have
reflected in the [heap]. Can someone please tell me (or point me in
the right direction) why is heap entry there only for some processes ?

Thanks in advance for any help..

Regards,
Aneesh Bhasin

Re: smap output - unnamed entries and heap

[Randy Dunlap]

On Tue, 15 Mar 2011 14:31:50 +0530 Aneesh Bhasin wrote:

> Hi All,
> 
> My apologies if this is not the right place for these type of
> questions - I do not know of a better place to ask this.
> 
> I am trying to take out some memory figures from the /proc/<pid>/smaps
> interface (linux v2.6.35). However, I could not find what do the
> entries that have no name associated to them (no library path or
> binary path with these entries) mean. So, I wrote a test C program
> that allocates 100MB of memory (using malloc) and then writes to it in
> chunks of 10MB. I could see that the 'Size' and 'RSS' of one of the
> unnamed entries was increasing in accordance with the code, but there
> were several other such unnamed sections too - is there a way to know
> what and where do they come from.
> 
> Also, I could see that for some process the smap interface was showing
> a '[heap]' entry but not for all - e.g. for my above test case, there
> was no '[heap]' entry, although I assumed the malloc should have
> reflected in the [heap]. Can someone please tell me (or point me in
> the right direction) why is heap entry there only for some processes ?
> 
> Thanks in advance for any help..

Did you look at Documentation/filesystems/proc.txt ?
It has some info on smaps and maps.

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Re: smap output - unnamed entries and heap

[Aneesh Bhasin]

On Tue, Mar 15, 2011 at 9:44 PM, Randy Dunlap <rdunlap@xenotime.net> wrote:
> On Tue, 15 Mar 2011 14:31:50 +0530 Aneesh Bhasin wrote:
>
>> Hi All,
>>
>> My apologies if this is not the right place for these type of
>> questions - I do not know of a better place to ask this.
>>
>> I am trying to take out some memory figures from the /proc/<pid>/smaps
>> interface (linux v2.6.35). However, I could not find what do the
>> entries that have no name associated to them (no library path or
>> binary path with these entries) mean. So, I wrote a test C program
>> that allocates 100MB of memory (using malloc) and then writes to it in
>> chunks of 10MB. I could see that the 'Size' and 'RSS' of one of the
>> unnamed entries was increasing in accordance with the code, but there
>> were several other such unnamed sections too - is there a way to know
>> what and where do they come from.
>>
>> Also, I could see that for some process the smap interface was showing
>> a '[heap]' entry but not for all - e.g. for my above test case, there
>> was no '[heap]' entry, although I assumed the malloc should have
>> reflected in the [heap]. Can someone please tell me (or point me in
>> the right direction) why is heap entry there only for some processes ?
>>
>> Thanks in advance for any help..
>
> Did you look at Documentation/filesystems/proc.txt ?
> It has some info on smaps and maps.
>
> ---
> ~Randy

Thanks for the pointer Randy. I had a look at it and here is my
understanding regarding this (please correct me if I am wrong)

The entries in smaps (or maps) that have no name associated with them
are anonymous memory regions. malloc, e.g. could allocate a memory
using anonymous mmap for large blocks of memory due to efficiency
reason. Are there any other such sources/reasons for creating
anonymous memory regions ?
I checked some of the above, by modifying my test program to allocate
1KB memory - this time it was attributed to [heap] whereas a memory
allocation of 1MB was attributed to one of the anonymous memory
regions.
So, does that mean that all anonymous regions can be treated
effectively as heap ? Also, is there a way in linux (without using
dedicated tools like valgrind), to see how much heap or anonymous
memory is attributed to code arising from a particular library/binary
?

Thanks for the help !

Regards,
Aneesh Bhasin

Re: smap output - unnamed entries and heap

[Pádraig Brady]

On 16/03/11 05:19, Aneesh Bhasin wrote:
> Also, is there a way in linux (without using
> dedicated tools like valgrind), to see how much heap or anonymous
> memory is attributed to code arising from a particular library/binary

You might find ps_mem.py useful.
If parses smaps and reports RAM usage of a program

http://www.pixelbeat.org/scripts/ps_mem.py

cheers,
Pádraig.

Re: smap output - unnamed entries and heap

[Aneesh Bhasin]

2011/3/16 Pádraig Brady <P@draigbrady.com>:
> On 16/03/11 05:19, Aneesh Bhasin wrote:
>> Also, is there a way in linux (without using
>> dedicated tools like valgrind), to see how much heap or anonymous
>> memory is attributed to code arising from a particular library/binary
>
> You might find ps_mem.py useful.
> If parses smaps and reports RAM usage of a program
>
> http://www.pixelbeat.org/scripts/ps_mem.py
>
> cheers,
> Pádraig.

I did try that script. Unfortunately, I not only want the memory used
for my program but also, which library is contributing to that memory.
To that end, I have written a small script which parses the smap
output in a format more suitable for my analysis.

However, as I wrote earlier, there a lot of anonymous memory regions
listed in smap output (with permission rwxp) - and I do not know of
any way in which I could associate these memory mapping to a
particular library..

Any pointers in this regard will be highly appreciated :)

Regards,
Aneesh

Re: smap output - unnamed entries and heap

[Jérôme Pouiller]

On Thursday 17 March 2011 12:46:13 Aneesh Bhasin wrote:
[..]
> However, as I wrote earlier, there a lot of anonymous memory regions
> listed in smap output (with permission rwxp) - and I do not know of
> any way in which I could associate these memory mapping to a
> particular library..
A few years ago, I had a similar need. I made a small library which overload 
malloc (and free) and display caller (using backtrace_symbols function). 
Next I loaded it using LD_PRELOAD. The main problems are :
  * it only work for malloced memory (not mmaped memory)
  * It is difficult to interpred efficiently information provided. Name of 
caller may not be sufficient to understand why memory is allocated.

--
Jérôme Pouiller <jezz@sysmic.org>