size of swapped-out part of the process

size of swapped-out part of the process

[Victoria Muntean]

How can I calculate size of swapped-out part of the process from
/proc/PID/*, even if approximately ?
For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
swap-in-use==0.
Hence (VmSize - VmRSS) is far from being swapped-out part of the
process. What is ?

Thanks
Viki

Re: size of swapped-out part of the process

[J.R. Mauro]

On Mon, Nov 17, 2008 at 7:53 AM, Victoria Muntean <vikimun@gmail.com> wrote:
> How can I calculate size of swapped-out part of the process from
> /proc/PID/*, even if approximately ?
> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
> swap-in-use==0.
> Hence (VmSize - VmRSS) is far from being swapped-out part of the
> process. What is ?

Doesn't top's SWAP field give you this?

>
> Thanks
> Viki
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: size of swapped-out part of the process

[Victoria Muntean]

On Mon, Nov 17, 2008 at 3:33 PM, J.R. Mauro <jrm8005@gmail.com> wrote:
> On Mon, Nov 17, 2008 at 7:53 AM, Victoria Muntean <vikimun@gmail.com> wrote:
>> How can I calculate size of swapped-out part of the process from
>> /proc/PID/*, even if approximately ?
>> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
>> swap-in-use==0.
>> Hence (VmSize - VmRSS) is far from being swapped-out part of the
>> process. What is ?
>
> Doesn't top's SWAP field give you this?

No. SWAP is global. I need per-process. Per-process participation in swap.

Re: size of swapped-out part of the process

[J.R. Mauro]

On Mon, Nov 17, 2008 at 8:47 AM, Victoria Muntean <vikimun@gmail.com> wrote:
> On Mon, Nov 17, 2008 at 3:33 PM, J.R. Mauro <jrm8005@gmail.com> wrote:
>> On Mon, Nov 17, 2008 at 7:53 AM, Victoria Muntean <vikimun@gmail.com> wrote:
>>> How can I calculate size of swapped-out part of the process from
>>> /proc/PID/*, even if approximately ?
>>> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
>>> swap-in-use==0.
>>> Hence (VmSize - VmRSS) is far from being swapped-out part of the
>>> process. What is ?
>>
>> Doesn't top's SWAP field give you this?
>
> No. SWAP is global. I need per-process. Per-process participation in swap.
>

Are you sure? I'm not talking about the Swap: BigNumber in the header.
If you hit f you can add a SWAP column which is per process. Or if it
isn't per-process, it's horribly misleading.

Re: size of swapped-out part of the process

[Hugh Dickins]

On Mon, 17 Nov 2008, Victoria Muntean wrote:
> How can I calculate size of swapped-out part of the process from
> /proc/PID/*, even if approximately ?
> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
> swap-in-use==0.
> Hence (VmSize - VmRSS) is far from being swapped-out part of the
> process. What is ?

I'm sorry, we don't record those numbers per process or per mm,
so you won't find them in /proc/PID/status or /proc/PID/statm.

And we'd probably resist accepting a patch to add them,
so as not to enlarge key data structures to hold them.

There's also an ambiguity about what "swapped-out" means: does
it include shared file pages which were once mapped into the
process, but have since been freed under memory pressure?

But it looks like you're interested in swapped-out to mean written
on swap.  In that case, if you have CONFIG_PROC_PAGE_MONITOR=y,
you can read /proc/PID/smaps and add up all the "Swap:" lines.

But don't keep reading it in a tight loop: that would
tend to detract from the performance of your system.

Hugh

Re: size of swapped-out part of the process

[Victoria Muntean]

On Mon, Nov 17, 2008 at 3:55 PM, J.R. Mauro <jrm8005@gmail.com> wrote:
> On Mon, Nov 17, 2008 at 8:47 AM, Victoria Muntean <vikimun@gmail.com> wrote:
>> On Mon, Nov 17, 2008 at 3:33 PM, J.R. Mauro <jrm8005@gmail.com> wrote:
>>> On Mon, Nov 17, 2008 at 7:53 AM, Victoria Muntean <vikimun@gmail.com> wrote:
>>>> How can I calculate size of swapped-out part of the process from
>>>> /proc/PID/*, even if approximately ?
>>>> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
>>>> swap-in-use==0.
>>>> Hence (VmSize - VmRSS) is far from being swapped-out part of the
>>>> process. What is ?
>>>
>>> Doesn't top's SWAP field give you this?
>>
>> No. SWAP is global. I need per-process. Per-process participation in swap.
>>
>
> Are you sure? I'm not talking about the Swap: BigNumber in the header.
> If you hit f you can add a SWAP column which is per process. Or if it
> isn't per-process, it's horribly misleading.

'top' naively shows (VmSize - VmRSS) in the SWAP column, which is not helpful.

Re: size of swapped-out part of the process

[Victoria Muntean]

On Mon, Nov 17, 2008 at 4:02 PM, Hugh Dickins <hugh@veritas.com> wrote:
> On Mon, 17 Nov 2008, Victoria Muntean wrote:
>> How can I calculate size of swapped-out part of the process from
>> /proc/PID/*, even if approximately ?
>> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
>> swap-in-use==0.
>> Hence (VmSize - VmRSS) is far from being swapped-out part of the
>> process. What is ?
>
> I'm sorry, we don't record those numbers per process or per mm,
> so you won't find them in /proc/PID/status or /proc/PID/statm.
>
> And we'd probably resist accepting a patch to add them,
> so as not to enlarge key data structures to hold them.
>
> There's also an ambiguity about what "swapped-out" means: does
> it include shared file pages which were once mapped into the
> process, but have since been freed under memory pressure?
>
> But it looks like you're interested in swapped-out to mean written
> on swap.  In that case, if you have CONFIG_PROC_PAGE_MONITOR=y,
> you can read /proc/PID/smaps and add up all the "Swap:" lines.

ok thanks

> But don't keep reading it in a tight loop: that would
> tend to detract from the performance of your system.

Re: size of swapped-out part of the process

[J.R. Mauro]

On Mon, Nov 17, 2008 at 9:08 AM, Victoria Muntean <vikimun@gmail.com> wrote:
> On Mon, Nov 17, 2008 at 3:55 PM, J.R. Mauro <jrm8005@gmail.com> wrote:
>> On Mon, Nov 17, 2008 at 8:47 AM, Victoria Muntean <vikimun@gmail.com> wrote:
>>> On Mon, Nov 17, 2008 at 3:33 PM, J.R. Mauro <jrm8005@gmail.com> wrote:
>>>> On Mon, Nov 17, 2008 at 7:53 AM, Victoria Muntean <vikimun@gmail.com> wrote:
>>>>> How can I calculate size of swapped-out part of the process from
>>>>> /proc/PID/*, even if approximately ?
>>>>> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
>>>>> swap-in-use==0.
>>>>> Hence (VmSize - VmRSS) is far from being swapped-out part of the
>>>>> process. What is ?
>>>>
>>>> Doesn't top's SWAP field give you this?
>>>
>>> No. SWAP is global. I need per-process. Per-process participation in swap.
>>>
>>
>> Are you sure? I'm not talking about the Swap: BigNumber in the header.
>> If you hit f you can add a SWAP column which is per process. Or if it
>> isn't per-process, it's horribly misleading.
>
> 'top' naively shows (VmSize - VmRSS) in the SWAP column, which is not helpful.
>

Ah, I see. The smap solution Hugh mentioned is what you're after, I guess.

Re: size of swapped-out part of the process

[Victoria Muntean]

On 11/17/08, Hugh Dickins <hugh@veritas.com> wrote:
> On Mon, 17 Nov 2008, Victoria Muntean wrote:
>> How can I calculate size of swapped-out part of the process from
>> /proc/PID/*, even if approximately ?
>> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
>> swap-in-use==0.
>> Hence (VmSize - VmRSS) is far from being swapped-out part of the
>> process. What is ?
>
> I'm sorry, we don't record those numbers per process or per mm,
> so you won't find them in /proc/PID/status or /proc/PID/statm.
>
> And we'd probably resist accepting a patch to add them,
> so as not to enlarge key data structures to hold them.
>
> There's also an ambiguity about what "swapped-out" means: does
> it include shared file pages which were once mapped into the
> process, but have since been freed under memory pressure?
>
> But it looks like you're interested in swapped-out to mean written
> on swap.  In that case, if you have CONFIG_PROC_PAGE_MONITOR=y,
> you can read /proc/PID/smaps and add up all the "Swap:" lines.

Can you possibly explain what types of pages create  "VSZ>>RSZ"
effect in total absence of swapping and plenty of free memory ?

Re: size of swapped-out part of the process

[Bryan Donlan]

On Mon, Nov 17, 2008 at 2:42 PM, Victoria Muntean <vikimun@gmail.com> wrote:
> On 11/17/08, Hugh Dickins <hugh@veritas.com> wrote:
>> On Mon, 17 Nov 2008, Victoria Muntean wrote:
>>> How can I calculate size of swapped-out part of the process from
>>> /proc/PID/*, even if approximately ?
>>> For example, I have VmSize=1216192 kB, VmRSS = 628788 kB, but global
>>> swap-in-use==0.
>>> Hence (VmSize - VmRSS) is far from being swapped-out part of the
>>> process. What is ?
>>
>> I'm sorry, we don't record those numbers per process or per mm,
>> so you won't find them in /proc/PID/status or /proc/PID/statm.
>>
>> And we'd probably resist accepting a patch to add them,
>> so as not to enlarge key data structures to hold them.
>>
>> There's also an ambiguity about what "swapped-out" means: does
>> it include shared file pages which were once mapped into the
>> process, but have since been freed under memory pressure?
>>
>> But it looks like you're interested in swapped-out to mean written
>> on swap.  In that case, if you have CONFIG_PROC_PAGE_MONITOR=y,
>> you can read /proc/PID/smaps and add up all the "Swap:" lines.
>
> Can you possibly explain what types of pages create  "VSZ>>RSZ"
> effect in total absence of swapping and plenty of free memory ?

mmap(2) a file, then don't touch some pages in it. This happens all
the time with shared libraries - for example, if you link in
imagemagick, but never use JPEG encoding/decoding, the pages of memory
corresponding to libjpeg will never be touched, and so will be
accounted in VSZ but not RSZ.

If there's memory pressure (even if you have plenty of cache, this can
occur just from IO activity), long-unaccessed pages from these mmaps
may be evicted after being read as well.

Also, some programs have a habit of allocating much more memory than
they actually use - the pages they don't access are not mapped, and so
are not accounted in RSZ either. If they never write to them, they
won't be present in swap either.

Re: size of swapped-out part of the process

[Andreas Schwab]

"Victoria Muntean" <vikimun@gmail.com> writes:

> Can you possibly explain what types of pages create  "VSZ>>RSZ"
> effect in total absence of swapping and plenty of free memory ?

Any mapping that hasn't been touched yet.  A page that is never read nor
written will never become resident.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

RE: size of swapped-out part of the process

[David Schwartz]

> On 11/17/08, Hugh Dickins <hugh@veritas.com> wrote:

> Can you possibly explain what types of pages create  "VSZ>>RSZ"
> effect in total absence of swapping and plenty of free memory ?

Rarely-used functions like 'malloc' and 'mmap' enlarge the virtual memory
size but (often) have no effect on the resident set size.

DS