Memory usage with qemu-kvm-0.12.1.1

Memory usage with qemu-kvm-0.12.1.1

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 1943 bytes --]

Hi, all!

I installed qemu-kvm-0.12.1.1 in one equipment of my house yesterday to
test it with Linux 2.6.32 compiled by myself from the source code of
kernel.org.

From the night of yesterday that I am observing a high use of swap. This
is the Service Log Entries from Nagios:

12-26-2009 21:57:33     12-26-2009 23:42:33     0d 1h 45m 0s    SERVICE WARNING (HARD)  SWAP WARNING - 30% free (142 MB out of 486 MB)
12-26-2009 23:42:33     12-27-2009 00:00:00     0d 0h 17m 27s   SERVICE CRITICAL (HARD) SWAP CRITICAL - 9% free (41 MB out of 486 MB)
12-27-2009 00:00:00     12-27-2009 06:27:33     0d 6h 27m 33s   SERVICE CRITICAL (HARD) SWAP CRITICAL - 5% free (22 MB out of 486 MB)
12-27-2009 06:27:33     12-27-2009 12:49:51     0d 6h 22m 18s+  SERVICE WARNING (HARD)  SWAP WARNING - 14% free (67 MB out of 486 MB)

The hour has been ART (GMT-3).

The VMs running in this host are the following:

+--------------------+----------+------------------+
|   OS               |    RAM   |      SWAP        |
+====================+==========+==================+
| Debian Lenny amd64 | 512 MB   | 1 GB / 0 used    |
| Debian Lenny amd64 | 512 MB   | 512 MB / 0 used  |
| Debian Lenny amd64 | 512 MB   | 1 GB / 584k used |
| Debian Lenny i386  | 512 MB   | 512 MB / 0 used  |
| OpenBSD 4.6  i386  | 512 MB   | 1 GB / 0 used    |
+--------------------+----------+------------------+

This is what I obtain with 'free' in the host:

root@ubuntu:~# free
             total       used       free     shared    buffers     cached
Mem:       4060340    3989512      70828          0    1804272      95212
-/+ buffers/cache:    2090028    1970312
Swap:       497972     415680      82292


To what that so high use of swap can be due?

Thank in advance for your replies.

Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Memory usage with qemu-kvm-0.12.1.1

[Avi Kivity]

On 12/27/2009 05:51 PM, Daniel Bareiro wrote:
> Hi, all!
>
> I installed qemu-kvm-0.12.1.1 in one equipment of my house yesterday to
> test it with Linux 2.6.32 compiled by myself from the source code of
> kernel.org.
>
>  From the night of yesterday that I am observing a high use of swap. This
> is the Service Log Entries from Nagios:
>
> 12-26-2009 21:57:33     12-26-2009 23:42:33     0d 1h 45m 0s    SERVICE WARNING (HARD)  SWAP WARNING - 30% free (142 MB out of 486 MB)
> 12-26-2009 23:42:33     12-27-2009 00:00:00     0d 0h 17m 27s   SERVICE CRITICAL (HARD) SWAP CRITICAL - 9% free (41 MB out of 486 MB)
> 12-27-2009 00:00:00     12-27-2009 06:27:33     0d 6h 27m 33s   SERVICE CRITICAL (HARD) SWAP CRITICAL - 5% free (22 MB out of 486 MB)
> 12-27-2009 06:27:33     12-27-2009 12:49:51     0d 6h 22m 18s+  SERVICE WARNING (HARD)  SWAP WARNING - 14% free (67 MB out of 486 MB)
>
> The hour has been ART (GMT-3).
>
> The VMs running in this host are the following:
>
> +--------------------+----------+------------------+
> |   OS               |    RAM   |      SWAP        |
> +====================+==========+==================+
> | Debian Lenny amd64 | 512 MB   | 1 GB / 0 used    |
> | Debian Lenny amd64 | 512 MB   | 512 MB / 0 used  |
> | Debian Lenny amd64 | 512 MB   | 1 GB / 584k used |
> | Debian Lenny i386  | 512 MB   | 512 MB / 0 used  |
> | OpenBSD 4.6  i386  | 512 MB   | 1 GB / 0 used    |
> +--------------------+----------+------------------+
>
> This is what I obtain with 'free' in the host:
>
> root@ubuntu:~# free
>               total       used       free     shared    buffers     cached
> Mem:       4060340    3989512      70828          0    1804272      95212
> -/+ buffers/cache:    2090028    1970312
> Swap:       497972     415680      82292
>
>
> To what that so high use of swap can be due?
>
>    

Probably a regression in Linux swapping.  Rik, Hugh, are you aware of 
any?  Hugh posted something but it appears to be performance related, 
not causing early swap.

Try setting /proc/sys/vm/swappiness to 0, and running your guests with 
cache=none (a good idea in any case).

Also, qemu might be leaking memory.  Please post 'pmap $pid' for all of 
your guests (do that before any of the other tests, on your swapped-out 
system).

-- 
error compiling committee.c: too many arguments to function

Re: Memory usage with qemu-kvm-0.12.1.1

[Rik van Riel]

On 12/27/2009 11:03 AM, Avi Kivity wrote:
> On 12/27/2009 05:51 PM, Daniel Bareiro wrote:
>> Hi, all!
>>
>> I installed qemu-kvm-0.12.1.1 in one equipment of my house yesterday to
>> test it with Linux 2.6.32 compiled by myself from the source code of
>> kernel.org.
>>
>> From the night of yesterday that I am observing a high use of swap. This
>> is the Service Log Entries from Nagios:
>>
>> 12-26-2009 21:57:33 12-26-2009 23:42:33 0d 1h 45m 0s SERVICE WARNING
>> (HARD) SWAP WARNING - 30% free (142 MB out of 486 MB)
>> 12-26-2009 23:42:33 12-27-2009 00:00:00 0d 0h 17m 27s SERVICE CRITICAL
>> (HARD) SWAP CRITICAL - 9% free (41 MB out of 486 MB)
>> 12-27-2009 00:00:00 12-27-2009 06:27:33 0d 6h 27m 33s SERVICE CRITICAL
>> (HARD) SWAP CRITICAL - 5% free (22 MB out of 486 MB)
>> 12-27-2009 06:27:33 12-27-2009 12:49:51 0d 6h 22m 18s+ SERVICE WARNING
>> (HARD) SWAP WARNING - 14% free (67 MB out of 486 MB)
>>
>> The hour has been ART (GMT-3).
>>
>> The VMs running in this host are the following:
>>
>> +--------------------+----------+------------------+
>> | OS | RAM | SWAP |
>> +====================+==========+==================+
>> | Debian Lenny amd64 | 512 MB | 1 GB / 0 used |
>> | Debian Lenny amd64 | 512 MB | 512 MB / 0 used |
>> | Debian Lenny amd64 | 512 MB | 1 GB / 584k used |
>> | Debian Lenny i386 | 512 MB | 512 MB / 0 used |
>> | OpenBSD 4.6 i386 | 512 MB | 1 GB / 0 used |
>> +--------------------+----------+------------------+
>>
>> This is what I obtain with 'free' in the host:
>>
>> root@ubuntu:~# free
>> total used free shared buffers cached
>> Mem: 4060340 3989512 70828 0 1804272 95212
>> -/+ buffers/cache: 2090028 1970312
>> Swap: 497972 415680 82292
>>
>>
>> To what that so high use of swap can be due?
>>
>
> Probably a regression in Linux swapping. Rik, Hugh, are you aware of
> any? Hugh posted something but it appears to be performance related, not
> causing early swap.

Yes, it is a smal bug in the VM.

A fix has been committed to 2.6.33 already.

-- 
All rights reversed.

Re: Memory usage with qemu-kvm-0.12.1.1

[Avi Kivity]

On 12/27/2009 06:32 PM, Rik van Riel wrote:
>> Probably a regression in Linux swapping. Rik, Hugh, are you aware of
>> any? Hugh posted something but it appears to be performance related, not
>> causing early swap.
>
>
> Yes, it is a smal bug in the VM.
>
> A fix has been committed to 2.6.33 already.
>

Is this b39415b273?

If so, it doesn't copy stable@kernel.org.  Is it queued for -stable?

-- 
error compiling committee.c: too many arguments to function

Re: Memory usage with qemu-kvm-0.12.1.1

[Rik van Riel]

On 12/27/2009 11:38 AM, Avi Kivity wrote:
> On 12/27/2009 06:32 PM, Rik van Riel wrote:
>>> Probably a regression in Linux swapping. Rik, Hugh, are you aware of
>>> any? Hugh posted something but it appears to be performance related, not
>>> causing early swap.
>>
>> Yes, it is a smal bug in the VM.
>>
>> A fix has been committed to 2.6.33 already.
>
> Is this b39415b273?

Indeed it is.

> If so, it doesn't copy stable@kernel.org. Is it queued for -stable?

I do not believe that it is queued for -stable.

Do performance fixes fit with -stable policy?

-- 
All rights reversed.

Re: Memory usage with qemu-kvm-0.12.1.1

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 29524 bytes --]

Hi, Avi.

On Sunday, 27 December 2009 18:03:18 +0200,
Avi Kivity wrote:

>> I installed qemu-kvm-0.12.1.1 in one equipment of my house yesterday
>> to test it with Linux 2.6.32 compiled by myself from the source code
>> of kernel.org.

>> [...]
>>
>> This is what I obtain with 'free' in the host:
>>
>> root@ubuntu:~# free
>>               total       used       free     shared    buffers     cached
>> Mem:       4060340    3989512      70828          0    1804272      95212
>> -/+ buffers/cache:    2090028    1970312
>> Swap:       497972     415680      82292
>>
>>
>> To what that so high use of swap can be due?

> Probably a regression in Linux swapping.  Rik, Hugh, are you aware of
> any?  Hugh posted something but it appears to be performance related,
> not causing early swap.

> Try setting /proc/sys/vm/swappiness to 0, and running your guests with
> cache=none (a good idea in any case).

Now that you mention "cache=none", I read it in the document of KVM
tuning [1], although I didn't apply it yet. I will consider it to future
in my other installations with KVM-88 and qemu-kvm-0.12.1.1 after
solving this problem.

> Also, qemu might be leaking memory.  Please post 'pmap $pid' for all of  
> your guests (do that before any of the other tests, on your swapped-out  
> system).

---------------------------------------------------------------------
root@ubuntu:~# pmap 3348
3348:   /usr/local/qemu-kvm/bin/qemu-system-x86_64 -hda /dev/vm/backup-raiz -hdb /dev/vm/backup-space -m 512 -boot c -net nic,vlan=0,macaddr=00:16:3e:00:00:32 -net tap -daemonize -vnc :1 -k es -localtime -serial telnet:localhost:4002,server,nowait -monitor telnet:localhost:4042,server,nowait
0000000000400000   2220K r-x--  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000082b000    124K rw---  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000084a000   8060K rw---    [ anon ]
000000004153d000      4K -----    [ anon ]
000000004153e000   8192K rw---    [ anon ]
0000000041d3e000      4K -----    [ anon ]
0000000041d3f000   8192K rw---    [ anon ]
00007f7c104a4000   3012K rw---    [ anon ]
00007f7c10795000    136K rw---    [ anon ]
00007f7c107b7000 540688K rw---    [ anon ]
00007f7c317bb000     40K r-x--  /lib/libnss_files-2.7.so
00007f7c317c5000   2048K -----  /lib/libnss_files-2.7.so
00007f7c319c5000      8K rw---  /lib/libnss_files-2.7.so
00007f7c319c7000     20K r-x--  /usr/lib/libXdmcp.so.6.0.0
00007f7c319cc000   2044K -----  /usr/lib/libXdmcp.so.6.0.0
00007f7c31bcb000      4K rw---  /usr/lib/libXdmcp.so.6.0.0
00007f7c31bcc000      8K r-x--  /usr/lib/libXau.so.6.0.0
00007f7c31bce000   2044K -----  /usr/lib/libXau.so.6.0.0
00007f7c31dcd000      4K rw---  /usr/lib/libXau.so.6.0.0
00007f7c31dce000     12K r-x--  /lib/libgpg-error.so.0.3.0
00007f7c31dd1000   2044K -----  /lib/libgpg-error.so.0.3.0
00007f7c31fd0000      4K rw---  /lib/libgpg-error.so.0.3.0
00007f7c31fd1000    108K r-x--  /usr/lib/libxcb.so.1.0.0
00007f7c31fec000   2044K -----  /usr/lib/libxcb.so.1.0.0
00007f7c321eb000      4K rw---  /usr/lib/libxcb.so.1.0.0
00007f7c321ec000      4K r-x--  /usr/lib/libxcb-xlib.so.0.0.0
00007f7c321ed000   2044K -----  /usr/lib/libxcb-xlib.so.0.0.0
00007f7c323ec000      4K rw---  /usr/lib/libxcb-xlib.so.0.0.0
00007f7c323ed000     80K r-x--  /usr/lib/libdirect-1.0.so.0.1.0
00007f7c32401000   2044K -----  /usr/lib/libdirect-1.0.so.0.1.0
00007f7c32600000      8K rw---  /usr/lib/libdirect-1.0.so.0.1.0
00007f7c32602000     32K r-x--  /usr/lib/libfusion-1.0.so.0.1.0
00007f7c3260a000   2044K -----  /usr/lib/libfusion-1.0.so.0.1.0
00007f7c32809000      4K rw---  /usr/lib/libfusion-1.0.so.0.1.0
00007f7c3280a000    428K r-x--  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f7c32875000   2048K -----  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f7c32a75000     12K rw---  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f7c32a78000    860K r-x--  /usr/lib/libasound.so.2.0.0
00007f7c32b4f000   2048K -----  /usr/lib/libasound.so.2.0.0
00007f7c32d4f000     28K rw---  /usr/lib/libasound.so.2.0.0
00007f7c32d56000    304K r-x--  /lib/libgcrypt.so.11.2.3
00007f7c32da2000   2044K -----  /lib/libgcrypt.so.11.2.3
00007f7c32fa1000     12K rw---  /lib/libgcrypt.so.11.2.3
00007f7c32fa4000     64K r-x--  /usr/lib/libtasn1.so.3.0.12
00007f7c32fb4000   2044K -----  /usr/lib/libtasn1.so.3.0.12
00007f7c331b3000      4K rw---  /usr/lib/libtasn1.so.3.0.12
00007f7c331b4000      8K r-x--  /lib/libdl-2.7.so
00007f7c331b6000   2048K -----  /lib/libdl-2.7.so
00007f7c333b6000      8K rw---  /lib/libdl-2.7.so
00007f7c333b8000   1376K r-x--  /lib/libc-2.7.so
00007f7c33510000   2048K -----  /lib/libc-2.7.so
00007f7c33710000     12K r----  /lib/libc-2.7.so
00007f7c33713000      8K rw---  /lib/libc-2.7.so
00007f7c33715000     20K rw---    [ anon ]
00007f7c3371a000    512K r-x--  /lib/libm-2.7.so
00007f7c3379a000   2044K -----  /lib/libm-2.7.so
00007f7c33999000      8K rw---  /lib/libm-2.7.so
00007f7c3399b000   1020K r-x--  /usr/lib/libX11.so.6.2.0
00007f7c33a9a000   2044K -----  /usr/lib/libX11.so.6.2.0
00007f7c33c99000     20K rw---  /usr/lib/libX11.so.6.2.0
00007f7c33c9e000    412K r-x--  /usr/lib/libSDL-1.2.so.0.11.1
00007f7c33d05000   2048K -----  /usr/lib/libSDL-1.2.so.0.11.1
00007f7c33f05000      8K rw---  /usr/lib/libSDL-1.2.so.0.11.1
00007f7c33f07000    176K rw---    [ anon ]
00007f7c33f33000    492K r-x--  /usr/lib/libgnutls.so.13.9.1
00007f7c33fae000   2048K -----  /usr/lib/libgnutls.so.13.9.1
00007f7c341ae000     40K rw---  /usr/lib/libgnutls.so.13.9.1
00007f7c341b8000    220K r-x--  /lib/libncurses.so.5.6
00007f7c341ef000   2044K -----  /lib/libncurses.so.5.6
00007f7c343ee000     20K rw---  /lib/libncurses.so.5.6
00007f7c343f3000     88K r-x--  /usr/lib/libz.so.1.2.3.3
00007f7c34409000   2048K -----  /usr/lib/libz.so.1.2.3.3
00007f7c34609000      4K rw---  /usr/lib/libz.so.1.2.3.3
00007f7c3460a000      8K r-x--  /lib/libutil-2.7.so
00007f7c3460c000   2044K -----  /lib/libutil-2.7.so
00007f7c3480b000      8K rw---  /lib/libutil-2.7.so
00007f7c3480d000     88K r-x--  /lib/libpthread-2.7.so
00007f7c34823000   2048K -----  /lib/libpthread-2.7.so
00007f7c34a23000      8K rw---  /lib/libpthread-2.7.so
00007f7c34a25000     16K rw---    [ anon ]
00007f7c34a29000     32K r-x--  /lib/librt-2.7.so
00007f7c34a31000   2044K -----  /lib/librt-2.7.so
00007f7c34c30000      8K rw---  /lib/librt-2.7.so
00007f7c34c32000    116K r-x--  /lib/ld-2.7.so
00007f7c34c68000   1472K rw---    [ anon ]
00007f7c34df9000    308K rw---    [ anon ]
00007f7c34e49000     12K rw-s-    [ anon ]
00007f7c34e4c000     12K rw---    [ anon ]
00007f7c34e4f000      8K rw---  /lib/ld-2.7.so
00007fff03de5000     84K rw---    [ stack ]
00007fff03dff000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
 total           626376K
---------------------------------------------------------------------

root@ubuntu:~# pmap 3357
3357:   /usr/local/qemu-kvm/bin/qemu-system-x86_64 -hda /dev/vm/central-raiz -hdb /dev/vm/central-space -m 512 -boot c -net nic,vlan=0,macaddr=00:16:3e:00:00:36 -net tap -daemonize -vnc :5 -k es -localtime -serial telnet:localhost:4005,server,nowait -monitor telnet:localhost:4045,server,nowait
0000000000400000   2220K r-x--  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000082b000    124K rw---  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000084a000   8156K rw---    [ anon ]
0000000040e5b000      4K -----    [ anon ]
0000000040e5c000   8192K rw---    [ anon ]
000000004165c000      4K -----    [ anon ]
000000004165d000   8192K rw---    [ anon ]
00007f00c5dd0000   3012K rw---    [ anon ]
00007f00c60c1000    136K rw---    [ anon ]
00007f00c60e3000 540688K rw---    [ anon ]
00007f00e70e7000     40K r-x--  /lib/libnss_files-2.7.so
00007f00e70f1000   2048K -----  /lib/libnss_files-2.7.so
00007f00e72f1000      8K rw---  /lib/libnss_files-2.7.so
00007f00e72f3000     20K r-x--  /usr/lib/libXdmcp.so.6.0.0
00007f00e72f8000   2044K -----  /usr/lib/libXdmcp.so.6.0.0
00007f00e74f7000      4K rw---  /usr/lib/libXdmcp.so.6.0.0
00007f00e74f8000      8K r-x--  /usr/lib/libXau.so.6.0.0
00007f00e74fa000   2044K -----  /usr/lib/libXau.so.6.0.0
00007f00e76f9000      4K rw---  /usr/lib/libXau.so.6.0.0
00007f00e76fa000     12K r-x--  /lib/libgpg-error.so.0.3.0
00007f00e76fd000   2044K -----  /lib/libgpg-error.so.0.3.0
00007f00e78fc000      4K rw---  /lib/libgpg-error.so.0.3.0
00007f00e78fd000    108K r-x--  /usr/lib/libxcb.so.1.0.0
00007f00e7918000   2044K -----  /usr/lib/libxcb.so.1.0.0
00007f00e7b17000      4K rw---  /usr/lib/libxcb.so.1.0.0
00007f00e7b18000      4K r-x--  /usr/lib/libxcb-xlib.so.0.0.0
00007f00e7b19000   2044K -----  /usr/lib/libxcb-xlib.so.0.0.0
00007f00e7d18000      4K rw---  /usr/lib/libxcb-xlib.so.0.0.0
00007f00e7d19000     80K r-x--  /usr/lib/libdirect-1.0.so.0.1.0
00007f00e7d2d000   2044K -----  /usr/lib/libdirect-1.0.so.0.1.0
00007f00e7f2c000      8K rw---  /usr/lib/libdirect-1.0.so.0.1.0
00007f00e7f2e000     32K r-x--  /usr/lib/libfusion-1.0.so.0.1.0
00007f00e7f36000   2044K -----  /usr/lib/libfusion-1.0.so.0.1.0
00007f00e8135000      4K rw---  /usr/lib/libfusion-1.0.so.0.1.0
00007f00e8136000    428K r-x--  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f00e81a1000   2048K -----  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f00e83a1000     12K rw---  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f00e83a4000    860K r-x--  /usr/lib/libasound.so.2.0.0
00007f00e847b000   2048K -----  /usr/lib/libasound.so.2.0.0
00007f00e867b000     28K rw---  /usr/lib/libasound.so.2.0.0
00007f00e8682000    304K r-x--  /lib/libgcrypt.so.11.2.3
00007f00e86ce000   2044K -----  /lib/libgcrypt.so.11.2.3
00007f00e88cd000     12K rw---  /lib/libgcrypt.so.11.2.3
00007f00e88d0000     64K r-x--  /usr/lib/libtasn1.so.3.0.12
00007f00e88e0000   2044K -----  /usr/lib/libtasn1.so.3.0.12
00007f00e8adf000      4K rw---  /usr/lib/libtasn1.so.3.0.12
00007f00e8ae0000      8K r-x--  /lib/libdl-2.7.so
00007f00e8ae2000   2048K -----  /lib/libdl-2.7.so
00007f00e8ce2000      8K rw---  /lib/libdl-2.7.so
00007f00e8ce4000   1376K r-x--  /lib/libc-2.7.so
00007f00e8e3c000   2048K -----  /lib/libc-2.7.so
00007f00e903c000     12K r----  /lib/libc-2.7.so
00007f00e903f000      8K rw---  /lib/libc-2.7.so
00007f00e9041000     20K rw---    [ anon ]
00007f00e9046000    512K r-x--  /lib/libm-2.7.so
00007f00e90c6000   2044K -----  /lib/libm-2.7.so
00007f00e92c5000      8K rw---  /lib/libm-2.7.so
00007f00e92c7000   1020K r-x--  /usr/lib/libX11.so.6.2.0
00007f00e93c6000   2044K -----  /usr/lib/libX11.so.6.2.0
00007f00e95c5000     20K rw---  /usr/lib/libX11.so.6.2.0
00007f00e95ca000    412K r-x--  /usr/lib/libSDL-1.2.so.0.11.1
00007f00e9631000   2048K -----  /usr/lib/libSDL-1.2.so.0.11.1
00007f00e9831000      8K rw---  /usr/lib/libSDL-1.2.so.0.11.1
00007f00e9833000    176K rw---    [ anon ]
00007f00e985f000    492K r-x--  /usr/lib/libgnutls.so.13.9.1
00007f00e98da000   2048K -----  /usr/lib/libgnutls.so.13.9.1
00007f00e9ada000     40K rw---  /usr/lib/libgnutls.so.13.9.1
00007f00e9ae4000    220K r-x--  /lib/libncurses.so.5.6
00007f00e9b1b000   2044K -----  /lib/libncurses.so.5.6
00007f00e9d1a000     20K rw---  /lib/libncurses.so.5.6
00007f00e9d1f000     88K r-x--  /usr/lib/libz.so.1.2.3.3
00007f00e9d35000   2048K -----  /usr/lib/libz.so.1.2.3.3
00007f00e9f35000      4K rw---  /usr/lib/libz.so.1.2.3.3
00007f00e9f36000      8K r-x--  /lib/libutil-2.7.so
00007f00e9f38000   2044K -----  /lib/libutil-2.7.so
00007f00ea137000      8K rw---  /lib/libutil-2.7.so
00007f00ea139000     88K r-x--  /lib/libpthread-2.7.so
00007f00ea14f000   2048K -----  /lib/libpthread-2.7.so
00007f00ea34f000      8K rw---  /lib/libpthread-2.7.so
00007f00ea351000     16K rw---    [ anon ]
00007f00ea355000     32K r-x--  /lib/librt-2.7.so
00007f00ea35d000   2044K -----  /lib/librt-2.7.so
00007f00ea55c000      8K rw---  /lib/librt-2.7.so
00007f00ea55e000    116K r-x--  /lib/ld-2.7.so
00007f00ea594000   1472K rw---    [ anon ]
00007f00ea725000    308K rw---    [ anon ]
00007f00ea775000     12K rw-s-    [ anon ]
00007f00ea778000     12K rw---    [ anon ]
00007f00ea77b000      8K rw---  /lib/ld-2.7.so
00007fff1f553000     84K rw---    [ stack ]
00007fff1f5ff000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
 total           626472K
---------------------------------------------------------------------

root@ubuntu:~# pmap 3366
3366:   /usr/local/qemu-kvm/bin/qemu-system-x86_64 -hda /dev/vm/ws2-raiz -hdb /dev/vm/ws2-space -m 512 -boot c -net nic,vlan=0,macaddr=00:16:3e:00:00:31 -net tap -daemonize -vnc :0 -k es -localtime -serial telnet:localhost:4000,server,nowait
0000000000400000   2220K r-x--  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000082b000    124K rw---  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000084a000   8080K rw---    [ anon ]
0000000041988000      4K -----    [ anon ]
0000000041989000   8192K rw---    [ anon ]
0000000042189000      4K -----    [ anon ]
000000004218a000   8192K rw---    [ anon ]
00007ff0ecb15000   3012K rw---    [ anon ]
00007ff0ece06000    136K rw---    [ anon ]
00007ff0ece28000 540688K rw---    [ anon ]
00007ff10de2c000     40K r-x--  /lib/libnss_files-2.7.so
00007ff10de36000   2048K -----  /lib/libnss_files-2.7.so
00007ff10e036000      8K rw---  /lib/libnss_files-2.7.so
00007ff10e038000     20K r-x--  /usr/lib/libXdmcp.so.6.0.0
00007ff10e03d000   2044K -----  /usr/lib/libXdmcp.so.6.0.0
00007ff10e23c000      4K rw---  /usr/lib/libXdmcp.so.6.0.0
00007ff10e23d000      8K r-x--  /usr/lib/libXau.so.6.0.0
00007ff10e23f000   2044K -----  /usr/lib/libXau.so.6.0.0
00007ff10e43e000      4K rw---  /usr/lib/libXau.so.6.0.0
00007ff10e43f000     12K r-x--  /lib/libgpg-error.so.0.3.0
00007ff10e442000   2044K -----  /lib/libgpg-error.so.0.3.0
00007ff10e641000      4K rw---  /lib/libgpg-error.so.0.3.0
00007ff10e642000    108K r-x--  /usr/lib/libxcb.so.1.0.0
00007ff10e65d000   2044K -----  /usr/lib/libxcb.so.1.0.0
00007ff10e85c000      4K rw---  /usr/lib/libxcb.so.1.0.0
00007ff10e85d000      4K r-x--  /usr/lib/libxcb-xlib.so.0.0.0
00007ff10e85e000   2044K -----  /usr/lib/libxcb-xlib.so.0.0.0
00007ff10ea5d000      4K rw---  /usr/lib/libxcb-xlib.so.0.0.0
00007ff10ea5e000     80K r-x--  /usr/lib/libdirect-1.0.so.0.1.0
00007ff10ea72000   2044K -----  /usr/lib/libdirect-1.0.so.0.1.0
00007ff10ec71000      8K rw---  /usr/lib/libdirect-1.0.so.0.1.0
00007ff10ec73000     32K r-x--  /usr/lib/libfusion-1.0.so.0.1.0
00007ff10ec7b000   2044K -----  /usr/lib/libfusion-1.0.so.0.1.0
00007ff10ee7a000      4K rw---  /usr/lib/libfusion-1.0.so.0.1.0
00007ff10ee7b000    428K r-x--  /usr/lib/libdirectfb-1.0.so.0.1.0
00007ff10eee6000   2048K -----  /usr/lib/libdirectfb-1.0.so.0.1.0
00007ff10f0e6000     12K rw---  /usr/lib/libdirectfb-1.0.so.0.1.0
00007ff10f0e9000    860K r-x--  /usr/lib/libasound.so.2.0.0
00007ff10f1c0000   2048K -----  /usr/lib/libasound.so.2.0.0
00007ff10f3c0000     28K rw---  /usr/lib/libasound.so.2.0.0
00007ff10f3c7000    304K r-x--  /lib/libgcrypt.so.11.2.3
00007ff10f413000   2044K -----  /lib/libgcrypt.so.11.2.3
00007ff10f612000     12K rw---  /lib/libgcrypt.so.11.2.3
00007ff10f615000     64K r-x--  /usr/lib/libtasn1.so.3.0.12
00007ff10f625000   2044K -----  /usr/lib/libtasn1.so.3.0.12
00007ff10f824000      4K rw---  /usr/lib/libtasn1.so.3.0.12
00007ff10f825000      8K r-x--  /lib/libdl-2.7.so
00007ff10f827000   2048K -----  /lib/libdl-2.7.so
00007ff10fa27000      8K rw---  /lib/libdl-2.7.so
00007ff10fa29000   1376K r-x--  /lib/libc-2.7.so
00007ff10fb81000   2048K -----  /lib/libc-2.7.so
00007ff10fd81000     12K r----  /lib/libc-2.7.so
00007ff10fd84000      8K rw---  /lib/libc-2.7.so
00007ff10fd86000     20K rw---    [ anon ]
00007ff10fd8b000    512K r-x--  /lib/libm-2.7.so
00007ff10fe0b000   2044K -----  /lib/libm-2.7.so
00007ff11000a000      8K rw---  /lib/libm-2.7.so
00007ff11000c000   1020K r-x--  /usr/lib/libX11.so.6.2.0
00007ff11010b000   2044K -----  /usr/lib/libX11.so.6.2.0
00007ff11030a000     20K rw---  /usr/lib/libX11.so.6.2.0
00007ff11030f000    412K r-x--  /usr/lib/libSDL-1.2.so.0.11.1
00007ff110376000   2048K -----  /usr/lib/libSDL-1.2.so.0.11.1
00007ff110576000      8K rw---  /usr/lib/libSDL-1.2.so.0.11.1
00007ff110578000    176K rw---    [ anon ]
00007ff1105a4000    492K r-x--  /usr/lib/libgnutls.so.13.9.1
00007ff11061f000   2048K -----  /usr/lib/libgnutls.so.13.9.1
00007ff11081f000     40K rw---  /usr/lib/libgnutls.so.13.9.1
00007ff110829000    220K r-x--  /lib/libncurses.so.5.6
00007ff110860000   2044K -----  /lib/libncurses.so.5.6
00007ff110a5f000     20K rw---  /lib/libncurses.so.5.6
00007ff110a64000     88K r-x--  /usr/lib/libz.so.1.2.3.3
00007ff110a7a000   2048K -----  /usr/lib/libz.so.1.2.3.3
00007ff110c7a000      4K rw---  /usr/lib/libz.so.1.2.3.3
00007ff110c7b000      8K r-x--  /lib/libutil-2.7.so
00007ff110c7d000   2044K -----  /lib/libutil-2.7.so
00007ff110e7c000      8K rw---  /lib/libutil-2.7.so
00007ff110e7e000     88K r-x--  /lib/libpthread-2.7.so
00007ff110e94000   2048K -----  /lib/libpthread-2.7.so
00007ff111094000      8K rw---  /lib/libpthread-2.7.so
00007ff111096000     16K rw---    [ anon ]
00007ff11109a000     32K r-x--  /lib/librt-2.7.so
00007ff1110a2000   2044K -----  /lib/librt-2.7.so
00007ff1112a1000      8K rw---  /lib/librt-2.7.so
00007ff1112a3000    116K r-x--  /lib/ld-2.7.so
00007ff1112d9000   1472K rw---    [ anon ]
00007ff11146a000    308K rw---    [ anon ]
00007ff1114ba000     12K rw-s-    [ anon ]
00007ff1114bd000     12K rw---    [ anon ]
00007ff1114c0000      8K rw---  /lib/ld-2.7.so
00007fff663c3000     84K rw---    [ stack ]
00007fff663ff000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
 total           626396K
---------------------------------------------------------------------

root@ubuntu:~# pmap 8636
8636:   /usr/local/qemu-kvm/bin/qemu-system-x86_64 -hda /dev/vm/test-disk -m 512 -boot c -net nic,vlan=0,macaddr=00:16:3e:00:00:34 -net tap -daemonize -vnc :3 -k es -localtime -serial telnet:localhost:5000,server,nowait -monitor telnet:localhost:5001,server,nowait
0000000000400000   2220K r-x--  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000082b000    124K rw---  /usr/local/qemu-kvm/bin/qemu-system-x86_64 (deleted)
000000000084a000  10060K rw---    [ anon ]
0000000040568000      4K -----    [ anon ]
0000000040569000   8192K rw---    [ anon ]
0000000040f20000      4K -----    [ anon ]
0000000040f21000   8192K rw---    [ anon ]
0000000041a86000      4K -----    [ anon ]
0000000041a87000   8192K rw---    [ anon ]
00007f8a36a5a000    772K rw---    [ anon ]
00007f8a36c48000   1036K rw---    [ anon ]
00007f8a36d4b000    136K rw---    [ anon ]
00007f8a36d6d000 540688K rw---    [ anon ]
00007f8a57d71000     40K r-x--  /lib/libnss_files-2.7.so
00007f8a57d7b000   2048K -----  /lib/libnss_files-2.7.so
00007f8a57f7b000      8K rw---  /lib/libnss_files-2.7.so
00007f8a57f7d000     20K r-x--  /usr/lib/libXdmcp.so.6.0.0
00007f8a57f82000   2044K -----  /usr/lib/libXdmcp.so.6.0.0
00007f8a58181000      4K rw---  /usr/lib/libXdmcp.so.6.0.0
00007f8a58182000      8K r-x--  /usr/lib/libXau.so.6.0.0
00007f8a58184000   2044K -----  /usr/lib/libXau.so.6.0.0
00007f8a58383000      4K rw---  /usr/lib/libXau.so.6.0.0
00007f8a58384000     12K r-x--  /lib/libgpg-error.so.0.3.0
00007f8a58387000   2044K -----  /lib/libgpg-error.so.0.3.0
00007f8a58586000      4K rw---  /lib/libgpg-error.so.0.3.0
00007f8a58587000    108K r-x--  /usr/lib/libxcb.so.1.0.0
00007f8a585a2000   2044K -----  /usr/lib/libxcb.so.1.0.0
00007f8a587a1000      4K rw---  /usr/lib/libxcb.so.1.0.0
00007f8a587a2000      4K r-x--  /usr/lib/libxcb-xlib.so.0.0.0
00007f8a587a3000   2044K -----  /usr/lib/libxcb-xlib.so.0.0.0
00007f8a589a2000      4K rw---  /usr/lib/libxcb-xlib.so.0.0.0
00007f8a589a3000     80K r-x--  /usr/lib/libdirect-1.0.so.0.1.0
00007f8a589b7000   2044K -----  /usr/lib/libdirect-1.0.so.0.1.0
00007f8a58bb6000      8K rw---  /usr/lib/libdirect-1.0.so.0.1.0
00007f8a58bb8000     32K r-x--  /usr/lib/libfusion-1.0.so.0.1.0
00007f8a58bc0000   2044K -----  /usr/lib/libfusion-1.0.so.0.1.0
00007f8a58dbf000      4K rw---  /usr/lib/libfusion-1.0.so.0.1.0
00007f8a58dc0000    428K r-x--  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f8a58e2b000   2048K -----  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f8a5902b000     12K rw---  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f8a5902e000    860K r-x--  /usr/lib/libasound.so.2.0.0
00007f8a59105000   2048K -----  /usr/lib/libasound.so.2.0.0
00007f8a59305000     28K rw---  /usr/lib/libasound.so.2.0.0
00007f8a5930c000    304K r-x--  /lib/libgcrypt.so.11.2.3
00007f8a59358000   2044K -----  /lib/libgcrypt.so.11.2.3
00007f8a59557000     12K rw---  /lib/libgcrypt.so.11.2.3
00007f8a5955a000     64K r-x--  /usr/lib/libtasn1.so.3.0.12
00007f8a5956a000   2044K -----  /usr/lib/libtasn1.so.3.0.12
00007f8a59769000      4K rw---  /usr/lib/libtasn1.so.3.0.12
00007f8a5976a000      8K r-x--  /lib/libdl-2.7.so
00007f8a5976c000   2048K -----  /lib/libdl-2.7.so
00007f8a5996c000      8K rw---  /lib/libdl-2.7.so
00007f8a5996e000   1376K r-x--  /lib/libc-2.7.so
00007f8a59ac6000   2048K -----  /lib/libc-2.7.so
00007f8a59cc6000     12K r----  /lib/libc-2.7.so
00007f8a59cc9000      8K rw---  /lib/libc-2.7.so
00007f8a59ccb000     20K rw---    [ anon ]
00007f8a59cd0000    512K r-x--  /lib/libm-2.7.so
00007f8a59d50000   2044K -----  /lib/libm-2.7.so
00007f8a59f4f000      8K rw---  /lib/libm-2.7.so
00007f8a59f51000   1020K r-x--  /usr/lib/libX11.so.6.2.0
00007f8a5a050000   2044K -----  /usr/lib/libX11.so.6.2.0
00007f8a5a24f000     20K rw---  /usr/lib/libX11.so.6.2.0
00007f8a5a254000    412K r-x--  /usr/lib/libSDL-1.2.so.0.11.1
00007f8a5a2bb000   2048K -----  /usr/lib/libSDL-1.2.so.0.11.1
00007f8a5a4bb000      8K rw---  /usr/lib/libSDL-1.2.so.0.11.1
00007f8a5a4bd000    176K rw---    [ anon ]
00007f8a5a4e9000    492K r-x--  /usr/lib/libgnutls.so.13.9.1
00007f8a5a564000   2048K -----  /usr/lib/libgnutls.so.13.9.1
00007f8a5a764000     40K rw---  /usr/lib/libgnutls.so.13.9.1
00007f8a5a76e000    220K r-x--  /lib/libncurses.so.5.6
00007f8a5a7a5000   2044K -----  /lib/libncurses.so.5.6
00007f8a5a9a4000     20K rw---  /lib/libncurses.so.5.6
00007f8a5a9a9000     88K r-x--  /usr/lib/libz.so.1.2.3.3
00007f8a5a9bf000   2048K -----  /usr/lib/libz.so.1.2.3.3
00007f8a5abbf000      4K rw---  /usr/lib/libz.so.1.2.3.3
00007f8a5abc0000      8K r-x--  /lib/libutil-2.7.so
00007f8a5abc2000   2044K -----  /lib/libutil-2.7.so
00007f8a5adc1000      8K rw---  /lib/libutil-2.7.so
00007f8a5adc3000     88K r-x--  /lib/libpthread-2.7.so
00007f8a5add9000   2048K -----  /lib/libpthread-2.7.so
00007f8a5afd9000      8K rw---  /lib/libpthread-2.7.so
00007f8a5afdb000     16K rw---    [ anon ]
00007f8a5afdf000     32K r-x--  /lib/librt-2.7.so
00007f8a5afe7000   2044K -----  /lib/librt-2.7.so
00007f8a5b1e6000      8K rw---  /lib/librt-2.7.so
00007f8a5b1e8000    116K r-x--  /lib/ld-2.7.so
00007f8a5b21e000   1260K rw---    [ anon ]
00007f8a5b36c000    136K rw---    [ anon ]
00007f8a5b3af000    308K rw---    [ anon ]
00007f8a5b3ff000     12K rw-s-    [ anon ]
00007f8a5b402000     12K rw---    [ anon ]
00007f8a5b405000      8K rw---  /lib/ld-2.7.so
00007fff3642e000     84K rw---    [ stack ]
00007fff3655e000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
 total           635292K
---------------------------------------------------------------------

root@ubuntu:~# pmap 10245
10245:   /usr/local/qemu-kvm/bin/qemu-system-x86_64 -hda /dev/vm/fugu-disk -m 512 -boot c -net nic,vlan=0,macaddr=00:16:3e:00:00:35,model=e1000 -net tap -daemonize -vnc :4 -k es -localtime -serial telnet:localhost:4001,server,nowait -monitor telnet:localhost:4041,server,nowait
0000000000400000   2220K r-x--  /usr/local/qemu-kvm/bin/qemu-system-x86_64
000000000082b000    124K rw---  /usr/local/qemu-kvm/bin/qemu-system-x86_64
000000000084a000   7512K rw---    [ anon ]
000000004003a000      4K -----    [ anon ]
000000004003b000   8192K rw---    [ anon ]
0000000040fcf000      4K -----    [ anon ]
0000000040fd0000   8192K rw---    [ anon ]
00007f196e91a000   2640K rw---    [ anon ]
00007f196ebae000    136K rw---    [ anon ]
00007f196ebd0000 540688K rw---    [ anon ]
00007f198fbd4000     40K r-x--  /lib/libnss_files-2.7.so
00007f198fbde000   2048K -----  /lib/libnss_files-2.7.so
00007f198fdde000      8K rw---  /lib/libnss_files-2.7.so
00007f198fde0000     20K r-x--  /usr/lib/libXdmcp.so.6.0.0
00007f198fde5000   2044K -----  /usr/lib/libXdmcp.so.6.0.0
00007f198ffe4000      4K rw---  /usr/lib/libXdmcp.so.6.0.0
00007f198ffe5000      8K r-x--  /usr/lib/libXau.so.6.0.0
00007f198ffe7000   2044K -----  /usr/lib/libXau.so.6.0.0
00007f19901e6000      4K rw---  /usr/lib/libXau.so.6.0.0
00007f19901e7000     12K r-x--  /lib/libgpg-error.so.0.3.0
00007f19901ea000   2044K -----  /lib/libgpg-error.so.0.3.0
00007f19903e9000      4K rw---  /lib/libgpg-error.so.0.3.0
00007f19903ea000    108K r-x--  /usr/lib/libxcb.so.1.0.0
00007f1990405000   2044K -----  /usr/lib/libxcb.so.1.0.0
00007f1990604000      4K rw---  /usr/lib/libxcb.so.1.0.0
00007f1990605000      4K r-x--  /usr/lib/libxcb-xlib.so.0.0.0
00007f1990606000   2044K -----  /usr/lib/libxcb-xlib.so.0.0.0
00007f1990805000      4K rw---  /usr/lib/libxcb-xlib.so.0.0.0
00007f1990806000     80K r-x--  /usr/lib/libdirect-1.0.so.0.1.0
00007f199081a000   2044K -----  /usr/lib/libdirect-1.0.so.0.1.0
00007f1990a19000      8K rw---  /usr/lib/libdirect-1.0.so.0.1.0
00007f1990a1b000     32K r-x--  /usr/lib/libfusion-1.0.so.0.1.0
00007f1990a23000   2044K -----  /usr/lib/libfusion-1.0.so.0.1.0
00007f1990c22000      4K rw---  /usr/lib/libfusion-1.0.so.0.1.0
00007f1990c23000    428K r-x--  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f1990c8e000   2048K -----  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f1990e8e000     12K rw---  /usr/lib/libdirectfb-1.0.so.0.1.0
00007f1990e91000    860K r-x--  /usr/lib/libasound.so.2.0.0
00007f1990f68000   2048K -----  /usr/lib/libasound.so.2.0.0
00007f1991168000     28K rw---  /usr/lib/libasound.so.2.0.0
00007f199116f000    304K r-x--  /lib/libgcrypt.so.11.2.3
00007f19911bb000   2044K -----  /lib/libgcrypt.so.11.2.3
00007f19913ba000     12K rw---  /lib/libgcrypt.so.11.2.3
00007f19913bd000     64K r-x--  /usr/lib/libtasn1.so.3.0.12
00007f19913cd000   2044K -----  /usr/lib/libtasn1.so.3.0.12
00007f19915cc000      4K rw---  /usr/lib/libtasn1.so.3.0.12
00007f19915cd000      8K r-x--  /lib/libdl-2.7.so
00007f19915cf000   2048K -----  /lib/libdl-2.7.so
00007f19917cf000      8K rw---  /lib/libdl-2.7.so
00007f19917d1000   1376K r-x--  /lib/libc-2.7.so
00007f1991929000   2048K -----  /lib/libc-2.7.so
00007f1991b29000     12K r----  /lib/libc-2.7.so
00007f1991b2c000      8K rw---  /lib/libc-2.7.so
00007f1991b2e000     20K rw---    [ anon ]
00007f1991b33000    512K r-x--  /lib/libm-2.7.so
00007f1991bb3000   2044K -----  /lib/libm-2.7.so
00007f1991db2000      8K rw---  /lib/libm-2.7.so
00007f1991db4000   1020K r-x--  /usr/lib/libX11.so.6.2.0
00007f1991eb3000   2044K -----  /usr/lib/libX11.so.6.2.0
00007f19920b2000     20K rw---  /usr/lib/libX11.so.6.2.0
00007f19920b7000    412K r-x--  /usr/lib/libSDL-1.2.so.0.11.1
00007f199211e000   2048K -----  /usr/lib/libSDL-1.2.so.0.11.1
00007f199231e000      8K rw---  /usr/lib/libSDL-1.2.so.0.11.1
00007f1992320000    176K rw---    [ anon ]
00007f199234c000    492K r-x--  /usr/lib/libgnutls.so.13.9.1
00007f19923c7000   2048K -----  /usr/lib/libgnutls.so.13.9.1
00007f19925c7000     40K rw---  /usr/lib/libgnutls.so.13.9.1
00007f19925d1000    220K r-x--  /lib/libncurses.so.5.6
00007f1992608000   2044K -----  /lib/libncurses.so.5.6
00007f1992807000     20K rw---  /lib/libncurses.so.5.6
00007f199280c000     88K r-x--  /usr/lib/libz.so.1.2.3.3
00007f1992822000   2048K -----  /usr/lib/libz.so.1.2.3.3
00007f1992a22000      4K rw---  /usr/lib/libz.so.1.2.3.3
00007f1992a23000      8K r-x--  /lib/libutil-2.7.so
00007f1992a25000   2044K -----  /lib/libutil-2.7.so
00007f1992c24000      8K rw---  /lib/libutil-2.7.so
00007f1992c26000     88K r-x--  /lib/libpthread-2.7.so
00007f1992c3c000   2048K -----  /lib/libpthread-2.7.so
00007f1992e3c000      8K rw---  /lib/libpthread-2.7.so
00007f1992e3e000     16K rw---    [ anon ]
00007f1992e42000     32K r-x--  /lib/librt-2.7.so
00007f1992e4a000   2044K -----  /lib/librt-2.7.so
00007f1993049000      8K rw---  /lib/librt-2.7.so
00007f199304b000    116K r-x--  /lib/ld-2.7.so
00007f1993071000   1536K rw---    [ anon ]
00007f1993212000    136K rw---    [ anon ]
00007f1993255000     40K rw---    [ anon ]
00007f1993262000     12K rw-s-    [ anon ]
00007f1993265000     12K rw---    [ anon ]
00007f1993268000      8K rw---  /lib/ld-2.7.so
00007fff37f8b000     84K rw---    [ stack ]
00007fff37fff000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
 total           625388K
---------------------------------------------------------------------


Thanks for so quick reply.

Regards,
Daniel

[1] http://www.linux-kvm.org/page/Tuning_KVM
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Memory usage with qemu-kvm-0.12.1.1

[Avi Kivity]

On 12/27/2009 06:45 PM, Rik van Riel wrote:
>
>> If so, it doesn't copy stable@kernel.org. Is it queued for -stable?
>
> I do not believe that it is queued for -stable.
>
> Do performance fixes fit with -stable policy?
>

If it is a serious regression, I believe it fits.

-- 
error compiling committee.c: too many arguments to function

Re: Memory usage with qemu-kvm-0.12.1.1

[Avi Kivity]

On 12/27/2009 07:00 PM, Daniel Bareiro wrote:
>
>> Also, qemu might be leaking memory.  Please post 'pmap $pid' for all of
>> your guests (do that before any of the other tests, on your swapped-out
>> system).
>>      
> ---------------------------------------------------------------------
>
>   total           626376K
>
>   total           626472K
>
>   total           626396K
>
>   total           635292K
>
>   total           625388K
>    

These all seem sane.  So it's a swap regression, hopefully 
2.6.32.something will have a fix.

-- 
error compiling committee.c: too many arguments to function

Re: Memory usage with qemu-kvm-0.12.1.1

[Rik van Riel]

On 12/27/2009 12:12 PM, Avi Kivity wrote:
> On 12/27/2009 06:45 PM, Rik van Riel wrote:
>>
>>> If so, it doesn't copy stable@kernel.org. Is it queued for -stable?
>>
>> I do not believe that it is queued for -stable.
>>
>> Do performance fixes fit with -stable policy?
>
> If it is a serious regression, I believe it fits.

It's probably been there since 2.6.28, though it might have been
introduced later with a cleanup patch.  It seems to go back at
least as far as March...

-- 
All rights reversed.

Re: Memory usage with qemu-kvm-0.12.1.1

[Avi Kivity]

On 12/27/2009 07:20 PM, Avi Kivity wrote:
> On 12/27/2009 07:00 PM, Daniel Bareiro wrote:
>>
>>> Also, qemu might be leaking memory.  Please post 'pmap $pid' for all of
>>> your guests (do that before any of the other tests, on your swapped-out
>>> system).
>> ---------------------------------------------------------------------
>>
>>   total           626376K
>>
>>   total           626472K
>>
>>   total           626396K
>>
>>   total           635292K
>>
>>   total           625388K
>
> These all seem sane.  So it's a swap regression, hopefully 
> 2.6.32.something will have a fix.
>

btw, does your system have ept?

$ cat /sys/module/kvm_intel/parameters/ept

-- 
error compiling committee.c: too many arguments to function

Re: Memory usage with qemu-kvm-0.12.1.1

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 1501 bytes --]

Hi, Avi.

On Sunday, 27 December 2009 19:33:31 +0200,
Avi Kivity wrote:

>>>> Also, qemu might be leaking memory.  Please post 'pmap $pid' for
>>>> all of your guests (do that before any of the other tests, on your
>>>> swapped-out system).
>>> ---------------------------------------------------------------------
>>>
>>>   total           626376K
>>>
>>>   total           626472K
>>>
>>>   total           626396K
>>>
>>>   total           635292K
>>>
>>>   total           625388K

>> These all seem sane.  So it's a swap regression, hopefully
>> 2.6.32.something will have a fix.

It draws attention to me that I didn't have this problem with
qemu-kvm-0.12.0-rc2 but after updating to qemu-kvm-0.12.1.1.

Mmmm... thinking a little, I made the update to see if with 0.12.1.1 no
longer had the problem with the e1000 network interface in OpenBSD.
Therefore, I only shutdown an boot this VM and not all. Could it have
caused that increase in the swap usage?

In both cases the used version of kernel was the same.

> btw, does your system have ept?
>
> $ cat /sys/module/kvm_intel/parameters/ept

It doesn't have:

root@ubuntu:~# ls /sys/module/kvm_amd/parameters/
nested  npt

root@ubuntu:~# uname -a
Linux ubuntu 2.6.32-dgb #1 SMP Mon Dec 14 06:18:06 ART 2009 x86_64
GNU/Linux


Thanks for your reply.

Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Memory usage with qemu-kvm-0.12.1.1

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

Hi, Avi.

On Sunday, 27 December 2009 18:03:18 +0200,
Avi Kivity wrote:

>> I installed qemu-kvm-0.12.1.1 in one equipment of my house yesterday
>> to test it with Linux 2.6.32 compiled by myself from the source code
>> of kernel.org.
>>
>> From the night of yesterday that I am observing a high use of swap.
>> This is the Service Log Entries from Nagios:

>> [...]

> Try setting /proc/sys/vm/swappiness to 0, and running your guests with
> cache=none (a good idea in any case).

Setting /proc/sys/vm/swappiness to 0 didn't take changes in the swap
usage until the moment.

What you consulted to me about /sys/module/kvm_intel/parameters/ept is
because it can serve like workaround until the publication of 2.6.32.x
that fix this problem?

Although, as I said in another mail, I don't have enabled this
parameter:

root@ubuntu:~# ls /sys/module/kvm_amd/parameters/
nested  npt

root@ubuntu:~# uname -a
Linux ubuntu 2.6.32-dgb #1 SMP Mon Dec 14 06:18:06 ART 2009 x86_64 GNU/Linux


Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Memory usage with qemu-kvm-0.12.1.1

[Hugh Dickins]

On Sun, 27 Dec 2009, Rik van Riel wrote:
> On 12/27/2009 12:12 PM, Avi Kivity wrote:
> > On 12/27/2009 06:45 PM, Rik van Riel wrote:
> > > 
> > > > If so, it doesn't copy stable@kernel.org. Is it queued for -stable?
> > > 
> > > I do not believe that it is queued for -stable.
> > > 
> > > Do performance fixes fit with -stable policy?
> > 
> > If it is a serious regression, I believe it fits.
> 
> It's probably been there since 2.6.28, though it might have been
> introduced later with a cleanup patch.  It seems to go back at
> least as far as March...

It does look as if it would have got worse in 2.6.31 and 2.6.32
- though I am reluctant to predict how these scans work out in
practice from just glancing at the code!

I agree with Avi (or with Avi's implication), that it would be
worth sending the fix to -stable - but it would be nice to hear
if the patch (inline below) actually does fix Daniel's problem.
It applies and builds correctly on 2.6.31.9 and 2.6.32.3.

We have certainly sent much more specialized performance fixes
to -stable (I'm thinking of Lee's anon_vma locking), and I see
this as more of a straightforward bugfix anyway - people get
worried by going into swap without understanding why.

I forget which patch of mine Avi was thinking of,
but I'm pretty sure Rik's will prove much the more relevant.

Hugh

commit b39415b2731d7dec5e612d2d12595da82399eedf
Author: Rik van Riel <riel@redhat.com>
Date:   Mon Dec 14 17:59:48 2009 -0800

    vmscan: do not evict inactive pages when skipping an active list scan
    
    In AIM7 runs, recent kernels start swapping out anonymous pages well
    before they should.  This is due to shrink_list falling through to
    shrink_inactive_list if !inactive_anon_is_low(zone, sc), when all we
    really wanted to do is pre-age some anonymous pages to give them extra
    time to be referenced while on the inactive list.
    
    The obvious fix is to make sure that shrink_list does not fall through to
    scanning/reclaiming inactive pages when we called it to scan one of the
    active lists.
    
    This change should be safe because the loop in shrink_zone ensures that we
    will still shrink the anon and file inactive lists whenever we should.
    
    [kosaki.motohiro@jp.fujitsu.com: inactive_file_is_low() should be inactive_anon_is_low()]
    Reported-by: Larry Woodman <lwoodman@redhat.com>
    Signed-off-by: Rik van Riel <riel@redhat.com>
    Acked-by: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Tomasz Chmielewski <mangoo@wpkg.org>
    Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 2ef59d5..0465818 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1463,20 +1463,26 @@ static int inactive_file_is_low(struct zone *zone, struct scan_control *sc)
 	return low;
 }
 
+static int inactive_list_is_low(struct zone *zone, struct scan_control *sc,
+				int file)
+{
+	if (file)
+		return inactive_file_is_low(zone, sc);
+	else
+		return inactive_anon_is_low(zone, sc);
+}
+
 static unsigned long shrink_list(enum lru_list lru, unsigned long nr_to_scan,
 	struct zone *zone, struct scan_control *sc, int priority)
 {
 	int file = is_file_lru(lru);
 
-	if (lru == LRU_ACTIVE_FILE && inactive_file_is_low(zone, sc)) {
-		shrink_active_list(nr_to_scan, zone, sc, priority, file);
+	if (is_active_lru(lru)) {
+		if (inactive_list_is_low(zone, sc, file))
+		    shrink_active_list(nr_to_scan, zone, sc, priority, file);
 		return 0;
 	}
 
-	if (lru == LRU_ACTIVE_ANON && inactive_anon_is_low(zone, sc)) {
-		shrink_active_list(nr_to_scan, zone, sc, priority, file);
-		return 0;
-	}
 	return shrink_inactive_list(nr_to_scan, zone, sc, priority, file);
 }
 

Re: Memory usage with qemu-kvm-0.12.1.1

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 2596 bytes --]

Hi, Hugh.

On Wednesday, 30 December 2009 18:14:30 +0000,
Hugh Dickins wrote:

> On Sun, 27 Dec 2009, Rik van Riel wrote:
> > On 12/27/2009 12:12 PM, Avi Kivity wrote:
> > > On 12/27/2009 06:45 PM, Rik van Riel wrote:
> > > > 
> > > > > If so, it doesn't copy stable@kernel.org. Is it queued for -stable?
> > > > 
> > > > I do not believe that it is queued for -stable.
> > > > 
> > > > Do performance fixes fit with -stable policy?
> > > 
> > > If it is a serious regression, I believe it fits.
> > 
> > It's probably been there since 2.6.28, though it might have been
> > introduced later with a cleanup patch.  It seems to go back at
> > least as far as March...
 
> It does look as if it would have got worse in 2.6.31 and 2.6.32
> - though I am reluctant to predict how these scans work out in
> practice from just glancing at the code!
> 
> I agree with Avi (or with Avi's implication), that it would be
> worth sending the fix to -stable - but it would be nice to hear
> if the patch (inline below) actually does fix Daniel's problem.
> It applies and builds correctly on 2.6.31.9 and 2.6.32.3.
> 
> We have certainly sent much more specialized performance fixes
> to -stable (I'm thinking of Lee's anon_vma locking), and I see
> this as more of a straightforward bugfix anyway - people get
> worried by going into swap without understanding why.
> 
> I forget which patch of mine Avi was thinking of,
> but I'm pretty sure Rik's will prove much the more relevant.

I shutdown the virtual machines yesterday and I boot them again and the
swap usage did not exceed the 9 MB / 486 MB with /proc/sys/vm/swappiness
set to 0. Now I tried restarting the host with the same running VM and,
at the moment, the use of swap is zero with /proc/sys/vm/swappiness by
default (60).

root@ubuntu:~# free
             total       used       free     shared    buffers     cached
Mem:       4060340    2299556    1760784          0     837668     133292
-/+ buffers/cache:    1328596    2731744
Swap:       497972          0     497972


What tests would be recommendable to make to reproduce the problem?

You say that the patch applies and builds correctly on 2.6.31.9 and
2.6.32.3. I would like to test it with 2.6.32.3, but I didn't find the
source code of it in [1]. It would have to be in another path?


Thanks for your reply and the patch.

Regards,
Daniel

[1] http://www.kernel.org/pub/linux/kernel/v2.6/
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Memory usage with qemu-kvm-0.12.1.1

[Hugh Dickins]

On Thu, 31 Dec 2009, Daniel Bareiro wrote:
> 
> What tests would be recommendable to make to reproduce the problem?

Oh, I thought you were the one seeing the problem!  If you cannot
easily reproduce it, then please don't spend too long over it.

I've never noticed it myself (perhaps because my workaday loads are
too light, and my testing loads too heavy): I was just judging by
the content of Rik's patch.  Rik cites AIM7 in his description,
but perhaps someone knows an easier way to demonstrate it; though
I was more interested in whether it also fixes the problem you saw.

> 
> You say that the patch applies and builds correctly on 2.6.31.9 and
> 2.6.32.3. I would like to test it with 2.6.32.3, but I didn't find the
> source code of it in [1]. It would have to be in another path?

Umm, may I pretend that, as we approach the New Year, I was gazing
into my crystal ball?  Or should I just admit to a stupid typo?
Such a dilemma, but I'll go for honesty: 2.6.32.2, sorry about that!

> 
> Thanks for your reply and the patch.

Thanks go to Rik for the patch.  And thanks for your report and
testing, but please don't put yourself out over it: the patch
already looks good, and I think it should go to -stable with
or without the commendation of whether it works for you.

Hugh

Re: Memory usage with qemu-kvm-0.12.1.1

[Rik van Riel]

On 12/31/2009 12:02 PM, Hugh Dickins wrote:
> On Thu, 31 Dec 2009, Daniel Bareiro wrote:
>>
>> What tests would be recommendable to make to reproduce the problem?
>
> Oh, I thought you were the one seeing the problem!  If you cannot
> easily reproduce it, then please don't spend too long over it.
>
> I've never noticed it myself (perhaps because my workaday loads are
> too light, and my testing loads too heavy): I was just judging by
> the content of Rik's patch.  Rik cites AIM7 in his description,
> but perhaps someone knows an easier way to demonstrate it; though
> I was more interested in whether it also fixes the problem you saw.

I believe Avi has seen the problem and is motivated to
get the patch into -stable :)

> Thanks go to Rik for the patch.  And thanks for your report and
> testing, but please don't put yourself out over it: the patch
> already looks good, and I think it should go to -stable with
> or without the commendation of whether it works for you.

I agree that the patch should be safe for -stable.

-- 
All rights reversed.

Re: Memory usage with qemu-kvm-0.12.1.1

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 2648 bytes --]

Hi, Hugh.

On Thursday, 31 December 2009 17:02:49 +0000,
Hugh Dickins wrote:

> On Thu, 31 Dec 2009, Daniel Bareiro wrote:
> > 
> > What tests would be recommendable to make to reproduce the problem?
> 
> Oh, I thought you were the one seeing the problem!  If you cannot
> easily reproduce it, then please don't spend too long over it.
> 
> I've never noticed it myself (perhaps because my workaday loads are
> too light, and my testing loads too heavy): I was just judging by
> the content of Rik's patch.  Rik cites AIM7 in his description,
> but perhaps someone knows an easier way to demonstrate it; though
> I was more interested in whether it also fixes the problem you saw.

Well... after of compiling three times the kernel 2.6.32.2 to test,
having the same running VMs, the swap usage reached 40% from 0%. Can it
be taken like as an evidence that the problem was reproduced?

root@ubuntu:~# free
             total       used       free     shared    buffers     cached
Mem:       4060340    3927632     132708          0      83704    2265672
-/+ buffers/cache:    1578256    2482084
Swap:       497972     202944     295028

> > You say that the patch applies and builds correctly on 2.6.31.9 and
> > 2.6.32.3. I would like to test it with 2.6.32.3, but I didn't find the
> > source code of it in [1]. It would have to be in another path?
> 
> Umm, may I pretend that, as we approach the New Year, I was gazing
> into my crystal ball?  Or should I just admit to a stupid typo?
> Such a dilemma, but I'll go for honesty: 2.6.32.2, sorry about that!

I thought maybe 2.6.32.3 was some development version and that is why I
chose to ask. Although I thought that it would be a typo. Don't worry
:-)

> > Thanks for your reply and the patch.
> 
> Thanks go to Rik for the patch.  And thanks for your report and
> testing, but please don't put yourself out over it: the patch
> already looks good, and I think it should go to -stable with
> or without the commendation of whether it works for you.

Also I was doing the same amount of compilations with the 2.6.32.2 with
Rik's patch and the amount of used swap is substantially lower:

root@ubuntu:~# free
             total       used       free     shared    buffers     cached
Mem:       4060340    3956872     103468          0      80912    2689772
-/+ buffers/cache:    1186188    2874152
Swap:       497972       8152     489820



From Buenos Aires I wish you a happy new year!!

Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Memory usage with qemu-kvm-0.12.1.1

[Avi Kivity]

On 12/31/2009 08:36 PM, Rik van Riel wrote:
>
> I believe Avi has seen the problem and is motivated to
> get the patch into -stable :)

I did not actually see the problem myself.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: Memory usage with qemu-kvm-0.12.1.1

[Thomas Fjellstrom]

On Sun December 27 2009, Avi Kivity wrote:
> On 12/27/2009 07:00 PM, Daniel Bareiro wrote:
> >> Also, qemu might be leaking memory.  Please post 'pmap $pid' for all
> >> of your guests (do that before any of the other tests, on your
> >> swapped-out system).
> >
> > ---------------------------------------------------------------------
> >
> >   total           626376K
> >
> >   total           626472K
> >
> >   total           626396K
> >
> >   total           635292K
> >
> >   total           625388K
> 
> These all seem sane.  So it's a swap regression, hopefully
> 2.6.32.something will have a fix.
> 

Sorry to butt in, but heres something I've found odd:

# ps aux | grep /usr/bin/kvm | grep -v grep | cut -f6 -d' ' | xargs -n 1 -i{} pmap {} | grep total
 total           845928K
 total           450336K
 total           441968K
 total           440740K
 total           845848K
 total           465808K

root     10466  2.6  6.2 845924 253804 ?       Sl    2009 2084:29 /usr/bin/kvm -S -M pc -m 512 -smp 1 -name awiki -uuid 330abdce-f657-e0e2-196b-5bf22c0e76f0 -monitor unix:/var/lib/libvirt/qemu/awiki.monitor,server,nowait -boot c -drive file=/dev/vg0/awiki-root,if=virtio,index=0,boot=on -drive file=/dev/vg0/awiki-swap,if=virtio,index=1 -drive file=/mnt/boris/data/pub/diskimage/debian-503-amd64-netinst.iso,if=ide,media=cdrom,index=2,format= -net nic,macaddr=52:54:00:35:8b:fb,vlan=0,model=virtio,name=virtio.0 -net tap,fd=19,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc 127.0.0.1:2 -k en-us -vga vmware
root     13953  0.2  1.3 450332 54832 ?        Sl    2009 167:25 /usr/bin/kvm -S -M pc -m 128 -smp 1 -name nginx -uuid 793160c1-5800-72cf-7b66-8484f931d396 -monitor unix:/var/lib/libvirt/qemu/nginx.monitor,server,nowait -boot c -drive file=/dev/vg0/nginx,if=virtio,index=0,boot=on -net nic,macaddr=52:54:00:06:49:d5,vlan=0,model=virtio,name=virtio.0 -net tap,fd=21,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc 127.0.0.1:3 -k en-us -vga vmware
root     14051 31.4  6.7 441964 273132 ?       Rl   01:19  30:35 /usr/bin/kvm -S -M pc -m 256 -smp 1 -name pfsense -uuid 0af4dfac-70f1-c348-9ce5-0df18e9bdc2c -monitor unix:/var/lib/libvirt/qemu/pfsense.monitor,server,nowait -boot c -drive file=/dev/vg0/pfsense,if=ide,index=0,boot=on -net nic,macaddr=00:19:5b:86:3e:fb,vlan=0,model=e1000,name=e1000.0 -net tap,fd=22,vlan=0,name=tap.0 -net nic,macaddr=52:54:00:53:62:b9,vlan=1,model=e1000,name=e1000.1 -net tap,fd=28,vlan=1,name=tap.1 -serial pty -parallel none -usb -vnc 0.0.0.0:0 -k en-us -vga vmware
root     15528 19.7  6.6 440736 270484 ?       Sl   01:37  15:38 /usr/bin/kvm -S -M pc -m 256 -smp 1 -name pfsense2 -uuid 2c4000a0-7565-b12d-1e2a-1e77cdb778d3 -monitor unix:/var/lib/libvirt/qemu/pfsense2.monitor,server,nowait -boot c -drive file=/dev/vg0/pfsense2,if=ide,index=0,boot=on -drive file=/mnt/boris/data/pub/diskimage/pfSense-1.2.2-LiveCD-Installer.iso,if=ide,media=cdrom,index=2,format= -net nic,macaddr=52:54:00:38:fc:a7,vlan=0,model=e1000,name=e1000.0 -net tap,fd=28,vlan=0,name=tap.0 -net nic,macaddr=00:24:1d:18:f8:f6,vlan=1,model=e1000,name=e1000.1 -net tap,fd=29,vlan=1,name=tap.1 -serial pty -parallel none -usb -vnc 127.0.0.1:1 -k en-us -vga vmware
root     27079  0.9  0.7 845700 30768 ?        SLl   2009 584:28 /usr/bin/kvm -S -M pc -m 512 -smp 1 -name asterisk -uuid a87d8fc1-ea90-0db4-d6fe-c04e8f2175e7 -monitor unix:/var/lib/libvirt/qemu/asterisk.monitor,server,nowait -boot c -drive file=/dev/vg0/asterisk,if=virtio,index=0,boot=on -net nic,macaddr=52:54:00:68:db:fc,vlan=0,model=virtio,name=virtio.0 -net tap,fd=23,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc 127.0.0.1:5 -k en-us -vga vmware -soundhw es1370
root     31214  0.6  2.9 465804 121476 ?       Sl    2009 207:08 /usr/bin/kvm -S -M pc -m 256 -smp 1 -name svn -uuid 6e30e0be-1781-7a68-fa5d-d3c69787e705 -monitor unix:/var/lib/libvirt/qemu/svn.monitor,server,nowait -boot c -drive file=/dev/vg0/svn-root,if=virtio,index=0,boot=on -net nic,macaddr=52:54:00:7d:f4:0b,vlan=0,model=virtio,name=virtio.0 -net tap,fd=27,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc 0.0.0.0:4 -k en-us -vga vmware

several of these vms are actually assigned less memory than is stated in -m,
since I used the virt-manager interface to shrink memory size. awiki is set to
256MB, yet is still somehow using over 800MB of virt? one of the "anon" maps
in pmap shows up as nearly 512MB (544788K). The rest of the vms show oddities
like that as well.

host is debian sid with the 2.6.31-2-amd64 kernel, kvm --version reports:

QEMU PC emulator version 0.10.50 (qemu-kvm-devel-88)

and just for kicks:

root@boris:~# free -m
             total       used       free     shared    buffers     cached
Mem:          3964       3891         72          0        108       1686
-/+ buffers/cache:       2096       1867
Swap:         7627        383       7244

-- 
Thomas Fjellstrom
tfjellstrom@shaw.ca

Re: Memory usage with qemu-kvm-0.12.1.1

[Thomas Fjellstrom]

On Sun January 3 2010, Thomas Fjellstrom wrote:
> On Sun December 27 2009, Avi Kivity wrote:
> > On 12/27/2009 07:00 PM, Daniel Bareiro wrote:
> > >> Also, qemu might be leaking memory.  Please post 'pmap $pid' for all
> > >> of your guests (do that before any of the other tests, on your
> > >> swapped-out system).
> > >
> > > ---------------------------------------------------------------------
> > >
> > >   total           626376K
> > >
> > >   total           626472K
> > >
> > >   total           626396K
> > >
> > >   total           635292K
> > >
> > >   total           625388K
> >
> > These all seem sane.  So it's a swap regression, hopefully
> > 2.6.32.something will have a fix.
> 
> Sorry to butt in, but heres something I've found odd:
> 
> # ps aux | grep /usr/bin/kvm | grep -v grep | cut -f6 -d' ' | xargs -n 1
>  -i{} pmap {} | grep total total           845928K
>  total           450336K
>  total           441968K
>  total           440740K
>  total           845848K
>  total           465808K
> 
> root     10466  2.6  6.2 845924 253804 ?       Sl    2009 2084:29
>  /usr/bin/kvm -S -M pc -m 512 -smp 1 -name awiki -uuid
>  330abdce-f657-e0e2-196b-5bf22c0e76f0 -monitor
>  unix:/var/lib/libvirt/qemu/awiki.monitor,server,nowait -boot c -drive
>  file=/dev/vg0/awiki-root,if=virtio,index=0,boot=on -drive
>  file=/dev/vg0/awiki-swap,if=virtio,index=1 -drive
>  file=/mnt/boris/data/pub/diskimage/debian-503-amd64-netinst.iso,if=ide,m
> edia=cdrom,index=2,format= -net
>  nic,macaddr=52:54:00:35:8b:fb,vlan=0,model=virtio,name=virtio.0 -net
>  tap,fd=19,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>  127.0.0.1:2 -k en-us -vga vmware root     13953  0.2  1.3 450332 54832 ?
>         Sl    2009 167:25 /usr/bin/kvm -S -M pc -m 128 -smp 1 -name nginx
>  -uuid 793160c1-5800-72cf-7b66-8484f931d396 -monitor
>  unix:/var/lib/libvirt/qemu/nginx.monitor,server,nowait -boot c -drive
>  file=/dev/vg0/nginx,if=virtio,index=0,boot=on -net
>  nic,macaddr=52:54:00:06:49:d5,vlan=0,model=virtio,name=virtio.0 -net
>  tap,fd=21,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>  127.0.0.1:3 -k en-us -vga vmware root     14051 31.4  6.7 441964 273132
>  ?       Rl   01:19  30:35 /usr/bin/kvm -S -M pc -m 256 -smp 1 -name
>  pfsense -uuid 0af4dfac-70f1-c348-9ce5-0df18e9bdc2c -monitor
>  unix:/var/lib/libvirt/qemu/pfsense.monitor,server,nowait -boot c -drive
>  file=/dev/vg0/pfsense,if=ide,index=0,boot=on -net
>  nic,macaddr=00:19:5b:86:3e:fb,vlan=0,model=e1000,name=e1000.0 -net
>  tap,fd=22,vlan=0,name=tap.0 -net
>  nic,macaddr=52:54:00:53:62:b9,vlan=1,model=e1000,name=e1000.1 -net
>  tap,fd=28,vlan=1,name=tap.1 -serial pty -parallel none -usb -vnc
>  0.0.0.0:0 -k en-us -vga vmware root     15528 19.7  6.6 440736 270484 ? 
>       Sl   01:37  15:38 /usr/bin/kvm -S -M pc -m 256 -smp 1 -name
>  pfsense2 -uuid 2c4000a0-7565-b12d-1e2a-1e77cdb778d3 -monitor
>  unix:/var/lib/libvirt/qemu/pfsense2.monitor,server,nowait -boot c -drive
>  file=/dev/vg0/pfsense2,if=ide,index=0,boot=on -drive
>  file=/mnt/boris/data/pub/diskimage/pfSense-1.2.2-LiveCD-Installer.iso,if
> =ide,media=cdrom,index=2,format= -net
>  nic,macaddr=52:54:00:38:fc:a7,vlan=0,model=e1000,name=e1000.0 -net
>  tap,fd=28,vlan=0,name=tap.0 -net
>  nic,macaddr=00:24:1d:18:f8:f6,vlan=1,model=e1000,name=e1000.1 -net
>  tap,fd=29,vlan=1,name=tap.1 -serial pty -parallel none -usb -vnc
>  127.0.0.1:1 -k en-us -vga vmware root     27079  0.9  0.7 845700 30768 ?
>         SLl   2009 584:28 /usr/bin/kvm -S -M pc -m 512 -smp 1 -name
>  asterisk -uuid a87d8fc1-ea90-0db4-d6fe-c04e8f2175e7 -monitor
>  unix:/var/lib/libvirt/qemu/asterisk.monitor,server,nowait -boot c -drive
>  file=/dev/vg0/asterisk,if=virtio,index=0,boot=on -net
>  nic,macaddr=52:54:00:68:db:fc,vlan=0,model=virtio,name=virtio.0 -net
>  tap,fd=23,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>  127.0.0.1:5 -k en-us -vga vmware -soundhw es1370 root     31214  0.6 
>  2.9 465804 121476 ?       Sl    2009 207:08 /usr/bin/kvm -S -M pc -m 256
>  -smp 1 -name svn -uuid 6e30e0be-1781-7a68-fa5d-d3c69787e705 -monitor
>  unix:/var/lib/libvirt/qemu/svn.monitor,server,nowait -boot c -drive
>  file=/dev/vg0/svn-root,if=virtio,index=0,boot=on -net
>  nic,macaddr=52:54:00:7d:f4:0b,vlan=0,model=virtio,name=virtio.0 -net
>  tap,fd=27,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>  0.0.0.0:4 -k en-us -vga vmware
> 
> several of these vms are actually assigned less memory than is stated in
>  -m, since I used the virt-manager interface to shrink memory size. awiki
>  is set to 256MB, yet is still somehow using over 800MB of virt? one of
>  the "anon" maps in pmap shows up as nearly 512MB (544788K). The rest of
>  the vms show oddities like that as well.
> 
> host is debian sid with the 2.6.31-2-amd64 kernel, kvm --version reports:
> 
> QEMU PC emulator version 0.10.50 (qemu-kvm-devel-88)
> 
> and just for kicks:
> 
> root@boris:~# free -m
>              total       used       free     shared    buffers     cached
> Mem:          3964       3891         72          0        108       1686
> -/+ buffers/cache:       2096       1867
> Swap:         7627        383       7244
> 

Would this be normal for my setup? The virt usage seems abnormally high for 
all of my guests, especially the ones using over 800MB virt.

-- 
Thomas Fjellstrom
tfjellstrom@shaw.ca

Re: Memory usage with qemu-kvm-0.12.1.1

[David S. Ahern]

On 01/04/2010 08:12 AM, Thomas Fjellstrom wrote:
> Would this be normal for my setup? The virt usage seems abnormally high for 
> all of my guests, especially the ones using over 800MB virt.
> 

As I understand it virtual memory usage shows the allocated address
ranges (library mappings, dynamic allocations, etc). The guest memory
will be one of the anonymous mappings - with size equal to the memory
allocated to the VM. Until the guest accesses all of its memory (or if
qemu initialized it after the malloc), even that memory is only a
notional allocation. This is standard memory usage for linux -- mallocs
only create address mappings/allocations; it is not backed with physical
RAM until accessed.

As an example I have a linux guest with 512MB of RAM. The VmSize at
startup is 892700kB, though RSS is only 59792kB. If I login to the guest
and make use of memory within it then the guest memory becomes backed
from the host side. e.g., I have a memuser program that does nothing
more than malloc memory and initialize it. For the 512M guest, I run
this program with an input arg of 512M and voila the RSS for the qemu
process jumps to 527444kB while the VmSize has not changed.

David

Re: Memory usage with qemu-kvm-0.12.1.1

[Avi Kivity]

On 01/04/2010 05:12 PM, Thomas Fjellstrom wrote:
> On Sun January 3 2010, Thomas Fjellstrom wrote:
>    
>> On Sun December 27 2009, Avi Kivity wrote:
>>      
>>> On 12/27/2009 07:00 PM, Daniel Bareiro wrote:
>>>        
>>>>> Also, qemu might be leaking memory.  Please post 'pmap $pid' for all
>>>>> of your guests (do that before any of the other tests, on your
>>>>> swapped-out system).
>>>>>            
>>>> ---------------------------------------------------------------------
>>>>
>>>>    total           626376K
>>>>
>>>>    total           626472K
>>>>
>>>>    total           626396K
>>>>
>>>>    total           635292K
>>>>
>>>>    total           625388K
>>>>          
>>> These all seem sane.  So it's a swap regression, hopefully
>>> 2.6.32.something will have a fix.
>>>        
>> Sorry to butt in, but heres something I've found odd:
>>
>> # ps aux | grep /usr/bin/kvm | grep -v grep | cut -f6 -d' ' | xargs -n 1
>>   -i{} pmap {} | grep total total           845928K
>>   total           450336K
>>   total           441968K
>>   total           440740K
>>   total           845848K
>>   total           465808K
>>
>> root     10466  2.6  6.2 845924 253804 ?       Sl    2009 2084:29
>>   /usr/bin/kvm -S -M pc -m 512 -smp 1 -name awiki -uuid
>>   330abdce-f657-e0e2-196b-5bf22c0e76f0 -monitor
>>   unix:/var/lib/libvirt/qemu/awiki.monitor,server,nowait -boot c -drive
>>   file=/dev/vg0/awiki-root,if=virtio,index=0,boot=on -drive
>>   file=/dev/vg0/awiki-swap,if=virtio,index=1 -drive
>>   file=/mnt/boris/data/pub/diskimage/debian-503-amd64-netinst.iso,if=ide,m
>> edia=cdrom,index=2,format= -net
>>   nic,macaddr=52:54:00:35:8b:fb,vlan=0,model=virtio,name=virtio.0 -net
>>   tap,fd=19,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>>   127.0.0.1:2 -k en-us -vga vmware root     13953  0.2  1.3 450332 54832 ?
>>          Sl    2009 167:25 /usr/bin/kvm -S -M pc -m 128 -smp 1 -name nginx
>>   -uuid 793160c1-5800-72cf-7b66-8484f931d396 -monitor
>>   unix:/var/lib/libvirt/qemu/nginx.monitor,server,nowait -boot c -drive
>>   file=/dev/vg0/nginx,if=virtio,index=0,boot=on -net
>>   nic,macaddr=52:54:00:06:49:d5,vlan=0,model=virtio,name=virtio.0 -net
>>   tap,fd=21,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>>   127.0.0.1:3 -k en-us -vga vmware root     14051 31.4  6.7 441964 273132
>>   ?       Rl   01:19  30:35 /usr/bin/kvm -S -M pc -m 256 -smp 1 -name
>>   pfsense -uuid 0af4dfac-70f1-c348-9ce5-0df18e9bdc2c -monitor
>>   unix:/var/lib/libvirt/qemu/pfsense.monitor,server,nowait -boot c -drive
>>   file=/dev/vg0/pfsense,if=ide,index=0,boot=on -net
>>   nic,macaddr=00:19:5b:86:3e:fb,vlan=0,model=e1000,name=e1000.0 -net
>>   tap,fd=22,vlan=0,name=tap.0 -net
>>   nic,macaddr=52:54:00:53:62:b9,vlan=1,model=e1000,name=e1000.1 -net
>>   tap,fd=28,vlan=1,name=tap.1 -serial pty -parallel none -usb -vnc
>>   0.0.0.0:0 -k en-us -vga vmware root     15528 19.7  6.6 440736 270484 ?
>>        Sl   01:37  15:38 /usr/bin/kvm -S -M pc -m 256 -smp 1 -name
>>   pfsense2 -uuid 2c4000a0-7565-b12d-1e2a-1e77cdb778d3 -monitor
>>   unix:/var/lib/libvirt/qemu/pfsense2.monitor,server,nowait -boot c -drive
>>   file=/dev/vg0/pfsense2,if=ide,index=0,boot=on -drive
>>   file=/mnt/boris/data/pub/diskimage/pfSense-1.2.2-LiveCD-Installer.iso,if
>> =ide,media=cdrom,index=2,format= -net
>>   nic,macaddr=52:54:00:38:fc:a7,vlan=0,model=e1000,name=e1000.0 -net
>>   tap,fd=28,vlan=0,name=tap.0 -net
>>   nic,macaddr=00:24:1d:18:f8:f6,vlan=1,model=e1000,name=e1000.1 -net
>>   tap,fd=29,vlan=1,name=tap.1 -serial pty -parallel none -usb -vnc
>>   127.0.0.1:1 -k en-us -vga vmware root     27079  0.9  0.7 845700 30768 ?
>>          SLl   2009 584:28 /usr/bin/kvm -S -M pc -m 512 -smp 1 -name
>>   asterisk -uuid a87d8fc1-ea90-0db4-d6fe-c04e8f2175e7 -monitor
>>   unix:/var/lib/libvirt/qemu/asterisk.monitor,server,nowait -boot c -drive
>>   file=/dev/vg0/asterisk,if=virtio,index=0,boot=on -net
>>   nic,macaddr=52:54:00:68:db:fc,vlan=0,model=virtio,name=virtio.0 -net
>>   tap,fd=23,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>>   127.0.0.1:5 -k en-us -vga vmware -soundhw es1370 root     31214  0.6
>>   2.9 465804 121476 ?       Sl    2009 207:08 /usr/bin/kvm -S -M pc -m 256
>>   -smp 1 -name svn -uuid 6e30e0be-1781-7a68-fa5d-d3c69787e705 -monitor
>>   unix:/var/lib/libvirt/qemu/svn.monitor,server,nowait -boot c -drive
>>   file=/dev/vg0/svn-root,if=virtio,index=0,boot=on -net
>>   nic,macaddr=52:54:00:7d:f4:0b,vlan=0,model=virtio,name=virtio.0 -net
>>   tap,fd=27,vlan=0,name=tap.0 -serial pty -parallel none -usb -vnc
>>   0.0.0.0:4 -k en-us -vga vmware
>>
>> several of these vms are actually assigned less memory than is stated in
>>   -m, since I used the virt-manager interface to shrink memory size. awiki
>>   is set to 256MB, yet is still somehow using over 800MB of virt? one of
>>   the "anon" maps in pmap shows up as nearly 512MB (544788K). The rest of
>>   the vms show oddities like that as well.
>>
>> host is debian sid with the 2.6.31-2-amd64 kernel, kvm --version reports:
>>
>> QEMU PC emulator version 0.10.50 (qemu-kvm-devel-88)
>>
>> and just for kicks:
>>
>> root@boris:~# free -m
>>               total       used       free     shared    buffers     cached
>> Mem:          3964       3891         72          0        108       1686
>> -/+ buffers/cache:       2096       1867
>> Swap:         7627        383       7244
>>
>>      
> Would this be normal for my setup? The virt usage seems abnormally high for
> all of my guests, especially the ones using over 800MB virt.
>    

That's expected due to how glibc manages the heap.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.