KSM with Debian GNU/Linux

KSM with Debian GNU/Linux

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 524 bytes --]

Hi all!

I have understood that in addition to support from the kernel and
qemu-kvm, there must be support in libc. So in testing I was doing on
Lenny, it did not work using libc from Debian repositories. Now that
squeeze is frozen, anyone knows if it will support KSM?

Has anyone had success making KSM works on Lenny? Michael?


Thanks in advance for your reply.

Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: KSM with Debian GNU/Linux

[Michael Tokarev]

09.08.2010 14:16, Daniel Bareiro wrote:
> Hi all!
>
> I have understood that in addition to support from the kernel and
> qemu-kvm, there must be support in libc. So in testing I was doing on
> Lenny, it did not work using libc from Debian repositories. Now that
> squeeze is frozen, anyone knows if it will support KSM?

KSM is working in qemu-kvm packages in bpo, for quite long
time already.  It has nothing to do with libc, and now it
does not even require recent-enough kernel headers to compile.
What's needed is recent-enough kernel that actually implements
the feature.  The kernel in Lenny does not KSM support, but
the one in bpo has.  Also note that Lenny's kernel is too old
for kvm.

/mjt

Re: KSM with Debian GNU/Linux

[Nikola Ciprich]

Hi,
I don't have clue what glibc is contained in debian, but it seems
to me that at least compile-time KSM support DOES depend on glibc.
I'm using RHEL5-based system and KSM support doesn't get compiled in
by default.
But since I use new kernel, I add following to CFLAGS while compiling
qemu-kvm and then it works:
-DMADV_MERGEABLE=12 -DMADV_UNMERGEABLE=13
just my 5cents
regards
nik


On Mon, Aug 09, 2010 at 02:27:12PM +0400, Michael Tokarev wrote:
> 09.08.2010 14:16, Daniel Bareiro wrote:
> >Hi all!
> >
> >I have understood that in addition to support from the kernel and
> >qemu-kvm, there must be support in libc. So in testing I was doing on
> >Lenny, it did not work using libc from Debian repositories. Now that
> >squeeze is frozen, anyone knows if it will support KSM?
> 
> KSM is working in qemu-kvm packages in bpo, for quite long
> time already.  It has nothing to do with libc, and now it
> does not even require recent-enough kernel headers to compile.
> What's needed is recent-enough kernel that actually implements
> the feature.  The kernel in Lenny does not KSM support, but
> the one in bpo has.  Also note that Lenny's kernel is too old
> for kvm.
> 
> /mjt
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
-------------------------------------
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava

tel.:   +420 596 603 142
fax:    +420 596 621 273
mobil:  +420 777 093 799
www.linuxbox.cz

mobil servis: +420 737 238 656
email servis: servis@linuxbox.cz
-------------------------------------

Re: KSM with Debian GNU/Linux

[Michael Tokarev]

09.08.2010 15:00, Nikola Ciprich пишет:
> Hi,
> I don't have clue what glibc is contained in debian, but it seems
> to me that at least compile-time KSM support DOES depend on glibc.
> I'm using RHEL5-based system and KSM support doesn't get compiled in
> by default.

For KSM to be compiled in, one need to have kernel headers of the
corresponding system interface (madvise() system call).  That is
not part of glibc (even while madvise function is declared by glibc)
but part of kernel headers which glibc uses itself during compilation.

In debian that comes in linux-libc-dev package, some other distribution
may bundle it together with glibc but that's pure packaging stuff,
these headers does not come from glibc sources.

> But since I use new kernel, I add following to CFLAGS while compiling
> qemu-kvm and then it works:
> -DMADV_MERGEABLE=12 -DMADV_UNMERGEABLE=13

This is what the patch in debian package does, to get KSM-ready binary
even without recent-enough kernel headers (like on Lenny).

/mjt

Re: KSM with Debian GNU/Linux

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 1968 bytes --]

On Monday, 09 August 2010 14:27:12 +0400,
Michael Tokarev wrote:

>> I have understood that in addition to support from the kernel and
>> qemu-kvm, there must be support in libc. So in testing I was doing on
>> Lenny, it did not work using libc from Debian repositories. Now that
>> squeeze is frozen, anyone knows if it will support KSM?

> KSM is working in qemu-kvm packages in bpo, for quite long
> time already.  It has nothing to do with libc, and now it
> does not even require recent-enough kernel headers to compile.
> What's needed is recent-enough kernel that actually implements
> the feature.  The kernel in Lenny does not KSM support, but
> the one in bpo has.  Also note that Lenny's kernel is too old
> for kvm.

I'm using Linux 2.6.32.3 compiled by myself with the kernel.org source
code with support for KSM:

# cat /boot/config-2.6.32.3-dgb | grep KSM
CONFIG_KSM=y


I'm also using qemu-kvm 0.12.1.2 compiled by myself with the source code
of SourceForge.


These are the checks that I was doing in the VMHost:

# ll /sys/kernel/mm/ksm/
total 0
-r--r--r-- 1 root root 4096 ago  9 06:28 full_scans
-rw-r--r-- 1 root root 4096 ago  9 06:28 max_kernel_pages
-r--r--r-- 1 root root 4096 ago  9 06:28 pages_shared
-r--r--r-- 1 root root 4096 ago  9 06:28 pages_sharing
-rw-r--r-- 1 root root 4096 ago  9 06:28 pages_to_scan
-r--r--r-- 1 root root 4096 ago  9 06:28 pages_unshared
-r--r--r-- 1 root root 4096 ago  9 06:28 pages_volatile
-rw-r--r-- 1 root root 4096 ago  9 06:27 run
-rw-r--r-- 1 root root 4096 ago  9 06:28 sleep_millisecs


# cat /sys/kernel/mm/ksm/max_kernel_pages
253500


But KSM seems not work:

# cat /sys/kernel/mm/ksm/pages_sharing
0


Should I have some additional consideration when compiling qemu-kvm?

Thanks for your reply.

Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: KSM with Debian GNU/Linux

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On Monday, 09 August 2010 13:00:49 +0200,
Nikola Ciprich wrote:

> Hi,

Hi, Nikola.

> I don't have clue what glibc is contained in debian, but it seems
> to me that at least compile-time KSM support DOES depend on glibc.
> I'm using RHEL5-based system and KSM support doesn't get compiled in
> by default.
> But since I use new kernel, I add following to CFLAGS while compiling
> qemu-kvm and then it works:
> -DMADV_MERGEABLE=12 -DMADV_UNMERGEABLE=13

My comments on the dependency of glibc support was based on reading this
bug report [1].

Regards,
Daniel

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=556631
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: KSM with Debian GNU/Linux

[Michael Tokarev]

09.08.2010 18:27, Daniel Bareiro wrote:
> On Monday, 09 August 2010 14:27:12 +0400,
> Michael Tokarev wrote:
>
>>> I have understood that in addition to support from the kernel and
>>> qemu-kvm, there must be support in libc. So in testing I was doing on
>>> Lenny, it did not work using libc from Debian repositories. Now that
>>> squeeze is frozen, anyone knows if it will support KSM?
>
>> KSM is working in qemu-kvm packages in bpo, for quite long
>> time already.  It has nothing to do with libc, and now it
>> does not even require recent-enough kernel headers to compile.
>> What's needed is recent-enough kernel that actually implements
>> the feature.  The kernel in Lenny does not KSM support, but
>> the one in bpo has.  Also note that Lenny's kernel is too old
>> for kvm.
>
> I'm using Linux 2.6.32.3 compiled by myself with the kernel.org source
> code with support for KSM:
>
> # cat /boot/config-2.6.32.3-dgb | grep KSM
> CONFIG_KSM=y
>
> I'm also using qemu-kvm 0.12.1.2 compiled by myself with the source code
> of SourceForge.

Note that this one has numerous bugs, some of which involves
data corruption.  JFYI.

> These are the checks that I was doing in the VMHost:
>
> # ll /sys/kernel/mm/ksm/
> total 0
> -r--r--r-- 1 root root 4096 ago  9 06:28 full_scans
> -rw-r--r-- 1 root root 4096 ago  9 06:28 max_kernel_pages
> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_shared
> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_sharing
> -rw-r--r-- 1 root root 4096 ago  9 06:28 pages_to_scan
> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_unshared
> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_volatile
> -rw-r--r-- 1 root root 4096 ago  9 06:27 run
> -rw-r--r-- 1 root root 4096 ago  9 06:28 sleep_millisecs
>
>
> # cat /sys/kernel/mm/ksm/max_kernel_pages
> 253500
>
> But KSM seems not work:
>
> # cat /sys/kernel/mm/ksm/pages_sharing
> 0

Did you actually enable it in the kernel?  Did you read
the ksm docs shipped with the kernel?

> Should I have some additional consideration when compiling qemu-kvm?

Please note what Nikola Ciprich said.  If you don't have
kernel headers that defines MADV_MERGEABLE, you'll have
to define it when compiling kvm.

Note again that debian qemu-kvm package (you're asking
about debian after all) includes support for KSM.  It
also includes a small patch to #define that symbol.

/mjt

Re: KSM with Debian GNU/Linux

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 2673 bytes --]

Hi, Michael.

On Monday, 09 August 2010 19:28:23 +0400,
Michael Tokarev wrote:

>> I'm also using qemu-kvm 0.12.1.2 compiled by myself with the source
>> code of SourceForge.

> Note that this one has numerous bugs, some of which involves data
> corruption. JFYI.

Thanks for the observation.

>> These are the checks that I was doing in the VMHost:
>>
>> # ll /sys/kernel/mm/ksm/
>> total 0
>> -r--r--r-- 1 root root 4096 ago  9 06:28 full_scans
>> -rw-r--r-- 1 root root 4096 ago  9 06:28 max_kernel_pages
>> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_shared
>> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_sharing
>> -rw-r--r-- 1 root root 4096 ago  9 06:28 pages_to_scan
>> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_unshared
>> -r--r--r-- 1 root root 4096 ago  9 06:28 pages_volatile
>> -rw-r--r-- 1 root root 4096 ago  9 06:27 run
>> -rw-r--r-- 1 root root 4096 ago  9 06:28 sleep_millisecs
>>
>>
>> # cat /sys/kernel/mm/ksm/max_kernel_pages
>> 253500
>>
>> But KSM seems not work:
>>
>> # cat /sys/kernel/mm/ksm/pages_sharing
>> 0

> Did you actually enable it in the kernel? Did you read the ksm docs
> shipped with the kernel?

Yes, I forgot to mention that I had enabled it:

# cat /sys/kernel/mm/ksm/run
1

>> Should I have some additional consideration when compiling qemu-kvm?

> Please note what Nikola Ciprich said. If you don't have kernel headers
> that defines MADV_MERGEABLE, you'll have to define it when compiling
> kvm.

> Note again that debian qemu-kvm package (you're asking about debian
> after all) includes support for KSM. It also includes a small patch to
> #define that symbol.

Keeping the kernel I had compiled and installing the qemu-kvm package in
Backports, now KSM is working:

# cat /sys/kernel/mm/ksm/pages_sharing
181406


This indicates that this was a problem in compiling qemu-kvm since the
kernel is still exactly the same. Now, according to what I was seeing,
headers of my kernel defines MADV_MERGEABLE:

# cat /usr/src/linux-2.6.32.3/debian/linux-headers-2.6.32.3-dgb/usr/src/linux-headers-2.6.32.3-dgb/include/asm-generic/mman-common.h | grep MERGEABLE
#define MADV_MERGEABLE   12             /* KSM may merge identical pages */
#define MADV_UNMERGEABLE 13             /* KSM may not merge identical pages */


Would it be that perhaps in my compilation of qemu-kvm, ksm was not
working because I would have to indicate the path
/usr/src/linux-2.6.32.3/debian/...?


Thanks for your reply.


Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: KSM with Debian GNU/Linux

[Daniel Bareiro]

[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]

Hi, all!

On Thursday, 12 August 2010 22:05:34 -0300,
Daniel Bareiro wrote:

> Keeping the kernel I had compiled and installing the qemu-kvm package
> in Backports, now KSM is working:
> 
> # cat /sys/kernel/mm/ksm/pages_sharing
> 181406

Looking at the statistics of the values obtained running 15 virtual
machines totaling 10.7 GB on a 4 GB VMHost, I get the following, which
is a very interesting memory savings:

# for ii in /sys/kernel/mm/ksm/* ; do echo -n "$ii: " ; cat $ii ; done
/sys/kernel/mm/ksm/full_scans: 4114
/sys/kernel/mm/ksm/max_kernel_pages: 253500
/sys/kernel/mm/ksm/pages_shared: 67064
/sys/kernel/mm/ksm/pages_sharing: 510990
/sys/kernel/mm/ksm/pages_to_scan: 100
/sys/kernel/mm/ksm/pages_unshared: 448079
/sys/kernel/mm/ksm/pages_volatile: 13595
/sys/kernel/mm/ksm/run: 1
/sys/kernel/mm/ksm/sleep_millisecs: 20

# free
             total       used       free     shared    buffers     cached
Mem:       4056468    2578728    1477740          0       3736      62156
-/+ buffers/cache:    2512836    1543632
Swap:       497848      25972     471876


Some recommendation about tunning of KSM?

I've no very clear about the difference between page_shared and
page_sharing. Somebody could clarify it?

Thanks for your reply.

Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Lenny - Linux user #188.598

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]