Luvalley-2 has been released: running KVM below any operating system

Luvalley-2 has been released: running KVM below any operating system

[Xiaodong Yi]

Luvalley is a Virtual Machine Monitor (VMM) spawned from the KVM
project. Its part of source codes are derived from KVM to virtualize
CPU instructions and memory management unit (MMU). However, its
overall architecture is completely different from KVM, but somewhat
like Xen. Luvalley runs outside of Linux, just like Xen's
architecture. Any operating system, including Linux, could be used as
Luvalley's scheduler, memory
manager, physical device driver provider and virtual IO device
emulator. Currently, Luvalley supports Linux and Windows. That is to
say, one may run Luvalley to boot a Linux or Windows, and then run
multiple virtualized operating systems on such Linux or Windows.

If you are interested in Luvalley project, you may download the source
codes from
    http://sourceforge.net/projects/luvalley/

This release (i.e., luvalley-2) updated Qemu in both Linux and
Windows. So the limitations of the previous release (i.e., luvalley-1)
have been partially eliminated. For example, the Qemu in Windows
allows to create disk images and install gueest operating systems.

This release has 4 tarballs:

 * luvalley-2.tgz: this is the source codes of the kernel part of
Luvalley, which should be compiled in Linux. It is running below any
operating system to provide virtualization extension for Intel's
X86/32 computers. See the README file of the tarball for how to
compile and run Luvalley.

 * qemu-linux.tgz: this is the source codes of the modified Qemu for
running virtualized guest OSs in Linux. It is derived from Qemu of
KVM-83. See the README file of the tarball for how to compile and run.

 * qemu-windows.tgz: this is the source codes of the modified Qemu for
running virtualized guest OSs in Windows. It is derived from
Qemu-0.10.2. See the README file of the tarball for how to build and
run.

 * WindowsXP-Bins.rar: this is the pre-compiled binaries for running
in Windows XP. It should be uncompressed in Windows XP operating
system with WinRAR or other programs that supports .rar format. See
the README file of the package for how to install and run in Windows
XP.

RE: Luvalley-2 has been released: running KVM below any operating system

[Zhang, Xiantao]

How about performance comparision with kvm guests?  
Xiantao 

-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Xiaodong Yi
Sent: Thursday, April 16, 2009 9:54 AM
To: kvm@vger.kernel.org
Subject: Luvalley-2 has been released: running KVM below any operating system

Luvalley is a Virtual Machine Monitor (VMM) spawned from the KVM
project. Its part of source codes are derived from KVM to virtualize
CPU instructions and memory management unit (MMU). However, its
overall architecture is completely different from KVM, but somewhat
like Xen. Luvalley runs outside of Linux, just like Xen's
architecture. Any operating system, including Linux, could be used as
Luvalley's scheduler, memory
manager, physical device driver provider and virtual IO device
emulator. Currently, Luvalley supports Linux and Windows. That is to
say, one may run Luvalley to boot a Linux or Windows, and then run
multiple virtualized operating systems on such Linux or Windows.

If you are interested in Luvalley project, you may download the source
codes from
    http://sourceforge.net/projects/luvalley/

This release (i.e., luvalley-2) updated Qemu in both Linux and
Windows. So the limitations of the previous release (i.e., luvalley-1)
have been partially eliminated. For example, the Qemu in Windows
allows to create disk images and install gueest operating systems.

This release has 4 tarballs:

 * luvalley-2.tgz: this is the source codes of the kernel part of
Luvalley, which should be compiled in Linux. It is running below any
operating system to provide virtualization extension for Intel's
X86/32 computers. See the README file of the tarball for how to
compile and run Luvalley.

 * qemu-linux.tgz: this is the source codes of the modified Qemu for
running virtualized guest OSs in Linux. It is derived from Qemu of
KVM-83. See the README file of the tarball for how to compile and run.

 * qemu-windows.tgz: this is the source codes of the modified Qemu for
running virtualized guest OSs in Windows. It is derived from
Qemu-0.10.2. See the README file of the tarball for how to build and
run.

 * WindowsXP-Bins.rar: this is the pre-compiled binaries for running
in Windows XP. It should be uncompressed in Windows XP operating
system with WinRAR or other programs that supports .rar format. See
the README file of the package for how to install and run in Windows
XP.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Luvalley-2 has been released: running KVM below any operating system

[Xiaodong Yi]

Hi,

I've tested the guest Linux using UnixBench 5.1.2. The platform is:
  * Intel's Core Due CPU with 2 cores, 2GB RAM
  * CentOS 5.2 as the dom0 Linux, i.e., the host Linux for KVM
  * CentOS 5.2 as the guest Linux, i.e., the Linux running on the
virtual machine provided by Qemu

The first set of results is for Luvalley, and the second one is for
KVM. As the result, Luvalley's guest Linux is 20% ~ 30% faster than
KVM's guest! It is very surprise to me. I had through Luvalley's guest
should be the same performance as KVM's.




Luvalley's result:

========================================================================
   BYTE UNIX Benchmarks (Version 5.1.2)

   System: localhost.localdomain: GNU/Linux
   OS: GNU/Linux -- 2.6.18-53.el5 -- #1 SMP Mon Nov 12 02:22:48 EST 2007
   Machine: i686 (i386)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: QEMU Virtual CPU version 0.9.1 (5326.4 bogomips)
          x86-64, MMX, Physical Address Ext
   CPU 1: QEMU Virtual CPU version 0.9.1 (5319.9 bogomips)
          x86-64, MMX, Physical Address Ext
   11:32:22 up 1 min,  1 user,  load average: 2.39, 1.07, 0.39; runlevel 5

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 11:32:22 - 11:44:22
2 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       10802400.5 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                    12287.7 MWIPS (10.0 s, 2 samples)
Execl Throughput                               1044.6 lps   (29.2 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        429860.0 KBps  (30.0 s, 1 samples)
File Copy 256 bufsize 500 maxblocks          125357.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks        990103.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                              803044.2 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                 124785.9 lps   (10.0 s, 2 samples)
Process Creation                               1861.8 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                   2338.6 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                    438.3 lpm   (60.1 s, 1 samples)
System Call Overhead                         709335.4 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   10802400.5    925.7
Double-Precision Whetstone                       55.0      12287.7   2234.1
Execl Throughput                                 43.0       1044.6    242.9
File Copy 1024 bufsize 2000 maxblocks          3960.0     429860.0   1085.5
File Copy 256 bufsize 500 maxblocks            1655.0     125357.0    757.4
File Copy 4096 bufsize 8000 maxblocks          5800.0     990103.0   1707.1
Pipe Throughput                               12440.0     803044.2    645.5
Pipe-based Context Switching                   4000.0     124785.9    312.0
Process Creation                                126.0       1861.8    147.8
Shell Scripts (1 concurrent)                     42.4       2338.6    551.6
Shell Scripts (8 concurrent)                      6.0        438.3    730.5
System Call Overhead                          15000.0     709335.4    472.9
                                                                   ========
System Benchmarks Index Score                                         631.2

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 11:44:22 - 11:56:05
2 CPUs in system; running 2 parallel copies of tests

Dhrystone 2 using register variables       22031241.9 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                    23862.7 MWIPS (10.0 s, 2 samples)
Execl Throughput                               1691.9 lps   (29.8 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        153766.0 KBps  (30.1 s, 1 samples)
File Copy 256 bufsize 500 maxblocks           45848.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks        432211.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                             1617636.7 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                 233890.1 lps   (10.0 s, 2 samples)
Process Creation                               3207.9 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                   3151.4 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                    437.8 lpm   (60.2 s, 1 samples)
System Call Overhead                        1386223.1 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   22031241.9   1887.9
Double-Precision Whetstone                       55.0      23862.7   4338.7
Execl Throughput                                 43.0       1691.9    393.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     153766.0    388.3
File Copy 256 bufsize 500 maxblocks            1655.0      45848.0    277.0
File Copy 4096 bufsize 8000 maxblocks          5800.0     432211.0    745.2
Pipe Throughput                               12440.0    1617636.7   1300.4
Pipe-based Context Switching                   4000.0     233890.1    584.7
Process Creation                                126.0       3207.9    254.6
Shell Scripts (1 concurrent)                     42.4       3151.4    743.3
Shell Scripts (8 concurrent)                      6.0        437.8    729.7
System Call Overhead                          15000.0    1386223.1    924.1
                                                                   ========
System Benchmarks Index Score                                         735.5





KVM's results:

========================================================================
   BYTE UNIX Benchmarks (Version 5.1.2)

   System: localhost.localdomain: GNU/Linux
   OS: GNU/Linux -- 2.6.18-53.el5 -- #1 SMP Mon Nov 12 02:22:48 EST 2007
   Machine: i686 (i386)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: QEMU Virtual CPU version 0.9.1 (5325.7 bogomips)
          x86-64, MMX, Physical Address Ext
   CPU 1: QEMU Virtual CPU version 0.9.1 (5319.6 bogomips)
          x86-64, MMX, Physical Address Ext
   12:02:30 up 1 min,  1 user,  load average: 2.37, 0.87, 0.31; runlevel 5

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 12:02:30 - 12:11:33
2 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       10599139.8 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                     2166.3 MWIPS (10.2 s, 2 samples)
Execl Throughput                                598.3 lps   (29.9 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        458264.0 KBps  (30.0 s, 1 samples)
File Copy 256 bufsize 500 maxblocks          125402.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks       1122309.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                              811955.6 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                 116759.0 lps   (10.0 s, 2 samples)
Process Creation                               1503.8 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                   1942.2 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                    374.9 lpm   (60.0 s, 1 samples)
System Call Overhead                         712668.8 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   10599139.8    908.2
Double-Precision Whetstone                       55.0       2166.3    393.9
Execl Throughput                                 43.0        598.3    139.1
File Copy 1024 bufsize 2000 maxblocks          3960.0     458264.0   1157.2
File Copy 256 bufsize 500 maxblocks            1655.0     125402.0    757.7
File Copy 4096 bufsize 8000 maxblocks          5800.0    1122309.0   1935.0
Pipe Throughput                               12440.0     811955.6    652.7
Pipe-based Context Switching                   4000.0     116759.0    291.9
Process Creation                                126.0       1503.8    119.4
Shell Scripts (1 concurrent)                     42.4       1942.2    458.1
Shell Scripts (8 concurrent)                      6.0        374.9    624.9
System Call Overhead                          15000.0     712668.8    475.1
                                                                   ========
System Benchmarks Index Score                                         502.8

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 12:11:33 - 12:20:43
2 CPUs in system; running 2 parallel copies of tests

Dhrystone 2 using register variables       21416721.5 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                     4928.6 MWIPS (10.1 s, 2 samples)
Execl Throughput                               1438.2 lps   (29.6 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        122731.0 KBps  (30.1 s, 1 samples)
File Copy 256 bufsize 500 maxblocks           32222.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks        308986.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                             1617083.9 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                 230390.0 lps   (10.0 s, 2 samples)
Process Creation                               2373.6 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                   2732.8 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                    373.4 lpm   (60.1 s, 1 samples)
System Call Overhead                        1393848.5 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   21416721.5   1835.2
Double-Precision Whetstone                       55.0       4928.6    896.1
Execl Throughput                                 43.0       1438.2    334.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     122731.0    309.9
File Copy 256 bufsize 500 maxblocks            1655.0      32222.0    194.7
File Copy 4096 bufsize 8000 maxblocks          5800.0     308986.0    532.7
Pipe Throughput                               12440.0    1617083.9   1299.9
Pipe-based Context Switching                   4000.0     230390.0    576.0
Process Creation                                126.0       2373.6    188.4
Shell Scripts (1 concurrent)                     42.4       2732.8    644.5
Shell Scripts (8 concurrent)                      6.0        373.4    622.4
System Call Overhead                          15000.0    1393848.5    929.2
                                                                   ========
System Benchmarks Index Score                                         558.9



Thanks for your attention and welcome further feedback.

Regards,

Xiaodong Yi


2009/4/16 Zhang, Xiantao <xiantao.zhang@intel.com>:
> How about performance comparision with kvm guests?
> Xiantao
>
> -----Original Message-----
> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Xiaodong Yi
> Sent: Thursday, April 16, 2009 9:54 AM
> To: kvm@vger.kernel.org
> Subject: Luvalley-2 has been released: running KVM below any operating system
>
> Luvalley is a Virtual Machine Monitor (VMM) spawned from the KVM
> project. Its part of source codes are derived from KVM to virtualize
> CPU instructions and memory management unit (MMU). However, its
> overall architecture is completely different from KVM, but somewhat
> like Xen. Luvalley runs outside of Linux, just like Xen's
> architecture. Any operating system, including Linux, could be used as
> Luvalley's scheduler, memory
> manager, physical device driver provider and virtual IO device
> emulator. Currently, Luvalley supports Linux and Windows. That is to
> say, one may run Luvalley to boot a Linux or Windows, and then run
> multiple virtualized operating systems on such Linux or Windows.
>
> If you are interested in Luvalley project, you may download the source
> codes from
>    http://sourceforge.net/projects/luvalley/
>
> This release (i.e., luvalley-2) updated Qemu in both Linux and
> Windows. So the limitations of the previous release (i.e., luvalley-1)
> have been partially eliminated. For example, the Qemu in Windows
> allows to create disk images and install gueest operating systems.
>
> This release has 4 tarballs:
>
>  * luvalley-2.tgz: this is the source codes of the kernel part of
> Luvalley, which should be compiled in Linux. It is running below any
> operating system to provide virtualization extension for Intel's
> X86/32 computers. See the README file of the tarball for how to
> compile and run Luvalley.
>
>  * qemu-linux.tgz: this is the source codes of the modified Qemu for
> running virtualized guest OSs in Linux. It is derived from Qemu of
> KVM-83. See the README file of the tarball for how to compile and run.
>
>  * qemu-windows.tgz: this is the source codes of the modified Qemu for
> running virtualized guest OSs in Windows. It is derived from
> Qemu-0.10.2. See the README file of the tarball for how to build and
> run.
>
>  * WindowsXP-Bins.rar: this is the pre-compiled binaries for running
> in Windows XP. It should be uncompressed in Windows XP operating
> system with WinRAR or other programs that supports .rar format. See
> the README file of the package for how to install and run in Windows
> XP.
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Luvalley-2 has been released: running KVM below any operating system

[Avi Kivity]

Xiaodong Yi wrote:
> Hi,
>
> I've tested the guest Linux using UnixBench 5.1.2. The platform is:
>   * Intel's Core Due CPU with 2 cores, 2GB RAM
>   * CentOS 5.2 as the dom0 Linux, i.e., the host Linux for KVM
>   * CentOS 5.2 as the guest Linux, i.e., the Linux running on the
> virtual machine provided by Qemu
>
> The first set of results is for Luvalley, and the second one is for
> KVM. As the result, Luvalley's guest Linux is 20% ~ 30% faster than
> KVM's guest! It is very surprise to me. I had through Luvalley's guest
> should be the same performance as KVM's.
>
>   

Yes, it is surprising.

> Double-Precision Whetstone                    12287.7 MWIPS (10.0 s, 2 samples)
> Double-Precision Whetstone                     2166.3 MWIPS (10.2 s, 2 samples

That's by far the biggest difference. Can you confirm it isn't a typo?

If not, then it looks like we have a bug in floating point handling. I
don't think this benchmark uses sse.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: Luvalley-2 has been released: running KVM below any operating system

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1172 bytes --]

Avi Kivity wrote:
> Xiaodong Yi wrote:
>> Hi,
>>
>> I've tested the guest Linux using UnixBench 5.1.2. The platform is:
>>   * Intel's Core Due CPU with 2 cores, 2GB RAM
>>   * CentOS 5.2 as the dom0 Linux, i.e., the host Linux for KVM
>>   * CentOS 5.2 as the guest Linux, i.e., the Linux running on the
>> virtual machine provided by Qemu
>>
>> The first set of results is for Luvalley, and the second one is for
>> KVM. As the result, Luvalley's guest Linux is 20% ~ 30% faster than
>> KVM's guest! It is very surprise to me. I had through Luvalley's guest
>> should be the same performance as KVM's.
>>
>>   
> 
> Yes, it is surprising.
> 
>> Double-Precision Whetstone                    12287.7 MWIPS (10.0 s, 2 samples)
>> Double-Precision Whetstone                     2166.3 MWIPS (10.2 s, 2 samples
> 
> That's by far the biggest difference. Can you confirm it isn't a typo?
> 
> If not, then it looks like we have a bug in floating point handling. I
> don't think this benchmark uses sse.
> 

Even the native Linux numbers are not that high, rather comparable to
KVM. I suspect Luvalley is fooling the benchmark here...

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: Luvalley-2 has been released: running KVM below any operating system

[Avi Kivity]

Jan Kiszka wrote:
> Avi Kivity wrote:
>   
>> Xiaodong Yi wrote:
>>     
>>> Hi,
>>>
>>> I've tested the guest Linux using UnixBench 5.1.2. The platform is:
>>>   * Intel's Core Due CPU with 2 cores, 2GB RAM
>>>   * CentOS 5.2 as the dom0 Linux, i.e., the host Linux for KVM
>>>   * CentOS 5.2 as the guest Linux, i.e., the Linux running on the
>>> virtual machine provided by Qemu
>>>
>>> The first set of results is for Luvalley, and the second one is for
>>> KVM. As the result, Luvalley's guest Linux is 20% ~ 30% faster than
>>> KVM's guest! It is very surprise to me. I had through Luvalley's guest
>>> should be the same performance as KVM's.
>>>
>>>   
>>>       
>> Yes, it is surprising.
>>
>>     
>>> Double-Precision Whetstone                    12287.7 MWIPS (10.0 s, 2 samples)
>>> Double-Precision Whetstone                     2166.3 MWIPS (10.2 s, 2 samples
>>>       
>> That's by far the biggest difference. Can you confirm it isn't a typo?
>>
>> If not, then it looks like we have a bug in floating point handling. I
>> don't think this benchmark uses sse.
>>
>>     
>
> Even the native Linux numbers are not that high, rather comparable to
> KVM. I suspect Luvalley is fooling the benchmark here...
>   

Most likely timing. Likely Luvally guests time runs to slowly, so they
achieve more loops per unit of guest time.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: Luvalley-2 has been released: running KVM below any operating system

[Xiaodong Yi]

It is not a typo. I copied from UnixBench output directly. Howver, it
must be a bug of Luvalley because even the native Linux benchmark on
Double-Precision Whetstone is not that high. I also noticed that other
benchmarks are all lower than native Linux.

About timing, Luvalley does nothing more than KVM except that Luvalley
implemented the VLAPIC timer using TSC while KVM uses the services of
Linux kernel. The other timers of both Luvalley and KVM, I think, are
all implemented in Qemu.

Moreover, I could not explain why Luvalley's benchmarks on process
creation, execl throughput, file copy and shell script are 20% ~ 40%
higher than KVM, while other benchmarks on pipe throughput, context
switching and syscall overhead are almost the same as KVM.

Thanks for your attention,

Xiaodong

2009/4/19 Avi Kivity <avi@redhat.com>:
> Jan Kiszka wrote:
>> Avi Kivity wrote:
>>
>>> Xiaodong Yi wrote:
>>>
>>>> Hi,
>>>>
>>>> I've tested the guest Linux using UnixBench 5.1.2. The platform is:
>>>>   * Intel's Core Due CPU with 2 cores, 2GB RAM
>>>>   * CentOS 5.2 as the dom0 Linux, i.e., the host Linux for KVM
>>>>   * CentOS 5.2 as the guest Linux, i.e., the Linux running on the
>>>> virtual machine provided by Qemu
>>>>
>>>> The first set of results is for Luvalley, and the second one is for
>>>> KVM. As the result, Luvalley's guest Linux is 20% ~ 30% faster than
>>>> KVM's guest! It is very surprise to me. I had through Luvalley's guest
>>>> should be the same performance as KVM's.
>>>>
>>>>
>>>>
>>> Yes, it is surprising.
>>>
>>>
>>>> Double-Precision Whetstone                    12287.7 MWIPS (10.0 s, 2 samples)
>>>> Double-Precision Whetstone                     2166.3 MWIPS (10.2 s, 2 samples
>>>>
>>> That's by far the biggest difference. Can you confirm it isn't a typo?
>>>
>>> If not, then it looks like we have a bug in floating point handling. I
>>> don't think this benchmark uses sse.
>>>
>>>
>>
>> Even the native Linux numbers are not that high, rather comparable to
>> KVM. I suspect Luvalley is fooling the benchmark here...
>>
>
> Most likely timing. Likely Luvally guests time runs to slowly, so they
> achieve more loops per unit of guest time.
>
> --
> Do not meddle in the internals of kernels, for they are subtle and quick to panic.
>
>

RE: Luvalley-2 has been released: running KVM below any operating system

[Dong, Eddie]

Xiaodong Yi wrote:
> It is not a typo. I copied from UnixBench output directly. Howver, it
> must be a bug of Luvalley because even the native Linux benchmark on
> Double-Precision Whetstone is not that high. I also noticed that other
> benchmarks are all lower than native Linux.
> 
> About timing, Luvalley does nothing more than KVM except that Luvalley
> implemented the VLAPIC timer using TSC while KVM uses the services of
> Linux kernel. The other timers of both Luvalley and KVM, I think, are
> all implemented in Qemu.
> 
> Moreover, I could not explain why Luvalley's benchmarks on process
> creation, execl throughput, file copy and shell script are 20% ~ 40%
> higher than KVM, while other benchmarks on pipe throughput, context
> switching and syscall overhead are almost the same as KVM.
> 

A typical issue in VMM benchmarking using OS benchmark such as what you used is time inaccurate issue.

Benchmarks using guest time won't be able to get a right time or right duration due to scheduler etc, and thus VMM benchmarks is using network time for measuring such as vConsolidate.  Spec.org is definning their benchmark for VMM, and I believe they will use network time too.

For simplicity, you may continue use OS benchmark to measure VMM, but then you need to calibrate guest time accuracy first such as using stop watch etc. In both Xen & KVM, we benchmark using OS benchmark too, but it is usually only to see improvement of a performance patch. Formal benchmark data needs to consult wall clock or stop watch.

Thx, eddie

Re: Luvalley-2 has been released: running KVM below any operating system

[Xiaodong Yi]

Thanks for your advice and hope your continue attention on Luvalley.

Regards,
Xiaodong

2009/5/11 Dong, Eddie <eddie.dong@intel.com>:
> Xiaodong Yi wrote:
>> It is not a typo. I copied from UnixBench output directly. Howver, it
>> must be a bug of Luvalley because even the native Linux benchmark on
>> Double-Precision Whetstone is not that high. I also noticed that other
>> benchmarks are all lower than native Linux.
>>
>> About timing, Luvalley does nothing more than KVM except that Luvalley
>> implemented the VLAPIC timer using TSC while KVM uses the services of
>> Linux kernel. The other timers of both Luvalley and KVM, I think, are
>> all implemented in Qemu.
>>
>> Moreover, I could not explain why Luvalley's benchmarks on process
>> creation, execl throughput, file copy and shell script are 20% ~ 40%
>> higher than KVM, while other benchmarks on pipe throughput, context
>> switching and syscall overhead are almost the same as KVM.
>>
>
> A typical issue in VMM benchmarking using OS benchmark such as what you used is time inaccurate issue.
>
> Benchmarks using guest time won't be able to get a right time or right duration due to scheduler etc, and thus VMM benchmarks is using network time for measuring such as vConsolidate.  Spec.org is definning their benchmark for VMM, and I believe they will use network time too.
>
> For simplicity, you may continue use OS benchmark to measure VMM, but then you need to calibrate guest time accuracy first such as using stop watch etc. In both Xen & KVM, we benchmark using OS benchmark too, but it is usually only to see improvement of a performance patch. Formal benchmark data needs to consult wall clock or stop watch.
>
> Thx, eddie

Re: Luvalley-2 has been released: running KVM below any operating system

[Xiaodong Yi]

Hi,

I also tested the performance of the dom0 Linux using UnixBench 5.1.2.
For Luvalley, the dom0 Linux means the first Linux that runs on top of
Luvalley. The dom0 Linux is responsible of driving devices and running
Qemu for more virtual machines. The platform is still:
 * Intel's Core Due CPU with 2 cores, 2GB RAM
 * CentOS 5.2 as the dom0 Linux, i.e., the host Linux for KVM

The benchmark shows that, after being virtualized by Luvalley, the
performance of Linux decreases to only a half of the native one. It is
also a surprise to me. I had thought that the performance of the Linux
running on top of Luvaley should be at least 80% of the native Linux.
Currently, all page faults of dom0 Linux cause vmexits. However, these
vmexits are unnecessary. I will try to eliminate them and test again.



The Linux benchmark when running on top of Luvalley:

========================================================================
   BYTE UNIX Benchmarks (Version 5.1.2)

   System: Yxd-Linux: GNU/Linux
   OS: GNU/Linux -- 2.6.18-92.el5 -- #1 SMP Tue Jun 10 18:49:47 EDT 2008
   Machine: i686 (i386)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz (5322.8 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext
   CPU 1: Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz (5319.9 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext
   10:13:06 up 13 min,  2 users,  load average: 0.01, 0.28, 0.29; runlevel 5

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 10:13:06 - 10:22:08
2 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       10964003.6 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                     2501.0 MWIPS (10.0 s, 2 samples)
Execl Throughput                               1262.5 lps   (29.1 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        136493.0 KBps  (30.0 s, 1 samples)
File Copy 256 bufsize 500 maxblocks           35172.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks        462014.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                              158474.5 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                  57442.5 lps   (10.0 s, 2 samples)
Process Creation                               2215.5 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                   2652.2 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                    602.1 lpm   (60.1 s, 1 samples)
System Call Overhead                         741086.6 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   10964003.6    939.5
Double-Precision Whetstone                       55.0       2501.0    454.7
Execl Throughput                                 43.0       1262.5    293.6
File Copy 1024 bufsize 2000 maxblocks          3960.0     136493.0    344.7
File Copy 256 bufsize 500 maxblocks            1655.0      35172.0    212.5
File Copy 4096 bufsize 8000 maxblocks          5800.0     462014.0    796.6
Pipe Throughput                               12440.0     158474.5    127.4
Pipe-based Context Switching                   4000.0      57442.5    143.6
Process Creation                                126.0       2215.5    175.8
Shell Scripts (1 concurrent)                     42.4       2652.2    625.5
Shell Scripts (8 concurrent)                      6.0        602.1   1003.5
System Call Overhead                          15000.0     741086.6    494.1
                                                                   ========
System Benchmarks Index Score                                         373.2

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 10:22:08 - 10:31:11
2 CPUs in system; running 2 parallel copies of tests

Dhrystone 2 using register variables       21800140.9 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                     4748.4 MWIPS (10.1 s, 2 samples)
Execl Throughput                               2572.8 lps   (29.0 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        196155.0 KBps  (30.0 s, 1 samples)
File Copy 256 bufsize 500 maxblocks           50331.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks        739819.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                              314696.4 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                 121641.5 lps   (10.0 s, 2 samples)
Process Creation                               6051.3 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                   4628.9 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                    639.8 lpm   (60.1 s, 1 samples)
System Call Overhead                        1446020.8 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   21800140.9   1868.0
Double-Precision Whetstone                       55.0       4748.4    863.4
Execl Throughput                                 43.0       2572.8    598.3
File Copy 1024 bufsize 2000 maxblocks          3960.0     196155.0    495.3
File Copy 256 bufsize 500 maxblocks            1655.0      50331.0    304.1
File Copy 4096 bufsize 8000 maxblocks          5800.0     739819.0   1275.5
Pipe Throughput                               12440.0     314696.4    253.0
Pipe-based Context Switching                   4000.0     121641.5    304.1
Process Creation                                126.0       6051.3    480.3
Shell Scripts (1 concurrent)                     42.4       4628.9   1091.7
Shell Scripts (8 concurrent)                      6.0        639.8   1066.4
System Call Overhead                          15000.0    1446020.8    964.0
                                                                   ========
System Benchmarks Index Score                                         666.5




The Linux benchmark when running directly:

========================================================================
   BYTE UNIX Benchmarks (Version 5.1.2)

   System: Yxd-Linux: GNU/Linux
   OS: GNU/Linux -- 2.6.18-92.el5 -- #1 SMP Tue Jun 10 18:49:47 EDT 2008
   Machine: i686 (i386)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz (5322.8 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, Intel
virtualization
   CPU 1: Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz (5319.9 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, Intel
virtualization
   10:40:31 up 1 min,  2 users,  load average: 0.66, 0.37, 0.14; runlevel 5

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 10:40:31 - 10:49:28
2 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       10774539.6 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                     2354.4 MWIPS (10.1 s, 2 samples)
Execl Throughput                               3016.7 lps   (29.8 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        353682.0 KBps  (30.0 s, 1 samples)
File Copy 256 bufsize 500 maxblocks           93716.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks        961845.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                              509890.9 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                 180117.9 lps   (10.0 s, 2 samples)
Process Creation                               9880.0 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                   5066.4 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                   1437.8 lpm   (60.0 s, 1 samples)
System Call Overhead                         756113.6 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   10774539.6    923.3
Double-Precision Whetstone                       55.0       2354.4    428.1
Execl Throughput                                 43.0       3016.7    701.6
File Copy 1024 bufsize 2000 maxblocks          3960.0     353682.0    893.1
File Copy 256 bufsize 500 maxblocks            1655.0      93716.0    566.3
File Copy 4096 bufsize 8000 maxblocks          5800.0     961845.0   1658.4
Pipe Throughput                               12440.0     509890.9    409.9
Pipe-based Context Switching                   4000.0     180117.9    450.3
Process Creation                                126.0       9880.0    784.1
Shell Scripts (1 concurrent)                     42.4       5066.4   1194.9
Shell Scripts (8 concurrent)                      6.0       1437.8   2396.3
System Call Overhead                          15000.0     756113.6    504.1
                                                                   ========
System Benchmarks Index Score                                         777.2

------------------------------------------------------------------------
Benchmark Run: 五  4月 17 2009 10:49:28 - 10:58:28
2 CPUs in system; running 2 parallel copies of tests

Dhrystone 2 using register variables       21949598.8 lps   (10.0 s, 2 samples)
Double-Precision Whetstone                     4866.7 MWIPS (10.1 s, 2 samples)
Execl Throughput                               8040.3 lps   (29.5 s, 1 samples)
File Copy 1024 bufsize 2000 maxblocks        394264.0 KBps  (30.0 s, 1 samples)
File Copy 256 bufsize 500 maxblocks          101466.0 KBps  (30.0 s, 1 samples)
File Copy 4096 bufsize 8000 maxblocks       1307480.0 KBps  (30.0 s, 1 samples)
Pipe Throughput                             1018531.7 lps   (10.0 s, 2 samples)
Pipe-based Context Switching                 399822.2 lps   (10.0 s, 2 samples)
Process Creation                              23748.0 lps   (30.0 s, 1 samples)
Shell Scripts (1 concurrent)                  10082.3 lpm   (60.0 s, 1 samples)
Shell Scripts (8 concurrent)                   1514.3 lpm   (60.0 s, 1 samples)
System Call Overhead                        1470446.3 lps   (10.0 s, 2 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   21949598.8   1880.9
Double-Precision Whetstone                       55.0       4866.7    884.9
Execl Throughput                                 43.0       8040.3   1869.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     394264.0    995.6
File Copy 256 bufsize 500 maxblocks            1655.0     101466.0    613.1
File Copy 4096 bufsize 8000 maxblocks          5800.0    1307480.0   2254.3
Pipe Throughput                               12440.0    1018531.7    818.8
Pipe-based Context Switching                   4000.0     399822.2    999.6
Process Creation                                126.0      23748.0   1884.8
Shell Scripts (1 concurrent)                     42.4      10082.3   2377.9
Shell Scripts (8 concurrent)                      6.0       1514.3   2523.8
System Call Overhead                          15000.0    1470446.3    980.3
                                                                   ========
System Benchmarks Index Score                                        1356.6


Welcome to feedback.

Regards,

Xiaodong Yi


2009/4/16 Zhang, Xiantao <xiantao.zhang@intel.com>:
> How about performance comparision with kvm guests?
> Xiantao
>
> -----Original Message-----
> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Xiaodong Yi
> Sent: Thursday, April 16, 2009 9:54 AM
> To: kvm@vger.kernel.org
> Subject: Luvalley-2 has been released: running KVM below any operating system
>
> Luvalley is a Virtual Machine Monitor (VMM) spawned from the KVM
> project. Its part of source codes are derived from KVM to virtualize
> CPU instructions and memory management unit (MMU). However, its
> overall architecture is completely different from KVM, but somewhat
> like Xen. Luvalley runs outside of Linux, just like Xen's
> architecture. Any operating system, including Linux, could be used as
> Luvalley's scheduler, memory
> manager, physical device driver provider and virtual IO device
> emulator. Currently, Luvalley supports Linux and Windows. That is to
> say, one may run Luvalley to boot a Linux or Windows, and then run
> multiple virtualized operating systems on such Linux or Windows.
>
> If you are interested in Luvalley project, you may download the source
> codes from
>    http://sourceforge.net/projects/luvalley/
>
> This release (i.e., luvalley-2) updated Qemu in both Linux and
> Windows. So the limitations of the previous release (i.e., luvalley-1)
> have been partially eliminated. For example, the Qemu in Windows
> allows to create disk images and install gueest operating systems.
>
> This release has 4 tarballs:
>
>  * luvalley-2.tgz: this is the source codes of the kernel part of
> Luvalley, which should be compiled in Linux. It is running below any
> operating system to provide virtualization extension for Intel's
> X86/32 computers. See the README file of the tarball for how to
> compile and run Luvalley.
>
>  * qemu-linux.tgz: this is the source codes of the modified Qemu for
> running virtualized guest OSs in Linux. It is derived from Qemu of
> KVM-83. See the README file of the tarball for how to compile and run.
>
>  * qemu-windows.tgz: this is the source codes of the modified Qemu for
> running virtualized guest OSs in Windows. It is derived from
> Qemu-0.10.2. See the README file of the tarball for how to build and
> run.
>
>  * WindowsXP-Bins.rar: this is the pre-compiled binaries for running
> in Windows XP. It should be uncompressed in Windows XP operating
> system with WinRAR or other programs that supports .rar format. See
> the README file of the package for how to install and run in Windows
> XP.
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Luvalley-2 has been released: running KVM below any operating system

[Anthony Liguori]

Xiaodong Yi wrote:
>  * qemu-windows.tgz: this is the source codes of the modified Qemu for
> running virtualized guest OSs in Windows. It is derived from
> Qemu-0.10.2. See the README file of the tarball for how to build and
> run.
>   

While your patch is still small, I'd suggest submitting it to 
qemu-devel.  I'm not crazy about the vmcall in userspace interface but 
I'm sure we can work something out.

I think we need to do a better job at not confusing KVM support with 
luvalley support too.  For instance, we want to introduce an 'info 
luvalley' command.

Regards,

Anthony Liguori