KVM performance

KVM performance

[Randy Broman]

I am using Intel Core2 Duo E6600, Kubuntu 8.04 with kernel 
2.6.24-21-generic,
kvm (as in "QEMU PC emulator version 0.9.1 (kvm-62)") and a WinXP SP3 
guest,
with bridged networking. My start command is:

sudo kvm -m 1024 -cdrom /dev/cdrom -boot c -net 
nic,macaddr=00:d0:13:b0:2d:32,
model=rtl8139 -net tap -soundhw all -localtime /home/rbroman/windows.img

All this is stable and generally works well, except that internet-based 
video and
audio performance is poor (choppy, skips) in comparison with performance 
under
WinXP running native on the same machine (it's a dual-boot). I would 
appreciate
recommendations to improve video and audio performance, and have the 
following
specific questions:

-I've tried both the default Cirrus adapter and the "-std-vga" option. 
Which is better?
I saw reference to another VMware-based adapter, but I can't figure out 
how to implement
it - would that be better?

-I notice we're up to kvm-79 vs my kvm-62. Should I move to the newer 
version? Do I
have to custom-compile my kernel to do so, and if so what kernel version 
and what
specific kernel options should I use?

-Are there other tuning steps I could take?

Please copy me directly as I'm not on this list. Thankyou

Re: KVM performance

[David S. Ahern]

See if boosting the priority of the VM (see man chrt), and locking it to
a core (see man taskset) helps. You'll want to do that for the vcpu
thread(s) (in the qmeu monitor, run 'info cpus' command).

david


Randy Broman wrote:
> I am using Intel Core2 Duo E6600, Kubuntu 8.04 with kernel
> 2.6.24-21-generic,
> kvm (as in "QEMU PC emulator version 0.9.1 (kvm-62)") and a WinXP SP3
> guest,
> with bridged networking. My start command is:
> 
> sudo kvm -m 1024 -cdrom /dev/cdrom -boot c -net
> nic,macaddr=00:d0:13:b0:2d:32,
> model=rtl8139 -net tap -soundhw all -localtime /home/rbroman/windows.img
> 
> All this is stable and generally works well, except that internet-based
> video and
> audio performance is poor (choppy, skips) in comparison with performance
> under
> WinXP running native on the same machine (it's a dual-boot). I would
> appreciate
> recommendations to improve video and audio performance, and have the
> following
> specific questions:
> 
> -I've tried both the default Cirrus adapter and the "-std-vga" option.
> Which is better?
> I saw reference to another VMware-based adapter, but I can't figure out
> how to implement
> it - would that be better?
> 
> -I notice we're up to kvm-79 vs my kvm-62. Should I move to the newer
> version? Do I
> have to custom-compile my kernel to do so, and if so what kernel version
> and what
> specific kernel options should I use?
> 
> -Are there other tuning steps I could take?
> 
> Please copy me directly as I'm not on this list. Thankyou
> 
> 
> 
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: KVM performance

[Avi Kivity]

Randy Broman wrote:
>
> -I've tried both the default Cirrus adapter and the "-std-vga" option. 
> Which is better?

Cirrus is generally better, but supports fewer resolutions.

> I saw reference to another VMware-based adapter, but I can't figure 
> out how to implement
> it - would that be better?
>

-vga vmware (with the new syntax needed by kvm-79); it should be better, 
but is less will tested.  I'm not at all sure the Windows driver will 
like it.

> -I notice we're up to kvm-79 vs my kvm-62. Should I move to the newer 
> version? 

Yes.

> Do I
> have to custom-compile my kernel to do so

No.

-- 
error compiling committee.c: too many arguments to function

Re: KVM performance

[Randy Broman]

After I submitted the initial question, I downloaded the latest kernel 
2.6.27.6, and compiled
with the following options, some of which are new since my previous 
kernel 2.6.24-21.

CONFIG_PARAVIRT_GUEST=y
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_VMI=y
CONFIG_KVM_CLOCK=y
CONFIG_KVM_GUEST=y
# CONFIG_LGUEST_GUEST is not set
CONFIG_PARAVIRT=y
CONFIG_PARAVIRT_CLOCK=y

Using my existing kvm-62 and the following invocation:

$ aoss kvm -m 1024 -cdrom /dev/cdrom -boot c -net 
nic,macaddr=00:d0:13:b0:2d:32,model=rtl8139 -net tap -soundhw all 
-localtime /home/rbroman/windows.img

CPU usage went down and performance was much better (no skips), for my 
video/audio feeds.

I then downloaded, compiled, installed kvm-79, and invoked using the 
following options

$ aoss ~/kvm-79/kvm -m 1024 --cdrom /dev/cdrom --mac=00:d0:13:b0:2d:32 
--nictype=rtl8139 --smp=2 /home/rbroman/windows.img

Note I'm using the new kvm in the compile directory, and I've confirmed 
that the kvm and kvm-intel
modules from the kvm-79 compile are what's loaded. Some of the options 
from the kvm-62 invocation
are missing - because they give errors - I understand that the command 
syntax/options have changed,
and I've checked ~/kvm-79/kvm --help for the new syntax, but I can't 
figure out how to invoke the
remaining options. One of the missing options seems to be the tap 
network, and the kvm-79 WinXP
guest now has no networking.

I also tried the "-vga vmware" option below, as well as -vga=vmware and 
various other permutations,
and I can't get that to work either.

Can someone help me resolve the above? Are there any README's, HowTo's 
or other documentation
on compiling, installing and using kvm-79?

Thanks, Randy

Avi Kivity wrote:
> Randy Broman wrote:
>>
>> -I've tried both the default Cirrus adapter and the "-std-vga" 
>> option. Which is better?
>
> Cirrus is generally better, but supports fewer resolutions.
>
>> I saw reference to another VMware-based adapter, but I can't figure 
>> out how to implement
>> it - would that be better?
>>
>
> -vga vmware (with the new syntax needed by kvm-79); it should be 
> better, but is less will tested.  I'm not at all sure the Windows 
> driver will like it.
>
>> -I notice we're up to kvm-79 vs my kvm-62. Should I move to the newer 
>> version? 
>
> Yes.
>
>> Do I
>> have to custom-compile my kernel to do so
>
> No.
>

Re: KVM performance

[Brian Jackson]

Don't use kvm in the tarball. It's not what you want. That's just a wrapper 
that calls qemu/kvm (possibly even the system one) after it mangles some 
command line options. Use qemu/x86_64-softmmu/qemu-system-x86_64 from the 
tarball if you aren't going to install it. Then you just use the same command 
line params as when you run the "kvm" that your distro installed.



On Sunday 16 November 2008 4:08:02 pm Randy Broman wrote:
> After I submitted the initial question, I downloaded the latest kernel
> 2.6.27.6, and compiled
> with the following options, some of which are new since my previous
> kernel 2.6.24-21.
>
> CONFIG_PARAVIRT_GUEST=y
> CONFIG_XEN_SAVE_RESTORE=y
> CONFIG_VMI=y
> CONFIG_KVM_CLOCK=y
> CONFIG_KVM_GUEST=y
> # CONFIG_LGUEST_GUEST is not set
> CONFIG_PARAVIRT=y
> CONFIG_PARAVIRT_CLOCK=y
>
> Using my existing kvm-62 and the following invocation:
>
> $ aoss kvm -m 1024 -cdrom /dev/cdrom -boot c -net
> nic,macaddr=00:d0:13:b0:2d:32,model=rtl8139 -net tap -soundhw all
> -localtime /home/rbroman/windows.img
>
> CPU usage went down and performance was much better (no skips), for my
> video/audio feeds.
>
> I then downloaded, compiled, installed kvm-79, and invoked using the
> following options
>
> $ aoss ~/kvm-79/kvm -m 1024 --cdrom /dev/cdrom --mac=00:d0:13:b0:2d:32
> --nictype=rtl8139 --smp=2 /home/rbroman/windows.img
>
> Note I'm using the new kvm in the compile directory, and I've confirmed
> that the kvm and kvm-intel
> modules from the kvm-79 compile are what's loaded. Some of the options
> from the kvm-62 invocation
> are missing - because they give errors - I understand that the command
> syntax/options have changed,
> and I've checked ~/kvm-79/kvm --help for the new syntax, but I can't
> figure out how to invoke the
> remaining options. One of the missing options seems to be the tap
> network, and the kvm-79 WinXP
> guest now has no networking.
>
> I also tried the "-vga vmware" option below, as well as -vga=vmware and
> various other permutations,
> and I can't get that to work either.
>
> Can someone help me resolve the above? Are there any README's, HowTo's
> or other documentation
> on compiling, installing and using kvm-79?
>
> Thanks, Randy
>
> Avi Kivity wrote:
> > Randy Broman wrote:
> >> -I've tried both the default Cirrus adapter and the "-std-vga"
> >> option. Which is better?
> >
> > Cirrus is generally better, but supports fewer resolutions.
> >
> >> I saw reference to another VMware-based adapter, but I can't figure
> >> out how to implement
> >> it - would that be better?
> >
> > -vga vmware (with the new syntax needed by kvm-79); it should be
> > better, but is less will tested.  I'm not at all sure the Windows
> > driver will like it.
> >
> >> -I notice we're up to kvm-79 vs my kvm-62. Should I move to the newer
> >> version?
> >
> > Yes.
> >
> >> Do I
> >> have to custom-compile my kernel to do so
> >
> > No.
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: KVM performance

[Avi Kivity]

Randy Broman wrote:
> After I submitted the initial question, I downloaded the latest kernel 
> 2.6.27.6, and compiled
> with the following options, some of which are new since my previous 
> kernel 2.6.24-21.
>
> CONFIG_PARAVIRT_GUEST=y
> CONFIG_XEN_SAVE_RESTORE=y
> CONFIG_VMI=y
> CONFIG_KVM_CLOCK=y
> CONFIG_KVM_GUEST=y
> # CONFIG_LGUEST_GUEST is not set
> CONFIG_PARAVIRT=y
> CONFIG_PARAVIRT_CLOCK=y
>
> Using my existing kvm-62 and the following invocation:
>
> $ aoss kvm -m 1024 -cdrom /dev/cdrom -boot c -net 
> nic,macaddr=00:d0:13:b0:2d:32,model=rtl8139 -net tap -soundhw all 
> -localtime /home/rbroman/windows.img

> and I've checked ~/kvm-79/kvm --help for the new syntax, but I can't 
> figure out how to invoke the
> remaining options. One of the missing options seems to be the tap 
> network, and the kvm-79 WinXP
> guest now has no networking.

These options have not changed.  Do you get any error messages?

Oh, and don't use the kvm python script,  I'll remove it from the 
repository.

>
> I also tried the "-vga vmware" option below, as well as -vga=vmware 
> and various other permutations,
> and I can't get that to work either.
>

What error message do you get?


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

KVM performance

[BRAUN, Stefanie]

Hallo,

as I want to switch from XEN to KVM I've made some performance tests
to see if KVM is as peformant as XEN. But tests with a VMU that receives
a streamed video, adds a small logo to the video and streams it to a
client
have shown that XEN performs much betten than KVM. 
In XEN the vlc (videolan client used to receive, process and send the
video) process 
within the vmu has a cpuload of 33,8 % whereas in KVM
the vlc process has a cpuload of 99.9 %.
I'am not sure why, does anybody now some settings to improve
the KVM performance?

Thank you.
Regards, Stefanie.


Used hardware and settings:
In the tests I've used the same host hardware for XEN and KVM:
- Dual Core AMD 2.2 GHz, 8 GB RAM
- Tested OSes for KVM Host: Fedora 10, 2.6.27.5-117.fc10.x86_64 with kvm
version 10.fc10 version 74
                            also tested in january: compiled kernel with
kvm-83

- KVM Guest settings: OS: Fedora 9 2.6.25-14.fc9.x86_64 (i386 also
tested)
                      RAM: 256 MB (same for XEN vmu)
                      CPU: 1 Core with 2,2 GHz (same for XEN vmu)
			    tested nic models: rtl8139, e1000, virtio

Tested Scenario: VMU receives a streamed video , adds a logo (watermark)
to the video stream and then streams it to a client

Results:
	
XEN:
Host cpu load (virt-manager): 		23%
VMU  cpu load (virt-manager):			18 %	
VLC process within VMU (top):			33,8%			

KVM:
no virt-manager cpu load as I started the vmu with the kvm command
Host cpu load :					52%
qemu-kvm process (top)				77-100%
VLC process within vmu (top):			80 - 99,9%

KVM command to start vmu
/usr/bin/qemu-kvm -boot c -hda /images/vmu01.raw -m 256 -net
nic,vlan=0,macaddr=aa:bb:cc:dd:ee:10,model=virtio -net
tap,ifname=tap0,vlan=0,script=/etc/kvm/qemu-ifup,downscript=/etc/kvm/qem
u-ifdown -vnc 127.0.0.1:1 -k de --daemonize





________________________________

Alcatel-Lucent Deutschland AG
Bell Labs Germany
Service Infrastructure, ZFZ-SI
Stefanie Braun
Phone:   +49.711.821-34865
Fax:     +49.711.821-32453

Postal address:
Alcatel-Lucent Deutschland AG
Lorenzstrasse 10
D-70435 STUTTGART

Mail: stefanie.braun@alcatel-lucent.de

 

Alcatel-Lucent Deutschland AG
Sitz der Gesellschaft: Stuttgart - Amtsgericht Stuttgart HRB 4026
Vorsitzender des Aufsichtsrats: Michael Oppenhoff Vorstand: Alf Henryk
Wulf (Vors.), Dr. Rainer Fechner

________________________________

Re: KVM performance

[Avi Kivity]

BRAUN, Stefanie wrote:
> Hallo,
>
> as I want to switch from XEN to KVM I've made some performance tests
> to see if KVM is as peformant as XEN. But tests with a VMU that receives
> a streamed video, adds a small logo to the video and streams it to a
> client
> have shown that XEN performs much betten than KVM. 
> In XEN the vlc (videolan client used to receive, process and send the
> video) process 
> within the vmu has a cpuload of 33,8 % whereas in KVM
> the vlc process has a cpuload of 99.9 %.
> I'am not sure why, does anybody now some settings to improve
> the KVM performance?
>   

Is this a tcp test?

Can you test receive and transmit separately?

-- 
error compiling committee.c: too many arguments to function

Re: KVM performance

[Hauke Hoffmann]

On Friday 03 April 2009 13:32:50 you wrote:
> Hallo,
>
> as I want to switch from XEN to KVM I've made some performance tests
> to see if KVM is as peformant as XEN. But tests with a VMU that receives
> a streamed video, adds a small logo to the video and streams it to a
> client
> have shown that XEN performs much betten than KVM.
> In XEN the vlc (videolan client used to receive, process and send the
> video) process
> within the vmu has a cpuload of 33,8 % whereas in KVM
> the vlc process has a cpuload of 99.9 %.
> I'am not sure why, does anybody now some settings to improve
> the KVM performance?
>
> Thank you.
> Regards, Stefanie.
>
>
> Used hardware and settings:
> In the tests I've used the same host hardware for XEN and KVM:
> - Dual Core AMD 2.2 GHz, 8 GB RAM
> - Tested OSes for KVM Host: Fedora 10, 2.6.27.5-117.fc10.x86_64 with kvm
> version 10.fc10 version 74
>                             also tested in january: compiled kernel with
> kvm-83
>
> - KVM Guest settings: OS: Fedora 9 2.6.25-14.fc9.x86_64 (i386 also
> tested)
>                       RAM: 256 MB (same for XEN vmu)
>                       CPU: 1 Core with 2,2 GHz (same for XEN vmu)
> 			    tested nic models: rtl8139, e1000, virtio
>
> Tested Scenario: VMU receives a streamed video , adds a logo (watermark)
> to the video stream and then streams it to a client
>
> Results:
>
> XEN:
> Host cpu load (virt-manager): 		23%
> VMU  cpu load (virt-manager):			18 %
> VLC process within VMU (top):			33,8%
>
> KVM:
> no virt-manager cpu load as I started the vmu with the kvm command
> Host cpu load :					52%
> qemu-kvm process (top)				77-100%
> VLC process within vmu (top):			80 - 99,9%
>
> KVM command to start vmu
> /usr/bin/qemu-kvm -boot c -hda /images/vmu01.raw -m 256 -net
> nic,vlan=0,macaddr=aa:bb:cc:dd:ee:10,model=virtio -net
> tap,ifname=tap0,vlan=0,script=/etc/kvm/qemu-ifup,downscript=/etc/kvm/qem
> u-ifdown -vnc 127.0.0.1:1 -k de --daemonize

Hi Stefanie,

does vlc perform operations on disc (eg caching, logging, ...)? 

When it cache you can use virtio also for the disk. 
Just change
	-hda /images/vmu01.raw
to
	-drive file=/images/vmu01.raw,if=virtio,boot=on

Regards
Hauke


>
>
>
>
>
> ________________________________
>
> Alcatel-Lucent Deutschland AG
> Bell Labs Germany
> Service Infrastructure, ZFZ-SI
> Stefanie Braun
> Phone:   +49.711.821-34865
> Fax:     +49.711.821-32453
>
> Postal address:
> Alcatel-Lucent Deutschland AG
> Lorenzstrasse 10
> D-70435 STUTTGART
>
> Mail: stefanie.braun@alcatel-lucent.de
>
>
>
> Alcatel-Lucent Deutschland AG
> Sitz der Gesellschaft: Stuttgart - Amtsgericht Stuttgart HRB 4026
> Vorsitzender des Aufsichtsrats: Michael Oppenhoff Vorstand: Alf Henryk
> Wulf (Vors.), Dr. Rainer Fechner
>
> ________________________________
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 

hauke hoffmann service and electronic systems

Moristeig 60, D-23556 Lübeck

Telefon: +49 (0) 451 8896462
Fax: +49 (0) 451 8896461
Mobil: +49 (0) 170 7580491
E-Mail: office@hauke-hoffmann.net
PGP public key: www.hauke-hoffmann.net/static/pgp/kontakt.asc

KVM performance

[Tim Chen]

Hi,

I did some testing of KVM on my woodcrest machine.  And I found that
building a 2.6.19 kernel with identical configuration takes 845 sec on a
guest and 210 sec on the host.  So the compile is about 4X slower on the
guest :(

The guest ran with 2.6.19 kernel and host ran with a 2.6.20-rc5 kernel.
The guest file image is created with raw format.  Wonder if people see
similar performance figure?  

Tim

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Fabian Deutsch]

Hey,

> Hi,
> 
> I did some testing of KVM on my woodcrest machine.  And I found that
> building a 2.6.19 kernel with identical configuration takes 845 sec on a
> guest and 210 sec on the host.  So the compile is about 4X slower on the
> guest :(

I wasn't able to compile some vanilla kernels, but i did two loops on
the host and guest.

First, with nearly no hd read/writes.
# time (for I in $(seq 1 100000) ; do echo $I > /dev/null ; done)
host:	3.7s	100%
guest:	4.1s	110%

And a second one with hd read/writes (might have a look at strace of
date, tells you that date open/closes a lot of files):
# time (for I in $(seq 1 1000) ; do date > /dev/null ; done)
host:	1s	100%
guest:	14s	1400%

i don't really know how much those tests say, but:
It seems as if the hd access is the bottleneck :)

fabian


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Avi Kivity]

Tim Chen wrote:
> Hi,
>
> I did some testing of KVM on my woodcrest machine.  And I found that
> building a 2.6.19 kernel with identical configuration takes 845 sec on a
> guest and 210 sec on the host.  So the compile is about 4X slower on the
> guest :(
>
> The guest ran with 2.6.19 kernel and host ran with a 2.6.20-rc5 kernel.
> The guest file image is created with raw format.  Wonder if people see
> similar performance figure?  
>
>   


http://virt.kernelnewbies.org/KVM/Performance cites a 2.5X difference, 
but for a different cpu.

It's probably better to use an lvm volume rather than a raw file, and to 
give the guest ample memory.  In addition, quite a few performance 
optimizations are missing from kvm:

- after modifying a pte, kvm doesn't preload the modified pte into 
shadow, but instead lets the guest fault it in
- disk access is blocking instead of non-blocking.  this will be fixed 
by merging qemu-cvs, which uses aio for disk access.
- better heuristics for recycling page tables are needed
- prefaulting for common access patterns can help
- kvm currently saves the entire fpu state on every exit, even if it has 
not been modified


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Rusty Russell]

On Sat, 2007-01-27 at 10:34 +0200, Avi Kivity wrote:
> In addition, quite a few performance optimizations are missing from kvm:

Hi Avi!

	Just thought I'd share my experience with some of these optimizations
in lguest.  I use virtbench (http://ozlabs.org/~rusty/virtbench) to try
to measure optimization results; it still needs more tests (and an
explicit kvm backend).

> - after modifying a pte, kvm doesn't preload the modified pte into 
> shadow, but instead lets the guest fault it in

lguest doesn't either, but don't you still want the fault to update the
host accessed bit?

> - disk access is blocking instead of non-blocking.  this will be fixed 
> by merging qemu-cvs, which uses aio for disk access.

Do you plan on multiple guest I/Os outstanding, too?  I would think that
I/O scheduling in the guest could be more effective than I/O scheduling
in the host if there is more than one guest sharing a host device.  It
could potentially reduce guest<->host transitions too.

> - better heuristics for recycling page tables are needed

Definitely a whole area of research here...

> - prefaulting for common access patterns can help

Hmm, interesting idea.  Any specific thoughts?

> - kvm currently saves the entire fpu state on every exit, even if it has 
> not been modified

This was measurable for lguest, but our transition is slow so your %
improvement might be greater.  With intelligent TS (2.13GHz Core Duo2):

 Time for one context switch via pipe: 53514 nsec
 Time for one Copy-on-Write fault: 14841 nsec
 Time to exec client once: 1155102 nsec
 Time for one fork/exit/wait: 764490 nsec
 Time for two PTE updates: 23800 nsec

Removing it and restoring FPU every time:

 Time for one context switch via pipe: 56229 nsec
 Time for one Copy-on-Write fault: 15989 nsec
 Time to exec client once: 1243624 nsec
 Time for one fork/exit/wait: 824647 nsec
 Time for two PTE updates: 25056 nsec

Cheers!
Rusty.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Avi Kivity]

Rusty Russell wrote:
> On Sat, 2007-01-27 at 10:34 +0200, Avi Kivity wrote:
>   
>> In addition, quite a few performance optimizations are missing from kvm:
>>     
>
> Hi Avi!
>
> 	Just thought I'd share my experience with some of these optimizations
> in lguest.  I use virtbench (http://ozlabs.org/~rusty/virtbench) to try
> to measure optimization results; it still needs more tests (and an
> explicit kvm backend).
>   

A kvm backend would be appreciated.

>   
>> - after modifying a pte, kvm doesn't preload the modified pte into 
>> shadow, but instead lets the guest fault it in
>>     
>
> lguest doesn't either, but don't you still want the fault to update the
> host accessed bit?
>   

The story here is that the guest is handling a pagefault and writing the 
new guest pte.  kvm traps the write (guest pagetables are write 
protected), and has the option of updating the shadow pte to reflect the 
guest pte.

The clever guest kernel will set the accessed bit (and the dirty bit on 
writable ptes) to avoid an rmw cycle by the hardware pagetable walker.

[two instrumented runs later]

Both Linux and Windows seem to do this optimization.

>   
>> - disk access is blocking instead of non-blocking.  this will be fixed 
>> by merging qemu-cvs, which uses aio for disk access.
>>     
>
> Do you plan on multiple guest I/Os outstanding, too?  I would think that
> I/O scheduling in the guest could be more effective than I/O scheduling
> in the host if there is more than one guest sharing a host device.  It
> could potentially reduce guest<->host transitions too.
>   

qemu-cvs has scsi emulation IIRC, which allows multiple outstanding 
requests.  But the big improvement comes from allowing even one 
outstanding request (current kvm will block the cpu as soon as a disk 
request is issued).


>> - prefaulting for common access patterns can help
>>     
>
> Hmm, interesting idea.  Any specific thoughts?
>
>   

Linear :)

Also, if we see a lot of host faults in a pagetable, but no guest 
faults, we can assume that we're rebuilding a recycled shadow page, and 
fault the entire page in at once.

>> - kvm currently saves the entire fpu state on every exit, even if it has 
>> not been modified
>>     
>
> This was measurable for lguest, but our transition is slow so your %
> improvement might be greater.  With intelligent TS (2.13GHz Core Duo2):
>
>  Time for one context switch via pipe: 53514 nsec
>  Time for one Copy-on-Write fault: 14841 nsec
>  Time to exec client once: 1155102 nsec
>  Time for one fork/exit/wait: 764490 nsec
>  Time for two PTE updates: 23800 nsec
>
> Removing it and restoring FPU every time:
>
>  Time for one context switch via pipe: 56229 nsec
>  Time for one Copy-on-Write fault: 15989 nsec
>  Time to exec client once: 1243624 nsec
>  Time for one fork/exit/wait: 824647 nsec
>  Time for two PTE updates: 25056 nsec
>   

Good, I hope it's worthwhile for kvm too.



-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Rusty Russell]

On Sun, 2007-01-28 at 11:40 +0200, Avi Kivity wrote:
> Rusty Russell wrote:
>  I use virtbench (http://ozlabs.org/~rusty/virtbench) to try
> > to measure optimization results; it still needs more tests (and an
> > explicit kvm backend).   
> 
> A kvm backend would be appreciated.

Yes, and patches are most welcome 8)  Actually, I'll work on this week.

> >> - after modifying a pte, kvm doesn't preload the modified pte into 
> >> shadow, but instead lets the guest fault it in  
> > lguest doesn't either, but don't you still want the fault to update the
> > host accessed bit?
> 
> The story here is that the guest is handling a pagefault and writing the 
> new guest pte.  kvm traps the write (guest pagetables are write 
> protected), and has the option of updating the shadow pte to reflect the 
> guest pte.
> 
> The clever guest kernel will set the accessed bit (and the dirty bit on 
> writable ptes) to avoid an rmw cycle by the hardware pagetable walker.
> 
> [two instrumented runs later]
> 
> Both Linux and Windows seem to do this optimization.

Right.  This (trivial!) optimization wins lguest a good 10%:

Before:
    Time for one Copy-on-Write fault: 13622 nsec
    Time to exec client once: 1085481 nsec
    Time for one fork/exit/wait: 700796 nsec
After:
    Time for one Copy-on-Write fault: 12036 nsec
    Time to exec client once: 969899 nsec
    Time for one fork/exit/wait: 664601 nsec

Thanks!
Rusty.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Avi Kivity]

Rusty Russell wrote:
>> The clever guest kernel will set the accessed bit (and the dirty bit on 
>> writable ptes) to avoid an rmw cycle by the hardware pagetable walker.
>>
>> [two instrumented runs later]
>>
>> Both Linux and Windows seem to do this optimization.
>>     
>
> Right.  This (trivial!) optimization wins lguest a good 10%:
>
> Before:
>     Time for one Copy-on-Write fault: 13622 nsec
>     Time to exec client once: 1085481 nsec
>     Time for one fork/exit/wait: 700796 nsec
> After:
>     Time for one Copy-on-Write fault: 12036 nsec
>     Time to exec client once: 969899 nsec
>     Time for one fork/exit/wait: 664601 nsec
>
>   

Another optimization is that we can stop having two parallel 
implementations of the same code.  I suggest we start thinking how to merge.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Rusty Russell]

On Tue, 2007-01-30 at 14:56 +0200, Avi Kivity wrote:
> Another optimization is that we can stop having two parallel 
> implementations of the same code.  I suggest we start thinking how to merge.

KVM has more ambition than lguest, so I was planning on letting you do
all the hard work then rebase my code on top once the mmu branch hits
mainline 8)  I've grabbed the release SVN tree and will look through it
to see how it'll fit...

(lguest's page_table.c is only 450 lines, BUT I doubt it would stay that
way if I implemented a decent one 8)

Cheers,
Rusty.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: KVM performance

[Anthony Liguori]

[-- Attachment #1: Type: text/plain, Size: 2467 bytes --]

Rusty Russell wrote:
> On Sun, 2007-01-28 at 11:40 +0200, Avi Kivity wrote:
>   
>> Rusty Russell wrote:
>>  I use virtbench (http://ozlabs.org/~rusty/virtbench) to try
>>     
>>> to measure optimization results; it still needs more tests (and an
>>> explicit kvm backend).   
>>>       
>> A kvm backend would be appreciated.
>>     
>  
> Yes, and patches are most welcome 8)  Actually, I'll work on this week.

Attached patch was my initial attempt.  I wanted it to work with mostly 
unmodified guest images so the only requirement is that a getty is 
spawned on ttyS0.  I ran in to quite a few problems with virtbench 
though once I started launching multiple guests and haven't gotten 
around to debugging those yet.

Regards,

Anthony Liguori

>  
>   
>>>> - after modifying a pte, kvm doesn't preload the modified pte into 
>>>> shadow, but instead lets the guest fault it in  
>>>>         
>>> lguest doesn't either, but don't you still want the fault to update the
>>> host accessed bit?
>>>       
>> The story here is that the guest is handling a pagefault and writing the 
>> new guest pte.  kvm traps the write (guest pagetables are write 
>> protected), and has the option of updating the shadow pte to reflect the 
>> guest pte.
>>
>> The clever guest kernel will set the accessed bit (and the dirty bit on 
>> writable ptes) to avoid an rmw cycle by the hardware pagetable walker.
>>
>> [two instrumented runs later]
>>
>> Both Linux and Windows seem to do this optimization.
>>     
>  
> Right.  This (trivial!) optimization wins lguest a good 10%:
>  
> Before:
>     Time for one Copy-on-Write fault: 13622 nsec
>     Time to exec client once: 1085481 nsec
>     Time for one fork/exit/wait: 700796 nsec
> After:
>     Time for one Copy-on-Write fault: 12036 nsec
>     Time to exec client once: 969899 nsec
>     Time for one fork/exit/wait: 664601 nsec
>  
> Thanks!
> Rusty.
>  
>  
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> kvm-devel mailing list
> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>   


[-- Attachment #2: virtbench-kvm.diff --]
[-- Type: text/x-patch, Size: 3265 bytes --]

diff -r 03813abef33e kvm/SETTINGS
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/kvm/SETTINGS	Mon Jan 15 15:50:09 2007 -0600
@@ -0,0 +1,3 @@
+# Sources from all scripts
+ROOT_FILE="/mnt/FC-5-i386.img"
+QEMU=qemu
diff -r 03813abef33e kvm/serial.py
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/kvm/serial.py	Mon Jan 15 16:00:13 2007 -0600
@@ -0,0 +1,58 @@
+import sys, socket, binascii, commands
+
+if len(sys.argv) != 9:
+    print 'Usage: serial IP PORT LOGIN PASSWORD PATH ID IP PORT'
+    sys.exit(1)
+
+sys.stderr.write('foo\n')
+
+login='login: '
+password='Password: '
+prompt=']# '
+
+s = socket.socket(socket.AF_INET)
+s.connect((sys.argv[1], int(sys.argv[2])))
+
+def wait_for(string):
+    buf = ''
+    while True:
+        f = s.recv(4096)
+        buf += f
+        if buf.endswith(string):
+            break
+
+s.sendall('\n')
+wait_for(login)
+s.sendall('%s\n' % sys.argv[3])
+
+wait_for(password)
+s.sendall('%s\n' % sys.argv[4])
+
+wait_for(prompt)
+status, output = commands.getstatusoutput('uuencode %s /tmp/virtclient' %
+                                          sys.argv[5])
+
+f = open(sys.argv[5])
+d = binascii.b2a_base64(f.read())
+f.close()
+
+s.sendall("stty -echo\n")
+wait_for(prompt)
+
+s.sendall("cat > /tmp/virtclient.b64 <<EOF\n")
+s.sendall(d)
+s.sendall("\nEOF\n")
+wait_for(prompt)
+
+s.sendall("""python
+import sys, binascii
+i = open('/tmp/virtclient.b64', 'r')
+f = open('/tmp/virtclient', 'w')
+f.write(binascii.a2b_base64(i.read()))
+sys.exit(0)
+""")
+wait_for(prompt)
+s.sendall('chmod 755 /tmp/virtclient\n')
+wait_for(prompt)
+s.sendall('/tmp/virtclient %s %s %s\n' % (sys.argv[6], sys.argv[7], sys.argv[8]))
+wait_for(prompt)
diff -r 03813abef33e kvm/start
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/kvm/start	Mon Jan 15 11:56:18 2007 -0600
@@ -0,0 +1,13 @@
+#! /bin/sh
+
+set -e
+
+. kvm/SETTINGS
+
+#if grep AuthenticAMD 2>/dev/null ; then
+#    modules="kvm kvm-amd"
+#else
+#    modules="kvm kvm-intel"
+#fi
+
+#[ -c /dev/kvm ] || modprobe $modules
diff -r 03813abef33e kvm/start_machine
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/kvm/start_machine	Sat Jan 27 11:09:40 2007 -0600
@@ -0,0 +1,10 @@
+#! /bin/sh
+
+. kvm/SETTINGS
+
+serial_port=$((1025 + $1))
+
+$QEMU -m 128 -hda "${ROOT_FILE}" -snapshot -serial tcp:localhost:${serial_port},server,nowait -kernel-kqemu &
+sleep 2
+python kvm/serial.py 127.0.0.1 ${serial_port} root ibm4xen virtclient $1 $2 $3 &
+jobs -p %1
diff -r 03813abef33e kvm/stop
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/kvm/stop	Mon Jan 15 11:32:55 2007 -0600
@@ -0,0 +1,4 @@
+#! /bin/sh
+
+. kvm/SETTINGS
+exit 0
diff -r 03813abef33e kvm/stop_machine
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/kvm/stop_machine	Mon Jan 15 11:43:29 2007 -0600
@@ -0,0 +1,5 @@
+#! /bin/sh
+
+. kvm/SETTINGS
+
+kill $1
diff -r 03813abef33e server.c
--- a/server.c	Thu Jan 11 13:56:30 2007 +1100
+++ b/server.c	Mon Jan 15 16:02:11 2007 -0600
@@ -227,7 +227,7 @@ static struct sockaddr_in get_server_add
 	socklen_t socklen = sizeof(saddr);
 
 	/* This assumes we have an eth0. */
-	strcpy(ifr.ifr_name, "eth0");
+	strcpy(ifr.ifr_name, "ath0");
 	sin->sin_family = AF_INET;
 	if (ioctl(sock, SIOCGIFADDR, &ifr) != 0)
 		err(1, "Getting interface address for eth0");

[-- Attachment #3: Type: text/plain, Size: 347 bytes --]

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[-- Attachment #4: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel