[ANNOUNCE] kvm userspace release 6

[ANNOUNCE] kvm userspace release 6

[Avi Kivity]

http://kvm.sourceforge.net

Changes:
- AMD 32-bit host support (Anthony Liguori)
- more migration work
- stabilization
- improved performance
- improved guest support on AMD

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Michael Riepe]

[-- Attachment #1: Type: text/plain, Size: 1867 bytes --]

Hi!

I immediately tried the latest version (SVN release 4064) on my Thinkpad
(Core Duo T2400, Linux 2.6.19) but I still have the same problems I had
with release 5.

Suse 10.1 (the German version) dies with:

> exception 13 (0)
> Aborted

no matter if I boot from DVD or disk.

OpenSolaris XXL gets hit even worse:

> unhandled vm exit:  0x80000021
> rax 0000000060000010 rbx 0000000000002000 rcx 000000000102808c rdx 0000000000101e50
> rsi 0000000000004f15 rdi 0000000000000000 rsp 0000000000004ffc rbp 0000000001028064
> r8  00000000010113c8 r9  00000000000007ff r10 00000000e0000011 r11 b7f04674b7f03ff4
> r12 bfa68a80b7f044c8 r13 b7f04674b7efae3c r14 00000001b79a03f0 r15 0000000000000001
> rip 00000000000021ab rflags 00223002
> Aborted

This one obviously points to an error on VM entry, according to the
Intel docs. Maybe the Solaris bootloader is messing with registers it
shouldn't touch?

Windows XP Professional works, but only if I turn off ACPI (-no-acpi).
The only thing that fails is poweroff/reboot (well, no ACPI, no way to
turn off the power, right?) but you can do that manually. If I turn ACPI
on, I get a bluescreen complaining about "IRQL_NOT_LESS_OR_EQUAL",
whatever that means.

On the other hand, some systems worked fine out of the box. So far I
managed to boot and use Knoppix 5.0 and the latest Qemu-Puppy (2.01-3).

All systems I tested work with -no-kvm (or the regular qemu 0.8.2).

The patch I attached is not essential. It fixes two error messages that
occur on a 32-bit host (where the offending registers don't exist).

By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages
in my syslog, but they don't seem to cause any further harm.

Please let me know if (and how, preferably) I can help debugging.

-- 
Michael "Tired" Riepe <michael-0QoEqw4nQxo@public.gmane.org>
X-Tired: Each morning I get up I die a little

[-- Attachment #2: kvm-4064.diff --]
[-- Type: text/plain, Size: 511 bytes --]

Index: kernel/vmx.c
===================================================================
--- kernel/vmx.c	(revision 4064)
+++ kernel/vmx.c	(working copy)
@@ -1175,8 +1175,10 @@
                                VM_ENTRY_CONTROLS, 0);
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0);  /* 22.2.1 */
 
+#ifdef CONFIG_X86_64
 	vmcs_writel(VIRTUAL_APIC_PAGE_ADDR, 0);
 	vmcs_writel(TPR_THRESHOLD, 0);
+#endif
 
 	vmcs_writel(CR0_GUEST_HOST_MASK, KVM_GUEST_CR0_MASK);
 	vmcs_writel(CR4_GUEST_HOST_MASK, KVM_GUEST_CR4_MASK);

[-- Attachment #3: Type: text/plain, Size: 347 bytes --]

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[-- Attachment #4: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [ANNOUNCE] kvm userspace release 6

[Dor Laor]

> 
> Hi!
> 
> I immediately tried the latest version (SVN release 4064) on my
Thinkpad
> (Core Duo T2400, Linux 2.6.19) but I still have the same problems I
had
> with release 5.
> 
> Suse 10.1 (the German version) dies with:
> 
> > exception 13 (0)
> > Aborted
> 
> no matter if I boot from DVD or disk.

Probably some real-mode emulation problem. I'm not absolutely sure, will
take a look tomorrow when I'll see the code.
There are real-modes instructions that we might fail to emulate
correctly. A temporal workaround is to try switch to textual boot mode,
it might help till we fix it.
Please tell if it worked for you.

> 
> OpenSolaris XXL gets hit even worse:
> 
> > unhandled vm exit:  0x80000021
> > rax 0000000060000010 rbx 0000000000002000 rcx 000000000102808c rdx
> 0000000000101e50
> > rsi 0000000000004f15 rdi 0000000000000000 rsp 0000000000004ffc rbp
> 0000000001028064
> > r8  00000000010113c8 r9  00000000000007ff r10 00000000e0000011 r11
> b7f04674b7f03ff4
> > r12 bfa68a80b7f044c8 r13 b7f04674b7efae3c r14 00000001b79a03f0 r15
> 0000000000000001
> > rip 00000000000021ab rflags 00223002
> > Aborted
> 
> This one obviously points to an error on VM entry, according to the
> Intel docs. Maybe the Solaris bootloader is messing with registers it
> shouldn't touch?

Right about this caused by problem in VM entry.
99% chance Solaris is not the blame (1% they are using the
BIG-real-mode).
Might be also a graphical boot loader problem, there is a debug.c file
that I need to revive that tells what bothered Intel.
In general, Intel tests to deeply before vmentery.

> 
> Windows XP Professional works, but only if I turn off ACPI (-no-acpi).
> The only thing that fails is poweroff/reboot (well, no ACPI, no way to
> turn off the power, right?) but you can do that manually. If I turn
ACPI
> on, I get a bluescreen complaining about "IRQL_NOT_LESS_OR_EQUAL",
> whatever that means.
> 

KVM has an issue with APIC. Currently the APIC is implemented within
Qemu.
I will send a patch that moves the APIC into the kernel that fixes the
problem you had above by the end of the week *so stay tuned*.
There is a simple workaround detailed in the KVM faq.

> On the other hand, some systems worked fine out of the box. So far I
> managed to boot and use Knoppix 5.0 and the latest Qemu-Puppy
(2.01-3).
> 
> All systems I tested work with -no-kvm (or the regular qemu 0.8.2).
> 
> The patch I attached is not essential. It fixes two error messages
that
> occur on a 32-bit host (where the offending registers don't exist).

Thanks, will be very soon embedded.

> 
> By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages
> in my syslog, but they don't seem to cause any further harm.
> 
> Please let me know if (and how, preferably) I can help debugging.
> 
> --
> Michael "Tired" Riepe <michael-0QoEqw4nQxo@public.gmane.org>
> X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Michael Riepe]

Hi!

Dor Laor wrote:
>>Hi!
>>
>>I immediately tried the latest version (SVN release 4064) on my
> 
> Thinkpad
> 
>>(Core Duo T2400, Linux 2.6.19) but I still have the same problems I
> 
> had
> 
>>with release 5.
>>
>>Suse 10.1 (the German version) dies with:
>>
>>
>>>exception 13 (0)
>>>Aborted
>>
>>no matter if I boot from DVD or disk.
> 
> 
> Probably some real-mode emulation problem. I'm not absolutely sure, will
> take a look tomorrow when I'll see the code.

I suspect that it's caused by a segmentation limit violation.

> There are real-modes instructions that we might fail to emulate
> correctly. A temporal workaround is to try switch to textual boot mode,
> it might help till we fix it.
> Please tell if it worked for you.

I'm not sure what you mean. The -nographic option doesn't help, and Suse
doesn't even come far enough to switch to graphics mode. It dies in the
bootloader.

>>OpenSolaris XXL gets hit even worse:
>>
>>
>>>unhandled vm exit:  0x80000021
>>>rax 0000000060000010 rbx 0000000000002000 rcx 000000000102808c rdx
>>
>>0000000000101e50
>>
>>>rsi 0000000000004f15 rdi 0000000000000000 rsp 0000000000004ffc rbp
>>
>>0000000001028064
>>
>>>r8  00000000010113c8 r9  00000000000007ff r10 00000000e0000011 r11
>>
>>b7f04674b7f03ff4
>>
>>>r12 bfa68a80b7f044c8 r13 b7f04674b7efae3c r14 00000001b79a03f0 r15
>>
>>0000000000000001
>>
>>>rip 00000000000021ab rflags 00223002
>>>Aborted
>>
>>This one obviously points to an error on VM entry, according to the
>>Intel docs. Maybe the Solaris bootloader is messing with registers it
>>shouldn't touch?
> 
> 
> Right about this caused by problem in VM entry.
> 99% chance Solaris is not the blame (1% they are using the
> BIG-real-mode).
> Might be also a graphical boot loader problem, there is a debug.c file
> that I need to revive that tells what bothered Intel.
> In general, Intel tests to deeply before vmentery.

This also happens while loading the kernel, in text mode.

Did I mention that kvm dies a horrible death ("Aiee, killing interrupt
handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently
I use CONFIG_PREEMPT_NONE.

-- 
Michael "Tired" Riepe <michael-0QoEqw4nQxo@public.gmane.org>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Avi Kivity]

Michael Riepe wrote:
> Did I mention that kvm dies a horrible death ("Aiee, killing interrupt
> handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently
> I use CONFIG_PREEMPT_NONE.
>   
I'll investigate this.  I thought we had this fixed.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Michael Riepe]

Avi Kivity wrote:
> Michael Riepe wrote:
> 
>> Did I mention that kvm dies a horrible death ("Aiee, killing interrupt
>> handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently
>> I use CONFIG_PREEMPT_NONE.
>>   
> 
> I'll investigate this.  I thought we had this fixed.

Maybe the problem was CONFIG_PREEMPT_BKL (which was set as well). Got to
check that again.

-- 
Michael "Tired" Riepe <michael-0QoEqw4nQxo@public.gmane.org>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Avi Kivity]

Michael Riepe wrote:
> Avi Kivity wrote:
>   
>> Michael Riepe wrote:
>>
>>     
>>> Did I mention that kvm dies a horrible death ("Aiee, killing interrupt
>>> handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently
>>> I use CONFIG_PREEMPT_NONE.
>>>   
>>>       
>> I'll investigate this.  I thought we had this fixed.
>>     
>
> Maybe the problem was CONFIG_PREEMPT_BKL (which was set as well). Got to
> check that again.
>
>   

If you were running from the trunk, then that would explain this. The 
trunk did not have the patch until a short while ago.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Gerd Hoffmann]

  Hi,

> I'm not sure what you mean. The -nographic option doesn't help, and Suse
> doesn't even come far enough to switch to graphics mode. It dies in the
> bootloader.

both isolinux and grub fail to switch into gfx mode, which likely is
caused by either incomplete real mode emulation or big real mode.

Workaround for grub: just turn off the eye candy, comment the gfxmenu
line in /boot/grub/menu.lst.

Kicking the installation is a bit harder as you can't just go and edit
the isolinux.cfg on the cdrom.  One way is to use normal qemu (without
kvm acceleration) for installation and fixing up menu.lst, then go ahead
with the kvm version ...

reproduce: fetch the boot iso[1], boot with "-cdrom /path/to/mini.iso
-boot d"

cheers,
  Gerd

[1]
http://ftp.opensuse.org/pub/opensuse/distribution/10.2/iso/cd/openSUSE-10.2-GM-i386-mini.iso

-- 
Gerd Hoffmann <kraxel-l3A5Bk7waGM@public.gmane.org>

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Michael Riepe]

Hi!

Gerd Hoffmann wrote:

> both isolinux and grub fail to switch into gfx mode, which likely is
> caused by either incomplete real mode emulation or big real mode.
> 
> Workaround for grub: just turn off the eye candy, comment the gfxmenu
> line in /boot/grub/menu.lst.

Yep, that did the trick :-)

> Kicking the installation is a bit harder as you can't just go and edit
> the isolinux.cfg on the cdrom.  One way is to use normal qemu (without
> kvm acceleration) for installation and fixing up menu.lst, then go ahead
> with the kvm version ...

That's how I got the system installed.

-- 
Michael "Tired" Riepe <michael-0QoEqw4nQxo@public.gmane.org>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Avi Kivity]

Dor Laor wrote:
>   
>> Windows XP Professional works, but only if I turn off ACPI (-no-acpi).
>> The only thing that fails is poweroff/reboot (well, no ACPI, no way to
>> turn off the power, right?) but you can do that manually. If I turn
>>     
> ACPI
>   
>> on, I get a bluescreen complaining about "IRQL_NOT_LESS_OR_EQUAL",
>> whatever that means.
>>
>>     
>
> KVM has an issue with APIC. Currently the APIC is implemented within
> Qemu.
> I will send a patch that moves the APIC into the kernel that fixes the
> problem you had above by the end of the week *so stay tuned*.
> There is a simple workaround detailed in the KVM faq.
>
>   

The -no-apic option seems to be an even easier workaround.  I'll update 
the faq.

It also explains why some of my guests don't have a problem -- they were 
installed with a pre-acpi qemu, hence they don't use the apic hal.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Avi Kivity]

Michael Riepe wrote:
> The patch I attached is not essential. It fixes two error messages that
> occur on a 32-bit host (where the offending registers don't exist).
>   

Applied.  Please add a proper Signed-off-by: line in future patches.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Avi Kivity]

Michael Riepe wrote:
> By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages
> in my syslog, but they don't seem to cause any further harm.
>   

Which guest does this happen with? AFAICT this should be handled well 
(vmx.c:vmx_get_msr() -> find_msr_entry())

If you'd like to debug this, you can add printk()s to see where 
vmx_get_msr() loses its way.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Michael Riepe]

Hi!

Avi Kivity wrote:
> Michael Riepe wrote:
> 
>> By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages
>> in my syslog, but they don't seem to cause any further harm.
>>   
> 
> 
> Which guest does this happen with? AFAICT this should be handled well
> (vmx.c:vmx_get_msr() -> find_msr_entry())

With all of them.

> If you'd like to debug this, you can add printk()s to see where
> vmx_get_msr() loses its way.

vmx_get_msr() actually works fine. The problem is in vmx_vcpu_setup(),
in the loop starting at line 1142. It tries to read all the MSRs listed
in vmx_msr_index, but it fails for MSR_K6_STAR. With earlier releases, I
also used to see the kernel message "kvm: msrs: 1" - that is, only
MSR_EFER is found. According to Intel's docs, that's correct - the
register does not exist on a Core Duo which isn't 64-bit capable (but,
of course, on a Core 2 Duo - or a Xeon 51xx/53xx, aka "Woodcrest" and
"Clovertown").

-- 
Michael "Tired" Riepe <michael-0QoEqw4nQxo@public.gmane.org>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [ANNOUNCE] kvm userspace release 6

[Avi Kivity]

Michael Riepe wrote:
> vmx_get_msr() actually works fine. The problem is in vmx_vcpu_setup(),
> in the loop starting at line 1142. It tries to read all the MSRs listed
> in vmx_msr_index, but it fails for MSR_K6_STAR. With earlier releases, I
> also used to see the kernel message "kvm: msrs: 1" - that is, only
> MSR_EFER is found. According to Intel's docs, that's correct - the
> register does not exist on a Core Duo which isn't 64-bit capable (but,
> of course, on a Core 2 Duo - or a Xeon 51xx/53xx, aka "Woodcrest" and
> "Clovertown").
>   

I see now.  So the fix would be to tell the cpu the syscall feature 
doesn't exist, presumably via cpuid (which is handled in userspace).

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV