Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 1297 bytes --]

Dear KVM maintainers, it appears that there is a gap in x86 emulation,
at least on a 32-bit host.  Stefan found this when running GRML, a live
distribution which can be downloaded from:
<http://download.grml.org/grml32-full_2013.02.iso>.  His original
reported is at <http://bugs.debian.org/707257>.

On Thu, 2013-05-16 at 13:26 +0200, Stefan Pietsch wrote:
> On 09.05.2013 20:56, Stefan Pietsch wrote:
> > On 09.05.2013 03:08, Ben Hutchings wrote:
> > 
> >> Please could you test some of the intermediate versions at
> >> <http://snapshot.debian.org/package/linux/> to find the first upstream
> >> version where this was broken.
> > 
> > The first version which does not work is 3.6.4-1~experimental.1.
> > 3.5.5-1~experimental.1 works.
> 
> 
> I was able to start KVM under kernel version 3.8.12-1 after loading the
> "kvm_intel" module with the option "emulate_invalid_guest_state=0".

And one of the many changes between 3.5 and 3.6 was to change the
default value of that parameter from 0 to 1.  So we don't know when the
the bug in emulation was introduced (or if it was always there).

Ben.

-- 
Ben Hutchings
The generation of random numbers is too important to be left to chance.
                                                            - Robert Coveyou

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> at least on a 32-bit host.  Stefan found this when running GRML, a live
> distribution which can be downloaded from:
> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
> reported is at <http://bugs.debian.org/707257>.
> 
Can you verify with latest linux.git HEAD? It works for me there on
64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
so it would be helpful if you'll test 32bit before I install one myself.

> On Thu, 2013-05-16 at 13:26 +0200, Stefan Pietsch wrote:
> > On 09.05.2013 20:56, Stefan Pietsch wrote:
> > > On 09.05.2013 03:08, Ben Hutchings wrote:
> > > 
> > >> Please could you test some of the intermediate versions at
> > >> <http://snapshot.debian.org/package/linux/> to find the first upstream
> > >> version where this was broken.
> > > 
> > > The first version which does not work is 3.6.4-1~experimental.1.
> > > 3.5.5-1~experimental.1 works.
> > 
> > 
> > I was able to start KVM under kernel version 3.8.12-1 after loading the
> > "kvm_intel" module with the option "emulate_invalid_guest_state=0".
> 
> And one of the many changes between 3.5 and 3.6 was to change the
> default value of that parameter from 0 to 1.  So we don't know when the
> the bug in emulation was introduced (or if it was always there).
> 
> Ben.
> 
> -- 
> Ben Hutchings
> The generation of random numbers is too important to be left to chance.
>                                                             - Robert Coveyou



--
			Gleb.

Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.05.2013 14:32, Gleb Natapov wrote:
> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>> at least on a 32-bit host.  Stefan found this when running GRML, a live
>> distribution which can be downloaded from:
>> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
>> reported is at <http://bugs.debian.org/707257>.
>>
> Can you verify with latest linux.git HEAD? It works for me there on
> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> so it would be helpful if you'll test 32bit before I install one myself.


Ben,

can you provide a 3.9 series kernel package?

Thanks.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Ben Hutchings]

On Wed, May 29, 2013 at 05:05:55PM +0200, Stefan Pietsch wrote:
> On 19.05.2013 14:32, Gleb Natapov wrote:
> > On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >> at least on a 32-bit host.  Stefan found this when running GRML, a live
> >> distribution which can be downloaded from:
> >> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
> >> reported is at <http://bugs.debian.org/707257>.
> >>
> > Can you verify with latest linux.git HEAD? It works for me there on
> > 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> > so it would be helpful if you'll test 32bit before I install one myself.
> 
> 
> Ben,
> 
> can you provide a 3.9 series kernel package?

I will do soon, but you should be able to build your own:

# ...unpack upstream source...
$ cp /boot/config-$(uname -r) .config
$ yes "" | make oldconfig
$ make deb-pkg -j$(nproc)

Ben.

-- 
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
                                                              - Albert Camus

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.05.2013 14:32, Gleb Natapov wrote:
> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>> at least on a 32-bit host.  Stefan found this when running GRML, a live
>> distribution which can be downloaded from:
>> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
>> reported is at <http://bugs.debian.org/707257>.
>>
> Can you verify with latest linux.git HEAD? It works for me there on
> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> so it would be helpful if you'll test 32bit before I install one myself.


Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.

The virtual machine tries to boot the kernel, but stops after a few
seconds and the kern.log shows:

kernel: [13851.000412] kvm [7482]: vcpu0 disabled perfctr wrmsr: 0xc1
data 0xffff


virtual machine was started with:
qemu-system-i386 -machine accel=kvm -m 512 -cdrom grml32-full_2013.02.iso

qemu-system-x86: 1.5.0+dfsg-3

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
> On 19.05.2013 14:32, Gleb Natapov wrote:
> > On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >> at least on a 32-bit host.  Stefan found this when running GRML, a live
> >> distribution which can be downloaded from:
> >> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
> >> reported is at <http://bugs.debian.org/707257>.
> >>
> > Can you verify with latest linux.git HEAD? It works for me there on
> > 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> > so it would be helpful if you'll test 32bit before I install one myself.
> 
> 
> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> 
> The virtual machine tries to boot the kernel, but stops after a few
> seconds and the kern.log shows:
At what point does it stop?

> 
> kernel: [13851.000412] kvm [7482]: vcpu0 disabled perfctr wrmsr: 0xc1
> data 0xffff
> 
That's harmless.

> 
> virtual machine was started with:
> qemu-system-i386 -machine accel=kvm -m 512 -cdrom grml32-full_2013.02.iso
> 
> qemu-system-x86: 1.5.0+dfsg-3

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 05.06.2013 14:10, Gleb Natapov wrote:
> On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
>> On 19.05.2013 14:32, Gleb Natapov wrote:
>>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>>>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>>>> at least on a 32-bit host.  Stefan found this when running GRML, a live
>>>> distribution which can be downloaded from:
>>>> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
>>>> reported is at <http://bugs.debian.org/707257>.
>>>>
>>> Can you verify with latest linux.git HEAD? It works for me there on
>>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
>>> so it would be helpful if you'll test 32bit before I install one myself.
>>
>>
>> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
>>
>> The virtual machine tries to boot the kernel, but stops after a few
>> seconds and the kern.log shows:
> At what point does it stop?


The machine stops at:

Performance Events: Broken PMU hardware detected, using software events
only.
Failed to access perfctr msr (MSR c1 is 0)
Enabling APIC mode:  Flat.  Using 1 I/O APICs

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
> On 05.06.2013 14:10, Gleb Natapov wrote:
> > On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
> >> On 19.05.2013 14:32, Gleb Natapov wrote:
> >>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >>>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >>>> at least on a 32-bit host.  Stefan found this when running GRML, a live
> >>>> distribution which can be downloaded from:
> >>>> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
> >>>> reported is at <http://bugs.debian.org/707257>.
> >>>>
> >>> Can you verify with latest linux.git HEAD? It works for me there on
> >>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> >>> so it would be helpful if you'll test 32bit before I install one myself.
> >>
> >>
> >> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> >>
> >> The virtual machine tries to boot the kernel, but stops after a few
> >> seconds and the kern.log shows:
> > At what point does it stop?
> 
> 
> The machine stops at:
> 
> Performance Events: Broken PMU hardware detected, using software events
> only.
> Failed to access perfctr msr (MSR c1 is 0)
> Enabling APIC mode:  Flat.  Using 1 I/O APICs
Timer initialization is what comes next.

I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and upstream
qemu and I cannot reproduce the problem. The guest boots fine.

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 06, 2013 at 09:42:40AM +0300, Gleb Natapov wrote:
> On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
> > On 05.06.2013 14:10, Gleb Natapov wrote:
> > > On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
> > >> On 19.05.2013 14:32, Gleb Natapov wrote:
> > >>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> > >>>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> > >>>> at least on a 32-bit host.  Stefan found this when running GRML, a live
> > >>>> distribution which can be downloaded from:
> > >>>> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
> > >>>> reported is at <http://bugs.debian.org/707257>.
> > >>>>
> > >>> Can you verify with latest linux.git HEAD? It works for me there on
> > >>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> > >>> so it would be helpful if you'll test 32bit before I install one myself.
> > >>
> > >>
> > >> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> > >>
> > >> The virtual machine tries to boot the kernel, but stops after a few
> > >> seconds and the kern.log shows:
> > > At what point does it stop?
> > 
> > 
> > The machine stops at:
> > 
> > Performance Events: Broken PMU hardware detected, using software events
> > only.
> > Failed to access perfctr msr (MSR c1 is 0)
> > Enabling APIC mode:  Flat.  Using 1 I/O APICs
> Timer initialization is what comes next.
> 
> I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and upstream
> qemu and I cannot reproduce the problem. The guest boots fine.
> 
Actually the branch I tested is "master" not "next", but this should not
make a difference.

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 06.06.2013 08:42, Gleb Natapov wrote:
> On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
>> On 05.06.2013 14:10, Gleb Natapov wrote:
>>> On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
>>>> On 19.05.2013 14:32, Gleb Natapov wrote:
>>>>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>>>>>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>>>>>> at least on a 32-bit host.  Stefan found this when running GRML, a live
>>>>>> distribution which can be downloaded from:
>>>>>> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
>>>>>> reported is at <http://bugs.debian.org/707257>.
>>>>>>
>>>>> Can you verify with latest linux.git HEAD? It works for me there on
>>>>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
>>>>> so it would be helpful if you'll test 32bit before I install one myself.
>>>>
>>>>
>>>> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
>>>>
>>>> The virtual machine tries to boot the kernel, but stops after a few
>>>> seconds and the kern.log shows:
>>> At what point does it stop?
>>
>>
>> The machine stops at:
>>
>> Performance Events: Broken PMU hardware detected, using software events
>> only.
>> Failed to access perfctr msr (MSR c1 is 0)
>> Enabling APIC mode:  Flat.  Using 1 I/O APICs
> Timer initialization is what comes next.
> 
> I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and upstream
> qemu and I cannot reproduce the problem. The guest boots fine.


I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).

The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:
> On 06.06.2013 08:42, Gleb Natapov wrote:
> > On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
> >> On 05.06.2013 14:10, Gleb Natapov wrote:
> >>> On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
> >>>> On 19.05.2013 14:32, Gleb Natapov wrote:
> >>>>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >>>>>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >>>>>> at least on a 32-bit host.  Stefan found this when running GRML, a live
> >>>>>> distribution which can be downloaded from:
> >>>>>> <http://download.grml.org/grml32-full_2013.02.iso>.  His original
> >>>>>> reported is at <http://bugs.debian.org/707257>.
> >>>>>>
> >>>>> Can you verify with latest linux.git HEAD? It works for me there on
> >>>>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> >>>>> so it would be helpful if you'll test 32bit before I install one myself.
> >>>>
> >>>>
> >>>> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> >>>>
> >>>> The virtual machine tries to boot the kernel, but stops after a few
> >>>> seconds and the kern.log shows:
> >>> At what point does it stop?
> >>
> >>
> >> The machine stops at:
> >>
> >> Performance Events: Broken PMU hardware detected, using software events
> >> only.
> >> Failed to access perfctr msr (MSR c1 is 0)
> >> Enabling APIC mode:  Flat.  Using 1 I/O APICs
> > Timer initialization is what comes next.
> > 
> > I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and upstream
> > qemu and I cannot reproduce the problem. The guest boots fine.
> 
> 
> I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).
> 
> The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
OK, since it looks like it hangs during timer initialization can you try
to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
Also can you provide the output of "cat /proc/cpuinfo" on your host? And
complete serial output before hang.

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 06.06.2013 13:40, Gleb Natapov wrote:
> On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:

>> I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).
>>
>> The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
> OK, since it looks like it hangs during timer initialization can you try
> to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
> Also can you provide the output of "cat /proc/cpuinfo" on your host? And
> complete serial output before hang.


command line:
qemu-system-i386 -machine accel=kvm -m 512 -cpu qemu64,-kvmclock -cdrom
grml32-full_2013.02.iso -serial file:ttyS0.log



/proc/cpuinfo:
##############

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 14
model name	: Intel(R) Core(TM) Duo CPU      L2400  @ 1.66GHz
stepping	: 12
microcode	: 0x54
cpu MHz		: 1000.000
cache size	: 2048 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fdiv_bug	: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc
arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm dtherm
bogomips	: 3325.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 32 bits physical, 32 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 14
model name	: Intel(R) Core(TM) Duo CPU      L2400  @ 1.66GHz
stepping	: 12
microcode	: 0x54
cpu MHz		: 1000.000
cache size	: 2048 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
apicid		: 1
initial apicid	: 1
fdiv_bug	: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc
arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm dtherm
bogomips	: 3325.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 32 bits physical, 32 bits virtual
power management:



ttyS0.log:
##########

[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.7-1-grml-486 (team@grml.org) (gcc version
4.7.2 (Debian 4.7.2-5) ) #1 Debian 3.7.9-1+grml.1
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff]
usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff]
reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff]
reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001fffdfff]
usable
[    0.000000] BIOS-e820: [mem 0x000000001fffe000-0x000000001fffffff]
reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff]
reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff]
reserved
[    0.000000] Notice: NX (Execute Disable) protection cannot be
enabled: non-PAE kernel!
[    0.000000] SMBIOS 2.4 present.
[    0.000000] Hypervisor detected: KVM
[    0.000000] e820: last_pfn = 0x1fffe max_arch_pfn = 0x100000
[    0.000000] PAT not supported by CPU.
[    0.000000] found SMP MP-table at [mem 0x000fdb00-0x000fdb0f] mapped
at [c00fdb00]
[    0.000000] init_memory_mapping: [mem 0x00000000-0x1fffdfff]
[    0.000000] RAMDISK: [mem 0x1f330000-0x1ffdbfff]
[    0.000000] ACPI: RSDP 000fd9a0 00014 (v00 BOCHS )
[    0.000000] ACPI: RSDT 1fffe4b0 00034 (v01 BOCHS  BXPCRSDT 00000001
BXPC 00000001)
[    0.000000] ACPI: FACP 1fffff80 00074 (v01 BOCHS  BXPCFACP 00000001
BXPC 00000001)
[    0.000000] ACPI: DSDT 1fffe4f0 011A9 (v01   BXPC   BXDSDT 00000001
INTL 20100528)
[    0.000000] ACPI: FACS 1fffff40 00040
[    0.000000] ACPI: SSDT 1ffff800 00735 (v01 BOCHS  BXPCSSDT 00000001
BXPC 00000001)
[    0.000000] ACPI: APIC 1ffff6e0 00078 (v01 BOCHS  BXPCAPIC 00000001
BXPC 00000001)
[    0.000000] ACPI: HPET 1ffff6a0 00038 (v01 BOCHS  BXPCHPET 00000001
BXPC 00000001)
[    0.000000] 0MB HIGHMEM available.
[    0.000000] 511MB LOWMEM available.
[    0.000000]   mapped low ram: 0 - 1fffe000
[    0.000000]   low ram: 0 - 1fffe000
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00010000-0x00ffffff]
[    0.000000]   Normal   [mem 0x01000000-0x1fffdfff]
[    0.000000]   HighMem  empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00010000-0x0009efff]
[    0.000000]   node   0: [mem 0x00100000-0x1fffdfff]
[    0.000000] Using APIC driver default
[    0.000000] ACPI: PM-Timer IO Port: 0xb008
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI
0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high
level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high
level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] KVM setup async PF for cpu 0
[    0.000000] kvm-stealtime: cpu 0, msr 1408000
[    0.000000] PM: Registered nosave memory: 000000000009f000 -
00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 -
00000000000f0000
[    0.000000] PM: Registered nosave memory: 00000000000f0000 -
0000000000100000
[    0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on KVM
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.
Total pages: 129933
[    0.000000] Kernel command line: initrd=/boot/grml32full/initrd.img
boot=live live-media-path=/live/grml32-full/ bootid=grml32full201302
apm=power-off console=ttyS0,9600 nomce  BOOT_IMAGE=/boot/grml32full/vmlinuz
[    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144
bytes)
[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072
bytes)
[    0.000000] __ex_table already sorted, skipping sort
[    0.000000] Initializing CPU#0
[    0.000000] Initializing HighMem for node 0 (00000000:00000000)
[    0.000000] Memory: 500936k/524280k available (2822k kernel code,
22892k reserved, 1523k data, 488k init, 0k highmem)
[    0.000000] virtual kernel memory layout:
[    0.000000]     fixmap  : 0xfffa1000 - 0xfffff000   ( 376 kB)
[    0.000000]     pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
[    0.000000]     vmalloc : 0xe07fe000 - 0xff7fe000   ( 496 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xdfffe000   ( 511 MB)
[    0.000000]       .init : 0xc143f000 - 0xc14b9000   ( 488 kB)
[    0.000000]       .data : 0xc12c19f2 - 0xc143e700   (1523 kB)
[    0.000000]       .text : 0xc1000000 - 0xc12c19f2   (2822 kB)
[    0.000000] Checking if this processor honours the WP bit even in
supervisor mode...Ok.
[    0.000000] NR_IRQS:2304 nr_irqs:256 16
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] tsc: Fast TSC calibration failed
[    0.000000] tsc: Unable to calibrate against PIT
[    0.000000] tsc: using HPET reference calibration
[    0.000000] tsc: Detected 1662.353 MHz processor
[    0.028002] Calibrating delay loop (skipped), value calculated using
timer frequency.. 3324.70 BogoMIPS (lpj=6649412)
[    0.033156] pid_max: default: 32768 minimum: 301
[    0.035077] Security Framework initialized
[    0.036026] AppArmor: AppArmor disabled by boot time parameter
[    0.038360] Mount-cache hash table entries: 512
[    0.040309] Initializing cgroup subsys cpuacct
[    0.042113] Initializing cgroup subsys memory
[    0.044017] Initializing cgroup subsys devices
[    0.045793] Initializing cgroup subsys freezer
[    0.048009] Initializing cgroup subsys net_cls
[    0.049792] Initializing cgroup subsys blkio
[    0.052008] Initializing cgroup subsys perf_event
[    0.054108] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.054108] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.054108] tlb_flushall_shift: 6
[    0.056007] CPU: Intel QEMU Virtual CPU version 1.5.0 (fam: 06,
model: 02, stepping: 03)
[    0.064662] ACPI: Core revision 20120913
[    0.067943] Performance Events: unsupported p6 CPU model 2 no PMU
driver, software events only.
[    0.072181] Enabling APIC mode:  Flat.  Using 1 I/O APICs

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 06, 2013 at 02:10:39PM +0200, Stefan Pietsch wrote:
> On 06.06.2013 13:40, Gleb Natapov wrote:
> > On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:
> 
> >> I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).
> >>
> >> The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
> > OK, since it looks like it hangs during timer initialization can you try
> > to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
> > Also can you provide the output of "cat /proc/cpuinfo" on your host? And
> > complete serial output before hang.
> 
> 
> command line:
> qemu-system-i386 -machine accel=kvm -m 512 -cpu qemu64,-kvmclock -cdrom
> grml32-full_2013.02.iso -serial file:ttyS0.log
> 
> 
> ttyS0.log:
> ##########
> 

Nothing out of ordinary here. Since you can reproduce the hang and I
cannot, can you try and bisect it? Also can trace kvm during the hang
http://www.linux-kvm.org/page/Tracing? Start the trace as close to hang
as possible and stop it as quick after it as possible too to make trace
file smaller.

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 09.06.2013 11:43, Gleb Natapov wrote:
> On Thu, Jun 06, 2013 at 02:10:39PM +0200, Stefan Pietsch wrote:
>> On 06.06.2013 13:40, Gleb Natapov wrote:
>>> On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:
>>
>>>> I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).
>>>>
>>>> The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
>>> OK, since it looks like it hangs during timer initialization can you try
>>> to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
>>> Also can you provide the output of "cat /proc/cpuinfo" on your host? And
>>> complete serial output before hang.
>>
>>
>> command line:
>> qemu-system-i386 -machine accel=kvm -m 512 -cpu qemu64,-kvmclock -cdrom
>> grml32-full_2013.02.iso -serial file:ttyS0.log
>>
>>
>> ttyS0.log:
>> ##########
>>
> 
> Nothing out of ordinary here. Since you can reproduce the hang and I
> cannot, can you try and bisect it? Also can trace kvm during the hang
> http://www.linux-kvm.org/page/Tracing? Start the trace as close to hang
> as possible and stop it as quick after it as possible too to make trace
> file smaller.


git bisect tells me:
79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit


This is my bisect log:

git bisect start
git bisect bad 9626357371b519f2b955fef399647181034a77fe
git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
git bisect good b5c78e04dd061b776978dad61dd85357081147b0
git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Paolo Bonzini]

Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
> git bisect tells me:
> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit

This is an s390 commit, so the bisect somehow went wrong.  Can you
confirm that 3.7 works and 3.8 doesn't?

Please check these pairs:

9e2d59a and 89f883372fa60f604d136924baf3e89ff1870e9e
39ab967 and 875b7679abbb232b584f2eec59fa6e45690dd6c4
10b3866 and ea4a0ce11160200410abbabd44ec9e75e93a95be
4ffd4eb and ccae663cd4f62890d862c660e5ed762eb9821c14
896ea17 and 66cdd0ceaf65a18996f561b770eedde1d123b019

Please tell us which pair introduced the failure.  Then:

- if you get a "bad and bad" pair, tell us and we'll figure out what's
next :)

- if you get a "good and bad" pair, do a "git bisect" between the two
commits in that pair.

Thanks!

Paolo

> This is my bisect log:
> 
> git bisect start
> git bisect bad 9626357371b519f2b955fef399647181034a77fe
> git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
> git bisect good b5c78e04dd061b776978dad61dd85357081147b0
> git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
> git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
> git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
> git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
> git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
> git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Paolo Bonzini]

Il 13/06/2013 09:42, Paolo Bonzini ha scritto:
> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
>> git bisect tells me:
>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
> 
> This is an s390 commit, so the bisect somehow went wrong.  Can you
> confirm that 3.7 works and 3.8 doesn't?

Sorry, 3.8 works and 3.9 doesn't
(66cdd0ceaf65a18996f561b770eedde1d123b019 was the 3.8 merge window
update, and your bisect shows it as good).

Can you double-check this with both normal "modprobe kvm_intel" and
"modprobe kvm_intel emulate_invalid_guest_state=0"?

Paolo

> Please check these pairs:
> 
> 9e2d59a and 89f883372fa60f604d136924baf3e89ff1870e9e
> 39ab967 and 875b7679abbb232b584f2eec59fa6e45690dd6c4
> 10b3866 and ea4a0ce11160200410abbabd44ec9e75e93a95be
> 4ffd4eb and ccae663cd4f62890d862c660e5ed762eb9821c14
> 896ea17 and 66cdd0ceaf65a18996f561b770eedde1d123b019
> 
> Please tell us which pair introduced the failure.  Then:
> 
> - if you get a "bad and bad" pair, tell us and we'll figure out what's
> next :)
> 
> - if you get a "good and bad" pair, do a "git bisect" between the two
> commits in that pair.
> 
> Thanks!
> 
> Paolo
> 
>> This is my bisect log:
>>
>> git bisect start
>> git bisect bad 9626357371b519f2b955fef399647181034a77fe
>> git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
>> git bisect good b5c78e04dd061b776978dad61dd85357081147b0
>> git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
>> git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
>> git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
>> git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
>> git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
>> git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 13.06.2013 15:42, Paolo Bonzini wrote:
> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
>> git bisect tells me:
>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
> 
> This is an s390 commit, so the bisect somehow went wrong.  Can you
> confirm that 3.7 works and 3.8 doesn't?

Confirmed. Something went wrong.
I replayed the bisect log and now I have

git bisect bad 9626357371b519f2b955fef399647181034a77fe
git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
git bisect good b5c78e04dd061b776978dad61dd85357081147b0
git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
git bisect bad aa11e3a8a6d9f92c3fe4b91a9aca5d8c23d55d4d
git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
git bisect bad d99e415275dd3f757b75981adad8645cdc26da45

So please wait for my results.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 13.06.2013 16:59, Stefan Pietsch wrote:
> On 13.06.2013 15:42, Paolo Bonzini wrote:
>> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
>>> git bisect tells me:
>>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
>>
>> This is an s390 commit, so the bisect somehow went wrong.  Can you
>> confirm that 3.7 works and 3.8 doesn't?
> 
> Confirmed. Something went wrong.
> I replayed the bisect log and now I have
> 
> git bisect bad 9626357371b519f2b955fef399647181034a77fe
> git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
> git bisect good b5c78e04dd061b776978dad61dd85357081147b0
> git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
> git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
> git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
> git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
> git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
> git bisect bad aa11e3a8a6d9f92c3fe4b91a9aca5d8c23d55d4d
> git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
> git bisect bad d99e415275dd3f757b75981adad8645cdc26da45
> 
> So please wait for my results.


Bisecting leads to

git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
git bisect good 218e763f458c44f30041c1b48b4371e130fd4317


first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
KVM: VMX: don't clobber segment AR of unusable segments.


25391454e73e3156202264eb3c473825afe4bc94
emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
emulate_invalid_guest_state=1 -> hangs

Please note, I had to compile some revisions with
3f0c3d0bb2bcc4b88b22452a7cf0073ee9a0f1e6 applied, caused by
9ae9febae9500a0a6f5ce29ee4b8d942b5332529.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Paolo Bonzini]

Il 16/06/2013 02:25, Stefan Pietsch ha scritto:
> Bisecting leads to
> 
> git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
> git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
> git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
> git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
> git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
> git bisect good 218e763f458c44f30041c1b48b4371e130fd4317
> 
> 
> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
> KVM: VMX: don't clobber segment AR of unusable segments.
> 
> 25391454e73e3156202264eb3c473825afe4bc94
> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
> emulate_invalid_guest_state=1 -> hangs
> 
> Please note, I had to compile some revisions with
> 3f0c3d0bb2bcc4b88b22452a7cf0073ee9a0f1e6 applied, caused by
> 9ae9febae9500a0a6f5ce29ee4b8d942b5332529.

Can you please execute "info registers" and "x/10i $pc" from the QEMU
monitor at the time of the hang, and include the output?  Using
"-monitor stdio" or the new GTK+ interface can help.

Also, can you run under tracing (for information on how to do this, see
http://www.linux-kvm.org/page/Tracing) and include the bottom of the log?

Thanks,

Paolo

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 17.06.2013 18:07, Paolo Bonzini wrote:
> Il 16/06/2013 02:25, Stefan Pietsch ha scritto:
>> Bisecting leads to
>>
>> git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
>> git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
>> git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
>> git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
>> git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
>> git bisect good 218e763f458c44f30041c1b48b4371e130fd4317
>>
>>
>> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
>> KVM: VMX: don't clobber segment AR of unusable segments.
>>
>> 25391454e73e3156202264eb3c473825afe4bc94
>> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
>> emulate_invalid_guest_state=1 -> hangs
>>
>> Please note, I had to compile some revisions with
>> 3f0c3d0bb2bcc4b88b22452a7cf0073ee9a0f1e6 applied, caused by
>> 9ae9febae9500a0a6f5ce29ee4b8d942b5332529.
> 
> Can you please execute "info registers" and "x/10i $pc" from the QEMU
> monitor at the time of the hang, and include the output?  Using
> "-monitor stdio" or the new GTK+ interface can help.
> 
> Also, can you run under tracing (for information on how to do this, see
> http://www.linux-kvm.org/page/Tracing) and include the bottom of the log?

Tested with 25391454e73e3156202264eb3c473825afe4bc94
 emulate_invalid_guest_state=1


(qemu) info registers
EAX=00010286 EBX=00000000 ECX=c12c527c EDX=00000000
ESI=00010286 EDI=c14c4744 EBP=c10161f5 ESP=de84df10
EIP=c1014a8d EFL=00010286 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0068 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
FS =0000 ffff0000 f0000fff 00f0ff00 DPL=3 CS64 [CRA]
GS =00e0 c1438b40 00000018 00409100 DPL=0 DS   [--A]
LDT=0000 ffff0000 f0000fff 00f0ff00 DPL=3 CS64 [CRA]
TR =0080 c1400f00 0000206b 00008b00 DPL=0 TSS32-busy
GDT=     c13f6000 000000ff
IDT=     c13f5000 000007ff
CR0=8005003b CR2=ffffffff CR3=014bc000 CR4=00000690
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000700000000
DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=f44d002c60000000 400d FPR1=80847fe700000000 400e
FPR2=fa007fa240000000 400e FPR3=80e88055f0000000 400e
FPR4=ea61009c40000000 400d FPR5=ea62009c40000000 400c
FPR6=800bf60000000000 4015 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000
XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000
XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000
XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000
XMM07=00000000000000000000000000000000

(qemu) x/10i $pc
0x00000000c1014a8d:  lea    0x0(%esi),%esi
0x00000000c1014a91:  ret
0x00000000c1014a92:  cli
0x00000000c1014a93:  nop
0x00000000c1014a94:  lea    0x0(%esi),%esi
0x00000000c1014a98:  ret
0x00000000c1014a99:  push   %eax
0x00000000c1014a9a:  call   0xc1014a84
0x00000000c1014a9f:  mov    %eax,(%esp)
0x00000000c1014aa2:  call   0xc1014a92


last 20 lines of the trace:
 qemu-system-x86-3575  [000]   542.279800: kvm_entry:            vcpu 0
 qemu-system-x86-3575  [000]   542.279802: kvm_inj_virq:         irq 48
 qemu-system-x86-3575  [000]   542.279802: kvm_entry:            vcpu 0
 qemu-system-x86-3575  [000]   542.279803: kvm_inj_virq:         irq 48
 qemu-system-x86-3575  [000]   542.279804: kvm_entry:            vcpu 0
 qemu-system-x86-3575  [000]   542.279805: kvm_inj_virq:         irq 48
 qemu-system-x86-3575  [000]   542.279806: kvm_entry:            vcpu 0
 qemu-system-x86-3575  [000]   542.279807: kvm_inj_virq:         irq 48
 qemu-system-x86-3575  [000]   542.279808: kvm_entry:            vcpu 0
 qemu-system-x86-3575  [000]   542.279809: kvm_inj_virq:         irq 48
 qemu-system-x86-3575  [000]   542.279810: kvm_entry:            vcpu 0
 qemu-system-x86-3575  [000]   542.279811: kvm_inj_virq:         irq 48
 qemu-system-x86-3575  [000]   542.279812: kvm_entry:            vcpu 0
 qemu-system-x86-3573  [001]   542.280010: kvm_set_irq:          gsi 0
level 1 source 0
 qemu-system-x86-3573  [001]   542.280013: kvm_pic_set_irq:      chip 0
pin 0 (edge|masked)
 qemu-system-x86-3573  [001]   542.280015: kvm_apic_accept_irq:  apicid
0 vec 48 (LowPrio|edge) (coalesced)
 qemu-system-x86-3573  [001]   542.280015: kvm_ioapic_set_irq:   pin 2
dst 1 vec=48 (LowPrio|logical|edge) (coalesced)
 qemu-system-x86-3573  [001]   542.280016: kvm_set_irq:          gsi 0
level 0 source 0
 qemu-system-x86-3573  [001]   542.280017: kvm_pic_set_irq:      chip 0
pin 0 (edge|masked)
 qemu-system-x86-3573  [001]   542.280017: kvm_ioapic_set_irq:   pin 2
dst 1 vec=48 (LowPrio|logical|edge)

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:
> On 13.06.2013 16:59, Stefan Pietsch wrote:
> > On 13.06.2013 15:42, Paolo Bonzini wrote:
> >> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
> >>> git bisect tells me:
> >>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
> >>
> >> This is an s390 commit, so the bisect somehow went wrong.  Can you
> >> confirm that 3.7 works and 3.8 doesn't?
> > 
> > Confirmed. Something went wrong.
> > I replayed the bisect log and now I have
> > 
> > git bisect bad 9626357371b519f2b955fef399647181034a77fe
> > git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
> > git bisect good b5c78e04dd061b776978dad61dd85357081147b0
> > git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
> > git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
> > git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
> > git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
> > git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
> > git bisect bad aa11e3a8a6d9f92c3fe4b91a9aca5d8c23d55d4d
> > git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
> > git bisect bad d99e415275dd3f757b75981adad8645cdc26da45
> > 
> > So please wait for my results.
> 
> 
> Bisecting leads to
> 
> git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
> git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
> git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
> git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
> git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
> git bisect good 218e763f458c44f30041c1b48b4371e130fd4317
> 
> 
> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
> KVM: VMX: don't clobber segment AR of unusable segments.
> 
> 
> 25391454e73e3156202264eb3c473825afe4bc94
> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
> emulate_invalid_guest_state=1 -> hangs
>
With emulate_invalid_guest_state=1 the commit does nothing. Can you
double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
with emulate_invalid_guest_state=1?
 
--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.06.2013 15:41, Gleb Natapov wrote:
> On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:
>>
>> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
>> KVM: VMX: don't clobber segment AR of unusable segments.
>>
>>
>> 25391454e73e3156202264eb3c473825afe4bc94
>> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
>> emulate_invalid_guest_state=1 -> hangs
>>
> With emulate_invalid_guest_state=1 the commit does nothing. Can you
> double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
> with emulate_invalid_guest_state=1?


218e763f458c44f30041c1b48b4371e130fd4317
emulate_invalid_guest_state=0 -> works
emulate_invalid_guest_state=1 -> hangs

25391454e73e3156202264eb3c473825afe4bc94 broke
emulate_invalid_guest_state=0.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Wed, Jun 19, 2013 at 04:12:04PM +0200, Stefan Pietsch wrote:
> On 19.06.2013 15:41, Gleb Natapov wrote:
> > On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:
> >>
> >> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
> >> KVM: VMX: don't clobber segment AR of unusable segments.
> >>
> >>
> >> 25391454e73e3156202264eb3c473825afe4bc94
> >> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
> >> emulate_invalid_guest_state=1 -> hangs
> >>
> > With emulate_invalid_guest_state=1 the commit does nothing. Can you
> > double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
> > with emulate_invalid_guest_state=1?
> 
> 
> 218e763f458c44f30041c1b48b4371e130fd4317
> emulate_invalid_guest_state=0 -> works
> emulate_invalid_guest_state=1 -> hangs
> 
> 25391454e73e3156202264eb3c473825afe4bc94 broke
> emulate_invalid_guest_state=0.
Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
monitor after the hang.

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.06.2013 16:38, Gleb Natapov wrote:
> On Wed, Jun 19, 2013 at 04:12:04PM +0200, Stefan Pietsch wrote:
>> On 19.06.2013 15:41, Gleb Natapov wrote:
>>> On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:
>>>>
>>>> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
>>>> KVM: VMX: don't clobber segment AR of unusable segments.
>>>>
>>>>
>>>> 25391454e73e3156202264eb3c473825afe4bc94
>>>> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
>>>> emulate_invalid_guest_state=1 -> hangs
>>>>
>>> With emulate_invalid_guest_state=1 the commit does nothing. Can you
>>> double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
>>> with emulate_invalid_guest_state=1?
>>
>>
>> 218e763f458c44f30041c1b48b4371e130fd4317
>> emulate_invalid_guest_state=0 -> works
>> emulate_invalid_guest_state=1 -> hangs
>>
>> 25391454e73e3156202264eb3c473825afe4bc94 broke
>> emulate_invalid_guest_state=0.
> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> monitor after the hang.


25391454e73e3156202264eb3c473825afe4bc94
 emulate_invalid_guest_state=0

(qemu) info registers
EAX=00000000 EBX=00000001 ECX=fffff000 EDX=fffff000
ESI=00195e93 EDI=00000000 EBP=de84c000 ESP=de84df64
EIP=c101611c EFL=00010246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0068 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
FS =0000 ffff0000 f0000fff 00f0ff00 DPL=3 CS64 [CRA]
GS =00e0 c1438b40 00000018 00409100 DPL=0 DS   [--A]
LDT=0000 ffff0000 f0000fff 00f0ff00 DPL=3 CS64 [CRA]
TR =0080 c1400f00 0000206b 00008b00 DPL=0 TSS32-busy
GDT=     c13f6000 000000ff
IDT=     c13f5000 000007ff
CR0=8005003b CR2=ffffffff CR3=014bc000 CR4=00000690
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000700000000
DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=f44d002c60000000 400d FPR1=80847fe700000000 400e
FPR2=fa007fa240000000 400e FPR3=80e88055f0000000 400e
FPR4=ea61009c40000000 400d FPR5=ea62009c40000000 400c
FPR6=800bf60000000000 4015 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000
XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000
XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000
XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000
XMM07=00000000000000000000000000000000

(qemu) x/20i $pc-20
0x00000000c1016108:  (bad)
0x00000000c1016109:  decl   0x158bc310(%ecx)
0x00000000c101610f:  les    -0x7b723ec0(%ebx),%eax
0x00000000c1016115:  adc    %al,(%eax)
0x00000000c1016117:  sar    $0xff,%bh
0x00000000c101611a:  mov    (%eax),%eax
0x00000000c101611c:  ret
0x00000000c101611d:  jmp    0xc1016121
0x00000000c101611f:  pause
0x00000000c1016121:  mov    (%eax),%edx
0x00000000c1016123:  test   %edx,%edx
0x00000000c1016125:  je     0xc101611f
0x00000000c1016127:  ret
0x00000000c1016128:  cmp    $0xfe,%eax
0x00000000c101612d:  setle  %al
0x00000000c1016130:  movzbl %al,%eax
0x00000000c1016133:  ret
0x00000000c1016134:  mov    0xc1407b64,%edx
0x00000000c101613a:  mov    $0x20,%eax
0x00000000c101613f:  call   *0x9c(%edx)


last 20 lines of the trace:
 qemu-system-x86-4042  [000]   295.592694: vcpu_match_mmio:      gva
0xffffb0d0 gpa 0xfee000d0 Read GVA
 qemu-system-x86-4042  [000]   295.592694: kvm_apic:
apic_read APIC_LDR = 0x1000000
 qemu-system-x86-4042  [000]   295.592695: kvm_mmio:             mmio
read len 4 gpa 0xfee000d0 val 0x1000000
 qemu-system-x86-4042  [000]   295.592696: kvm_entry:            vcpu 0
 qemu-system-x86-4042  [000]   295.592699: kvm_exit:             reason
EXCEPTION_NMI rip 0xc101611a info ffffb080 80000b0e
 qemu-system-x86-4042  [000]   295.592700: kvm_page_fault:       address
ffffb080 error_code 9
 qemu-system-x86-4042  [000]   295.592701: kvm_emulate_insn:
0:c101611a:8b 00 (prot32)
 qemu-system-x86-4042  [000]   295.592702: vcpu_match_mmio:      gva
0xffffb080 gpa 0xfee00080 Read GVA
 qemu-system-x86-4042  [000]   295.592703: kvm_apic:
apic_read APIC_TASKPRI = 0x0
 qemu-system-x86-4042  [000]   295.592703: kvm_mmio:             mmio
read len 4 gpa 0xfee00080 val 0x0
 qemu-system-x86-4042  [000]   295.592704: kvm_userspace_exit:   reason
KVM_EXIT_TPR_ACCESS (12)
 qemu-system-x86-4042  [000]   295.592805: kvm_entry:            vcpu 0
 qemu-system-x86-4042  [000]   295.592808: kvm_exit:             reason
 rip 0xc101611c info 0 80000b0e
 qemu-system-x86-4042  [000]   295.592809: kvm_userspace_exit:   reason
KVM_EXIT_FAIL_ENTRY (9)
 qemu-system-x86-4040  [001]   295.594993: kvm_set_irq:          gsi 0
level 1 source 0
 qemu-system-x86-4040  [001]   295.594995: kvm_pic_set_irq:      chip 0
pin 0 (edge)
 qemu-system-x86-4040  [001]   295.594997: kvm_ioapic_set_irq:   pin 2
dst 0 vec=0 (Fixed|physical|edge|masked)
 qemu-system-x86-4040  [001]   295.594998: kvm_set_irq:          gsi 0
level 0 source 0
 qemu-system-x86-4040  [001]   295.594998: kvm_pic_set_irq:      chip 0
pin 0 (edge)
 qemu-system-x86-4040  [001]   295.594999: kvm_ioapic_set_irq:   pin 2
dst 0 vec=0 (Fixed|physical|edge|masked)

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> > Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> > and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> > monitor after the hang.
> 
> 
> 25391454e73e3156202264eb3c473825afe4bc94
>  emulate_invalid_guest_state=0
> 
Very interesting. Looks like somewhere during TPR access FS
register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
and try again? This will disable some code paths during TPR access and
will narrow down the issue.

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 23.06.2013 09:51, Gleb Natapov wrote:
> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
>>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
>>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
>>> monitor after the hang.
>>
>>
>> 25391454e73e3156202264eb3c473825afe4bc94
>>  emulate_invalid_guest_state=0
>>
> Very interesting. Looks like somewhere during TPR access FS
> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> and try again? This will disable some code paths during TPR access and
> will narrow down the issue.


Doing this, qemu complains
"Could not open option rom 'kvmvapic.bin': No such file or directory",
but the virtual machine boots successful with
emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.

kvmvapic.bin comes with Debian package "seabios 1.7.2-3".

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
> On 23.06.2013 09:51, Gleb Natapov wrote:
> > On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> >>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> >>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> >>> monitor after the hang.
> >>
> >>
> >> 25391454e73e3156202264eb3c473825afe4bc94
> >>  emulate_invalid_guest_state=0
> >>
> > Very interesting. Looks like somewhere during TPR access FS
> > register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> > and try again? This will disable some code paths during TPR access and
> > will narrow down the issue.
> 
> 
> Doing this, qemu complains
> "Could not open option rom 'kvmvapic.bin': No such file or directory",
> but the virtual machine boots successful with
> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
> 
Hmm, I think we ate close. Can you try with upstream qemu?

> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 23.06.2013 19:36, Gleb Natapov wrote:
> On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
>> On 23.06.2013 09:51, Gleb Natapov wrote:
>>> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
>>>>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
>>>>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
>>>>> monitor after the hang.
>>>>
>>>>
>>>> 25391454e73e3156202264eb3c473825afe4bc94
>>>>  emulate_invalid_guest_state=0
>>>>
>>> Very interesting. Looks like somewhere during TPR access FS
>>> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
>>> and try again? This will disable some code paths during TPR access and
>>> will narrow down the issue.
>>
>>
>> Doing this, qemu complains
>> "Could not open option rom 'kvmvapic.bin': No such file or directory",
>> but the virtual machine boots successful with
>> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
>>
> Hmm, I think we ate close. Can you try with upstream qemu?
> 
>> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".

I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Mon, Jun 24, 2013 at 01:43:26PM +0200, Stefan Pietsch wrote:
> On 23.06.2013 19:36, Gleb Natapov wrote:
> > On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
> >> On 23.06.2013 09:51, Gleb Natapov wrote:
> >>> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> >>>>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> >>>>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> >>>>> monitor after the hang.
> >>>>
> >>>>
> >>>> 25391454e73e3156202264eb3c473825afe4bc94
> >>>>  emulate_invalid_guest_state=0
> >>>>
> >>> Very interesting. Looks like somewhere during TPR access FS
> >>> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> >>> and try again? This will disable some code paths during TPR access and
> >>> will narrow down the issue.
> >>
> >>
> >> Doing this, qemu complains
> >> "Could not open option rom 'kvmvapic.bin': No such file or directory",
> >> but the virtual machine boots successful with
> >> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
> >>
> > Hmm, I think we ate close. Can you try with upstream qemu?
> > 
> >> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".
> 
> I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.
And it didn't work? Mind trying some debug kernel patches? I suspect
your CPU does something no CPU I have do, so I want to verify it.

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 24.06.2013 13:47, Gleb Natapov wrote:
> On Mon, Jun 24, 2013 at 01:43:26PM +0200, Stefan Pietsch wrote:
>> On 23.06.2013 19:36, Gleb Natapov wrote:
>>> On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
>>>> On 23.06.2013 09:51, Gleb Natapov wrote:
>>>>> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
>>>>>>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
>>>>>>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
>>>>>>> monitor after the hang.
>>>>>>
>>>>>>
>>>>>> 25391454e73e3156202264eb3c473825afe4bc94
>>>>>>  emulate_invalid_guest_state=0
>>>>>>
>>>>> Very interesting. Looks like somewhere during TPR access FS
>>>>> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
>>>>> and try again? This will disable some code paths during TPR access and
>>>>> will narrow down the issue.
>>>>
>>>>
>>>> Doing this, qemu complains
>>>> "Could not open option rom 'kvmvapic.bin': No such file or directory",
>>>> but the virtual machine boots successful with
>>>> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
>>>>
>>> Hmm, I think we ate close. Can you try with upstream qemu?
>>>
>>>> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".
>>
>> I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.
> And it didn't work? Mind trying some debug kernel patches? I suspect
> your CPU does something no CPU I have do, so I want to verify it.


As soon as I remove "kvmvapic.bin" the virtual machine boots with
qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
"emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
no difference.

Please send your patches.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
> On 24.06.2013 13:47, Gleb Natapov wrote:
> > On Mon, Jun 24, 2013 at 01:43:26PM +0200, Stefan Pietsch wrote:
> >> On 23.06.2013 19:36, Gleb Natapov wrote:
> >>> On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
> >>>> On 23.06.2013 09:51, Gleb Natapov wrote:
> >>>>> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> >>>>>>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> >>>>>>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> >>>>>>> monitor after the hang.
> >>>>>>
> >>>>>>
> >>>>>> 25391454e73e3156202264eb3c473825afe4bc94
> >>>>>>  emulate_invalid_guest_state=0
> >>>>>>
> >>>>> Very interesting. Looks like somewhere during TPR access FS
> >>>>> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> >>>>> and try again? This will disable some code paths during TPR access and
> >>>>> will narrow down the issue.
> >>>>
> >>>>
> >>>> Doing this, qemu complains
> >>>> "Could not open option rom 'kvmvapic.bin': No such file or directory",
> >>>> but the virtual machine boots successful with
> >>>> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
> >>>>
> >>> Hmm, I think we ate close. Can you try with upstream qemu?
> >>>
> >>>> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".
> >>
> >> I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.
> > And it didn't work? Mind trying some debug kernel patches? I suspect
> > your CPU does something no CPU I have do, so I want to verify it.
> 
> 
> As soon as I remove "kvmvapic.bin" the virtual machine boots with
> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
> no difference.
> 
> Please send your patches.
Here it is, run with it and kvmvapic.bin present. See what is printed in
dmesg after the failure.


diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f4a5b3f..65488a4 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 	u32 ar;
+	unsigned long rip;
 
 	if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
 		*var = vmx->rmode.segs[seg];
@@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
 	var->db = (ar >> 14) & 1;
 	var->g = (ar >> 15) & 1;
 	var->unusable = (ar >> 16) & 1;
+	rip = kvm_rip_read(vcpu);
+	if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
+		printk("base=%p limit=%p selector=%x ar=%x\n", var->base, var->limit, var->selector, ar);
 }
 
 static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 24.06.2013 14:30, Gleb Natapov wrote:
> On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
>> As soon as I remove "kvmvapic.bin" the virtual machine boots with
>> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
>> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
>> no difference.
>>
>> Please send your patches.
> Here it is, run with it and kvmvapic.bin present. See what is printed in
> dmesg after the failure.
> 
> 
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index f4a5b3f..65488a4 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>  {
>  	struct vcpu_vmx *vmx = to_vmx(vcpu);
>  	u32 ar;
> +	unsigned long rip;
>  
>  	if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
>  		*var = vmx->rmode.segs[seg];
> @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>  	var->db = (ar >> 14) & 1;
>  	var->g = (ar >> 15) & 1;
>  	var->unusable = (ar >> 16) & 1;
> +	rip = kvm_rip_read(vcpu);
> +	if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
> +		printk("base=%p limit=%p selector=%x ar=%x\n", var->base, var->limit, var->selector, ar);
>  }
>  
>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)


Booting kernel Linux 3.10-rc5 with your patch applied produces these
messages in dmesg when starting a virtual machine:

emulate_invalid_guest_state=0
[  118.732151] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  118.732341] base=ffff0000 limit=  (null) selector=f0000fff ar=0

emulate_invalid_guest_state=1
[  196.481653] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481700] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481706] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481711] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481716] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481720] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481725] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481730] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481735] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481739] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.481777] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482068] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482073] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482079] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482084] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482131] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482136] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482142] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482146] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482193] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482198] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482203] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482208] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482255] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482259] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482265] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482269] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482316] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482321] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482326] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482331] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482378] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482382] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482388] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482392] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482439] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482444] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482449] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482454] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482501] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482505] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482511] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482516] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482562] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482567] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482573] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.482577] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483137] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483142] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483147] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483152] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483712] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483716] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483722] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.483727] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484321] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484326] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484333] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484337] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484897] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484901] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484907] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.484911] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.487824] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.487830] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.487836] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.487841] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.488842] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.488847] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.488853] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.488858] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.489416] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.489420] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.489426] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.489431] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490052] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490057] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490062] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490067] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490148] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490152] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490158] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490162] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490262] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490266] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490272] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.490277] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529018] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529025] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529032] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529036] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529099] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529103] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529109] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529114] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529219] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529223] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529229] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529234] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529353] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529357] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529363] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529367] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529407] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529412] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529417] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529422] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529621] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529625] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529631] base=ffff0000 limit=  (null) selector=f0000fff ar=0
[  196.529636] base=ffff0000 limit=  (null) selector=f0000fff ar=0

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Mon, Jun 24, 2013 at 10:42:57PM +0200, Stefan Pietsch wrote:
> On 24.06.2013 14:30, Gleb Natapov wrote:
> > On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
> >> As soon as I remove "kvmvapic.bin" the virtual machine boots with
> >> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
> >> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
> >> no difference.
> >>
> >> Please send your patches.
> > Here it is, run with it and kvmvapic.bin present. See what is printed in
> > dmesg after the failure.
> > 
> > 
> > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> > index f4a5b3f..65488a4 100644
> > --- a/arch/x86/kvm/vmx.c
> > +++ b/arch/x86/kvm/vmx.c
> > @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >  {
> >  	struct vcpu_vmx *vmx = to_vmx(vcpu);
> >  	u32 ar;
> > +	unsigned long rip;
> >  
> >  	if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
> >  		*var = vmx->rmode.segs[seg];
> > @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >  	var->db = (ar >> 14) & 1;
> >  	var->g = (ar >> 15) & 1;
> >  	var->unusable = (ar >> 16) & 1;
> > +	rip = kvm_rip_read(vcpu);
> > +	if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
> > +		printk("base=%p limit=%p selector=%x ar=%x\n", var->base, var->limit, var->selector, ar);
> >  }
> >  
> >  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
> 
> 
> Booting kernel Linux 3.10-rc5 with your patch applied produces these
> messages in dmesg when starting a virtual machine:
> 
> emulate_invalid_guest_state=0
> [  118.732151] base=ffff0000 limit=  (null) selector=f0000fff ar=0
> [  118.732341] base=ffff0000 limit=  (null) selector=f0000fff ar=0
> 
I've butchered printk format, but it gives me the idea of what is going
on anyway. Can you try the patch below with
emulate_invalid_guest_state=0|1?


diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f4a5b3f..eb062ce 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -3395,19 +3395,20 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
 		var->selector = vmx_read_guest_seg_selector(vmx, seg);
 		return;
 	}
+
 	var->base = vmx_read_guest_seg_base(vmx, seg);
 	var->limit = vmx_read_guest_seg_limit(vmx, seg);
 	var->selector = vmx_read_guest_seg_selector(vmx, seg);
 	ar = vmx_read_guest_seg_ar(vmx, seg);
+	var->unusable = (ar >> 16) & 1;
 	var->type = ar & 15;
 	var->s = (ar >> 4) & 1;
 	var->dpl = (ar >> 5) & 3;
-	var->present = (ar >> 7) & 1;
+	var->present = !var->unusable;
 	var->avl = (ar >> 12) & 1;
 	var->l = (ar >> 13) & 1;
 	var->db = (ar >> 14) & 1;
 	var->g = (ar >> 15) & 1;
-	var->unusable = (ar >> 16) & 1;
 }
 
 static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)

--
			Gleb.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 26.06.2013 12:47, Gleb Natapov wrote:
> On Mon, Jun 24, 2013 at 10:42:57PM +0200, Stefan Pietsch wrote:
>> On 24.06.2013 14:30, Gleb Natapov wrote:
>>> On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
>>>> As soon as I remove "kvmvapic.bin" the virtual machine boots with
>>>> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
>>>> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
>>>> no difference.
>>>>
>>>> Please send your patches.
>>> Here it is, run with it and kvmvapic.bin present. See what is printed in
>>> dmesg after the failure.
>>>
>>>
>>> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
>>> index f4a5b3f..65488a4 100644
>>> --- a/arch/x86/kvm/vmx.c
>>> +++ b/arch/x86/kvm/vmx.c
>>> @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>>>  {
>>>  	struct vcpu_vmx *vmx = to_vmx(vcpu);
>>>  	u32 ar;
>>> +	unsigned long rip;
>>>  
>>>  	if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
>>>  		*var = vmx->rmode.segs[seg];
>>> @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>>>  	var->db = (ar >> 14) & 1;
>>>  	var->g = (ar >> 15) & 1;
>>>  	var->unusable = (ar >> 16) & 1;
>>> +	rip = kvm_rip_read(vcpu);
>>> +	if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
>>> +		printk("base=%p limit=%p selector=%x ar=%x\n", var->base, var->limit, var->selector, ar);
>>>  }
>>>  
>>>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
>>
>>
>> Booting kernel Linux 3.10-rc5 with your patch applied produces these
>> messages in dmesg when starting a virtual machine:
>>
>> emulate_invalid_guest_state=0
>> [  118.732151] base=ffff0000 limit=  (null) selector=f0000fff ar=0
>> [  118.732341] base=ffff0000 limit=  (null) selector=f0000fff ar=0
>>
> I've butchered printk format, but it gives me the idea of what is going
> on anyway. Can you try the patch below with
> emulate_invalid_guest_state=0|1?
> 
> 
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index f4a5b3f..eb062ce 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -3395,19 +3395,20 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>  		var->selector = vmx_read_guest_seg_selector(vmx, seg);
>  		return;
>  	}
> +
>  	var->base = vmx_read_guest_seg_base(vmx, seg);
>  	var->limit = vmx_read_guest_seg_limit(vmx, seg);
>  	var->selector = vmx_read_guest_seg_selector(vmx, seg);
>  	ar = vmx_read_guest_seg_ar(vmx, seg);
> +	var->unusable = (ar >> 16) & 1;
>  	var->type = ar & 15;
>  	var->s = (ar >> 4) & 1;
>  	var->dpl = (ar >> 5) & 3;
> -	var->present = (ar >> 7) & 1;
> +	var->present = !var->unusable;
>  	var->avl = (ar >> 12) & 1;
>  	var->l = (ar >> 13) & 1;
>  	var->db = (ar >> 14) & 1;
>  	var->g = (ar >> 15) & 1;
> -	var->unusable = (ar >> 16) & 1;
>  }
>  
>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)


Kernel 3.10-rc5 with your latest patch applied can successfully boot the
virtual machine with emulate_invalid_guest_state 0 or 1.

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 27, 2013 at 04:09:50PM +0200, Stefan Pietsch wrote:
> On 26.06.2013 12:47, Gleb Natapov wrote:
> > On Mon, Jun 24, 2013 at 10:42:57PM +0200, Stefan Pietsch wrote:
> >> On 24.06.2013 14:30, Gleb Natapov wrote:
> >>> On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
> >>>> As soon as I remove "kvmvapic.bin" the virtual machine boots with
> >>>> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
> >>>> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
> >>>> no difference.
> >>>>
> >>>> Please send your patches.
> >>> Here it is, run with it and kvmvapic.bin present. See what is printed in
> >>> dmesg after the failure.
> >>>
> >>>
> >>> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> >>> index f4a5b3f..65488a4 100644
> >>> --- a/arch/x86/kvm/vmx.c
> >>> +++ b/arch/x86/kvm/vmx.c
> >>> @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >>>  {
> >>>  	struct vcpu_vmx *vmx = to_vmx(vcpu);
> >>>  	u32 ar;
> >>> +	unsigned long rip;
> >>>  
> >>>  	if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
> >>>  		*var = vmx->rmode.segs[seg];
> >>> @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >>>  	var->db = (ar >> 14) & 1;
> >>>  	var->g = (ar >> 15) & 1;
> >>>  	var->unusable = (ar >> 16) & 1;
> >>> +	rip = kvm_rip_read(vcpu);
> >>> +	if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
> >>> +		printk("base=%p limit=%p selector=%x ar=%x\n", var->base, var->limit, var->selector, ar);
> >>>  }
> >>>  
> >>>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
> >>
> >>
> >> Booting kernel Linux 3.10-rc5 with your patch applied produces these
> >> messages in dmesg when starting a virtual machine:
> >>
> >> emulate_invalid_guest_state=0
> >> [  118.732151] base=ffff0000 limit=  (null) selector=f0000fff ar=0
> >> [  118.732341] base=ffff0000 limit=  (null) selector=f0000fff ar=0
> >>
> > I've butchered printk format, but it gives me the idea of what is going
> > on anyway. Can you try the patch below with
> > emulate_invalid_guest_state=0|1?
> > 
> > 
> > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> > index f4a5b3f..eb062ce 100644
> > --- a/arch/x86/kvm/vmx.c
> > +++ b/arch/x86/kvm/vmx.c
> > @@ -3395,19 +3395,20 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >  		var->selector = vmx_read_guest_seg_selector(vmx, seg);
> >  		return;
> >  	}
> > +
> >  	var->base = vmx_read_guest_seg_base(vmx, seg);
> >  	var->limit = vmx_read_guest_seg_limit(vmx, seg);
> >  	var->selector = vmx_read_guest_seg_selector(vmx, seg);
> >  	ar = vmx_read_guest_seg_ar(vmx, seg);
> > +	var->unusable = (ar >> 16) & 1;
> >  	var->type = ar & 15;
> >  	var->s = (ar >> 4) & 1;
> >  	var->dpl = (ar >> 5) & 3;
> > -	var->present = (ar >> 7) & 1;
> > +	var->present = !var->unusable;
> >  	var->avl = (ar >> 12) & 1;
> >  	var->l = (ar >> 13) & 1;
> >  	var->db = (ar >> 14) & 1;
> >  	var->g = (ar >> 15) & 1;
> > -	var->unusable = (ar >> 16) & 1;
> >  }
> >  
> >  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
> 
> 
> Kernel 3.10-rc5 with your latest patch applied can successfully boot the
> virtual machine with emulate_invalid_guest_state 0 or 1.
Excellent. I will send the patch to the list. Thank you for your help
in tracking it.

--
			Gleb.