-smp >=2 + -no-kvm-irqchip broken

-smp >=2 + -no-kvm-irqchip broken

[Jan Kiszka]

Hi,

notice while testing v2 of my vcpu state series: Starting SMP guests
like Linux or Vista without in-kernel irqchip and with more than one CPU
make them lock up during boot (Vista) or spit out messages like

udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
in libc-2.9.so[7fc4345b2000+14f000]

Affected are all recent versions (stable-0.11 didn't even start the
bios, probably due to a different historic bug).

Any idea?

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: -smp >=2 + -no-kvm-irqchip broken

[Gleb Natapov]

On Tue, Feb 02, 2010 at 05:29:53PM +0100, Jan Kiszka wrote:
> Hi,
> 
> notice while testing v2 of my vcpu state series: Starting SMP guests
> like Linux or Vista without in-kernel irqchip and with more than one CPU
> make them lock up during boot (Vista) or spit out messages like
> 
> udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
> in libc-2.9.so[7fc4345b2000+14f000]
> 
Looks like interrupt is injected as exception. Can you try to bisect?

--
			Gleb.

Re: -smp >=2 + -no-kvm-irqchip broken

[Jan Kiszka]

Gleb Natapov wrote:
> On Tue, Feb 02, 2010 at 05:29:53PM +0100, Jan Kiszka wrote:
>> Hi,
>>
>> notice while testing v2 of my vcpu state series: Starting SMP guests
>> like Linux or Vista without in-kernel irqchip and with more than one CPU
>> make them lock up during boot (Vista) or spit out messages like
>>
>> udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
>> in libc-2.9.so[7fc4345b2000+14f000]
>>
> Looks like interrupt is injected as exception. Can you try to bisect?

Already started (as low prio background task), first looking for a good
version. If you know a recent one, I'm a taker.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: -smp >=2 + -no-kvm-irqchip broken

[Gleb Natapov]

On Tue, Feb 02, 2010 at 05:41:03PM +0100, Jan Kiszka wrote:
> Gleb Natapov wrote:
> > On Tue, Feb 02, 2010 at 05:29:53PM +0100, Jan Kiszka wrote:
> >> Hi,
> >>
> >> notice while testing v2 of my vcpu state series: Starting SMP guests
> >> like Linux or Vista without in-kernel irqchip and with more than one CPU
> >> make them lock up during boot (Vista) or spit out messages like
> >>
> >> udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
> >> in libc-2.9.so[7fc4345b2000+14f000]
> >>
> > Looks like interrupt is injected as exception. Can you try to bisect?
> 
> Already started (as low prio background task), first looking for a good
> version. If you know a recent one, I'm a taker.
> 

Nope, didn't try no kernel irqchip for a long time.

--
			Gleb.

Re: -smp >=2 + -no-kvm-irqchip broken

[Jan Kiszka]

Gleb Natapov wrote:
> On Tue, Feb 02, 2010 at 05:41:03PM +0100, Jan Kiszka wrote:
>> Gleb Natapov wrote:
>>> On Tue, Feb 02, 2010 at 05:29:53PM +0100, Jan Kiszka wrote:
>>>> Hi,
>>>>
>>>> notice while testing v2 of my vcpu state series: Starting SMP guests
>>>> like Linux or Vista without in-kernel irqchip and with more than one CPU
>>>> make them lock up during boot (Vista) or spit out messages like
>>>>
>>>> udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
>>>> in libc-2.9.so[7fc4345b2000+14f000]
>>>>
>>> Looks like interrupt is injected as exception. Can you try to bisect?
>> Already started (as low prio background task), first looking for a good
>> version. If you know a recent one, I'm a taker.
>>
> 
> Nope, didn't try no kernel irqchip for a long time.

Kernel plays the key role here, either kvm-kmod or actually the kvm
module: kvm-kmod-2.6.31.6 works, the 2.6.32 series and later do not.
Digging deeper...

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: -smp >=2 + -no-kvm-irqchip broken

[Jan Kiszka]

Jan Kiszka wrote:
> Gleb Natapov wrote:
>> On Tue, Feb 02, 2010 at 05:41:03PM +0100, Jan Kiszka wrote:
>>> Gleb Natapov wrote:
>>>> On Tue, Feb 02, 2010 at 05:29:53PM +0100, Jan Kiszka wrote:
>>>>> Hi,
>>>>>
>>>>> notice while testing v2 of my vcpu state series: Starting SMP guests
>>>>> like Linux or Vista without in-kernel irqchip and with more than one CPU
>>>>> make them lock up during boot (Vista) or spit out messages like
>>>>>
>>>>> udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
>>>>> in libc-2.9.so[7fc4345b2000+14f000]
>>>>>
>>>> Looks like interrupt is injected as exception. Can you try to bisect?
>>> Already started (as low prio background task), first looking for a good
>>> version. If you know a recent one, I'm a taker.
>>>
>> Nope, didn't try no kernel irqchip for a long time.
> 
> Kernel plays the key role here, either kvm-kmod or actually the kvm
> module: kvm-kmod-2.6.31.6 works, the 2.6.32 series and later do not.
> Digging deeper...

I take this back: It works for Linux guests, but Vista crashes also with
old kvm-kmod. So there is no way around real debugging.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: -smp >=2 + -no-kvm-irqchip broken

[Gleb Natapov]

On Tue, Feb 02, 2010 at 07:56:42PM +0100, Jan Kiszka wrote:
> Jan Kiszka wrote:
> > Gleb Natapov wrote:
> >> On Tue, Feb 02, 2010 at 05:41:03PM +0100, Jan Kiszka wrote:
> >>> Gleb Natapov wrote:
> >>>> On Tue, Feb 02, 2010 at 05:29:53PM +0100, Jan Kiszka wrote:
> >>>>> Hi,
> >>>>>
> >>>>> notice while testing v2 of my vcpu state series: Starting SMP guests
> >>>>> like Linux or Vista without in-kernel irqchip and with more than one CPU
> >>>>> make them lock up during boot (Vista) or spit out messages like
> >>>>>
> >>>>> udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
> >>>>> in libc-2.9.so[7fc4345b2000+14f000]
> >>>>>
> >>>> Looks like interrupt is injected as exception. Can you try to bisect?
> >>> Already started (as low prio background task), first looking for a good
> >>> version. If you know a recent one, I'm a taker.
> >>>
> >> Nope, didn't try no kernel irqchip for a long time.
> > 
> > Kernel plays the key role here, either kvm-kmod or actually the kvm
> > module: kvm-kmod-2.6.31.6 works, the 2.6.32 series and later do not.
> > Digging deeper...
> 
> I take this back: It works for Linux guests, but Vista crashes also with
> old kvm-kmod. So there is no way around real debugging.
> 
Upstream qemu-kvm/kvm works for me. Boot linux with -smp 2 and
windows2008-32 with -smp 4.

--
			Gleb.

Re: -smp >=2 + -no-kvm-irqchip broken

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1764 bytes --]

Gleb Natapov wrote:
> On Tue, Feb 02, 2010 at 07:56:42PM +0100, Jan Kiszka wrote:
>> Jan Kiszka wrote:
>>> Gleb Natapov wrote:
>>>> On Tue, Feb 02, 2010 at 05:41:03PM +0100, Jan Kiszka wrote:
>>>>> Gleb Natapov wrote:
>>>>>> On Tue, Feb 02, 2010 at 05:29:53PM +0100, Jan Kiszka wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> notice while testing v2 of my vcpu state series: Starting SMP guests
>>>>>>> like Linux or Vista without in-kernel irqchip and with more than one CPU
>>>>>>> make them lock up during boot (Vista) or spit out messages like
>>>>>>>
>>>>>>> udevd[112] trap invalid opcode ip:7fc434630950 sp:7fff3cd3fcb8 error:0
>>>>>>> in libc-2.9.so[7fc4345b2000+14f000]
>>>>>>>
>>>>>> Looks like interrupt is injected as exception. Can you try to bisect?
>>>>> Already started (as low prio background task), first looking for a good
>>>>> version. If you know a recent one, I'm a taker.
>>>>>
>>>> Nope, didn't try no kernel irqchip for a long time.
>>> Kernel plays the key role here, either kvm-kmod or actually the kvm
>>> module: kvm-kmod-2.6.31.6 works, the 2.6.32 series and later do not.
>>> Digging deeper...
>> I take this back: It works for Linux guests, but Vista crashes also with
>> old kvm-kmod. So there is no way around real debugging.
>>
> Upstream qemu-kvm/kvm works for me. Boot linux with -smp 2 and
> windows2008-32 with -smp 4.

We seem to have two issues:
 - kvm.git master breaks 64-bit linux guests under -no-kvm-irqchip +
   smp (#UD pop up in guest user space). 2.6.33 kvm is fine. Will try to
   bisect this the next days.
 - My Vista x64 image apparently suffer from a race (I suspect in user
   space) that lets it deadlock on a spinlock, independent of the
   underlying kernel kvm version.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]