BUG: soft lockup - CPU#0 stuck for 61s!

BUG: soft lockup - CPU#0 stuck for 61s!

[Andres Lagar-Cavilla]

Ian,

I came across the subject line on a 96GB server with over 85 VMs running.
Completely frozen and unresponsive, qemu-dm processes hung on event
channels. I'm using the XenServer 6.0 dom0 kernel on top of the
xen-unstable tip hypervisor.

I believe you solved the issue backporting some event channel patches to
the 2.6.32 kernel, as described in this thread
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=638172

I am missing a bit of context in order to retrieve the patches and apply
them to my setup. What trees are involved here? I'm looking in particular
for "multiple fixes to PIRQ event channel handling" and friends, but I
don't have commit identifiers, nor a tree URL (neither I am too sure what
changesets from which upstream tree do the backports aggregate).

Any help appreciated. Thanks
Andres

Re: BUG: soft lockup - CPU#0 stuck for 61s!

[Ian Campbell]

On Thu, 2012-03-29 at 16:43 +0100, Andres Lagar-Cavilla wrote:
> Ian,
> 
> I came across the subject line on a 96GB server with over 85 VMs running.
> Completely frozen and unresponsive, qemu-dm processes hung on event
> channels. I'm using the XenServer 6.0 dom0 kernel on top of the
> xen-unstable tip hypervisor.
> 
> I believe you solved the issue backporting some event channel patches to
> the 2.6.32 kernel, as described in this thread
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=638172
> 
> I am missing a bit of context in order to retrieve the patches and apply
> them to my setup. What trees are involved here?

This was a backport to a 2.6.32 pvops kernel. The Debian Squeeze kernel
is a snapshot from Jeremy's xen.git at some point in the past.
drivers/xen/events.c is pretty different to the classic-Xen kernels like
XenServers.

>  I'm looking in particular
> for "multiple fixes to PIRQ event channel handling" and friends, but I
> don't have commit identifiers, nor a tree URL (neither I am too sure what
> changesets from which upstream tree do the backports aggregate).

I think the pirq stuff was because the aforementioned snapshot happened
to be taken in the middle of a transition in how pirqs were handled, so
I was back porting the remainder.

Anyway, if you want them you  can find them in the Debian source
package, they are listed in debian/patches/series/40-extra, the paths
there are relative to debian/patches.

Also, I think (but I'm not 100% sure any more) that these patches
correspond to this branch in my xenbits tree:

  git://xenbits.xen.org/people/ianc/linux-2.6.git debian/squeeze/evtchn

Ian

Re: BUG: soft lockup - CPU#0 stuck for 61s!

[Andres Lagar-Cavilla]

> On Thu, 2012-03-29 at 16:43 +0100, Andres Lagar-Cavilla wrote:
>> Ian,
>>
>> I came across the subject line on a 96GB server with over 85 VMs
>> running.
>> Completely frozen and unresponsive, qemu-dm processes hung on event
>> channels. I'm using the XenServer 6.0 dom0 kernel on top of the
>> xen-unstable tip hypervisor.
>>
>> I believe you solved the issue backporting some event channel patches to
>> the 2.6.32 kernel, as described in this thread
>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=638172
>>
>> I am missing a bit of context in order to retrieve the patches and apply
>> them to my setup. What trees are involved here?
>
> This was a backport to a 2.6.32 pvops kernel. The Debian Squeeze kernel
> is a snapshot from Jeremy's xen.git at some point in the past.
> drivers/xen/events.c is pretty different to the classic-Xen kernels like
> XenServers.
>
>>  I'm looking in particular
>> for "multiple fixes to PIRQ event channel handling" and friends, but I
>> don't have commit identifiers, nor a tree URL (neither I am too sure
>> what
>> changesets from which upstream tree do the backports aggregate).
>
> I think the pirq stuff was because the aforementioned snapshot happened
> to be taken in the middle of a transition in how pirqs were handled, so
> I was back porting the remainder.
>
> Anyway, if you want them you  can find them in the Debian source
> package, they are listed in debian/patches/series/40-extra, the paths
> there are relative to debian/patches.
>
> Also, I think (but I'm not 100% sure any more) that these patches
> correspond to this branch in my xenbits tree:
>
>   git://xenbits.xen.org/people/ianc/linux-2.6.git debian/squeeze/evtchn
>
Thanks very much, this will get me going
Andres

> Ian
>
>