Deadload between sched_adjust() in schedule.c and compat_failsafe_callback in entry.S

Deadload between sched_adjust() in schedule.c and compat_failsafe_callback in entry.S

[hj lee]

[-- Attachment #1.1: Type: text/plain, Size: 1142 bytes --]

Hi,

We have a deadlock in dom0 running X86_64 CentOs 5.2 when dom0 runs libvirtd
and xentop together, this deadlock is easily reproducible. The dom0 has four
vcpus assigned. The libvirt is running on vcpu#0 and xentop is running on
vcpu#3. The vcpu#0 is processing XEN_DOMCTL_scheduler_op of domctl.c which
calls sched_adjust(). The sched_adjust() calls vcpu_pause(v) for each vcpu
in the domain, and vcpu_pause(v) calls vcpu_sleep_sync(v) where it waits for
vcpu#3 pause. On the other hand vcpu#3 is executing vcpu_runstate_get() in
schedule.c called from XEN_SYSCTL_getdomaininfolist in sysctl.c. At the time
of deadlock somehow this vcpu#3's exception RIP is pointing
[compat_failsafe_callback+86], which is cmpb $0x0,87987(%rip)    #
0xffff828c8019ef00 <domctl_lock.10183>. I am not sure how vcpu#3 gets into
this code, but what I believe it is trying to get the spinlock on
domctl_lock. But vcpu#0 had a lock on the domctl_lock when it enters
do_comctl(). So two vcpus are in deadlock.

Can anybody explain how and when compat_failsafe_callback in entry.S is get
called? Why does it try to get a lock on domctl_lock?

Thanks in advance

[-- Attachment #1.2: Type: text/html, Size: 1179 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Deadload between sched_adjust() in schedule.c and compat_failsafe_callback in entry.S

[Keir Fraser]

[-- Attachment #1.1: Type: text/plain, Size: 1296 bytes --]

This should be easily fixable. I¹ll look into it.

 -- Keir

On 08/04/2009 06:41, "hj lee" <kerdosa@gmail.com> wrote:

> We have a deadlock in dom0 running X86_64 CentOs 5.2 when dom0 runs libvirtd
> and xentop together, this deadlock is easily reproducible. The dom0 has four
> vcpus assigned. The libvirt is running on vcpu#0 and xentop is running on
> vcpu#3. The vcpu#0 is processing XEN_DOMCTL_scheduler_op of domctl.c which
> calls sched_adjust(). The sched_adjust() calls vcpu_pause(v) for each vcpu in
> the domain, and vcpu_pause(v) calls vcpu_sleep_sync(v) where it waits for
> vcpu#3 pause. On the other hand vcpu#3 is executing vcpu_runstate_get() in
> schedule.c called from XEN_SYSCTL_getdomaininfolist in sysctl.c. At the time
> of deadlock somehow this vcpu#3's exception RIP is pointing
> [compat_failsafe_callback+86], which is cmpb $0x0,87987(%rip)    #
> 0xffff828c8019ef00 <domctl_lock.10183>. I am not sure how vcpu#3 gets into
> this code, but what I believe it is trying to get the spinlock on domctl_lock.
> But vcpu#0 had a lock on the domctl_lock when it enters do_comctl(). So two
> vcpus are in deadlock.
> 
> Can anybody explain how and when compat_failsafe_callback in entry.S is get
> called? Why does it try to get a lock on domctl_lock?


[-- Attachment #1.2: Type: text/html, Size: 1727 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Deadload between sched_adjust() in schedule.c and compat_failsafe_callback in entry.S

[hj lee]

[-- Attachment #1.1: Type: text/plain, Size: 238 bytes --]

Now I figured out that this is not a C code error, it is linker error!

Thanks

On Wed, Apr 8, 2009 at 12:03 AM, Keir Fraser <keir.fraser@eu.citrix.com>wrote:

>  This should be easily fixable. I’ll look into it.
>
>  -- Keir
>

[-- Attachment #1.2: Type: text/html, Size: 670 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Deadload between sched_adjust() in schedule.c and compat_failsafe_callback in entry.S

[hj lee]

[-- Attachment #1.1: Type: text/plain, Size: 386 bytes --]

I was wrong about link error, there is no link error. The changeset 19519 by
Keir was committed in to xen-unstable.hq to fix this deadlock.

Thank you very much


>
> On Wed, Apr 8, 2009 at 12:03 AM, Keir Fraser <keir.fraser@eu.citrix.com>wrote:
>
>>  This should be easily fixable. I’ll look into it.
>>
>>  -- Keir
>>
>
>


-- 
Dream with longterm vision!
kerdosa

[-- Attachment #1.2: Type: text/html, Size: 1058 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel