Re: Re: No shutdown of domU: xenbus_dev_shutdown: device/console/0: Initialising != Connected, skipping

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 11/18/2010 11:57 AM, Sander Eikelenboom wrote:
> Hi Jeremy/Keir,
>
> I have tried to add the following lines:
>        if (!strcmp(type, "console"))
>                return 0;
>
> But it seems to be a red herring .. although the "the xenbus_dev_shutdown: device/console/0: Initialising != Connected, skipping" warning is gone.
> The PV domU guest still doesn't get completely halted:
> - It stays in a running state
> - It uses 100% cpu according to xentop
> - I can still connect to it's console
>
> So probably something is stuck in a waiting loop, which doesn't sleep in between and consumes 100% cpu.

Hm, I generally see clean shutdowns of my multi-VCPU domains.

Can you do

# gdbsx -a <domid> <arch-size> 9999        (arch-size is 32 or 64)

then

$ gdb vmlinux
(gdb) tar rem :9999
(gdb) thread apply all bt

to see what's going on?  (You may need to enable CONFIG_DEBUG_INFO to
get useful info if you haven't already.)

    J


> Last lines of domU's console(with some additional printk's in xenbus_dev_shutdown):
>
> Cannot access the Hardware Clock via any known method.
> Use the --debug option to see the details of our search for an access method.
> Stopping enhanced syslogd: rsyslogd.
> Asking all remaining processes to terminate...done.
> All processes ended within 2 seconds....done.
> Deconfiguring network interfaces...done.
> Cleaning up ifupdown....
> Deactivating swap...done.
> Unmounting local filesystems...done.
> Will now halt.
> [   46.643035] md: stopping all md devices.
> [   47.643320] xenbus_dev_shutdown:  trying shutdown of device/vif/0: Connected
> [   47.716448] xenbus_dev_shutdown: result of shutdown of device/vif/0: Closed
> [   47.716461] xenbus_dev_shutdown:  trying shutdown of device/vbd/51714: Connected
> [   47.772293] xenbus_dev_shutdown: result of shutdown of device/vbd/51714: Closed
> [   47.772306] xenbus_dev_shutdown:  trying shutdown of device/vbd/51713: Connected
> [   47.829384] xenbus_dev_shutdown: result of shutdown of device/vbd/51713: Closed
> [   47.829415] System halted.
>
>
>
> it's a domU using a 2.6.37-rc2 kernel, file: based disk access, nothing very special.
>
> the only difference that triggers it is:
> vcpus=1 works fine ..
> vcpus>1 symptoms above ..
>
> I have also added xend.log, with first a boot and shutdown with vpcus=1 and then a boot and shutdown with vpcus=4
> I left the domain(with 4vcpus) running with 100% cpu for 10 minutes, but doesn't seem to hit any timeout, and keeps running.
> On 20:40 it ends up stuck, on 20:51 i'm shooting the domain off with xm destroy.
>
>
> Any pointers for adding extra debug info ?
>
> --
> Sander
>
>
>
>
> Wednesday, November 17, 2010, 10:57:47 PM, you wrote:
>
>> On 11/17/2010 12:22 PM, Sander Eikelenboom wrote:
>>> Ah yes, 2.6.18 does contain the following line  in xenbus_probe.c and 2.6.37-rc2 not:
>>>
>>> -       if (!strcmp(type, "console"))
>>> -               return 0;
>>> It seems to be changed,
>>>
>>>
>>> There seems to have been a patch http://lists.xensource.com/archives/html/xen-devel/2009-11/msg01072.html but it seems it's not applied to xenbus in linux upstream.
>> And does this patch fix things for you?
>> But I have to say that's pretty gross.  Is it really the right thing to do?
>>     J
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel