blkback stuck in infinite loop in xen_blkbk_discard()

blkback stuck in infinite loop in xen_blkbk_discard()

[Jeremy Fitzhardinge]

I just "xl destroy"d a domain, and now my dom0 kernel is sitting there
infinitely spewing:

vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type
vbd vbd-83-51712: 5 reading type

which seems to be coming from the error case in
drivers/block/xen-blkback/xenbus.c:xen_blkbk_discard().

I don't know what the backtrace is.  The system seems overall fine,
despite the console spew, though I have a pile of dying domains sitting
in odd states rather than being cleaned up.

    J

Re: blkback stuck in infinite loop in xen_blkbk_discard()

[Jan Beulich]

>>> On 28.09.11 at 23:40, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> I just "xl destroy"d a domain, and now my dom0 kernel is sitting there
> infinitely spewing:
> 
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> vbd vbd-83-51712: 5 reading type
> 
> which seems to be coming from the error case in
> drivers/block/xen-blkback/xenbus.c:xen_blkbk_discard().
> 
> I don't know what the backtrace is.  The system seems overall fine,
> despite the console spew, though I have a pile of dying domains sitting
> in odd states rather than being cleaned up.

I wonder how the function gets called during destroy in the first place,
and how that would prevent destroying a domain (unless there's a
feedback loop due to the use of xenbus_dev_fatal() here, causing
some xenstore entry to get written over and over again, triggering
the respective watch that blkback has active).

But irrespective of this I would think the function should bail without
doing anything if blkif->blk_backend_type was already set (or
couldn't get set). But that would then also indicate a more general
problem in connect(), as that function shouldn't do anything either
when a domain gets destroyed.

Jan

Re: blkback stuck in infinite loop in xen_blkbk_discard()

[Jeremy Fitzhardinge]

On 09/29/2011 02:55 AM, Jan Beulich wrote:
>>>> On 28.09.11 at 23:40, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>> I just "xl destroy"d a domain, and now my dom0 kernel is sitting there
>> infinitely spewing:
>>
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>> vbd vbd-83-51712: 5 reading type
>>
>> which seems to be coming from the error case in
>> drivers/block/xen-blkback/xenbus.c:xen_blkbk_discard().
>>
>> I don't know what the backtrace is.  The system seems overall fine,
>> despite the console spew, though I have a pile of dying domains sitting
>> in odd states rather than being cleaned up.
> I wonder how the function gets called during destroy in the first place,
> and how that would prevent destroying a domain (unless there's a
> feedback loop due to the use of xenbus_dev_fatal() here, causing
> some xenstore entry to get written over and over again, triggering
> the respective watch that blkback has active).
>
> But irrespective of this I would think the function should bail without
> doing anything if blkif->blk_backend_type was already set (or
> couldn't get set). But that would then also indicate a more general
> problem in connect(), as that function shouldn't do anything either
> when a domain gets destroyed.

Well, it could have got into that state before I destroyed the domain -
it was a test kernel that crashed very early (before setting up its own
frontend) after booting from pvgrub, so perhaps there was some race in
that handoff.

    J