blktap3: libxl complaining when a domain gets destroyed

blktap3: libxl complaining when a domain gets destroyed

[Thanos Makatos]

I'm testing blktap3 and when I destroy a domain via "xl destroy 21" I get the following behaviour: "xl destroy 21" seems to pause for a couple of seconds and then spits out the following:
libxl: error: libxl_device.c:882:device_backend_callback: unable to remove device with path /local/domain/0/backend/vbd3/21/51712
libxl: error: libxl.c:1453:devices_destroy_cb: libxl__devices_destroy failed for 21
Apart from these messages, the domain seems to be correctly destroyed and there aren't any leftovers in XenStore. I'm able to successfully start and stop the domain multiple times. What is libxl complaining about?

--
Thanos Makatos

Re: blktap3: libxl complaining when a domain gets destroyed

[Ian Campbell]

On Wed, 2013-04-24 at 12:46 +0100, Thanos Makatos wrote:
> I'm testing blktap3 and when I destroy a domain via "xl destroy 21" I
> get the following behaviour: "xl destroy 21" seems to pause for a
> couple of seconds and then spits out the following:
> libxl: error: libxl_device.c:882:device_backend_callback: unable to
> remove device with path /local/domain/0/backend/vbd3/21/51712
> libxl: error: libxl.c:1453:devices_destroy_cb: libxl__devices_destroy
> failed for 21
> Apart from these messages, the domain seems to be correctly destroyed
> and there aren't any leftovers in XenStore. I'm able to successfully
> start and stop the domain multiple times. What is libxl complaining
> about?

does xl -vvv destroy give any more hints?

The message results from a failure in the call
libxl__ev_devstate_wait(...XenbusStateClosed). Perhaps blktap3 is
cleaning up a bit too diligently and removing something which the
toolstack expects to cleanup?

Ian.

Re: blktap3: libxl complaining when a domain gets destroyed

[Roger Pau Monné]

On 24/04/13 13:46, Thanos Makatos wrote:
> I'm testing blktap3 and when I destroy a domain via "xl destroy 21" I get the following behaviour: "xl destroy 21" seems to pause for a couple of seconds and then spits out the following:
> libxl: error: libxl_device.c:882:device_backend_callback: unable to remove device with path /local/domain/0/backend/vbd3/21/51712

This is because libxl will wait for the backend to reach state '6'
(closed). Even if state 6 is not reached libxl will clean up the backend
path, you should check why blktap3 doesn't set the backend to state 6.

> libxl: error: libxl.c:1453:devices_destroy_cb: libxl__devices_destroy failed for 21
> Apart from these messages, the domain seems to be correctly destroyed and there aren't any leftovers in XenStore. I'm able to successfully start and stop the domain multiple times. What is libxl complaining about?
> 
> --
> Thanos Makatos
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: blktap3: libxl complaining when a domain gets destroyed

[Thanos Makatos]

> This is because libxl will wait for the backend to reach state '6'
> (closed). Even if state 6 is not reached libxl will clean up the
> backend path, you should check why blktap3 doesn't set the backend to
> state 6.

At some point during "xl destroy ..." the backend state is set to 5 (assuming by libxl?). I get a XenStore watch triggering on this event. So how should I handle this event? Setting the backend state to 6 fails with EPERM, which I don't understand since I do modify the backend state when going from initialised to connected etc.

Re: blktap3: libxl complaining when a domain gets destroyed

[Thanos Makatos]

> > This is because libxl will wait for the backend to reach state '6'
> > (closed). Even if state 6 is not reached libxl will clean up the
> > backend path, you should check why blktap3 doesn't set the backend to
> > state 6.
> 
> At some point during "xl destroy ..." the backend state is set to 5
> (assuming by libxl?). I get a XenStore watch triggering on this event.
> So how should I handle this event? Setting the backend state to 6 fails
> with EPERM, which I don't understand since I do modify the backend
> state when going from initialised to connected etc.

Hmm I'm looking at the code in libxl__initiate_device_remove (libxl_device.c) and I see that for qemu back-ends libxl doesn't wait for the backend to switch to state 6, it does something different that I don't understand. If I just jump to the "out" label it seems to work fine, though I don't know the semantics of the domain destruction process.