problem about changing state to XenbusStateClosed resulting in vbd entry removed from xenstore

problem about changing state to XenbusStateClosed resulting in vbd entry removed from xenstore

[Max Zhen]

Hi,

I encountered a problem while porting Solaris as a guest OS on Xen.

The problem I found is that when frontend vbd driver get unloaded, it
will close the vbd interface by changing the state to XenbusStateClosed.
This appears to cause the dom0 hotplug scripts to remove the vbd entry 
from xenstore for the domU guest, preventing the vbd from being re-attached.

So, my questions are:

+ This doesn't seem correct since vbd's are created and destroyed from 
Dom0 as part of domain creation 'xm create' or explicitly by executing
'xm block-attach'. Therefore, it would seem that devices should be
removed either when a domU terminates or the device is explicitly
removed from dom0 by 'xm block-detach'.

+ Is there any other way to flush all the I/O to the disk?
Currently, the only way to flush the I/Os is to change the frontend
state to XenbusStateClosed.
Since changing the state to XenbusStateClosed is a dangerous thing to do
(cause all the vbd interface information to be removed), I cannot just
flush the I/O, while keep the frontend and backend connected.
Could there be any new state or command to do that?

Thanks,
Max

Re: problem about changing state to XenbusStateClosed resulting in vbd entry removed from xenstore

[Ewan Mellor]

On Fri, May 19, 2006 at 12:26:32AM +0800, Max Zhen wrote:

> Hi,
> 
> I encountered a problem while porting Solaris as a guest OS on Xen.
> 
> The problem I found is that when frontend vbd driver get unloaded, it
> will close the vbd interface by changing the state to XenbusStateClosed.
> This appears to cause the dom0 hotplug scripts to remove the vbd entry 
> from xenstore for the domU guest, preventing the vbd from being re-attached.
>
> So, my questions are:
> 
> + This doesn't seem correct since vbd's are created and destroyed from 
> Dom0 as part of domain creation 'xm create' or explicitly by executing
> 'xm block-attach'. Therefore, it would seem that devices should be
> removed either when a domU terminates or the device is explicitly
> removed from dom0 by 'xm block-detach'.

You could argue that you ought to be able to remove the frontend driver
module, and then load that module again and have all the devices reconnect.
I've no idea whether that would have knock-on consequences though.

> + Is there any other way to flush all the I/O to the disk?
> Currently, the only way to flush the I/Os is to change the frontend
> state to XenbusStateClosed.
> Since changing the state to XenbusStateClosed is a dangerous thing to do
> (cause all the vbd interface information to be removed), I cannot just
> flush the I/O, while keep the frontend and backend connected.
> Could there be any new state or command to do that?

A block-detach should be switching the backend to XenbusStateClosing, which
the frontend will observe, allowing it to flush remaining I/O.  Only when the
frontend is done should it switch to Closed, which will then be seen by the
backend and then the backend can finish up and close itself.

Are you not seeing this?

Cheers,

Ewan.

Re: problem about changing state to XenbusStateClosed resulting in vbd entry removed from xenstore

[Keir Fraser]

On 19 May 2006, at 00:00, Ewan Mellor wrote:

>> + This doesn't seem correct since vbd's are created and destroyed from
>> Dom0 as part of domain creation 'xm create' or explicitly by executing
>> 'xm block-attach'. Therefore, it would seem that devices should be
>> removed either when a domU terminates or the device is explicitly
>> removed from dom0 by 'xm block-detach'.
>
> You could argue that you ought to be able to remove the frontend driver
> module, and then load that module again and have all the devices 
> reconnect.
> I've no idea whether that would have knock-on consequences though.

I think we really ought to be able to support this, but it hardly 
sounds a priority. OTOH if it's simply a case of stopping the hotplug 
scripts from zapping the xenstore entries, and instead do that in the 
domain-destruction and vbd-remove paths of xend, then maybe it's not so 
hard to fix. Anyway, I'd certainly consider the current behaviour to be 
a bug, albeit perhaps a minor one.

  -- Keir

Re: problem about changing state to XenbusStateClosed resulting in vbd entry removed from xenstore

[Max Zhen]

>>+ Is there any other way to flush all the I/O to the disk?
>>Currently, the only way to flush the I/Os is to change the frontend
>>state to XenbusStateClosed.
>>Since changing the state to XenbusStateClosed is a dangerous thing to do
>>(cause all the vbd interface information to be removed), I cannot just
>>flush the I/O, while keep the frontend and backend connected.
>>Could there be any new state or command to do that?
>>    
>>
>
>A block-detach should be switching the backend to XenbusStateClosing, which
>the frontend will observe, allowing it to flush remaining I/O.  Only when the
>frontend is done should it switch to Closed, which will then be seen by the
>backend and then the backend can finish up and close itself.
>
>Are you not seeing this?
>  
>
Yes, I've seen this.
But, the key point is that how to flush the I/O without disconnecting 
between frontend and backend.
It seems to me that I/O flushing is more like a command, say, 
BLKIF_OP_FLUSH, than changing the driver to various states, especially 
XenbusStateClosing/Closed.

Any idea?

Thanks,
Max

>Cheers,
>
>Ewan.
>  
>