Another migration question

Another migration question

[Stefan Berger]

Hi!

  I had a look at the migration code in the kernel 
(arch/xen/kernel/reboot:__do_suspend()). I am surprised that migration 
actually seems possible when the block device frontend has mounted a 
device. Shouldn't it rather refuse to be suspended, assuming that the 
partition can be migrated to another machine and possibly harm a 
filesystem there? I would also think that there should be a user-level 
daemon trying to unmount hard drive partitons before any migration is 
initited. I suppose the same problem will arise with the USB driver.

Regards,
    Stefan Berger



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: Another migration question

[Keir Fraser]

On 17 Mar 2005, at 04:13, Stefan Berger wrote:

>   I had a look at the migration code in the kernel
> (arch/xen/kernel/reboot:__do_suspend()). I am surprised that migration
> actually seems possible when the block device frontend has mounted a
> device. Shouldn't it rather refuse to be suspended, assuming that the
> partition can be migrated to another machine and possibly harm a
> filesystem there? I would also think that there should be a user-level
> daemon trying to unmount hard drive partitons before any migration is
> initited. I suppose the same problem will arise with the USB driver.

The guest you are migrating should have only location-transparent 
device connections (e.g, network-attached storage). Then the device 
connections are remade during the resume process as if nothing happened 
(from the pov of the rest of the OS).

Punting the suspend request up to user space so that administrators can 
introduce appropriate local customisations would not be that hard.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: Another migration question

[M.A. Williamson]

>   I had a look at the migration code in the kernel 
> (arch/xen/kernel/reboot:__do_suspend()). I am surprised that migration 
> actually seems possible when the block device frontend has mounted a 
> device. Shouldn't it rather refuse to be suspended, assuming that the 
> partition can be migrated to another machine and possibly harm a 
> filesystem there? I would also think that there should be a user-level 
> daemon trying to unmount hard drive partitons before any migration is 
> initited. I suppose the same problem will arise with the USB driver.

The idea is that the block device should be available at the destination 
(e.g. by using some network storage system, SAN or other mechanism). The 
device channel can then just reconnect and carry on.

USB is a little trickier because there isn't a straightforward way to 
ensure the guest can still access the device after the migration nor to 
make the process transparent to the guest USB stack. For this reason, USB 
doesn't support suspend, so you can't suspend a USB frontend domain. We can 
probably do a bit better e.g. fake out port disconnects on suspend then 
allow a device reconnect on resume - it won't be transparent but it'll be 
more useful.

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: Another migration question

[Harry Butterworth]

On Thu, 2005-03-17 at 14:31 +0000, M.A. Williamson wrote:
> USB is a little trickier because there isn't a straightforward way to 
> ensure the guest can still access the device after the migration nor to 
> make the process transparent to the guest USB stack. For this reason, USB 
> doesn't support suspend, so you can't suspend a USB frontend domain. We can 
> probably do a bit better e.g. fake out port disconnects on suspend then 
> allow a device reconnect on resume - it won't be transparent but it'll be 
> more useful.

If the inter-domain communication interface was network transparent it
would allow you to migrate a domain within a cluster for load-balancing
whilst leaving the usb devices where they were.

Harry.



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: Another migration question

[Jacob Gorm Hansen]

Harry Butterworth wrote:
> On Thu, 2005-03-17 at 14:31 +0000, M.A. Williamson wrote:
> 
>>USB is a little trickier because there isn't a straightforward way to 
>>ensure the guest can still access the device after the migration nor to 
>>make the process transparent to the guest USB stack. For this reason, USB 
>>doesn't support suspend, so you can't suspend a USB frontend domain. We can 
>>probably do a bit better e.g. fake out port disconnects on suspend then 
>>allow a device reconnect on resume - it won't be transparent but it'll be 
>>more useful.
> 
> 
> If the inter-domain communication interface was network transparent it
> would allow you to migrate a domain within a cluster for load-balancing
> whilst leaving the usb devices where they were.

There is an argument against making any of this transparent. If you 
migrate stuff somewhere, you would like to be independent of the 
originating machine once you arrive. Network transparent IPC/RPC, 
filesystems, and process migration, were all hot topics in systems 
research some 10-15 years ago, and, apart from NFS (which is now being 
replaced by iSCSI) none of these fancy transparent schemes survived in 
the real world.

I've done some test with self-migration (the right way to do all of this 
IMHO) using software RAID-1. In this setup it is trivial to mirror a 
block device onto the target node, and when the mirror is synced up you 
migrate the OS itself. I guess it is not currently possible to upgrade a 
remote iSCSI mount to a local Xen-block device mount.

There is no need to forward any migration events into userspace, because 
userspace is what is controlling the whole thing anyway. The TCP 
connection is opened and fed from userspace, the ARP reply is sent from 
userspace, even the protocol implementation running at the remote end is 
uploaded from userspace.

You could argue that self-migration, implemented inside XenoLinux, is 
less portable. However, I recently counted, and it seems that today 
there are more different VMMs than there are operating systems.

But that is just my view. I am likely to be wrong, or at least suffering 
from the 'second system effect' ;-).

Jacob


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: Another migration question

[Eric S. Johansson]

Jacob Gorm Hansen wrote:
> There is an argument against making any of this transparent. If you 
> migrate stuff somewhere, you would like to be independent of the 
> originating machine once you arrive. Network transparent IPC/RPC, 
> filesystems, and process migration, were all hot topics in systems 
> research some 10-15 years ago, and, apart from NFS (which is now being 
> replaced by iSCSI) none of these fancy transparent schemes survived in 
> the real world.

from an operational perspective, I could not agree with this more.  If 
the task is moving from machine A to machine B, I am comfortable with 
specifying everything *as long as* there exists a tool to compare source 
against destination and point out to me what is different.

for example, if the source requires three disk images, I should be able 
to clear the system and be that three disk images are necessary.  I also 
should be able to change the size of the destination images and even the 
file system (in theory) used.

in other words, there's no crime in not automating as long as sufficient 
information can be provided in a way that leads the user to a reasonably 
correct solution.

as for the USB issue, moving from one system to the other is exactly the 
same as unplugging the device.  get out of the chair and move the 
device.  No need for fancy technological solutions there.  ;-)

> But that is just my view. I am likely to be wrong, or at least suffering 
> from the 'second system effect' ;-).

having acquired much scar tissue in the OS realm almost 20 years past, I 
don't believe you are suffering from second system effect.

---eric


-- 
http://www.wired.com/wired/archive/13.03/view.html?pg=5

The result of the duopoly that currently defines "competition" is that
prices and service suck. We're the world's leader in Internet
technology - except that we're not.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

RE: Another migration question

[Ian Pratt]

>   I had a look at the migration code in the kernel 
> (arch/xen/kernel/reboot:__do_suspend()). I am surprised that 
> migration 
> actually seems possible when the block device frontend has mounted a 
> device. Shouldn't it rather refuse to be suspended, assuming that the 
> partition can be migrated to another machine and possibly harm a 
> filesystem there? 

We could build in more idiot proofing into xend, but the current
situation works fine for someone that knows what they're doing. 

For migration to work it obviously has to be possible to access the old
block device in the new place. You can do this with a SAN, iSCSI, GNBD,
drdb etc.

> I would also think that there should be a 
> user-level 
> daemon trying to unmount hard drive partitons before any migration is 
> initited. I suppose the same problem will arise with the USB driver.

It's not possible to unmount your root filesystem. I guess we could put
a few sanity checks in to ensure that the contents of the device after
the migrate appears to be the same as before, but this is non-trivial
and could potetnailly trigger false positives e.g. in the case of GFS or
OCFS2.

The next generation tools might provide more saftey checks to help stop
people shooting their own feet...

Ian




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click