[RFD] External device migration for block devices

[RFD] External device migration for block devices

[Michael Paesold]

Hi everyone, hi Stefan!

I want to use Stefan's work on device migration to implement 
block-device migration for devices, where the regular model (hotplug 
add/remove) does not work, i.e. when the device migration script is 
required.
I would like to get some feedback on my ideas.

1) Currently, the device migration script does not get device details 
for the device it should migrate, obviously because it's not needed for 
vtpm. On the other hand, there can be several block devices for each 
domain, and each might have to be handled differently. So the script 
needs to get the details for the device it should migrate.

Should I add a arguments to the device migration script, that will not 
be set for vtpm?

2) I think there should be some way to configure the migration 
capabilities of block devices using the domain config files. Some block 
device might be migrated automatically using regular hotplug add/remove 
(e.g. nbd/enbd). Others might not be migratable at all (e.g., 
phy:/dev/sdb). For my current use case (DRBD), I use phy:/dev/drbd. 
These devices are migratable, but Xend does not know this.

I don't think it is good if all this knowledge must be put into the 
migration scripts itself. So I suggest to add a device config option to 
explicitly tell Xend what to do on migrate.

I suggest adding optional parameters to the end of the block device 
definition, which is currently "phy:UNAME,DEV,MODE"; I would extend that 
to "phy:UNAME,DEV,MODE[,OPTIONS]".

Examples:
phy:sdb,xvda,w,migrate=reject  # reject migration
nbd:sample,xvda1,r,migrate=ok  # migration is ok, done through hotplug
phy:drbd1,xvda,w,migrate=drbd  # migration script called with -type drbd
phy:sdb,xvda,r                 # ok is default, for backwards compatib.

The "reject" argument can be "reject", "ok", or any other string. In the 
latter case, it's assumed to be the type given to the migration script. 
(Which can be used to redirect this to a different shell script.)

I appreciate your comments and suggestions.

Best Regards,
Michael Paesold

Re: [RFD] External device migration for block devices

[Stefan Berger]

[-- Attachment #1.1: Type: text/plain, Size: 3564 bytes --]

xen-devel-bounces@lists.xensource.com wrote on 04/21/2006 07:13:15 AM:

> Hi everyone, hi Stefan!
> 
> I want to use Stefan's work on device migration to implement 
> block-device migration for devices, where the regular model (hotplug 
> add/remove) does not work, i.e. when the device migration script is 
> required.
> I would like to get some feedback on my ideas.
> 
> 1) Currently, the device migration script does not get device details 
> for the device it should migrate, obviously because it's not needed for 
> vtpm. On the other hand, there can be several block devices for each 
> domain, and each might have to be handled differently. So the script 
> needs to get the details for the device it should migrate.
> 
> Should I add a arguments to the device migration script, that will not 
> be set for vtpm?

I think this is the best way of doing this. Currently the external device 
migration script only parses the parameters

-step <1,2,3>
-host <destination host>
-typ  <type of device {'vtpm',...}>
-domnam <name of domain to migrate>

which I thought were pretty generic for any kind of migration but not a 
complete list. If you don't want to extend the list of what is parsed in 
that script, the best would be to append '$*' to the call into the more 
specific migration scripts that would then if needed implement command 
line parameter parsing themselves again. This call currently looks like 
this:

func="$typ"_migration_step
eval $func $host $domname $step [$*  <- add]

So for vtpm it's first sourcing vtpm-migration.sh and afterwards calls the 
function vtpm_migration_step in that script.

> 
> 2) I think there should be some way to configure the migration 
> capabilities of block devices using the domain config files. Some block 
> device might be migrated automatically using regular hotplug add/remove 
> (e.g. nbd/enbd). Others might not be migratable at all (e.g., 
> phy:/dev/sdb). For my current use case (DRBD), I use phy:/dev/drbd. 
> These devices are migratable, but Xend does not know this.
> 
> I don't think it is good if all this knowledge must be put into the 
> migration scripts itself. So I suggest to add a device config option to 
> explicitly tell Xend what to do on migrate.
> 
> I suggest adding optional parameters to the end of the block device 
> definition, which is currently "phy:UNAME,DEV,MODE"; I would extend that 

> to "phy:UNAME,DEV,MODE[,OPTIONS]".
> 
> Examples:
> phy:sdb,xvda,w,migrate=reject  # reject migration
> nbd:sample,xvda1,r,migrate=ok  # migration is ok, done through hotplug
> phy:drbd1,xvda,w,migrate=drbd  # migration script called with -type drbd
> phy:sdb,xvda,r                 # ok is default, for backwards compatib.
> 
> The "reject" argument can be "reject", "ok", or any other string. In the 

> latter case, it's assumed to be the type given to the migration script. 
> (Which can be used to redirect this to a different shell script.)

If there's a difference between migrating these types of block devices 
then what you show is probably the best way of doing this. Otherwise if 
migrateability and which script to call can be recognized by the name of 
the device (sda1 vs drbd1 vs hda), then we could take that kind of 
intelligence and put it into XenD's blkif.py and take the configuration 
task away from the user.

   Stefan
> 
> I appreciate your comments and suggestions.
> 
> Best Regards,
> Michael Paesold
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

[-- Attachment #1.2: Type: text/html, Size: 4593 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFD] External device migration for block devices

[Michael Paesold]

Stefan Berger wrote:
> xen-devel-bounces@lists.xensource.com wrote on 04/21/2006 07:13:15 AM:
...
>  > 1) Currently, the device migration script does not get device details
>  > for the device it should migrate, obviously because it's not needed for
>  > vtpm. On the other hand, there can be several block devices for each
>  > domain, and each might have to be handled differently. So the script
>  > needs to get the details for the device it should migrate.
>  >
>  > Should I add a arguments to the device migration script, that will not
>  > be set for vtpm?
> 
> I think this is the best way of doing this. Currently the external 
> device migration script only parses the parameters
> 
> -step <1,2,3>
> -host <destination host>
> -typ  <type of device {'vtpm',...}>
> -domnam <name of domain to migrate>
> 
> which I thought were pretty generic for any kind of migration but not a 
> complete list. If you don't want to extend the list of what is parsed in 
> that script, the best would be to append '$*' to the call into the more 
> specific migration scripts that would then if needed implement command 
> line parameter parsing themselves again. This call currently looks like 
> this:
> 
> func="$typ"_migration_step
> eval $func $host $domname $step [$*  <- add]

Yeah, that's a good way of doing it. I don't think the parent script 
should be bloated with all possible options for all device types.

> So for vtpm it's first sourcing vtpm-migration.sh and afterwards calls 
> the function vtpm_migration_step in that script.
> 
>  > 2) I think there should be some way to configure the migration
>  > capabilities of block devices using the domain config files. Some block
>  > device might be migrated automatically using regular hotplug add/remove
>  > (e.g. nbd/enbd). Others might not be migratable at all (e.g.,
>  > phy:/dev/sdb). For my current use case (DRBD), I use phy:/dev/drbd.
>  > These devices are migratable, but Xend does not know this.
>  >
>  > I don't think it is good if all this knowledge must be put into the
>  > migration scripts itself. So I suggest to add a device config option to
>  > explicitly tell Xend what to do on migrate.
>  >
>  > I suggest adding optional parameters to the end of the block device
>  > definition, which is currently "phy:UNAME,DEV,MODE"; I would extend that
>  > to "phy:UNAME,DEV,MODE[,OPTIONS]".
>  >
>  > Examples:
>  > phy:sdb,xvda,w,migrate=reject  # reject migration
>  > nbd:sample,xvda1,r,migrate=ok  # migration is ok, done through hotplug
>  > phy:drbd1,xvda,w,migrate=drbd  # migration script called with -type drbd
>  > phy:sdb,xvda,r                 # ok is default, for backwards compatib.
>  >
>  > The "reject" argument can be "reject", "ok", or any other string. In the
>  > latter case, it's assumed to be the type given to the migration script.
>  > (Which can be used to redirect this to a different shell script.)
> 
> If there's a difference between migrating these types of block devices 
> then what you show is probably the best way of doing this. Otherwise if 
> migrateability and which script to call can be recognized by the name of 
> the device (sda1 vs drbd1 vs hda), then we could take that kind of 
> intelligence and put it into XenD's blkif.py and take the configuration 
> task away from the user.

I think there are so many different types of block devices that it's 
infeasible for blkif.py to know about all of them. For example, if you 
use Cluster LVM and a SAN, you can have activated the LVs on all dom0s, 
so no migration is neccessary at all -- in such a case phy:/dev/vg/lv1 
is available on all dom0s. In a different setup, phy:/dev/vg/lv1 might 
have some other backend that does need a migration procedure.

It might be possible to use the block device type (phy:,file:,nbd:,...) 
as argument to be passed to the migration script, but that means one 
also has to create a corresponding block-* script for hotplug (even if 
that doesn't do anything useful).

So "nbd:sdb,xvda,w" should use nbd-migrate.sh, and "drbd:drbd1,xvda,w" 
should use drbd-migrate.sh.

Is that a preferable solution?

Best Regards,
Michael Paesold

P.S. perhaps it's even better to pass "-type block -subtype drbd", so 
that a name of "block-drbd-migrate.sh" would be constructed, which would 
live next to "block-drbd" for hotplug.