Live Migration, DRBD

Live Migration, DRBD

[Kent Borg]

I am very happy to discover that KVM does live migration.  Now I am
figuring out whether it will work for me. 

What I have in mind is to use DRBD for the file system image.  The
problem is that during the migration I want to shift the file system
access at the moment when the VM has quit running on the host it is
leaving but before it starts running on the host where it is arriving. 
Is there a hook to let me do stuff at this point?

This is what I want to do:

On the departing machine...

  - VM has stopped here
  - umount the volume with the VM file system image
  - mark volume in DRDB as secondary


On the arriving machine...

  - mark volume in DRBD as primary
  - mount the volume with the VM file system image
  - VM can now start here


Is there a way?


Thanks,

-kb

Re: Live Migration, DRBD

[Dor Laor]

Kent Borg wrote:
> I am very happy to discover that KVM does live migration.  Now I am
> figuring out whether it will work for me. 
>
> What I have in mind is to use DRBD for the file system image.  The
> problem is that during the migration I want to shift the file system
> access at the moment when the VM has quit running on the host it is
> leaving but before it starts running on the host where it is arriving. 
> Is there a hook to let me do stuff at this point?
>
> This is what I want to do:
>
> On the departing machine...
>
>   - VM has stopped here
>   - umount the volume with the VM file system image
>   - mark volume in DRDB as secondary
>
>
> On the arriving machine...
>
>   - mark volume in DRBD as primary
>   - mount the volume with the VM file system image
>   - VM can now start here
>
>
> Is there a way?
>
>   
No, but one can add such pretty easy. The whole migration code is in one 
file qemu/migration.c
You can add a parameter to qemu migration command to specify a script 
that should be called on
migration end event (similar to the tap script).
> Thanks,
>
> -kb
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

Re: Live Migration, DRBD

[Jim]

Kent Borg <kentborg <at> borg.org> writes:

> 
> I am very happy to discover that KVM does live migration.  Now I am
> figuring out whether it will work for me. 
> 
> What I have in mind is to use DRBD for the file system image.  The
> problem is that during the migration I want to shift the file system
> access at the moment when the VM has quit running on the host it is
> leaving but before it starts running on the host where it is arriving. 
> Is there a hook to let me do stuff at this point?
> 
> This is what I want to do:
> 
> On the departing machine...
> 
>   - VM has stopped here
>   - umount the volume with the VM file system image
>   - mark volume in DRDB as secondary
> 
> On the arriving machine...
> 
>   - mark volume in DRBD as primary
>   - mount the volume with the VM file system image
>   - VM can now start here
> 

Yes, there is a way, but first your setup is a little strange. Why do you
take a device (the DRBD) then put a file system on it which just contains a
file with the system image? Why not use the DRBD device directly as your system
disk?

e.g. qemu-system-86_64 -hda /dev/drbdX

This way you do not get an extra layer of filesystem slowing things down
and taking up space, the whole of the DRBD device is directly accessible to
the guest.

Most importantly it saves the mount/umount steps in your above procedures.

When using DRBD devices directly live migration simply requires that the
device is accessible on both nodes at the same time. In other words live
migration assumes a shared device, which you have. The only problem is that
it needs to be opened read/write on both nodes at the same time, which means
you need to go Primary/Primary.

The recent DRBD versions support Primary/Primary, you just need to add
"net { allow-two-primaries; }"
to the resource section in drbd.conf

With that done you can go to the target node, make the device primary there
too, start up qemu to accept the incoming migration and migrate from the
source node.

Afterwards it is advisable to set the source node to secondary.

This procedure is safe, as apparently qemu won't start accessing the target
device until the source has been finished with and flushed. I have tested
the procedure and it worked very well.

Hope that helps,

Jim

P.S. I'm not subscribed to this list so please email me directly if you
need to.

Re: Live Migration, DRBD

[Kent Borg]

Jim wrote:
> Yes, there is a way, but first your setup is a little strange. Why do you
> take a device (the DRBD) then put a file system on it which just contains a
> file with the system image? Why not use the DRBD device directly as your system
> disk?
>
> e.g. qemu-system-86_64 -hda /dev/drbdX
>   

That does sound smarter. I guess I was assuming files because they are
easier to otherwise manipulate.


Thanks!

-kb