Live Migration fails

Live Migration fails

[Daniel Hasler]

Hi all

I ran into a problem testing live migration.
I have two Identical Dell R200 Intel Boxes with 4G RAM and 2.5Ghz Intel 
CPUs (Quad-Cores).
I identically installed Ubuntu Intrepid Alpha-3 x86_64 (OK - a bit on 
the bleeding edge, but offers kvm-71 at the moment).
As a guest OS I use Ubunte Jeos 8.04.1 i686.

I have a iSCSI device, where I have the disk images as raw block devices.
ISO-Images for CDs/DVDs are on a NFS share.
I use bridged networking and virtio.
The two systems are on the same subnet.
The QEMU Monitor is used on a socket bound to localhost, the same is 
true for VNC.

I now start the virtual machine to migrate on host A:
 >/usr/bin/qemu -name vm1 -boot c -M pc -smp 1 -m 512 -monitor 
tcp:127.0.0.1:4101,server,nowait -vnc 127.0.0.1:1 -net
 >nic,macaddr=00:16:3e:0c:ed:c1,vlan=0,model=rtl8139 -net 
tap,script=/home/administrator/my-ifup.sh,vlan=0,ifname=vnet1 -drive
 >file=/dev/disk/by-path/ip-10.0.1.20:3260-iscsi-iqn.2006-01.com.openfiler:vm1-lun-0,if=ide,index=0 
-drive
 >file=/mnt/iso-images/jeos-8.04.1-jeos-i386.iso,if=ide,media=cdrom,index=2

Wait for the VM to fully boot into the OS.

Now, I start the VM in migration-listen mode on host B:
 >/usr/bin/kvm -S -name vm1 -boot c -M pc -smp 1 -m 512 -monitor 
tcp:127.0.0.1:4101,server,nowait -vnc 127.0.0.1:1 -net
 >nic,macaddr=00:16:3e:0c:ed:c1,vlan=0,model=virtio -net 
tap,script=/home/administrator/my-ifup.sh,vlan=0,ifname=vnet1 -drive
 >file=/dev/disk/by-path/ip-10.0.1.20:3260-iscsi-iqn.2006-01.com.openfiler:vm1-lun-0,if=ide,index=0,boot=on 
-drive
 >file=/mnt/iso-images/jeos-8.04.1-jeos-i386.iso,if=ide,media=cdrom,index=2 
-incoming tcp://10.0.1.30:4444

Now, on host A, using telnet, I connect to the QEMU monitor:
 >telnet localhost 4101
and start the migration
 > (QEMU) migrate tcp://10.0.1.30:4444

After a while, on host B (the target), I see the following error:
 >migration: wait_for_go: read error len=0 (Interrupted system call)
 >Migration failed rc=210

while on host A, I read following error message:
 >WAIT FOR ACK: timeout reached
 >Migration failed! ret=0 error=13

I am sure I tried the same scenario on Ubuntu Hardy 8.04 with kvm-62 
successfully (on different hardware).

I am doing something wrong? Anyone ran in the same issue and maybe has a 
resolution?
I googled for hours by now, but did not find many articles about kvm 
live migration.

Any help is appreciated,

Thanks in advance
Dani

Re: Live Migration fails

[Chris Lalancette]

Daniel Hasler wrote:
> After a while, on host B (the target), I see the following error:
>  >migration: wait_for_go: read error len=0 (Interrupted system call)
>  >Migration failed rc=210
> 
> while on host A, I read following error message:
>  >WAIT FOR ACK: timeout reached
>  >Migration failed! ret=0 error=13

No, this is a bug I just fixed yesterday in Qemu proper.  The next time that Avi
rebases KVM to Qemu (which he said he would do as soon as the fix went into
Qemu), then this should be fixed.

FYI, the problem is basically that the i2c restore on the destination side was
expecting 4 bytes, while the i2c save on the sending side was only sending 1
byte, so they eventually timed out while waiting for each other.

Chris Lalancette

Re: Live Migration fails

[Anthony Liguori]

Chris Lalancette wrote:
> Daniel Hasler wrote:
>   
>> After a while, on host B (the target), I see the following error:
>>  >migration: wait_for_go: read error len=0 (Interrupted system call)
>>  >Migration failed rc=210
>>
>> while on host A, I read following error message:
>>  >WAIT FOR ACK: timeout reached
>>  >Migration failed! ret=0 error=13
>>     
>
> No, this is a bug I just fixed yesterday in Qemu proper.  The next time that Avi
> rebases KVM to Qemu (which he said he would do as soon as the fix went into
> Qemu), then this should be fixed.
>   

Migration is also known to be broken with virtio.

Regards,

Anthony Liguori

> FYI, the problem is basically that the i2c restore on the destination side was
> expecting 4 bytes, while the i2c save on the sending side was only sending 1
> byte, so they eventually timed out while waiting for each other.
>
> Chris Lalancette
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>