Re: [Qemu-devel] live migration between different qemu versions

[Alexey <alukardd@alukardd.org>]

On 2016-01-12 13:04, Dr. David Alan Gilbert wrote:
> * Alexey (alukardd@alukardd.org) wrote:
>> Hello,
>> 
>> On 2016-01-12 12:19, Dr. David Alan Gilbert wrote:
>> >* Alexey (alukardd@alukardd.org) wrote:
>> >>Hi David.
>> >>
>> >>On 2016-01-11 22:51, Dr. David Alan Gilbert wrote:
>> >>>* Alexey (alukardd@alukardd.org) wrote:
>> >>>>Hello.
>> >>>
>> >>>Hi,
>> >>>
>> >>>>I have two servers between which I need have live migration.
>> >>>>
>> >>>>First server have QEMU emulator version 2.3.0
>> >>>>Second server have QEMU emulator version 2.5.0
>> >>>>
>> >>>>Migration command look like this:
>> >>>>/usr/bin/virsh migrate --live DOMAIN_NAME --migrateuri
>> >>>>tcp://second.server
>> >>>>qemu+ssh://second.server/system --verbose
>> >>>>
>> >>>>I got next error on console:
>> >>>>error: operation failed: migration job: unexpectedly failed
>> >>>>
>> >>>>And next error in log on second server:
>> >>>>2016-01-11T15:07:12.888647Z qemu-system-x86_64: Length too large:
>> >>>>/rom@genroms/efi-virtio.rom: 0x32000 > 0x2d000: Invalid argument
>> >>>>2016-01-11T15:07:12.888685Z qemu-system-x86_64: error while loading
>> >>>>state
>> >>>>for instance 0x0 of device 'ram'
>> >>>>2016-01-11T15:07:12.889733Z qemu-system-x86_64: load of migration
>> >>>>failed:
>> >>>>Invalid argument
>> >>>>
>> >>>>In my libvirt domain I have option <rom bar="off"/>.
>> >>>>
>> >>>>I need to do a live migration in any possible way. How can I do this
>> >>>>one?
>> >>
>> >>>Is the efi-virtio ROM that you're using on the two versions the same
>> >>>identical size?
>> >
>> >>No, I have different efi-virtio.rom files, and I try to find a way which
>> >>allows me not to copy the same efi-virtio.rom to all servers.
>> >
>> >Right, the error does look like it's just complaining about the ROM size:
>> >
>> >  2016-01-11T15:07:12.888647Z qemu-system-x86_64: Length too large:
>> >  /rom@genroms/efi-virtio.rom: 0x32000 > 0x2d000: Invalid argument
>> >
>> >It copes with small variations, but as soon as it flips over a 4K (?)
>> >boundary then it has nowhere to put the extra data coming from the
>> >larger ROM.
>> old server with qemu 2.3.0:
>> -rw-r--r-- 1 root root 203264 Nov 21  2014 
>> /usr/lib/ipxe/qemu/efi-virtio.rom
>> 
>> new server with qemu 2.5.0:
>> -rw-r--r-- 1 root root 184320 Oct 19  2014 
>> /usr/lib/ipxe/qemu/efi-virtio.rom
>> 
>> See, that destination server has smaller rom than source.
>> I interested in way which I could do live migration between different 
>> ROMs.
> 
> You can't do it if the ROMs dont approximately match in size;  what 
> happens here
> is that the 2.3.0 machine allocates just over 203k for the ROM 
> (203264=0x31A00
> and it's rounded that upto 0x32000).
> On the destination it's size is based on the 184320 (=0x2d000), so it 
> hasn't
> got room to receive the larger ROM.
> One way to fix this is to pad the ROMs; so just adding blank space to 
> the end
> of the newer efi-virtio.rom to make it the same size as the old one.
Ok, thanks. I'll try to do this.
Tell me please right way to append zeros to "BIOS (ia32) ROM Ext. 
(137*512)" file?

It will be grate, if I could specify ROM size as option to qemu.

>> Also I interested in migration in both side: from old to new and from 
>> new to
>> old.
> 
> OK, but note that backwards migration isn't tested much in
> upstream qemu.
> 
>> >>>Also, can you post the commandline that you're using please.
>> >>I already done this in my first email: virsh migrate...
>> >>>>/usr/bin/virsh migrate --live DOMAIN_NAME --migrateuri
>> >>>>tcp://second.server
>> >>>>qemu+ssh://second.server/system --verbose
>> >Do you mean running domain command line?
>> >I meant for the QEMU itself; but if we've already found the ROMs are
>> >different sizes, lets fix that first and see how you go.
>> Do you mean command line of running domain?
>> 
>> qemu-system-x86_64 -enable-kvm -name 208664-73 -S -machine
>> pc-i440fx-1.7,accel=kvm,usb=off -cpu
>> kvm64,+lahf_lm,+popcnt,+sse4.2,+sse4.1,+ssse3,+ss,-mca,-mce -m 1024
>> -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid
>> 08cf6bf2-a701-4a00-a0cb-4ecbe514c3d7 -no-user-config -nodefaults 
>> -chardev 
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/208664-73.monitor,server,nowait
>> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc 
>> -no-shutdown
>> -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 
>> -device
>> virtio-scsi-pci,id=scsi0,num_queues=4,bus=pci.0,addr=0x4 -device
>> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive 
>> file=/dev/disk/vbd/208664-73,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none,discard=unmap,aio=native
>> -device 
>> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2
>> -netdev tap,fd=53,id=hostnet0,vhost=on,vhostfd=54 -device 
>> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:00:40:b9,bus=pci.0,addr=0x3,rombar=0
>> -chardev pty,id=charserial0 -device
>> isa-serial,chardev=charserial0,id=serial0 -chardev 
>> socket,id=charchannel0,path=/var/lib/libvirt/qemu/208664-73.agent,server,nowait
>> -device 
>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
>> -device usb-mouse,id=input0 -device usb-kbd,id=input1 -vnc 
>> [::]:12,password
>> -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2 -device
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -object
>> rng-random,id=objrng0,filename=/dev/random -device 
>> virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=1024,period=2000,bus=pci.0,addr=0x7
>> -msg timestamp=on
> 
> OK, nothing too unusual there.
> 
> Dave
> 
>> >
>> >Dave
>> >
>> >>
>> >>Regards,
>> >>Alexey Mochkin.
>> >--
>> >Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK