live migration to qemu.git fails

live migration to qemu.git fails

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 1518 bytes --]

Doing a live migration of a HVM domU from staging-4.5 with its included
qemu-xen to staging-4.7 with qemu-xen from qemu.org (either master or
2.6.0) fails:

# cat /var/log/xen/qemu-dm-fv-x64-sles12sp1-clean--incoming.log
char device redirected to /dev/pts/4 (label serial0)
xen_ram_alloc: do not alloc f800000 bytes of ram at 0 when runstate is INMIGRATE
xen_ram_alloc: do not alloc 800000 bytes of ram at f800000 when runstate is INMIGRATE
xen_ram_alloc: do not alloc 10000 bytes of ram at 10000000 when runstate is INMIGRATE
xen_ram_alloc: do not alloc 40000 bytes of ram at 10010000 when runstate is INMIGRATE
VNC server running on 127.0.0.1:5900
qemu-system-i386: Expected vmdescription section, but got 0


'xl vnc domU' on the receiver side just shows "guest has not initialized
display yet". For some reason the domU is still alive, 'vmstat 1' via
ssh keeps running.

'xl shutdown domU' fails. It looks like the guest OS shuts down, the ssh
connection gets terminated, but the domU becomes busy according to 'xl
top', and never shuts down.

domU.cfg looks like this:
name="fv-x64-sles12sp1-clean"
memory=256
serial="pty"
builder="hvm"
boot="cd"
disk=[
        'file:/disk0.raw,hda,w',
        'file:/DVD1.iso,hdc:cdrom,r',
]
vif=[
        'bridge=br0'
]
keymap="de"
vfb = [
        'type=vnc,vncunused=1,keymap=de'
]

usb=1
usbdevice='tablet'
device_model_override="/root/dm.sh"


The override is to make sure to use qemu-xen as included in xen-4.5 on
the sending side and qemu.git on the receiving side.

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 346 bytes --]

On Tue, Aug 02, Olaf Hering wrote:

> Doing a live migration of a HVM domU from staging-4.5 with its included
> qemu-xen to staging-4.7 with qemu-xen from qemu.org (either master or
> 2.6.0) fails:

It fails also with staging-4.6 and staging-4.7 and their included
qemu-xen binaries, and staging-4.7 with qemu-xen from qemu.org as target.


Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Wei Liu]

Cc Anthony and Stefano

On Tue, Aug 02, 2016 at 11:58:05AM +0200, Olaf Hering wrote:
> Doing a live migration of a HVM domU from staging-4.5 with its included
> qemu-xen to staging-4.7 with qemu-xen from qemu.org (either master or
> 2.6.0) fails:
> 
> # cat /var/log/xen/qemu-dm-fv-x64-sles12sp1-clean--incoming.log
> char device redirected to /dev/pts/4 (label serial0)
> xen_ram_alloc: do not alloc f800000 bytes of ram at 0 when runstate is INMIGRATE
> xen_ram_alloc: do not alloc 800000 bytes of ram at f800000 when runstate is INMIGRATE
> xen_ram_alloc: do not alloc 10000 bytes of ram at 10000000 when runstate is INMIGRATE
> xen_ram_alloc: do not alloc 40000 bytes of ram at 10010000 when runstate is INMIGRATE
> VNC server running on 127.0.0.1:5900
> qemu-system-i386: Expected vmdescription section, but got 0
> 
> 
> 'xl vnc domU' on the receiver side just shows "guest has not initialized
> display yet". For some reason the domU is still alive, 'vmstat 1' via
> ssh keeps running.
> 
> 'xl shutdown domU' fails. It looks like the guest OS shuts down, the ssh
> connection gets terminated, but the domU becomes busy according to 'xl
> top', and never shuts down.
> 
> domU.cfg looks like this:
> name="fv-x64-sles12sp1-clean"
> memory=256
> serial="pty"
> builder="hvm"
> boot="cd"
> disk=[
>         'file:/disk0.raw,hda,w',
>         'file:/DVD1.iso,hdc:cdrom,r',
> ]
> vif=[
>         'bridge=br0'
> ]
> keymap="de"
> vfb = [
>         'type=vnc,vncunused=1,keymap=de'
> ]
> 
> usb=1
> usbdevice='tablet'
> device_model_override="/root/dm.sh"
> 
> 
> The override is to make sure to use qemu-xen as included in xen-4.5 on
> the sending side and qemu.git on the receiving side.
> 
> Olaf



> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Anthony PERARD]

On Wed, Aug 03, 2016 at 10:41:25AM +0100, Wei Liu wrote:
> Cc Anthony and Stefano
> 
> On Tue, Aug 02, 2016 at 11:58:05AM +0200, Olaf Hering wrote:
> > Doing a live migration of a HVM domU from staging-4.5 with its included
> > qemu-xen to staging-4.7 with qemu-xen from qemu.org (either master or
> > 2.6.0) fails:
> > 
> > # cat /var/log/xen/qemu-dm-fv-x64-sles12sp1-clean--incoming.log
> > char device redirected to /dev/pts/4 (label serial0)
> > xen_ram_alloc: do not alloc f800000 bytes of ram at 0 when runstate is INMIGRATE
> > xen_ram_alloc: do not alloc 800000 bytes of ram at f800000 when runstate is INMIGRATE
> > xen_ram_alloc: do not alloc 10000 bytes of ram at 10000000 when runstate is INMIGRATE
> > xen_ram_alloc: do not alloc 40000 bytes of ram at 10010000 when runstate is INMIGRATE
> > VNC server running on 127.0.0.1:5900
> > qemu-system-i386: Expected vmdescription section, but got 0
> > 
> > 
> > 'xl vnc domU' on the receiver side just shows "guest has not initialized
> > display yet". For some reason the domU is still alive, 'vmstat 1' via
> > ssh keeps running.
> > 
> > 'xl shutdown domU' fails. It looks like the guest OS shuts down, the ssh
> > connection gets terminated, but the domU becomes busy according to 'xl
> > top', and never shuts down.

Does the patch "xen: handle inbound migration of VMs without ioreq
server pages" from Paul fix the issue?
<1470042985-680-1-git-send-email-paul.durrant@citrix.com>
https://lists.xen.org/archives/html/xen-devel/2016-08/msg00020.html

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 440 bytes --]

On Wed, Aug 03, Anthony PERARD wrote:

> Does the patch "xen: handle inbound migration of VMs without ioreq
> server pages" from Paul fix the issue?
> <1470042985-680-1-git-send-email-paul.durrant@citrix.com>
> https://lists.xen.org/archives/html/xen-devel/2016-08/msg00020.html

Would that help with VMs created by staging-4.7 and its qemu-xen?
I think that change is just for staging-4.4 and its qemu-xen.
But I will try it anyway.

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Anthony PERARD]

On Wed, Aug 03, 2016 at 02:52:54PM +0200, Olaf Hering wrote:
> On Wed, Aug 03, Anthony PERARD wrote:
> 
> > Does the patch "xen: handle inbound migration of VMs without ioreq
> > server pages" from Paul fix the issue?
> > <1470042985-680-1-git-send-email-paul.durrant@citrix.com>
> > https://lists.xen.org/archives/html/xen-devel/2016-08/msg00020.html
> 
> Would that help with VMs created by staging-4.7 and its qemu-xen?
> I think that change is just for staging-4.4 and its qemu-xen.
> But I will try it anyway.

I quote your original email:

On Tue, Aug 02, 2016 at 11:58:05AM +0200, Olaf Hering wrote:
> Doing a live migration of a HVM domU from staging-4.5 with its included
> qemu-xen to staging-4.7 with qemu-xen from qemu.org (either master or
> 2.6.0) fails:

Haven't you try to create a guest with Xen 4.5 and qemu-xen-4.5, and
then migrate to Xen 4.7 with QEMU-2.6/master?

If you did a live migration with Xen 4.7 from qemu-xen-4.7 to
QEMU-master, the patch might not help.


-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 269 bytes --]

On Wed, Aug 03, Anthony PERARD wrote:

> Haven't you try to create a guest with Xen 4.5 and qemu-xen-4.5, and
> then migrate to Xen 4.7 with QEMU-2.6/master?

In the end I tried xen-4.5/6/7/8 as source and their qemu-xen, and
migrated to qemu#master. All failed.

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 647 bytes --]

On Wed, Aug 03, Olaf Hering wrote:

> On Wed, Aug 03, Anthony PERARD wrote:
> 
> > Haven't you try to create a guest with Xen 4.5 and qemu-xen-4.5, and
> > then migrate to Xen 4.7 with QEMU-2.6/master?
> 
> In the end I tried xen-4.5/6/7/8 as source and their qemu-xen, and
> migrated to qemu#master. All failed.

Today I built qemu binaries from stable-2.4/stable-2.5/v2.6.0 to prepare
a bisect. Because 2.4/2.5 does not compile with staging-4.7 I went back
to staging-4.6. And to my surprise live migration works in every
combination I have tried. Not sure what went wrong with earlier testing.

I will see what I find..

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Anthony PERARD]

On Fri, Aug 05, 2016 at 03:37:32PM +0200, Olaf Hering wrote:
> On Wed, Aug 03, Olaf Hering wrote:
> 
> > On Wed, Aug 03, Anthony PERARD wrote:
> > 
> > > Haven't you try to create a guest with Xen 4.5 and qemu-xen-4.5, and
> > > then migrate to Xen 4.7 with QEMU-2.6/master?
> > 
> > In the end I tried xen-4.5/6/7/8 as source and their qemu-xen, and
> > migrated to qemu#master. All failed.
> 
> Today I built qemu binaries from stable-2.4/stable-2.5/v2.6.0 to prepare
> a bisect. Because 2.4/2.5 does not compile with staging-4.7 I went back
> to staging-4.6.

Well, 4.7 introduces new libraries and older QEMUs do not support it out
of the box. You'll need to add some CFLAGS and LDFLADS to have them
working, like does xen.git (staging-4.7) when compiling qemu-xen.

If you compile older qemu out if xen tree, you would need to at least
add:
--extra-cflags="-DXC_WANT_COMPAT_EVTCHN_API=1 -DXC_WANT_COMPAT_GNTTAB_API=1 -DXC_WANT_COMPAT_MAP_FOREIGN_API=1"

> And to my surprise live migration works in every
> combination I have tried. Not sure what went wrong with earlier testing.

Good to hear.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 630 bytes --]

On Fri, Aug 05, Anthony PERARD wrote:

> On Fri, Aug 05, 2016 at 03:37:32PM +0200, Olaf Hering wrote:

> If you compile older qemu out if xen tree, you would need to at least
> add:
> --extra-cflags="-DXC_WANT_COMPAT_EVTCHN_API=1 -DXC_WANT_COMPAT_GNTTAB_API=1 -DXC_WANT_COMPAT_MAP_FOREIGN_API=1"

Thanks, I will try that to fix compile errors.

> > And to my surprise live migration works in every
> > combination I have tried. Not sure what went wrong with earlier testing.
> 
> Good to hear.

After more testing even the staging-4.7 packages work again.
Not sure what change was made to hide the error.

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: live migration to qemu.git fails

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 469 bytes --]

On Fri, Aug 05, Olaf Hering wrote:

> Not sure what change was made to hide the error.

On the receiving side, /usr/bin/qemu-system-i386 works while
/usr/bin/qemu-system-x86_64 fails. Since the sending side is
/usr/lib/xen/bin/qemu-system-i386 there is appearenly an incompatibality
between the i386 and the x86_64 binary.

Not sure if there is any gain in running qemu-system-x86_64, so I think
its best to adjust our --with-system-qemu to use qemu-system-i386.

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel