Commit 331b51 breaks live migration on FreeBSD/Xen dom0

Commit 331b51 breaks live migration on FreeBSD/Xen dom0

[Roger Pau Monné]

Hello,

Commit 331b51 "xen: don't use xenstore to save/restore physmap
anymore" breaks live migration on FreeBSD dom0. I've checked reverting
just this specific commit and live migration works again.

The symptoms are the following, xl migrate will fail with:

migration target: Ready to receive domain.
Saving to migration stream new xl format (info 0x3/0x0/1415)
Loading new save file <incoming migration stream> (new xl fmt info 0x3/0x0/1415)
 Savefile contains xl domain config in JSON format
Parsing config from <saved>
xc: info: Saving domain 18, type x86 HVM
xc: info: Found x86 HVM domain from Xen 4.11
xc: info: Restoring domain
xc: info: Restore successful
xc: info: XenStore: mfn 0xfeffc, dom 0, evt 1
xc: info: Console: mfn 0xfefff, dom 0, evt 2
libxl: error: libxl_dm.c:2427:device_model_spawn_outcome: Domain 19:domain 19 device model: spawn failed (rc=-3)
libxl: error: libxl_create.c:1562:domcreate_devmodel_started: Domain 19:device model did not start: -3
libxl: error: libxl_dm.c:2541:kill_device_model: Device Model already exited
libxl: error: libxl_domain.c:1034:libxl__destroy_domid: Domain 19:Non-existant domain
libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 19:Unable to destroy guest
libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 19:Destruction of domain failed
migration target: Domain creation failed (code -3).
libxl: error: libxl_utils.c:510:libxl_read_exactly: file/stream truncated reading ready message from migration receiver stream
libxl: info: libxl_exec.c:118:libxl_report_child_exitstatus: migration transport process [12952] exited with error status 1
Migration failed, resuming at sender.

The log of the incoming QEMU is:

qemu-system-i386: -serial pty: char device redirected to /dev/pts/4 (label serial0)
xen: shared page at pfn feff0
xen: buffered io page at pfn feff1
xen: buffered io evtchn is 4
xen_mapcache: xen_map_cache_init, nr_buckets = 8000 size 1572864
xen_ram_alloc: do not alloc 1f800000 bytes of ram at 0 when runstate is INMIGRATE
xen_ram_alloc: do not alloc 800000 bytes of ram at 1f800000 when runstate is INMIGRATE
xen_ram_alloc: do not alloc 10000 bytes of ram at 20000000 when runstate is INMIGRATE
xen_ram_alloc: do not alloc 40000 bytes of ram at 20010000 when runstate is INMIGRATE
VNC server running on 0.0.0.0:5900
xen: mapping vram to f0000000 - f0800000
Replacing a dummy mapcache entry for 000000001f800000 with 00000000f0000000
Assertion failed: (p && p == memory_region_get_ram_ptr(mr)), function xen_add_to_physmap, file /usr/ports/sysutils/xen-tools/work/xen-4.11.1/tools/qemu-xen/hw/i386/xen/xen-hvm.c, line 392.

This is with Xen 4.11.1.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Commit 331b51 breaks live migration on FreeBSD/Xen dom0

[Igor Druzhinin]

On 14/03/2019 15:54, Roger Pau Monné wrote:
> The log of the incoming QEMU is:
> 
> qemu-system-i386: -serial pty: char device redirected to /dev/pts/4 (label serial0)
> xen: shared page at pfn feff0
> xen: buffered io page at pfn feff1
> xen: buffered io evtchn is 4
> xen_mapcache: xen_map_cache_init, nr_buckets = 8000 size 1572864
> xen_ram_alloc: do not alloc 1f800000 bytes of ram at 0 when runstate is INMIGRATE
> xen_ram_alloc: do not alloc 800000 bytes of ram at 1f800000 when runstate is INMIGRATE
> xen_ram_alloc: do not alloc 10000 bytes of ram at 20000000 when runstate is INMIGRATE
> xen_ram_alloc: do not alloc 40000 bytes of ram at 20010000 when runstate is INMIGRATE
> VNC server running on 0.0.0.0:5900
> xen: mapping vram to f0000000 - f0800000
> Replacing a dummy mapcache entry for 000000001f800000 with 00000000f0000000
> Assertion failed: (p && p == memory_region_get_ram_ptr(mr)), function xen_add_to_physmap, file /usr/ports/sysutils/xen-tools/work/xen-4.11.1/tools/qemu-xen/hw/i386/xen/xen-hvm.c, line 392.
> 

The change was supposed to be platform independent - it relies on the
fact addr argument to mmap() works as expected - mmaps at the specified
address. If this argument might be ignored on FreeBSD - that is a
problem. Other then that the change was platform independent.

You could also manually enable XEN_COMPAT_PHYSMAP while building QEMU
(it's now gated on only Xen version) and see if it solves your problem.
We may probably keep it enabled on FreeBSD all the time if there is no
other way.

Could you also print p and memory_region_get_ram_ptr(mr) so we could be
sure it's mmap disregarding hint address?

Igor

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Commit 331b51 breaks live migration on FreeBSD/Xen dom0

[Marek Marczykowski-Górecki]

[-- Attachment #1.1: Type: text/plain, Size: 509 bytes --]

On Thu, Mar 14, 2019 at 04:07:55PM +0000, Igor Druzhinin wrote:
> The change was supposed to be platform independent - it relies on the
> fact addr argument to mmap() works as expected - mmaps at the specified
> address. 

You have this guarantee only with MAP_FIXED flag, otherwise the addr
argument is only a hint.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel