* [libvirt test] 92667: regressions - FAIL
@ 2016-04-25 11:26 osstest service owner
2016-04-27 21:58 ` Jim Fehlig
0 siblings, 1 reply; 7+ messages in thread
From: osstest service owner @ 2016-04-25 11:26 UTC (permalink / raw)
To: xen-devel, osstest-admin
[-- Attachment #1: Type: text/plain, Size: 6090 bytes --]
flight 92667 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/92667/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-i386-libvirt 14 guest-saverestore fail REGR. vs. 91479
test-amd64-amd64-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
test-amd64-i386-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 91479
test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
test-amd64-amd64-libvirt 14 guest-saverestore fail REGR. vs. 91479
Tests which did not succeed, but are not blocking:
test-armhf-armhf-libvirt-xsm 12 migrate-support-check fail never pass
test-armhf-armhf-libvirt-xsm 14 guest-saverestore fail never pass
test-amd64-i386-libvirt 12 migrate-support-check fail never pass
test-amd64-amd64-libvirt-xsm 12 migrate-support-check fail never pass
test-armhf-armhf-libvirt-raw 13 guest-saverestore fail never pass
test-armhf-armhf-libvirt-raw 11 migrate-support-check fail never pass
test-armhf-armhf-libvirt 14 guest-saverestore fail never pass
test-armhf-armhf-libvirt 12 migrate-support-check fail never pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass
test-amd64-i386-libvirt-xsm 12 migrate-support-check fail never pass
test-amd64-amd64-libvirt-vhd 11 migrate-support-check fail never pass
test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass
test-amd64-amd64-libvirt 12 migrate-support-check fail never pass
test-armhf-armhf-libvirt-qcow2 11 migrate-support-check fail never pass
test-armhf-armhf-libvirt-qcow2 13 guest-saverestore fail never pass
version targeted for testing:
libvirt 856e84a51675a0dccf0bde3e5e7bcf83eb708c02
baseline version:
libvirt 8b62c65d24bdb20121d3147b4f4dc98bac4f024b
Last test of basis 91479 2016-04-15 05:33:51 Z 10 days
Failing since 91597 2016-04-16 04:20:46 Z 9 days 10 attempts
Testing same since 92533 2016-04-24 04:28:00 Z 1 days 2 attempts
------------------------------------------------------------
People who touched revisions under test:
Andrea Bolognani <abologna@redhat.com>
Cole Robinson <crobinso@redhat.com>
Dmitry Andreev <dandreev@virtuozzo.com>
Erik Skultety <eskultet@redhat.com>
Jason J. Herne <jjherne@linux.vnet.ibm.com>
Jim Fehlig <jfehlig@suse.com>
Jiri Denemark <jdenemar@redhat.com>
John Ferlan <jferlan@redhat.com>
Ján Tomko <jtomko@redhat.com>
Laine Stump <laine@laine.org>
Martin Kletzander <mkletzan@redhat.com>
Maxim Nestratov <mnestratov@virtuozzo.com>
Michal Privoznik <mprivozn@redhat.com>
Mikhail Feoktistov <mfeoktistov@virtuozzo.com>
Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
Nitesh Konkar <niteshkonkar.libvirt@gmail.com>
Nitesh Konkar <nitkon12@linux.vnet.ibm.com>
Olga Krishtal <okrishtal@virtuozzo.com>
Peter Krempa <pkrempa@redhat.com>
Richard Laager <rlaager@wiktel.com>
Richard W.M. Jones <rjones@redhat.com>
Roman Bogorodskiy <bogorodskiy@gmail.com>
Simon Arlott <bugzilla.redhat.simon@arlott.org>
jobs:
build-amd64-xsm pass
build-armhf-xsm pass
build-i386-xsm pass
build-amd64 pass
build-armhf pass
build-i386 pass
build-amd64-libvirt pass
build-armhf-libvirt pass
build-i386-libvirt pass
build-amd64-pvops pass
build-armhf-pvops pass
build-i386-pvops pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm fail
test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm fail
test-amd64-amd64-libvirt-xsm fail
test-armhf-armhf-libvirt-xsm fail
test-amd64-i386-libvirt-xsm fail
test-amd64-amd64-libvirt fail
test-armhf-armhf-libvirt fail
test-amd64-i386-libvirt fail
test-amd64-amd64-libvirt-pair fail
test-amd64-i386-libvirt-pair fail
test-armhf-armhf-libvirt-qcow2 fail
test-armhf-armhf-libvirt-raw fail
test-amd64-amd64-libvirt-vhd fail
------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
Not pushing.
(No revision log; it would be 1823 lines long.)
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [libvirt test] 92667: regressions - FAIL
2016-04-25 11:26 [libvirt test] 92667: regressions - FAIL osstest service owner
@ 2016-04-27 21:58 ` Jim Fehlig
2016-04-27 22:22 ` Andrew Cooper
0 siblings, 1 reply; 7+ messages in thread
From: Jim Fehlig @ 2016-04-27 21:58 UTC (permalink / raw)
To: osstest service owner, xen-devel; +Cc: Andrew Cooper, Wei Liu
On 04/25/2016 05:26 AM, osstest service owner wrote:
> flight 92667 libvirt real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/92667/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-amd64-i386-libvirt 14 guest-saverestore fail REGR. vs. 91479
> test-amd64-amd64-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
> test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
> test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
> test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
> test-amd64-i386-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
> test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 91479
> test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
> test-amd64-amd64-libvirt 14 guest-saverestore fail REGR. vs. 91479
All of these save/restore and migration failures show the following error on the
restore side
2016-04-25 10:16:18 UTC libxl: error:
libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [26771] exited
with error status 1
2016-04-25 10:16:18 UTC libxl: error: libxl_utils.c:507:libxl_read_exactly:
file/stream truncated reading ipc msg header from domain 1 save/restore helper
stdout pipe
2016-04-25 10:16:18 UTC libxl: error:
libxl_exec.c:129:libxl_report_child_exitstatus: domain 1 save/restore helper
[26772] died due to fatal signal Terminated
I'm not sure if this problem has already been addressed by recent
migration-related fixes.
Regards,
Jim
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [libvirt test] 92667: regressions - FAIL
2016-04-27 21:58 ` Jim Fehlig
@ 2016-04-27 22:22 ` Andrew Cooper
2016-04-28 2:49 ` Jim Fehlig
2016-04-28 10:20 ` Wei Liu
0 siblings, 2 replies; 7+ messages in thread
From: Andrew Cooper @ 2016-04-27 22:22 UTC (permalink / raw)
To: Jim Fehlig, osstest service owner, xen-devel; +Cc: Wei Liu
On 27/04/2016 22:58, Jim Fehlig wrote:
> On 04/25/2016 05:26 AM, osstest service owner wrote:
>> flight 92667 libvirt real [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/92667/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>> test-amd64-i386-libvirt 14 guest-saverestore fail REGR. vs. 91479
>> test-amd64-amd64-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
>> test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
>> test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
>> test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
>> test-amd64-i386-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
>> test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 91479
>> test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
>> test-amd64-amd64-libvirt 14 guest-saverestore fail REGR. vs. 91479
> All of these save/restore and migration failures show the following error on the
> restore side
>
> 2016-04-25 10:16:18 UTC libxl: error:
> libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [26771] exited
> with error status 1
> 2016-04-25 10:16:18 UTC libxl: error: libxl_utils.c:507:libxl_read_exactly:
> file/stream truncated reading ipc msg header from domain 1 save/restore helper
> stdout pipe
> 2016-04-25 10:16:18 UTC libxl: error:
> libxl_exec.c:129:libxl_report_child_exitstatus: domain 1 save/restore helper
> [26772] died due to fatal signal Terminated
>
> I'm not sure if this problem has already been addressed by recent
> migration-related fixes.
This is testing two different versions of libvirt against the same
version of libxl.
Looking at
http://logs.test-lab.xenproject.org/osstest/logs/92667/test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm/italia0---var-log-libvirt-libxl-libxl-driver.log
2016-04-25 08:36:03 UTC xc: progress: End of stream: 0/0 0%
indicates that the save side is in v2 format (which is expected). (I
should add at least an info print in libxl_stream_write() indicating the
pertinent details).
On the restore side,
2016-04-25 08:36:20 UTC libxl: debug:
libxl_stream_read.c:358:stream_header_done: Stream v2 (from legacy)
2016-04-25 08:36:20 UTC libxl: debug:
libxl_stream_read.c:574:process_record: Record: 1, length 0
2016-04-25 08:36:20 UTC libxl: error:
libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [3909]
exited with error status 1
which means that the restore code was told that the stream was in legacy
format. The legacy conversion script was forked and found that the
stream wasn't legacy. (I have no idea where the real error message went
from that - it should be plumbed through into a info message, and
definitely does work when running `xl` on the command line).
I suspect this is breakage from the LIBXL_ABI_VERSION changes.
Because of the short-sightest mess that legacy migration was, it is not
possible for libxl to distinguish a legacy stream from a v2 stream in
libxl_domain_create_restore(). The caller (i.e. libvirt) must provide
the correct stream version in libxl_domain_restore_params.
~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [libvirt test] 92667: regressions - FAIL
2016-04-27 22:22 ` Andrew Cooper
@ 2016-04-28 2:49 ` Jim Fehlig
2016-04-28 9:54 ` Wei Liu
2016-04-28 10:20 ` Wei Liu
1 sibling, 1 reply; 7+ messages in thread
From: Jim Fehlig @ 2016-04-28 2:49 UTC (permalink / raw)
To: Andrew Cooper, osstest service owner, xen-devel; +Cc: Wei Liu
On 04/27/2016 04:22 PM, Andrew Cooper wrote:
> On 27/04/2016 22:58, Jim Fehlig wrote:
>> On 04/25/2016 05:26 AM, osstest service owner wrote:
>>> flight 92667 libvirt real [real]
>>> http://logs.test-lab.xenproject.org/osstest/logs/92667/
>>>
>>> Regressions :-(
>>>
>>> Tests which did not succeed and are blocking,
>>> including tests which could not be run:
>>> test-amd64-i386-libvirt 14 guest-saverestore fail REGR. vs. 91479
>>> test-amd64-amd64-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
>>> test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
>>> test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
>>> test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
>>> test-amd64-i386-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
>>> test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 91479
>>> test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
>>> test-amd64-amd64-libvirt 14 guest-saverestore fail REGR. vs. 91479
>> All of these save/restore and migration failures show the following error on the
>> restore side
>>
>> 2016-04-25 10:16:18 UTC libxl: error:
>> libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [26771] exited
>> with error status 1
>> 2016-04-25 10:16:18 UTC libxl: error: libxl_utils.c:507:libxl_read_exactly:
>> file/stream truncated reading ipc msg header from domain 1 save/restore helper
>> stdout pipe
>> 2016-04-25 10:16:18 UTC libxl: error:
>> libxl_exec.c:129:libxl_report_child_exitstatus: domain 1 save/restore helper
>> [26772] died due to fatal signal Terminated
>>
>> I'm not sure if this problem has already been addressed by recent
>> migration-related fixes.
> This is testing two different versions of libvirt against the same
> version of libxl.
>
> Looking at
> http://logs.test-lab.xenproject.org/osstest/logs/92667/test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm/italia0---var-log-libvirt-libxl-libxl-driver.log
>
> 2016-04-25 08:36:03 UTC xc: progress: End of stream: 0/0 0%
>
> indicates that the save side is in v2 format (which is expected). (I
> should add at least an info print in libxl_stream_write() indicating the
> pertinent details).
>
> On the restore side,
>
> 2016-04-25 08:36:20 UTC libxl: debug:
> libxl_stream_read.c:358:stream_header_done: Stream v2 (from legacy)
> 2016-04-25 08:36:20 UTC libxl: debug:
> libxl_stream_read.c:574:process_record: Record: 1, length 0
> 2016-04-25 08:36:20 UTC libxl: error:
> libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [3909]
> exited with error status 1
>
> which means that the restore code was told that the stream was in legacy
> format. The legacy conversion script was forked and found that the
> stream wasn't legacy. (I have no idea where the real error message went
> from that - it should be plumbed through into a info message, and
> definitely does work when running `xl` on the command line).
>
> I suspect this is breakage from the LIBXL_ABI_VERSION changes.
>
> Because of the short-sightest mess that legacy migration was, it is not
> possible for libxl to distinguish a legacy stream from a v2 stream in
> libxl_domain_create_restore(). The caller (i.e. libvirt) must provide
> the correct stream version in libxl_domain_restore_params.
How do I handle the case of a libvirt+Xen migV2 host migrating a domain to a
libvirt+Xen migV1 host? Do you know how that scenario is handled in xl? Or is
migrating a domain from migV2 host to migV1 host not supported?
Thanks for the help.
Regards,
Jim
>
> ~Andrew
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [libvirt test] 92667: regressions - FAIL
2016-04-28 2:49 ` Jim Fehlig
@ 2016-04-28 9:54 ` Wei Liu
2016-04-28 10:06 ` Andrew Cooper
0 siblings, 1 reply; 7+ messages in thread
From: Wei Liu @ 2016-04-28 9:54 UTC (permalink / raw)
To: Jim Fehlig; +Cc: Andrew Cooper, xen-devel, Wei Liu, osstest service owner
On Wed, Apr 27, 2016 at 08:49:23PM -0600, Jim Fehlig wrote:
> On 04/27/2016 04:22 PM, Andrew Cooper wrote:
> > On 27/04/2016 22:58, Jim Fehlig wrote:
> >> On 04/25/2016 05:26 AM, osstest service owner wrote:
> >>> flight 92667 libvirt real [real]
> >>> http://logs.test-lab.xenproject.org/osstest/logs/92667/
> >>>
> >>> Regressions :-(
> >>>
> >>> Tests which did not succeed and are blocking,
> >>> including tests which could not be run:
> >>> test-amd64-i386-libvirt 14 guest-saverestore fail REGR. vs. 91479
> >>> test-amd64-amd64-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
> >>> test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
> >>> test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
> >>> test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
> >>> test-amd64-i386-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
> >>> test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 91479
> >>> test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
> >>> test-amd64-amd64-libvirt 14 guest-saverestore fail REGR. vs. 91479
> >> All of these save/restore and migration failures show the following error on the
> >> restore side
> >>
> >> 2016-04-25 10:16:18 UTC libxl: error:
> >> libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [26771] exited
> >> with error status 1
> >> 2016-04-25 10:16:18 UTC libxl: error: libxl_utils.c:507:libxl_read_exactly:
> >> file/stream truncated reading ipc msg header from domain 1 save/restore helper
> >> stdout pipe
> >> 2016-04-25 10:16:18 UTC libxl: error:
> >> libxl_exec.c:129:libxl_report_child_exitstatus: domain 1 save/restore helper
> >> [26772] died due to fatal signal Terminated
> >>
> >> I'm not sure if this problem has already been addressed by recent
> >> migration-related fixes.
> > This is testing two different versions of libvirt against the same
> > version of libxl.
> >
> > Looking at
> > http://logs.test-lab.xenproject.org/osstest/logs/92667/test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm/italia0---var-log-libvirt-libxl-libxl-driver.log
> >
> > 2016-04-25 08:36:03 UTC xc: progress: End of stream: 0/0 0%
> >
> > indicates that the save side is in v2 format (which is expected). (I
> > should add at least an info print in libxl_stream_write() indicating the
> > pertinent details).
> >
> > On the restore side,
> >
> > 2016-04-25 08:36:20 UTC libxl: debug:
> > libxl_stream_read.c:358:stream_header_done: Stream v2 (from legacy)
> > 2016-04-25 08:36:20 UTC libxl: debug:
> > libxl_stream_read.c:574:process_record: Record: 1, length 0
> > 2016-04-25 08:36:20 UTC libxl: error:
> > libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [3909]
> > exited with error status 1
> >
> > which means that the restore code was told that the stream was in legacy
> > format. The legacy conversion script was forked and found that the
> > stream wasn't legacy. (I have no idea where the real error message went
> > from that - it should be plumbed through into a info message, and
> > definitely does work when running `xl` on the command line).
> >
> > I suspect this is breakage from the LIBXL_ABI_VERSION changes.
> >
> > Because of the short-sightest mess that legacy migration was, it is not
> > possible for libxl to distinguish a legacy stream from a v2 stream in
> > libxl_domain_create_restore(). The caller (i.e. libvirt) must provide
> > the correct stream version in libxl_domain_restore_params.
>
> How do I handle the case of a libvirt+Xen migV2 host migrating a domain to a
> libvirt+Xen migV1 host? Do you know how that scenario is handled in xl? Or is
> migrating a domain from migV2 host to migV1 host not supported?
>
I don't think this configuration is supported. Obviously an old xen
system wouldn't have knowledge of the new format.
Wei.
> Thanks for the help.
>
> Regards,
> Jim
> >
> > ~Andrew
> >
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [libvirt test] 92667: regressions - FAIL
2016-04-28 9:54 ` Wei Liu
@ 2016-04-28 10:06 ` Andrew Cooper
0 siblings, 0 replies; 7+ messages in thread
From: Andrew Cooper @ 2016-04-28 10:06 UTC (permalink / raw)
To: Wei Liu, Jim Fehlig; +Cc: xen-devel, osstest service owner
On 28/04/16 10:54, Wei Liu wrote:
> On Wed, Apr 27, 2016 at 08:49:23PM -0600, Jim Fehlig wrote:
>> On 04/27/2016 04:22 PM, Andrew Cooper wrote:
>>> On 27/04/2016 22:58, Jim Fehlig wrote:
>>>> On 04/25/2016 05:26 AM, osstest service owner wrote:
>>>>> flight 92667 libvirt real [real]
>>>>> http://logs.test-lab.xenproject.org/osstest/logs/92667/
>>>>>
>>>>> Regressions :-(
>>>>>
>>>>> Tests which did not succeed and are blocking,
>>>>> including tests which could not be run:
>>>>> test-amd64-i386-libvirt 14 guest-saverestore fail REGR. vs. 91479
>>>>> test-amd64-amd64-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
>>>>> test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
>>>>> test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
>>>>> test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
>>>>> test-amd64-i386-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
>>>>> test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 91479
>>>>> test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
>>>>> test-amd64-amd64-libvirt 14 guest-saverestore fail REGR. vs. 91479
>>>> All of these save/restore and migration failures show the following error on the
>>>> restore side
>>>>
>>>> 2016-04-25 10:16:18 UTC libxl: error:
>>>> libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [26771] exited
>>>> with error status 1
>>>> 2016-04-25 10:16:18 UTC libxl: error: libxl_utils.c:507:libxl_read_exactly:
>>>> file/stream truncated reading ipc msg header from domain 1 save/restore helper
>>>> stdout pipe
>>>> 2016-04-25 10:16:18 UTC libxl: error:
>>>> libxl_exec.c:129:libxl_report_child_exitstatus: domain 1 save/restore helper
>>>> [26772] died due to fatal signal Terminated
>>>>
>>>> I'm not sure if this problem has already been addressed by recent
>>>> migration-related fixes.
>>> This is testing two different versions of libvirt against the same
>>> version of libxl.
>>>
>>> Looking at
>>> http://logs.test-lab.xenproject.org/osstest/logs/92667/test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm/italia0---var-log-libvirt-libxl-libxl-driver.log
>>>
>>> 2016-04-25 08:36:03 UTC xc: progress: End of stream: 0/0 0%
>>>
>>> indicates that the save side is in v2 format (which is expected). (I
>>> should add at least an info print in libxl_stream_write() indicating the
>>> pertinent details).
>>>
>>> On the restore side,
>>>
>>> 2016-04-25 08:36:20 UTC libxl: debug:
>>> libxl_stream_read.c:358:stream_header_done: Stream v2 (from legacy)
>>> 2016-04-25 08:36:20 UTC libxl: debug:
>>> libxl_stream_read.c:574:process_record: Record: 1, length 0
>>> 2016-04-25 08:36:20 UTC libxl: error:
>>> libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [3909]
>>> exited with error status 1
>>>
>>> which means that the restore code was told that the stream was in legacy
>>> format. The legacy conversion script was forked and found that the
>>> stream wasn't legacy. (I have no idea where the real error message went
>>> from that - it should be plumbed through into a info message, and
>>> definitely does work when running `xl` on the command line).
>>>
>>> I suspect this is breakage from the LIBXL_ABI_VERSION changes.
>>>
>>> Because of the short-sightest mess that legacy migration was, it is not
>>> possible for libxl to distinguish a legacy stream from a v2 stream in
>>> libxl_domain_create_restore(). The caller (i.e. libvirt) must provide
>>> the correct stream version in libxl_domain_restore_params.
>> How do I handle the case of a libvirt+Xen migV2 host migrating a domain to a
>> libvirt+Xen migV1 host? Do you know how that scenario is handled in xl? Or is
>> migrating a domain from migV2 host to migV1 host not supported?
>>
> I don't think this configuration is supported. Obviously an old xen
> system wouldn't have knowledge of the new format.
An older Xen (4.5 or earlier) will always produce "legacy" format streams.
A newer Xen (4.6 and later) will always produce v2 format streams. They
can also accept and convert incoming legacy streams, for backwards
compatibility.
Basically, libvirt needs to know the version of libxl on the source side
of the migration, and use this information to inform the local libxl
whether it is being given a legacy or a v2 stream.
I realise this position sucks, but the legacy migration format was
really that broken.
Migration v2 has a deliberate marker which can be unambiguously
distinguished from a legacy stream, but the the issue for libxl is that
the migration fd is typically unseekable, so it can't peek ahead.
~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [libvirt test] 92667: regressions - FAIL
2016-04-27 22:22 ` Andrew Cooper
2016-04-28 2:49 ` Jim Fehlig
@ 2016-04-28 10:20 ` Wei Liu
1 sibling, 0 replies; 7+ messages in thread
From: Wei Liu @ 2016-04-28 10:20 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Jim Fehlig, xen-devel, Wei Liu, osstest service owner
On Wed, Apr 27, 2016 at 11:22:37PM +0100, Andrew Cooper wrote:
> On 27/04/2016 22:58, Jim Fehlig wrote:
> > On 04/25/2016 05:26 AM, osstest service owner wrote:
> >> flight 92667 libvirt real [real]
> >> http://logs.test-lab.xenproject.org/osstest/logs/92667/
> >>
> >> Regressions :-(
> >>
> >> Tests which did not succeed and are blocking,
> >> including tests which could not be run:
> >> test-amd64-i386-libvirt 14 guest-saverestore fail REGR. vs. 91479
> >> test-amd64-amd64-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
> >> test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
> >> test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail REGR. vs. 91479
> >> test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
> >> test-amd64-i386-libvirt-xsm 14 guest-saverestore fail REGR. vs. 91479
> >> test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 91479
> >> test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 guest-saverestore fail REGR. vs. 91479
> >> test-amd64-amd64-libvirt 14 guest-saverestore fail REGR. vs. 91479
> > All of these save/restore and migration failures show the following error on the
> > restore side
> >
> > 2016-04-25 10:16:18 UTC libxl: error:
> > libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [26771] exited
> > with error status 1
> > 2016-04-25 10:16:18 UTC libxl: error: libxl_utils.c:507:libxl_read_exactly:
> > file/stream truncated reading ipc msg header from domain 1 save/restore helper
> > stdout pipe
> > 2016-04-25 10:16:18 UTC libxl: error:
> > libxl_exec.c:129:libxl_report_child_exitstatus: domain 1 save/restore helper
> > [26772] died due to fatal signal Terminated
> >
> > I'm not sure if this problem has already been addressed by recent
> > migration-related fixes.
>
> This is testing two different versions of libvirt against the same
> version of libxl.
>
> Looking at
> http://logs.test-lab.xenproject.org/osstest/logs/92667/test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm/italia0---var-log-libvirt-libxl-libxl-driver.log
>
> 2016-04-25 08:36:03 UTC xc: progress: End of stream: 0/0 0%
>
> indicates that the save side is in v2 format (which is expected). (I
> should add at least an info print in libxl_stream_write() indicating the
> pertinent details).
>
> On the restore side,
>
> 2016-04-25 08:36:20 UTC libxl: debug:
> libxl_stream_read.c:358:stream_header_done: Stream v2 (from legacy)
> 2016-04-25 08:36:20 UTC libxl: debug:
> libxl_stream_read.c:574:process_record: Record: 1, length 0
> 2016-04-25 08:36:20 UTC libxl: error:
> libxl_exec.c:118:libxl_report_child_exitstatus: conversion helper [3909]
> exited with error status 1
>
> which means that the restore code was told that the stream was in legacy
> format. The legacy conversion script was forked and found that the
> stream wasn't legacy. (I have no idea where the real error message went
> from that - it should be plumbed through into a info message, and
> definitely does work when running `xl` on the command line).
>
> I suspect this is breakage from the LIBXL_ABI_VERSION changes.
>
Correct.
Now libvirt specifies API version to be 4.2. The function
libxl_domain_create_restore_0x040200 calls
libxl_domain_restore_params_init, which explicitly sets stream_version
to 1.
A stopgap solution would be to use newer version of the API -- maybe
bump to 4.6 or something. Then we can think about sorting out cross
version migration in libvirt.
FWIW xl has a way of figuring out which version the stream is and set
the version accordingly.
Wei.
> Because of the short-sightest mess that legacy migration was, it is not
> possible for libxl to distinguish a legacy stream from a v2 stream in
> libxl_domain_create_restore(). The caller (i.e. libvirt) must provide
> the correct stream version in libxl_domain_restore_params.
>
> ~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-04-28 10:20 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-04-25 11:26 [libvirt test] 92667: regressions - FAIL osstest service owner
2016-04-27 21:58 ` Jim Fehlig
2016-04-27 22:22 ` Andrew Cooper
2016-04-28 2:49 ` Jim Fehlig
2016-04-28 9:54 ` Wei Liu
2016-04-28 10:06 ` Andrew Cooper
2016-04-28 10:20 ` Wei Liu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).