Re: [xen-unstable bisection] complete test-amd64-amd64-i386-pvgrub

[Julien Grall <julien.grall@linaro.org>]

Hi Wei,

On 07/11/17 15:13, Wei Liu wrote:
> On Tue, Nov 07, 2017 at 03:09:07PM +0000, Julien Grall wrote:
>> Hi Wei,
>>
>> On 06/11/17 14:55, Wei Liu wrote:
>>> On Mon, Nov 06, 2017 at 01:47:56PM +0000, osstest service owner wrote:
>>>> branch xen-unstable
>>>> xenbranch xen-unstable
>>>> job test-amd64-amd64-i386-pvgrub
>>>> testid guest-start
>>>>
>>>> Tree: linux git://xenbits.xen.org/linux-pvops.git
>>>> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
>>>> Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
>>>> Tree: qemuu git://xenbits.xen.org/qemu-xen.git
>>>> Tree: xen git://xenbits.xen.org/xen.git
>>>>
>>>> *** Found and reproduced problem changeset ***
>>>>
>>>>     Bug is in tree:  xen git://xenbits.xen.org/xen.git
>>>>     Bug introduced:  f48b5449dabc770acdde6d25cfbd265cfb71034d
>>>>     Bug not present: 86cf189a957129ea1ad6468fe9a0887b9e2819f3
>>>>     Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/115612/
>>>>
>>>>
>>>>     commit f48b5449dabc770acdde6d25cfbd265cfb71034d
>>>>     Author: Wei Liu <wei.liu2@citrix.com>
>>>>     Date:   Thu Oct 12 20:19:07 2017 +0100
>>>>         tools/dombuilder: Switch to using gfn terminology for console and xenstore rings
>>>>         The sole use of xc_dom_translated() and xc_dom_p2m() outside of the domain
>>>>         builder is for libxl_dom() to translate the console and xenstore pfns back
>>>>         into useful values.  PV guest pfns are only interesting to the domain builder,
>>>>         and gfns are the address space used by all other hypercalls.
>>>>         Renaming the fields in xc_dom_image is deliberate, as it will cause
>>>>         out-of-tree users of the dombuilder to notice the different semantics.
>>>>         Correct the terminology throughout xc_dom_gnttab{_hvm,}_seed(), which are all
>>>>         using gfns despite the existing variable names.
>>>>         Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>>         Reviewed-by: Roger Pau Monn?? <roger.pau@citrix.com>
>>>>         Acked-by: Wei Liu <wei.liu2@citrix.com>
>>>>         Tested-by: Julien Grall <julien.grall@arm.com>
>>>>         Release-acked-by: Julien Grall <julien.grall@linaro.org>
>>>>         [ wei: fix stubdom build ]
>>>>         Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>>>
>>> This has broken pvgrub. The problem is more than just the name of the
>>> variables. I have reverted this and its successor patch.
>>
>> It looks like osstest is still broken after the patches you reverted (see
>> [1] and [2]).
>>
>> AFAICT, the only series between the two flights is the dombuilder, there are
>> 2 patches not reverted.
>>
>> Do you have an idea of what's going on?
>>
>> Cheers,
>>
>> [1] http://logs.test-lab.xenproject.org/osstest/logs/115624/
>> [2]
>> https://lists.xenproject.org/archives/html/xen-devel/2017-11/msg00391.html
>>
> 
> test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs.  115526
> test-armhf-armhf-xl-vhd     15 guest-start/debian.repeat fail REGR. vs.  115526

The log for the xl-vhd contains ([1])

libxl: error: libxl_bootloader.c:283:bootloader_local_detached_cb: Domain 11:unable to detach locally attached disk
libxl: error: libxl_create.c:1246:domcreate_rebuild_done: Domain 11:cannot (re-)build domain: -3
libxl: debug: libxl_domain.c:1138:devices_destroy_cb: Domain 11:Forked pid 5103 for destroy of domain
libxl: debug: libxl_create.c:1683:do_domain_create: Domain 0:ao 0x5d6e8: inprogress: poller=0x56ad8, flags=i
libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x5d6e8: complete, rc=-3
libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x5d6e8: destroy
libxl: debug: libxl_domain.c:868:libxl_domain_destroy: Domain 11:ao 0x5a170: create: how=(nil) callback=(nil) poller=0x56ad8
libxl: error: libxl_domain.c:1000:libxl__destroy_domid: Domain 11:Non-existant domain
libxl: error: libxl_domain.c:959:domain_destroy_callback: Domain 11:Unable to destroy guest
libxl: error: libxl_domain.c:886:domain_destroy_cb: Domain 11:Destruction of domain failed
libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x5a170: complete, rc=-21
libxl: debug: libxl_domain.c:877:libxl_domain_destroy: Domain 11:ao 0x5a170: inprogress: poller=0x56ad8, flags=ic
libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x5a170: destroy

It is in guest repeat and has succeed few times before.

Looking at the success/failure ([2]), the same configuration passed on the Arndale
(see 115580) but fails reliably on the cubietruck.

My guess would be the disk is not detached by the previous guest in time.
Now the question is why? I am not familiar with this area, any ideas? 

> 
> These aren't related to dombuilder at first glance.
> 

Cheers,

[1] http://logs.test-lab.xenproject.org/osstest/logs/115624/test-armhf-armhf-xl-vhd/15.ts-repeat-test.log
[2] http://logs.test-lab.xenproject.org/osstest/results/history/test-armhf-armhf-xl-vhd/xen-unstable

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel