Re: Windows "heinsenbug" (WAS: Re: Notes Design Session: Making Releases Lessons Learned: Improving Our Release Process and Tooling)

[Paul Durrant <Paul.Durrant@citrix.com>]

> -----Original Message-----
> From: Julien Grall [mailto:julien.grall@arm.com]
> Sent: 14 September 2017 03:49
> To: Paul Durrant <Paul.Durrant@citrix.com>; Ian Jackson
> <Ian.Jackson@citrix.com>; Jan Beuli ch <jbeulich@suse.com>
> Cc: Juergen Gross <jgross@suse.com>; Wei Liu <wei.liu2@citrix.com>;
> ross.philipson@gmail.com; lars.kurth.xen@gmail.com; xen-
> devel@lists.xen.org; committers@xenproject.org
> Subject: Re: [Xen-devel] Windows "heinsenbug" (WAS: Re: Notes Design
> Session: Making Releases Lessons Learned: Improving Our Release Process
> and Tooling)
> 
> Hi Paul,
> 
> On 13/09/17 19:26, Paul Durrant wrote:
> >> -----Original Message-----
> >> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
> >> Julien Grall
> >> Sent: 13 September 2017 10:52
> >> To: Ian Jackson <Ian.Jackson@citrix.com>; Jan Beuli ch
> <jbeulich@suse.com>
> >> Cc: Juergen Gross <jgross@suse.com>; Wei Liu <wei.liu2@citrix.com>;
> >> ross.philipson@gmail.com; lars.kurth.xen@gmail.com; xen-
> >> devel@lists.xen.org; committers@xenproject.org
> >> Subject: [Xen-devel] Windows "heinsenbug" (WAS: Re: Notes Design
> >> Session: Making Releases Lessons Learned: Improving Our Release
> Process
> >> and Tooling)
> >>
> >> Hi all,
> >>
> >> On 07/27/2017 05:56 PM, Ian Jackson wrote:
> >>> Jan Beulich writes ("Re: Notes Design Session: Making Releases Lessons
> >> Learned: Improving Our Release Process and Tooling"):
> >>>> [Lars:]
> >>>>> Ian: These are most likely software problems, most likely in Xen.
> >>>>> ISSUE: nobody wants to debug Windows Heisenbugs
> >>>>
> >>>> Hmm, I think we believe to have understood what the issue is, it's just
> >> that
> >>>> the change needed is on the tool stack side (gracefully deal with a
> domain
> >>>> rebooting while being migrated).
> >>>
> >>> No, the problem is that we don't understand why the domain is
> >>> rebooting.  It should not be.
> >>
> >> This bug still seem to be around and likely going to impact once again
> >> the release testing.
> >>
> >> As discussed during the summit, I am really thinking to make this
> >> "heinsenbug" a blocker until somehow finally spend sometimes
> >> investigating the problem and suggest actions.
> >
> > If testing was done with PV drivers installed then we could get a lot more
> information into the logs. BSODs vs. clean shutdowns (perhaps due to
> Windows Update) would then become obvious.
> 
> Would it be feasible to get those installed in osstest?
> 

Julien,

  That should not be too hard. We have a driver install agent for XenServer that will manage all the necessary driver staging and required reboots. Hopefully it should mainly be a question of building that installer to pick up drivers from the upstream master branches rather than the internal XenServer branches. AFAIK osstest is using XenRT customized Windows images so hopefully we can use the slipstreamed XenRT agent to then fetch and run the driver installer. IIRC it sets some form of cookie in xenstore once it is done, so the rest of the test cycle would just need to wait for that to appear.

  I'll look into how to progress this when I get back from the US.

  Cheers,

    Paul

> Cheers,
> 
> --
> Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel