Re: [xen-unstable test] 57852: regressions - FAIL

[Ian Campbell <ian.campbell@citrix.com>]

On Mon, 2015-06-08 at 09:50 -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Jun 08, 2015 at 10:27:32AM +0100, Ian Campbell wrote:
> > On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote:
> > > >>> On 08.06.15 at 10:53, <ian.campbell@citrix.com> wrote:
> > > > That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1. Which
> > > > differs form the apparent xen-unstable failure rate. But I wouldn't take
> > > > this as evidence that the two systems differ significantly, despite how
> > > > the unstable results looked at first glance.
> > > 
> > > So we can basically rule out just one of the hosts being the culprit;
> > > it's either both or our software. Considering that (again at the
> > > example of the recent 4.2 flight) the guest is apparently waiting for
> > > a timer (or other) interrupt (on a HLT instruction), this is very likely
> > > interrupt delivery related, yet (as said before, albeit wrongly for
> > > 4.3) 4.2 doesn't have APICV support yet (4.3 only lack the option
> > > to disable it), so it can't be that (alone).
> > > 
> > > Looking at the hardware - are fiano[01], in terms of CPU and
> > > chipset, perhaps the newest or oldest in the pool? (I'm trying to
> > > make myself a picture of what debugging options we have.)
> > 
> > I don't know much about the hardware in the pool other than what can be
> > gathered from the serial and dmesg logs.
> > 
> > http://logs.test-lab.xenproject.org/osstest/logs/58028/test-amd64-amd64-xl-qemuu-win7-amd64/info.html
> > 
> > >From the serial log and this:
> > 
> > Jun  6 12:09:27.089020 (XEN) VMX: Supported advanced features:
> > Jun  6 12:09:27.089052 (XEN)  - APIC MMIO access virtualisation
> > Jun  6 12:09:27.097051 (XEN)  - APIC TPR shadow
> > Jun  6 12:09:27.097088 (XEN)  - Extended Page Tables (EPT)
> > Jun  6 12:09:27.097118 (XEN)  - Virtual-Processor Identifiers (VPID)
> > Jun  6 12:09:27.105066 (XEN)  - Virtual NMI
> > Jun  6 12:09:27.105100 (XEN)  - MSR direct-access bitmap
> > Jun  6 12:09:27.105130 (XEN)  - Unrestricted Guest
> > Jun  6 12:09:27.113269 (XEN)  - APIC Register Virtualization
> > Jun  6 12:09:27.113290 (XEN)  - Virtual Interrupt Delivery
> > Jun  6 12:09:27.113328 (XEN)  - Posted Interrupt Processing
> > Jun  6 12:09:27.121180 (XEN) HVM: ASIDs enabled.
> > Jun  6 12:09:27.121235 (XEN) HVM: VMX enabled
> > Jun  6 12:09:27.121267 (XEN) HVM: Hardware Assisted Paging (HAP) detected
> > Jun  6 12:09:27.129069 (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> > 
> > I guess they are pretty new?
> 
> Could it be an missing microcode update? I don't know if the OSSTest does
> the ucode=scan or updates the microcode later?

I rather suspect it doesn't do microcode updates at all. (It probably
should)

Is there some reason to expect APICV (or something else) would cause
these failures if microcode wasn't up to date?

Ian.