Windows install problems on Xeon E5-2403 v2

Windows install problems on Xeon E5-2403 v2

[Jan Beulich]

All,

we're seeing recurring but intermittent failures in osstest on just the
two hosts using these particular CPU types. The closest similar ones,
using Xeon E5-2407 v2, do not exhibit this same behavior. We do
note that the latter run at microcode level 0x427, while the problem
ones are only at 0x416, and IanC is in the process of trying to
integrate microcode updates with osstest. However, since there
doesn't seem to be any matching erratum documented of these CPU
models, we aren't really assuming that a microcode update (0x428
appears to be the newest available) would help here, and hence
we'd like to ask for your assistance in possible further diagnostic
steps, or suggestions towards a resolution.

For more context, please see also the mail thread starting at
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg00522.html.

Thanks, Jan

Re: Windows install problems on Xeon E5-2403 v2

[Tian, Kevin]

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Thursday, June 11, 2015 2:58 PM
> 
> All,
> 
> we're seeing recurring but intermittent failures in osstest on just the
> two hosts using these particular CPU types. The closest similar ones,
> using Xeon E5-2407 v2, do not exhibit this same behavior. We do
> note that the latter run at microcode level 0x427, while the problem
> ones are only at 0x416, and IanC is in the process of trying to
> integrate microcode updates with osstest. However, since there
> doesn't seem to be any matching erratum documented of these CPU
> models, we aren't really assuming that a microcode update (0x428
> appears to be the newest available) would help here, and hence
> we'd like to ask for your assistance in possible further diagnostic
> steps, or suggestions towards a resolution.
> 
> For more context, please see also the mail thread starting at
> http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg00522.html.
> 
> Thanks, Jan

That's a pretty long thread. Could you summarize what's the
error phenomenon? Is the only clue related to microcode so far?

I'll ping our internal development team whether we can find a
same model to reproduce. Could someone post the model number 
here?

Thanks
Kevin

Re: Windows install problems on Xeon E5-2403 v2

[Jan Beulich]

>>> On 11.06.15 at 09:37, <kevin.tian@intel.com> wrote:
>>  From: Jan Beulich [mailto:JBeulich@suse.com]
>> Sent: Thursday, June 11, 2015 2:58 PM
>> 
>> All,
>> 
>> we're seeing recurring but intermittent failures in osstest on just the
>> two hosts using these particular CPU types. The closest similar ones,
>> using Xeon E5-2407 v2, do not exhibit this same behavior. We do
>> note that the latter run at microcode level 0x427, while the problem
>> ones are only at 0x416, and IanC is in the process of trying to
>> integrate microcode updates with osstest. However, since there
>> doesn't seem to be any matching erratum documented of these CPU
>> models, we aren't really assuming that a microcode update (0x428
>> appears to be the newest available) would help here, and hence
>> we'd like to ask for your assistance in possible further diagnostic
>> steps, or suggestions towards a resolution.
>> 
>> For more context, please see also the mail thread starting at
>> http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg00522.html.
> 
> That's a pretty long thread. Could you summarize what's the
> error phenomenon?

Did you at least take a look at the logs of the referenced failure? I
ask because the failure _is_ the phenomenon; we can't really see
what may be wrong with the guest. We can only guess that maybe
it's not receiving (all intended) interrupts anymore.

Ad-hoc flights run by IanC hint towards "no-apicv" eliminating the
bad behavior, but didn't give 100% confidence in that.

> Is the only clue related to microcode so far?

That's not even a clue, but a wild guess.

Jan

Re: Windows install problems on Xeon E5-2403 v2

[Ian Campbell]

On Thu, 2015-06-11 at 09:01 +0100, Jan Beulich wrote:
> >>> On 11.06.15 at 09:37, <kevin.tian@intel.com> wrote:
> >>  From: Jan Beulich [mailto:JBeulich@suse.com]
> >> Sent: Thursday, June 11, 2015 2:58 PM
> >> 
> >> All,
> >> 
> >> we're seeing recurring but intermittent failures in osstest on just the
> >> two hosts using these particular CPU types. The closest similar ones,
> >> using Xeon E5-2407 v2, do not exhibit this same behavior. We do
> >> note that the latter run at microcode level 0x427, while the problem
> >> ones are only at 0x416, and IanC is in the process of trying to
> >> integrate microcode updates with osstest.

microcode is now being updated automatically in osstest however we are
still seeing incidences of this sort of failure (e.g. [0]). Possibly
less frequent although it is too early to say, either way it still needs
investigation IMHO.

Ian.

[0] http://logs.test-lab.xenproject.org/osstest/logs/58727/test-amd64-amd64-xl-qemuu-win7-amd64/info.html

>  However, since there
> >> doesn't seem to be any matching erratum documented of these CPU
> >> models, we aren't really assuming that a microcode update (0x428
> >> appears to be the newest available) would help here, and hence
> >> we'd like to ask for your assistance in possible further diagnostic
> >> steps, or suggestions towards a resolution.
> >> 
> >> For more context, please see also the mail thread starting at
> >> http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg00522.html.
> > 
> > That's a pretty long thread. Could you summarize what's the
> > error phenomenon?
> 
> Did you at least take a look at the logs of the referenced failure? I
> ask because the failure _is_ the phenomenon; we can't really see
> what may be wrong with the guest. We can only guess that maybe
> it's not receiving (all intended) interrupts anymore.
> 
> Ad-hoc flights run by IanC hint towards "no-apicv" eliminating the
> bad behavior, but didn't give 100% confidence in that.
> 
> > Is the only clue related to microcode so far?
> 
> That's not even a clue, but a wild guess.
> 
> Jan
>