vmcs_revision_id: do we care?

vmcs_revision_id: do we care?

[Daniel Li]

Right now when xen finds out a SMP host has processors with different 
vmcs_revision_id, it considers it a bug.

xen/arch/x86/hvm/vmx/vmcs.c:vmx_init_vmcs_config():

rdmsr(MSR_IA32_VMX_BASIC_MSR, vmx_msr_low, vmx_msr_high);

if ( smp_processor_id() == 0 )
{
    vmcs_revision_id = vmx_msr_low;
}
else
{
    BUG_ON(vmcs_revision_id != vmx_msr_low);
}

Does anyone know why are we doing this? Can we change that check into a 
warning message?

Re: vmcs_revision_id: do we care?

[Keir Fraser]

No, we'll simply fail a VMPTRLD the first time we load a VMCS onto a
different processor than the one it was initialised on. This configuration
is actually quite hard to support and I don't see that it's worth it (even
though I know there are a few boxes from a certain tier-1 vendor out there
exhibiting this issue). It's really not a valid SMP system configuration.

 -- Keir

On 8/1/08 22:40, "Daniel Li" <dli@virtualiron.com> wrote:

> Right now when xen finds out a SMP host has processors with different
> vmcs_revision_id, it considers it a bug.
> 
> xen/arch/x86/hvm/vmx/vmcs.c:vmx_init_vmcs_config():
> 
> rdmsr(MSR_IA32_VMX_BASIC_MSR, vmx_msr_low, vmx_msr_high);
> 
> if ( smp_processor_id() == 0 )
> {
>     vmcs_revision_id = vmx_msr_low;
> }
> else
> {
>     BUG_ON(vmcs_revision_id != vmx_msr_low);
> }
> 
> Does anyone know why are we doing this? Can we change that check into a
> warning message?
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: vmcs_revision_id: do we care?

[Daniel Li]

Based on what I've heard, both IBM and Intel have been telling people 
this configuration is valid.  From what I was able to dig out, Intel in 
their System Programming Guide: Virtual Machine Monitor Programming 
Considerations, published Nov. 2007, suggested when loading a VMCS onto 
a processor with a different vmcs_revision_id, instead of using VMPTRLD, 
one should use VMREAD to read out each field then use VMWRITE to write 
them back. Is that more or less  the kind of work we are facing here? Or 
is there more?

Thanks,
Dan

Re: vmcs_revision_id: do we care?

[Samuel Thibault]

Daniel Li, le Wed 09 Jan 2008 11:13:48 -0500, a écrit :
> Based on what I've heard, both IBM and Intel have been telling people 
> this configuration is valid.  From what I was able to dig out, Intel in 
> their System Programming Guide: Virtual Machine Monitor Programming 
> Considerations, published Nov. 2007, suggested when loading a VMCS onto 
> a processor with a different vmcs_revision_id, instead of using VMPTRLD, 
> one should use VMREAD to read out each field then use VMWRITE to write 
> them back.

Which would mean quite some burden, even if valid... Buying a machine
that may require such slowdown is a no-go for me :)

Samuel

Re: vmcs_revision_id: do we care?

[Keir Fraser]

Yes, it would be along those lines. Also HVM save/restore should probably
not be checking for identical CPU stepping number, if it's actually allowed
that steppings can differ even within the same machine!

It's probably not too hard to knock up for 3.2.1, and maybe in time for
3.1.3, but it's bound to have some performance impact (but only on
mixed-stepping systems). I'm not sure how much that impact will be.

 -- Keir

On 9/1/08 16:13, "Daniel Li" <dli@virtualiron.com> wrote:

> Based on what I've heard, both IBM and Intel have been telling people
> this configuration is valid.  From what I was able to dig out, Intel in
> their System Programming Guide: Virtual Machine Monitor Programming
> Considerations, published Nov. 2007, suggested when loading a VMCS onto
> a processor with a different vmcs_revision_id, instead of using VMPTRLD,
> one should use VMREAD to read out each field then use VMWRITE to write
> them back. Is that more or less  the kind of work we are facing here? Or
> is there more?
> 
> Thanks,
> Dan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: vmcs_revision_id: do we care?

[Keir Fraser]

The specific situation we are seeing here, I think, is described in the
following Intel Product Change Notification:
http://intel.pcnalert.com/content/eolpcn/PCN107467-00.pdf

It affects Xeon 51xx and 53xx parts, whose steppings change from B-2 or B-3
to G-0, with some accompanying visible changes to feature set.

However, notice that to properly support this mixed-stepping configuration
there is supposed to be a microcode and BIOS update to hide the change
(Customer Impact of Change and Recommended Action). This will presumably
modify CPU behaviour to advertise matching stepping numbers and features.

So, presumably some IBM boxes do not have a suitably modified BIOS and hence
OS software can see the mixed steppings. That's not really a preferred mode
of operation as far as Intel are concerned, I'm pretty sure. The one
sentence in the Intel reference manual referring to mixed VMCS version
numbers doesn't convince me otherwise!

Looking at e.g., 
https://www-304.ibm.com/jct01004c/systems/support/supportsite.wss/docdisplay
?lndocid=MIGR-64501&brandind=5000008 there's no mention in the BIOS revision
history that support for B/G mixed steppings has been fixed. :-(

 -- Keir

On 9/1/08 16:13, "Daniel Li" <dli@virtualiron.com> wrote:

> Based on what I've heard, both IBM and Intel have been telling people
> this configuration is valid.  From what I was able to dig out, Intel in
> their System Programming Guide: Virtual Machine Monitor Programming
> Considerations, published Nov. 2007, suggested when loading a VMCS onto
> a processor with a different vmcs_revision_id, instead of using VMPTRLD,
> one should use VMREAD to read out each field then use VMWRITE to write
> them back. Is that more or less  the kind of work we are facing here? Or
> is there more?
> 
> Thanks,
> Dan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel