[RFC] Overriding MSR values via xl.cfg

[RFC] Overriding MSR values via xl.cfg

[Eric Shelton]

Sometimes, it is helpful to persuade a guest OS that it is running on
a particular CPU model, or that a CPU has (or does not have)
particular features.  For example, this may ease migrating guests
across a heterogeneous pool of systems.  Currently, via an xl.cfg file
you can specify specific masks or values to be returned for the CPUID
instruction.  This is an example of the syntax being used:

cpuid = [ '0:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
           '1:eax=0x06b1,
              ecx=xxxxxxxxxx0000xx00xxx0000000xx0,
              edx=xx00000xxxxxxx0xxxxxxxxx0xxxxxx',
           '4:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
  '0x80000000:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0']

MSRs provide another mechanism for a guest to collect information
about the system on which it is running.  For much the same reasons it
can be useful to change CPUID values returned to a guest, it could
also be useful to be able to override and specify particular MSR
values to be returned to a guest, and for this to be done via an
xl.cfg.  This would only affect RDMSR return values, and would not
affect WRMSR behavior.  The syntax in xl.cfg could be simlar to what
is used for CPUID.

Additionally, it seems like this capability would be useful for debug,
development, and testing of guest operating systems, or the Xen
hypervisor itself.

I think the current implementation for the CPUID instruction, both in
the hypervisor and toolchain, provides a reasonable prototype for
implementing the above functionality.  However, before I pursue this,
I wanted to gauge the acceptability of and interest in adding this
capability to Xen.

Thanks,
Eric

Re: [RFC] Overriding MSR values via xl.cfg

[Jan Beulich]

>>> On 17.09.14 at 19:37, <eshelton@pobox.com> wrote:
> Sometimes, it is helpful to persuade a guest OS that it is running on
> a particular CPU model, or that a CPU has (or does not have)
> particular features.  For example, this may ease migrating guests
> across a heterogeneous pool of systems.  Currently, via an xl.cfg file
> you can specify specific masks or values to be returned for the CPUID
> instruction.  This is an example of the syntax being used:
> 
> cpuid = [ '0:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
>            '1:eax=0x06b1,
>               ecx=xxxxxxxxxx0000xx00xxx0000000xx0,
>               edx=xx00000xxxxxxx0xxxxxxxxx0xxxxxx',
>            '4:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
>   '0x80000000:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0']
> 
> MSRs provide another mechanism for a guest to collect information
> about the system on which it is running.  For much the same reasons it
> can be useful to change CPUID values returned to a guest, it could
> also be useful to be able to override and specify particular MSR
> values to be returned to a guest, and for this to be done via an
> xl.cfg.  This would only affect RDMSR return values, and would not
> affect WRMSR behavior.  The syntax in xl.cfg could be simlar to what
> is used for CPUID.
> 
> Additionally, it seems like this capability would be useful for debug,
> development, and testing of guest operating systems, or the Xen
> hypervisor itself.
> 
> I think the current implementation for the CPUID instruction, both in
> the hypervisor and toolchain, provides a reasonable prototype for
> implementing the above functionality.  However, before I pursue this,
> I wanted to gauge the acceptability of and interest in adding this
> capability to Xen.

While from an abstract perspective this would seem a feature that
could indeed be useful here, I think the resemblance with CPUID
handling is less than what you describe: The value calculation done
in the tool stack, the physical value to merge with (in case of a
partial bit-wise override) is ambiguous: The tool stack (running in
Dom0) can only see whatever it gets to see, which (a) may be
different between PV and PVH Dom0, (b) would likely differ from
what the guest would get to see in the absence of an override,
and (c) might be more heavily dependent on the pCPU it's being
determined on.

Jan

Re: [RFC] Overriding MSR values via xl.cfg

[Konrad Rzeszutek Wilk]

On Thu, Sep 18, 2014 at 08:48:46AM +0100, Jan Beulich wrote:
> >>> On 17.09.14 at 19:37, <eshelton@pobox.com> wrote:
> > Sometimes, it is helpful to persuade a guest OS that it is running on
> > a particular CPU model, or that a CPU has (or does not have)
> > particular features.  For example, this may ease migrating guests
> > across a heterogeneous pool of systems.  Currently, via an xl.cfg file
> > you can specify specific masks or values to be returned for the CPUID
> > instruction.  This is an example of the syntax being used:
> > 
> > cpuid = [ '0:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
> >            '1:eax=0x06b1,
> >               ecx=xxxxxxxxxx0000xx00xxx0000000xx0,
> >               edx=xx00000xxxxxxx0xxxxxxxxx0xxxxxx',
> >            '4:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
> >   '0x80000000:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0']
> > 
> > MSRs provide another mechanism for a guest to collect information
> > about the system on which it is running.  For much the same reasons it
> > can be useful to change CPUID values returned to a guest, it could
> > also be useful to be able to override and specify particular MSR
> > values to be returned to a guest, and for this to be done via an
> > xl.cfg.  This would only affect RDMSR return values, and would not
> > affect WRMSR behavior.  The syntax in xl.cfg could be simlar to what
> > is used for CPUID.
> > 
> > Additionally, it seems like this capability would be useful for debug,
> > development, and testing of guest operating systems, or the Xen
> > hypervisor itself.
> > 
> > I think the current implementation for the CPUID instruction, both in
> > the hypervisor and toolchain, provides a reasonable prototype for
> > implementing the above functionality.  However, before I pursue this,
> > I wanted to gauge the acceptability of and interest in adding this
> > capability to Xen.
> 
> While from an abstract perspective this would seem a feature that
> could indeed be useful here, I think the resemblance with CPUID
> handling is less than what you describe: The value calculation done
> in the tool stack, the physical value to merge with (in case of a
> partial bit-wise override) is ambiguous: The tool stack (running in
> Dom0) can only see whatever it gets to see, which (a) may be
> different between PV and PVH Dom0, (b) would likely differ from
> what the guest would get to see in the absence of an override,
> and (c) might be more heavily dependent on the pCPU it's being
> determined on.

The problems you describe are based more on the 'sourcing' of the
CPUID values and how choose different paths for PV and HVM (and PVH).

One starts with a nice clean plate and adds bits while the other
takes what it sees and clears the bits. It ought to really be
streamlined in one code path.

I think Eric is describing the cases for 'string' parsing of
what feature we want enabled/disabled or the hex magic. The 'string'
parsing and then uploading which MSR we want and what value it
we want to report to the guest would be quite useful.

There is no way it can be done for Xen 4.5 but this would be quite
a neat thing to get done Xen 4.6.

Eric, is that what you were thinking about?

> 
> Jan
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [RFC] Overriding MSR values via xl.cfg

[Andrew Cooper]

On 17/09/14 18:37, Eric Shelton wrote:
> Sometimes, it is helpful to persuade a guest OS that it is running on
> a particular CPU model, or that a CPU has (or does not have)
> particular features.  For example, this may ease migrating guests
> across a heterogeneous pool of systems.  Currently, via an xl.cfg file
> you can specify specific masks or values to be returned for the CPUID
> instruction.  This is an example of the syntax being used:
>
> cpuid = [ '0:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
>            '1:eax=0x06b1,
>               ecx=xxxxxxxxxx0000xx00xxx0000000xx0,
>               edx=xx00000xxxxxxx0xxxxxxxxx0xxxxxx',
>            '4:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
>   '0x80000000:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0']
>
> MSRs provide another mechanism for a guest to collect information
> about the system on which it is running.  For much the same reasons it
> can be useful to change CPUID values returned to a guest, it could
> also be useful to be able to override and specify particular MSR
> values to be returned to a guest, and for this to be done via an
> xl.cfg.  This would only affect RDMSR return values, and would not
> affect WRMSR behavior.  The syntax in xl.cfg could be simlar to what
> is used for CPUID.
>
> Additionally, it seems like this capability would be useful for debug,
> development, and testing of guest operating systems, or the Xen
> hypervisor itself.
>
> I think the current implementation for the CPUID instruction, both in
> the hypervisor and toolchain, provides a reasonable prototype for
> implementing the above functionality.  However, before I pursue this,
> I wanted to gauge the acceptability of and interest in adding this
> capability to Xen.
>
> Thanks,
> Eric

Which MSRs are you considering for overrides?  In principle I can see
this being useful, although can't currently think of an MSR which is
safe to blanket-override in this way.

The CPUID framework currently has some problems.  Xen itself, and all
components higher up, lack any concept of per-core and per-package CPUID
values.  What currently exists is fine for uni-vcpu guests, but was
never fixed when mutli-vcpu guest support was added.

I will be trying to fix this in the 4.6 development timeframe, but
exactly the same kind of distinction would need to be made for MSRs.

~Andrew