PVH cleanups after 4.5

PVH cleanups after 4.5

[Tim Deegan]

Hi all,

At the Hackathon in Rackspace we discussed a plan to tidy up the
hypervisor side of PVH code so that 'PVH' stops being a separate guest
type, becoming instead a HVM guest with various features disabled (and
one or two other tweaks). 

Although I had hoped to work on that in the meantime, various other
things have got in the way.  But I'd like to at least document the
plan in the hope of getting back on track in the 4.6 development
cycle.

The plan goes something like this:

1. Merge the PVH and HVM hypercall tables.  Patches were already
   posted for this but will need rebasing, and probably more scrutiny.

2. Make PVH-ness a flag on a HVM guest, retaining all the current
   has_hvm_container/is_pvh_domain tests as-is but dropping the
   three-way guest type.

3. Add feature flags to HVM guests that disable various features. 
   These flags should be set once, at domain creation/build, and
   for sanity of testing we should only allow two combinations
   of settings, corresponding to the current HVM and PVH types.
   Make sure that all PVH guests have the PVH feature set.

4. Replace tests for pvh-ness with feature-flag tests wherever
   possible.

5. Fix any outstanding is_pvh tests -- expect this mostly to be 
   feature compatibility with other HVM features (e.g. paging).
   Hopefully we can remove those constraints, or make them constraints
   on a feature set (e.g. can't have PCI passthrough w/out an
   emulated PCI controller).  This also needs an audit of is_hvm tests,
   which are implicitly !is_pvh at the moment.

6. Remove 'PVH' from the hypercall interface, and remove the PVH flag
   from the HVM domain struct.  At this point Xen no longer treats PVH
   and HVM as different (though the tools and the guests themselves
   can maintain the distinction).

Potential feature flags, based on whiteboard notes at the session.
Things that are 'Yes' in both columns might not need actual flags :)

                     'HVM'       'PVH'
64bit hypercalls      Yes         Yes
32bit hypercalls      Yes         No
Paging assistance     Yes         Yes
ioreq-servers         Yes         No
HVM-style CPUID       Yes         Yes
Interrupt controllers Yes         No     ([x2]apic, ioapic, pic &c)
Timers                Yes         No     (rtc, hpet, pit, pmtimer)
Machine Check regs    Yes         Yes
Emulated PCI          Yes         No     (PVH to use pcifront?)

This plan doesn't explicitly add things that we might like to happen
for PVH in 4.6 (e.g. PCI passthrough, compat hypercalls) but it ought
to make some of them easier, and we might get some (e.g. shadow
pagetables) for free as the differences between PVH and HVM go away.

I would like to work on this stuff, but I can't really guarantee to
get anything done for 4.6 in the time I have available.  Any
volunteers to help out would be welcomed!  Likewise, any feedback on
the overall plan is welcome before any more work gets done. :)

Cheers,

Tim.

Re: PVH cleanups after 4.5

[Jan Beulich]

>>> On 04.12.14 at 18:25, <tim@xen.org> wrote:
> Potential feature flags, based on whiteboard notes at the session.
> Things that are 'Yes' in both columns might not need actual flags :)
> 
>                      'HVM'       'PVH'
> 64bit hypercalls      Yes         Yes
> 32bit hypercalls      Yes         No

Iiuc the lack of support of 32-bit hypercalls is simply because PVH
guests aren't expected to use them as being always 64-bit right
now. I.e. I can't really see why we couldn't just enable them once
the 64-bit hypercall tables got combined, in which case we wouldn't
need a feature flag here either.

Jan

Re: PVH cleanups after 4.5

[Tim Deegan]

At 09:20 +0000 on 05 Dec (1417767654), Jan Beulich wrote:
> >>> On 04.12.14 at 18:25, <tim@xen.org> wrote:
> > Potential feature flags, based on whiteboard notes at the session.
> > Things that are 'Yes' in both columns might not need actual flags :)
> > 
> >                      'HVM'       'PVH'
> > 64bit hypercalls      Yes         Yes
> > 32bit hypercalls      Yes         No
> 
> Iiuc the lack of support of 32-bit hypercalls is simply because PVH
> guests aren't expected to use them as being always 64-bit right
> now. I.e. I can't really see why we couldn't just enable them once
> the 64-bit hypercall tables got combined, in which case we wouldn't
> need a feature flag here either.

Agreed -- I think the same will apply to a few other things, like shadow
pagetables and some of the other MM tricks.  

Tim.

Re: PVH cleanups after 4.5

[Ian Campbell]

On Fri, 2014-12-05 at 10:49 +0100, Tim Deegan wrote:
> At 09:20 +0000 on 05 Dec (1417767654), Jan Beulich wrote:
> > >>> On 04.12.14 at 18:25, <tim@xen.org> wrote:
> > > Potential feature flags, based on whiteboard notes at the session.
> > > Things that are 'Yes' in both columns might not need actual flags :)
> > > 
> > >                      'HVM'       'PVH'
> > > 64bit hypercalls      Yes         Yes
> > > 32bit hypercalls      Yes         No
> > 
> > Iiuc the lack of support of 32-bit hypercalls is simply because PVH
> > guests aren't expected to use them as being always 64-bit right
> > now. I.e. I can't really see why we couldn't just enable them once
> > the 64-bit hypercall tables got combined, in which case we wouldn't
> > need a feature flag here either.
> 
> Agreed -- I think the same will apply to a few other things, like shadow
> pagetables and some of the other MM tricks.  

Might we want to constrain a given PVH domain to only make 32- or 64-bit
hypercalls?

Or do we consider already having crossed that bridge with HVM enough
reason to allow it for PVH? I'm wonder if that, even if it is
technically possible to support not, doing so might mitigate some
potential security issues down the line. There's obviously a tradeoff
against in-guest flexibility though.

Ian.

Re: PVH cleanups after 4.5

[Andrew Cooper]

On 05/12/14 09:54, Ian Campbell wrote:
> On Fri, 2014-12-05 at 10:49 +0100, Tim Deegan wrote:
>> At 09:20 +0000 on 05 Dec (1417767654), Jan Beulich wrote:
>>>>>> On 04.12.14 at 18:25, <tim@xen.org> wrote:
>>>> Potential feature flags, based on whiteboard notes at the session.
>>>> Things that are 'Yes' in both columns might not need actual flags :)
>>>>
>>>>                      'HVM'       'PVH'
>>>> 64bit hypercalls      Yes         Yes
>>>> 32bit hypercalls      Yes         No
>>> Iiuc the lack of support of 32-bit hypercalls is simply because PVH
>>> guests aren't expected to use them as being always 64-bit right
>>> now. I.e. I can't really see why we couldn't just enable them once
>>> the 64-bit hypercall tables got combined, in which case we wouldn't
>>> need a feature flag here either.
>> Agreed -- I think the same will apply to a few other things, like shadow
>> pagetables and some of the other MM tricks.  
> Might we want to constrain a given PVH domain to only make 32- or 64-bit
> hypercalls?
>
> Or do we consider already having crossed that bridge with HVM enough
> reason to allow it for PVH? I'm wonder if that, even if it is
> technically possible to support not, doing so might mitigate some
> potential security issues down the line. There's obviously a tradeoff
> against in-guest flexibility though.

Madating a 32/64bit split serves only to cause booting issues; you need
to know a-priori what the eventual kernel is going to be before you
build the domain. This is an awkward issue with PV domains which
*really* wants not to apply to PVH as well.

PVH guests with the plan of "HVM - qemu" should be able to fully choose
their operating mode, and allow for in-guest bootstrapping which is far
superior from a security/isolation point of view than toolstack
bootstrapping.

~Andrew

Re: PVH cleanups after 4.5

[David Vrabel]

On 04/12/14 17:25, Tim Deegan wrote:
>                      'HVM'       'PVH'
> 64bit hypercalls      Yes         Yes
> 32bit hypercalls      Yes         No
> Paging assistance     Yes         Yes
> ioreq-servers         Yes         No

                                    Perhaps, but no default one.  This
would be required for supporting virtual GPU passthrough to a PVH guest.

> HVM-style CPUID       Yes         Yes
> Interrupt controllers Yes         No     ([x2]apic, ioapic, pic &c)

                                    Yes, if enough APIC virtualization
hardware is available.

> Timers                Yes         No     (rtc, hpet, pit, pmtimer)
> Machine Check regs    Yes         Yes
> Emulated PCI          Yes         No     (PVH to use pcifront?)

David

Re: PVH cleanups after 4.5

[Konrad Rzeszutek Wilk]

On Fri, Dec 05, 2014 at 10:42:27AM +0000, Andrew Cooper wrote:
> On 05/12/14 09:54, Ian Campbell wrote:
> > On Fri, 2014-12-05 at 10:49 +0100, Tim Deegan wrote:
> >> At 09:20 +0000 on 05 Dec (1417767654), Jan Beulich wrote:
> >>>>>> On 04.12.14 at 18:25, <tim@xen.org> wrote:
> >>>> Potential feature flags, based on whiteboard notes at the session.
> >>>> Things that are 'Yes' in both columns might not need actual flags :)
> >>>>
> >>>>                      'HVM'       'PVH'
> >>>> 64bit hypercalls      Yes         Yes
> >>>> 32bit hypercalls      Yes         No
> >>> Iiuc the lack of support of 32-bit hypercalls is simply because PVH
> >>> guests aren't expected to use them as being always 64-bit right
> >>> now. I.e. I can't really see why we couldn't just enable them once
> >>> the 64-bit hypercall tables got combined, in which case we wouldn't
> >>> need a feature flag here either.
> >> Agreed -- I think the same will apply to a few other things, like shadow
> >> pagetables and some of the other MM tricks.  
> > Might we want to constrain a given PVH domain to only make 32- or 64-bit
> > hypercalls?
> >
> > Or do we consider already having crossed that bridge with HVM enough
> > reason to allow it for PVH? I'm wonder if that, even if it is
> > technically possible to support not, doing so might mitigate some
> > potential security issues down the line. There's obviously a tradeoff
> > against in-guest flexibility though.
> 
> Madating a 32/64bit split serves only to cause booting issues; you need
> to know a-priori what the eventual kernel is going to be before you
> build the domain. This is an awkward issue with PV domains which
> *really* wants not to apply to PVH as well.
> 
> PVH guests with the plan of "HVM - qemu" should be able to fully choose
> their operating mode, and allow for in-guest bootstrapping which is far
> superior from a security/isolation point of view than toolstack
> bootstrapping.

Or another use-case: kexec-ing from within an 64-bit PVH guest to an
32-bit PVH or vice-versa.

> 
> ~Andrew
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel