Re: Xen Project policy on feature flags

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Fri, Sep 26, 2014 at 03:54:40PM +0100, Stefano Stabellini wrote:
> On Fri, 26 Sep 2014, Konrad Rzeszutek Wilk wrote:
> > On Fri, Sep 26, 2014 at 03:26:33PM +0100, David Vrabel wrote:
> > > On 26/09/14 14:24, Stefano Stabellini wrote:
> > > > Hello all,
> > > > I am writing to request a clarification on Xen feature flags
> > > > (XENFEAT_*) and backward compatibility:
> > > >     
> > > > is the hypervisor allowed to remove any feature flags in a future
> > > > release, even though doing so might break some existing guests?
> > > > 
> > > > For example one could write a PV on HVM guest that requires
> > > > XENFEAT_hvm_callback_vector (regardless of PVH), could a future Xen
> > > > release remove that feature? Or is it now part of our ABI, therefore
> > > > maintained for backward compatibility, following the rule that we don't
> > > > break existing guests?
> > > > 
> > > > 
> > > > I always thought that any XENFEAT feature flags could be removed going
> > > > forward, if the hypervisor maintainers decide to do so. However Ian
> > > > Campbell is of the opposite opinion, so I think we should have a clear
> > > > policy regarding them.
> > > 
> > > A guest that runs on Xen version V /must/ continue to run on V+1.
> > 
> > Not if the feature is 'experimental' (I don't know offhand if this
> > is experimental or not).
> > 
> > > 
> > > The is similar to the policy the Linux kernel has for the user space ABI.
> > 
> > Sure, but there are also ioctls and such which can change between releases
> > such that application for V+1 will _not_ work with V. Look at 'perf'
> > as an example.
> > 
> > > 
> > > This does permit support for features to be removed but only if no guest
> > > would be broken by its removal.  But since it it is not possible to know
> > > what guests people have running and what features they require, I can't
> > > see how any feature could be safely removed.
> > 
> > The guest is already broken even by taking advantage of this feature.
> 
> The guest is capable of recognizing whether the feature is available or
> not and doesn't try to use the feature if it is not advertized by the
> XENFEAT flag.
> 
> Unfortunately the fall back at the moment is breaking up farther along
> the way.

HA! To summarize:

 - With this feature available we break.
 - Without this feature available we break.

In either way removing this feature will mean we are still broken - it is
just a different shade of it.

> 
>  
> > > > In any case I think that it is generally useful to have optional flags
> > > > that advertise the presence of a feature but can also be removed going
> > > > forward. If XENFEAT feature flags are not them, then we might still want
> > > > to introduce them as a separate concept.
> > > 
> > > I don't think "optional" feature flags are any different.  You can
> > > specify that guests must handle the feature being missing but that's no
> > > guarantee that guest will actually implement the fallback.
> > 
> > That would be a odd - as it would mean guests were written
> > to work with V+1 but not with V-1.
> > 
> > Granted that is the case for booting Linux as dom0. It can only work
> > with Xen 4.1 and later. But you could argue that the fallback for booting
> > with V-1 (Xen 4.0) is to not boot at all as it did not have the
> > minimum required features.
> > 
> > So was this feature an required feature or an enhancement?
> 
> >From a practical perspective it is a required feature for Linux.

Hehe. We have a required feature that we want to be broken, eh :-)

Can the CONFIG_PCI_DMA_ARE_64_BIT (or whatever it is) be part of an
non-LPAE build? That would have fixed the issue too right?

> 
> >From a theoretical and ABI standpoint, one could write a Dom0 kernel
> that doesn't need this feature to work properly (and in fact you can
> still have it by reverting Zoltan's changes or going back to 3.15).

Oh boy. Have we dug ourselves in a hole.