Addition of a staging subdirectory to virt/ for in-development features

Addition of a staging subdirectory to virt/ for in-development features

[Amy Parker]

Hello all.

Has the idea of having a staging subdirectory for KVM been proposed
before? It could become a buffer place for KVM features currently in
development/not production ready, but that should be able to be
conditionally included into the kernel. With a staging directory,
these features would be divorced from KVM mainline, but would be able
to be rolled out promptly to users while they are being refined.

Any thoughts on this?

    - amyip

Re: Addition of a staging subdirectory to virt/ for in-development features

[Sean Christopherson]

On Wed, Oct 20, 2021, Amy Parker wrote:
> Hello all.
> 
> Has the idea of having a staging subdirectory for KVM been proposed before?

Not that I know of.

> It could become a buffer place for KVM features currently in development/not
> production ready, but that should be able to be conditionally included into
> the kernel. With a staging directory, these features would be divorced from
> KVM mainline, but would be able to be rolled out promptly to users while they
> are being refined.

Can you give an example?

I'm struggling to envision a feature that is both large enough to warrant
"staging", yet isolated enough to actually be "divorced from KVM mainline".
One of the rules for staging/drivers is that the drivers are standalone and don't
require changes to the kernel proper outside of docs, firmware, and perhaps
exports.

Re: Addition of a staging subdirectory to virt/ for in-development features

[Amy Parker]

On Wed, Oct 20, 2021 at 4:11 PM Sean Christopherson <seanjc@google.com> wrote:
>
> On Wed, Oct 20, 2021, Amy Parker wrote:
> > Hello all.
> >
> > Has the idea of having a staging subdirectory for KVM been proposed before?
>
> Not that I know of.

Alright, thank you for clarification.

>
> > It could become a buffer place for KVM features currently in development/not
> > production ready, but that should be able to be conditionally included into
> > the kernel. With a staging directory, these features would be divorced from
> > KVM mainline, but would be able to be rolled out promptly to users while they
> > are being refined.
>
> Can you give an example?
>
> I'm struggling to envision a feature that is both large enough to warrant
> "staging", yet isolated enough to actually be "divorced from KVM mainline".
> One of the rules for staging/drivers is that the drivers are standalone and don't
> require changes to the kernel proper outside of docs, firmware, and perhaps
> exports.

I can think of three categories of implementations:

1. Supporting new hardware virtualization technologies, which may take
a while to develop but should still be available to users for testing
and non-production purposes while being refined, such as on new
architectures or by new manufacturers. It's entirely possible that
there will be new competitors in the x86_64 space soon, especially
with the increasing popularity of the concept of 128-bit computing
architectures.
2. Experimental algorithms which optimize current functions; these may
not be immediately production-ready, but should be available to the
end user in something like the staging folder.
3. Other future virtualization technologies, should they ever be
added, such as cross-binary architecture and syscall compatibility.

    - amyip

Re: Addition of a staging subdirectory to virt/ for in-development features

[Paolo Bonzini]

Hi Amy,

I don't think any of these justify a different approach than what we are 
doing already.

On 21/10/21 04:24, Amy Parker wrote:
> 1. Supporting new hardware virtualization technologies, which may take
> a while to develop but should still be available to users for testing
> and non-production purposes while being refined, such as on new
> architectures or by new manufacturers. It's entirely possible that
> there will be new competitors in the x86_64 space soon, especially
> with the increasing popularity of the concept of 128-bit computing
> architectures.

The issue here would be, first and foremost, whether the architectures 
would be supported by Linux.  As KVM RISC-V is going to be included in 
Linux 5.16, I'm not sure what other architectures would be supported by 
Linux but only have experimental status in KVM.

In particular, the recent disagreements about including KVM for RISC-V 
in some kind of "staging" code arose only after the architectural 
specification had reached considerable stability, and so had the KVM code.

> 2. Experimental algorithms which optimize current functions; these may > not be immediately production-ready, but should be available to the
> end user in something like the staging folder.

These are also not suitable for a separate staging subtree.  Almost 
always such optimizations require changes to existing code, in such a 
way that the existing code needs to be aware of what the more 
experimental code is doing.

It is okay to include experimental algorithms and optimizations before 
they have reached feature parity with what they're supposed to replace. 
  See for example the "TDP MMU" code that has been contributed over the 
past year and has only reached feature parity in 5.15.

> 3. Other future virtualization technologies, should they ever be
> added, such as cross-binary architecture and syscall compatibility.

This is the same as the previous point: we have some code in Linux that 
is not used widely (KVM for s390 has some microcode testing 
functionality that is only used internally at IBM, and the Xen emulation 
layer is only used by Amazon as far as I know), but it's okay to include 
it if it's maintainable, reasonably mature, and covered by test cases. 
However, contributions to KVM should follow the same quality standards 
as the rest of Linux.

Thanks,

Paolo