Re: RFC: A KVM-specific alternative to UserfaultFD

[Oliver Upton <oliver.upton@linux.dev>]

On Tue, Nov 07, 2023 at 12:04:21PM -0800, David Matlack wrote:
> On Tue, Nov 7, 2023 at 8:25 AM Paolo Bonzini <pbonzini@redhat.com> wrote:

[...]

> >  My
> > gut feeling even without reading everything was (and it was confirmed
> > after): I am open to merging some specific features that close holes in
> > the userfaultfd API, but in general I like the unification between
> > guest, userspace *and kernel* accesses that userfaultfd brings. The fact
> > that it includes VGIC on Arm is a cherry on top. :)
> 
> Can you explain how VGIC interacts with UFFD? I'd like to understand
> if/how that could work with a KVM-specific solution.

The VGIC implementation is completely unaware of the existence of UFFD,
which is rather elegant.

There is no ioctl that allows userspace to directly get/set the VGIC
state. Instead, when userspace wants to migrate a VM it needs to flush
the cached state out of KVM's representation into guest memory. I would
expect the VMM to do this right before collecting the final dirty
bitmap.

If UFFD is off the table then it would appear there are two options:

 - Instrument these ioctls to request pages not marked as present in the
   theorized KVM-owned demand paging interface

 - Mandate that userspace has transferred all of the required VGIC / ITS
   pages before resuming on the target

The former increases the maintenance burden of supporting post-copy
upstream and the latter *will* fail spectacularly. Ideally we use a
mechanism that doesn't require us to think about instrumenting
post-copy for every new widget that we will want to virtualize.

> So in the short term we could provide a partial solution for
> HugeTLB-backed VMs (at least unblocking Google's use-case) and in the
> long-term there's line of sight of a unified solution.

Who do we expect to look after the upstreamed short-term solution once
Google has moved on to something else?

-- 
Thanks,
Oliver