Re: [RFC PATCH 0/3] KVM: Dirty page logging for guest_memfd-only memslots

[Alexandru Elisei <alexandru.elisei@arm.com>]

Hi Oliver,

On Thu, Jul 09, 2026 at 01:33:30PM -0700, Oliver Upton wrote:
> On Tue, Jul 07, 2026 at 05:58:49PM +0100, Alexandru Elisei wrote:
> > Hi Sean,
> > 
> > On Mon, Jul 06, 2026 at 05:56:12PM -0700, Sean Christopherson wrote:
> > > On Thu, Jul 02, 2026, Alexandru Elisei wrote:
> > > > The memory represented by guest_memfd-only memslots
> > > > (kvm_memslot_is_gmem_only() is true) is shared with userspace, which can
> > > > freely mmap it and access it. The only thing that is preventing dirty page
> > > > logging for such memslots is that KVM doesn't allow slots backed by
> > > > guest_memfd to have their flags changed; they can only be created and
> > > > deleted.
> > > 
> > > Please (publicly) document *why*  you want to add dirty-logging support.  It's
> > > all but impossible to review new uAPI without knowing the use case.
> > 
> > Of course, my mistake, I was so deep in this that I didn't realise that
> > there might be different perspectives.
> > 
> > My thinking was that since guest_memfd created with GUEST_MEMFD_FLAG_MMAP +
> > GUEST_MEMFD_FLAG_INIT_SHARED is extremely similar from a userspace point of
> > view to using an anonymous file (created with memfd_create()), that
> > supporting dirty page logging and migration would be a natural next step
> > and would expand the usefulness of guest_memfd. It has nothing to do with
> > confidential compute.
> > 
> > As to why I'm working on it now, it's because of an arm64 feature that
> > requires that memory remains mapped at stage 2, called Statistical
> > Profiling Extension (SPE), similar to Intel's PEBS or AMD's IBS. Exposing
> > the feature to a guest requires that memory remains mapped at stage 2
> > outside of userspace explicitely unmapping it, and guest_memfd, with the
> > patch to ignore the MMU notifiers [1], has this property.  I wanted to
> > expand the functionality of guest_memfd to support migration of virtual
> > machines when that arm64 feature is exposed to guests.
> 
> Can you please expand a bit on how you actually expect dirty tracking to
> work with SPE? Taking write permission faults seem to be at odds with
> "thou shalt not fault".

For the initial posting on the mailing list, my approach was to keep things
as simple as possible: allow memory to be made read-only and then have the
host map the memory as writable at stage 2 when SPE reports a permission
fault, if the buffer happens to be enabled while dirty page logging is also
enabled. Of course, that means that the quality of the profiling data might
be degraded. I was planning to point this out, to start a discussion about
it, but we can talk about it now.

Another approach that builds on that, which I have prototyped locally, is
to have a VCPU exit to userspace when/if the buffer is enabled and there is
at least one memslot with dirty page logging enabled (with a specific
hardware_exit_reason) and then let userspace decide what to do: resume the
VCPU and live with the blackout window, or keep the VCPU stopped, and the
guest gets non-degraded profiling data as the expense of the VCPU making
progress. My hope is that if migration is fast enough, the VCPU not running
for (possibly) the entire duration of the migration process won't be
noticeable, but I might be naive in thinking that.

> 
> Seems to me like you'd need hardware dirty state and the "noabort"
> flavor of BBML2 to actually get things down to page mappings. Oh, and an
> SPE implementation that actually respects write permissions... Is there
> even an implementation out there where the stars align?

According to the Neoverse v3 errata document (MP168), r0p0 is affected,
r0p1 is not affected. A quick google search revealed that Microsoft's
Cobalt 200 CPU is based on Neoverse v3, same for the Arm AGI CPU. Wikipedia
also says that Graviton 5 is based on Neoverse v3. Don't know the exact
revision though, for either of them.

I was planning to add a kernel parameter that permits SPE on the affected
parts, so people who trust their guests* can still use it from a VM. I
don't think migration would be possible in this case because SPE bypasses
the read-only permission at stage 2, and the host would not know to mark
that memory as dirty (for dirty page logging).

* Ignoring migration, guest_memfd is always mapped as writable at stage 2,
so a guest would have to point the buffer at memory that KVM represents
with a KVM_MEM_READONLY memslot, like the CFI flash device used by EDK2.

> 
> Fixing rotten architecture would be a more appropriate starting point :-/

I like to think that the architecture is constantly improving :)

Thanks,
Alex