Re: [PATCH v3 03/26] mm: introduce AS_NO_DIRECT_MAP

Re: [PATCH v3 03/26] mm: introduce AS_NO_DIRECT_MAP

[Takahiro Itazuri]

On Thu, 3 Sep 2026 07:06:02 -0700, Yosry Ahmed <yosry@kernel.org> wrote:
> > If that work is not already underway,
> > I would like to respin the AS_NO_DIRECT_MAP introduction and secretmem
> > adoption as a standalone MM series, separate from ALLOC_UNMAPPED and
> > mermap.
[...]
> Perhaps wait until I send the new version, then we can discuss if we
> want to split it based on how the feedback looks like. Does this work
> for you?

Following up on this with an update. Yosry let me know off-list that
ALLOC_UNMAPPED is on hold for now (quoted with permission):

> I wanted to give you an update. I was working on v4 of the
> ALLOC_UNMAPPED series while our team was simultaneously working on
> integrating that design without internal ASI implementation.
> Unfortunately, there were multiple complications with that
> integration, and we will be holding off on sending new versions of the
> ALLOC_UNMAPPED series until we have higher confidence that this
> approach will meet ASI's requirements (ideally in a production
> experiment).

Since the integration with Google's internal ASI implementation needs
more time, Yosry also agreed that I pick up the AS_NO_DIRECT_MAP work
for guest_memfd, separate from ALLOC_UNMAPPED. Here is how I plan to
proceed. Feedback is very welcome, especially from Sean on section 2
below.

1. MM and KVM parts together

I'm withdrawing my earlier proposal to post AS_NO_DIRECT_MAP and the
secretmem conversion as a standalone MM series first. secretmem
removes kernel access to its memory entirely, but KVM still needs to
access some guest memory from the kernel. If the MM part lands first
and the guest_memfd integration then requires a different design, we
would have to rework it right after it lands. So I'd like to settle
the design with guest_memfd as a real user, and post the MM and KVM
parts together in a single series so that both sides can review the
whole picture. Once we agree on the overall design, I'm of course
happy to split it up so that each part can go through the MM and KVM
trees.

2. Kernel access to direct-map-removed memory

I agree with Sean [1] that the way to access such memory from the
kernel, whether it ends up as an extension of GUP or a new interface,
should be generic rather than specific to guest_memfd, so that other
users of direct-map-removed pages that may come along can use it too.
It is also still open who should establish the kernel mapping: MM
could do it transparently, or callers could create one explicitly on
demand, either with existing MM mechanisms or with a new one like the
mermap. I don't have a concrete design in mind yet.

As a first step towards that, I'd like to:

  - List the use cases that stop working when GUP rejects
    AS_NO_DIRECT_MAP mappings. On the KVM side, these are mainly the
    gfn_to_pfn_cache users (kvmclock and the Xen shared pages) and
    kvm_vcpu_map() (mostly nested VMX/SVM). Accesses through uaccess,
    such as kvm_read_guest(), are not affected.

  - Look into each use case and sort out its requirements, e.g.
    whether it needs only the PFN, a short-lived kernel mapping, or a
    kernel mapping that stays valid for as long as the vCPU or VM
    uses it.

and then work out an appropriate design based on those requirements.
I'd only consider supporting the use cases incrementally where that
doesn't leave guest_memfd with a list of caveats or an ad hoc
interface.

3. ALLOC_UNMAPPED and the mermap

AS_NO_DIRECT_MAP itself works without them, by zapping the direct map
of each folio. However, guest_memfd will eventually need a temporary
kernel mapping to populate guest memory via write() [2], because
folios are already unmapped once they are added to the page cache.
The cost of TLB flushes and direct map fragmentation also matters for
performance, so I'd like to keep discussing these with folks
upstream, building on the direct map fragmentation discussion that
Kevin led at LPC [3].

Thanks,
Takahiro

[1] https://lore.kernel.org/all/anZfv81jLSU2qY9G@google.com/
[2] https://lore.kernel.org/all/20251114151828.98165-1-kalyazin@amazon.com/
[3] https://lpc.events/event/20/contributions/2420/

Re: [PATCH v3 03/26] mm: introduce AS_NO_DIRECT_MAP

[Yosry Ahmed]

On Tue, Oct 13, 2026 at 8:26 AM Takahiro Itazuri <itazur@amazon.com> wrote:
>
> On Thu, 3 Sep 2026 07:06:02 -0700, Yosry Ahmed <yosry@kernel.org> wrote:
> > > If that work is not already underway,
> > > I would like to respin the AS_NO_DIRECT_MAP introduction and secretmem
> > > adoption as a standalone MM series, separate from ALLOC_UNMAPPED and
> > > mermap.
> [...]
> > Perhaps wait until I send the new version, then we can discuss if we
> > want to split it based on how the feedback looks like. Does this work
> > for you?
>
> Following up on this with an update. Yosry let me know off-list that
> ALLOC_UNMAPPED is on hold for now (quoted with permission):
>
> > I wanted to give you an update. I was working on v4 of the
> > ALLOC_UNMAPPED series while our team was simultaneously working on
> > integrating that design without internal ASI implementation.
> > Unfortunately, there were multiple complications with that
> > integration, and we will be holding off on sending new versions of the
> > ALLOC_UNMAPPED series until we have higher confidence that this
> > approach will meet ASI's requirements (ideally in a production
> > experiment).
>
> Since the integration with Google's internal ASI implementation needs
> more time, Yosry also agreed that I pick up the AS_NO_DIRECT_MAP work
> for guest_memfd, separate from ALLOC_UNMAPPED. Here is how I plan to
> proceed. Feedback is very welcome, especially from Sean on section 2
> below.

Thanks for the update, Takahrio!

> 3. ALLOC_UNMAPPED and the mermap
>
> AS_NO_DIRECT_MAP itself works without them, by zapping the direct map
> of each folio. However, guest_memfd will eventually need a temporary
> kernel mapping to populate guest memory via write() [2], because
> folios are already unmapped once they are added to the page cache.
> The cost of TLB flushes and direct map fragmentation also matters for
> performance, so I'd like to keep discussing these with folks
> upstream, building on the direct map fragmentation discussion that
> Kevin led at LPC [3].

FWIW, I think the mermap can still be used without ALLOC_UNMAPPED, and
I did have a newer version of the x86 patches repurposing the LDT area
for the mermap, and simplifying the mermap implementation a bit. I can
share them privately if you want to take a look.

Re: [PATCH v3 03/26] mm: introduce AS_NO_DIRECT_MAP

[Takahiro Itazuri]

On Tue, 13 Oct 2026 08:36:54 -0700, Yosry Ahmed <yosry@kernel.org> wrote:
> > 3. ALLOC_UNMAPPED and the mermap
> >
> > AS_NO_DIRECT_MAP itself works without them, by zapping the direct map
> > of each folio. However, guest_memfd will eventually need a temporary
> > kernel mapping to populate guest memory via write() [2], because
> > folios are already unmapped once they are added to the page cache.
[...]
>
> FWIW, I think the mermap can still be used without ALLOC_UNMAPPED, and
> I did have a newer version of the x86 patches repurposing the LDT area
> for the mermap, and simplifying the mermap implementation a bit. I can
> share them privately if you want to take a look.

Yes please, that would be very helpful. Thanks!

Agreed that the mermap can be used without ALLOC_UNMAPPED. To give a
bit more context from the guest_memfd side: I can't say anything
definite until I've gone through the use cases, but as far as I can
see now, guest_memfd needs the mermap only for write() support.
write() is an optimization for populating guest memory (e.g. when
restoring a snapshot) and isn't functionally required, because
userspace can also populate guest memory through its own mapping of
guest_memfd. So I currently plan to keep write() support, and with it
the mermap, out of the guest_memfd direct map removal series itself.
Since ALLOC_UNMAPPED is the main user of the mermap, it may make more
sense for the two to land together later.

One thing I'd like to look into is arm64. Firecracker also runs on
arm64, so write() needs to work there too, but the mermap currently
relies on the x86 mm-local region, and it sounds like your newer
version relies on the LDT area. Kevin, do you happen to have any
ideas on how something similar could be done on arm64?

Thanks,
Takahiro

Re: [PATCH v3 03/26] mm: introduce AS_NO_DIRECT_MAP

[Yosry Ahmed]

On Tue, Oct 13, 2026 at 9:06 AM Takahiro Itazuri <itazur@amazon.com> wrote:
>
> On Tue, 13 Oct 2026 08:36:54 -0700, Yosry Ahmed <yosry@kernel.org> wrote:
> > > 3. ALLOC_UNMAPPED and the mermap
> > >
> > > AS_NO_DIRECT_MAP itself works without them, by zapping the direct map
> > > of each folio. However, guest_memfd will eventually need a temporary
> > > kernel mapping to populate guest memory via write() [2], because
> > > folios are already unmapped once they are added to the page cache.
> [...]
> >
> > FWIW, I think the mermap can still be used without ALLOC_UNMAPPED, and
> > I did have a newer version of the x86 patches repurposing the LDT area
> > for the mermap, and simplifying the mermap implementation a bit. I can
> > share them privately if you want to take a look.
>
> Yes please, that would be very helpful. Thanks!
>
> Agreed that the mermap can be used without ALLOC_UNMAPPED. To give a
> bit more context from the guest_memfd side: I can't say anything
> definite until I've gone through the use cases, but as far as I can
> see now, guest_memfd needs the mermap only for write() support.
> write() is an optimization for populating guest memory (e.g. when
> restoring a snapshot) and isn't functionally required, because
> userspace can also populate guest memory through its own mapping of
> guest_memfd. So I currently plan to keep write() support, and with it
> the mermap, out of the guest_memfd direct map removal series itself.
> Since ALLOC_UNMAPPED is the main user of the mermap, it may make more
> sense for the two to land together later.
>
> One thing I'd like to look into is arm64. Firecracker also runs on
> arm64, so write() needs to work there too, but the mermap currently
> relies on the x86 mm-local region, and it sounds like your newer
> version relies on the LDT area. Kevin, do you happen to have any
> ideas on how something similar could be done on arm64?

Small correction: Brendan's version already reworked the LDT area, I
just reshuffled things and simplified the implementation.

>
> Thanks,
> Takahiro