Re: [RFC PATCH 00/12] dax: Add DAX to guest memfd support for KVM

[Ira Weiny <iweiny@fastmail.com>]

Gregory Price wrote:
> On Thu, Apr 23, 2026 at 10:02:07AM -0700, Dave Jiang wrote:
> > This RFC series is created as a proof of concept to connect device DAX to guest
> > memory by riding on top of guest memfd in order to prove out that device DAX
> > can be used as guest memory. The series seeks to jump start a discussion on
> > if there are interests in creating a DAX bridge to utilize CXL memory for guest
> > memory until the N_PRIVATE implementation by Gregory [1] is available upstream
> > and DAX users are ready to move to the new scheme. Once there's an established
> > consensus of interest, we can move the discussion to the best way to implement
> > the DAX bridge and the future of device DAX as guest.
> >
> > I did the bare minimal to get the PoC to pass a modified version of KVM gmem
> > selftest (guest_memfd_test) in order to prove out that DAX can go in the gmem
> > path. A DAX char dev is created and the fd is passed in user space with
> > vm_set_user_memory_region2(). The DAX region is passed in as a whole when used
> > unlike memfd where any size can be passed in to be allocated.
> > 
> > The folks on the cc line are people that Dan Williams has mentioned that may be
> > of interest to this.
> 
> I see these as *mildly* orthogonal, but I think maybe you should propose
> a discussion at LSF to talk about this.

Sorry I was a bit delayed on this thread due to some email issues.

Yes this should be talked about at LSF if possible.  But I also think this
is something which is a ways off based on the responses we have seen here.

> 
> guest_memfd in particular wants the host to never map the memory - and
> guests *generally* want 1GB huge page support (TLB go brrrrr).

That is _not_ going to be true forever.  There is work ongoing to create
shared gmem for various reasons.  For secure guests this is at least
useful for initial population of memory before handing to a guest.

> 
> There's a real argument for just handing a physical memory region over
> to guest_memfd and making it manage the region manually, rather than
> doing a bunch of nonsense just so you can call alloc_pages_node()

Agreed.

> 
> So I see an extension like this as genuinely useful regardless of
> whether private nodes actually end up merged.  It's a matter of
> flexibility and use cases.

Yep, the initial talks we had with Dan were to try and get DAX FDs to be
more mainstream.  Given some of the other work it may be better to
deprecate DAX FDs.  But deprecations can take a long time so what Dave
came up with here is trying to help modernize those fds to be more useful
for guest computing.

Also depending on existing use cases this may be easier for folks to
adopt?  But it may have more rough edges than it is worth?

> 
> With this plumbing, you get less flexible use of the memory (you're tied
> to dax abstractions), whereas with private nodes you can build slightly
> more flexible general-system support.
> 
> IN THEORY you could add something like an NP_OP_NOMAP to private nodes
> to make the buddy manage pages that don't have a direct map - BUT - in
> practice that's likely to be more of a bodge rather than a good design.
> 
> So I will say - to the detriment of private nodes ;] - I like this idea.

I've investigated using private nodes as a mechanism for guest_memfd to
draw from.  I think this is along the lines of what Frank mentioned
elsewhere in this thread.

> 
> The question is ultimately how much flexibility you need to shuffle this
> capacity from one guest to another.

Yep.  And how much control one needs over which exact CXL/DAX devices the
memory comes from.  As you know from our community calls that is one thing
I'm not sure the private node idea is great at.  But it could be that is
not really required.  Or is best handled as a carve out.

Ira