Re: [RFC PATCH 0/8] dax/kmem: support for private NUMA nodes

[Gregory Price <gourry@gourry.net>]

On Fri, Jul 24, 2026 at 03:45:23PM +0800, Richard Cheng wrote:
> On Wed, Jul 22, 2026 at 02:35:11PM +0800, Gregory Price wrote:
> 
> Hi Gregory,
> 
> I have some thoughts especially for ownership and configs.
> 
> I think it would be good to expose dax control of private node, just with
> the following guarantees.
> 
> Let's make it clear that userspace should never become the owner
> of private node, they should be driver-owned and 1-to-1 mapping relationship.
> 

The intent here is 100% 1-to-1 driver:node mappings.

Technically there's nothing unreasonable about single-owner multi-node, but
really we just don't want multiple devices on one node.

> A userspace tool can request to reconfigure some capability of the DAX device, and
> maybe how the driver expose it.
> 
> A dax range may have several deployment rules, e.g. devdax, system-ram or private, with
> the patch series' tool, it looks like a userspace tool can convert a DAX device between
> dax and system-ram mode, I think that's unsafe especially when the region is online as
> system-ram.
> 
> And it will cause coherence issue to be resolved, if no special or strong use cases, I would
> prevent having such ability to convert the rules in runtime, especially for fixed-purpose device.
> 

Right now we can't really prevent a user from doing:

echo dax0.0 > dax/drivers/devdax/unbind
echo dax0.0 > dax/drivers/kmem/bind

And in fact we depend on kmem being the default target for CXL to
auto-probe through to hotplug.  For CXL accelerators, the current
intended auto-probe behavior is to terminate at an offline kmem dax
binding, and then expect the user to either rebind the dax device, or
rebind the entire cxl device:

echo $DEVICE > ../drivers/cxl_pci/unbind
echo $DEVICE > ../pci/my_cxl_device_driver/bind

I have some old patches that augment how drivers select the dax mode,
I can see about reviving that and maybe letting the source driver
also dictate which dax modes are valid according to the source.

I'll look into that.

> For capability bitmap part, I think it's reasonable to expose it, but with 2 concepts
> - supported_caps : immutable, supplied by the provider/vendor
> - enabled-caps : selected policy, subset of supported_caps
> 
> The invariant should always hold
> """
> enabled_caps & ~supported_caps == 0
> """
> 
> This way we can prevent userspace from touching things that they shouldn't.
> 

This I like, and i think you've convinced me that this should be a
separate dax device driver so we're not overloading kmem.

That said, for a dax device that was not created by a driver, but
instead created via something like "memmap=" boot parameter, i'm we
can't really dictate supported caps - but for a driver that *does* want
to limit support, I think we can manage that.

> Maybe a demo with a driver would be nice ?
> 

Sure! I think i'm going to include my anon-memory only compressed ram
service back into v6 w/ the dax driver extensions you've suggested.

~Gregory