Re: qemu cxl memory expander shows numa_node -1

[Jonathan Cameron <Jonathan.Cameron@Huawei.com>]

On Fri, 26 Jan 2024 10:43:43 -0500
Gregory Price <gregory.price@memverge.com> wrote:

> On Fri, Jan 26, 2024 at 12:39:26PM +0000, Jonathan Cameron wrote:
> > On Thu, 25 Jan 2024 13:45:09 +0530
> > Sajjan Rao <sajjanr@gmail.com> wrote:
> >   
> > > Looks like something changed in QEMU 8.2 that broke running code out
> > > of CXL memory with KVM disabled.
> > > I used "numactl --membind 2 ls" as suggested by Dimitrios earlier,
> > > this worked for me until I updated to the latest QEMU.
> > > 
> > > Is this a known issue? Or am I missing something?  
> > 
> > I'm confused on how the description below ever worked.
> > Assigning the underlying memdev=cxl-mem1 to a numa node isn't going
> > to correctly build the connections the CFMWS PA range.
> >   
> 
> I've now seen 3-4 occasions where people have done this and run into
> trouble (for obvious reasons).  Is there anything we can do to disallow
> the double-registering of a single memdev to both a numa node and a cxl
> device?
> 
It would be novel for us to prevent people shooting themselves
in the foot ;) but I guess this should be fairly easy as the
numa node logic prevents the same one being used multiple times so can
copy how that is done.

This should do the trick (very lightly tested).
It's end of day Friday here so a formal patch can wait for next week.


diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index f29346fae7..d4194bb757 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -827,6 +827,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
             error_setg(errp, "volatile memdev must have backing device");
             return false;
         }
+        if (host_memory_backend_is_mapped(ct3d->hostvmem)) {
+            error_setg(errp, "memory backend %s can't be used multiple times.",
+               object_get_canonical_path_component(OBJECT(ct3d->hostvmem)));
+            return false;
+        }
         memory_region_set_nonvolatile(vmr, false);
         memory_region_set_enabled(vmr, true);
         host_memory_backend_set_mapped(ct3d->hostvmem, true);
@@ -850,6 +855,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
             error_setg(errp, "persistent memdev must have backing device");
             return false;
         }
+        if (host_memory_backend_is_mapped(ct3d->hostpmem)) {
+            error_setg(errp, "memory backend %s can't be used multiple times.",
+               object_get_canonical_path_component(OBJECT(ct3d->hostpmem)));
+            return false;
+        }
         memory_region_set_nonvolatile(pmr, true);
         memory_region_set_enabled(pmr, true);
         host_memory_backend_set_mapped(ct3d->hostpmem, true);
@@ -880,6 +890,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
             error_setg(errp, "dynamic capacity must have backing device");
             return false;
         }
+        if (host_memory_backend_is_mapped(ct3d->dc.host_dc)) {
+            error_setg(errp, "memory backend %s can't be used multiple times.",
+               object_get_canonical_path_component(OBJECT(ct3d->dc.host_dc)));
+            return false;
+        }
         /* FIXME: set dc as nonvolatile for now */
         memory_region_set_nonvolatile(dc_mr, true);
         memory_region_set_enabled(dc_mr, true);





> ~Gregory