[CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

[CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

[Samarth Saxena]

[-- Attachment #1: Type: text/plain, Size: 1468 bytes --]

Hi All,

I am trying the following command to run Qemu with Kernel 5.14.

qemu-system-x86_64 -M q35,accel=kvm,nvdimm=on,cxl=on,hmat=on -m 8448M,slots=2,maxmem=16G -smp 8,sockets=2,cores=2,threads=2 -hda /lan/dscratch/singhabh/shradha/ubuntu-20.10-64_with_orig_driver_backup.qcow2 -enable-kvm -usbdevice tablet -L $VB_ROOT/etc/vm/common/ -object memory-backend-ram,id=cxl-ram,share=on,size=256M -device "pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-base[0]=0x4c0000000,memdev[0]=cxl-ram" -device cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0 -device cxl-type3,bus=rp0,memdev=cxl-ram,id=cxl-vmem0,size=256M -numa node,nodeid=0,memdev=cxl-ram -object memory-backend-ram,id=other-ram,size=8G,prealloc=n,share=off -numa node,nodeid=1,memdev=other-ram,initiator=0 -numa cpu,node-id=0,socket-id=0 -numa cpu,node-id=0,socket-id=1

I get the following crash before the machine starts to boot.

qemu-system-x86_64: ../softmmu/memory.c:2443: memory_region_add_subregion_common: Assertion `!subregion->container' failed.


Please help me find what's wrong here.

Regards,
[CadenceLogoRed185Regcopy1583174817new51584636989.png]<https://www.cadence.com/en_US/home.html>
Samarth Saxena
Sr Principal Software Engineer
T: +911204308300
[UIcorrectsize1583179003.png]<https://www.cadence.com/en_US/home.html>
[16066EmailSignatureFortune100Best2021White92x1271617625037.png]<https://www.cadence.com/en_US/home/company/careers.html>





[-- Attachment #2: Type: text/html, Size: 5900 bytes --]

Re: [CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

[Dr. David Alan Gilbert]

* Samarth Saxena (samarths@cadence.com) wrote:
> Hi All,
> 
> I am trying the following command to run Qemu with Kernel 5.14.

cc'ing in Ben who I think owns the CXL stuff.

> qemu-system-x86_64 -M q35,accel=kvm,nvdimm=on,cxl=on,hmat=on -m 8448M,slots=2,maxmem=16G -smp 8,sockets=2,cores=2,threads=2 -hda /lan/dscratch/singhabh/shradha/ubuntu-20.10-64_with_orig_driver_backup.qcow2 -enable-kvm -usbdevice tablet -L $VB_ROOT/etc/vm/common/ -object memory-backend-ram,id=cxl-ram,share=on,size=256M -device "pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-base[0]=0x4c0000000,memdev[0]=cxl-ram" -device cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0 -device cxl-type3,bus=rp0,memdev=cxl-ram,id=cxl-vmem0,size=256M -numa node,nodeid=0,memdev=cxl-ram -object memory-backend-ram,id=other-ram,size=8G,prealloc=n,share=off -numa node,nodeid=1,memdev=other-ram,initiator=0 -numa cpu,node-id=0,socket-id=0 -numa cpu,node-id=0,socket-id=1

You probably need to state which qemu tree and version you're using
here.

> I get the following crash before the machine starts to boot.
> 
> qemu-system-x86_64: ../softmmu/memory.c:2443: memory_region_add_subregion_common: Assertion `!subregion->container' failed.

It's probably best to check with Ben, but you'll want a backtrace and
figure out which subregion and region you're dealing with.

Dave

> 
> Please help me find what's wrong here.
> 
> Regards,
> [CadenceLogoRed185Regcopy1583174817new51584636989.png]<https://www.cadence.com/en_US/home.html>
> Samarth Saxena
> Sr Principal Software Engineer
> T: +911204308300
> [UIcorrectsize1583179003.png]<https://www.cadence.com/en_US/home.html>
> [16066EmailSignatureFortune100Best2021White92x1271617625037.png]<https://www.cadence.com/en_US/home/company/careers.html>
> 
> 
> 
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

RE: [CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

[Samarth Saxena]

Thanks Dave,

The Qemu version is qemu-6.0.50.

I am trying to capture the stack and will place it ASAP.

Regards,
Samarth


-----Original Message-----
From: Dr. David Alan Gilbert <dgilbert@redhat.com> 
Sent: Tuesday, August 10, 2021 4:58 PM
To: Samarth Saxena <samarths@cadence.com>; ben.widawsky@intel.com
Cc: qemu-devel@nongnu.org
Subject: Re: [CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

EXTERNAL MAIL


* Samarth Saxena (samarths@cadence.com) wrote:
> Hi All,
> 
> I am trying the following command to run Qemu with Kernel 5.14.

cc'ing in Ben who I think owns the CXL stuff.

> qemu-system-x86_64 -M q35,accel=kvm,nvdimm=on,cxl=on,hmat=on -m 
> 8448M,slots=2,maxmem=16G -smp 8,sockets=2,cores=2,threads=2 -hda 
> /lan/dscratch/singhabh/shradha/ubuntu-20.10-64_with_orig_driver_backup
> .qcow2 -enable-kvm -usbdevice tablet -L $VB_ROOT/etc/vm/common/ 
> -object memory-backend-ram,id=cxl-ram,share=on,size=256M -device 
> "pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-
> base[0]=0x4c0000000,memdev[0]=cxl-ram" -device 
> cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0 -device 
> cxl-type3,bus=rp0,memdev=cxl-ram,id=cxl-vmem0,size=256M -numa 
> node,nodeid=0,memdev=cxl-ram -object 
> memory-backend-ram,id=other-ram,size=8G,prealloc=n,share=off -numa 
> node,nodeid=1,memdev=other-ram,initiator=0 -numa 
> cpu,node-id=0,socket-id=0 -numa cpu,node-id=0,socket-id=1

You probably need to state which qemu tree and version you're using here.

> I get the following crash before the machine starts to boot.
> 
> qemu-system-x86_64: ../softmmu/memory.c:2443: memory_region_add_subregion_common: Assertion `!subregion->container' failed.

It's probably best to check with Ben, but you'll want a backtrace and figure out which subregion and region you're dealing with.

Dave

> 
> Please help me find what's wrong here.
> 
> Regards,
> [CadenceLogoRed185Regcopy1583174817new51584636989.png]<https://www.cad
> ence.com/en_US/home.html>
> Samarth Saxena
> Sr Principal Software Engineer
> T: +911204308300
> [UIcorrectsize1583179003.png]<https://www.cadence.com/en_US/home.html>
> [16066EmailSignatureFortune100Best2021White92x1271617625037.png]<https
> ://www.cadence.com/en_US/home/company/careers.html>
> 
> 
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

[Ben Widawsky]

Thanks Dave.

Samarth,

Easiest is to just use our run_qemu and figure out the diffs (--cmdline will
print the qemu commandline):
https://github.com/pmem/run_qemu

If you're not able to figure it out after that, please let me know.

On 21-08-10 17:38:16, Samarth Saxena wrote:
> Thanks Dave,
> 
> The Qemu version is qemu-6.0.50.
> 
> I am trying to capture the stack and will place it ASAP.
> 
> Regards,
> Samarth
> 
> 
> -----Original Message-----
> From: Dr. David Alan Gilbert <dgilbert@redhat.com> 
> Sent: Tuesday, August 10, 2021 4:58 PM
> To: Samarth Saxena <samarths@cadence.com>; ben.widawsky@intel.com
> Cc: qemu-devel@nongnu.org
> Subject: Re: [CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion
> 
> EXTERNAL MAIL
> 
> 
> * Samarth Saxena (samarths@cadence.com) wrote:
> > Hi All,
> > 
> > I am trying the following command to run Qemu with Kernel 5.14.
> 
> cc'ing in Ben who I think owns the CXL stuff.
> 
> > qemu-system-x86_64 -M q35,accel=kvm,nvdimm=on,cxl=on,hmat=on -m 
> > 8448M,slots=2,maxmem=16G -smp 8,sockets=2,cores=2,threads=2 -hda 
> > /lan/dscratch/singhabh/shradha/ubuntu-20.10-64_with_orig_driver_backup
> > .qcow2 -enable-kvm -usbdevice tablet -L $VB_ROOT/etc/vm/common/ 
> > -object memory-backend-ram,id=cxl-ram,share=on,size=256M -device 
> > "pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-
> > base[0]=0x4c0000000,memdev[0]=cxl-ram" -device 
> > cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0 -device 
> > cxl-type3,bus=rp0,memdev=cxl-ram,id=cxl-vmem0,size=256M -numa 
> > node,nodeid=0,memdev=cxl-ram -object 
> > memory-backend-ram,id=other-ram,size=8G,prealloc=n,share=off -numa 
> > node,nodeid=1,memdev=other-ram,initiator=0 -numa 
> > cpu,node-id=0,socket-id=0 -numa cpu,node-id=0,socket-id=1
> 
> You probably need to state which qemu tree and version you're using here.
> 
> > I get the following crash before the machine starts to boot.
> > 
> > qemu-system-x86_64: ../softmmu/memory.c:2443: memory_region_add_subregion_common: Assertion `!subregion->container' failed.
> 
> It's probably best to check with Ben, but you'll want a backtrace and figure out which subregion and region you're dealing with.
> 
> Dave
> 
> > 
> > Please help me find what's wrong here.
> > 
> > Regards,
> > [CadenceLogoRed185Regcopy1583174817new51584636989.png]<https://www.cad
> > ence.com/en_US/home.html>
> > Samarth Saxena
> > Sr Principal Software Engineer
> > T: +911204308300
> > [UIcorrectsize1583179003.png]<https://www.cadence.com/en_US/home.html>
> > [16066EmailSignatureFortune100Best2021White92x1271617625037.png]<https
> > ://www.cadence.com/en_US/home/company/careers.html>
> > 
> > 
> > 
> > 
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> 

RE: [CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

[Samarth Saxena]

Thanks Ben !

Does this command support volatile memory too?
We are looking to run Volatile memory.

Regards,
Samarth

-----Original Message-----
From: Ben Widawsky <ben.widawsky@intel.com> 
Sent: Tuesday, August 10, 2021 11:43 PM
To: Samarth Saxena <samarths@cadence.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>; qemu-devel@nongnu.org
Subject: Re: [CXL volatile MEM] - Qemu command to turn on HMAT and NUMA fails with assertion

EXTERNAL MAIL


Thanks Dave.

Samarth,

Easiest is to just use our run_qemu and figure out the diffs (--cmdline will print the qemu commandline):
https://urldefense.com/v3/__https://github.com/pmem/run_qemu__;!!EHscmS1ygiU1lA!Unz0pDVxMVCy_xrhyJ2NNrJg3z-bb2dughtewVIoD5hKzlg0zzNMUj0s7N8OWLc$ 

If you're not able to figure it out after that, please let me know.

On 21-08-10 17:38:16, Samarth Saxena wrote:
> Thanks Dave,
> 
> The Qemu version is qemu-6.0.50.
> 
> I am trying to capture the stack and will place it ASAP.
> 
> Regards,
> Samarth
> 
> 
> -----Original Message-----
> From: Dr. David Alan Gilbert <dgilbert@redhat.com>
> Sent: Tuesday, August 10, 2021 4:58 PM
> To: Samarth Saxena <samarths@cadence.com>; ben.widawsky@intel.com
> Cc: qemu-devel@nongnu.org
> Subject: Re: [CXL volatile MEM] - Qemu command to turn on HMAT and 
> NUMA fails with assertion
> 
> EXTERNAL MAIL
> 
> 
> * Samarth Saxena (samarths@cadence.com) wrote:
> > Hi All,
> > 
> > I am trying the following command to run Qemu with Kernel 5.14.
> 
> cc'ing in Ben who I think owns the CXL stuff.
> 
> > qemu-system-x86_64 -M q35,accel=kvm,nvdimm=on,cxl=on,hmat=on -m 
> > 8448M,slots=2,maxmem=16G -smp 8,sockets=2,cores=2,threads=2 -hda 
> > /lan/dscratch/singhabh/shradha/ubuntu-20.10-64_with_orig_driver_back
> > up
> > .qcow2 -enable-kvm -usbdevice tablet -L $VB_ROOT/etc/vm/common/ 
> > -object memory-backend-ram,id=cxl-ram,share=on,size=256M -device
> > "pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,windo
> > w- base[0]=0x4c0000000,memdev[0]=cxl-ram" -device
> > cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0 -device 
> > cxl-type3,bus=rp0,memdev=cxl-ram,id=cxl-vmem0,size=256M -numa 
> > node,nodeid=0,memdev=cxl-ram -object 
> > memory-backend-ram,id=other-ram,size=8G,prealloc=n,share=off -numa
> > node,nodeid=1,memdev=other-ram,initiator=0 -numa
> > cpu,node-id=0,socket-id=0 -numa cpu,node-id=0,socket-id=1
> 
> You probably need to state which qemu tree and version you're using here.
> 
> > I get the following crash before the machine starts to boot.
> > 
> > qemu-system-x86_64: ../softmmu/memory.c:2443: memory_region_add_subregion_common: Assertion `!subregion->container' failed.
> 
> It's probably best to check with Ben, but you'll want a backtrace and figure out which subregion and region you're dealing with.
> 
> Dave
> 
> > 
> > Please help me find what's wrong here.
> > 
> > Regards,
> > [CadenceLogoRed185Regcopy1583174817new51584636989.png]<https://urlde
> > fense.com/v3/__https://www.cad__;!!EHscmS1ygiU1lA!Unz0pDVxMVCy_xrhyJ
> > 2NNrJg3z-bb2dughtewVIoD5hKzlg0zzNMUj0sZqFMTcY$
> > ence.com/en_US/home.html>
> > Samarth Saxena
> > Sr Principal Software Engineer
> > T: +911204308300
> > [UIcorrectsize1583179003.png]<https://www.cadence.com/en_US/home.htm
> > l> 
> > [16066EmailSignatureFortune100Best2021White92x1271617625037.png]<htt
> > ps ://www.cadence.com/en_US/home/company/careers.html>
> > 
> > 
> > 
> > 
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>