Re: ndctl cxl test suite fails in arm64 QEMU

[Itaru Kitayama <itaru.kitayama@linux.dev>]

> On Mar 13, 2025, at 18:11, Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote:
> 
> On Fri, 7 Mar 2025 10:44:23 +0900
> Itaru Kitayama <itaru.kitayama@linux.dev> wrote:
> 
>> Hi Jonathan, Alison,
>> 
>>> On Feb 28, 2025, at 23:34, Itaru Kitayama <itaru.kitayama@linux.dev> wrote:
>>> 
>>> On Tue, Feb 25, 2025 at 02:40:25PM -0800, Alison Schofield wrote:  
>>>> On Tue, Feb 25, 2025 at 01:09:57PM +0900, Itaru Kitayama wrote:  
>>>>> Hi,
>>>>> 
>>>>> Has anyone noticed the ndctl cxl test suite failures I reported below on arm64, QEMU emulation?
>>>>> 
>>>>> https://github.com/pmem/ndctl/issues/278
>>>>> 
>>>>> I’m using Jonathan’s latest CXL capable QEMU [1], and the latest CXL kernel [2].  
>>>> 
>>>> Hi Itaru,
>>>> 
>>>> Looking at region.c:size_store() it seems that alloc_hpa() is failing.
>>>> (kstrtou64 is arch independent and it works for me with same script &
>>>> value)
>>>> 
>>>> Check the dmesg log at the time of the failure. alloc_hpa() may be
>>>> emitting a message.  (In the run_qemu.sh cmdline add '--cxl-debug')  
>>> 
>>> dynamic debug reported this (kernel is without kaslr, today's cxl/next):
>>> 
>>> [  193.267649] device: 'region4': device_add
>>> [  193.267931] bus: 'cxl': add device region4
>>> [  193.268284] cxl region4: bus: 'cxl': __driver_probe_device: matched device with driver cxl_region
>>> [  193.268333] cxl region4: bus: 'cxl': really_probe: probing driver cxl_region with device
>>> [  193.268401] cxl_region region4: no default pinctrl state
>>> [  193.268482] cxl_region region4: probe with driver cxl_region rejects match -6
>>> 
>>> does the above show the region4 was not added properly? The sysfs
>>> entries are there after the out of range failure.  
>> 
>> The driver/cxl/core/region.c’s size_store() triggers a sequence of: 
>> 
>> alloc_hpa()
>> alloc_free_mem_region()
>> get_free_mem_region() // with flags set to 0, retruns -ERANGE
>> 
>> While I am looking at the resource core code, do you think this is caused by the old firmware I am using
>> when booting QEMU built off of your CXL topic branch? Suggestions are welcome.
> Unlikely as I don't think there have been any relevant changes in edk2 for
> a long long time.

I am trying to run the run_qemu.sh script which is maintained by the Intel CXL folk with:

$ qemu=$HOME/projects/qemu/build/qemu-system-aarch64 run_qemu.sh --roots ~/ubuntu24.img -r none --no-cxl --no-kvm  --hmat

and I get:

mkosi 12
/home/itaru/projects/qemu/build/qemu-system-aarch64 -machine virt,accel=tcg,nvdimm=on,hmat=on -m 8192M,slots=4,maxmem=40964M -smp 4,sockets=2,cores=2,threads=1 -display none -nographic -drive if=pflash,format=raw,unit=0,file=/home/itaru/edk2-aarch64-code.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=/home/itaru/qemu-arm64-efivars.test -drive file=/home/itaru/ubuntu24.img,format=raw,media=disk -device e1000,netdev=net0,mac=52:54:00:12:34:56 -netdev user,id=net0,hostfwd=tcp::10022-:22 -snapshot -object memory-backend-ram,id=mem0,size=2048M -numa node,nodeid=0,memdev=mem0,initiator=0 -numa cpu,node-id=0,socket-id=0 -object memory-backend-ram,id=mem1,size=2048M -numa node,nodeid=1,memdev=mem1,initiator=1 -numa cpu,node-id=1,socket-id=1 -object memory-backend-ram,id=mem2,size=2048M -numa node,nodeid=2,memdev=mem2,initiator=0 -object memory-backend-ram,id=mem3,size=2048M -numa node,nodeid=3,memdev=mem3,initiator=1 -numa node,nodeid=4,initiator=0 -object memory-backend-file,id=nvmem0,share=on,mem-path=nvdimm-0,size=16384M,align=1G -device nvdimm,memdev=nvmem0,id=nv0,label-size=2M,node=4 -numa node,nodeid=5,initiator=1 -object memory-backend-file,id=nvmem1,share=on,mem-path=nvdimm-1,size=16384M,align=1G -device nvdimm,memdev=nvmem1,id=nv1,label-size=2M,node=5 -numa dist,src=0,dst=0,val=10 -numa dist,src=0,dst=1,val=21 -numa dist,src=0,dst=2,val=12 -numa dist,src=0,dst=3,val=21 -numa dist,src=0,dst=4,val=17 -numa dist,src=0,dst=5,val=28 -numa dist,src=1,dst=1,val=10 -numa dist,src=1,dst=2,val=21 -numa dist,src=1,dst=3,val=12 -numa dist,src=1,dst=4,val=28 -numa dist,src=1,dst=5,val=17 -numa dist,src=2,dst=2,val=10 -numa dist,src=2,dst=3,val=21 -numa dist,src=2,dst=4,val=28 -numa dist,src=2,dst=5,val=28 -numa dist,src=3,dst=3,val=10 -numa dist,src=3,dst=4,val=28 -numa dist,src=3,dst=5,val=28 -numa dist,src=4,dst=4,val=10 -numa dist,src=4,dst=5,val=28 -numa dist,src=5,dst=5,val=10 -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=5 -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=2000M -numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=20 -numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=1000M -numa hmat-lb,initiator=0,target=2,hierarchy=memory,data-type=access-latency,latency=10 -numa hmat-lb,initiator=0,target=2,hierarchy=memory,data-type=access-bandwidth,bandwidth=1500M -numa hmat-lb,initiator=0,target=3,hierarchy=memory,data-type=access-latency,latency=20 -numa hmat-lb,initiator=0,target=3,hierarchy=memory,data-type=access-bandwidth,bandwidth=1000M -numa hmat-lb,initiator=0,target=4,hierarchy=memory,data-type=access-latency,latency=30 -numa hmat-lb,initiator=0,target=4,hierarchy=memory,data-type=access-bandwidth,bandwidth=1000M -numa hmat-lb,initiator=0,target=5,hierarchy=memory,data-type=access-latency,latency=40 -numa hmat-lb,initiator=0,target=5,hierarchy=memory,data-type=access-bandwidth,bandwidth=500M -numa hmat-cache,node-id=0,size=10K,level=1,associativity=direct,policy=write-back,line=64 -numa hmat-lb,initiator=1,target=0,hierarchy=memory,data-type=access-latency,latency=20 -numa hmat-lb,initiator=1,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=1000M -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-latency,latency=5 -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=2000M -numa hmat-lb,initiator=1,target=2,hierarchy=memory,data-type=access-latency,latency=20 -numa hmat-lb,initiator=1,target=2,hierarchy=memory,data-type=access-bandwidth,bandwidth=1000M -numa hmat-lb,initiator=1,target=3,hierarchy=memory,data-type=access-latency,latency=10 -numa hmat-lb,initiator=1,target=3,hierarchy=memory,data-type=access-bandwidth,bandwidth=1500M -numa hmat-lb,initiator=1,target=4,hierarchy=memory,data-type=access-latency,latency=40 -numa hmat-lb,initiator=1,target=4,hierarchy=memory,data-type=access-bandwidth,bandwidth=500M -numa hmat-lb,initiator=1,target=5,hierarchy=memory,data-type=access-latency,latency=30 -numa hmat-lb,initiator=1,target=5,hierarchy=memory,data-type=access-bandwidth,bandwidth=1000M -numa hmat-cache,node-id=1,size=10K,level=1,associativity=direct,policy=write-back,line=64
qemu-system-aarch64: -device nvdimm,memdev=nvmem0,id=nv0,label-size=2M,node=4: memory hotplug is not enabled: missing acpi-ged device

QEMU is built off of your latest update, and I could not find the acpi-ged device what do I do?

Thanks,
Itaru.

> 
> Jonathan
> 
>> 
>> Itaru. 
>> 
>>> 
>>> Itaru.
>>> 
>>>> 
>>>> Sounds like Marc is on to something and maybe the KASLR is known
>>>> to cause this problem.
>>>> 
>>>> Thanks for posting to this list and send any further questions or
>>>> comments. Happy to have you using run_qemu.sh and the cxl tests!
>>>> 
>>>> Alison
>>>> 
>>>> 
>>>>> 
>>>>> [1] https://gitlab.com/jic23/qemu/ cxl-2025-02-20
>>>>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/
>>>>> 
>>>>> Thanks,
>>>>> Itaru.