Re: Unexpected error in rme_configure_one() at ../target/arm/kvm-rme.c:159

[Jean-Philippe Brucker <jean-philippe@linaro.org>]

Hi Gavin,

On Sat, Jun 01, 2024 at 08:14:46PM +1000, Gavin Shan wrote:
> ---> guest edk2
> 
> # git clone https://git.codelinaro.org/linaro/dcap/edk2.git edk2-guest
> # cd edk2-guest; git checkout origin/cca/v2 -b cca/v2
> # git submodule update --init --recursive;  \
>   source edksetup.sh; make -j -C BaseTools; \
>   export GCC5_AARCH64_PREFIX=;              \

Doesn't this needs a cross-compiler, something like "aarch64-linux-gnu-" ?

>   build -b DEBUG -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtQemu.dsc
>    :
>   WriteSections64(): /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll AARCH64 small code model requires identical ELF and PE/COFF section offsets modulo 4 KB.
> cp -p -f /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/OvmfPkg/VirtioFsDxe/VirtioFsDxe/DEBUG/VirtioFsDxe.dll /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/OvmfPkg/VirtioFsDxe/VirtioFsDxe/DEBUG/VirtioFsDxe.debug
> cp -p -f /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/Disk/PartitionDxe/PartitionDxe/DEBUG/PartitionDxe.debug /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/PartitionDxe.debug
> "gcc" -MMD -MF /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/CryptoPkg/Library/OpensslLib/OpensslLibCrypto/OUTPUT/openssl/crypto/asn1/x_sig.obj.deps @/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/CryptoPkg/Library/OpensslLib/OpensslLibCrypto/OUTPUT/cc_resp.txt  -c -o /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/CryptoPkg/Library/OpensslLib/OpensslLibCrypto/OUTPUT/openssl/crypto/asn1/x_sig.obj  /home/gavin/sandbox/CCA/edk2-guest/CryptoPkg/Library/OpensslLib/openssl/crypto/asn1/x_sig.c
> "GenFw" -e DXE_CORE -o /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/OUTPUT/DxeCore.efi /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll
> GenSec -s EFI_SECTION_USER_INTERFACE -n ArmCpuDxe -o /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/FV/Ffs/B8D9777E-D72A-451F-9BDB-BAFB52A68415ArmCpuDxe/B8D9777E-D72A-451F-9BDB-BAFB52A68415SEC3.ui
> cp -p -f /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/DisplayEngineDxe/DisplayEngineDxe/DEBUG/*.map /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/DisplayEngineDxe/DisplayEngineDxe/OUTPUT
> cp -p -f /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/Disk/UdfDxe/UdfDxe/OUTPUT/UdfDxe.efi /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/Disk/UdfDxe/UdfDxe/DEBUG
> GenFw: ERROR 3000: Invalid
>   :
> build.py...
>  : error 7000: Failed to execute command
> 	make tbuild [/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore]
> 
> 
> build.py...
>  : error F002: Failed to build module
> 	/home/gavin/sandbox/CCA/edk2-guest/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf [AARCH64, GCC5, DEBUG]
> 
> - Failed -
> Build end time: 05:42:19, Jun.01 2024
> Build total time: 00:00:31
> 
> ---> Use the edk2 image from the latest QEMU source

Unfortunately this can't work at the moment because edk2 needs several
changes in order to run in a Realm

> 
> # cd /home/gavin/sandbox/CCA
> # cp /home/gavin/sandbox/qemu.main/build/pc-bios/edk2-aarch64-code.fd ./
> 
> ---> Start the guest and no output from the console
> 
> host# mount | grep 9p
> shr0 on /mnt/shr0 type 9p (rw,relatime,access=client,trans=virtio)
> host# export SHR_DIR="/mnt/shr0"
> host# qemu-system-aarch64 -accel kvm                              \
>       -machine virt,gic-version=3,confidential-guest-support=rme0 \
>       -cpu host -smp 2 -m 512M                                    \
>       -object 'rme-guest,id=rme0,measurement-algo=sha512'         \
>       -monitor none -serial mon:stdio -nographic                  \
>       -bios /mnt/edk2-aarch64-code.fd                             \
>       -kernel ${SHR_DIR}/linux/arch/arm64/boot/Image              \
>       -initrd ${SHR_DIR}/buildroot/output/images/rootfs.cpio      \
>       -append 'console=ttyAMA0'
>         :
>       <no output from the console>
>         :
>       (QEMU) q
> 
> There are some messages from host's console indicating RMI/RMM servicing
> states when the guest is running at background. After the guest is terminated,
> the host crashes.
> 
> SMC_RMM_RTT_CREATE            102dff000 122c2e000 1e00000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 1234a7000 2000000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 1235bd000 2200000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 12387c000 2400000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 123a5a000 2600000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 12407d000 2800000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 124109000 2a00000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 123e49000 2c00000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 124275000 2e00000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 123138000 3000000 3 > RMI_SUCCESS
> SMC_RMM_RTT_CREATE            102dff000 124d07000 3200000 3 > RMI_SUCCESS
>  :
>  :
> [22768.994481] rcu: INFO: rcu_preempt self-detected stall on CPU

This is a warning rather than a crash. The current KVM patches spend too
much time tearing down guest page tables (SMC calls to RMM) while holding
the mmu lock. Not very nice but harmless.

Thanks,
Jean

> [22769.006861] rcu: 	3-....: (2751 ticks this GP) idle=93ec/1/0x4000000000000000 softirq=114451/115721 fqs=1160
> [22769.020475] rcu: 	(t=5257 jiffies g=531913 q=7 ncpus=8)
> [22769.030547] CPU: 3 PID: 198 Comm: qemu-system-aar Not tainted 6.9.0-rc1-gavin-gfcfc92d6ff07 #1
> [22769.041847] Hardware name: QEMU QEMU Virtual Machine, BIOS unknown 2/2/2022
> [22769.050548] pstate: 60402009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [22769.059382] pc : kvm_realm_unmap_range+0x308/0x32c
> [22769.070275] lr : kvm_realm_unmap_range+0x304/0x32c
> [22769.075893] sp : ffff800080a3b930
> [22769.079929] x29: ffff800080a3b930 x28: 00000000003d7000 x27: 00000000003d6000
> [22769.092990] x26: 00000000c4000152 x25: ffffffffffffffff x24: 0000000000000000
> [22769.101150] x23: 0000010000000000 x22: 00000000c4000155 x21: 0000000102dff000
> [22769.109056] x20: ffff8000801a5e00 x19: 0000000000000000 x18: 0000000000000001
> [22769.117042] x17: 0000000000000000 x16: 000000000000000e x15: 0000000000000000
> [22769.124991] x14: 0000ffff7fa14000 x13: 0000000000000002 x12: 000000000010d594
> [22769.134213] x11: 0000000000000002 x10: 00000000ffffffff x9 : ffffffffffffffff
> [22769.142951] x8 : 0000000000000000 x7 : 0000000000000000 x6 : 00000000c2dff000
> [22769.151413] x5 : 0000000102f56000 x4 : 0000000000000015 x3 : 0000000000000000
> [22769.159932] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000
> [22769.169674] Call trace:
> [22769.174026]  kvm_realm_unmap_range+0x308/0x32c
> [22769.181046]  __unmap_stage2_range+0x60/0x7c
> [22769.186396]  kvm_free_stage2_pgd+0xa0/0xd4
> [22769.191766]  kvm_arch_flush_shadow_all+0x1c/0x34
> [22769.197879]  kvm_mmu_notifier_release+0x30/0x84
> [22769.203304]  __mmu_notifier_release+0x7c/0x1f8
> [22769.209340]  exit_mmap+0x264/0x274
> [22769.213992]  __mmput+0x40/0x150
> [22769.218635]  mmput+0x50/0x5c
> [22769.222606]  do_exit+0x288/0x92c
> [22769.226935]  do_group_exit+0x34/0x90
> [22769.231359]  get_signal+0x814/0x820
> [22769.236537]  do_signal+0x90/0x1320
> [22769.241145]  do_notify_resume+0xc8/0x140
> [22769.246458]  el0_svc+0xc8/0xdc
> [22769.250913]  el0t_64_sync_handler+0x13c/0x158
> [22769.256045]  el0t_64_sync+0x190/0x194
> 
> Thanks,
> Gavin
> 
> 
> 
>