Re: Unexpected error in rme_configure_one() at ../target/arm/kvm-rme.c:159

[Gavin Shan <gshan@redhat.com>]

On 5/31/24 14:19, Itaru Kitayama wrote:
>> On May 30, 2024, at 22:30, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>> Cc'ing more developers
>>
>> On 30/5/24 06:30, Itaru Kitayama wrote:
>>> Hi,
>>> When I see a Realm VM creation fails with:
>>> Unexpected error in rme_configure_one() at ../target/arm/kvm-rme.c:159:
>>> qemu-system-aarch64: RME: failed to configure SVE: Invalid argument
>>> test.sh: line 8:  2502 Aborted                 qemu-system-aarch64 -M 'virt,acpi=off,gic-version=3' -cpu host -enable-kvm -smp 2 -m 512M -overcommit 'mem-lock=on' -M 'confidential-guest-support=rme0' -object 'rme-guest,id=rme0,measurement-algo=sha512,num-pmu-counters=6,sve-vector-length=256' -kernel Image -initrd rootfs.cpio -append 'earycon console=ttyAMA0 rdinit=/sbin/init' -nographic -net none
>>> do I need to suspect first the VMM, QEMU, or the Image? The kernel is built with LLVM, does it matter?
>>> Thanks,
>>> Itaru.
>>
> 
> I’m testing Jean’s repo at:
> 
> https://git.codelinaro.org/linaro/dcap/qemu/-/tree/cca/v2?ref_type=heads
> 

I got a chance to try CCA software components, suggested by [1]. However, the edk2
is stuck somewhere. I didn't reach to stage of loading guest kernel yet. I'm replying
to see if anyone has a idea.

[1] https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/Building+an+RME+stack+for+QEMU


---> tf-rmm

# git clone https://git.codelinaro.org/linaro/dcap/rmm.git tf-rmm
# cd tf-rmm
# git checkout origin/cca/v2 -b cca/v2
# git submodule update --init --recursive
# export CROSS_COMPILE=
# cmake -DCMAKE_BUILD_TYPE=Debug -DRMM_CONFIG=qemu_virt_defcfg -B build-qemu
# cmake --build build-qemu

---> edk2

# git clone git@github.com:tianocore/edk2.git
# cd edk2
# git submodule update --init --recursive
# source edksetup.sh
# make -j -C BaseTools
# export GCC5_AARCH64_PREFIX=
# build -b RELEASE -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtQemuKernel.dsc

---> tf-a

# git clone https://git.codelinaro.org/linaro/dcap/tf-a/trusted-firmware-a.git tf-a
# cd tf-a
# git checkout origin/cca/v2 -b cca/v2
# make -j CROSS_COMPILE= PLAT=qemu ENABLE_RME=1 DEBUG=1 LOG_LEVEL=40 \
   QEMU_USE_GIC_DRIVER=QEMU_GICV3 RMM=../rmm/build-qemu/Debug/rmm.img \
   BL33=../edk2/Build/ArmVirtQemuKernel-AARCH64/RELEASE_GCC5/FV/QEMU_EFI.fd all fip
# dd if=build/qemu/debug/bl1.bin of=flash.bin
# dd if=build/qemu/debug/fip.bin of=flash.bin seek=64 bs=4096

---> QEMU

# git clone https://git.qemu.org/git/qemu.git qemu.main
# cd qemu.main
# ./configure --target-list=aarch64-softmmu
# make -j 60

---> boot the host on the emulated hardware

/home/gavin/sandbox/qemu.main/build/qemu-system-aarch64                            \
-M virt,virtualization=on,secure=on,gic-version=3,acpi=off                         \
-cpu max,x-rme=on -m 8G -smp 8                                                     \
-nodefaults -monitor none -serial mon:stdio -nographic                             \
-bios /home/gavin/sandbox/CCA/tf-a/flash.bin                                       \
-kernel /home/gavin/sandbox/CCA/linux/arch/arm64/boot/Image                        \
-drive format=raw,if=none,file=${CCA}/buildroot/output/images/rootfs.ext4,id=hd0   \
-device virtio-blk-pci,drive=hd0                                                   \
-append "root=/dev/vda console=ttyAMA0"
   :
NOTICE:  Booting Trusted Firmware
NOTICE:  BL1: v2.10.0(debug):99e0b97aa-dirty
NOTICE:  BL1: Built : 00:31:35, May 31 2024
INFO:    BL1: RAM 0xe0ee000 - 0xe0f7000
INFO:    BL1: Loading BL2
INFO:    Loading image id=1 at address 0xe06b000
INFO:    Image id=1 loaded: 0xe06b000 - 0xe0742d1
NOTICE:  BL1: Booting BL2
INFO:    Entry point address = 0xe06b000
INFO:    SPSR = 0x3cd
INFO:    [GPT] Boot Configuration
INFO:      PPS/T:     0x2/40
INFO:      PGS/P:     0x0/12
INFO:      L0GPTSZ/S: 0x0/30
INFO:      PAS count: 0x6
INFO:      L0 base:   0xedfe000
INFO:    [GPT] PAS[0]: base 0xe001000, size 0xff000, GPI 0xa, type 0x1
INFO:    [GPT] PAS[1]: base 0xe100000, size 0xcfe000, GPI 0x8, type 0x1
INFO:    [GPT] PAS[2]: base 0xedfe000, size 0x202000, GPI 0xa, type 0x1
INFO:    [GPT] PAS[3]: base 0x40000000, size 0x100000, GPI 0x9, type 0x1
INFO:    [GPT] PAS[4]: base 0x40100000, size 0x2800000, GPI 0xb, type 0x1
INFO:    [GPT] PAS[5]: base 0x42900000, size 0x1fd700000, GPI 0x9, type 0x1
INFO:    Enabling Granule Protection Checks
NOTICE:  BL2: v2.10.0(debug):99e0b97aa-dirty
NOTICE:  BL2: Built : 00:31:35, May 31 2024
INFO:    BL2: Doing platform setup
INFO:    Reserved RMM memory [0x40100000, 0x428fffff] in Device tree
INFO:    BL2: Loading image id 3
INFO:    Loading image id=3 at address 0xe0a0000
INFO:    Image id=3 loaded: 0xe0a0000 - 0xe0b10c4
INFO:    BL2: Loading image id 35
INFO:    Loading image id=35 at address 0x40100000
INFO:    Image id=35 loaded: 0x40100000 - 0x403033b0
INFO:    BL2: Loading image id 5
INFO:    Loading image id=5 at address 0x60000000
INFO:    Image id=5 loaded: 0x60000000 - 0x60200000
NOTICE:  BL2: Booting BL31
INFO:    Entry point address = 0xe0a0000
INFO:    SPSR = 0x3cd
NOTICE:  BL31: v2.10.0(debug):99e0b97aa-dirty
NOTICE:  BL31: Built : 00:31:35, May 31 2024
INFO:    GICv3 without legacy support detected.
INFO:    ARM GICv3 driver initialized in EL3
INFO:    Maximum SPI INTID supported: 287
INFO:    BL31: Initializing runtime services
INFO:    RMM setup done.
INFO:    BL31: Initializing RMM
INFO:    RMM init start.
Booting RMM v.0.4.0(debug) 17924bc Built with GCC 11.4.1
RMM-EL3 Interface v.0.2
Boot Manifest Interface v.0.3
RMI/RSI ABI v.1.0/1.0 built: May 31 2024 00:21:59
INFO:    RMM init end.
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x60000000
INFO:    SPSR = 0x3c9
UEFI firmware (version  built at 01:31:23 on May 31 2024)

The boot is stuck and no more output after that. I tried adding more verbose output
from edk2 and found it's stuck at the following point.


ArmVirtPkg/PrePi/PrePi.c::PrePiMain
rmVirtPkg/Library/PlatformPeiLib/PlatformPeiLib.c::PlatformPeim

  #ifdef MDE_CPU_AARCH64
   //
   // Set the SMCCC conduit to SMC if executing at EL2, which is typically the
   // exception level that services HVCs rather than the one that invokes them.
   //
   if (ArmReadCurrentEL () == AARCH64_EL2) {
     Status = PcdSetBoolS (PcdMonitorConduitHvc, FALSE);       // The function is never returned in my case
     ASSERT_EFI_ERROR (Status);
   }
  #endif

Thanks,
Gavin