Re: Unexpected error in rme_configure_one() at ../target/arm/kvm-rme.c:159

[Jean-Philippe Brucker <jean-philippe@linaro.org>]

On Wed, Jun 05, 2024 at 11:28:47AM +1000, Gavin Shan wrote:
> > >    WriteSections64(): /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll AARCH64 small code model requires identical ELF and PE/COFF section offsets modulo 4 KB.
> > 
> > Ah I've seen this once but it disappeared as I tried to investigate and
> > I've since changed the implementation, so I don't have many notes about
> > it.
> > 
> > Maybe you could try to bisect from "ArmVirtPkg: ArmCcaIoMmu: Provide an
> > implementation for SetAttribute", but it may give false positives if the
> > error depends on some random linker placement. Could be
> > "ArmVirtPkg/ArmPlatformLibQemu: Setup early UART mapping in a Realm" which
> > adds a 4k page to the data section for the ealy RSI config call, though
> > that has explicit 4kB alignment.
> > 
> > In my notes I also wrote that changing "-z common-page-size=0x20" to 4k in
> > the link flags may have made the error disappear, but I doubt it's the
> > right fix.
> > 
> > I'll try GCC 11 to see if I can reproduce.
> > 
> 
> Ok. I run a git-bisect and the first problematic commit is 1153ae939c
> ("ArmVirtPkg/ArmPlatformLibQemu: Add a third-level page table for the UART idmap")

Ah thanks, I'm able to reproduce the problem now, it was my local config
that masked it.

> 
> I'm not familiar with edk2. The error is raised by BaseTools/Source/C/GenFw/Elf64Convert.c::WriteSections64()
> where the relocatable address isn't properly aligned to 4KB. So I modified the code
> as below, but I have to run two consecutive builds. In the first attempt build, I
> still hit the same error.

This seems to be because GenFw generates a file even on error, so it
doesn't retry the second time.

This commit moves the page tables from .rodata to .data. When linking
IdMap.obj into ArmPlatformPrePeiCore.dll, the alignment of the .text
section changes from 0x1000 to 0x800. This change comes from the linker
script putting .rodata into .text. I don't know why the included .rodata
alignment affects the .text alignment, but I don't think it matters here.

In GenFw, ScanSections64() calculates a mCoffAlignment as the max
.text/.data/.hii section alignement. Since with this commit, .data
alignement (0x1000) becomes larger than .text (0x800), it picks 0x1000 as
the output text offset, and then WriteSections64() complains that this
offset isn't equal to the input .text alignment modulo 0x1000.

The linker script says:

  /*
   * The alignment of the .data section should be less than or equal to the
   * alignment of the .text section. This ensures that the relative offset
   * between these sections is the same in the ELF and the PE/COFF versions of
   * this binary.
   */

but that's not what we're getting. I don't have a fix yet, other than
forcing the .text and .data alignment to 4k.

> ---> VirtPkg/Library/ArmPlatformLibQemu/IdMap.S
> 
>   .align    12
>   .globl    idmap
>   .globl    uart_pte
>   .section  ".data.idmap", "aw", %progbits
>   .align    12
> 
> # source edksetup.sh; export GCC5_AARCH64_PREFIX=
> # make -j -C BaseTools; \                                               <<< Failed on the first attempt
>   build -b DEBUG -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtQemu.dsc
>    :
> WriteSections64(): /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll AARCH64 small code model requires identical ELF and PE/COFF section offsets modulo 4 KB.
> make: *** [GNUmakefile:405: /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/OUTPUT/ArmPlatformPrePeiCore.efi] Error 2
> 
> # make -j -C BaseTools; \                                              <<< Succeed on the second attempt
>   build -b DEBUG -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtQemu.dsc
>    :
> Generating FVMAIN FV
> ######
> Fd File Name:QEMU_VARS (/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/FV/QEMU_VARS.fd)
>    :
> - Done -
> Build end time: 21:04:05, Jun.04 2024
> Build total time: 00:00:06
> 
> After that, I'm unable to start the guest with the edk2 image successfully.
> 
> host# # mount | grep 9p
> shr0 on /mnt/shr0 type 9p (rw,relatime,access=client,trans=virtio)
> host# cat ./realm.sh
> #!/bin/sh
> 
> SHR_DIR="/mnt/shr0"
> 
> qemu-system-aarch64 -accel kvm                              \
> -machine virt,gic-version=3,confidential-guest-support=rme0 \
> -cpu host -smp 2 -m 512M                                    \
> -object 'rme-guest,id=rme0,measurement-algo=sha512'         \
> -monitor none -serial mon:stdio -nographic                  \
> -bios ${SHR_DIR}/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/FV/QEMU_EFI.fd \
> -kernel ${SHR_DIR}/linux/arch/arm64/boot/Image              \
> -initrd ${SHR_DIR}/buildroot/output/images/rootfs.cpio      \
> -append 'console=ttyAMA0'
> 
> host# ./realm.sh
> UEFI firmware (version  built at 19:56:47 on Jun  4 2024)
> add-symbol-file /home/gavin/sandbox/C                              <<< I don't see more output after it

I'm guessing in this case the firmware was corrupted because GenFw fails the
first time and never generated a complete binary

> > 
> > Note that the guest edk2 is optional and experimental, you can use direct
> > kernel boot to get a working demo quicker.
> > 
> 
> I never did this before. Could you please provide the detailed steps on this?

Removing the -bios parameter to QEMU should be enough. You can also add
'earlycon' to -append to show early boot errors.

Thanks,
Jean