Re: [PATCH v2 07/39] efi: Add a script for building and testing U-Boot on UEFI

[Heinrich Schuchardt <xypron.glpk@gmx.de>]

On 9/25/21 2:30 AM, Simon Glass wrote:
> It is quite complicating to run U-Boot on qemu since we have four

Nits:
%s/complicating/complicated/   %/qemu/QEMU/

> different builds and they must use different versions of qemu and the

QEMU

> UEFI binaries.
>
> Add a script to help.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> Changes in v2:
> - Add MAINTAINERS entry
> - Add support for creating a partition table with a filesystem inside
> - Add support for running qemu with just a serial console (no display)
>
>   MAINTAINERS                        |   1 +
>   doc/develop/uefi/u-boot_on_efi.rst |  62 ++++++++++
>   scripts/build-efi.sh               | 188 +++++++++++++++++++++++++++++
>   3 files changed, 251 insertions(+)
>   create mode 100755 scripts/build-efi.sh
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 019c87592c5..84750be81a9 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -706,6 +706,7 @@ M:	Heinrich Schuchardt <xypron.glpk@gmx.de>
>   S:	Maintained
>   W:	https://u-boot.readthedocs.io/en/latest/develop/uefi/u-boot_on_efi.html
>   F:	lib/efi/efi_app.c
> +F:	scripts/build-efi.sh
>
>   EFI PAYLOAD
>   M:	Heinrich Schuchardt <xypron.glpk@gmx.de>
> diff --git a/doc/develop/uefi/u-boot_on_efi.rst b/doc/develop/uefi/u-boot_on_efi.rst
> index c9a41bc919f..4b2a733076d 100644
> --- a/doc/develop/uefi/u-boot_on_efi.rst
> +++ b/doc/develop/uefi/u-boot_on_efi.rst
> @@ -96,6 +96,11 @@ that EFI does not support booting a 64-bit application from a 32-bit
>   EFI (or vice versa). Also it will often fail to print an error message if
>   you get this wrong.
>
> +You may find the script `scripts/build-efi.sh` helpful for building and testing
> +U-Boot on UEFI on QEMU. It also includes links to UEFI binaries dating from
> +2021.
> +
> +See `Example run`_ for an example run.
>
>   Inner workings
>   --------------
> @@ -191,6 +196,63 @@ of code is built this way (see the extra- line in lib/efi/Makefile).
>   Everything else is built as a normal U-Boot, so is always 32-bit on x86 at
>   present.
>
> +Example run
> +-----------
> +
> +This shows running with serial enabled (see `include/configs/efi-x86_app.h`)::
> +
> +   $ scripts/build-efi.sh -wsPr
> +   Packaging efi-x86_app32
> +   Running qemu-system-i386
> +
> +   BdsDxe: failed to load Boot0001 "UEFI QEMU HARDDISK QM00005 " from PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0xFFFF,0x0): Not Found
> +   BdsDxe: loading Boot0002 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)
> +   BdsDxe: starting Boot0002 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)
> +
> +   UEFI Interactive Shell v2.2
> +   EDK II
> +   UEFI v2.70 (EDK II, 0x00010000)
> +   Mapping table
> +         FS0: Alias(s):HD0a65535a1:;BLK1:
> +             PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0xFFFF,0x0)/HD(1,GPT,0FFD5E61-3B0C-4326-8049-BDCDC910AF72,0x800,0xB000)
> +        BLK0: Alias(s):
> +             PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0xFFFF,0x0)
> +
> +   Press ESC in 5 seconds to skip startup.nsh or any other key to continue.
> +   Shell> fs0:u-boot-app.efi
> +   U-Boot EFI App (using allocated RAM address 47d4000) key=8d4, image=06a6f610
> +   starting
> +
> +
> +   U-Boot 2022.01-rc4 (Sep 19 2021 - 14:03:20 -0600)
> +
> +   CPU: x86, vendor Intel, device 663h
> +   DRAM:  32 MiB
> +    0: efi_media_0  PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0xFFFF,0x0)
> +    1: <partition>  PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0xFFFF,0x0)/HD(1,GPT,0FFD5E61-3B0C-4326-8049-BDCDC910AF72,0x800,0xB000)
> +   Loading Environment from nowhere... OK
> +   Model: EFI x86 Application
> +   Hit any key to stop autoboot:  0
> +
> +   Partition Map for EFI device 0  --   Partition Type: EFI
> +
> +   Part    Start LBA       End LBA            Name
> +           Attributes
> +           Type GUID
> +           Partition GUID
> +     1     0x00000800      0x0000b7ff      "boot"
> +           attrs:  0x0000000000000000
> +           type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
> +           guid:   0ffd5e61-3b0c-4326-8049-bdcdc910af72
> +          19   startup.nsh
> +      528384   u-boot-app.efi
> +       10181   NvVars
> +
> +   3 file(s), 0 dir(s)
> +
> +   => QEMU: Terminated
> +
> +
>   Future work
>   -----------
>   This work could be extended in a number of ways:
> diff --git a/scripts/build-efi.sh b/scripts/build-efi.sh
> new file mode 100755
> index 00000000000..e6c035a7495
> --- /dev/null
> +++ b/scripts/build-efi.sh
> @@ -0,0 +1,188 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0+
> +#
> +# Script to build an EFI thing suitable for booting with qemu, possibly running
> +# it also.
> +
> +# This just an example. It assumes that
> +
> +# - you build U-Boot in /tmp/b/<name> where <name> is the U-Boot board config
> +# - /mnt/x is a directory used for mounting
> +# - you have access to the 'pure UEFI' builds for qemu
> +#
> +# UEFI binaries for QEMU used for testing this script:
> +#
> +# OVMF-pure-efi.i386.fd at
> +# https://drive.google.com/file/d/1jWzOAZfQqMmS2_dAK2G518GhIgj9r2RY/view?usp=sharing
> +
> +# OVMF-pure-efi.x64.fd at
> +# https://drive.google.com/file/d/1c39YI9QtpByGQ4V0UNNQtGqttEzS-eFV/view?usp=sharing
> +
> +set -e
> +
> +usage() {
> +	echo "Usage: $0 [-a | -p] [other opts]" 1>&2
> +	echo 1>&2
> +	echo "   -a   - Package up the app" 1>&2
> +	echo "   -o   - Use old EFI app build (before 32/64 split)" 1>&2

Why is this option needed?

> +	echo "   -p   - Package up the payload" 1>&2
> +	echo "   -P   - Create a partition table" 1>&2
> +	echo "   -r   - Run qemu with the image" 1>&2

QEMU

> +	echo "   -s   - Run qemu with serial only (no display)" 1>&2

QEMU

Best regards

Heinrich

> +	echo "   -w   - Use word version (32-bit)" 1>&2
> +	exit 1
> +}
> +
> +# 32- or 64-bit EFI
> +bitness=64
> +
> +# app or payload ?
> +type=app
> +
> +# create a partition table and put the filesystem in that (otherwise put the
> +# filesystem in the raw device)
> +part=
> +
> +# run the image with qemu
> +run=
> +
> +# run qemu without a display (U-Boot must be set to stdout=serial)
> +serial=
> +
> +# before the 32/64 split of the app
> +old=
> +
> +while getopts "aopPrsw" opt; do
> +	case "${opt}" in
> +	a)
> +		type=app
> +		;;
> +	p)
> +		type=payload
> +		;;
> +	r)
> +		run=1
> +		;;
> +	s)
> +		serial=1
> +		;;
> +	w)
> +		bitness=32
> +		;;
> +	o)
> +		old=1
> +		;;
> +	P)
> +		part=1
> +		;;
> +	*)
> +		usage
> +		;;
> +	esac
> +done
> +
> +run_qemu() {
> +	extra=
> +	if [[ "${bitness}" = "64" ]]; then
> +		qemu=qemu-system-x86_64
> +		bios=OVMF-pure-efi.x64.fd
> +	else
> +		qemu=qemu-system-i386
> +		bios=OVMF-pure-efi.i386.fd
> +	fi
> +	if [[ -n "${serial}" ]]; then
> +		extra="-display none -serial mon:stdio"
> +	fi
> +	echo "Running ${qemu}"
> +	"${qemu}" -bios "${bios}" \
> +		-drive id=disk,file="${IMG}",if=none,format=raw \
> +		-nic none -device ahci,id=ahci \
> +		-device ide-hd,drive=disk,bus=ahci.0 ${extra}
> +}
> +
> +# Put files in /tmp/b/${BUILD}
> +setup_files() {
> +	echo "Packaging ${BUILD}"
> +	mkdir -p $TMP
> +	cat >$TMP/startup.nsh <<EOF
> +fs0:u-boot-${type}.efi
> +EOF
> +	sudo cp /tmp/b/$BUILD/u-boot-${type}.efi $TMP
> +
> +	# Can copy in other files here:
> +	#sudo cp /tmp/b/$BUILD/image.bin $TMP/chromeos.rom
> +	#sudo cp /boot/vmlinuz-5.4.0-77-generic $TMP/vmlinuz
> +}
> +
> +# Copy files into the filesystem
> +copy_files() {
> +	sudo cp $TMP/* $MNT
> +}
> +
> +# Create a filesystem on a raw device and copy in the files
> +setup_raw() {
> +	mkfs.vfat "${IMG}" >/dev/null
> +	sudo mount -o loop "${IMG}" $MNT
> +	copy_files
> +	sudo umount $MNT
> +}
> +
> +# Create a partition table and put the filesystem in the first partition
> +# then copy in the files
> +setup_part() {
> +	# Create a gpt partition table with one parittion
> +	parted "${IMG}" mklabel gpt 2>/dev/null
> +
> +	# This doesn't work correctly. It creates:
> +	# Number  Start   End     Size    File system  Name  Flags
> +	#  1      1049kB  24.1MB  23.1MB               boot  msftdata
> +	# Odd if the same is entered interactively it does set the FS type
> +	parted -s -a optimal -- "${IMG}" mkpart boot fat32 1MiB 23MiB
> +
> +	# Map this partition to a loop device
> +	kp="$(sudo kpartx -av ${IMG})"
> +	read boot_dev<<<$(grep -o 'loop.*p.' <<< "${kp}")
> +	test "${boot_dev}"
> +	dev="/dev/mapper/${boot_dev}"
> +
> +	mkfs.vfat "${dev}" >/dev/null
> +
> +	sudo mount -o loop "${dev}" $MNT
> +
> +	copy_files
> +
> +	# Sync here since this makes kpartx more likely to work the first time
> +	sync
> +	sudo umount $MNT
> +
> +	# For some reason this needs a sleep or it sometimes fails, if it was
> +	# run recently (in the last few seconds)
> +	if ! sudo kpartx -d "${IMG}" > /dev/null; then
> +		sleep .5
> +		sudo kpartx -d "${IMG}" > /dev/null || \
> +			echo "Failed to remove ${boot_dev}, use: sudo kpartx -d ${IMG}"
> +	fi
> +}
> +
> +TMP="/tmp/efi${bitness}${type}"
> +MNT=/mnt/x
> +BUILD="efi-x86_${type}${bitness}"
> +IMG=try.img
> +
> +if [[ -n "${old}" && "${bitness}" = "32" ]]; then
> +	BUILD="efi-x86_${type}"
> +fi
> +
> +setup_files
> +
> +qemu-img create "${IMG}" 24M >/dev/null
> +
> +if [[ -n "${part}" ]]; then
> +	setup_part
> +else
> +	setup_raw
> +fi
> +
> +if [[ -n "${run}" ]]; then
> +	run_qemu
> +fi
>