[PATCH 00/18] x86: Improve operation under QEMU

[PATCH 00/18] x86: Improve operation under QEMU

[Simon Glass]

U-Boot can start and boot an OS in both qemu-x86 and qemu-x86_64 but it
is not perfect.

With both builds, executing the VESA ROM causes an intermittent hang, at
least on some AMD CPUs.

With qemu-x86_64 kvm cannot be used since the move to long mode (64-bit)
is done in a way that works on real hardware but not with QEMU. This
means that performance is 4-5x slower than it could be, at least on my
CPU.

We can work around the first problem by using Bochs, which is anyway a
better choice than VESA for QEMU. But this results in the Ubuntu
installer failing to boot via EFI. It isn't clear whether this due to
a problem with the UEFI GOP, or something else. It hangs before the
Ubuntu logo appears.

So this series is unfinished. I thought it best to post what is here in
the hope that others can help work out the kinks.


Simon Glass (18):
  scripts: Add a script for building and booting QEMU
  x86: Expand x86_64 early memory
  RFC: x86: qemu: Switch to bochs display
  x86: qemu: Enable dhrystone
  x86: qemu: Avoid accessing BSS too early
  x86: Drop mpspec from the SPL build
  x86: Add some log categories
  x86: Drop use of CONFIG_REALMODE_DEBUG
  x86: Avoid clearing the VESA display
  x86: Add 64-bit entries to the GDT
  x86: Use defines for the cache flags
  x86: spl: Drop duplicate CPU init
  x86: Drop the message about features missing in 64-bit
  x86: Include stdbool.h in interrupt header
  x86: Tidy up the GDT size in start/16.S
  x86: Disable paging before changing to long mode
  x86: Use the same GDT when jumping to long mode
  x86: Use a simple jump into long mode

 MAINTAINERS                      |   8 ++
 arch/x86/cpu/i386/call64.S       |  35 +++----
 arch/x86/cpu/i386/cpu.c          |  18 +++-
 arch/x86/cpu/qemu/qemu.c         |  20 ++--
 arch/x86/cpu/start.S             |   4 +-
 arch/x86/cpu/start16.S           |   3 +-
 arch/x86/include/asm/interrupt.h |   1 +
 arch/x86/include/asm/processor.h |   5 +-
 arch/x86/lib/Makefile            |   2 +
 arch/x86/lib/bios.c              |  27 +++--
 arch/x86/lib/bios_interrupts.c   |   8 +-
 arch/x86/lib/i8259.c             |   2 +
 arch/x86/lib/spl.c               |   4 +-
 configs/qemu-x86_64_defconfig    |  10 +-
 configs/qemu-x86_defconfig       |   6 +-
 doc/board/emulation/index.rst    |   1 +
 doc/board/emulation/script.rst   |  61 +++++++++++
 scripts/build-qemu.sh            | 175 +++++++++++++++++++++++++++++++
 18 files changed, 331 insertions(+), 59 deletions(-)
 create mode 100644 doc/board/emulation/script.rst
 create mode 100755 scripts/build-qemu.sh

-- 
2.34.1

[PATCH 01/18] scripts: Add a script for building and booting QEMU

[Simon Glass]

It is handy to be able to quickly build and boot a QEMU image for a
particular architecture and distro.

Add a script for this purpose. It supports only arm and x86 at present.
For distros it only supports Ubuntu. Both 32- and 64-bit builds are
supported.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 MAINTAINERS                    |   8 ++
 doc/board/emulation/index.rst  |   1 +
 doc/board/emulation/script.rst |  61 ++++++++++++
 scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
 4 files changed, 245 insertions(+)
 create mode 100644 doc/board/emulation/script.rst
 create mode 100755 scripts/build-qemu.sh

diff --git a/MAINTAINERS b/MAINTAINERS
index 0399ed1dbf6..b45bb96d5a5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1110,6 +1110,14 @@ F:	tools/efivar.py
 F:	tools/file2include.c
 F:	tools/mkeficapsule.c
 
+EMULATION
+M:	Simon Glass <sjg@chromium.org>
+S:	Maintained
+W:	https://docs.u-boot.org/en/latest/board/emulation/script.html
+F:	configs/qemu_x86*
+F:	doc/board/emulation/script.rst
+F:	scripts/build-qemu.sh
+
 ENVIRONMENT
 M:	Joe Hershberger <joe.hershberger@ni.com>
 S:	Maintained
diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
index f8908166276..5a2a00ae225 100644
--- a/doc/board/emulation/index.rst
+++ b/doc/board/emulation/index.rst
@@ -8,6 +8,7 @@ Emulation
 
    acpi
    blkdev
+   script
    qemu-arm
    qemu-mips
    qemu-ppce500
diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
new file mode 100644
index 00000000000..23981e333cb
--- /dev/null
+++ b/doc/board/emulation/script.rst
@@ -0,0 +1,61 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+Script for building and running
+===============================
+
+You may find the script `scripts/build-qemu.sh` helpful for building and testing
+U-Boot on QEMU.
+
+If uses a environment variables to control how it works:
+
+ubdir
+    base directory for building U-Boot, with each board being in its own
+    subdirectory
+
+imagedir
+    directory containing OS images, containin a subdirectory for each distro
+    type (e.g. ubuntu/
+
+Once configured, you can build and run QEMU for arm64 like this::
+
+    scripts/build-qemu.sh -rsw
+
+No support is currently included for specifying a root disk, so this script can
+only be used to start installers.
+
+Options
+~~~~~~~
+
+Options are available to control the script:
+
+-a <arch>
+    Select architecture (default arm, x86)
+
+-B
+    Don't build; assume a build exists
+
+-k
+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
+    emulator
+
+-o <os>
+    Run an Operating System. For now this only supports 'ubuntu'. The name of
+    the OS file must remain unchanged from its standard name on the Ubuntu
+    website.
+
+-r
+    Run QEMU with the image (by default this is not done)
+
+-R
+    Select OS release (e.g. 24.04).
+
+-s
+    Use serial only (no display)
+
+-w
+    Use word version (32-bit). By default, 64-bit is used
+
+.. note::
+
+    Note: For now this is a shell script, but if it expands it might be better
+    as Python, accepting the slower startup.
diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
new file mode 100755
index 00000000000..0ff53593cf9
--- /dev/null
+++ b/scripts/build-qemu.sh
@@ -0,0 +1,175 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Script to build U-Boot suitable for booting with QEMU, possibly running
+# it, possibly with an OS image
+
+# This just an example. It assumes that
+
+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
+# - your OS images are in ${imagedir}/{distroname}/...
+
+# So far the script supports only ARM and x86.
+
+set -e
+
+usage() {
+	(
+	if [[ -n "$1" ]]; then
+		echo "$1"
+		echo
+	fi
+	echo "Usage: $0 -aBkrsw"
+	echo
+	echo "   -a   - Select architecture (arm, x86)"
+	echo "   -B   - Don't build; assume a build exists"
+	echo "   -k   - Use kvm (kernel-based Virtual Machine)"
+	echo "   -o   - Run Operating System ('ubuntu' only for now)"
+	echo "   -r   - Run QEMU with the image"
+	echo "   -R   - Select OS release (e.g. 24.04)"
+	echo "   -s   - Use serial only (no display)"
+	echo "   -w   - Use word version (32-bit)" ) >&2
+	exit 1
+}
+
+# Directory tree for OS images
+imagedir=${imagedir-/vid/software/linux}
+
+# architecture (arm or x86)
+arch=arm
+
+# 32- or 64-bit build
+bitness=64
+
+# Build U-Boot
+build=yes
+
+# Extra setings
+extra=
+
+# Operating System to boot (ubuntu)
+os=
+
+release=24.04.1
+
+# run the image with QEMU
+run=
+
+# run QEMU without a display (U-Boot must be set to stdout=serial)
+serial=
+
+# Use kvm
+kvm=
+
+# Set ubdir to the build directory where you build U-Boot out-of-tree
+# We avoid in-tree build because it gets confusing trying different builds
+ubdir=${ubdir-/tmp/b}
+
+while getopts "a:Bko:rR:sw" opt; do
+	case "${opt}" in
+	a)
+		arch=$OPTARG
+		;;
+	B)
+		build=
+		;;
+	k)
+		kvm="-enable-kvm"
+		;;
+	o)
+		os=$OPTARG
+
+		# Expand memory and CPUs
+		extra+=" -m 4G -smp 4"
+		;;
+	r)
+		run=1
+		;;
+	R)
+		release=$OPTARG
+		;;
+	s)
+		serial=1
+		;;
+	w)
+		bitness=32
+		;;
+	*)
+		usage
+		;;
+	esac
+done
+
+# Build U-Boot for the selected board
+build_u_boot() {
+	buildman -w -o $DIR --board $BOARD -I || exit $?
+}
+
+# Run QEMU with U-Boot
+run_qemu() {
+	if [[ -n "${os_image}" ]]; then
+		extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
+	fi
+	if [[ -n "${serial}" ]]; then
+		extra+=" -display none -serial mon:stdio"
+	else
+		extra+=" -serial mon:stdio"
+	fi
+	echo "Running ${qemu} ${extra}"
+	"${qemu}" -bios "$DIR/${BIOS}" \
+		-m 512 \
+		-nic none \
+		${kvm} \
+		${extra}
+}
+
+# Check architecture
+case "${arch}" in
+arm)
+	BOARD="qemu_arm"
+	BIOS="u-boot.bin"
+	qemu=qemu-system-arm
+	extra+=" -machine virt"
+	suffix="arm"
+	if [[ "${bitness}" == "64" ]]; then
+		BOARD="qemu_arm64"
+		qemu=qemu-system-aarch64
+		extra+=" -cpu cortex-a57"
+		suffix="arm64"
+	fi
+	;;
+x86)
+	BOARD="qemu-x86"
+	BIOS="u-boot.rom"
+	qemu=qemu-system-i386
+	suffix="i386"
+	if [[ "${bitness}" == "64" ]]; then
+		BOARD="qemu-x86_64"
+		qemu=qemu-system-x86_64
+		suffix="amd64"
+	fi
+	;;
+*)
+	usage "Unknown architecture '${arch}'"
+esac
+
+# Check OS
+case "${os}" in
+ubuntu)
+	os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
+	;;
+"")
+	;;
+*)
+	usage "Unknown OS '${os}'"
+esac
+
+DIR=${ubdir}/${BOARD}
+
+if [[ -n "${build}" ]]; then
+	build_u_boot
+fi
+
+if [[ -n "${run}" ]]; then
+	run_qemu
+fi
-- 
2.34.1

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Tue, Nov 12, 2024 at 06:58:54AM -0700, Simon Glass wrote:

> It is handy to be able to quickly build and boot a QEMU image for a
> particular architecture and distro.
> 
> Add a script for this purpose. It supports only arm and x86 at present.
> For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> supported.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>

I don't know that this is an appropriate thing to merge. It's missing
all of the other architectures and platforms we support via QEMU and run
in CI all of the time. I feel it would be better to document how to
"read" the combination of CI yaml files and u-boot-test-hooks to see the
common cases of "fire up QEMU with U-Boot" instead of adding yet another
tool to support.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Heinrich Schuchardt]

Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
>It is handy to be able to quickly build and boot a QEMU image for a
>particular architecture and distro.
>
>Add a script for this purpose. It supports only arm and x86 at present.
>For distros it only supports Ubuntu. Both 32- and 64-bit builds are
>supported.
>
>Signed-off-by: Simon Glass <sjg@chromium.org>
>---
>
> MAINTAINERS                    |   8 ++
> doc/board/emulation/index.rst  |   1 +
> doc/board/emulation/script.rst |  61 ++++++++++++
> scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> 4 files changed, 245 insertions(+)
> create mode 100644 doc/board/emulation/script.rst
> create mode 100755 scripts/build-qemu.sh
>
>diff --git a/MAINTAINERS b/MAINTAINERS
>index 0399ed1dbf6..b45bb96d5a5 100644
>--- a/MAINTAINERS
>+++ b/MAINTAINERS
>@@ -1110,6 +1110,14 @@ F:	tools/efivar.py
> F:	tools/file2include.c
> F:	tools/mkeficapsule.c
> 
>+EMULATION
>+M:	Simon Glass <sjg@chromium.org>
>+S:	Maintained
>+W:	https://docs.u-boot.org/en/latest/board/emulation/script.html
>+F:	configs/qemu_x86*
>+F:	doc/board/emulation/script.rst
>+F:	scripts/build-qemu.sh

Please, avoid misnomers. This script does not build QEMU.

>+
> ENVIRONMENT
> M:	Joe Hershberger <joe.hershberger@ni.com>
> S:	Maintained
>diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
>index f8908166276..5a2a00ae225 100644
>--- a/doc/board/emulation/index.rst
>+++ b/doc/board/emulation/index.rst
>@@ -8,6 +8,7 @@ Emulation
> 
>    acpi
>    blkdev
>+   script
>    qemu-arm
>    qemu-mips
>    qemu-ppce500
>diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst

Just another misnomer. This page is not about script.sh.

>new file mode 100644
>index 00000000000..23981e333cb
>--- /dev/null
>+++ b/doc/board/emulation/script.rst
>@@ -0,0 +1,61 @@
>+.. SPDX-License-Identifier: GPL-2.0+

This is not a valid SPDX identifier.

>+
>+Script for building and running
>+===============================
>+
>+You may find the script `scripts/build-qemu.sh` helpful for building and testing
>+U-Boot on QEMU.
>+
>+If uses a environment variables to control how it works:
>+
>+ubdir
>+    base directory for building U-Boot, with each board being in its own
>+    subdirectory
>+
>+imagedir
>+    directory containing OS images, containin a subdirectory for each distro
>+    type (e.g. ubuntu/
>+
>+Once configured, you can build and run QEMU for arm64 like this::

This downloads the QEMU source and builds it?

>+
>+    scripts/build-qemu.sh -rsw
>+
>+No support is currently included for specifying a root disk, so this script can
>+only be used to start installers.
>+
>+Options
>+~~~~~~~
>+
>+Options are available to control the script:
>+
>+-a <arch>
>+    Select architecture (default arm, x86)
>+
>+-B
>+    Don't build; assume a build exists
>+
>+-k
>+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
>+    emulator
>+
>+-o <os>
>+    Run an Operating System. For now this only supports 'ubuntu'. The name of
>+    the OS file must remain unchanged from its standard name on the Ubuntu
>+    website.

The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?

Use the URL of the image as argument.

>+
>+-r
>+    Run QEMU with the image (by default this is not done)
>+
>+-R
>+    Select OS release (e.g. 24.04).
>+
>+-s
>+    Use serial only (no display)
>+
>+-w
>+    Use word version (32-bit). By default, 64-bit is used

"word version" is not helpful as explanation.

Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.

>+
>+.. note::
>+
>+    Note: For now this is a shell script, but if it expands it might be better
>+    as Python, accepting the slower startup.
>diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
>new file mode 100755
>index 00000000000..0ff53593cf9
>--- /dev/null
>+++ b/scripts/build-qemu.sh
>@@ -0,0 +1,175 @@
>+#!/bin/bash
>+# SPDX-License-Identifier: GPL-2.0+

This is not a valid SPDX identifier.

>+#
>+# Script to build U-Boot suitable for booting with QEMU, possibly running
>+# it, possibly with an OS image
>+
>+# This just an example. It assumes that
>+
>+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
>+# - your OS images are in ${imagedir}/{distroname}/...
>+
>+# So far the script supports only ARM and x86.

Why support obsolete i386 but not riscv64?

>+
>+set -e
>+
>+usage() {
>+	(
>+	if [[ -n "$1" ]]; then
>+		echo "$1"
>+		echo
>+	fi
>+	echo "Usage: $0 -aBkrsw"
>+	echo
>+	echo "   -a   - Select architecture (arm, x86)"
>+	echo "   -B   - Don't build; assume a build exists"
>+	echo "   -k   - Use kvm (kernel-based Virtual Machine)"
>+	echo "   -o   - Run Operating System ('ubuntu' only for now)"
>+	echo "   -r   - Run QEMU with the image"
>+	echo "   -R   - Select OS release (e.g. 24.04)"
>+	echo "   -s   - Use serial only (no display)"
>+	echo "   -w   - Use word version (32-bit)" ) >&2
>+	exit 1
>+}
>+
>+# Directory tree for OS images
>+imagedir=${imagedir-/vid/software/linux}
>+
>+# architecture (arm or x86)
>+arch=arm
>+
>+# 32- or 64-bit build
>+bitness=64
>+
>+# Build U-Boot
>+build=yes
>+
>+# Extra setings
>+extra=
>+
>+# Operating System to boot (ubuntu)
>+os=
>+
>+release=24.04.1
>+
>+# run the image with QEMU
>+run=
>+
>+# run QEMU without a display (U-Boot must be set to stdout=serial)
>+serial=
>+
>+# Use kvm
>+kvm=
>+
>+# Set ubdir to the build directory where you build U-Boot out-of-tree
>+# We avoid in-tree build because it gets confusing trying different builds
>+ubdir=${ubdir-/tmp/b}
>+
>+while getopts "a:Bko:rR:sw" opt; do
>+	case "${opt}" in
>+	a)
>+		arch=$OPTARG
>+		;;
>+	B)
>+		build=
>+		;;
>+	k)
>+		kvm="-enable-kvm"
>+		;;
>+	o)
>+		os=$OPTARG
>+
>+		# Expand memory and CPUs
>+		extra+=" -m 4G -smp 4"
>+		;;
>+	r)
>+		run=1
>+		;;
>+	R)
>+		release=$OPTARG
>+		;;
>+	s)
>+		serial=1
>+		;;
>+	w)
>+		bitness=32
>+		;;
>+	*)
>+		usage
>+		;;
>+	esac
>+done
>+
>+# Build U-Boot for the selected board
>+build_u_boot() {
>+	buildman -w -o $DIR --board $BOARD -I || exit $?
>+}
>+
>+# Run QEMU with U-Boot
>+run_qemu() {
>+	if [[ -n "${os_image}" ]]; then
>+		extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
>+	fi
>+	if [[ -n "${serial}" ]]; then
>+		extra+=" -display none -serial mon:stdio"
>+	else
>+		extra+=" -serial mon:stdio"
>+	fi
>+	echo "Running ${qemu} ${extra}"
>+	"${qemu}" -bios "$DIR/${BIOS}" \
>+		-m 512 \

Ubuntu suggests 4 GiB as minimum for a desktop.

>+		-nic none \

Who wants to run without network?
Use the virtio nic.

>+		${kvm} \
>+		${extra}
>+}
>+
>+# Check architecture
>+case "${arch}" in
>+arm)
>+	BOARD="qemu_arm"
>+	BIOS="u-boot.bin"
>+	qemu=qemu-system-arm
>+	extra+=" -machine virt"
>+	suffix="arm"
>+	if [[ "${bitness}" == "64" ]]; then
>+		BOARD="qemu_arm64"
>+		qemu=qemu-system-aarch64
>+		extra+=" -cpu cortex-a57"

That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.

-cpu max works fine with both tcg and kvm.

>+		suffix="arm64"

>+	fi
>+	;;
>+x86)
>+	BOARD="qemu-x86"
>+	BIOS="u-boot.rom"
>+	qemu=qemu-system-i386
>+	suffix="i386"
>+	if [[ "${bitness}" == "64" ]]; then
>+		BOARD="qemu-x86_64"
>+		qemu=qemu-system-x86_64
>+		suffix="amd64"
>+	fi
>+	;;
>+*)
>+	usage "Unknown architecture '${arch}'"
>+esac
>+
>+# Check OS
>+case "${os}" in
>+ubuntu)
>+	os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"

There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.

Running a foreign architecture desktop with tcg is not enjoyable.

For testing U-Boot a server image is all it takes.

Best regards

Heinrich


>+	;;
>+"")
>+	;;
>+*)
>+	usage "Unknown OS '${os}'"
>+esac
>+
>+DIR=${ubdir}/${BOARD}
>+
>+if [[ -n "${build}" ]]; then
>+	build_u_boot
>+fi
>+
>+if [[ -n "${run}" ]]; then
>+	run_qemu
>+fi

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 9399 bytes --]

On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> >It is handy to be able to quickly build and boot a QEMU image for a
> >particular architecture and distro.
> >
> >Add a script for this purpose. It supports only arm and x86 at present.
> >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> >supported.
> >
> >Signed-off-by: Simon Glass <sjg@chromium.org>
> >---
> >
> > MAINTAINERS                    |   8 ++
> > doc/board/emulation/index.rst  |   1 +
> > doc/board/emulation/script.rst |  61 ++++++++++++
> > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > 4 files changed, 245 insertions(+)
> > create mode 100644 doc/board/emulation/script.rst
> > create mode 100755 scripts/build-qemu.sh
> >
> >diff --git a/MAINTAINERS b/MAINTAINERS
> >index 0399ed1dbf6..b45bb96d5a5 100644
> >--- a/MAINTAINERS
> >+++ b/MAINTAINERS
> >@@ -1110,6 +1110,14 @@ F:	tools/efivar.py
> > F:	tools/file2include.c
> > F:	tools/mkeficapsule.c
> > 
> >+EMULATION
> >+M:	Simon Glass <sjg@chromium.org>
> >+S:	Maintained
> >+W:	https://docs.u-boot.org/en/latest/board/emulation/script.html
> >+F:	configs/qemu_x86*
> >+F:	doc/board/emulation/script.rst
> >+F:	scripts/build-qemu.sh
> 
> Please, avoid misnomers. This script does not build QEMU.
> 
> >+
> > ENVIRONMENT
> > M:	Joe Hershberger <joe.hershberger@ni.com>
> > S:	Maintained
> >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> >index f8908166276..5a2a00ae225 100644
> >--- a/doc/board/emulation/index.rst
> >+++ b/doc/board/emulation/index.rst
> >@@ -8,6 +8,7 @@ Emulation
> > 
> >    acpi
> >    blkdev
> >+   script
> >    qemu-arm
> >    qemu-mips
> >    qemu-ppce500
> >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> 
> Just another misnomer. This page is not about script.sh.
> 
> >new file mode 100644
> >index 00000000000..23981e333cb
> >--- /dev/null
> >+++ b/doc/board/emulation/script.rst
> >@@ -0,0 +1,61 @@
> >+.. SPDX-License-Identifier: GPL-2.0+
> 
> This is not a valid SPDX identifier.
> 
> >+
> >+Script for building and running
> >+===============================
> >+
> >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> >+U-Boot on QEMU.
> >+
> >+If uses a environment variables to control how it works:
> >+
> >+ubdir
> >+    base directory for building U-Boot, with each board being in its own
> >+    subdirectory
> >+
> >+imagedir
> >+    directory containing OS images, containin a subdirectory for each distro
> >+    type (e.g. ubuntu/
> >+
> >+Once configured, you can build and run QEMU for arm64 like this::
> 
> This downloads the QEMU source and builds it?
> 
> >+
> >+    scripts/build-qemu.sh -rsw
> >+
> >+No support is currently included for specifying a root disk, so this script can
> >+only be used to start installers.
> >+
> >+Options
> >+~~~~~~~
> >+
> >+Options are available to control the script:
> >+
> >+-a <arch>
> >+    Select architecture (default arm, x86)
> >+
> >+-B
> >+    Don't build; assume a build exists
> >+
> >+-k
> >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> >+    emulator
> >+
> >+-o <os>
> >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> >+    the OS file must remain unchanged from its standard name on the Ubuntu
> >+    website.
> 
> The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> 
> Use the URL of the image as argument.
> 
> >+
> >+-r
> >+    Run QEMU with the image (by default this is not done)
> >+
> >+-R
> >+    Select OS release (e.g. 24.04).
> >+
> >+-s
> >+    Use serial only (no display)
> >+
> >+-w
> >+    Use word version (32-bit). By default, 64-bit is used
> 
> "word version" is not helpful as explanation.
> 
> Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> 
> >+
> >+.. note::
> >+
> >+    Note: For now this is a shell script, but if it expands it might be better
> >+    as Python, accepting the slower startup.
> >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> >new file mode 100755
> >index 00000000000..0ff53593cf9
> >--- /dev/null
> >+++ b/scripts/build-qemu.sh
> >@@ -0,0 +1,175 @@
> >+#!/bin/bash
> >+# SPDX-License-Identifier: GPL-2.0+
> 
> This is not a valid SPDX identifier.
> 
> >+#
> >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> >+# it, possibly with an OS image
> >+
> >+# This just an example. It assumes that
> >+
> >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> >+# - your OS images are in ${imagedir}/{distroname}/...
> >+
> >+# So far the script supports only ARM and x86.
> 
> Why support obsolete i386 but not riscv64?
> 
> >+
> >+set -e
> >+
> >+usage() {
> >+	(
> >+	if [[ -n "$1" ]]; then
> >+		echo "$1"
> >+		echo
> >+	fi
> >+	echo "Usage: $0 -aBkrsw"
> >+	echo
> >+	echo "   -a   - Select architecture (arm, x86)"
> >+	echo "   -B   - Don't build; assume a build exists"
> >+	echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> >+	echo "   -o   - Run Operating System ('ubuntu' only for now)"
> >+	echo "   -r   - Run QEMU with the image"
> >+	echo "   -R   - Select OS release (e.g. 24.04)"
> >+	echo "   -s   - Use serial only (no display)"
> >+	echo "   -w   - Use word version (32-bit)" ) >&2
> >+	exit 1
> >+}
> >+
> >+# Directory tree for OS images
> >+imagedir=${imagedir-/vid/software/linux}
> >+
> >+# architecture (arm or x86)
> >+arch=arm
> >+
> >+# 32- or 64-bit build
> >+bitness=64
> >+
> >+# Build U-Boot
> >+build=yes
> >+
> >+# Extra setings
> >+extra=
> >+
> >+# Operating System to boot (ubuntu)
> >+os=
> >+
> >+release=24.04.1
> >+
> >+# run the image with QEMU
> >+run=
> >+
> >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> >+serial=
> >+
> >+# Use kvm
> >+kvm=
> >+
> >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> >+# We avoid in-tree build because it gets confusing trying different builds
> >+ubdir=${ubdir-/tmp/b}
> >+
> >+while getopts "a:Bko:rR:sw" opt; do
> >+	case "${opt}" in
> >+	a)
> >+		arch=$OPTARG
> >+		;;
> >+	B)
> >+		build=
> >+		;;
> >+	k)
> >+		kvm="-enable-kvm"
> >+		;;
> >+	o)
> >+		os=$OPTARG
> >+
> >+		# Expand memory and CPUs
> >+		extra+=" -m 4G -smp 4"
> >+		;;
> >+	r)
> >+		run=1
> >+		;;
> >+	R)
> >+		release=$OPTARG
> >+		;;
> >+	s)
> >+		serial=1
> >+		;;
> >+	w)
> >+		bitness=32
> >+		;;
> >+	*)
> >+		usage
> >+		;;
> >+	esac
> >+done
> >+
> >+# Build U-Boot for the selected board
> >+build_u_boot() {
> >+	buildman -w -o $DIR --board $BOARD -I || exit $?
> >+}
> >+
> >+# Run QEMU with U-Boot
> >+run_qemu() {
> >+	if [[ -n "${os_image}" ]]; then
> >+		extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> >+	fi
> >+	if [[ -n "${serial}" ]]; then
> >+		extra+=" -display none -serial mon:stdio"
> >+	else
> >+		extra+=" -serial mon:stdio"
> >+	fi
> >+	echo "Running ${qemu} ${extra}"
> >+	"${qemu}" -bios "$DIR/${BIOS}" \
> >+		-m 512 \
> 
> Ubuntu suggests 4 GiB as minimum for a desktop.
> 
> >+		-nic none \
> 
> Who wants to run without network?
> Use the virtio nic.
> 
> >+		${kvm} \
> >+		${extra}
> >+}
> >+
> >+# Check architecture
> >+case "${arch}" in
> >+arm)
> >+	BOARD="qemu_arm"
> >+	BIOS="u-boot.bin"
> >+	qemu=qemu-system-arm
> >+	extra+=" -machine virt"
> >+	suffix="arm"
> >+	if [[ "${bitness}" == "64" ]]; then
> >+		BOARD="qemu_arm64"
> >+		qemu=qemu-system-aarch64
> >+		extra+=" -cpu cortex-a57"
> 
> That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> 
> -cpu max works fine with both tcg and kvm.
> 
> >+		suffix="arm64"
> 
> >+	fi
> >+	;;
> >+x86)
> >+	BOARD="qemu-x86"
> >+	BIOS="u-boot.rom"
> >+	qemu=qemu-system-i386
> >+	suffix="i386"
> >+	if [[ "${bitness}" == "64" ]]; then
> >+		BOARD="qemu-x86_64"
> >+		qemu=qemu-system-x86_64
> >+		suffix="amd64"
> >+	fi
> >+	;;
> >+*)
> >+	usage "Unknown architecture '${arch}'"
> >+esac
> >+
> >+# Check OS
> >+case "${os}" in
> >+ubuntu)
> >+	os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> 
> There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> 
> Running a foreign architecture desktop with tcg is not enjoyable.
> 
> For testing U-Boot a server image is all it takes.

And _all_ of this is why I don't want to add a useful personal script
as an additional tool we support. I've seen how much work goes in to the
OpenEmbedded runqemu script, we don't have the spare cycles for
something like that. Doubly so when ultimately I believe we would be
well served by having a document that says (in much more words) to look
at u-boot-test-hooks for how to invoke QEMU for a large number of
architectures and platforms and to then further leverage general QEMU
tips and guides on how to run an OS of your choice with that.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Simon Glass]

Hi Tom,

On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > >It is handy to be able to quickly build and boot a QEMU image for a
> > >particular architecture and distro.
> > >
> > >Add a script for this purpose. It supports only arm and x86 at present.
> > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > >supported.
> > >
> > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > >---
> > >
> > > MAINTAINERS                    |   8 ++
> > > doc/board/emulation/index.rst  |   1 +
> > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > 4 files changed, 245 insertions(+)
> > > create mode 100644 doc/board/emulation/script.rst
> > > create mode 100755 scripts/build-qemu.sh
> > >
> > >diff --git a/MAINTAINERS b/MAINTAINERS
> > >index 0399ed1dbf6..b45bb96d5a5 100644
> > >--- a/MAINTAINERS
> > >+++ b/MAINTAINERS
> > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > F:  tools/file2include.c
> > > F:  tools/mkeficapsule.c
> > >
> > >+EMULATION
> > >+M:  Simon Glass <sjg@chromium.org>
> > >+S:  Maintained
> > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > >+F:  configs/qemu_x86*
> > >+F:  doc/board/emulation/script.rst
> > >+F:  scripts/build-qemu.sh
> >
> > Please, avoid misnomers. This script does not build QEMU.
> >
> > >+
> > > ENVIRONMENT
> > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > S:  Maintained
> > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > >index f8908166276..5a2a00ae225 100644
> > >--- a/doc/board/emulation/index.rst
> > >+++ b/doc/board/emulation/index.rst
> > >@@ -8,6 +8,7 @@ Emulation
> > >
> > >    acpi
> > >    blkdev
> > >+   script
> > >    qemu-arm
> > >    qemu-mips
> > >    qemu-ppce500
> > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> >
> > Just another misnomer. This page is not about script.sh.
> >
> > >new file mode 100644
> > >index 00000000000..23981e333cb
> > >--- /dev/null
> > >+++ b/doc/board/emulation/script.rst
> > >@@ -0,0 +1,61 @@
> > >+.. SPDX-License-Identifier: GPL-2.0+
> >
> > This is not a valid SPDX identifier.
> >
> > >+
> > >+Script for building and running
> > >+===============================
> > >+
> > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > >+U-Boot on QEMU.
> > >+
> > >+If uses a environment variables to control how it works:
> > >+
> > >+ubdir
> > >+    base directory for building U-Boot, with each board being in its own
> > >+    subdirectory
> > >+
> > >+imagedir
> > >+    directory containing OS images, containin a subdirectory for each distro
> > >+    type (e.g. ubuntu/
> > >+
> > >+Once configured, you can build and run QEMU for arm64 like this::
> >
> > This downloads the QEMU source and builds it?
> >
> > >+
> > >+    scripts/build-qemu.sh -rsw
> > >+
> > >+No support is currently included for specifying a root disk, so this script can
> > >+only be used to start installers.
> > >+
> > >+Options
> > >+~~~~~~~
> > >+
> > >+Options are available to control the script:
> > >+
> > >+-a <arch>
> > >+    Select architecture (default arm, x86)
> > >+
> > >+-B
> > >+    Don't build; assume a build exists
> > >+
> > >+-k
> > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > >+    emulator
> > >+
> > >+-o <os>
> > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > >+    website.
> >
> > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> >
> > Use the URL of the image as argument.
> >
> > >+
> > >+-r
> > >+    Run QEMU with the image (by default this is not done)
> > >+
> > >+-R
> > >+    Select OS release (e.g. 24.04).
> > >+
> > >+-s
> > >+    Use serial only (no display)
> > >+
> > >+-w
> > >+    Use word version (32-bit). By default, 64-bit is used
> >
> > "word version" is not helpful as explanation.
> >
> > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> >
> > >+
> > >+.. note::
> > >+
> > >+    Note: For now this is a shell script, but if it expands it might be better
> > >+    as Python, accepting the slower startup.
> > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > >new file mode 100755
> > >index 00000000000..0ff53593cf9
> > >--- /dev/null
> > >+++ b/scripts/build-qemu.sh
> > >@@ -0,0 +1,175 @@
> > >+#!/bin/bash
> > >+# SPDX-License-Identifier: GPL-2.0+
> >
> > This is not a valid SPDX identifier.
> >
> > >+#
> > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > >+# it, possibly with an OS image
> > >+
> > >+# This just an example. It assumes that
> > >+
> > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > >+# - your OS images are in ${imagedir}/{distroname}/...
> > >+
> > >+# So far the script supports only ARM and x86.
> >
> > Why support obsolete i386 but not riscv64?
> >
> > >+
> > >+set -e
> > >+
> > >+usage() {
> > >+    (
> > >+    if [[ -n "$1" ]]; then
> > >+            echo "$1"
> > >+            echo
> > >+    fi
> > >+    echo "Usage: $0 -aBkrsw"
> > >+    echo
> > >+    echo "   -a   - Select architecture (arm, x86)"
> > >+    echo "   -B   - Don't build; assume a build exists"
> > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > >+    echo "   -r   - Run QEMU with the image"
> > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > >+    echo "   -s   - Use serial only (no display)"
> > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > >+    exit 1
> > >+}
> > >+
> > >+# Directory tree for OS images
> > >+imagedir=${imagedir-/vid/software/linux}
> > >+
> > >+# architecture (arm or x86)
> > >+arch=arm
> > >+
> > >+# 32- or 64-bit build
> > >+bitness=64
> > >+
> > >+# Build U-Boot
> > >+build=yes
> > >+
> > >+# Extra setings
> > >+extra=
> > >+
> > >+# Operating System to boot (ubuntu)
> > >+os=
> > >+
> > >+release=24.04.1
> > >+
> > >+# run the image with QEMU
> > >+run=
> > >+
> > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > >+serial=
> > >+
> > >+# Use kvm
> > >+kvm=
> > >+
> > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > >+# We avoid in-tree build because it gets confusing trying different builds
> > >+ubdir=${ubdir-/tmp/b}
> > >+
> > >+while getopts "a:Bko:rR:sw" opt; do
> > >+    case "${opt}" in
> > >+    a)
> > >+            arch=$OPTARG
> > >+            ;;
> > >+    B)
> > >+            build=
> > >+            ;;
> > >+    k)
> > >+            kvm="-enable-kvm"
> > >+            ;;
> > >+    o)
> > >+            os=$OPTARG
> > >+
> > >+            # Expand memory and CPUs
> > >+            extra+=" -m 4G -smp 4"
> > >+            ;;
> > >+    r)
> > >+            run=1
> > >+            ;;
> > >+    R)
> > >+            release=$OPTARG
> > >+            ;;
> > >+    s)
> > >+            serial=1
> > >+            ;;
> > >+    w)
> > >+            bitness=32
> > >+            ;;
> > >+    *)
> > >+            usage
> > >+            ;;
> > >+    esac
> > >+done
> > >+
> > >+# Build U-Boot for the selected board
> > >+build_u_boot() {
> > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > >+}
> > >+
> > >+# Run QEMU with U-Boot
> > >+run_qemu() {
> > >+    if [[ -n "${os_image}" ]]; then
> > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > >+    fi
> > >+    if [[ -n "${serial}" ]]; then
> > >+            extra+=" -display none -serial mon:stdio"
> > >+    else
> > >+            extra+=" -serial mon:stdio"
> > >+    fi
> > >+    echo "Running ${qemu} ${extra}"
> > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > >+            -m 512 \
> >
> > Ubuntu suggests 4 GiB as minimum for a desktop.
> >
> > >+            -nic none \
> >
> > Who wants to run without network?
> > Use the virtio nic.
> >
> > >+            ${kvm} \
> > >+            ${extra}
> > >+}
> > >+
> > >+# Check architecture
> > >+case "${arch}" in
> > >+arm)
> > >+    BOARD="qemu_arm"
> > >+    BIOS="u-boot.bin"
> > >+    qemu=qemu-system-arm
> > >+    extra+=" -machine virt"
> > >+    suffix="arm"
> > >+    if [[ "${bitness}" == "64" ]]; then
> > >+            BOARD="qemu_arm64"
> > >+            qemu=qemu-system-aarch64
> > >+            extra+=" -cpu cortex-a57"
> >
> > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> >
> > -cpu max works fine with both tcg and kvm.
> >
> > >+            suffix="arm64"
> >
> > >+    fi
> > >+    ;;
> > >+x86)
> > >+    BOARD="qemu-x86"
> > >+    BIOS="u-boot.rom"
> > >+    qemu=qemu-system-i386
> > >+    suffix="i386"
> > >+    if [[ "${bitness}" == "64" ]]; then
> > >+            BOARD="qemu-x86_64"
> > >+            qemu=qemu-system-x86_64
> > >+            suffix="amd64"
> > >+    fi
> > >+    ;;
> > >+*)
> > >+    usage "Unknown architecture '${arch}'"
> > >+esac
> > >+
> > >+# Check OS
> > >+case "${os}" in
> > >+ubuntu)
> > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> >
> > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> >
> > Running a foreign architecture desktop with tcg is not enjoyable.
> >
> > For testing U-Boot a server image is all it takes.
>
> And _all_ of this is why I don't want to add a useful personal script
> as an additional tool we support. I've seen how much work goes in to the
> OpenEmbedded runqemu script, we don't have the spare cycles for
> something like that. Doubly so when ultimately I believe we would be
> well served by having a document that says (in much more words) to look
> at u-boot-test-hooks for how to invoke QEMU for a large number of
> architectures and platforms and to then further leverage general QEMU
> tips and guides on how to run an OS of your choice with that.

I'm not sure it matters that much. Everyone is going to have their
preference as to how this script should look, but no one else has
taken the time to write one...

People are free to send patches to enhance it. But I believe it is
helpful, e.g. for repeating problems caused by recent lmb patches.

Re the test hooks, I just get tired of looking them up and trying to
figure out what to do. Every board name and arch is slightly
different. Just a hassle that I don't need.

We can put it in scripts/contrib if you like.

Regards,
Simon

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 12191 bytes --]

On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > >particular architecture and distro.
> > > >
> > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > >supported.
> > > >
> > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > >---
> > > >
> > > > MAINTAINERS                    |   8 ++
> > > > doc/board/emulation/index.rst  |   1 +
> > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > 4 files changed, 245 insertions(+)
> > > > create mode 100644 doc/board/emulation/script.rst
> > > > create mode 100755 scripts/build-qemu.sh
> > > >
> > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > >--- a/MAINTAINERS
> > > >+++ b/MAINTAINERS
> > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > F:  tools/file2include.c
> > > > F:  tools/mkeficapsule.c
> > > >
> > > >+EMULATION
> > > >+M:  Simon Glass <sjg@chromium.org>
> > > >+S:  Maintained
> > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > >+F:  configs/qemu_x86*
> > > >+F:  doc/board/emulation/script.rst
> > > >+F:  scripts/build-qemu.sh
> > >
> > > Please, avoid misnomers. This script does not build QEMU.
> > >
> > > >+
> > > > ENVIRONMENT
> > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > S:  Maintained
> > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > >index f8908166276..5a2a00ae225 100644
> > > >--- a/doc/board/emulation/index.rst
> > > >+++ b/doc/board/emulation/index.rst
> > > >@@ -8,6 +8,7 @@ Emulation
> > > >
> > > >    acpi
> > > >    blkdev
> > > >+   script
> > > >    qemu-arm
> > > >    qemu-mips
> > > >    qemu-ppce500
> > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > >
> > > Just another misnomer. This page is not about script.sh.
> > >
> > > >new file mode 100644
> > > >index 00000000000..23981e333cb
> > > >--- /dev/null
> > > >+++ b/doc/board/emulation/script.rst
> > > >@@ -0,0 +1,61 @@
> > > >+.. SPDX-License-Identifier: GPL-2.0+
> > >
> > > This is not a valid SPDX identifier.
> > >
> > > >+
> > > >+Script for building and running
> > > >+===============================
> > > >+
> > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > >+U-Boot on QEMU.
> > > >+
> > > >+If uses a environment variables to control how it works:
> > > >+
> > > >+ubdir
> > > >+    base directory for building U-Boot, with each board being in its own
> > > >+    subdirectory
> > > >+
> > > >+imagedir
> > > >+    directory containing OS images, containin a subdirectory for each distro
> > > >+    type (e.g. ubuntu/
> > > >+
> > > >+Once configured, you can build and run QEMU for arm64 like this::
> > >
> > > This downloads the QEMU source and builds it?
> > >
> > > >+
> > > >+    scripts/build-qemu.sh -rsw
> > > >+
> > > >+No support is currently included for specifying a root disk, so this script can
> > > >+only be used to start installers.
> > > >+
> > > >+Options
> > > >+~~~~~~~
> > > >+
> > > >+Options are available to control the script:
> > > >+
> > > >+-a <arch>
> > > >+    Select architecture (default arm, x86)
> > > >+
> > > >+-B
> > > >+    Don't build; assume a build exists
> > > >+
> > > >+-k
> > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > >+    emulator
> > > >+
> > > >+-o <os>
> > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > >+    website.
> > >
> > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > >
> > > Use the URL of the image as argument.
> > >
> > > >+
> > > >+-r
> > > >+    Run QEMU with the image (by default this is not done)
> > > >+
> > > >+-R
> > > >+    Select OS release (e.g. 24.04).
> > > >+
> > > >+-s
> > > >+    Use serial only (no display)
> > > >+
> > > >+-w
> > > >+    Use word version (32-bit). By default, 64-bit is used
> > >
> > > "word version" is not helpful as explanation.
> > >
> > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > >
> > > >+
> > > >+.. note::
> > > >+
> > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > >+    as Python, accepting the slower startup.
> > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > >new file mode 100755
> > > >index 00000000000..0ff53593cf9
> > > >--- /dev/null
> > > >+++ b/scripts/build-qemu.sh
> > > >@@ -0,0 +1,175 @@
> > > >+#!/bin/bash
> > > >+# SPDX-License-Identifier: GPL-2.0+
> > >
> > > This is not a valid SPDX identifier.
> > >
> > > >+#
> > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > >+# it, possibly with an OS image
> > > >+
> > > >+# This just an example. It assumes that
> > > >+
> > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > >+
> > > >+# So far the script supports only ARM and x86.
> > >
> > > Why support obsolete i386 but not riscv64?
> > >
> > > >+
> > > >+set -e
> > > >+
> > > >+usage() {
> > > >+    (
> > > >+    if [[ -n "$1" ]]; then
> > > >+            echo "$1"
> > > >+            echo
> > > >+    fi
> > > >+    echo "Usage: $0 -aBkrsw"
> > > >+    echo
> > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > >+    echo "   -B   - Don't build; assume a build exists"
> > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > >+    echo "   -r   - Run QEMU with the image"
> > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > >+    echo "   -s   - Use serial only (no display)"
> > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > >+    exit 1
> > > >+}
> > > >+
> > > >+# Directory tree for OS images
> > > >+imagedir=${imagedir-/vid/software/linux}
> > > >+
> > > >+# architecture (arm or x86)
> > > >+arch=arm
> > > >+
> > > >+# 32- or 64-bit build
> > > >+bitness=64
> > > >+
> > > >+# Build U-Boot
> > > >+build=yes
> > > >+
> > > >+# Extra setings
> > > >+extra=
> > > >+
> > > >+# Operating System to boot (ubuntu)
> > > >+os=
> > > >+
> > > >+release=24.04.1
> > > >+
> > > >+# run the image with QEMU
> > > >+run=
> > > >+
> > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > >+serial=
> > > >+
> > > >+# Use kvm
> > > >+kvm=
> > > >+
> > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > >+ubdir=${ubdir-/tmp/b}
> > > >+
> > > >+while getopts "a:Bko:rR:sw" opt; do
> > > >+    case "${opt}" in
> > > >+    a)
> > > >+            arch=$OPTARG
> > > >+            ;;
> > > >+    B)
> > > >+            build=
> > > >+            ;;
> > > >+    k)
> > > >+            kvm="-enable-kvm"
> > > >+            ;;
> > > >+    o)
> > > >+            os=$OPTARG
> > > >+
> > > >+            # Expand memory and CPUs
> > > >+            extra+=" -m 4G -smp 4"
> > > >+            ;;
> > > >+    r)
> > > >+            run=1
> > > >+            ;;
> > > >+    R)
> > > >+            release=$OPTARG
> > > >+            ;;
> > > >+    s)
> > > >+            serial=1
> > > >+            ;;
> > > >+    w)
> > > >+            bitness=32
> > > >+            ;;
> > > >+    *)
> > > >+            usage
> > > >+            ;;
> > > >+    esac
> > > >+done
> > > >+
> > > >+# Build U-Boot for the selected board
> > > >+build_u_boot() {
> > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > >+}
> > > >+
> > > >+# Run QEMU with U-Boot
> > > >+run_qemu() {
> > > >+    if [[ -n "${os_image}" ]]; then
> > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > >+    fi
> > > >+    if [[ -n "${serial}" ]]; then
> > > >+            extra+=" -display none -serial mon:stdio"
> > > >+    else
> > > >+            extra+=" -serial mon:stdio"
> > > >+    fi
> > > >+    echo "Running ${qemu} ${extra}"
> > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > >+            -m 512 \
> > >
> > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > >
> > > >+            -nic none \
> > >
> > > Who wants to run without network?
> > > Use the virtio nic.
> > >
> > > >+            ${kvm} \
> > > >+            ${extra}
> > > >+}
> > > >+
> > > >+# Check architecture
> > > >+case "${arch}" in
> > > >+arm)
> > > >+    BOARD="qemu_arm"
> > > >+    BIOS="u-boot.bin"
> > > >+    qemu=qemu-system-arm
> > > >+    extra+=" -machine virt"
> > > >+    suffix="arm"
> > > >+    if [[ "${bitness}" == "64" ]]; then
> > > >+            BOARD="qemu_arm64"
> > > >+            qemu=qemu-system-aarch64
> > > >+            extra+=" -cpu cortex-a57"
> > >
> > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > >
> > > -cpu max works fine with both tcg and kvm.
> > >
> > > >+            suffix="arm64"
> > >
> > > >+    fi
> > > >+    ;;
> > > >+x86)
> > > >+    BOARD="qemu-x86"
> > > >+    BIOS="u-boot.rom"
> > > >+    qemu=qemu-system-i386
> > > >+    suffix="i386"
> > > >+    if [[ "${bitness}" == "64" ]]; then
> > > >+            BOARD="qemu-x86_64"
> > > >+            qemu=qemu-system-x86_64
> > > >+            suffix="amd64"
> > > >+    fi
> > > >+    ;;
> > > >+*)
> > > >+    usage "Unknown architecture '${arch}'"
> > > >+esac
> > > >+
> > > >+# Check OS
> > > >+case "${os}" in
> > > >+ubuntu)
> > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > >
> > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > >
> > > Running a foreign architecture desktop with tcg is not enjoyable.
> > >
> > > For testing U-Boot a server image is all it takes.
> >
> > And _all_ of this is why I don't want to add a useful personal script
> > as an additional tool we support. I've seen how much work goes in to the
> > OpenEmbedded runqemu script, we don't have the spare cycles for
> > something like that. Doubly so when ultimately I believe we would be
> > well served by having a document that says (in much more words) to look
> > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > architectures and platforms and to then further leverage general QEMU
> > tips and guides on how to run an OS of your choice with that.
> 
> I'm not sure it matters that much. Everyone is going to have their
> preference as to how this script should look, but no one else has
> taken the time to write one...
> 
> People are free to send patches to enhance it. But I believe it is
> helpful, e.g. for repeating problems caused by recent lmb patches.
> 
> Re the test hooks, I just get tired of looking them up and trying to
> figure out what to do. Every board name and arch is slightly
> different. Just a hassle that I don't need.
> 
> We can put it in scripts/contrib if you like.

It's handy to point people to scripts, yes. I frequently point people at
my wrappers around buildman for example for "how do I find code bloat?"
and similar. But no, I don't think this rises to the level of
"scripts/contrib".

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Simon Glass]

Hi Tom,

On Fri, 15 Nov 2024 at 08:14, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > >particular architecture and distro.
> > > > >
> > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > >supported.
> > > > >
> > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > >---
> > > > >
> > > > > MAINTAINERS                    |   8 ++
> > > > > doc/board/emulation/index.rst  |   1 +
> > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > 4 files changed, 245 insertions(+)
> > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > create mode 100755 scripts/build-qemu.sh
> > > > >
> > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > >--- a/MAINTAINERS
> > > > >+++ b/MAINTAINERS
> > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > F:  tools/file2include.c
> > > > > F:  tools/mkeficapsule.c
> > > > >
> > > > >+EMULATION
> > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > >+S:  Maintained
> > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > >+F:  configs/qemu_x86*
> > > > >+F:  doc/board/emulation/script.rst
> > > > >+F:  scripts/build-qemu.sh
> > > >
> > > > Please, avoid misnomers. This script does not build QEMU.
> > > >
> > > > >+
> > > > > ENVIRONMENT
> > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > S:  Maintained
> > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > >index f8908166276..5a2a00ae225 100644
> > > > >--- a/doc/board/emulation/index.rst
> > > > >+++ b/doc/board/emulation/index.rst
> > > > >@@ -8,6 +8,7 @@ Emulation
> > > > >
> > > > >    acpi
> > > > >    blkdev
> > > > >+   script
> > > > >    qemu-arm
> > > > >    qemu-mips
> > > > >    qemu-ppce500
> > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > >
> > > > Just another misnomer. This page is not about script.sh.
> > > >
> > > > >new file mode 100644
> > > > >index 00000000000..23981e333cb
> > > > >--- /dev/null
> > > > >+++ b/doc/board/emulation/script.rst
> > > > >@@ -0,0 +1,61 @@
> > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > >
> > > > This is not a valid SPDX identifier.
> > > >
> > > > >+
> > > > >+Script for building and running
> > > > >+===============================
> > > > >+
> > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > >+U-Boot on QEMU.
> > > > >+
> > > > >+If uses a environment variables to control how it works:
> > > > >+
> > > > >+ubdir
> > > > >+    base directory for building U-Boot, with each board being in its own
> > > > >+    subdirectory
> > > > >+
> > > > >+imagedir
> > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > >+    type (e.g. ubuntu/
> > > > >+
> > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > >
> > > > This downloads the QEMU source and builds it?
> > > >
> > > > >+
> > > > >+    scripts/build-qemu.sh -rsw
> > > > >+
> > > > >+No support is currently included for specifying a root disk, so this script can
> > > > >+only be used to start installers.
> > > > >+
> > > > >+Options
> > > > >+~~~~~~~
> > > > >+
> > > > >+Options are available to control the script:
> > > > >+
> > > > >+-a <arch>
> > > > >+    Select architecture (default arm, x86)
> > > > >+
> > > > >+-B
> > > > >+    Don't build; assume a build exists
> > > > >+
> > > > >+-k
> > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > >+    emulator
> > > > >+
> > > > >+-o <os>
> > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > >+    website.
> > > >
> > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > >
> > > > Use the URL of the image as argument.
> > > >
> > > > >+
> > > > >+-r
> > > > >+    Run QEMU with the image (by default this is not done)
> > > > >+
> > > > >+-R
> > > > >+    Select OS release (e.g. 24.04).
> > > > >+
> > > > >+-s
> > > > >+    Use serial only (no display)
> > > > >+
> > > > >+-w
> > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > >
> > > > "word version" is not helpful as explanation.
> > > >
> > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > >
> > > > >+
> > > > >+.. note::
> > > > >+
> > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > >+    as Python, accepting the slower startup.
> > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > >new file mode 100755
> > > > >index 00000000000..0ff53593cf9
> > > > >--- /dev/null
> > > > >+++ b/scripts/build-qemu.sh
> > > > >@@ -0,0 +1,175 @@
> > > > >+#!/bin/bash
> > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > >
> > > > This is not a valid SPDX identifier.
> > > >
> > > > >+#
> > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > >+# it, possibly with an OS image
> > > > >+
> > > > >+# This just an example. It assumes that
> > > > >+
> > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > >+
> > > > >+# So far the script supports only ARM and x86.
> > > >
> > > > Why support obsolete i386 but not riscv64?
> > > >
> > > > >+
> > > > >+set -e
> > > > >+
> > > > >+usage() {
> > > > >+    (
> > > > >+    if [[ -n "$1" ]]; then
> > > > >+            echo "$1"
> > > > >+            echo
> > > > >+    fi
> > > > >+    echo "Usage: $0 -aBkrsw"
> > > > >+    echo
> > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > >+    echo "   -r   - Run QEMU with the image"
> > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > >+    echo "   -s   - Use serial only (no display)"
> > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > >+    exit 1
> > > > >+}
> > > > >+
> > > > >+# Directory tree for OS images
> > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > >+
> > > > >+# architecture (arm or x86)
> > > > >+arch=arm
> > > > >+
> > > > >+# 32- or 64-bit build
> > > > >+bitness=64
> > > > >+
> > > > >+# Build U-Boot
> > > > >+build=yes
> > > > >+
> > > > >+# Extra setings
> > > > >+extra=
> > > > >+
> > > > >+# Operating System to boot (ubuntu)
> > > > >+os=
> > > > >+
> > > > >+release=24.04.1
> > > > >+
> > > > >+# run the image with QEMU
> > > > >+run=
> > > > >+
> > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > >+serial=
> > > > >+
> > > > >+# Use kvm
> > > > >+kvm=
> > > > >+
> > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > >+ubdir=${ubdir-/tmp/b}
> > > > >+
> > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > >+    case "${opt}" in
> > > > >+    a)
> > > > >+            arch=$OPTARG
> > > > >+            ;;
> > > > >+    B)
> > > > >+            build=
> > > > >+            ;;
> > > > >+    k)
> > > > >+            kvm="-enable-kvm"
> > > > >+            ;;
> > > > >+    o)
> > > > >+            os=$OPTARG
> > > > >+
> > > > >+            # Expand memory and CPUs
> > > > >+            extra+=" -m 4G -smp 4"
> > > > >+            ;;
> > > > >+    r)
> > > > >+            run=1
> > > > >+            ;;
> > > > >+    R)
> > > > >+            release=$OPTARG
> > > > >+            ;;
> > > > >+    s)
> > > > >+            serial=1
> > > > >+            ;;
> > > > >+    w)
> > > > >+            bitness=32
> > > > >+            ;;
> > > > >+    *)
> > > > >+            usage
> > > > >+            ;;
> > > > >+    esac
> > > > >+done
> > > > >+
> > > > >+# Build U-Boot for the selected board
> > > > >+build_u_boot() {
> > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > >+}
> > > > >+
> > > > >+# Run QEMU with U-Boot
> > > > >+run_qemu() {
> > > > >+    if [[ -n "${os_image}" ]]; then
> > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > >+    fi
> > > > >+    if [[ -n "${serial}" ]]; then
> > > > >+            extra+=" -display none -serial mon:stdio"
> > > > >+    else
> > > > >+            extra+=" -serial mon:stdio"
> > > > >+    fi
> > > > >+    echo "Running ${qemu} ${extra}"
> > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > >+            -m 512 \
> > > >
> > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > >
> > > > >+            -nic none \
> > > >
> > > > Who wants to run without network?
> > > > Use the virtio nic.
> > > >
> > > > >+            ${kvm} \
> > > > >+            ${extra}
> > > > >+}
> > > > >+
> > > > >+# Check architecture
> > > > >+case "${arch}" in
> > > > >+arm)
> > > > >+    BOARD="qemu_arm"
> > > > >+    BIOS="u-boot.bin"
> > > > >+    qemu=qemu-system-arm
> > > > >+    extra+=" -machine virt"
> > > > >+    suffix="arm"
> > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > >+            BOARD="qemu_arm64"
> > > > >+            qemu=qemu-system-aarch64
> > > > >+            extra+=" -cpu cortex-a57"
> > > >
> > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > >
> > > > -cpu max works fine with both tcg and kvm.
> > > >
> > > > >+            suffix="arm64"
> > > >
> > > > >+    fi
> > > > >+    ;;
> > > > >+x86)
> > > > >+    BOARD="qemu-x86"
> > > > >+    BIOS="u-boot.rom"
> > > > >+    qemu=qemu-system-i386
> > > > >+    suffix="i386"
> > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > >+            BOARD="qemu-x86_64"
> > > > >+            qemu=qemu-system-x86_64
> > > > >+            suffix="amd64"
> > > > >+    fi
> > > > >+    ;;
> > > > >+*)
> > > > >+    usage "Unknown architecture '${arch}'"
> > > > >+esac
> > > > >+
> > > > >+# Check OS
> > > > >+case "${os}" in
> > > > >+ubuntu)
> > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > >
> > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > >
> > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > >
> > > > For testing U-Boot a server image is all it takes.
> > >
> > > And _all_ of this is why I don't want to add a useful personal script
> > > as an additional tool we support. I've seen how much work goes in to the
> > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > something like that. Doubly so when ultimately I believe we would be
> > > well served by having a document that says (in much more words) to look
> > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > architectures and platforms and to then further leverage general QEMU
> > > tips and guides on how to run an OS of your choice with that.
> >
> > I'm not sure it matters that much. Everyone is going to have their
> > preference as to how this script should look, but no one else has
> > taken the time to write one...
> >
> > People are free to send patches to enhance it. But I believe it is
> > helpful, e.g. for repeating problems caused by recent lmb patches.
> >
> > Re the test hooks, I just get tired of looking them up and trying to
> > figure out what to do. Every board name and arch is slightly
> > different. Just a hassle that I don't need.
> >
> > We can put it in scripts/contrib if you like.
>
> It's handy to point people to scripts, yes. I frequently point people at
> my wrappers around buildman for example for "how do I find code bloat?"
> and similar. But no, I don't think this rises to the level of
> "scripts/contrib".

I find this useful and it makes it easier to report problems with
booting if we are using the same tool. I plan to add RISC-V at some
point.

Regards,
Simon

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 13277 bytes --]

On Fri, Nov 15, 2024 at 09:14:03AM -0600, Tom Rini wrote:
> On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > Hi Tom,
> > 
> > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > >particular architecture and distro.
> > > > >
> > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > >supported.
> > > > >
> > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > >---
> > > > >
> > > > > MAINTAINERS                    |   8 ++
> > > > > doc/board/emulation/index.rst  |   1 +
> > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > 4 files changed, 245 insertions(+)
> > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > create mode 100755 scripts/build-qemu.sh
> > > > >
> > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > >--- a/MAINTAINERS
> > > > >+++ b/MAINTAINERS
> > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > F:  tools/file2include.c
> > > > > F:  tools/mkeficapsule.c
> > > > >
> > > > >+EMULATION
> > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > >+S:  Maintained
> > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > >+F:  configs/qemu_x86*
> > > > >+F:  doc/board/emulation/script.rst
> > > > >+F:  scripts/build-qemu.sh
> > > >
> > > > Please, avoid misnomers. This script does not build QEMU.
> > > >
> > > > >+
> > > > > ENVIRONMENT
> > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > S:  Maintained
> > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > >index f8908166276..5a2a00ae225 100644
> > > > >--- a/doc/board/emulation/index.rst
> > > > >+++ b/doc/board/emulation/index.rst
> > > > >@@ -8,6 +8,7 @@ Emulation
> > > > >
> > > > >    acpi
> > > > >    blkdev
> > > > >+   script
> > > > >    qemu-arm
> > > > >    qemu-mips
> > > > >    qemu-ppce500
> > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > >
> > > > Just another misnomer. This page is not about script.sh.
> > > >
> > > > >new file mode 100644
> > > > >index 00000000000..23981e333cb
> > > > >--- /dev/null
> > > > >+++ b/doc/board/emulation/script.rst
> > > > >@@ -0,0 +1,61 @@
> > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > >
> > > > This is not a valid SPDX identifier.
> > > >
> > > > >+
> > > > >+Script for building and running
> > > > >+===============================
> > > > >+
> > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > >+U-Boot on QEMU.
> > > > >+
> > > > >+If uses a environment variables to control how it works:
> > > > >+
> > > > >+ubdir
> > > > >+    base directory for building U-Boot, with each board being in its own
> > > > >+    subdirectory
> > > > >+
> > > > >+imagedir
> > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > >+    type (e.g. ubuntu/
> > > > >+
> > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > >
> > > > This downloads the QEMU source and builds it?
> > > >
> > > > >+
> > > > >+    scripts/build-qemu.sh -rsw
> > > > >+
> > > > >+No support is currently included for specifying a root disk, so this script can
> > > > >+only be used to start installers.
> > > > >+
> > > > >+Options
> > > > >+~~~~~~~
> > > > >+
> > > > >+Options are available to control the script:
> > > > >+
> > > > >+-a <arch>
> > > > >+    Select architecture (default arm, x86)
> > > > >+
> > > > >+-B
> > > > >+    Don't build; assume a build exists
> > > > >+
> > > > >+-k
> > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > >+    emulator
> > > > >+
> > > > >+-o <os>
> > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > >+    website.
> > > >
> > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > >
> > > > Use the URL of the image as argument.
> > > >
> > > > >+
> > > > >+-r
> > > > >+    Run QEMU with the image (by default this is not done)
> > > > >+
> > > > >+-R
> > > > >+    Select OS release (e.g. 24.04).
> > > > >+
> > > > >+-s
> > > > >+    Use serial only (no display)
> > > > >+
> > > > >+-w
> > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > >
> > > > "word version" is not helpful as explanation.
> > > >
> > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > >
> > > > >+
> > > > >+.. note::
> > > > >+
> > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > >+    as Python, accepting the slower startup.
> > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > >new file mode 100755
> > > > >index 00000000000..0ff53593cf9
> > > > >--- /dev/null
> > > > >+++ b/scripts/build-qemu.sh
> > > > >@@ -0,0 +1,175 @@
> > > > >+#!/bin/bash
> > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > >
> > > > This is not a valid SPDX identifier.
> > > >
> > > > >+#
> > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > >+# it, possibly with an OS image
> > > > >+
> > > > >+# This just an example. It assumes that
> > > > >+
> > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > >+
> > > > >+# So far the script supports only ARM and x86.
> > > >
> > > > Why support obsolete i386 but not riscv64?
> > > >
> > > > >+
> > > > >+set -e
> > > > >+
> > > > >+usage() {
> > > > >+    (
> > > > >+    if [[ -n "$1" ]]; then
> > > > >+            echo "$1"
> > > > >+            echo
> > > > >+    fi
> > > > >+    echo "Usage: $0 -aBkrsw"
> > > > >+    echo
> > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > >+    echo "   -r   - Run QEMU with the image"
> > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > >+    echo "   -s   - Use serial only (no display)"
> > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > >+    exit 1
> > > > >+}
> > > > >+
> > > > >+# Directory tree for OS images
> > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > >+
> > > > >+# architecture (arm or x86)
> > > > >+arch=arm
> > > > >+
> > > > >+# 32- or 64-bit build
> > > > >+bitness=64
> > > > >+
> > > > >+# Build U-Boot
> > > > >+build=yes
> > > > >+
> > > > >+# Extra setings
> > > > >+extra=
> > > > >+
> > > > >+# Operating System to boot (ubuntu)
> > > > >+os=
> > > > >+
> > > > >+release=24.04.1
> > > > >+
> > > > >+# run the image with QEMU
> > > > >+run=
> > > > >+
> > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > >+serial=
> > > > >+
> > > > >+# Use kvm
> > > > >+kvm=
> > > > >+
> > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > >+ubdir=${ubdir-/tmp/b}
> > > > >+
> > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > >+    case "${opt}" in
> > > > >+    a)
> > > > >+            arch=$OPTARG
> > > > >+            ;;
> > > > >+    B)
> > > > >+            build=
> > > > >+            ;;
> > > > >+    k)
> > > > >+            kvm="-enable-kvm"
> > > > >+            ;;
> > > > >+    o)
> > > > >+            os=$OPTARG
> > > > >+
> > > > >+            # Expand memory and CPUs
> > > > >+            extra+=" -m 4G -smp 4"
> > > > >+            ;;
> > > > >+    r)
> > > > >+            run=1
> > > > >+            ;;
> > > > >+    R)
> > > > >+            release=$OPTARG
> > > > >+            ;;
> > > > >+    s)
> > > > >+            serial=1
> > > > >+            ;;
> > > > >+    w)
> > > > >+            bitness=32
> > > > >+            ;;
> > > > >+    *)
> > > > >+            usage
> > > > >+            ;;
> > > > >+    esac
> > > > >+done
> > > > >+
> > > > >+# Build U-Boot for the selected board
> > > > >+build_u_boot() {
> > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > >+}
> > > > >+
> > > > >+# Run QEMU with U-Boot
> > > > >+run_qemu() {
> > > > >+    if [[ -n "${os_image}" ]]; then
> > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > >+    fi
> > > > >+    if [[ -n "${serial}" ]]; then
> > > > >+            extra+=" -display none -serial mon:stdio"
> > > > >+    else
> > > > >+            extra+=" -serial mon:stdio"
> > > > >+    fi
> > > > >+    echo "Running ${qemu} ${extra}"
> > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > >+            -m 512 \
> > > >
> > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > >
> > > > >+            -nic none \
> > > >
> > > > Who wants to run without network?
> > > > Use the virtio nic.
> > > >
> > > > >+            ${kvm} \
> > > > >+            ${extra}
> > > > >+}
> > > > >+
> > > > >+# Check architecture
> > > > >+case "${arch}" in
> > > > >+arm)
> > > > >+    BOARD="qemu_arm"
> > > > >+    BIOS="u-boot.bin"
> > > > >+    qemu=qemu-system-arm
> > > > >+    extra+=" -machine virt"
> > > > >+    suffix="arm"
> > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > >+            BOARD="qemu_arm64"
> > > > >+            qemu=qemu-system-aarch64
> > > > >+            extra+=" -cpu cortex-a57"
> > > >
> > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > >
> > > > -cpu max works fine with both tcg and kvm.
> > > >
> > > > >+            suffix="arm64"
> > > >
> > > > >+    fi
> > > > >+    ;;
> > > > >+x86)
> > > > >+    BOARD="qemu-x86"
> > > > >+    BIOS="u-boot.rom"
> > > > >+    qemu=qemu-system-i386
> > > > >+    suffix="i386"
> > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > >+            BOARD="qemu-x86_64"
> > > > >+            qemu=qemu-system-x86_64
> > > > >+            suffix="amd64"
> > > > >+    fi
> > > > >+    ;;
> > > > >+*)
> > > > >+    usage "Unknown architecture '${arch}'"
> > > > >+esac
> > > > >+
> > > > >+# Check OS
> > > > >+case "${os}" in
> > > > >+ubuntu)
> > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > >
> > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > >
> > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > >
> > > > For testing U-Boot a server image is all it takes.
> > >
> > > And _all_ of this is why I don't want to add a useful personal script
> > > as an additional tool we support. I've seen how much work goes in to the
> > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > something like that. Doubly so when ultimately I believe we would be
> > > well served by having a document that says (in much more words) to look
> > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > architectures and platforms and to then further leverage general QEMU
> > > tips and guides on how to run an OS of your choice with that.
> > 
> > I'm not sure it matters that much. Everyone is going to have their
> > preference as to how this script should look, but no one else has
> > taken the time to write one...
> > 
> > People are free to send patches to enhance it. But I believe it is
> > helpful, e.g. for repeating problems caused by recent lmb patches.
> > 
> > Re the test hooks, I just get tired of looking them up and trying to
> > figure out what to do. Every board name and arch is slightly
> > different. Just a hassle that I don't need.
> > 
> > We can put it in scripts/contrib if you like.
> 
> It's handy to point people to scripts, yes. I frequently point people at
> my wrappers around buildman for example for "how do I find code bloat?"
> and similar. But no, I don't think this rises to the level of
> "scripts/contrib".

Coming back to this question again. I'd be willing to make a new
top-level repository for "contributor tooling" and also make that more
widely writable. But I also think you're underestimating the level of
work required to have a "generic" script here that works on arbitrary
developer machines.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Simon Glass]

Hi Tom,

On Mon, 6 Jan 2025 at 07:55, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Nov 15, 2024 at 09:14:03AM -0600, Tom Rini wrote:
> > On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > > >particular architecture and distro.
> > > > > >
> > > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > > >supported.
> > > > > >
> > > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > >---
> > > > > >
> > > > > > MAINTAINERS                    |   8 ++
> > > > > > doc/board/emulation/index.rst  |   1 +
> > > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > > 4 files changed, 245 insertions(+)
> > > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > > create mode 100755 scripts/build-qemu.sh
> > > > > >
> > > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > > >--- a/MAINTAINERS
> > > > > >+++ b/MAINTAINERS
> > > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > > F:  tools/file2include.c
> > > > > > F:  tools/mkeficapsule.c
> > > > > >
> > > > > >+EMULATION
> > > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > > >+S:  Maintained
> > > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > > >+F:  configs/qemu_x86*
> > > > > >+F:  doc/board/emulation/script.rst
> > > > > >+F:  scripts/build-qemu.sh
> > > > >
> > > > > Please, avoid misnomers. This script does not build QEMU.
> > > > >
> > > > > >+
> > > > > > ENVIRONMENT
> > > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > > S:  Maintained
> > > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > > >index f8908166276..5a2a00ae225 100644
> > > > > >--- a/doc/board/emulation/index.rst
> > > > > >+++ b/doc/board/emulation/index.rst
> > > > > >@@ -8,6 +8,7 @@ Emulation
> > > > > >
> > > > > >    acpi
> > > > > >    blkdev
> > > > > >+   script
> > > > > >    qemu-arm
> > > > > >    qemu-mips
> > > > > >    qemu-ppce500
> > > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > > >
> > > > > Just another misnomer. This page is not about script.sh.
> > > > >
> > > > > >new file mode 100644
> > > > > >index 00000000000..23981e333cb
> > > > > >--- /dev/null
> > > > > >+++ b/doc/board/emulation/script.rst
> > > > > >@@ -0,0 +1,61 @@
> > > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > > >
> > > > > This is not a valid SPDX identifier.
> > > > >
> > > > > >+
> > > > > >+Script for building and running
> > > > > >+===============================
> > > > > >+
> > > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > > >+U-Boot on QEMU.
> > > > > >+
> > > > > >+If uses a environment variables to control how it works:
> > > > > >+
> > > > > >+ubdir
> > > > > >+    base directory for building U-Boot, with each board being in its own
> > > > > >+    subdirectory
> > > > > >+
> > > > > >+imagedir
> > > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > > >+    type (e.g. ubuntu/
> > > > > >+
> > > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > > >
> > > > > This downloads the QEMU source and builds it?
> > > > >
> > > > > >+
> > > > > >+    scripts/build-qemu.sh -rsw
> > > > > >+
> > > > > >+No support is currently included for specifying a root disk, so this script can
> > > > > >+only be used to start installers.
> > > > > >+
> > > > > >+Options
> > > > > >+~~~~~~~
> > > > > >+
> > > > > >+Options are available to control the script:
> > > > > >+
> > > > > >+-a <arch>
> > > > > >+    Select architecture (default arm, x86)
> > > > > >+
> > > > > >+-B
> > > > > >+    Don't build; assume a build exists
> > > > > >+
> > > > > >+-k
> > > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > > >+    emulator
> > > > > >+
> > > > > >+-o <os>
> > > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > > >+    website.
> > > > >
> > > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > > >
> > > > > Use the URL of the image as argument.
> > > > >
> > > > > >+
> > > > > >+-r
> > > > > >+    Run QEMU with the image (by default this is not done)
> > > > > >+
> > > > > >+-R
> > > > > >+    Select OS release (e.g. 24.04).
> > > > > >+
> > > > > >+-s
> > > > > >+    Use serial only (no display)
> > > > > >+
> > > > > >+-w
> > > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > > >
> > > > > "word version" is not helpful as explanation.
> > > > >
> > > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > > >
> > > > > >+
> > > > > >+.. note::
> > > > > >+
> > > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > > >+    as Python, accepting the slower startup.
> > > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > > >new file mode 100755
> > > > > >index 00000000000..0ff53593cf9
> > > > > >--- /dev/null
> > > > > >+++ b/scripts/build-qemu.sh
> > > > > >@@ -0,0 +1,175 @@
> > > > > >+#!/bin/bash
> > > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > > >
> > > > > This is not a valid SPDX identifier.
> > > > >
> > > > > >+#
> > > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > > >+# it, possibly with an OS image
> > > > > >+
> > > > > >+# This just an example. It assumes that
> > > > > >+
> > > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > > >+
> > > > > >+# So far the script supports only ARM and x86.
> > > > >
> > > > > Why support obsolete i386 but not riscv64?
> > > > >
> > > > > >+
> > > > > >+set -e
> > > > > >+
> > > > > >+usage() {
> > > > > >+    (
> > > > > >+    if [[ -n "$1" ]]; then
> > > > > >+            echo "$1"
> > > > > >+            echo
> > > > > >+    fi
> > > > > >+    echo "Usage: $0 -aBkrsw"
> > > > > >+    echo
> > > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > > >+    echo "   -r   - Run QEMU with the image"
> > > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > > >+    echo "   -s   - Use serial only (no display)"
> > > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > > >+    exit 1
> > > > > >+}
> > > > > >+
> > > > > >+# Directory tree for OS images
> > > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > > >+
> > > > > >+# architecture (arm or x86)
> > > > > >+arch=arm
> > > > > >+
> > > > > >+# 32- or 64-bit build
> > > > > >+bitness=64
> > > > > >+
> > > > > >+# Build U-Boot
> > > > > >+build=yes
> > > > > >+
> > > > > >+# Extra setings
> > > > > >+extra=
> > > > > >+
> > > > > >+# Operating System to boot (ubuntu)
> > > > > >+os=
> > > > > >+
> > > > > >+release=24.04.1
> > > > > >+
> > > > > >+# run the image with QEMU
> > > > > >+run=
> > > > > >+
> > > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > > >+serial=
> > > > > >+
> > > > > >+# Use kvm
> > > > > >+kvm=
> > > > > >+
> > > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > > >+ubdir=${ubdir-/tmp/b}
> > > > > >+
> > > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > > >+    case "${opt}" in
> > > > > >+    a)
> > > > > >+            arch=$OPTARG
> > > > > >+            ;;
> > > > > >+    B)
> > > > > >+            build=
> > > > > >+            ;;
> > > > > >+    k)
> > > > > >+            kvm="-enable-kvm"
> > > > > >+            ;;
> > > > > >+    o)
> > > > > >+            os=$OPTARG
> > > > > >+
> > > > > >+            # Expand memory and CPUs
> > > > > >+            extra+=" -m 4G -smp 4"
> > > > > >+            ;;
> > > > > >+    r)
> > > > > >+            run=1
> > > > > >+            ;;
> > > > > >+    R)
> > > > > >+            release=$OPTARG
> > > > > >+            ;;
> > > > > >+    s)
> > > > > >+            serial=1
> > > > > >+            ;;
> > > > > >+    w)
> > > > > >+            bitness=32
> > > > > >+            ;;
> > > > > >+    *)
> > > > > >+            usage
> > > > > >+            ;;
> > > > > >+    esac
> > > > > >+done
> > > > > >+
> > > > > >+# Build U-Boot for the selected board
> > > > > >+build_u_boot() {
> > > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > > >+}
> > > > > >+
> > > > > >+# Run QEMU with U-Boot
> > > > > >+run_qemu() {
> > > > > >+    if [[ -n "${os_image}" ]]; then
> > > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > > >+    fi
> > > > > >+    if [[ -n "${serial}" ]]; then
> > > > > >+            extra+=" -display none -serial mon:stdio"
> > > > > >+    else
> > > > > >+            extra+=" -serial mon:stdio"
> > > > > >+    fi
> > > > > >+    echo "Running ${qemu} ${extra}"
> > > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > > >+            -m 512 \
> > > > >
> > > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > > >
> > > > > >+            -nic none \
> > > > >
> > > > > Who wants to run without network?
> > > > > Use the virtio nic.
> > > > >
> > > > > >+            ${kvm} \
> > > > > >+            ${extra}
> > > > > >+}
> > > > > >+
> > > > > >+# Check architecture
> > > > > >+case "${arch}" in
> > > > > >+arm)
> > > > > >+    BOARD="qemu_arm"
> > > > > >+    BIOS="u-boot.bin"
> > > > > >+    qemu=qemu-system-arm
> > > > > >+    extra+=" -machine virt"
> > > > > >+    suffix="arm"
> > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > >+            BOARD="qemu_arm64"
> > > > > >+            qemu=qemu-system-aarch64
> > > > > >+            extra+=" -cpu cortex-a57"
> > > > >
> > > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > > >
> > > > > -cpu max works fine with both tcg and kvm.
> > > > >
> > > > > >+            suffix="arm64"
> > > > >
> > > > > >+    fi
> > > > > >+    ;;
> > > > > >+x86)
> > > > > >+    BOARD="qemu-x86"
> > > > > >+    BIOS="u-boot.rom"
> > > > > >+    qemu=qemu-system-i386
> > > > > >+    suffix="i386"
> > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > >+            BOARD="qemu-x86_64"
> > > > > >+            qemu=qemu-system-x86_64
> > > > > >+            suffix="amd64"
> > > > > >+    fi
> > > > > >+    ;;
> > > > > >+*)
> > > > > >+    usage "Unknown architecture '${arch}'"
> > > > > >+esac
> > > > > >+
> > > > > >+# Check OS
> > > > > >+case "${os}" in
> > > > > >+ubuntu)
> > > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > > >
> > > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > > >
> > > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > > >
> > > > > For testing U-Boot a server image is all it takes.
> > > >
> > > > And _all_ of this is why I don't want to add a useful personal script
> > > > as an additional tool we support. I've seen how much work goes in to the
> > > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > > something like that. Doubly so when ultimately I believe we would be
> > > > well served by having a document that says (in much more words) to look
> > > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > > architectures and platforms and to then further leverage general QEMU
> > > > tips and guides on how to run an OS of your choice with that.
> > >
> > > I'm not sure it matters that much. Everyone is going to have their
> > > preference as to how this script should look, but no one else has
> > > taken the time to write one...
> > >
> > > People are free to send patches to enhance it. But I believe it is
> > > helpful, e.g. for repeating problems caused by recent lmb patches.
> > >
> > > Re the test hooks, I just get tired of looking them up and trying to
> > > figure out what to do. Every board name and arch is slightly
> > > different. Just a hassle that I don't need.
> > >
> > > We can put it in scripts/contrib if you like.
> >
> > It's handy to point people to scripts, yes. I frequently point people at
> > my wrappers around buildman for example for "how do I find code bloat?"
> > and similar. But no, I don't think this rises to the level of
> > "scripts/contrib".
>
> Coming back to this question again. I'd be willing to make a new
> top-level repository for "contributor tooling" and also make that more
> widely writable. But I also think you're underestimating the level of
> work required to have a "generic" script here that works on arbitrary
> developer machines.

Having it in a separate repo seems like too much of a pain, to me.
When things change in U-Boot I would want to update the script (e.g.
to add UPL support, booting Ubuntu and the like).

Regards,
Simon

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 15217 bytes --]

On Thu, Jan 09, 2025 at 05:36:34AM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Mon, 6 Jan 2025 at 07:55, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Fri, Nov 15, 2024 at 09:14:03AM -0600, Tom Rini wrote:
> > > On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > > > Hi Tom,
> > > >
> > > > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > > > >particular architecture and distro.
> > > > > > >
> > > > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > > > >supported.
> > > > > > >
> > > > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > >---
> > > > > > >
> > > > > > > MAINTAINERS                    |   8 ++
> > > > > > > doc/board/emulation/index.rst  |   1 +
> > > > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > > > 4 files changed, 245 insertions(+)
> > > > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > > > create mode 100755 scripts/build-qemu.sh
> > > > > > >
> > > > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > > > >--- a/MAINTAINERS
> > > > > > >+++ b/MAINTAINERS
> > > > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > > > F:  tools/file2include.c
> > > > > > > F:  tools/mkeficapsule.c
> > > > > > >
> > > > > > >+EMULATION
> > > > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > > > >+S:  Maintained
> > > > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > > > >+F:  configs/qemu_x86*
> > > > > > >+F:  doc/board/emulation/script.rst
> > > > > > >+F:  scripts/build-qemu.sh
> > > > > >
> > > > > > Please, avoid misnomers. This script does not build QEMU.
> > > > > >
> > > > > > >+
> > > > > > > ENVIRONMENT
> > > > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > > > S:  Maintained
> > > > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > > > >index f8908166276..5a2a00ae225 100644
> > > > > > >--- a/doc/board/emulation/index.rst
> > > > > > >+++ b/doc/board/emulation/index.rst
> > > > > > >@@ -8,6 +8,7 @@ Emulation
> > > > > > >
> > > > > > >    acpi
> > > > > > >    blkdev
> > > > > > >+   script
> > > > > > >    qemu-arm
> > > > > > >    qemu-mips
> > > > > > >    qemu-ppce500
> > > > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > > > >
> > > > > > Just another misnomer. This page is not about script.sh.
> > > > > >
> > > > > > >new file mode 100644
> > > > > > >index 00000000000..23981e333cb
> > > > > > >--- /dev/null
> > > > > > >+++ b/doc/board/emulation/script.rst
> > > > > > >@@ -0,0 +1,61 @@
> > > > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > > > >
> > > > > > This is not a valid SPDX identifier.
> > > > > >
> > > > > > >+
> > > > > > >+Script for building and running
> > > > > > >+===============================
> > > > > > >+
> > > > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > > > >+U-Boot on QEMU.
> > > > > > >+
> > > > > > >+If uses a environment variables to control how it works:
> > > > > > >+
> > > > > > >+ubdir
> > > > > > >+    base directory for building U-Boot, with each board being in its own
> > > > > > >+    subdirectory
> > > > > > >+
> > > > > > >+imagedir
> > > > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > > > >+    type (e.g. ubuntu/
> > > > > > >+
> > > > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > > > >
> > > > > > This downloads the QEMU source and builds it?
> > > > > >
> > > > > > >+
> > > > > > >+    scripts/build-qemu.sh -rsw
> > > > > > >+
> > > > > > >+No support is currently included for specifying a root disk, so this script can
> > > > > > >+only be used to start installers.
> > > > > > >+
> > > > > > >+Options
> > > > > > >+~~~~~~~
> > > > > > >+
> > > > > > >+Options are available to control the script:
> > > > > > >+
> > > > > > >+-a <arch>
> > > > > > >+    Select architecture (default arm, x86)
> > > > > > >+
> > > > > > >+-B
> > > > > > >+    Don't build; assume a build exists
> > > > > > >+
> > > > > > >+-k
> > > > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > > > >+    emulator
> > > > > > >+
> > > > > > >+-o <os>
> > > > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > > > >+    website.
> > > > > >
> > > > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > > > >
> > > > > > Use the URL of the image as argument.
> > > > > >
> > > > > > >+
> > > > > > >+-r
> > > > > > >+    Run QEMU with the image (by default this is not done)
> > > > > > >+
> > > > > > >+-R
> > > > > > >+    Select OS release (e.g. 24.04).
> > > > > > >+
> > > > > > >+-s
> > > > > > >+    Use serial only (no display)
> > > > > > >+
> > > > > > >+-w
> > > > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > > > >
> > > > > > "word version" is not helpful as explanation.
> > > > > >
> > > > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > > > >
> > > > > > >+
> > > > > > >+.. note::
> > > > > > >+
> > > > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > > > >+    as Python, accepting the slower startup.
> > > > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > > > >new file mode 100755
> > > > > > >index 00000000000..0ff53593cf9
> > > > > > >--- /dev/null
> > > > > > >+++ b/scripts/build-qemu.sh
> > > > > > >@@ -0,0 +1,175 @@
> > > > > > >+#!/bin/bash
> > > > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > > > >
> > > > > > This is not a valid SPDX identifier.
> > > > > >
> > > > > > >+#
> > > > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > > > >+# it, possibly with an OS image
> > > > > > >+
> > > > > > >+# This just an example. It assumes that
> > > > > > >+
> > > > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > > > >+
> > > > > > >+# So far the script supports only ARM and x86.
> > > > > >
> > > > > > Why support obsolete i386 but not riscv64?
> > > > > >
> > > > > > >+
> > > > > > >+set -e
> > > > > > >+
> > > > > > >+usage() {
> > > > > > >+    (
> > > > > > >+    if [[ -n "$1" ]]; then
> > > > > > >+            echo "$1"
> > > > > > >+            echo
> > > > > > >+    fi
> > > > > > >+    echo "Usage: $0 -aBkrsw"
> > > > > > >+    echo
> > > > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > > > >+    echo "   -r   - Run QEMU with the image"
> > > > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > > > >+    echo "   -s   - Use serial only (no display)"
> > > > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > > > >+    exit 1
> > > > > > >+}
> > > > > > >+
> > > > > > >+# Directory tree for OS images
> > > > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > > > >+
> > > > > > >+# architecture (arm or x86)
> > > > > > >+arch=arm
> > > > > > >+
> > > > > > >+# 32- or 64-bit build
> > > > > > >+bitness=64
> > > > > > >+
> > > > > > >+# Build U-Boot
> > > > > > >+build=yes
> > > > > > >+
> > > > > > >+# Extra setings
> > > > > > >+extra=
> > > > > > >+
> > > > > > >+# Operating System to boot (ubuntu)
> > > > > > >+os=
> > > > > > >+
> > > > > > >+release=24.04.1
> > > > > > >+
> > > > > > >+# run the image with QEMU
> > > > > > >+run=
> > > > > > >+
> > > > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > > > >+serial=
> > > > > > >+
> > > > > > >+# Use kvm
> > > > > > >+kvm=
> > > > > > >+
> > > > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > > > >+ubdir=${ubdir-/tmp/b}
> > > > > > >+
> > > > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > > > >+    case "${opt}" in
> > > > > > >+    a)
> > > > > > >+            arch=$OPTARG
> > > > > > >+            ;;
> > > > > > >+    B)
> > > > > > >+            build=
> > > > > > >+            ;;
> > > > > > >+    k)
> > > > > > >+            kvm="-enable-kvm"
> > > > > > >+            ;;
> > > > > > >+    o)
> > > > > > >+            os=$OPTARG
> > > > > > >+
> > > > > > >+            # Expand memory and CPUs
> > > > > > >+            extra+=" -m 4G -smp 4"
> > > > > > >+            ;;
> > > > > > >+    r)
> > > > > > >+            run=1
> > > > > > >+            ;;
> > > > > > >+    R)
> > > > > > >+            release=$OPTARG
> > > > > > >+            ;;
> > > > > > >+    s)
> > > > > > >+            serial=1
> > > > > > >+            ;;
> > > > > > >+    w)
> > > > > > >+            bitness=32
> > > > > > >+            ;;
> > > > > > >+    *)
> > > > > > >+            usage
> > > > > > >+            ;;
> > > > > > >+    esac
> > > > > > >+done
> > > > > > >+
> > > > > > >+# Build U-Boot for the selected board
> > > > > > >+build_u_boot() {
> > > > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > > > >+}
> > > > > > >+
> > > > > > >+# Run QEMU with U-Boot
> > > > > > >+run_qemu() {
> > > > > > >+    if [[ -n "${os_image}" ]]; then
> > > > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > > > >+    fi
> > > > > > >+    if [[ -n "${serial}" ]]; then
> > > > > > >+            extra+=" -display none -serial mon:stdio"
> > > > > > >+    else
> > > > > > >+            extra+=" -serial mon:stdio"
> > > > > > >+    fi
> > > > > > >+    echo "Running ${qemu} ${extra}"
> > > > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > > > >+            -m 512 \
> > > > > >
> > > > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > > > >
> > > > > > >+            -nic none \
> > > > > >
> > > > > > Who wants to run without network?
> > > > > > Use the virtio nic.
> > > > > >
> > > > > > >+            ${kvm} \
> > > > > > >+            ${extra}
> > > > > > >+}
> > > > > > >+
> > > > > > >+# Check architecture
> > > > > > >+case "${arch}" in
> > > > > > >+arm)
> > > > > > >+    BOARD="qemu_arm"
> > > > > > >+    BIOS="u-boot.bin"
> > > > > > >+    qemu=qemu-system-arm
> > > > > > >+    extra+=" -machine virt"
> > > > > > >+    suffix="arm"
> > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > >+            BOARD="qemu_arm64"
> > > > > > >+            qemu=qemu-system-aarch64
> > > > > > >+            extra+=" -cpu cortex-a57"
> > > > > >
> > > > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > > > >
> > > > > > -cpu max works fine with both tcg and kvm.
> > > > > >
> > > > > > >+            suffix="arm64"
> > > > > >
> > > > > > >+    fi
> > > > > > >+    ;;
> > > > > > >+x86)
> > > > > > >+    BOARD="qemu-x86"
> > > > > > >+    BIOS="u-boot.rom"
> > > > > > >+    qemu=qemu-system-i386
> > > > > > >+    suffix="i386"
> > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > >+            BOARD="qemu-x86_64"
> > > > > > >+            qemu=qemu-system-x86_64
> > > > > > >+            suffix="amd64"
> > > > > > >+    fi
> > > > > > >+    ;;
> > > > > > >+*)
> > > > > > >+    usage "Unknown architecture '${arch}'"
> > > > > > >+esac
> > > > > > >+
> > > > > > >+# Check OS
> > > > > > >+case "${os}" in
> > > > > > >+ubuntu)
> > > > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > > > >
> > > > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > > > >
> > > > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > > > >
> > > > > > For testing U-Boot a server image is all it takes.
> > > > >
> > > > > And _all_ of this is why I don't want to add a useful personal script
> > > > > as an additional tool we support. I've seen how much work goes in to the
> > > > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > > > something like that. Doubly so when ultimately I believe we would be
> > > > > well served by having a document that says (in much more words) to look
> > > > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > > > architectures and platforms and to then further leverage general QEMU
> > > > > tips and guides on how to run an OS of your choice with that.
> > > >
> > > > I'm not sure it matters that much. Everyone is going to have their
> > > > preference as to how this script should look, but no one else has
> > > > taken the time to write one...
> > > >
> > > > People are free to send patches to enhance it. But I believe it is
> > > > helpful, e.g. for repeating problems caused by recent lmb patches.
> > > >
> > > > Re the test hooks, I just get tired of looking them up and trying to
> > > > figure out what to do. Every board name and arch is slightly
> > > > different. Just a hassle that I don't need.
> > > >
> > > > We can put it in scripts/contrib if you like.
> > >
> > > It's handy to point people to scripts, yes. I frequently point people at
> > > my wrappers around buildman for example for "how do I find code bloat?"
> > > and similar. But no, I don't think this rises to the level of
> > > "scripts/contrib".
> >
> > Coming back to this question again. I'd be willing to make a new
> > top-level repository for "contributor tooling" and also make that more
> > widely writable. But I also think you're underestimating the level of
> > work required to have a "generic" script here that works on arbitrary
> > developer machines.
> 
> Having it in a separate repo seems like too much of a pain, to me.
> When things change in U-Boot I would want to update the script (e.g.
> to add UPL support, booting Ubuntu and the like).

Being external means it's easier to use for bisect'ing problems and you
still have to handle UPL / no UPL and so on.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Simon Glass]

Hi Tom,

On Thu, 9 Jan 2025 at 07:56, Tom Rini <trini@konsulko.com> wrote:
>
> On Thu, Jan 09, 2025 at 05:36:34AM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Mon, 6 Jan 2025 at 07:55, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Fri, Nov 15, 2024 at 09:14:03AM -0600, Tom Rini wrote:
> > > > On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > > > > Hi Tom,
> > > > >
> > > > > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > > > > >particular architecture and distro.
> > > > > > > >
> > > > > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > > > > >supported.
> > > > > > > >
> > > > > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > >---
> > > > > > > >
> > > > > > > > MAINTAINERS                    |   8 ++
> > > > > > > > doc/board/emulation/index.rst  |   1 +
> > > > > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > > > > 4 files changed, 245 insertions(+)
> > > > > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > > > > create mode 100755 scripts/build-qemu.sh
> > > > > > > >
> > > > > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > > > > >--- a/MAINTAINERS
> > > > > > > >+++ b/MAINTAINERS
> > > > > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > > > > F:  tools/file2include.c
> > > > > > > > F:  tools/mkeficapsule.c
> > > > > > > >
> > > > > > > >+EMULATION
> > > > > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > > > > >+S:  Maintained
> > > > > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > > > > >+F:  configs/qemu_x86*
> > > > > > > >+F:  doc/board/emulation/script.rst
> > > > > > > >+F:  scripts/build-qemu.sh
> > > > > > >
> > > > > > > Please, avoid misnomers. This script does not build QEMU.
> > > > > > >
> > > > > > > >+
> > > > > > > > ENVIRONMENT
> > > > > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > > > > S:  Maintained
> > > > > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > > > > >index f8908166276..5a2a00ae225 100644
> > > > > > > >--- a/doc/board/emulation/index.rst
> > > > > > > >+++ b/doc/board/emulation/index.rst
> > > > > > > >@@ -8,6 +8,7 @@ Emulation
> > > > > > > >
> > > > > > > >    acpi
> > > > > > > >    blkdev
> > > > > > > >+   script
> > > > > > > >    qemu-arm
> > > > > > > >    qemu-mips
> > > > > > > >    qemu-ppce500
> > > > > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > > > > >
> > > > > > > Just another misnomer. This page is not about script.sh.
> > > > > > >
> > > > > > > >new file mode 100644
> > > > > > > >index 00000000000..23981e333cb
> > > > > > > >--- /dev/null
> > > > > > > >+++ b/doc/board/emulation/script.rst
> > > > > > > >@@ -0,0 +1,61 @@
> > > > > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > > > > >
> > > > > > > This is not a valid SPDX identifier.
> > > > > > >
> > > > > > > >+
> > > > > > > >+Script for building and running
> > > > > > > >+===============================
> > > > > > > >+
> > > > > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > > > > >+U-Boot on QEMU.
> > > > > > > >+
> > > > > > > >+If uses a environment variables to control how it works:
> > > > > > > >+
> > > > > > > >+ubdir
> > > > > > > >+    base directory for building U-Boot, with each board being in its own
> > > > > > > >+    subdirectory
> > > > > > > >+
> > > > > > > >+imagedir
> > > > > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > > > > >+    type (e.g. ubuntu/
> > > > > > > >+
> > > > > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > > > > >
> > > > > > > This downloads the QEMU source and builds it?
> > > > > > >
> > > > > > > >+
> > > > > > > >+    scripts/build-qemu.sh -rsw
> > > > > > > >+
> > > > > > > >+No support is currently included for specifying a root disk, so this script can
> > > > > > > >+only be used to start installers.
> > > > > > > >+
> > > > > > > >+Options
> > > > > > > >+~~~~~~~
> > > > > > > >+
> > > > > > > >+Options are available to control the script:
> > > > > > > >+
> > > > > > > >+-a <arch>
> > > > > > > >+    Select architecture (default arm, x86)
> > > > > > > >+
> > > > > > > >+-B
> > > > > > > >+    Don't build; assume a build exists
> > > > > > > >+
> > > > > > > >+-k
> > > > > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > > > > >+    emulator
> > > > > > > >+
> > > > > > > >+-o <os>
> > > > > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > > > > >+    website.
> > > > > > >
> > > > > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > > > > >
> > > > > > > Use the URL of the image as argument.
> > > > > > >
> > > > > > > >+
> > > > > > > >+-r
> > > > > > > >+    Run QEMU with the image (by default this is not done)
> > > > > > > >+
> > > > > > > >+-R
> > > > > > > >+    Select OS release (e.g. 24.04).
> > > > > > > >+
> > > > > > > >+-s
> > > > > > > >+    Use serial only (no display)
> > > > > > > >+
> > > > > > > >+-w
> > > > > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > > > > >
> > > > > > > "word version" is not helpful as explanation.
> > > > > > >
> > > > > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > > > > >
> > > > > > > >+
> > > > > > > >+.. note::
> > > > > > > >+
> > > > > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > > > > >+    as Python, accepting the slower startup.
> > > > > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > > > > >new file mode 100755
> > > > > > > >index 00000000000..0ff53593cf9
> > > > > > > >--- /dev/null
> > > > > > > >+++ b/scripts/build-qemu.sh
> > > > > > > >@@ -0,0 +1,175 @@
> > > > > > > >+#!/bin/bash
> > > > > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > > > > >
> > > > > > > This is not a valid SPDX identifier.
> > > > > > >
> > > > > > > >+#
> > > > > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > > > > >+# it, possibly with an OS image
> > > > > > > >+
> > > > > > > >+# This just an example. It assumes that
> > > > > > > >+
> > > > > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > > > > >+
> > > > > > > >+# So far the script supports only ARM and x86.
> > > > > > >
> > > > > > > Why support obsolete i386 but not riscv64?
> > > > > > >
> > > > > > > >+
> > > > > > > >+set -e
> > > > > > > >+
> > > > > > > >+usage() {
> > > > > > > >+    (
> > > > > > > >+    if [[ -n "$1" ]]; then
> > > > > > > >+            echo "$1"
> > > > > > > >+            echo
> > > > > > > >+    fi
> > > > > > > >+    echo "Usage: $0 -aBkrsw"
> > > > > > > >+    echo
> > > > > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > > > > >+    echo "   -r   - Run QEMU with the image"
> > > > > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > > > > >+    echo "   -s   - Use serial only (no display)"
> > > > > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > > > > >+    exit 1
> > > > > > > >+}
> > > > > > > >+
> > > > > > > >+# Directory tree for OS images
> > > > > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > > > > >+
> > > > > > > >+# architecture (arm or x86)
> > > > > > > >+arch=arm
> > > > > > > >+
> > > > > > > >+# 32- or 64-bit build
> > > > > > > >+bitness=64
> > > > > > > >+
> > > > > > > >+# Build U-Boot
> > > > > > > >+build=yes
> > > > > > > >+
> > > > > > > >+# Extra setings
> > > > > > > >+extra=
> > > > > > > >+
> > > > > > > >+# Operating System to boot (ubuntu)
> > > > > > > >+os=
> > > > > > > >+
> > > > > > > >+release=24.04.1
> > > > > > > >+
> > > > > > > >+# run the image with QEMU
> > > > > > > >+run=
> > > > > > > >+
> > > > > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > > > > >+serial=
> > > > > > > >+
> > > > > > > >+# Use kvm
> > > > > > > >+kvm=
> > > > > > > >+
> > > > > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > > > > >+ubdir=${ubdir-/tmp/b}
> > > > > > > >+
> > > > > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > > > > >+    case "${opt}" in
> > > > > > > >+    a)
> > > > > > > >+            arch=$OPTARG
> > > > > > > >+            ;;
> > > > > > > >+    B)
> > > > > > > >+            build=
> > > > > > > >+            ;;
> > > > > > > >+    k)
> > > > > > > >+            kvm="-enable-kvm"
> > > > > > > >+            ;;
> > > > > > > >+    o)
> > > > > > > >+            os=$OPTARG
> > > > > > > >+
> > > > > > > >+            # Expand memory and CPUs
> > > > > > > >+            extra+=" -m 4G -smp 4"
> > > > > > > >+            ;;
> > > > > > > >+    r)
> > > > > > > >+            run=1
> > > > > > > >+            ;;
> > > > > > > >+    R)
> > > > > > > >+            release=$OPTARG
> > > > > > > >+            ;;
> > > > > > > >+    s)
> > > > > > > >+            serial=1
> > > > > > > >+            ;;
> > > > > > > >+    w)
> > > > > > > >+            bitness=32
> > > > > > > >+            ;;
> > > > > > > >+    *)
> > > > > > > >+            usage
> > > > > > > >+            ;;
> > > > > > > >+    esac
> > > > > > > >+done
> > > > > > > >+
> > > > > > > >+# Build U-Boot for the selected board
> > > > > > > >+build_u_boot() {
> > > > > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > > > > >+}
> > > > > > > >+
> > > > > > > >+# Run QEMU with U-Boot
> > > > > > > >+run_qemu() {
> > > > > > > >+    if [[ -n "${os_image}" ]]; then
> > > > > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > > > > >+    fi
> > > > > > > >+    if [[ -n "${serial}" ]]; then
> > > > > > > >+            extra+=" -display none -serial mon:stdio"
> > > > > > > >+    else
> > > > > > > >+            extra+=" -serial mon:stdio"
> > > > > > > >+    fi
> > > > > > > >+    echo "Running ${qemu} ${extra}"
> > > > > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > > > > >+            -m 512 \
> > > > > > >
> > > > > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > > > > >
> > > > > > > >+            -nic none \
> > > > > > >
> > > > > > > Who wants to run without network?
> > > > > > > Use the virtio nic.
> > > > > > >
> > > > > > > >+            ${kvm} \
> > > > > > > >+            ${extra}
> > > > > > > >+}
> > > > > > > >+
> > > > > > > >+# Check architecture
> > > > > > > >+case "${arch}" in
> > > > > > > >+arm)
> > > > > > > >+    BOARD="qemu_arm"
> > > > > > > >+    BIOS="u-boot.bin"
> > > > > > > >+    qemu=qemu-system-arm
> > > > > > > >+    extra+=" -machine virt"
> > > > > > > >+    suffix="arm"
> > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > >+            BOARD="qemu_arm64"
> > > > > > > >+            qemu=qemu-system-aarch64
> > > > > > > >+            extra+=" -cpu cortex-a57"
> > > > > > >
> > > > > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > > > > >
> > > > > > > -cpu max works fine with both tcg and kvm.
> > > > > > >
> > > > > > > >+            suffix="arm64"
> > > > > > >
> > > > > > > >+    fi
> > > > > > > >+    ;;
> > > > > > > >+x86)
> > > > > > > >+    BOARD="qemu-x86"
> > > > > > > >+    BIOS="u-boot.rom"
> > > > > > > >+    qemu=qemu-system-i386
> > > > > > > >+    suffix="i386"
> > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > >+            BOARD="qemu-x86_64"
> > > > > > > >+            qemu=qemu-system-x86_64
> > > > > > > >+            suffix="amd64"
> > > > > > > >+    fi
> > > > > > > >+    ;;
> > > > > > > >+*)
> > > > > > > >+    usage "Unknown architecture '${arch}'"
> > > > > > > >+esac
> > > > > > > >+
> > > > > > > >+# Check OS
> > > > > > > >+case "${os}" in
> > > > > > > >+ubuntu)
> > > > > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > > > > >
> > > > > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > > > > >
> > > > > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > > > > >
> > > > > > > For testing U-Boot a server image is all it takes.
> > > > > >
> > > > > > And _all_ of this is why I don't want to add a useful personal script
> > > > > > as an additional tool we support. I've seen how much work goes in to the
> > > > > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > > > > something like that. Doubly so when ultimately I believe we would be
> > > > > > well served by having a document that says (in much more words) to look
> > > > > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > > > > architectures and platforms and to then further leverage general QEMU
> > > > > > tips and guides on how to run an OS of your choice with that.
> > > > >
> > > > > I'm not sure it matters that much. Everyone is going to have their
> > > > > preference as to how this script should look, but no one else has
> > > > > taken the time to write one...
> > > > >
> > > > > People are free to send patches to enhance it. But I believe it is
> > > > > helpful, e.g. for repeating problems caused by recent lmb patches.
> > > > >
> > > > > Re the test hooks, I just get tired of looking them up and trying to
> > > > > figure out what to do. Every board name and arch is slightly
> > > > > different. Just a hassle that I don't need.
> > > > >
> > > > > We can put it in scripts/contrib if you like.
> > > >
> > > > It's handy to point people to scripts, yes. I frequently point people at
> > > > my wrappers around buildman for example for "how do I find code bloat?"
> > > > and similar. But no, I don't think this rises to the level of
> > > > "scripts/contrib".
> > >
> > > Coming back to this question again. I'd be willing to make a new
> > > top-level repository for "contributor tooling" and also make that more
> > > widely writable. But I also think you're underestimating the level of
> > > work required to have a "generic" script here that works on arbitrary
> > > developer machines.
> >
> > Having it in a separate repo seems like too much of a pain, to me.
> > When things change in U-Boot I would want to update the script (e.g.
> > to add UPL support, booting Ubuntu and the like).
>
> Being external means it's easier to use for bisect'ing problems and you
> still have to handle UPL / no UPL and so on.

There are trade-offs, for sure. Sometimes I use buildman from a
separate tree when trying to debug something which changes buildman.
But I don't think that warrants creating an entirely new tree for
scripts.

Regards,
Simon

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 17243 bytes --]

On Fri, Jan 10, 2025 at 06:39:19AM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Thu, 9 Jan 2025 at 07:56, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Thu, Jan 09, 2025 at 05:36:34AM -0700, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Mon, 6 Jan 2025 at 07:55, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Fri, Nov 15, 2024 at 09:14:03AM -0600, Tom Rini wrote:
> > > > > On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > > > > > Hi Tom,
> > > > > >
> > > > > > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > > > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > > > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > > > > > >particular architecture and distro.
> > > > > > > > >
> > > > > > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > > > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > > > > > >supported.
> > > > > > > > >
> > > > > > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > > >---
> > > > > > > > >
> > > > > > > > > MAINTAINERS                    |   8 ++
> > > > > > > > > doc/board/emulation/index.rst  |   1 +
> > > > > > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > > > > > 4 files changed, 245 insertions(+)
> > > > > > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > > > > > create mode 100755 scripts/build-qemu.sh
> > > > > > > > >
> > > > > > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > > > > > >--- a/MAINTAINERS
> > > > > > > > >+++ b/MAINTAINERS
> > > > > > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > > > > > F:  tools/file2include.c
> > > > > > > > > F:  tools/mkeficapsule.c
> > > > > > > > >
> > > > > > > > >+EMULATION
> > > > > > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > > > > > >+S:  Maintained
> > > > > > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > > > > > >+F:  configs/qemu_x86*
> > > > > > > > >+F:  doc/board/emulation/script.rst
> > > > > > > > >+F:  scripts/build-qemu.sh
> > > > > > > >
> > > > > > > > Please, avoid misnomers. This script does not build QEMU.
> > > > > > > >
> > > > > > > > >+
> > > > > > > > > ENVIRONMENT
> > > > > > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > > > > > S:  Maintained
> > > > > > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > > > > > >index f8908166276..5a2a00ae225 100644
> > > > > > > > >--- a/doc/board/emulation/index.rst
> > > > > > > > >+++ b/doc/board/emulation/index.rst
> > > > > > > > >@@ -8,6 +8,7 @@ Emulation
> > > > > > > > >
> > > > > > > > >    acpi
> > > > > > > > >    blkdev
> > > > > > > > >+   script
> > > > > > > > >    qemu-arm
> > > > > > > > >    qemu-mips
> > > > > > > > >    qemu-ppce500
> > > > > > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > > > > > >
> > > > > > > > Just another misnomer. This page is not about script.sh.
> > > > > > > >
> > > > > > > > >new file mode 100644
> > > > > > > > >index 00000000000..23981e333cb
> > > > > > > > >--- /dev/null
> > > > > > > > >+++ b/doc/board/emulation/script.rst
> > > > > > > > >@@ -0,0 +1,61 @@
> > > > > > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > > > > > >
> > > > > > > > This is not a valid SPDX identifier.
> > > > > > > >
> > > > > > > > >+
> > > > > > > > >+Script for building and running
> > > > > > > > >+===============================
> > > > > > > > >+
> > > > > > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > > > > > >+U-Boot on QEMU.
> > > > > > > > >+
> > > > > > > > >+If uses a environment variables to control how it works:
> > > > > > > > >+
> > > > > > > > >+ubdir
> > > > > > > > >+    base directory for building U-Boot, with each board being in its own
> > > > > > > > >+    subdirectory
> > > > > > > > >+
> > > > > > > > >+imagedir
> > > > > > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > > > > > >+    type (e.g. ubuntu/
> > > > > > > > >+
> > > > > > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > > > > > >
> > > > > > > > This downloads the QEMU source and builds it?
> > > > > > > >
> > > > > > > > >+
> > > > > > > > >+    scripts/build-qemu.sh -rsw
> > > > > > > > >+
> > > > > > > > >+No support is currently included for specifying a root disk, so this script can
> > > > > > > > >+only be used to start installers.
> > > > > > > > >+
> > > > > > > > >+Options
> > > > > > > > >+~~~~~~~
> > > > > > > > >+
> > > > > > > > >+Options are available to control the script:
> > > > > > > > >+
> > > > > > > > >+-a <arch>
> > > > > > > > >+    Select architecture (default arm, x86)
> > > > > > > > >+
> > > > > > > > >+-B
> > > > > > > > >+    Don't build; assume a build exists
> > > > > > > > >+
> > > > > > > > >+-k
> > > > > > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > > > > > >+    emulator
> > > > > > > > >+
> > > > > > > > >+-o <os>
> > > > > > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > > > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > > > > > >+    website.
> > > > > > > >
> > > > > > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > > > > > >
> > > > > > > > Use the URL of the image as argument.
> > > > > > > >
> > > > > > > > >+
> > > > > > > > >+-r
> > > > > > > > >+    Run QEMU with the image (by default this is not done)
> > > > > > > > >+
> > > > > > > > >+-R
> > > > > > > > >+    Select OS release (e.g. 24.04).
> > > > > > > > >+
> > > > > > > > >+-s
> > > > > > > > >+    Use serial only (no display)
> > > > > > > > >+
> > > > > > > > >+-w
> > > > > > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > > > > > >
> > > > > > > > "word version" is not helpful as explanation.
> > > > > > > >
> > > > > > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > > > > > >
> > > > > > > > >+
> > > > > > > > >+.. note::
> > > > > > > > >+
> > > > > > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > > > > > >+    as Python, accepting the slower startup.
> > > > > > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > > > > > >new file mode 100755
> > > > > > > > >index 00000000000..0ff53593cf9
> > > > > > > > >--- /dev/null
> > > > > > > > >+++ b/scripts/build-qemu.sh
> > > > > > > > >@@ -0,0 +1,175 @@
> > > > > > > > >+#!/bin/bash
> > > > > > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > > > > > >
> > > > > > > > This is not a valid SPDX identifier.
> > > > > > > >
> > > > > > > > >+#
> > > > > > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > > > > > >+# it, possibly with an OS image
> > > > > > > > >+
> > > > > > > > >+# This just an example. It assumes that
> > > > > > > > >+
> > > > > > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > > > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > > > > > >+
> > > > > > > > >+# So far the script supports only ARM and x86.
> > > > > > > >
> > > > > > > > Why support obsolete i386 but not riscv64?
> > > > > > > >
> > > > > > > > >+
> > > > > > > > >+set -e
> > > > > > > > >+
> > > > > > > > >+usage() {
> > > > > > > > >+    (
> > > > > > > > >+    if [[ -n "$1" ]]; then
> > > > > > > > >+            echo "$1"
> > > > > > > > >+            echo
> > > > > > > > >+    fi
> > > > > > > > >+    echo "Usage: $0 -aBkrsw"
> > > > > > > > >+    echo
> > > > > > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > > > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > > > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > > > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > > > > > >+    echo "   -r   - Run QEMU with the image"
> > > > > > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > > > > > >+    echo "   -s   - Use serial only (no display)"
> > > > > > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > > > > > >+    exit 1
> > > > > > > > >+}
> > > > > > > > >+
> > > > > > > > >+# Directory tree for OS images
> > > > > > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > > > > > >+
> > > > > > > > >+# architecture (arm or x86)
> > > > > > > > >+arch=arm
> > > > > > > > >+
> > > > > > > > >+# 32- or 64-bit build
> > > > > > > > >+bitness=64
> > > > > > > > >+
> > > > > > > > >+# Build U-Boot
> > > > > > > > >+build=yes
> > > > > > > > >+
> > > > > > > > >+# Extra setings
> > > > > > > > >+extra=
> > > > > > > > >+
> > > > > > > > >+# Operating System to boot (ubuntu)
> > > > > > > > >+os=
> > > > > > > > >+
> > > > > > > > >+release=24.04.1
> > > > > > > > >+
> > > > > > > > >+# run the image with QEMU
> > > > > > > > >+run=
> > > > > > > > >+
> > > > > > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > > > > > >+serial=
> > > > > > > > >+
> > > > > > > > >+# Use kvm
> > > > > > > > >+kvm=
> > > > > > > > >+
> > > > > > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > > > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > > > > > >+ubdir=${ubdir-/tmp/b}
> > > > > > > > >+
> > > > > > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > > > > > >+    case "${opt}" in
> > > > > > > > >+    a)
> > > > > > > > >+            arch=$OPTARG
> > > > > > > > >+            ;;
> > > > > > > > >+    B)
> > > > > > > > >+            build=
> > > > > > > > >+            ;;
> > > > > > > > >+    k)
> > > > > > > > >+            kvm="-enable-kvm"
> > > > > > > > >+            ;;
> > > > > > > > >+    o)
> > > > > > > > >+            os=$OPTARG
> > > > > > > > >+
> > > > > > > > >+            # Expand memory and CPUs
> > > > > > > > >+            extra+=" -m 4G -smp 4"
> > > > > > > > >+            ;;
> > > > > > > > >+    r)
> > > > > > > > >+            run=1
> > > > > > > > >+            ;;
> > > > > > > > >+    R)
> > > > > > > > >+            release=$OPTARG
> > > > > > > > >+            ;;
> > > > > > > > >+    s)
> > > > > > > > >+            serial=1
> > > > > > > > >+            ;;
> > > > > > > > >+    w)
> > > > > > > > >+            bitness=32
> > > > > > > > >+            ;;
> > > > > > > > >+    *)
> > > > > > > > >+            usage
> > > > > > > > >+            ;;
> > > > > > > > >+    esac
> > > > > > > > >+done
> > > > > > > > >+
> > > > > > > > >+# Build U-Boot for the selected board
> > > > > > > > >+build_u_boot() {
> > > > > > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > > > > > >+}
> > > > > > > > >+
> > > > > > > > >+# Run QEMU with U-Boot
> > > > > > > > >+run_qemu() {
> > > > > > > > >+    if [[ -n "${os_image}" ]]; then
> > > > > > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > > > > > >+    fi
> > > > > > > > >+    if [[ -n "${serial}" ]]; then
> > > > > > > > >+            extra+=" -display none -serial mon:stdio"
> > > > > > > > >+    else
> > > > > > > > >+            extra+=" -serial mon:stdio"
> > > > > > > > >+    fi
> > > > > > > > >+    echo "Running ${qemu} ${extra}"
> > > > > > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > > > > > >+            -m 512 \
> > > > > > > >
> > > > > > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > > > > > >
> > > > > > > > >+            -nic none \
> > > > > > > >
> > > > > > > > Who wants to run without network?
> > > > > > > > Use the virtio nic.
> > > > > > > >
> > > > > > > > >+            ${kvm} \
> > > > > > > > >+            ${extra}
> > > > > > > > >+}
> > > > > > > > >+
> > > > > > > > >+# Check architecture
> > > > > > > > >+case "${arch}" in
> > > > > > > > >+arm)
> > > > > > > > >+    BOARD="qemu_arm"
> > > > > > > > >+    BIOS="u-boot.bin"
> > > > > > > > >+    qemu=qemu-system-arm
> > > > > > > > >+    extra+=" -machine virt"
> > > > > > > > >+    suffix="arm"
> > > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > > >+            BOARD="qemu_arm64"
> > > > > > > > >+            qemu=qemu-system-aarch64
> > > > > > > > >+            extra+=" -cpu cortex-a57"
> > > > > > > >
> > > > > > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > > > > > >
> > > > > > > > -cpu max works fine with both tcg and kvm.
> > > > > > > >
> > > > > > > > >+            suffix="arm64"
> > > > > > > >
> > > > > > > > >+    fi
> > > > > > > > >+    ;;
> > > > > > > > >+x86)
> > > > > > > > >+    BOARD="qemu-x86"
> > > > > > > > >+    BIOS="u-boot.rom"
> > > > > > > > >+    qemu=qemu-system-i386
> > > > > > > > >+    suffix="i386"
> > > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > > >+            BOARD="qemu-x86_64"
> > > > > > > > >+            qemu=qemu-system-x86_64
> > > > > > > > >+            suffix="amd64"
> > > > > > > > >+    fi
> > > > > > > > >+    ;;
> > > > > > > > >+*)
> > > > > > > > >+    usage "Unknown architecture '${arch}'"
> > > > > > > > >+esac
> > > > > > > > >+
> > > > > > > > >+# Check OS
> > > > > > > > >+case "${os}" in
> > > > > > > > >+ubuntu)
> > > > > > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > > > > > >
> > > > > > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > > > > > >
> > > > > > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > > > > > >
> > > > > > > > For testing U-Boot a server image is all it takes.
> > > > > > >
> > > > > > > And _all_ of this is why I don't want to add a useful personal script
> > > > > > > as an additional tool we support. I've seen how much work goes in to the
> > > > > > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > > > > > something like that. Doubly so when ultimately I believe we would be
> > > > > > > well served by having a document that says (in much more words) to look
> > > > > > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > > > > > architectures and platforms and to then further leverage general QEMU
> > > > > > > tips and guides on how to run an OS of your choice with that.
> > > > > >
> > > > > > I'm not sure it matters that much. Everyone is going to have their
> > > > > > preference as to how this script should look, but no one else has
> > > > > > taken the time to write one...
> > > > > >
> > > > > > People are free to send patches to enhance it. But I believe it is
> > > > > > helpful, e.g. for repeating problems caused by recent lmb patches.
> > > > > >
> > > > > > Re the test hooks, I just get tired of looking them up and trying to
> > > > > > figure out what to do. Every board name and arch is slightly
> > > > > > different. Just a hassle that I don't need.
> > > > > >
> > > > > > We can put it in scripts/contrib if you like.
> > > > >
> > > > > It's handy to point people to scripts, yes. I frequently point people at
> > > > > my wrappers around buildman for example for "how do I find code bloat?"
> > > > > and similar. But no, I don't think this rises to the level of
> > > > > "scripts/contrib".
> > > >
> > > > Coming back to this question again. I'd be willing to make a new
> > > > top-level repository for "contributor tooling" and also make that more
> > > > widely writable. But I also think you're underestimating the level of
> > > > work required to have a "generic" script here that works on arbitrary
> > > > developer machines.
> > >
> > > Having it in a separate repo seems like too much of a pain, to me.
> > > When things change in U-Boot I would want to update the script (e.g.
> > > to add UPL support, booting Ubuntu and the like).
> >
> > Being external means it's easier to use for bisect'ing problems and you
> > still have to handle UPL / no UPL and so on.
> 
> There are trade-offs, for sure. Sometimes I use buildman from a
> separate tree when trying to debug something which changes buildman.
> But I don't think that warrants creating an entirely new tree for
> scripts.

It would also encourage others. I for example might put the scripts I
use for having buildman do various tasks there as well.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Simon Glass]

Hi Tom,

On Fri, 10 Jan 2025 at 09:01, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Jan 10, 2025 at 06:39:19AM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Thu, 9 Jan 2025 at 07:56, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Thu, Jan 09, 2025 at 05:36:34AM -0700, Simon Glass wrote:
> > > > Hi Tom,
> > > >
> > > > On Mon, 6 Jan 2025 at 07:55, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Fri, Nov 15, 2024 at 09:14:03AM -0600, Tom Rini wrote:
> > > > > > On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > > > > > > Hi Tom,
> > > > > > >
> > > > > > > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > > > > > > >
> > > > > > > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > > > > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > > > > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > > > > > > >particular architecture and distro.
> > > > > > > > > >
> > > > > > > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > > > > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > > > > > > >supported.
> > > > > > > > > >
> > > > > > > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > > > >---
> > > > > > > > > >
> > > > > > > > > > MAINTAINERS                    |   8 ++
> > > > > > > > > > doc/board/emulation/index.rst  |   1 +
> > > > > > > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > > > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > > > > > > 4 files changed, 245 insertions(+)
> > > > > > > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > > > > > > create mode 100755 scripts/build-qemu.sh
> > > > > > > > > >
> > > > > > > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > > > > > > >--- a/MAINTAINERS
> > > > > > > > > >+++ b/MAINTAINERS
> > > > > > > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > > > > > > F:  tools/file2include.c
> > > > > > > > > > F:  tools/mkeficapsule.c
> > > > > > > > > >
> > > > > > > > > >+EMULATION
> > > > > > > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > > > > > > >+S:  Maintained
> > > > > > > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > > > > > > >+F:  configs/qemu_x86*
> > > > > > > > > >+F:  doc/board/emulation/script.rst
> > > > > > > > > >+F:  scripts/build-qemu.sh
> > > > > > > > >
> > > > > > > > > Please, avoid misnomers. This script does not build QEMU.
> > > > > > > > >
> > > > > > > > > >+
> > > > > > > > > > ENVIRONMENT
> > > > > > > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > > > > > > S:  Maintained
> > > > > > > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > > > > > > >index f8908166276..5a2a00ae225 100644
> > > > > > > > > >--- a/doc/board/emulation/index.rst
> > > > > > > > > >+++ b/doc/board/emulation/index.rst
> > > > > > > > > >@@ -8,6 +8,7 @@ Emulation
> > > > > > > > > >
> > > > > > > > > >    acpi
> > > > > > > > > >    blkdev
> > > > > > > > > >+   script
> > > > > > > > > >    qemu-arm
> > > > > > > > > >    qemu-mips
> > > > > > > > > >    qemu-ppce500
> > > > > > > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > > > > > > >
> > > > > > > > > Just another misnomer. This page is not about script.sh.
> > > > > > > > >
> > > > > > > > > >new file mode 100644
> > > > > > > > > >index 00000000000..23981e333cb
> > > > > > > > > >--- /dev/null
> > > > > > > > > >+++ b/doc/board/emulation/script.rst
> > > > > > > > > >@@ -0,0 +1,61 @@
> > > > > > > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > > > > > > >
> > > > > > > > > This is not a valid SPDX identifier.
> > > > > > > > >
> > > > > > > > > >+
> > > > > > > > > >+Script for building and running
> > > > > > > > > >+===============================
> > > > > > > > > >+
> > > > > > > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > > > > > > >+U-Boot on QEMU.
> > > > > > > > > >+
> > > > > > > > > >+If uses a environment variables to control how it works:
> > > > > > > > > >+
> > > > > > > > > >+ubdir
> > > > > > > > > >+    base directory for building U-Boot, with each board being in its own
> > > > > > > > > >+    subdirectory
> > > > > > > > > >+
> > > > > > > > > >+imagedir
> > > > > > > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > > > > > > >+    type (e.g. ubuntu/
> > > > > > > > > >+
> > > > > > > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > > > > > > >
> > > > > > > > > This downloads the QEMU source and builds it?
> > > > > > > > >
> > > > > > > > > >+
> > > > > > > > > >+    scripts/build-qemu.sh -rsw
> > > > > > > > > >+
> > > > > > > > > >+No support is currently included for specifying a root disk, so this script can
> > > > > > > > > >+only be used to start installers.
> > > > > > > > > >+
> > > > > > > > > >+Options
> > > > > > > > > >+~~~~~~~
> > > > > > > > > >+
> > > > > > > > > >+Options are available to control the script:
> > > > > > > > > >+
> > > > > > > > > >+-a <arch>
> > > > > > > > > >+    Select architecture (default arm, x86)
> > > > > > > > > >+
> > > > > > > > > >+-B
> > > > > > > > > >+    Don't build; assume a build exists
> > > > > > > > > >+
> > > > > > > > > >+-k
> > > > > > > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > > > > > > >+    emulator
> > > > > > > > > >+
> > > > > > > > > >+-o <os>
> > > > > > > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > > > > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > > > > > > >+    website.
> > > > > > > > >
> > > > > > > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > > > > > > >
> > > > > > > > > Use the URL of the image as argument.
> > > > > > > > >
> > > > > > > > > >+
> > > > > > > > > >+-r
> > > > > > > > > >+    Run QEMU with the image (by default this is not done)
> > > > > > > > > >+
> > > > > > > > > >+-R
> > > > > > > > > >+    Select OS release (e.g. 24.04).
> > > > > > > > > >+
> > > > > > > > > >+-s
> > > > > > > > > >+    Use serial only (no display)
> > > > > > > > > >+
> > > > > > > > > >+-w
> > > > > > > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > > > > > > >
> > > > > > > > > "word version" is not helpful as explanation.
> > > > > > > > >
> > > > > > > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > > > > > > >
> > > > > > > > > >+
> > > > > > > > > >+.. note::
> > > > > > > > > >+
> > > > > > > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > > > > > > >+    as Python, accepting the slower startup.
> > > > > > > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > > > > > > >new file mode 100755
> > > > > > > > > >index 00000000000..0ff53593cf9
> > > > > > > > > >--- /dev/null
> > > > > > > > > >+++ b/scripts/build-qemu.sh
> > > > > > > > > >@@ -0,0 +1,175 @@
> > > > > > > > > >+#!/bin/bash
> > > > > > > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > > > > > > >
> > > > > > > > > This is not a valid SPDX identifier.
> > > > > > > > >
> > > > > > > > > >+#
> > > > > > > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > > > > > > >+# it, possibly with an OS image
> > > > > > > > > >+
> > > > > > > > > >+# This just an example. It assumes that
> > > > > > > > > >+
> > > > > > > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > > > > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > > > > > > >+
> > > > > > > > > >+# So far the script supports only ARM and x86.
> > > > > > > > >
> > > > > > > > > Why support obsolete i386 but not riscv64?
> > > > > > > > >
> > > > > > > > > >+
> > > > > > > > > >+set -e
> > > > > > > > > >+
> > > > > > > > > >+usage() {
> > > > > > > > > >+    (
> > > > > > > > > >+    if [[ -n "$1" ]]; then
> > > > > > > > > >+            echo "$1"
> > > > > > > > > >+            echo
> > > > > > > > > >+    fi
> > > > > > > > > >+    echo "Usage: $0 -aBkrsw"
> > > > > > > > > >+    echo
> > > > > > > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > > > > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > > > > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > > > > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > > > > > > >+    echo "   -r   - Run QEMU with the image"
> > > > > > > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > > > > > > >+    echo "   -s   - Use serial only (no display)"
> > > > > > > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > > > > > > >+    exit 1
> > > > > > > > > >+}
> > > > > > > > > >+
> > > > > > > > > >+# Directory tree for OS images
> > > > > > > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > > > > > > >+
> > > > > > > > > >+# architecture (arm or x86)
> > > > > > > > > >+arch=arm
> > > > > > > > > >+
> > > > > > > > > >+# 32- or 64-bit build
> > > > > > > > > >+bitness=64
> > > > > > > > > >+
> > > > > > > > > >+# Build U-Boot
> > > > > > > > > >+build=yes
> > > > > > > > > >+
> > > > > > > > > >+# Extra setings
> > > > > > > > > >+extra=
> > > > > > > > > >+
> > > > > > > > > >+# Operating System to boot (ubuntu)
> > > > > > > > > >+os=
> > > > > > > > > >+
> > > > > > > > > >+release=24.04.1
> > > > > > > > > >+
> > > > > > > > > >+# run the image with QEMU
> > > > > > > > > >+run=
> > > > > > > > > >+
> > > > > > > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > > > > > > >+serial=
> > > > > > > > > >+
> > > > > > > > > >+# Use kvm
> > > > > > > > > >+kvm=
> > > > > > > > > >+
> > > > > > > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > > > > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > > > > > > >+ubdir=${ubdir-/tmp/b}
> > > > > > > > > >+
> > > > > > > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > > > > > > >+    case "${opt}" in
> > > > > > > > > >+    a)
> > > > > > > > > >+            arch=$OPTARG
> > > > > > > > > >+            ;;
> > > > > > > > > >+    B)
> > > > > > > > > >+            build=
> > > > > > > > > >+            ;;
> > > > > > > > > >+    k)
> > > > > > > > > >+            kvm="-enable-kvm"
> > > > > > > > > >+            ;;
> > > > > > > > > >+    o)
> > > > > > > > > >+            os=$OPTARG
> > > > > > > > > >+
> > > > > > > > > >+            # Expand memory and CPUs
> > > > > > > > > >+            extra+=" -m 4G -smp 4"
> > > > > > > > > >+            ;;
> > > > > > > > > >+    r)
> > > > > > > > > >+            run=1
> > > > > > > > > >+            ;;
> > > > > > > > > >+    R)
> > > > > > > > > >+            release=$OPTARG
> > > > > > > > > >+            ;;
> > > > > > > > > >+    s)
> > > > > > > > > >+            serial=1
> > > > > > > > > >+            ;;
> > > > > > > > > >+    w)
> > > > > > > > > >+            bitness=32
> > > > > > > > > >+            ;;
> > > > > > > > > >+    *)
> > > > > > > > > >+            usage
> > > > > > > > > >+            ;;
> > > > > > > > > >+    esac
> > > > > > > > > >+done
> > > > > > > > > >+
> > > > > > > > > >+# Build U-Boot for the selected board
> > > > > > > > > >+build_u_boot() {
> > > > > > > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > > > > > > >+}
> > > > > > > > > >+
> > > > > > > > > >+# Run QEMU with U-Boot
> > > > > > > > > >+run_qemu() {
> > > > > > > > > >+    if [[ -n "${os_image}" ]]; then
> > > > > > > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > > > > > > >+    fi
> > > > > > > > > >+    if [[ -n "${serial}" ]]; then
> > > > > > > > > >+            extra+=" -display none -serial mon:stdio"
> > > > > > > > > >+    else
> > > > > > > > > >+            extra+=" -serial mon:stdio"
> > > > > > > > > >+    fi
> > > > > > > > > >+    echo "Running ${qemu} ${extra}"
> > > > > > > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > > > > > > >+            -m 512 \
> > > > > > > > >
> > > > > > > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > > > > > > >
> > > > > > > > > >+            -nic none \
> > > > > > > > >
> > > > > > > > > Who wants to run without network?
> > > > > > > > > Use the virtio nic.
> > > > > > > > >
> > > > > > > > > >+            ${kvm} \
> > > > > > > > > >+            ${extra}
> > > > > > > > > >+}
> > > > > > > > > >+
> > > > > > > > > >+# Check architecture
> > > > > > > > > >+case "${arch}" in
> > > > > > > > > >+arm)
> > > > > > > > > >+    BOARD="qemu_arm"
> > > > > > > > > >+    BIOS="u-boot.bin"
> > > > > > > > > >+    qemu=qemu-system-arm
> > > > > > > > > >+    extra+=" -machine virt"
> > > > > > > > > >+    suffix="arm"
> > > > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > > > >+            BOARD="qemu_arm64"
> > > > > > > > > >+            qemu=qemu-system-aarch64
> > > > > > > > > >+            extra+=" -cpu cortex-a57"
> > > > > > > > >
> > > > > > > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > > > > > > >
> > > > > > > > > -cpu max works fine with both tcg and kvm.
> > > > > > > > >
> > > > > > > > > >+            suffix="arm64"
> > > > > > > > >
> > > > > > > > > >+    fi
> > > > > > > > > >+    ;;
> > > > > > > > > >+x86)
> > > > > > > > > >+    BOARD="qemu-x86"
> > > > > > > > > >+    BIOS="u-boot.rom"
> > > > > > > > > >+    qemu=qemu-system-i386
> > > > > > > > > >+    suffix="i386"
> > > > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > > > >+            BOARD="qemu-x86_64"
> > > > > > > > > >+            qemu=qemu-system-x86_64
> > > > > > > > > >+            suffix="amd64"
> > > > > > > > > >+    fi
> > > > > > > > > >+    ;;
> > > > > > > > > >+*)
> > > > > > > > > >+    usage "Unknown architecture '${arch}'"
> > > > > > > > > >+esac
> > > > > > > > > >+
> > > > > > > > > >+# Check OS
> > > > > > > > > >+case "${os}" in
> > > > > > > > > >+ubuntu)
> > > > > > > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > > > > > > >
> > > > > > > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > > > > > > >
> > > > > > > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > > > > > > >
> > > > > > > > > For testing U-Boot a server image is all it takes.
> > > > > > > >
> > > > > > > > And _all_ of this is why I don't want to add a useful personal script
> > > > > > > > as an additional tool we support. I've seen how much work goes in to the
> > > > > > > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > > > > > > something like that. Doubly so when ultimately I believe we would be
> > > > > > > > well served by having a document that says (in much more words) to look
> > > > > > > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > > > > > > architectures and platforms and to then further leverage general QEMU
> > > > > > > > tips and guides on how to run an OS of your choice with that.
> > > > > > >
> > > > > > > I'm not sure it matters that much. Everyone is going to have their
> > > > > > > preference as to how this script should look, but no one else has
> > > > > > > taken the time to write one...
> > > > > > >
> > > > > > > People are free to send patches to enhance it. But I believe it is
> > > > > > > helpful, e.g. for repeating problems caused by recent lmb patches.
> > > > > > >
> > > > > > > Re the test hooks, I just get tired of looking them up and trying to
> > > > > > > figure out what to do. Every board name and arch is slightly
> > > > > > > different. Just a hassle that I don't need.
> > > > > > >
> > > > > > > We can put it in scripts/contrib if you like.
> > > > > >
> > > > > > It's handy to point people to scripts, yes. I frequently point people at
> > > > > > my wrappers around buildman for example for "how do I find code bloat?"
> > > > > > and similar. But no, I don't think this rises to the level of
> > > > > > "scripts/contrib".
> > > > >
> > > > > Coming back to this question again. I'd be willing to make a new
> > > > > top-level repository for "contributor tooling" and also make that more
> > > > > widely writable. But I also think you're underestimating the level of
> > > > > work required to have a "generic" script here that works on arbitrary
> > > > > developer machines.
> > > >
> > > > Having it in a separate repo seems like too much of a pain, to me.
> > > > When things change in U-Boot I would want to update the script (e.g.
> > > > to add UPL support, booting Ubuntu and the like).
> > >
> > > Being external means it's easier to use for bisect'ing problems and you
> > > still have to handle UPL / no UPL and so on.
> >
> > There are trade-offs, for sure. Sometimes I use buildman from a
> > separate tree when trying to debug something which changes buildman.
> > But I don't think that warrants creating an entirely new tree for
> > scripts.
>
> It would also encourage others. I for example might put the scripts I
> use for having buildman do various tasks there as well.

OK well I think we've reached a dead end here. If you'd like to create
a repo for it, please do. I can then try it out and see how useful it
is for me. I wonder if it should share the u-boot-test-hooks thing to
reduce the number of repos? Rename to u-boot-extra ?

For u-boot-test-hooks at some point you said you don't want to take
patches specific to my setup, so it has become a complete mess on my
side, with different versions for upstream Labgrid, etc. Just some
feedback...

Regards,
Simon

Re: [PATCH 01/18] scripts: Add a script for building and booting QEMU

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 20221 bytes --]

On Fri, Jan 10, 2025 at 12:50:20PM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Fri, 10 Jan 2025 at 09:01, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Fri, Jan 10, 2025 at 06:39:19AM -0700, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Thu, 9 Jan 2025 at 07:56, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Thu, Jan 09, 2025 at 05:36:34AM -0700, Simon Glass wrote:
> > > > > Hi Tom,
> > > > >
> > > > > On Mon, 6 Jan 2025 at 07:55, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Fri, Nov 15, 2024 at 09:14:03AM -0600, Tom Rini wrote:
> > > > > > > On Fri, Nov 15, 2024 at 07:21:47AM -0700, Simon Glass wrote:
> > > > > > > > Hi Tom,
> > > > > > > >
> > > > > > > > On Tue, 12 Nov 2024 at 17:54, Tom Rini <trini@konsulko.com> wrote:
> > > > > > > > >
> > > > > > > > > On Wed, Nov 13, 2024 at 01:49:30AM +0100, Heinrich Schuchardt wrote:
> > > > > > > > > > Am 12. November 2024 14:58:54 MEZ schrieb Simon Glass <sjg@chromium.org>:
> > > > > > > > > > >It is handy to be able to quickly build and boot a QEMU image for a
> > > > > > > > > > >particular architecture and distro.
> > > > > > > > > > >
> > > > > > > > > > >Add a script for this purpose. It supports only arm and x86 at present.
> > > > > > > > > > >For distros it only supports Ubuntu. Both 32- and 64-bit builds are
> > > > > > > > > > >supported.
> > > > > > > > > > >
> > > > > > > > > > >Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > > > > >---
> > > > > > > > > > >
> > > > > > > > > > > MAINTAINERS                    |   8 ++
> > > > > > > > > > > doc/board/emulation/index.rst  |   1 +
> > > > > > > > > > > doc/board/emulation/script.rst |  61 ++++++++++++
> > > > > > > > > > > scripts/build-qemu.sh          | 175 +++++++++++++++++++++++++++++++++
> > > > > > > > > > > 4 files changed, 245 insertions(+)
> > > > > > > > > > > create mode 100644 doc/board/emulation/script.rst
> > > > > > > > > > > create mode 100755 scripts/build-qemu.sh
> > > > > > > > > > >
> > > > > > > > > > >diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > > > > > >index 0399ed1dbf6..b45bb96d5a5 100644
> > > > > > > > > > >--- a/MAINTAINERS
> > > > > > > > > > >+++ b/MAINTAINERS
> > > > > > > > > > >@@ -1110,6 +1110,14 @@ F:    tools/efivar.py
> > > > > > > > > > > F:  tools/file2include.c
> > > > > > > > > > > F:  tools/mkeficapsule.c
> > > > > > > > > > >
> > > > > > > > > > >+EMULATION
> > > > > > > > > > >+M:  Simon Glass <sjg@chromium.org>
> > > > > > > > > > >+S:  Maintained
> > > > > > > > > > >+W:  https://docs.u-boot.org/en/latest/board/emulation/script.html
> > > > > > > > > > >+F:  configs/qemu_x86*
> > > > > > > > > > >+F:  doc/board/emulation/script.rst
> > > > > > > > > > >+F:  scripts/build-qemu.sh
> > > > > > > > > >
> > > > > > > > > > Please, avoid misnomers. This script does not build QEMU.
> > > > > > > > > >
> > > > > > > > > > >+
> > > > > > > > > > > ENVIRONMENT
> > > > > > > > > > > M:  Joe Hershberger <joe.hershberger@ni.com>
> > > > > > > > > > > S:  Maintained
> > > > > > > > > > >diff --git a/doc/board/emulation/index.rst b/doc/board/emulation/index.rst
> > > > > > > > > > >index f8908166276..5a2a00ae225 100644
> > > > > > > > > > >--- a/doc/board/emulation/index.rst
> > > > > > > > > > >+++ b/doc/board/emulation/index.rst
> > > > > > > > > > >@@ -8,6 +8,7 @@ Emulation
> > > > > > > > > > >
> > > > > > > > > > >    acpi
> > > > > > > > > > >    blkdev
> > > > > > > > > > >+   script
> > > > > > > > > > >    qemu-arm
> > > > > > > > > > >    qemu-mips
> > > > > > > > > > >    qemu-ppce500
> > > > > > > > > > >diff --git a/doc/board/emulation/script.rst b/doc/board/emulation/script.rst
> > > > > > > > > >
> > > > > > > > > > Just another misnomer. This page is not about script.sh.
> > > > > > > > > >
> > > > > > > > > > >new file mode 100644
> > > > > > > > > > >index 00000000000..23981e333cb
> > > > > > > > > > >--- /dev/null
> > > > > > > > > > >+++ b/doc/board/emulation/script.rst
> > > > > > > > > > >@@ -0,0 +1,61 @@
> > > > > > > > > > >+.. SPDX-License-Identifier: GPL-2.0+
> > > > > > > > > >
> > > > > > > > > > This is not a valid SPDX identifier.
> > > > > > > > > >
> > > > > > > > > > >+
> > > > > > > > > > >+Script for building and running
> > > > > > > > > > >+===============================
> > > > > > > > > > >+
> > > > > > > > > > >+You may find the script `scripts/build-qemu.sh` helpful for building and testing
> > > > > > > > > > >+U-Boot on QEMU.
> > > > > > > > > > >+
> > > > > > > > > > >+If uses a environment variables to control how it works:
> > > > > > > > > > >+
> > > > > > > > > > >+ubdir
> > > > > > > > > > >+    base directory for building U-Boot, with each board being in its own
> > > > > > > > > > >+    subdirectory
> > > > > > > > > > >+
> > > > > > > > > > >+imagedir
> > > > > > > > > > >+    directory containing OS images, containin a subdirectory for each distro
> > > > > > > > > > >+    type (e.g. ubuntu/
> > > > > > > > > > >+
> > > > > > > > > > >+Once configured, you can build and run QEMU for arm64 like this::
> > > > > > > > > >
> > > > > > > > > > This downloads the QEMU source and builds it?
> > > > > > > > > >
> > > > > > > > > > >+
> > > > > > > > > > >+    scripts/build-qemu.sh -rsw
> > > > > > > > > > >+
> > > > > > > > > > >+No support is currently included for specifying a root disk, so this script can
> > > > > > > > > > >+only be used to start installers.
> > > > > > > > > > >+
> > > > > > > > > > >+Options
> > > > > > > > > > >+~~~~~~~
> > > > > > > > > > >+
> > > > > > > > > > >+Options are available to control the script:
> > > > > > > > > > >+
> > > > > > > > > > >+-a <arch>
> > > > > > > > > > >+    Select architecture (default arm, x86)
> > > > > > > > > > >+
> > > > > > > > > > >+-B
> > > > > > > > > > >+    Don't build; assume a build exists
> > > > > > > > > > >+
> > > > > > > > > > >+-k
> > > > > > > > > > >+    Use kvm - kernel-based Virtual Machine. By default QEMU uses its own
> > > > > > > > > > >+    emulator
> > > > > > > > > > >+
> > > > > > > > > > >+-o <os>
> > > > > > > > > > >+    Run an Operating System. For now this only supports 'ubuntu'. The name of
> > > > > > > > > > >+    the OS file must remain unchanged from its standard name on the Ubuntu
> > > > > > > > > > >+    website.
> > > > > > > > > >
> > > > > > > > > > The U-Boot project should remain open to all operating systems. How will this work with OpenBSD?
> > > > > > > > > >
> > > > > > > > > > Use the URL of the image as argument.
> > > > > > > > > >
> > > > > > > > > > >+
> > > > > > > > > > >+-r
> > > > > > > > > > >+    Run QEMU with the image (by default this is not done)
> > > > > > > > > > >+
> > > > > > > > > > >+-R
> > > > > > > > > > >+    Select OS release (e.g. 24.04).
> > > > > > > > > > >+
> > > > > > > > > > >+-s
> > > > > > > > > > >+    Use serial only (no display)
> > > > > > > > > > >+
> > > > > > > > > > >+-w
> > > > > > > > > > >+    Use word version (32-bit). By default, 64-bit is used
> > > > > > > > > >
> > > > > > > > > > "word version" is not helpful as explanation.
> > > > > > > > > >
> > > > > > > > > > Look at <https://en.m.wikipedia.org/wiki/Word_(computer_architecture)> which says a word is 64 bit on a 64-bit system and 16 bit on a 16-bit system.
> > > > > > > > > >
> > > > > > > > > > >+
> > > > > > > > > > >+.. note::
> > > > > > > > > > >+
> > > > > > > > > > >+    Note: For now this is a shell script, but if it expands it might be better
> > > > > > > > > > >+    as Python, accepting the slower startup.
> > > > > > > > > > >diff --git a/scripts/build-qemu.sh b/scripts/build-qemu.sh
> > > > > > > > > > >new file mode 100755
> > > > > > > > > > >index 00000000000..0ff53593cf9
> > > > > > > > > > >--- /dev/null
> > > > > > > > > > >+++ b/scripts/build-qemu.sh
> > > > > > > > > > >@@ -0,0 +1,175 @@
> > > > > > > > > > >+#!/bin/bash
> > > > > > > > > > >+# SPDX-License-Identifier: GPL-2.0+
> > > > > > > > > >
> > > > > > > > > > This is not a valid SPDX identifier.
> > > > > > > > > >
> > > > > > > > > > >+#
> > > > > > > > > > >+# Script to build U-Boot suitable for booting with QEMU, possibly running
> > > > > > > > > > >+# it, possibly with an OS image
> > > > > > > > > > >+
> > > > > > > > > > >+# This just an example. It assumes that
> > > > > > > > > > >+
> > > > > > > > > > >+# - you build U-Boot in ${ubdir}/<name> where <name> is the U-Boot board config
> > > > > > > > > > >+# - your OS images are in ${imagedir}/{distroname}/...
> > > > > > > > > > >+
> > > > > > > > > > >+# So far the script supports only ARM and x86.
> > > > > > > > > >
> > > > > > > > > > Why support obsolete i386 but not riscv64?
> > > > > > > > > >
> > > > > > > > > > >+
> > > > > > > > > > >+set -e
> > > > > > > > > > >+
> > > > > > > > > > >+usage() {
> > > > > > > > > > >+    (
> > > > > > > > > > >+    if [[ -n "$1" ]]; then
> > > > > > > > > > >+            echo "$1"
> > > > > > > > > > >+            echo
> > > > > > > > > > >+    fi
> > > > > > > > > > >+    echo "Usage: $0 -aBkrsw"
> > > > > > > > > > >+    echo
> > > > > > > > > > >+    echo "   -a   - Select architecture (arm, x86)"
> > > > > > > > > > >+    echo "   -B   - Don't build; assume a build exists"
> > > > > > > > > > >+    echo "   -k   - Use kvm (kernel-based Virtual Machine)"
> > > > > > > > > > >+    echo "   -o   - Run Operating System ('ubuntu' only for now)"
> > > > > > > > > > >+    echo "   -r   - Run QEMU with the image"
> > > > > > > > > > >+    echo "   -R   - Select OS release (e.g. 24.04)"
> > > > > > > > > > >+    echo "   -s   - Use serial only (no display)"
> > > > > > > > > > >+    echo "   -w   - Use word version (32-bit)" ) >&2
> > > > > > > > > > >+    exit 1
> > > > > > > > > > >+}
> > > > > > > > > > >+
> > > > > > > > > > >+# Directory tree for OS images
> > > > > > > > > > >+imagedir=${imagedir-/vid/software/linux}
> > > > > > > > > > >+
> > > > > > > > > > >+# architecture (arm or x86)
> > > > > > > > > > >+arch=arm
> > > > > > > > > > >+
> > > > > > > > > > >+# 32- or 64-bit build
> > > > > > > > > > >+bitness=64
> > > > > > > > > > >+
> > > > > > > > > > >+# Build U-Boot
> > > > > > > > > > >+build=yes
> > > > > > > > > > >+
> > > > > > > > > > >+# Extra setings
> > > > > > > > > > >+extra=
> > > > > > > > > > >+
> > > > > > > > > > >+# Operating System to boot (ubuntu)
> > > > > > > > > > >+os=
> > > > > > > > > > >+
> > > > > > > > > > >+release=24.04.1
> > > > > > > > > > >+
> > > > > > > > > > >+# run the image with QEMU
> > > > > > > > > > >+run=
> > > > > > > > > > >+
> > > > > > > > > > >+# run QEMU without a display (U-Boot must be set to stdout=serial)
> > > > > > > > > > >+serial=
> > > > > > > > > > >+
> > > > > > > > > > >+# Use kvm
> > > > > > > > > > >+kvm=
> > > > > > > > > > >+
> > > > > > > > > > >+# Set ubdir to the build directory where you build U-Boot out-of-tree
> > > > > > > > > > >+# We avoid in-tree build because it gets confusing trying different builds
> > > > > > > > > > >+ubdir=${ubdir-/tmp/b}
> > > > > > > > > > >+
> > > > > > > > > > >+while getopts "a:Bko:rR:sw" opt; do
> > > > > > > > > > >+    case "${opt}" in
> > > > > > > > > > >+    a)
> > > > > > > > > > >+            arch=$OPTARG
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    B)
> > > > > > > > > > >+            build=
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    k)
> > > > > > > > > > >+            kvm="-enable-kvm"
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    o)
> > > > > > > > > > >+            os=$OPTARG
> > > > > > > > > > >+
> > > > > > > > > > >+            # Expand memory and CPUs
> > > > > > > > > > >+            extra+=" -m 4G -smp 4"
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    r)
> > > > > > > > > > >+            run=1
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    R)
> > > > > > > > > > >+            release=$OPTARG
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    s)
> > > > > > > > > > >+            serial=1
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    w)
> > > > > > > > > > >+            bitness=32
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    *)
> > > > > > > > > > >+            usage
> > > > > > > > > > >+            ;;
> > > > > > > > > > >+    esac
> > > > > > > > > > >+done
> > > > > > > > > > >+
> > > > > > > > > > >+# Build U-Boot for the selected board
> > > > > > > > > > >+build_u_boot() {
> > > > > > > > > > >+    buildman -w -o $DIR --board $BOARD -I || exit $?
> > > > > > > > > > >+}
> > > > > > > > > > >+
> > > > > > > > > > >+# Run QEMU with U-Boot
> > > > > > > > > > >+run_qemu() {
> > > > > > > > > > >+    if [[ -n "${os_image}" ]]; then
> > > > > > > > > > >+            extra+=" -drive if=virtio,file=${os_image},format=raw,id=hd0"
> > > > > > > > > > >+    fi
> > > > > > > > > > >+    if [[ -n "${serial}" ]]; then
> > > > > > > > > > >+            extra+=" -display none -serial mon:stdio"
> > > > > > > > > > >+    else
> > > > > > > > > > >+            extra+=" -serial mon:stdio"
> > > > > > > > > > >+    fi
> > > > > > > > > > >+    echo "Running ${qemu} ${extra}"
> > > > > > > > > > >+    "${qemu}" -bios "$DIR/${BIOS}" \
> > > > > > > > > > >+            -m 512 \
> > > > > > > > > >
> > > > > > > > > > Ubuntu suggests 4 GiB as minimum for a desktop.
> > > > > > > > > >
> > > > > > > > > > >+            -nic none \
> > > > > > > > > >
> > > > > > > > > > Who wants to run without network?
> > > > > > > > > > Use the virtio nic.
> > > > > > > > > >
> > > > > > > > > > >+            ${kvm} \
> > > > > > > > > > >+            ${extra}
> > > > > > > > > > >+}
> > > > > > > > > > >+
> > > > > > > > > > >+# Check architecture
> > > > > > > > > > >+case "${arch}" in
> > > > > > > > > > >+arm)
> > > > > > > > > > >+    BOARD="qemu_arm"
> > > > > > > > > > >+    BIOS="u-boot.bin"
> > > > > > > > > > >+    qemu=qemu-system-arm
> > > > > > > > > > >+    extra+=" -machine virt"
> > > > > > > > > > >+    suffix="arm"
> > > > > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > > > > >+            BOARD="qemu_arm64"
> > > > > > > > > > >+            qemu=qemu-system-aarch64
> > > > > > > > > > >+            extra+=" -cpu cortex-a57"
> > > > > > > > > >
> > > > > > > > > > That CPU is 12 years old and not all distros are stuck on ARM v8.0. See <https://en.opensuse.org/Arm_architecture_support>.
> > > > > > > > > >
> > > > > > > > > > -cpu max works fine with both tcg and kvm.
> > > > > > > > > >
> > > > > > > > > > >+            suffix="arm64"
> > > > > > > > > >
> > > > > > > > > > >+    fi
> > > > > > > > > > >+    ;;
> > > > > > > > > > >+x86)
> > > > > > > > > > >+    BOARD="qemu-x86"
> > > > > > > > > > >+    BIOS="u-boot.rom"
> > > > > > > > > > >+    qemu=qemu-system-i386
> > > > > > > > > > >+    suffix="i386"
> > > > > > > > > > >+    if [[ "${bitness}" == "64" ]]; then
> > > > > > > > > > >+            BOARD="qemu-x86_64"
> > > > > > > > > > >+            qemu=qemu-system-x86_64
> > > > > > > > > > >+            suffix="amd64"
> > > > > > > > > > >+    fi
> > > > > > > > > > >+    ;;
> > > > > > > > > > >+*)
> > > > > > > > > > >+    usage "Unknown architecture '${arch}'"
> > > > > > > > > > >+esac
> > > > > > > > > > >+
> > > > > > > > > > >+# Check OS
> > > > > > > > > > >+case "${os}" in
> > > > > > > > > > >+ubuntu)
> > > > > > > > > > >+    os_image="${imagedir}/${os}/${os}-${release}-desktop-${suffix}.iso"
> > > > > > > > > >
> > > > > > > > > > There is no ARM 32-bit Ubuntu desktop. And for i386 there is no image in 24.04.1.
> > > > > > > > > >
> > > > > > > > > > Running a foreign architecture desktop with tcg is not enjoyable.
> > > > > > > > > >
> > > > > > > > > > For testing U-Boot a server image is all it takes.
> > > > > > > > >
> > > > > > > > > And _all_ of this is why I don't want to add a useful personal script
> > > > > > > > > as an additional tool we support. I've seen how much work goes in to the
> > > > > > > > > OpenEmbedded runqemu script, we don't have the spare cycles for
> > > > > > > > > something like that. Doubly so when ultimately I believe we would be
> > > > > > > > > well served by having a document that says (in much more words) to look
> > > > > > > > > at u-boot-test-hooks for how to invoke QEMU for a large number of
> > > > > > > > > architectures and platforms and to then further leverage general QEMU
> > > > > > > > > tips and guides on how to run an OS of your choice with that.
> > > > > > > >
> > > > > > > > I'm not sure it matters that much. Everyone is going to have their
> > > > > > > > preference as to how this script should look, but no one else has
> > > > > > > > taken the time to write one...
> > > > > > > >
> > > > > > > > People are free to send patches to enhance it. But I believe it is
> > > > > > > > helpful, e.g. for repeating problems caused by recent lmb patches.
> > > > > > > >
> > > > > > > > Re the test hooks, I just get tired of looking them up and trying to
> > > > > > > > figure out what to do. Every board name and arch is slightly
> > > > > > > > different. Just a hassle that I don't need.
> > > > > > > >
> > > > > > > > We can put it in scripts/contrib if you like.
> > > > > > >
> > > > > > > It's handy to point people to scripts, yes. I frequently point people at
> > > > > > > my wrappers around buildman for example for "how do I find code bloat?"
> > > > > > > and similar. But no, I don't think this rises to the level of
> > > > > > > "scripts/contrib".
> > > > > >
> > > > > > Coming back to this question again. I'd be willing to make a new
> > > > > > top-level repository for "contributor tooling" and also make that more
> > > > > > widely writable. But I also think you're underestimating the level of
> > > > > > work required to have a "generic" script here that works on arbitrary
> > > > > > developer machines.
> > > > >
> > > > > Having it in a separate repo seems like too much of a pain, to me.
> > > > > When things change in U-Boot I would want to update the script (e.g.
> > > > > to add UPL support, booting Ubuntu and the like).
> > > >
> > > > Being external means it's easier to use for bisect'ing problems and you
> > > > still have to handle UPL / no UPL and so on.
> > >
> > > There are trade-offs, for sure. Sometimes I use buildman from a
> > > separate tree when trying to debug something which changes buildman.
> > > But I don't think that warrants creating an entirely new tree for
> > > scripts.
> >
> > It would also encourage others. I for example might put the scripts I
> > use for having buildman do various tasks there as well.
> 
> OK well I think we've reached a dead end here. If you'd like to create
> a repo for it, please do. I can then try it out and see how useful it
> is for me. I wonder if it should share the u-boot-test-hooks thing to
> reduce the number of repos? Rename to u-boot-extra ?

I've created a "u-boot-extras" repository at the top level, added you
as well to the maintainers (aside, any other custodians that want to
contribute scripts/etc here are welcome to DM me for write access), and
added "u-boot-size-test.sh" to "contrib/trini". Please feel free to
start populating "contrib/sjg" with any/all scripts you feel are ready
for public viewing / use.

> For u-boot-test-hooks at some point you said you don't want to take
> patches specific to my setup, so it has become a complete mess on my
> side, with different versions for upstream Labgrid, etc. Just some
> feedback...

FWIW, your labgrid is in u-boot-test-hooks as well and has been for a
while. I had to rebase my local branches on top of that, but it was
mostly progmatic sed'ing. I'm still less clear on the value of putting
all of the lab details in a public repository instead of just example
levels, but I'm open to changing my mind there if that will really help.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

[PATCH 02/18] x86: Expand x86_64 early memory

[Simon Glass]

The SPL and pre-reloc malloc()-space is not large enough to start up
with a display. Expand it.

Switch the order of SPL_SYS_MALLOC_F_LEN and SPL_TEXT_BASE since this
matches what 'savedefconfig' gives us.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 configs/qemu-x86_64_defconfig | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/configs/qemu-x86_64_defconfig b/configs/qemu-x86_64_defconfig
index 008eb46a01c..218c3535d15 100644
--- a/configs/qemu-x86_64_defconfig
+++ b/configs/qemu-x86_64_defconfig
@@ -1,13 +1,13 @@
 CONFIG_X86=y
 CONFIG_TEXT_BASE=0x1110000
-CONFIG_SYS_MALLOC_F_LEN=0x1000
+CONFIG_SYS_MALLOC_F_LEN=0x1800
 CONFIG_NR_DRAM_BANKS=8
 CONFIG_ENV_SIZE=0x40000
 CONFIG_MAX_CPUS=2
 CONFIG_SPL_DM_SPI=y
 CONFIG_DEFAULT_DEVICE_TREE="qemu-x86_i440fx"
+CONFIG_SPL_SYS_MALLOC_F_LEN=0x4800
 CONFIG_SPL_TEXT_BASE=0xfffd4000
-CONFIG_SPL_SYS_MALLOC_F_LEN=0x2000
 CONFIG_DEBUG_UART_BASE=0x3f8
 CONFIG_DEBUG_UART_CLOCK=1843200
 CONFIG_X86_RUN_64BIT=y
-- 
2.34.1

[PATCH 03/18] RFC: x86: qemu: Switch to bochs display

[Simon Glass]

The vesa display is widely used on hardware, but it is a bit of a pain
with QEMU. It requires executing option ROMs, which either doesn't work
with kvm, or is difficult to do in a kvm/QEMU-friendly way.

THe bochs display is probably better anyway, so switch to that. It works
fine with kvm as it doesn't need an option ROM.

Unfortunately this causes the Ubuntu 22.04 installer to stop booting,
which needs further investigation

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/31
---

 configs/qemu-x86_64_defconfig | 5 ++---
 configs/qemu-x86_defconfig    | 5 ++---
 2 files changed, 4 insertions(+), 6 deletions(-)

diff --git a/configs/qemu-x86_64_defconfig b/configs/qemu-x86_64_defconfig
index 218c3535d15..1670c122002 100644
--- a/configs/qemu-x86_64_defconfig
+++ b/configs/qemu-x86_64_defconfig
@@ -81,9 +81,8 @@ CONFIG_SYS_NS16550_PORT_MAPPED=y
 CONFIG_SPI=y
 CONFIG_USB_KEYBOARD=y
 CONFIG_CONSOLE_TRUETYPE=y
-CONFIG_FRAMEBUFFER_SET_VESA_MODE=y
-CONFIG_FRAMEBUFFER_VESA_MODE_USER=y
-CONFIG_FRAMEBUFFER_VESA_MODE=0x144
+CONFIG_VIDEO_BOCHS=y
+# CONFIG_VIDEO_VESA is not set
 CONFIG_CONSOLE_SCROLL_LINES=5
 CONFIG_SPL_VIDEO=y
 # CONFIG_SPL_USE_TINY_PRINTF is not set
diff --git a/configs/qemu-x86_defconfig b/configs/qemu-x86_defconfig
index 947d15cd727..40c2f1cd362 100644
--- a/configs/qemu-x86_defconfig
+++ b/configs/qemu-x86_defconfig
@@ -58,9 +58,8 @@ CONFIG_SYS_NS16550_PORT_MAPPED=y
 CONFIG_SPI=y
 CONFIG_USB_KEYBOARD=y
 CONFIG_CONSOLE_TRUETYPE=y
-CONFIG_FRAMEBUFFER_SET_VESA_MODE=y
-CONFIG_FRAMEBUFFER_VESA_MODE_USER=y
-CONFIG_FRAMEBUFFER_VESA_MODE=0x144
+CONFIG_VIDEO_BOCHS=y
+# CONFIG_VIDEO_VESA is not set
 CONFIG_CONSOLE_SCROLL_LINES=5
 CONFIG_GENERATE_ACPI_TABLE=y
 # CONFIG_GZIP is not set
-- 
2.34.1

[PATCH 04/18] x86: qemu: Enable dhrystone

[Simon Glass]

Provide the 'dhry' command, which helps to check that kvm is being used
properly with QEMU.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 configs/qemu-x86_64_defconfig | 1 +
 configs/qemu-x86_defconfig    | 1 +
 2 files changed, 2 insertions(+)

diff --git a/configs/qemu-x86_64_defconfig b/configs/qemu-x86_64_defconfig
index 1670c122002..151c5f3e0e7 100644
--- a/configs/qemu-x86_64_defconfig
+++ b/configs/qemu-x86_64_defconfig
@@ -87,4 +87,5 @@ CONFIG_CONSOLE_SCROLL_LINES=5
 CONFIG_SPL_VIDEO=y
 # CONFIG_SPL_USE_TINY_PRINTF is not set
 CONFIG_GENERATE_ACPI_TABLE=y
+CONFIG_CMD_DHRYSTONE=y
 # CONFIG_GZIP is not set
diff --git a/configs/qemu-x86_defconfig b/configs/qemu-x86_defconfig
index 40c2f1cd362..5c5cacd734c 100644
--- a/configs/qemu-x86_defconfig
+++ b/configs/qemu-x86_defconfig
@@ -62,4 +62,5 @@ CONFIG_VIDEO_BOCHS=y
 # CONFIG_VIDEO_VESA is not set
 CONFIG_CONSOLE_SCROLL_LINES=5
 CONFIG_GENERATE_ACPI_TABLE=y
+CONFIG_CMD_DHRYSTONE=y
 # CONFIG_GZIP is not set
-- 
2.34.1

[PATCH 05/18] x86: qemu: Avoid accessing BSS too early

[Simon Glass]

BSS is placed in DRAM which is actually available early with QEMU. But
it is cleared by the init sequence, so values stored there are lost.

Move the system-type flag into a function, instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/cpu/qemu/qemu.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/arch/x86/cpu/qemu/qemu.c b/arch/x86/cpu/qemu/qemu.c
index 563f63e2bc8..e846ccd44aa 100644
--- a/arch/x86/cpu/qemu/qemu.c
+++ b/arch/x86/cpu/qemu/qemu.c
@@ -15,14 +15,21 @@
 #include <asm/arch/qemu.h>
 #include <asm/u-boot-x86.h>
 
-static bool i440fx;
-
 #if CONFIG_IS_ENABLED(QFW_PIO)
 U_BOOT_DRVINFO(x86_qfw_pio) = {
 	.name = "qfw_pio",
 };
 #endif
 
+static bool is_i440fx(void)
+{
+	u16 device;
+
+	pci_read_config16(PCI_BDF(0, 0, 0), PCI_DEVICE_ID, &device);
+
+	return device == PCI_DEVICE_ID_INTEL_82441;
+}
+
 static void enable_pm_piix(void)
 {
 	u8 en;
@@ -50,16 +57,17 @@ static void enable_pm_ich9(void)
 
 void qemu_chipset_init(void)
 {
-	u16 device, xbcs;
+	bool i440fx;
+	u16 xbcs;
 	int pam, i;
 
+	i440fx = is_i440fx();
+
 	/*
 	 * i440FX and Q35 chipset have different PAM register offset, but with
 	 * the same bitfield layout. Here we determine the offset based on its
 	 * PCI device ID.
 	 */
-	pci_read_config16(PCI_BDF(0, 0, 0), PCI_DEVICE_ID, &device);
-	i440fx = (device == PCI_DEVICE_ID_INTEL_82441);
 	pam = i440fx ? I440FX_PAM : Q35_PAM;
 
 	/*
@@ -123,7 +131,7 @@ int mp_determine_pci_dstirq(int bus, int dev, int func, int pirq)
 {
 	u8 irq;
 
-	if (i440fx) {
+	if (is_i440fx()) {
 		/*
 		 * Not like most x86 platforms, the PIRQ[A-D] on PIIX3 are not
 		 * connected to I/O APIC INTPIN#16-19. Instead they are routed
-- 
2.34.1

[PATCH 06/18] x86: Drop mpspec from the SPL build

[Simon Glass]

This is not needed in SPL, so drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/lib/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
index 43e6a1de77d..a908356e8a6 100644
--- a/arch/x86/lib/Makefile
+++ b/arch/x86/lib/Makefile
@@ -26,7 +26,9 @@ obj-y	+= e820.o
 obj-y	+= init_helpers.o
 obj-y	+= interrupts.o
 obj-y	+= lpc-uclass.o
+ifndef CONFIG_XPL_BUILD
 obj-y	+= mpspec.o
+endif
 obj-$(CONFIG_$(PHASE_)ACPIGEN) += acpi_nhlt.o
 obj-y	+= northbridge-uclass.o
 obj-$(CONFIG_I8259_PIC) += i8259.o
-- 
2.34.1

[PATCH 07/18] x86: Add some log categories

[Simon Glass]

Add categories for i8259 and bios files, so that log statements have the
right category.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/lib/bios.c            | 3 +++
 arch/x86/lib/bios_interrupts.c | 2 ++
 arch/x86/lib/i8259.c           | 2 ++
 3 files changed, 7 insertions(+)

diff --git a/arch/x86/lib/bios.c b/arch/x86/lib/bios.c
index 03f7360032c..acae635cf19 100644
--- a/arch/x86/lib/bios.c
+++ b/arch/x86/lib/bios.c
@@ -5,6 +5,9 @@
  * Copyright (C) 2007 Advanced Micro Devices, Inc.
  * Copyright (C) 2009-2010 coresystems GmbH
  */
+
+#define LOG_CATEGRORY	LOGC_ARCH
+
 #include <compiler.h>
 #include <bios_emul.h>
 #include <irq_func.h>
diff --git a/arch/x86/lib/bios_interrupts.c b/arch/x86/lib/bios_interrupts.c
index b2cf1527b1c..3ae6c193ec7 100644
--- a/arch/x86/lib/bios_interrupts.c
+++ b/arch/x86/lib/bios_interrupts.c
@@ -7,6 +7,8 @@
  * Copyright (C) 2007-2009 coresystems GmbH
  */
 
+#define LOG_CATEGRORY	LOGC_ARCH
+
 #include <log.h>
 #include <asm/pci.h>
 #include "bios_emul.h"
diff --git a/arch/x86/lib/i8259.c b/arch/x86/lib/i8259.c
index 465ff70146f..088f78f4661 100644
--- a/arch/x86/lib/i8259.c
+++ b/arch/x86/lib/i8259.c
@@ -13,6 +13,8 @@
  * Programmable Interrupt Controllers.
  */
 
+#define LOG_CATEGORY	UCLASS_IRQ
+
 #include <log.h>
 #include <asm/io.h>
 #include <asm/i8259.h>
-- 
2.34.1

[PATCH 08/18] x86: Drop use of CONFIG_REALMODE_DEBUG

[Simon Glass]

This option is not actually defined in Kconfig anymore. Use a normal
debug print instead, which has a similar effect.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/lib/bios.c            | 18 ++++++++----------
 arch/x86/lib/bios_interrupts.c |  6 ++----
 2 files changed, 10 insertions(+), 14 deletions(-)

diff --git a/arch/x86/lib/bios.c b/arch/x86/lib/bios.c
index acae635cf19..5dfe5a647eb 100644
--- a/arch/x86/lib/bios.c
+++ b/arch/x86/lib/bios.c
@@ -301,16 +301,14 @@ asmlinkage int interrupt_handler(u32 intnumber, u32 gsfs, u32 dses,
 	cs = cs_ip >> 16;
 	flags = stackflags;
 
-#ifdef CONFIG_REALMODE_DEBUG
-	debug("oprom: INT# 0x%x\n", intnumber);
-	debug("oprom: eax: %08x ebx: %08x ecx: %08x edx: %08x\n",
-	      eax, ebx, ecx, edx);
-	debug("oprom: ebp: %08x esp: %08x edi: %08x esi: %08x\n",
-	      ebp, esp, edi, esi);
-	debug("oprom:  ip: %04x      cs: %04x   flags: %08x\n",
-	      ip, cs, flags);
-	debug("oprom: stackflags = %04x\n", stackflags);
-#endif
+	log_debug("oprom: INT# 0x%x\n", intnumber);
+	log_debug("oprom: eax: %08x ebx: %08x ecx: %08x edx: %08x\n",
+		  eax, ebx, ecx, edx);
+	log_debug("oprom: ebp: %08x esp: %08x edi: %08x esi: %08x\n",
+		  ebp, esp, edi, esi);
+	log_debug("oprom:  ip: %04x      cs: %04x   flags: %08x\n",
+		  ip, cs, flags);
+	log_debug("oprom: stackflags = %04x\n", stackflags);
 
 	/*
 	 * Fetch arguments from the stack and put them to a place
diff --git a/arch/x86/lib/bios_interrupts.c b/arch/x86/lib/bios_interrupts.c
index 3ae6c193ec7..e0c2284a901 100644
--- a/arch/x86/lib/bios_interrupts.c
+++ b/arch/x86/lib/bios_interrupts.c
@@ -200,10 +200,8 @@ int int1a_handler(void)
 			dm_pci_write_config32(dev, reg, dword);
 			break;
 		}
-#ifdef CONFIG_REALMODE_DEBUG
-		debug("0x%x: bus %d devfn 0x%x reg 0x%x val 0x%x\n", func,
-		      bus, devfn, reg, M.x86.R_ECX);
-#endif
+		log_debug("0x%x: bus %d devfn 0x%x reg 0x%x val 0x%x\n", func,
+			  bus, devfn, reg, M.x86.R_ECX);
 		M.x86.R_EAX &= 0xffff00ff; /* Clear AH */
 		M.x86.R_EAX |= PCIBIOS_SUCCESSFUL;
 		retval = 1;
-- 
2.34.1

[PATCH 09/18] x86: Avoid clearing the VESA display

[Simon Glass]

U-Boot clears the display when it starts up, so there is no need to ask
the VESA driver to do this. Fix this and add a comment explaining the
flags.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/lib/bios.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/lib/bios.c b/arch/x86/lib/bios.c
index 5dfe5a647eb..de4578666fb 100644
--- a/arch/x86/lib/bios.c
+++ b/arch/x86/lib/bios.c
@@ -231,7 +231,11 @@ static void vbe_set_graphics(int vesa_mode, struct vesa_state *mode_info)
 {
 	unsigned char *framebuffer;
 
-	mode_info->video_mode = (1 << 14) | vesa_mode;
+	/*
+	 * bit 14 is linear-framebuffer mode
+	 * bit 15 means don't clear the display
+	 */
+	mode_info->video_mode = (1 << 14) | (1 << 15) | vesa_mode;
 	vbe_get_mode_info(mode_info);
 
 	framebuffer = (unsigned char *)(ulong)mode_info->vesa.phys_base_ptr;
-- 
2.34.1

[PATCH 10/18] x86: Add 64-bit entries to the GDT

[Simon Glass]

At present it is not possible to execution 64-bit code without
installing an entire new Global Descriptor Table. This is inconvenient
since kvm does not seem to like switching into long mode with a new
table.

It isn't actually necessary, since we can just extend the existing
table. Add some new entries to this effect.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/cpu/i386/cpu.c          | 3 +++
 arch/x86/include/asm/processor.h | 5 ++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/x86/cpu/i386/cpu.c b/arch/x86/cpu/i386/cpu.c
index a51a24498a7..3bbad3b2eca 100644
--- a/arch/x86/cpu/i386/cpu.c
+++ b/arch/x86/cpu/i386/cpu.c
@@ -160,6 +160,9 @@ void arch_setup_gd(gd_t *new_gd)
 
 	gdt_addr[X86_GDT_ENTRY_16BIT_FLAT_CS] = GDT_ENTRY(0x809b, 0, 0xfffff);
 	gdt_addr[X86_GDT_ENTRY_16BIT_FLAT_DS] = GDT_ENTRY(0x8093, 0, 0xfffff);
+	gdt_addr[X86_GDT_ENTRY_64BIT_CS] = GDT_ENTRY(0xaf9b, 0, 0xfffff);
+	gdt_addr[X86_GDT_ENTRY_64BIT_TS1] = GDT_ENTRY(0x8980, 0, 0xfffff);
+	gdt_addr[X86_GDT_ENTRY_64BIT_TS2] = 0;
 
 	load_gdt(gdt_addr, X86_GDT_NUM_ENTRIES);
 	load_ds(X86_GDT_ENTRY_32BIT_DS);
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index d7b68367861..ad8240be387 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -18,7 +18,10 @@
 #define X86_GDT_ENTRY_16BIT_DS		6
 #define X86_GDT_ENTRY_16BIT_FLAT_CS	7
 #define X86_GDT_ENTRY_16BIT_FLAT_DS	8
-#define X86_GDT_NUM_ENTRIES		9
+#define X86_GDT_ENTRY_64BIT_CS		9
+#define X86_GDT_ENTRY_64BIT_TS1		10
+#define X86_GDT_ENTRY_64BIT_TS2		11
+#define X86_GDT_NUM_ENTRIES		12
 
 #define X86_GDT_SIZE		(X86_GDT_NUM_ENTRIES * X86_GDT_ENTRY_SIZE)
 
-- 
2.34.1

[PATCH 11/18] x86: Use defines for the cache flags

[Simon Glass]

Use some named flags when setting up the cache, so it is easier to see
what is going on.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/cpu/i386/cpu.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/arch/x86/cpu/i386/cpu.c b/arch/x86/cpu/i386/cpu.c
index 3bbad3b2eca..845e00ca439 100644
--- a/arch/x86/cpu/i386/cpu.c
+++ b/arch/x86/cpu/i386/cpu.c
@@ -592,6 +592,13 @@ int cpu_has_64bit(void)
 #define PAGETABLE_BASE		0x80000
 #define PAGETABLE_SIZE		(6 * 4096)
 
+#define _PRES BIT(0)	/* present */
+#define _RW   BIT(1)	/* write allowed */
+#define _US   BIT(2)	/* user-access allowed */
+#define _A    BIT(5)	/* has been accessed */
+#define _D    BIT(6)	/* has been written to */
+#define _PS   BIT(7)	/* indicates 2MB page size here */
+
 /**
  * build_pagetable() - build a flat 4GiB page table structure for 64-bti mode
  *
@@ -604,15 +611,17 @@ static void build_pagetable(uint32_t *pgtable)
 	memset(pgtable, '\0', PAGETABLE_SIZE);
 
 	/* Level 4 needs a single entry */
-	pgtable[0] = (ulong)&pgtable[1024] + 7;
+	pgtable[0] = (ulong)&pgtable[1024] + _PRES + _RW + _US + _A;
 
 	/* Level 3 has one 64-bit entry for each GiB of memory */
 	for (i = 0; i < 4; i++)
-		pgtable[1024 + i * 2] = (ulong)&pgtable[2048] + 0x1000 * i + 7;
+		pgtable[1024 + i * 2] = (ulong)&pgtable[2048] + 0x1000 * i +
+			_PRES + _RW + _US + _A;
 
 	/* Level 2 has 2048 64-bit entries, each repesenting 2MiB */
 	for (i = 0; i < 2048; i++)
-		pgtable[2048 + i * 2] = 0x183 + (i << 21UL);
+		pgtable[2048 + i * 2] = _PRES + _RW + _US + _PS + _A +  _D +
+					 (i << 21UL);
 }
 
 int cpu_jump_to_64bit(ulong setup_base, ulong target)
-- 
2.34.1

[PATCH 12/18] x86: spl: Drop duplicate CPU init

[Simon Glass]

x86_cpu_init_f() is called by arch_cpu_init() a few lines below this
code. Drop the duplicate call.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/lib/spl.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/x86/lib/spl.c b/arch/x86/lib/spl.c
index f761fbc8bc3..2586a81093b 100644
--- a/arch/x86/lib/spl.c
+++ b/arch/x86/lib/spl.c
@@ -84,8 +84,6 @@ static int x86_spl_init(void)
 	log_debug("x86 spl starting\n");
 	if (IS_ENABLED(TPL))
 		ret = x86_cpu_reinit_f();
-	else
-		ret = x86_cpu_init_f();
 	ret = spl_init();
 	if (ret) {
 		log_debug("spl_init() failed (err=%d)\n", ret);
-- 
2.34.1

[PATCH 13/18] x86: Drop the message about features missing in 64-bit

[Simon Glass]

This functions normally and has done for a while, so drop this scary
message.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/lib/spl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/lib/spl.c b/arch/x86/lib/spl.c
index 2586a81093b..715c8cdc3ac 100644
--- a/arch/x86/lib/spl.c
+++ b/arch/x86/lib/spl.c
@@ -281,7 +281,7 @@ void __noreturn jump_to_image_no_args(struct spl_image_info *spl_image)
 {
 	int ret;
 
-	printf("Jumping to 64-bit U-Boot: Note many features are missing\n");
+	log_debug("Jumping to 64-bit U-Boot\n");
 	ret = cpu_jump_to_64bit_uboot(spl_image->entry_point);
 	debug("ret=%d\n", ret);
 	hang();
-- 
2.34.1

[PATCH 14/18] x86: Include stdbool.h in interrupt header

[Simon Glass]

This makes use of a 'bool' type, so include the required header.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/include/asm/interrupt.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/include/asm/interrupt.h b/arch/x86/include/asm/interrupt.h
index e23fb2c8e72..c689fc23d08 100644
--- a/arch/x86/include/asm/interrupt.h
+++ b/arch/x86/include/asm/interrupt.h
@@ -10,6 +10,7 @@
 #ifndef __ASM_INTERRUPT_H_
 #define __ASM_INTERRUPT_H_ 1
 
+#include <stdbool.h>
 #include <asm/types.h>
 
 #define SYS_NUM_IRQS	16
-- 
2.34.1

[PATCH 15/18] x86: Tidy up the GDT size in start/16.S

[Simon Glass]

Use a symbol to select the size of the GDT, rather than hard-coding a
value. This matches how it is done in start64

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/cpu/start.S   | 4 +++-
 arch/x86/cpu/start16.S | 3 ++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/x86/cpu/start.S b/arch/x86/cpu/start.S
index 0ef27cc5a00..385a691265e 100644
--- a/arch/x86/cpu/start.S
+++ b/arch/x86/cpu/start.S
@@ -254,7 +254,7 @@ multiboot_header:
 	 * GDT is setup in a safe location in RAM
 	 */
 gdt_ptr2:
-	.word	0x1f		/* limit (31 bytes = 4 GDT entries - 1) */
+	.word	gdt2_end - gdt_ptr2 - 1
 	.long	gdt_rom2	/* base */
 
 	/* Some CPUs are picky about GDT alignment... */
@@ -313,4 +313,6 @@ gdt_rom2:
 	.byte	0x93		/* access */
 	.byte	0xcf		/* flags + limit_high */
 	.byte	0x00		/* base_high */
+gdt2_end:
+
 #endif
diff --git a/arch/x86/cpu/start16.S b/arch/x86/cpu/start16.S
index 865a49731e5..8d9acb193e0 100644
--- a/arch/x86/cpu/start16.S
+++ b/arch/x86/cpu/start16.S
@@ -61,7 +61,7 @@ idt_ptr:
 	 * GDT is setup in a safe location in RAM
 	 */
 gdt_ptr:
-	.word	0x1f		/* limit (31 bytes = 4 GDT entries - 1) */
+	.word	gdt_end - gdt_rom - 1
 	.long	BOOT_SEG + gdt_rom	/* base */
 
 	/* Some CPUs are picky about GDT alignment... */
@@ -120,3 +120,4 @@ gdt_rom:
 	.byte	0x93		/* access */
 	.byte	0xcf		/* flags + limit_high */
 	.byte	0x00		/* base_high */
+gdt_end:
-- 
2.34.1

[PATCH 16/18] x86: Disable paging before changing to long mode

[Simon Glass]

This is required as part of the procedure. The existing code works
because it changes the GDT at the same time, but this makes kvm
unhappy.

Update the algorithm to disable and then re-enable paging.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/cpu/i386/call64.S | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/x86/cpu/i386/call64.S b/arch/x86/cpu/i386/call64.S
index d81bcc6f8f4..e5a0420d1ba 100644
--- a/arch/x86/cpu/i386/call64.S
+++ b/arch/x86/cpu/i386/call64.S
@@ -25,6 +25,11 @@ cpu_call64:
 	push	%edx		/* arg1 = setup_base */
 	mov	%eax, %ebx
 
+	# disable paging
+	movl	%cr0, %eax
+	andl	$~X86_CR0_PG, %eax
+	movl	%eax, %cr0
+
 	/* Load new GDT with the 64bit segments using 32bit descriptor */
 	leal	gdt, %eax
 	movl	%eax, gdt+2
@@ -67,7 +72,8 @@ cpu_call64:
 	pushl	%eax
 
 	/* Enter paged protected Mode, activating Long Mode */
-	movl	$(X86_CR0_PG | X86_CR0_PE), %eax
+	movl	%cr0, %eax
+	orl	$X86_CR0_PG, %eax
 	movl	%eax, %cr0
 
 	/* Jump from 32bit compatibility mode into 64bit mode. */
-- 
2.34.1

[PATCH 17/18] x86: Use the same GDT when jumping to long mode

[Simon Glass]

Make use the existing GDT which now includes entries for 64-bit code.
Leave the interrupt descriptors alone. They can be tidied up once U-Boot
starts up.

With this, kvm mode works with QEMU.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/31
---

 arch/x86/cpu/i386/call64.S | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)

diff --git a/arch/x86/cpu/i386/call64.S b/arch/x86/cpu/i386/call64.S
index e5a0420d1ba..3137ec17d31 100644
--- a/arch/x86/cpu/i386/call64.S
+++ b/arch/x86/cpu/i386/call64.S
@@ -7,6 +7,7 @@
  */
 
 #include <asm/msr-index.h>
+#include <asm/processor.h>
 #include <asm/processor-flags.h>
 
 .code32
@@ -30,11 +31,6 @@ cpu_call64:
 	andl	$~X86_CR0_PG, %eax
 	movl	%eax, %cr0
 
-	/* Load new GDT with the 64bit segments using 32bit descriptor */
-	leal	gdt, %eax
-	movl	%eax, gdt+2
-	lgdt	gdt
-
 	/* Enable PAE mode */
 	movl	$(X86_CR4_PAE), %eax
 	movl	%eax, %cr4
@@ -49,12 +45,6 @@ cpu_call64:
 	btsl	$_EFER_LME, %eax
 	wrmsr
 
-	/* After gdt is loaded */
-	xorl	%eax, %eax
-	lldt	%ax
-	movl    $0x20, %eax
-	ltr	%ax
-
 	/*
 	 * Setup for the jump to 64bit mode
 	 *
@@ -67,7 +57,7 @@ cpu_call64:
 	 */
 	pop	%esi			/* setup_base */
 
-	pushl	$0x10
+	pushl	$(X86_GDT_ENTRY_64BIT_CS * X86_GDT_ENTRY_SIZE)
 	leal	lret_target, %eax
 	pushl	%eax
 
-- 
2.34.1

[PATCH 18/18] x86: Use a simple jump into long mode

[Simon Glass]

With the 64-bit descriptor we can use a jump instruction, rather than
pushing things on the stack.

Since the processor is in 64-bit mode by this point, pop a 64-bit value
from the stack, containing the target address.

This simplifies the code slightly, in particular its use of the stack.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/x86/cpu/i386/call64.S | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/arch/x86/cpu/i386/call64.S b/arch/x86/cpu/i386/call64.S
index 3137ec17d31..c6308b92e25 100644
--- a/arch/x86/cpu/i386/call64.S
+++ b/arch/x86/cpu/i386/call64.S
@@ -22,6 +22,7 @@ cpu_call64:
 	 * ecx - target
 	 */
 	cli
+	pushl	$0		/* top 64-bits of target */
 	push	%ecx		/* arg2 = target */
 	push	%edx		/* arg1 = setup_base */
 	mov	%eax, %ebx
@@ -32,7 +33,8 @@ cpu_call64:
 	movl	%eax, %cr0
 
 	/* Enable PAE mode */
-	movl	$(X86_CR4_PAE), %eax
+	movl	%cr4, %eax
+	orl	$X86_CR4_PAE, %eax
 	movl	%eax, %cr4
 
 	/* Enable the boot page tables */
@@ -57,23 +59,18 @@ cpu_call64:
 	 */
 	pop	%esi			/* setup_base */
 
-	pushl	$(X86_GDT_ENTRY_64BIT_CS * X86_GDT_ENTRY_SIZE)
-	leal	lret_target, %eax
-	pushl	%eax
-
 	/* Enter paged protected Mode, activating Long Mode */
 	movl	%cr0, %eax
 	orl	$X86_CR0_PG, %eax
 	movl	%eax, %cr0
 
 	/* Jump from 32bit compatibility mode into 64bit mode. */
-	lret
+	ljmp	$(X86_GDT_ENTRY_64BIT_CS * X86_GDT_ENTRY_SIZE), $lret_target
 
-code64:
+.code64
 lret_target:
-	pop	%eax			/* target */
-	mov	%eax, %eax		/* Clear bits 63:32 */
-	jmp	*%eax			/* Jump to the 64-bit target */
+	pop	%rax			/* target */
+	jmp	*%rax			/* Jump to the 64-bit target */
 
 .globl call64_stub_size
 call64_stub_size:
-- 
2.34.1