[PATCH v2 00/39] efi: Add a test for EFI bootmeth

[PATCH v2 00/39] efi: Add a test for EFI bootmeth

[Simon Glass]

The test coverage for the EFI bootmeth is incomplete since it does not
actually boot the application.

This series creates a simple test for this purpose. It includes a
surprising number patches to make this work:

- sandbox memory-mapping conflict with PCI
- the fix for that causes the mbr test to crash as it sets up pointers
  instead of addresses for its 'mmc' commands
- the mmc and read commands which cast addresses to pointers
- ANSI output from the EFI loader confusing the unit-testing checker
- bug in bootdev_next_prio() which causes an infinite loop in EFI test
- Hang in sandbox virtio due to EFI probing all block devices
- a tricky bug to do with USB keyboard and stdio
- a few other minor things

Changes in v2:
- Fix 'use' typo
- Add many new patches to resolve all the outstanding test issues

Simon Glass (39):
  nvmxip: Drop the message on probe
  nvmxip: Avoid probing on boot
  bootstd: Add UT_TESTF_CONSOLE_REC to bootflow tests
  test/py: Fix some pylint warnings in test_ut.py
  scripts: Update pylint.base
  bootstd: Create a function to reset USB
  usb: Drop old non-DM code
  log: Add a new log category for the console
  usb: Add DEV_FLAGS_DM to stdio for USB keyboard
  dm: usb: Deal with USB keyboard persisting across tests
  test: mbr: Adjust test to use lower-case hex
  test: mbr: Adjust test to drop 0x
  sandbox: Change the range used for memory-mapping tags
  sandbox: Update cpu to use logging
  sandbox: Unmap old tags
  sandbox: Add some debugging to pci_io
  sandbox: Implement reference counting for address mapping
  mmc: Use map_sysmem() with buffers in the mmc command
  read: Use map_sysmem() with buffers in the read command
  cmd: Fix memory-mapping in cmp command
  test: mbr: Unmap the buffers after use
  test: mbr: Use a constant for the block size
  test: mbr: Use RAM for the buffers
  test: mbr: Drop a duplicate test
  efi: Use puts() in cout so that console recording works
  efi_loader: Put back copyright message
  efi_loader: Rename and move CMD_BOOTEFI_HELLO_COMPILE
  efi_loader: Shorten the app rules
  efi_loader: Shorten the app rules further
  efi: Show the vendor in helloworld
  Revert "bootdev: avoid infinite probe loop"
  bootstd: Make bootdev_next_prio() continue after failure
  efi: Use the same filename for all sandbox builds
  bootstd: Add debugging for efi bootmeth
  efi: Disable ANSI output for tests
  efi: Add a test app
  efi: Avoid using sandbox virtio devices
  test: Set up an image suitable for EFI testing
  efi: Add a test for the efi bootmeth

 arch/arm/lib/Makefile               |   2 +-
 arch/riscv/lib/Makefile             |   2 +-
 arch/sandbox/cpu/cpu.c              |  38 ++-
 arch/sandbox/cpu/state.c            |   9 +-
 arch/sandbox/dts/test.dts           |   2 +-
 arch/sandbox/include/asm/state.h    |   3 +
 arch/sandbox/lib/pci_io.c           |   9 +-
 arch/x86/lib/Makefile               |   2 +-
 boot/bootdev-uclass.c               |  23 +-
 boot/bootmeth_efi.c                 |  11 +-
 cmd/Kconfig                         |  14 +-
 cmd/mem.c                           |  26 +-
 cmd/mmc.c                           |  15 +-
 cmd/read.c                          |  10 +-
 cmd/usb.c                           |  20 --
 common/console.c                    |  36 +++
 common/log.c                        |   1 +
 common/usb_kbd.c                    |  74 +----
 configs/octeontx2_95xx_defconfig    |   2 +-
 configs/octeontx2_96xx_defconfig    |   2 +-
 configs/octeontx_81xx_defconfig     |   2 +-
 configs/octeontx_83xx_defconfig     |   2 +-
 doc/arch/sandbox/sandbox.rst        |  21 +-
 doc/develop/uefi/uefi.rst           |   2 +-
 drivers/mtd/nvmxip/nvmxip-uclass.c  |  10 +-
 drivers/usb/Kconfig                 |   3 +-
 include/console.h                   |   8 +
 include/efi_default_filename.h      |  25 +-
 include/efi_loader.h                |  21 +-
 include/log.h                       |   2 +
 include/usb.h                       |  20 +-
 lib/efi_loader/Kconfig              |  22 ++
 lib/efi_loader/Makefile             |  47 +--
 lib/efi_loader/efi_console.c        |  28 +-
 lib/efi_loader/efi_disk.c           |  14 +-
 lib/efi_loader/helloworld.c         |   6 +
 lib/efi_loader/testapp.c            |  68 ++++
 scripts/pylint.base                 | 461 ++++++++++++++++------------
 test/boot/bootdev.c                 |  37 ++-
 test/boot/bootflow.c                | 119 +++++--
 test/boot/bootstd_common.c          |  12 +
 test/boot/bootstd_common.h          |   8 +
 test/cmd/mbr.c                      | 132 ++++----
 test/py/tests/bootstd/flash1.img.xz | Bin 0 -> 5016 bytes
 test/py/tests/test_efi_fit.py       |   2 +-
 test/py/tests/test_efi_loader.py    |   2 +-
 test/py/tests/test_ut.py            | 146 +++++----
 test/test-main.c                    |  38 +++
 48 files changed, 949 insertions(+), 610 deletions(-)
 create mode 100644 lib/efi_loader/testapp.c
 create mode 100644 test/py/tests/bootstd/flash1.img.xz

-- 
2.34.1

[PATCH v2 01/39] nvmxip: Drop the message on probe

[Simon Glass]

We should not need to announce this device. Drop the message.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 drivers/mtd/nvmxip/nvmxip-uclass.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/mtd/nvmxip/nvmxip-uclass.c b/drivers/mtd/nvmxip/nvmxip-uclass.c
index 254f04e0b99..58e8c3fb74b 100644
--- a/drivers/mtd/nvmxip/nvmxip-uclass.c
+++ b/drivers/mtd/nvmxip/nvmxip-uclass.c
@@ -47,7 +47,8 @@ int nvmxip_probe(struct udevice *udev)
 		return ret;
 	}
 
-	log_info("[%s]: the block device %s ready for use\n", udev->name, bdev_name);
+	log_debug("[%s]: the block device %s ready for use\n", udev->name,
+		  bdev_name);
 
 	return 0;
 }
-- 
2.34.1

[PATCH v2 02/39] nvmxip: Avoid probing on boot

[Simon Glass]

Devices should be probed when they are used, not before. Drop this
boot-time probing.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 drivers/mtd/nvmxip/nvmxip-uclass.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/drivers/mtd/nvmxip/nvmxip-uclass.c b/drivers/mtd/nvmxip/nvmxip-uclass.c
index 58e8c3fb74b..d18bd0e3d6b 100644
--- a/drivers/mtd/nvmxip/nvmxip-uclass.c
+++ b/drivers/mtd/nvmxip/nvmxip-uclass.c
@@ -53,14 +53,7 @@ int nvmxip_probe(struct udevice *udev)
 	return 0;
 }
 
-static int nvmxip_post_bind(struct udevice *udev)
-{
-	dev_or_flags(udev, DM_FLAG_PROBE_AFTER_BIND);
-	return 0;
-}
-
 UCLASS_DRIVER(nvmxip) = {
 	.name	   = "nvmxip",
 	.id	   = UCLASS_NVMXIP,
-	.post_bind = nvmxip_post_bind,
 };
-- 
2.34.1

[PATCH v2 03/39] bootstd: Add UT_TESTF_CONSOLE_REC to bootflow tests

[Simon Glass]

Quite a few tests use console recording without indicating this, using
the UT_TESTF_CONSOLE_REC flag. Fix this to avoid strange failures.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/boot/bootflow.c | 59 +++++++++++++++++++++++---------------------
 1 file changed, 31 insertions(+), 28 deletions(-)

diff --git a/test/boot/bootflow.c b/test/boot/bootflow.c
index 8b46256fa48..8ea091a36af 100644
--- a/test/boot/bootflow.c
+++ b/test/boot/bootflow.c
@@ -50,7 +50,6 @@ static int inject_response(struct unit_test_state *uts)
 /* Check 'bootflow scan/list' commands */
 static int bootflow_cmd(struct unit_test_state *uts)
 {
-	console_record_reset_enable();
 	ut_assertok(run_command("bootdev select 1", 0));
 	ut_assert_console_end();
 	ut_assertok(run_command("bootflow scan -lH", 0));
@@ -76,14 +75,14 @@ static int bootflow_cmd(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /* Check 'bootflow scan' with a label / seq */
 static int bootflow_cmd_label(struct unit_test_state *uts)
 {
 	test_set_eth_enable(false);
 
-	console_record_reset_enable();
 	ut_assertok(run_command("bootflow scan -lH mmc1", 0));
 	ut_assert_nextline("Scanning for bootflows with label 'mmc1'");
 	ut_assert_skip_to_line("(1 bootflow, 1 valid)");
@@ -124,7 +123,7 @@ static int bootflow_cmd_label(struct unit_test_state *uts)
 	return 0;
 }
 BOOTSTD_TEST(bootflow_cmd_label, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
-	     UT_TESTF_ETH_BOOTDEV);
+	     UT_TESTF_ETH_BOOTDEV | UT_TESTF_CONSOLE_REC);
 
 /* Check 'bootflow scan/list' commands using all bootdevs */
 static int bootflow_cmd_glob(struct unit_test_state *uts)
@@ -156,14 +155,14 @@ static int bootflow_cmd_glob(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd_glob, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd_glob, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /* Check 'bootflow scan -e' */
 static int bootflow_cmd_scan_e(struct unit_test_state *uts)
 {
 	ut_assertok(bootstd_test_drop_bootdev_order(uts));
 
-	console_record_reset_enable();
 	ut_assertok(run_command("bootflow scan -aleGH", 0));
 	ut_assert_nextline("Scanning for bootflows in all bootdevs");
 	ut_assert_nextline("Seq  Method       State   Uclass    Part  Name                      Filename");
@@ -207,12 +206,12 @@ static int bootflow_cmd_scan_e(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd_scan_e, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd_scan_e, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /* Check 'bootflow info' */
 static int bootflow_cmd_info(struct unit_test_state *uts)
 {
-	console_record_reset_enable();
 	ut_assertok(run_command("bootdev select 1", 0));
 	ut_assert_console_end();
 	ut_assertok(run_command("bootflow scan", 0));
@@ -248,12 +247,12 @@ static int bootflow_cmd_info(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd_info, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd_info, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /* Check 'bootflow scan -b' to boot the first available bootdev */
 static int bootflow_scan_boot(struct unit_test_state *uts)
 {
-	console_record_reset_enable();
 	ut_assertok(inject_response(uts));
 	ut_assertok(run_command("bootflow scan -b", 0));
 	ut_assert_nextline(
@@ -270,7 +269,8 @@ static int bootflow_scan_boot(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_scan_boot, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_scan_boot, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /* Check iterating through available bootflows */
 static int bootflow_iter(struct unit_test_state *uts)
@@ -368,7 +368,8 @@ static int bootflow_iter(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_iter, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_iter, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 #if defined(CONFIG_SANDBOX) && defined(CONFIG_BOOTMETH_GLOBAL)
 /* Check using the system bootdev */
@@ -386,7 +387,6 @@ static int bootflow_system(struct unit_test_state *uts)
 
 	/* We should get a single 'bootmgr' method right at the end */
 	bootstd_clear_glob();
-	console_record_reset_enable();
 	ut_assertok(run_command("bootflow scan -lH", 0));
 	ut_assert_skip_to_line(
 		"  0  efi_mgr      ready   (none)       0  <NULL>                    ");
@@ -397,7 +397,7 @@ static int bootflow_system(struct unit_test_state *uts)
 	return 0;
 }
 BOOTSTD_TEST(bootflow_system, UT_TESTF_DM | UT_TESTF_SCAN_PDATA |
-	     UT_TESTF_SCAN_FDT);
+	     UT_TESTF_SCAN_FDT | UT_TESTF_CONSOLE_REC);
 #endif
 
 /* Check disabling a bootmethod if it requests it */
@@ -438,7 +438,9 @@ static int bootflow_iter_disable(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_iter_disable, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_iter_disable, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC
+);
 
 /* Check 'bootflow scan' with a bootmeth ordering including a global bootmeth */
 static int bootflow_scan_glob_bootmeth(struct unit_test_state *uts)
@@ -452,7 +454,6 @@ static int bootflow_scan_glob_bootmeth(struct unit_test_state *uts)
 	 * Make sure that the -G flag makes the scan fail, since this is not
 	 * supported when an ordering is provided
 	 */
-	console_record_reset_enable();
 	ut_assertok(bootmeth_set_order("efi firmware0"));
 	ut_assertok(run_command("bootflow scan -lGH", 0));
 	ut_assert_nextline("Scanning for bootflows in all bootdevs");
@@ -479,12 +480,13 @@ static int bootflow_scan_glob_bootmeth(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_scan_glob_bootmeth, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_scan_glob_bootmeth, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC
+);
 
 /* Check 'bootflow boot' to boot a selected bootflow */
 static int bootflow_cmd_boot(struct unit_test_state *uts)
 {
-	console_record_reset_enable();
 	ut_assertok(run_command("bootdev select 1", 0));
 	ut_assert_console_end();
 	ut_assertok(run_command("bootflow scan", 0));
@@ -508,7 +510,8 @@ static int bootflow_cmd_boot(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd_boot, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd_boot, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /**
  * prep_mmc_bootdev() - Set up an mmc bootdev so we can access other distros
@@ -675,7 +678,8 @@ static int bootflow_cmd_menu(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd_menu, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd_menu, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /* Check 'bootflow scan -m' to select a bootflow using a menu */
 static int bootflow_scan_menu(struct unit_test_state *uts)
@@ -783,7 +787,6 @@ static int bootflow_cmd_hunt_single(struct unit_test_state *uts)
 
 	ut_assertok(bootstd_test_drop_bootdev_order(uts));
 
-	console_record_reset_enable();
 	ut_assertok(run_command("bootflow scan -l mmc1", 0));
 	ut_assert_nextline("Scanning for bootflows with label 'mmc1'");
 	ut_assert_skip_to_line("(1 bootflow, 1 valid)");
@@ -794,7 +797,8 @@ static int bootflow_cmd_hunt_single(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd_hunt_single, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd_hunt_single, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /* Check searching for a uclass label using the hunters */
 static int bootflow_cmd_hunt_label(struct unit_test_state *uts)
@@ -808,7 +812,6 @@ static int bootflow_cmd_hunt_label(struct unit_test_state *uts)
 	test_set_eth_enable(false);
 	ut_assertok(bootstd_test_drop_bootdev_order(uts));
 
-	console_record_reset_enable();
 	ut_assertok(run_command("bootflow scan -l mmc", 0));
 
 	/* check that the hunter was used */
@@ -831,7 +834,8 @@ static int bootflow_cmd_hunt_label(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmd_hunt_label, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
+BOOTSTD_TEST(bootflow_cmd_hunt_label, UT_TESTF_DM | UT_TESTF_SCAN_FDT |
+	     UT_TESTF_CONSOLE_REC);
 
 /**
  * check_font() - Check that the font size for an item matches expectations
@@ -1127,7 +1131,6 @@ static int bootflow_cmdline(struct unit_test_state *uts)
 {
 	ut_assertok(run_command("bootflow scan mmc", 0));
 	ut_assertok(run_command("bootflow sel 0", 0));
-	console_record_reset_enable();
 
 	ut_asserteq(1, run_command("bootflow cmdline get fred", 0));
 	ut_assert_nextline("Argument not found");
@@ -1157,7 +1160,7 @@ static int bootflow_cmdline(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cmdline, 0);
+BOOTSTD_TEST(bootflow_cmdline, UT_TESTF_CONSOLE_REC);
 
 /* test a few special changes to a long command line */
 static int bootflow_cmdline_special(struct unit_test_state *uts)
@@ -1198,7 +1201,7 @@ static int bootflow_cros(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_cros, 0);
+BOOTSTD_TEST(bootflow_cros, UT_TESTF_CONSOLE_REC);
 
 /* Test Android bootmeth  */
 static int bootflow_android(struct unit_test_state *uts)
@@ -1221,4 +1224,4 @@ static int bootflow_android(struct unit_test_state *uts)
 
 	return 0;
 }
-BOOTSTD_TEST(bootflow_android, 0);
+BOOTSTD_TEST(bootflow_android, UT_TESTF_CONSOLE_REC);
-- 
2.34.1

[PATCH v2 04/39] test/py: Fix some pylint warnings in test_ut.py

[Simon Glass]

Tidy up most of these warnings. Remaining are four of these:

   R0914: Too many local variables

which can only by fixed by splitting things into functions, so that is
left for another time.

Part of this change was done by the flynt tool.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/py/tests/test_ut.py | 94 ++++++++++++++++++++--------------------
 1 file changed, 48 insertions(+), 46 deletions(-)

diff --git a/test/py/tests/test_ut.py b/test/py/tests/test_ut.py
index 05e15830590..39aa1035e34 100644
--- a/test/py/tests/test_ut.py
+++ b/test/py/tests/test_ut.py
@@ -1,6 +1,12 @@
 # SPDX-License-Identifier: GPL-2.0
-# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+"""
+Unit-test runner
+
+Provides a test_ut() function which is used by conftest.py to run each unit
+test one at a time, as well setting up some files needed by the tests.
 
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+"""
 import collections
 import getpass
 import gzip
@@ -44,8 +50,8 @@ def setup_image(cons, mmc_dev, part_type, second_part=False):
     if second_part:
         spec += '\ntype=c'
 
-    u_boot_utils.run_and_log(cons, 'qemu-img create %s 20M' % fname)
-    u_boot_utils.run_and_log(cons, 'sudo sfdisk %s' % fname,
+    u_boot_utils.run_and_log(cons, f'qemu-img create {fname} 20M')
+    u_boot_utils.run_and_log(cons, f'sudo sfdisk {fname}',
                              stdin=spec.encode('utf-8'))
     return fname, mnt
 
@@ -61,13 +67,13 @@ def mount_image(cons, fname, mnt, fstype):
     Returns:
         str: Name of loop device used
     """
-    out = u_boot_utils.run_and_log(cons, 'sudo losetup --show -f -P %s' % fname)
+    out = u_boot_utils.run_and_log(cons, f'sudo losetup --show -f -P {fname}')
     loop = out.strip()
     part = f'{loop}p1'
     u_boot_utils.run_and_log(cons, f'sudo mkfs.{fstype} {part}')
     opts = ''
     if fstype == 'vfat':
-         opts += f' -o uid={os.getuid()},gid={os.getgid()}'
+        opts += f' -o uid={os.getuid()},gid={os.getgid()}'
     u_boot_utils.run_and_log(cons, f'sudo mount -o loop {part} {mnt}{opts}')
     u_boot_utils.run_and_log(cons, f'sudo chown {getpass.getuser()} {mnt}')
     return loop
@@ -82,9 +88,7 @@ def copy_prepared_image(cons, mmc_dev, fname):
     """
     infname = os.path.join(cons.config.source_dir,
                            f'test/py/tests/bootstd/mmc{mmc_dev}.img.xz')
-    u_boot_utils.run_and_log(
-        cons,
-        ['sh', '-c', 'xz -dc %s >%s' % (infname, fname)])
+    u_boot_utils.run_and_log(cons, ['sh', '-c', f'xz -dc {infname} >{fname}'])
 
 def setup_bootmenu_image(cons):
     """Create a 20MB disk image with a single ext4 partition
@@ -101,9 +105,6 @@ def setup_bootmenu_image(cons):
         loop = mount_image(cons, fname, mnt, 'ext4')
         mounted = True
 
-        vmlinux = 'Image'
-        initrd = 'uInitrd'
-        dtbdir = 'dtb'
         script = '''# DO NOT EDIT THIS FILE
 #
 # Please edit /boot/armbianEnv.txt to set supported parameters
@@ -177,12 +178,12 @@ booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r}
 
 # Recompile with:
 # mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr
-''' % (mmc_dev)
+'''
         bootdir = os.path.join(mnt, 'boot')
         mkdir_cond(bootdir)
         cmd_fname = os.path.join(bootdir, 'boot.cmd')
         scr_fname = os.path.join(bootdir, 'boot.scr')
-        with open(cmd_fname, 'w') as outf:
+        with open(cmd_fname, 'w', encoding='ascii') as outf:
             print(script, file=outf)
 
         infname = os.path.join(cons.config.source_dir,
@@ -212,13 +213,12 @@ booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r}
         complete = True
 
     except ValueError as exc:
-        print('Falled to create image, failing back to prepared copy: %s',
-              str(exc))
+        print(f'Falled to create image, failing back to prepared copy: {exc}')
     finally:
         if mounted:
-            u_boot_utils.run_and_log(cons, 'sudo umount --lazy %s' % mnt)
+            u_boot_utils.run_and_log(cons, f'sudo umount --lazy {mnt}')
         if loop:
-            u_boot_utils.run_and_log(cons, 'sudo losetup -d %s' % loop)
+            u_boot_utils.run_and_log(cons, f'sudo losetup -d {loop}')
 
     if not complete:
         copy_prepared_image(cons, mmc_dev, fname)
@@ -254,32 +254,32 @@ label Fedora-Workstation-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl)
         ext = os.path.join(mnt, 'extlinux')
         mkdir_cond(ext)
 
-        with open(os.path.join(ext, 'extlinux.conf'), 'w') as fd:
+        conf = os.path.join(ext, 'extlinux.conf')
+        with open(conf, 'w', encoding='ascii') as fd:
             print(script, file=fd)
 
         inf = os.path.join(cons.config.persistent_data_dir, 'inf')
         with open(inf, 'wb') as fd:
             fd.write(gzip.compress(b'vmlinux'))
-        u_boot_utils.run_and_log(cons, 'mkimage -f auto -d %s %s' %
-                                 (inf, os.path.join(mnt, vmlinux)))
+        u_boot_utils.run_and_log(
+            cons, f'mkimage -f auto -d {inf} {os.path.join(mnt, vmlinux)}')
 
-        with open(os.path.join(mnt, initrd), 'w') as fd:
+        with open(os.path.join(mnt, initrd), 'w', encoding='ascii') as fd:
             print('initrd', file=fd)
 
         mkdir_cond(os.path.join(mnt, dtbdir))
 
-        dtb_file = os.path.join(mnt, '%s/sandbox.dtb' % dtbdir)
+        dtb_file = os.path.join(mnt, f'{dtbdir}/sandbox.dtb')
         u_boot_utils.run_and_log(
-            cons, 'dtc -o %s' % dtb_file, stdin=b'/dts-v1/; / {};')
+            cons, f'dtc -o {dtb_file}', stdin=b'/dts-v1/; / {};')
         complete = True
     except ValueError as exc:
-        print('Falled to create image, failing back to prepared copy: %s',
-              str(exc))
+        print(f'Falled to create image, failing back to prepared copy: {exc}')
     finally:
         if mounted:
-            u_boot_utils.run_and_log(cons, 'sudo umount --lazy %s' % mnt)
+            u_boot_utils.run_and_log(cons, f'sudo umount --lazy {mnt}')
         if loop:
-            u_boot_utils.run_and_log(cons, 'sudo losetup -d %s' % loop)
+            u_boot_utils.run_and_log(cons, f'sudo losetup -d {loop}')
 
     if not complete:
         copy_prepared_image(cons, mmc_dev, fname)
@@ -303,7 +303,8 @@ def setup_cros_image(cons):
         Return:
             bytes: Packed-kernel data
         """
-        kern_part = os.path.join(cons.config.result_dir, 'kern-part-{arch}.bin')
+        kern_part = os.path.join(cons.config.result_dir,
+                                 f'kern-part-{arch}.bin')
         u_boot_utils.run_and_log(
             cons,
             f'futility vbutil_kernel --pack {kern_part} '
@@ -332,7 +333,7 @@ def setup_cros_image(cons):
 
     mmc_dev = 5
     fname = os.path.join(cons.config.source_dir, f'mmc{mmc_dev}.img')
-    u_boot_utils.run_and_log(cons, 'qemu-img create %s 20M' % fname)
+    u_boot_utils.run_and_log(cons, f'qemu-img create {fname} 20M')
     #mnt = os.path.join(cons.config.persistent_data_dir, 'mnt')
     #mkdir_cond(mnt)
     u_boot_utils.run_and_log(cons, f'cgpt create {fname}')
@@ -381,20 +382,20 @@ def setup_cros_image(cons):
 
     u_boot_utils.run_and_log(cons, f'cgpt boot -p {fname}')
     out = u_boot_utils.run_and_log(cons, f'cgpt show -q {fname}')
-    '''We expect something like this:
-        8239        2048       1  Basic data
-          45        2048       2  ChromeOS kernel
-        8238           1       3  ChromeOS rootfs
-        2093        2048       4  ChromeOS kernel
-        8237           1       5  ChromeOS rootfs
-          41           1       6  ChromeOS kernel
-          42           1       7  ChromeOS rootfs
-        4141        2048       8  Basic data
-          43           1       9  ChromeOS reserved
-          44           1      10  ChromeOS reserved
-          40           1      11  ChromeOS firmware
-        6189        2048      12  EFI System Partition
-    '''
+
+    # We expect something like this:
+    #   8239        2048       1  Basic data
+    #     45        2048       2  ChromeOS kernel
+    #   8238           1       3  ChromeOS rootfs
+    #   2093        2048       4  ChromeOS kernel
+    #   8237           1       5  ChromeOS rootfs
+    #     41           1       6  ChromeOS kernel
+    #     42           1       7  ChromeOS rootfs
+    #   4141        2048       8  Basic data
+    #     43           1       9  ChromeOS reserved
+    #     44           1      10  ChromeOS reserved
+    #     40           1      11  ChromeOS firmware
+    #   6189        2048      12  EFI System Partition
 
     # Create a dict (indexed by partition number) containing the above info
     for line in out.splitlines():
@@ -446,7 +447,7 @@ def setup_android_image(cons):
 
     mmc_dev = 7
     fname = os.path.join(cons.config.source_dir, f'mmc{mmc_dev}.img')
-    u_boot_utils.run_and_log(cons, 'qemu-img create %s 20M' % fname)
+    u_boot_utils.run_and_log(cons, f'qemu-img create {fname} 20M')
     u_boot_utils.run_and_log(cons, f'cgpt create {fname}')
 
     ptr = 40
@@ -498,11 +499,12 @@ def setup_android_image(cons):
     with open(fname, 'wb') as outf:
         outf.write(disk_data)
 
-    print('wrote to {}'.format(fname))
+    print(f'wrote to {fname}')
 
     return fname
 
 def setup_cedit_file(cons):
+    """Set up a .dtb file for use with testing expo and configuration editor"""
     infname = os.path.join(cons.config.source_dir,
                            'test/boot/files/expo_layout.dts')
     inhname = os.path.join(cons.config.source_dir,
@@ -584,7 +586,7 @@ def test_ut(u_boot_console, ut_subtest):
         # ut hush hush_test_simple_dollar prints "Unknown command" on purpose.
         with u_boot_console.disable_check('unknown_command'):
             output = u_boot_console.run_command('ut ' + ut_subtest)
-        assert('Unknown command \'quux\' - try \'help\'' in output)
+        assert 'Unknown command \'quux\' - try \'help\'' in output
     else:
         output = u_boot_console.run_command('ut ' + ut_subtest)
     assert output.endswith('Failures: 0')
-- 
2.34.1

[PATCH v2 05/39] scripts: Update pylint.base

[Simon Glass]

There have been quite a few changes in the Python scripts, so update the
pylint baseline.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 scripts/pylint.base | 461 ++++++++++++++++++++++++++------------------
 1 file changed, 271 insertions(+), 190 deletions(-)

diff --git a/scripts/pylint.base b/scripts/pylint.base
index c7d141ed393..b2209a1d10f 100644
--- a/scripts/pylint.base
+++ b/scripts/pylint.base
@@ -1,229 +1,310 @@
-test_conftest.py 6.56
-test_multiplexed_log.py 7.49
-test_test.py 8.18
+test_conftest.py 6.78
+test_multiplexed_log.py 7.68
+test_test.py 9.00
+test_tests_fit_util.py 7.19
+test_tests_fs_helper.py 10.00
 test_tests_test_000_version.py 7.50
-test_tests_test_android_test_ab.py 6.50
-test_tests_test_android_test_abootimg.py 6.09
-test_tests_test_android_test_avb.py 5.52
-test_tests_test_bind.py -2.99
+test_tests_test_android_test_ab.py 7.00
+test_tests_test_android_test_abootimg.py 6.52
+test_tests_test_android_test_avb.py 6.38
+test_tests_test_bind.py 8.89
 test_tests_test_bootmenu.py 10.00
+test_tests_test_bootstage.py 7.14
 test_tests_test_button.py 3.33
-test_tests_test_dfu.py 5.45
-test_tests_test_dm.py 9.52
-test_tests_test_efi_capsule_capsule_defs.py 6.67
-test_tests_test_efi_capsule_conftest.py 1.86
-test_tests_test_efi_capsule_test_capsule_firmware.py 4.52
-test_tests_test_efi_capsule_test_capsule_firmware_signed.py 4.85
-test_tests_test_efi_fit.py 8.16
-test_tests_test_efi_loader.py 7.38
-test_tests_test_efi_secboot_conftest.py -3.29
-test_tests_test_efi_secboot_defs.py 6.67
-test_tests_test_efi_secboot_test_authvar.py 8.93
-test_tests_test_efi_secboot_test_signed.py 8.41
-test_tests_test_efi_secboot_test_signed_intca.py 8.10
-test_tests_test_efi_secboot_test_unsigned.py 8.00
-test_tests_test_efi_selftest.py 6.36
-test_tests_test_env.py 7.15
-test_tests_test_extension.py 2.14
-test_tests_test_fit.py 6.83
-test_tests_test_fit_ecdsa.py 7.94
-test_tests_test_fit_hashes.py 7.70
-test_tests_test_fpga.py 1.81
-test_tests_test_fs_conftest.py 5.13
+test_tests_test_cat_conftest.py 10.00
+test_tests_test_cat_test_cat.py 10.00
+test_tests_test_cleanup_build.py 10.00
+test_tests_test_dfu.py 6.23
+test_tests_test_dm.py 8.98
+test_tests_test_efi_bootmgr_conftest.py 10.00
+test_tests_test_efi_bootmgr_test_efi_bootmgr.py 10.00
+test_tests_test_efi_capsule_capsule_common.py 10.00
+test_tests_test_efi_capsule_capsule_defs.py 10.00
+test_tests_test_efi_capsule_conftest.py 3.26
+test_tests_test_efi_capsule_test_capsule_firmware_fit.py 10.00
+test_tests_test_efi_capsule_test_capsule_firmware_raw.py 9.62
+test_tests_test_efi_capsule_test_capsule_firmware_signed_fit.py 9.74
+test_tests_test_efi_capsule_test_capsule_firmware_signed_raw.py 9.75
+test_tests_test_efi_fit.py 8.84
+test_tests_test_efi_loader.py 8.00
+test_tests_test_efi_secboot_conftest.py 0.00
+test_tests_test_efi_secboot_defs.py 10.00
+test_tests_test_efi_secboot_test_authvar.py 9.38
+test_tests_test_efi_secboot_test_signed.py 8.60
+test_tests_test_efi_secboot_test_signed_intca.py 8.81
+test_tests_test_efi_secboot_test_unsigned.py 8.75
+test_tests_test_efi_selftest.py 8.07
+test_tests_test_eficonfig_conftest.py 10.00
+test_tests_test_eficonfig_test_eficonfig.py 9.47
+test_tests_test_env.py 7.76
+test_tests_test_event_dump.py 2.22
+test_tests_test_extension.py 2.50
+test_tests_test_fit.py 7.54
+test_tests_test_fit_auto_signed.py 9.09
+test_tests_test_fit_ecdsa.py 8.29
+test_tests_test_fit_hashes.py 7.94
+test_tests_test_fpga.py 2.94
+test_tests_test_fs_conftest.py 5.21
 test_tests_test_fs_fstest_defs.py 8.33
 test_tests_test_fs_fstest_helpers.py 4.29
-test_tests_test_fs_test_basic.py 0.60
-test_tests_test_fs_test_ext.py 0.00
+test_tests_test_fs_test_basic.py 1.90
+test_tests_test_fs_test_erofs.py 8.97
+test_tests_test_fs_test_ext.py 1.48
 test_tests_test_fs_test_fs_cmd.py 8.00
-test_tests_test_fs_test_mkdir.py 1.96
-test_tests_test_fs_test_squashfs_sqfs_common.py 8.41
-test_tests_test_fs_test_squashfs_test_sqfs_load.py 7.46
-test_tests_test_fs_test_squashfs_test_sqfs_ls.py 8.00
-test_tests_test_fs_test_symlink.py 1.22
-test_tests_test_fs_test_unlink.py 2.78
-test_tests_test_gpio.py 6.09
-test_tests_test_gpt.py 7.67
+test_tests_test_fs_test_fs_fat.py 2.50
+test_tests_test_fs_test_mkdir.py 3.04
+test_tests_test_fs_test_squashfs_sqfs_common.py 8.38
+test_tests_test_fs_test_squashfs_test_sqfs_load.py 7.63
+test_tests_test_fs_test_squashfs_test_sqfs_ls.py 8.04
+test_tests_test_fs_test_symlink.py 2.04
+test_tests_test_fs_test_unlink.py 4.07
+test_tests_test_gpio.py 7.60
+test_tests_test_gpt.py 8.55
 test_tests_test_handoff.py 5.00
-test_tests_test_help.py 5.00
-test_tests_test_hush_if_test.py 9.27
-test_tests_test_log.py 8.64
+test_tests_test_help.py 8.64
+test_tests_test_i2c.py 7.42
+test_tests_test_kconfig.py 5.38
+test_tests_test_log.py 8.75
 test_tests_test_lsblk.py 8.00
 test_tests_test_md.py 3.64
+test_tests_test_mdio.py 6.82
+test_tests_test_memtest.py 8.39
+test_tests_test_mii.py 8.55
+test_tests_test_mmc.py 7.01
 test_tests_test_mmc_rd.py 6.05
 test_tests_test_mmc_wr.py 3.33
-test_tests_test_net.py 6.84
+test_tests_test_net.py 8.43
+test_tests_test_net_boot.py 8.23
+test_tests_test_of_migrate.py 7.86
 test_tests_test_ofplatdata.py 5.71
+test_tests_test_optee_rpmb.py 0.00
 test_tests_test_part.py 8.00
-test_tests_test_pinmux.py 3.27
+test_tests_test_pinmux.py 3.40
 test_tests_test_pstore.py 2.31
 test_tests_test_qfw.py 8.75
+test_tests_test_reset.py 9.55
 test_tests_test_sandbox_exit.py 6.50
+test_tests_test_sandbox_opts.py 1.11
+test_tests_test_saveenv.py 7.87
 test_tests_test_scp03.py 3.33
-test_tests_test_sf.py 7.13
+test_tests_test_scsi.py 8.47
+test_tests_test_semihosting_conftest.py 10.00
+test_tests_test_semihosting_test_hostfs.py 10.00
+test_tests_test_sf.py 7.45
 test_tests_test_shell_basics.py 9.58
-test_tests_test_sleep.py 7.78
-test_tests_test_spl.py 2.22
+test_tests_test_sleep.py 8.28
+test_tests_test_smbios.py 9.47
+test_tests_test_source.py 7.20
+test_tests_test_spl.py 6.67
 test_tests_test_stackprotector.py 5.71
-test_tests_test_tpm2.py 8.51
-test_tests_test_ums.py 6.32
+test_tests_test_tpm2.py 8.45
+test_tests_test_trace.py 8.70
+test_tests_test_ums.py 5.92
 test_tests_test_unknown_cmd.py 5.00
-test_tests_test_ut.py 7.06
-test_tests_test_vboot.py 6.01
-test_tests_vboot_evil.py 8.95
+test_tests_test_usb.py 7.08
+test_tests_test_ut.py 9.44
+test_tests_test_vbe.py 7.22
+test_tests_test_vbe_vpl.py 6.11
+test_tests_test_vboot.py 5.37
+test_tests_test_vpl.py 2.22
+test_tests_test_xxd_conftest.py 10.00
+test_tests_test_xxd_test_xxd.py 10.00
+test_tests_test_zynq_secure.py 7.60
+test_tests_test_zynqmp_rpu.py 7.54
+test_tests_test_zynqmp_secure.py 7.68
+test_tests_vboot_evil.py 9.45
 test_tests_vboot_forge.py 9.22
-test_u_boot_console_base.py 7.08
-test_u_boot_console_exec_attach.py 9.23
-test_u_boot_console_sandbox.py 8.06
-test_u_boot_spawn.py 7.65
-test_u_boot_utils.py 6.94
-tools_binman_bintool 8.59
+test_u_boot_console_base.py 7.73
+test_u_boot_console_exec_attach.py 9.62
+test_u_boot_console_sandbox.py 8.64
+test_u_boot_spawn.py 8.57
+test_u_boot_utils.py 7.83
+tools_binman_bintool 9.16
 tools_binman_bintool_test 9.87
-tools_binman_btool__testing 6.09
+tools_binman_btool__testing 6.52
+tools_binman_btool_bootgen 4.50
+tools_binman_btool_btool_gzip 0.00
+tools_binman_btool_bzip2 0.00
 tools_binman_btool_cbfstool 7.83
-tools_binman_btool_fiptool 7.62
-tools_binman_btool_futility 7.39
+tools_binman_btool_cst 5.00
+tools_binman_btool_fdt_add_pubkey 7.00
+tools_binman_btool_fdtgrep 5.20
+tools_binman_btool_fiptool 7.22
+tools_binman_btool_futility 6.67
 tools_binman_btool_ifwitool 3.81
-tools_binman_btool_lz4 6.30
+tools_binman_btool_lz4 4.76
 tools_binman_btool_lzma_alone 6.97
-tools_binman_btool_mkimage 7.86
-tools_binman_cbfs_util 8.46
-tools_binman_cbfs_util_test 9.38
-tools_binman_cmdline 9.03
-tools_binman_comp_util 6.88
-tools_binman_control 5.01
-tools_binman_elf 6.98
-tools_binman_elf_test 5.62
-tools_binman_entry 3.55
-tools_binman_entry_test 5.34
-tools_binman_etype__testing 0.83
-tools_binman_etype_atf_bl31 -6.00
-tools_binman_etype_atf_fip 0.29
-tools_binman_etype_blob -1.58
-tools_binman_etype_blob_dtb -10.00
-tools_binman_etype_blob_ext -19.09
+tools_binman_btool_lzop 0.00
+tools_binman_btool_mkeficapsule 7.69
+tools_binman_btool_mkimage 6.36
+tools_binman_btool_openssl 4.63
+tools_binman_btool_xz 0.00
+tools_binman_btool_zstd 0.00
+tools_binman_cbfs_util 8.93
+tools_binman_cbfs_util_test 9.81
+tools_binman_cmdline 9.33
+tools_binman_control 6.92
+tools_binman_elf 7.52
+tools_binman_elf_test 8.40
+tools_binman_entry 6.40
+tools_binman_entry_test 6.99
+tools_binman_etype__testing 2.02
+tools_binman_etype_alternates_fdt 5.09
+tools_binman_etype_atf_bl31 0.00
+tools_binman_etype_atf_fip 0.44
+tools_binman_etype_blob 0.41
+tools_binman_etype_blob_dtb 0.21
+tools_binman_etype_blob_ext 0.00
 tools_binman_etype_blob_ext_list 0.00
-tools_binman_etype_blob_named_by_arg -7.78
-tools_binman_etype_blob_phase -5.00
-tools_binman_etype_cbfs -1.44
+tools_binman_etype_blob_named_by_arg 0.00
+tools_binman_etype_blob_phase 0.50
+tools_binman_etype_cbfs 1.86
 tools_binman_etype_collection 2.67
-tools_binman_etype_cros_ec_rw -6.00
-tools_binman_etype_fdtmap -3.28
-tools_binman_etype_files -7.43
-tools_binman_etype_fill -6.43
-tools_binman_etype_fit 6.31
-tools_binman_etype_fmap -0.29
-tools_binman_etype_gbb 0.83
-tools_binman_etype_image_header 5.77
-tools_binman_etype_intel_cmc -12.50
+tools_binman_etype_cros_ec_rw 0.00
+tools_binman_etype_efi_capsule 3.04
+tools_binman_etype_efi_empty_capsule 0.00
+tools_binman_etype_encrypted 1.43
+tools_binman_etype_fdtmap 0.16
+tools_binman_etype_files 0.00
+tools_binman_etype_fill 0.00
+tools_binman_etype_fit 7.52
+tools_binman_etype_fmap 0.54
+tools_binman_etype_gbb 1.35
+tools_binman_etype_image_header 4.81
+tools_binman_etype_intel_cmc 0.00
 tools_binman_etype_intel_descriptor 4.62
 tools_binman_etype_intel_fit 0.00
 tools_binman_etype_intel_fit_ptr 2.35
-tools_binman_etype_intel_fsp -12.50
-tools_binman_etype_intel_fsp_m -12.50
-tools_binman_etype_intel_fsp_s -12.50
-tools_binman_etype_intel_fsp_t -12.50
-tools_binman_etype_intel_ifwi 2.88
-tools_binman_etype_intel_me -12.50
-tools_binman_etype_intel_mrc -10.00
-tools_binman_etype_intel_refcode -10.00
-tools_binman_etype_intel_vbt -12.50
-tools_binman_etype_intel_vga -12.50
-tools_binman_etype_mkimage 1.47
-tools_binman_etype_opensbi -6.00
-tools_binman_etype_powerpc_mpc85xx_bootpg_resetvec -10.00
-tools_binman_etype_scp -6.00
-tools_binman_etype_section 4.57
-tools_binman_etype_tee_os -6.00
-tools_binman_etype_text -0.48
-tools_binman_etype_u_boot -15.71
-tools_binman_etype_u_boot_dtb -12.22
-tools_binman_etype_u_boot_dtb_with_ucode 0.39
-tools_binman_etype_u_boot_elf -8.42
+tools_binman_etype_intel_fsp 0.00
+tools_binman_etype_intel_fsp_m 0.00
+tools_binman_etype_intel_fsp_s 0.00
+tools_binman_etype_intel_fsp_t 0.00
+tools_binman_etype_intel_ifwi 3.13
+tools_binman_etype_intel_me 0.00
+tools_binman_etype_intel_mrc 0.00
+tools_binman_etype_intel_refcode 0.00
+tools_binman_etype_intel_vbt 0.00
+tools_binman_etype_intel_vga 0.00
+tools_binman_etype_mkimage 4.88
+tools_binman_etype_null 0.00
+tools_binman_etype_nxp_imx8mcst 2.44
+tools_binman_etype_nxp_imx8mimage 0.00
+tools_binman_etype_opensbi 0.00
+tools_binman_etype_powerpc_mpc85xx_bootpg_resetvec 0.00
+tools_binman_etype_pre_load 3.68
+tools_binman_etype_rockchip_tpl 0.00
+tools_binman_etype_scp 0.00
+tools_binman_etype_section 6.04
+tools_binman_etype_tee_os 4.00
+tools_binman_etype_text 0.00
+tools_binman_etype_ti_board_config 5.40
+tools_binman_etype_ti_dm 0.00
+tools_binman_etype_ti_secure 4.22
+tools_binman_etype_ti_secure_rom 2.22
+tools_binman_etype_u_boot 0.00
+tools_binman_etype_u_boot_dtb 0.00
+tools_binman_etype_u_boot_dtb_with_ucode 1.73
+tools_binman_etype_u_boot_elf 0.00
 tools_binman_etype_u_boot_env 0.74
-tools_binman_etype_u_boot_expanded -10.00
-tools_binman_etype_u_boot_img -15.71
-tools_binman_etype_u_boot_nodtb -15.71
-tools_binman_etype_u_boot_spl -10.91
-tools_binman_etype_u_boot_spl_bss_pad -9.29
-tools_binman_etype_u_boot_spl_dtb -12.22
-tools_binman_etype_u_boot_spl_elf -15.71
-tools_binman_etype_u_boot_spl_expanded -9.09
-tools_binman_etype_u_boot_spl_nodtb -10.91
-tools_binman_etype_u_boot_spl_with_ucode_ptr -5.00
-tools_binman_etype_u_boot_tpl -10.91
-tools_binman_etype_u_boot_tpl_bss_pad -9.29
-tools_binman_etype_u_boot_tpl_dtb -12.22
-tools_binman_etype_u_boot_tpl_dtb_with_ucode -7.50
-tools_binman_etype_u_boot_tpl_elf -15.71
-tools_binman_etype_u_boot_tpl_expanded -9.09
-tools_binman_etype_u_boot_tpl_nodtb -10.91
-tools_binman_etype_u_boot_tpl_with_ucode_ptr -20.83
+tools_binman_etype_u_boot_expanded 0.00
+tools_binman_etype_u_boot_img 0.00
+tools_binman_etype_u_boot_nodtb 0.00
+tools_binman_etype_u_boot_spl 0.00
+tools_binman_etype_u_boot_spl_bss_pad 0.00
+tools_binman_etype_u_boot_spl_dtb 0.00
+tools_binman_etype_u_boot_spl_elf 0.00
+tools_binman_etype_u_boot_spl_expanded 0.00
+tools_binman_etype_u_boot_spl_nodtb 0.00
+tools_binman_etype_u_boot_spl_pubkey_dtb 1.21
+tools_binman_etype_u_boot_spl_with_ucode_ptr 0.00
+tools_binman_etype_u_boot_tpl 0.00
+tools_binman_etype_u_boot_tpl_bss_pad 0.00
+tools_binman_etype_u_boot_tpl_dtb 0.00
+tools_binman_etype_u_boot_tpl_dtb_with_ucode 0.00
+tools_binman_etype_u_boot_tpl_elf 0.00
+tools_binman_etype_u_boot_tpl_expanded 0.00
+tools_binman_etype_u_boot_tpl_nodtb 0.00
+tools_binman_etype_u_boot_tpl_with_ucode_ptr 0.00
 tools_binman_etype_u_boot_ucode 1.52
-tools_binman_etype_u_boot_with_ucode_ptr -0.71
-tools_binman_etype_vblock 0.27
-tools_binman_etype_x86_reset16 -15.71
-tools_binman_etype_x86_reset16_spl -15.71
-tools_binman_etype_x86_reset16_tpl -15.71
-tools_binman_etype_x86_start16 -15.71
-tools_binman_etype_x86_start16_spl -15.71
-tools_binman_etype_x86_start16_tpl -15.71
-tools_binman_fdt_test 10.00
+tools_binman_etype_u_boot_vpl 0.00
+tools_binman_etype_u_boot_vpl_bss_pad 0.00
+tools_binman_etype_u_boot_vpl_dtb 0.00
+tools_binman_etype_u_boot_vpl_elf 0.00
+tools_binman_etype_u_boot_vpl_expanded 0.00
+tools_binman_etype_u_boot_vpl_nodtb 0.00
+tools_binman_etype_u_boot_with_ucode_ptr 0.00
+tools_binman_etype_vblock 0.79
+tools_binman_etype_x509_cert 3.10
+tools_binman_etype_x86_reset16 0.00
+tools_binman_etype_x86_reset16_spl 0.00
+tools_binman_etype_x86_reset16_tpl 0.00
+tools_binman_etype_x86_start16 0.00
+tools_binman_etype_x86_start16_spl 0.00
+tools_binman_etype_x86_start16_tpl 0.00
+tools_binman_etype_xilinx_bootgen 6.06
+tools_binman_fdt_test 7.74
 tools_binman_fip_util 9.85
 tools_binman_fip_util_test 10.00
-tools_binman_fmap_util 6.88
-tools_binman_ftest 7.46
-tools_binman_image 7.12
-tools_binman_image_test 4.48
-tools_binman_main 4.86
-tools_binman_setup 5.00
-tools_binman_state 4.15
-tools_buildman_board 7.82
-tools_buildman_bsettings 1.71
-tools_buildman_builder 6.92
-tools_buildman_builderthread 7.48
-tools_buildman_cfgutil 7.83
-tools_buildman_cmdline 8.89
-tools_buildman_control 8.12
-tools_buildman_func_test 7.18
-tools_buildman_kconfiglib 7.49
-tools_buildman_main -1.11
-tools_buildman_test 6.56
-tools_buildman_toolchain 6.44
-tools_concurrencytest_concurrencytest 7.26
-tools_dtoc_dtb_platdata 7.90
-tools_dtoc_fdt 4.46
-tools_dtoc_fdt_util 6.80
-tools_dtoc_main 7.78
-tools_dtoc_setup 5.00
-tools_dtoc_src_scan 8.91
-tools_dtoc_test_dtoc 8.56
-tools_dtoc_test_fdt 6.88
-tools_dtoc_test_src_scan 9.43
-tools_efivar 6.71
+tools_binman_fmap_util 6.94
+tools_binman_ftest 8.04
+tools_binman_image 7.29
+tools_binman_image_test 5.52
+tools_binman_main 5.63
+tools_binman_setup 0.00
+tools_binman_state 4.88
+tools_buildman_board 6.36
+tools_buildman_boards 9.72
+tools_buildman_bsettings 5.00
+tools_buildman_builder 7.66
+tools_buildman_builderthread 9.63
+tools_buildman_cfgutil 10.00
+tools_buildman_cmdline 10.00
+tools_buildman_control 9.26
+tools_buildman_func_test 8.38
+tools_buildman_kconfiglib 8.33
+tools_buildman_main 8.10
+tools_buildman_test 7.16
+tools_buildman_toolchain 6.99
+tools_dtoc_dtb_platdata 8.10
+tools_dtoc_fdt 6.31
+tools_dtoc_fdt_util 7.62
+tools_dtoc_main 8.54
+tools_dtoc_setup 0.00
+tools_dtoc_src_scan 9.14
+tools_dtoc_test_dtoc 8.97
+tools_dtoc_test_fdt 9.93
+tools_dtoc_test_src_scan 9.46
+tools_efivar 7.39
 tools_endian-swap 9.29
-tools_microcode-tool 7.25
-tools_moveconfig 8.34
+tools_expo 9.72
+tools_key2dtsi 7.14
+tools_microcode-tool 6.55
 tools_patman___init__ 0.00
-tools_patman_checkpatch 8.48
-tools_patman_command 5.51
-tools_patman_commit 4.50
-tools_patman_control 8.14
-tools_patman_cros_subprocess 7.76
-tools_patman_func_test 8.51
-tools_patman_get_maintainer 7.06
-tools_patman_gitutil 6.65
-tools_patman_main 7.90
-tools_patman_patchstream 9.11
+tools_patman___main__ 9.44
+tools_patman_checkpatch 8.90
+tools_patman_cmdline 10.00
+tools_patman_commit 6.43
+tools_patman_control 8.29
+tools_patman_func_test 9.02
+tools_patman_get_maintainer 7.50
+tools_patman_gitutil 7.37
+tools_patman_patchstream 9.21
 tools_patman_project 7.78
-tools_patman_series 6.16
-tools_patman_settings 5.89
+tools_patman_series 7.54
+tools_patman_settings 7.94
 tools_patman_setup 5.00
-tools_patman_status 8.62
-tools_patman_terminal 8.00
-tools_patman_test_checkpatch 7.75
-tools_patman_test_util 7.64
-tools_patman_tools 5.68
-tools_patman_tout 5.31
-tools_rkmux 6.90
-tools_rmboard 7.76
+tools_patman_status 8.52
+tools_patman_test_checkpatch 8.51
+tools_patman_test_settings 8.78
+tools_qconfig 9.79
+tools_rkmux 7.10
+tools_rmboard 8.06
+tools_u_bootlib___init__.py 0.00
+tools_u_bootlib___main__.py 7.78
+tools_u_bootlib_command.py 6.48
+tools_u_bootlib_cros_subprocess.py 9.25
+tools_u_bootlib_terminal.py 8.50
+tools_u_bootlib_test_util.py 7.31
+tools_u_bootlib_tools.py 6.97
+tools_u_bootlib_tout.py 6.56
 tools_zynqmp_pm_cfg_obj_convert 6.67
-- 
2.34.1

[PATCH v2 06/39] bootstd: Create a function to reset USB

[Simon Glass]

Set up a function for this, since it needs to be used from multiple test
files.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/boot/bootdev.c        | 19 ++++++-------------
 test/boot/bootstd_common.c | 12 ++++++++++++
 test/boot/bootstd_common.h |  8 ++++++++
 3 files changed, 26 insertions(+), 13 deletions(-)

diff --git a/test/boot/bootdev.c b/test/boot/bootdev.c
index 1bf5929c396..de16a51956d 100644
--- a/test/boot/bootdev.c
+++ b/test/boot/bootdev.c
@@ -16,13 +16,6 @@
 #include <test/ut.h>
 #include "bootstd_common.h"
 
-/* Allow reseting the USB-started flag */
-#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
-extern bool usb_started;
-#else
-#include <usb.h>
-#endif
-
 /* Check 'bootdev list' command */
 static int bootdev_test_cmd_list(struct unit_test_state *uts)
 {
@@ -201,7 +194,7 @@ static int bootdev_test_order(struct unit_test_state *uts)
 	test_set_skip_delays(true);
 
 	/* Start up USB which gives us three additional bootdevs */
-	usb_started = false;
+	bootstd_reset_usb();
 	ut_assertok(run_command("usb start", 0));
 
 	/*
@@ -317,7 +310,7 @@ static int bootdev_test_prio(struct unit_test_state *uts)
 	test_set_eth_enable(false);
 
 	/* Start up USB which gives us three additional bootdevs */
-	usb_started = false;
+	bootstd_reset_usb();
 	ut_assertok(run_command("usb start", 0));
 
 	ut_assertok(bootstd_test_drop_bootdev_order(uts));
@@ -357,7 +350,7 @@ static int bootdev_test_hunter(struct unit_test_state *uts)
 {
 	struct bootstd_priv *std;
 
-	usb_started = false;
+	bootstd_reset_usb();
 	test_set_skip_delays(true);
 
 	/* get access to the used hunters */
@@ -398,7 +391,7 @@ static int bootdev_test_cmd_hunt(struct unit_test_state *uts)
 	struct bootstd_priv *std;
 
 	test_set_skip_delays(true);
-	usb_started = false;
+	bootstd_reset_usb();
 
 	/* get access to the used hunters */
 	ut_assertok(bootstd_get_priv(&std));
@@ -527,7 +520,7 @@ BOOTSTD_TEST(bootdev_test_bootable, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
 /* Check hunting for bootdev of a particular priority */
 static int bootdev_test_hunt_prio(struct unit_test_state *uts)
 {
-	usb_started = false;
+	bootstd_reset_usb();
 	test_set_skip_delays(true);
 
 	console_record_reset_enable();
@@ -556,7 +549,7 @@ static int bootdev_test_hunt_label(struct unit_test_state *uts)
 	struct bootstd_priv *std;
 	int mflags;
 
-	usb_started = false;
+	bootstd_reset_usb();
 
 	/* get access to the used hunters */
 	ut_assertok(bootstd_get_priv(&std));
diff --git a/test/boot/bootstd_common.c b/test/boot/bootstd_common.c
index e50539500a0..ff0aff4bbe7 100644
--- a/test/boot/bootstd_common.c
+++ b/test/boot/bootstd_common.c
@@ -20,6 +20,13 @@
 /* tracks whether bootstd_setup_for_tests() has been run yet */
 bool vbe_setup_done;
 
+/* Allow resetting the USB-started flag */
+#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
+extern bool usb_started;
+#else
+#include <usb.h>
+#endif
+
 /* set up MMC for VBE tests */
 int bootstd_setup_for_tests(void)
 {
@@ -88,6 +95,11 @@ int bootstd_test_check_mmc_hunter(struct unit_test_state *uts)
 	return 0;
 }
 
+void bootstd_reset_usb(void)
+{
+	usb_started = false;
+}
+
 int do_ut_bootstd(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 {
 	struct unit_test *tests = UNIT_TEST_SUITE_START(bootstd_test);
diff --git a/test/boot/bootstd_common.h b/test/boot/bootstd_common.h
index 4a126e43ff4..e29036c897c 100644
--- a/test/boot/bootstd_common.h
+++ b/test/boot/bootstd_common.h
@@ -53,4 +53,12 @@ int bootstd_setup_for_tests(void);
  */
 int bootstd_test_check_mmc_hunter(struct unit_test_state *uts);
 
+/**
+ * bootstd_reset_usb() - Reset the USB subsystem
+ *
+ * Resets USB so that it can be started (and scanning) again. This is useful in
+ * tests which need to use USB.
+ */
+void bootstd_reset_usb(void);
+
 #endif
-- 
2.34.1

[PATCH v2 07/39] usb: Drop old non-DM code

[Simon Glass]

The driver model deadline for USB was in 2019, so drop the old USB
keyboard code, to avoid needing to deal with the extra code path.

Drop the unnecessary #ifdef around USB_KBD_BOOT_REPORT_SIZE while we
are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 cmd/usb.c           | 20 --------------
 common/usb_kbd.c    | 67 ---------------------------------------------
 drivers/usb/Kconfig |  3 +-
 include/usb.h       |  8 ------
 4 files changed, 2 insertions(+), 96 deletions(-)

diff --git a/cmd/usb.c b/cmd/usb.c
index 16c081bf128..13a2996c1f0 100644
--- a/cmd/usb.c
+++ b/cmd/usb.c
@@ -560,17 +560,6 @@ static int do_usbboot(struct cmd_tbl *cmdtp, int flag, int argc,
 }
 #endif /* CONFIG_USB_STORAGE */
 
-static int do_usb_stop_keyboard(int force)
-{
-#if !defined CONFIG_DM_USB && defined CONFIG_USB_KEYBOARD
-	if (usb_kbd_deregister(force) != 0) {
-		printf("USB not stopped: usbkbd still using USB\n");
-		return 1;
-	}
-#endif
-	return 0;
-}
-
 static void do_usb_start(void)
 {
 	bootstage_mark_name(BOOTSTAGE_ID_USB_START, "usb_start");
@@ -583,11 +572,6 @@ static void do_usb_start(void)
 	/* try to recognize storage devices immediately */
 	usb_stor_curr_dev = usb_stor_scan(1);
 # endif
-#ifndef CONFIG_DM_USB
-# ifdef CONFIG_USB_KEYBOARD
-	drv_usb_kbd_init();
-# endif
-#endif /* !CONFIG_DM_USB */
 }
 
 #ifdef CONFIG_DM_USB
@@ -633,8 +617,6 @@ static int do_usb(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 
 	if (strncmp(argv[1], "reset", 5) == 0) {
 		printf("resetting USB...\n");
-		if (do_usb_stop_keyboard(1) != 0)
-			return 1;
 		usb_stop();
 		do_usb_start();
 		return 0;
@@ -642,8 +624,6 @@ static int do_usb(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 	if (strncmp(argv[1], "stop", 4) == 0) {
 		if (argc != 2)
 			console_assign(stdin, "serial");
-		if (do_usb_stop_keyboard(0) != 0)
-			return 1;
 		printf("stopping USB..\n");
 		usb_stop();
 		return 0;
diff --git a/common/usb_kbd.c b/common/usb_kbd.c
index f3b4a3c94e6..b834b2f703d 100644
--- a/common/usb_kbd.c
+++ b/common/usb_kbd.c
@@ -643,71 +643,6 @@ static int probe_usb_keyboard(struct usb_device *dev)
 	return 0;
 }
 
-#if !CONFIG_IS_ENABLED(DM_USB)
-/* Search for keyboard and register it if found. */
-int drv_usb_kbd_init(void)
-{
-	int error, i;
-
-	debug("%s: Probing for keyboard\n", __func__);
-	/* Scan all USB Devices */
-	for (i = 0; i < USB_MAX_DEVICE; i++) {
-		struct usb_device *dev;
-
-		/* Get USB device. */
-		dev = usb_get_dev_index(i);
-		if (!dev)
-			break;
-
-		if (dev->devnum == -1)
-			continue;
-
-		error = probe_usb_keyboard(dev);
-		if (!error)
-			return 1;
-		if (error && error != -ENOENT)
-			return error;
-	}
-
-	/* No USB Keyboard found */
-	return -1;
-}
-
-/* Deregister the keyboard. */
-int usb_kbd_deregister(int force)
-{
-#if CONFIG_IS_ENABLED(SYS_STDIO_DEREGISTER)
-	struct stdio_dev *dev;
-	struct usb_device *usb_kbd_dev;
-	struct usb_kbd_pdata *data;
-
-	dev = stdio_get_by_name(DEVNAME);
-	if (dev) {
-		usb_kbd_dev = (struct usb_device *)dev->priv;
-		data = usb_kbd_dev->privptr;
-#if CONFIG_IS_ENABLED(CONSOLE_MUX)
-		if (iomux_replace_device(stdin, DEVNAME, force ? "nulldev" : ""))
-			return 1;
-#endif
-		if (stdio_deregister_dev(dev, force) != 0)
-			return 1;
-#ifdef CONFIG_SYS_USB_EVENT_POLL_VIA_INT_QUEUE
-		destroy_int_queue(usb_kbd_dev, data->intq);
-#endif
-		free(data->new);
-		free(data);
-	}
-
-	return 0;
-#else
-	return 1;
-#endif
-}
-
-#endif
-
-#if CONFIG_IS_ENABLED(DM_USB)
-
 static int usb_kbd_probe(struct udevice *dev)
 {
 	struct usb_device *udev = dev_get_parent_priv(dev);
@@ -788,5 +723,3 @@ static const struct usb_device_id kbd_id_table[] = {
 };
 
 U_BOOT_USB_DEVICE(usb_kbd, kbd_id_table);
-
-#endif
diff --git a/drivers/usb/Kconfig b/drivers/usb/Kconfig
index a972d87c7ad..311aaa7e67f 100644
--- a/drivers/usb/Kconfig
+++ b/drivers/usb/Kconfig
@@ -99,7 +99,8 @@ config USB_STORAGE
 
 config USB_KEYBOARD
 	bool "USB Keyboard support"
-	select DM_KEYBOARD if DM_USB
+	depends on DM_USB
+	select DM_KEYBOARD
 	select SYS_STDIO_DEREGISTER
 	---help---
 	  Say Y here if you want to use a USB keyboard for U-Boot command line
diff --git a/include/usb.h b/include/usb.h
index fcbe2146f7d..e37f853482c 100644
--- a/include/usb.h
+++ b/include/usb.h
@@ -250,20 +250,12 @@ int usb_host_eth_scan(int mode);
 
 #endif
 
-#ifdef CONFIG_USB_KEYBOARD
-
 /*
  * USB Keyboard reports are 8 bytes in boot protocol.
  * Appendix B of HID Device Class Definition 1.11
  */
 #define USB_KBD_BOOT_REPORT_SIZE 8
 
-int drv_usb_kbd_init(void);
-int usb_kbd_deregister(int force);
-
-#endif
-/* routines */
-
 /*
  * usb_init() - initialize the USB Controllers
  *
-- 
2.34.1

[PATCH v2 08/39] log: Add a new log category for the console

[Simon Glass]

Add a new category which covers the console, including the stdio
drivers.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 common/console.c | 2 ++
 common/log.c     | 1 +
 include/log.h    | 2 ++
 3 files changed, 5 insertions(+)

diff --git a/common/console.c b/common/console.c
index 63f78004fdb..034942ec4a9 100644
--- a/common/console.c
+++ b/common/console.c
@@ -4,6 +4,8 @@
  * Paolo Scaffardi, AIRVENT SAM s.p.a - RIMINI(ITALY), arsenio@tin.it
  */
 
+#define LOG_CATEGORY	LOGC_CONSOLE
+
 #include <console.h>
 #include <debug_uart.h>
 #include <display_options.h>
diff --git a/common/log.c b/common/log.c
index dfee250b158..b83a6618900 100644
--- a/common/log.c
+++ b/common/log.c
@@ -31,6 +31,7 @@ static const char *const log_cat_name[] = {
 	"event",
 	"fs",
 	"expo",
+	"console",
 };
 
 _Static_assert(ARRAY_SIZE(log_cat_name) == LOGC_COUNT - LOGC_NONE,
diff --git a/include/log.h b/include/log.h
index fc0d5984472..2b86c7da3d4 100644
--- a/include/log.h
+++ b/include/log.h
@@ -104,6 +104,8 @@ enum log_category_t {
 	LOGC_FS,
 	/** @LOGC_EXPO: Related to expo handling */
 	LOGC_EXPO,
+	/** @LOGC_CONSOLE: Related to the console and stdio */
+	LOGC_CONSOLE,
 	/** @LOGC_COUNT: Number of log categories */
 	LOGC_COUNT,
 	/** @LOGC_END: Sentinel value for lists of log categories */
-- 
2.34.1

[PATCH v2 09/39] usb: Add DEV_FLAGS_DM to stdio for USB keyboard

[Simon Glass]

This device contains a pointer to struct udevice so set the flag
indicating that, just to be tidy.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 common/usb_kbd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/common/usb_kbd.c b/common/usb_kbd.c
index b834b2f703d..1d9845b01ed 100644
--- a/common/usb_kbd.c
+++ b/common/usb_kbd.c
@@ -612,7 +612,7 @@ static int probe_usb_keyboard(struct usb_device *dev)
 	debug("USB KBD: register.\n");
 	memset(&usb_kbd_dev, 0, sizeof(struct stdio_dev));
 	strcpy(usb_kbd_dev.name, DEVNAME);
-	usb_kbd_dev.flags =  DEV_FLAGS_INPUT;
+	usb_kbd_dev.flags = DEV_FLAGS_INPUT | DEV_FLAGS_DM;
 	usb_kbd_dev.getc = usb_kbd_getc;
 	usb_kbd_dev.tstc = usb_kbd_testc;
 	usb_kbd_dev.priv = (void *)dev;
-- 
2.34.1

[PATCH v2 10/39] dm: usb: Deal with USB keyboard persisting across tests

[Simon Glass]

Clear any USB-keyboard devices before running a unit test, to avoid
using a stale udevice pointer in stdio. Add a long comment to explain
this situation and why this solution seems best, at least for now.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 common/console.c  | 34 ++++++++++++++++++++++++++++++++++
 common/usb_kbd.c  |  5 +++++
 include/console.h |  8 ++++++++
 include/usb.h     | 12 ++++++++++++
 test/test-main.c  | 38 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 97 insertions(+)

diff --git a/common/console.c b/common/console.c
index 034942ec4a9..17dceba40b8 100644
--- a/common/console.c
+++ b/common/console.c
@@ -1241,3 +1241,37 @@ int console_init_r(void)
 }
 
 #endif /* CONFIG_IS_ENABLED(SYS_CONSOLE_IS_IN_ENV) */
+
+int console_remove_by_name(const char *name)
+{
+	int err = 0;
+
+#if CONFIG_IS_ENABLED(CONSOLE_MUX)
+	int fnum;
+
+	log_debug("removing console device %s\n", name);
+	for (fnum = 0; fnum < MAX_FILES; fnum++) {
+		struct stdio_dev **src, **dest;
+		int i;
+
+		log_debug("file %d: %d devices: ", fnum, cd_count[fnum]);
+		src = console_devices[fnum];
+		dest = src;
+		for (i = 0; i < cd_count[fnum]; i++, src++) {
+			struct stdio_dev *sdev = *src;
+			int ret = 0;
+
+			if (!strcmp(sdev->name, name))
+				ret = stdio_deregister_dev(sdev, true);
+			else
+				*dest++ = *src;
+			if (ret && !err)
+				err = ret;
+		}
+		cd_count[fnum] = dest - console_devices[fnum];
+		log_debug("now %d\n", cd_count[fnum]);
+	}
+#endif /* CONSOLE_MUX */
+
+	return err;
+}
diff --git a/common/usb_kbd.c b/common/usb_kbd.c
index 1d9845b01ed..bbfee23bc26 100644
--- a/common/usb_kbd.c
+++ b/common/usb_kbd.c
@@ -137,6 +137,11 @@ extern int __maybe_unused net_busy_flag;
 /* The period of time between two calls of usb_kbd_testc(). */
 static unsigned long kbd_testc_tms;
 
+int usb_kbd_remove_for_test(void)
+{
+	return console_remove_by_name(DEVNAME);
+}
+
 /* Puts character in the queue and sets up the in and out pointer. */
 static void usb_kbd_put_queue(struct usb_kbd_pdata *data, u8 c)
 {
diff --git a/include/console.h b/include/console.h
index 2617e160073..125f498facb 100644
--- a/include/console.h
+++ b/include/console.h
@@ -179,6 +179,14 @@ void console_puts_select_stderr(bool serial_only, const char *s);
  */
 int console_clear(void);
 
+/**
+ * console_remove_by_name() - Remove a console by its stdio name
+ *
+ * This must only be used in tests. It removes any use of the named stdio device
+ * from the console tables.
+ */
+int console_remove_by_name(const char *name);
+
 /*
  * CONSOLE multiplexing.
  */
diff --git a/include/usb.h b/include/usb.h
index e37f853482c..be37ed272e1 100644
--- a/include/usb.h
+++ b/include/usb.h
@@ -1092,4 +1092,16 @@ struct usb_generic_descriptor **usb_emul_find_descriptor(
  */
 void usb_show_tree(void);
 
+/**
+ * usb_kbd_remove_for_test() - Remove any USB keyboard
+ *
+ * This can only be called from test_pre_run(). It removes the USB keyboard from
+ * the console system so that the USB device can be dropped
+ */
+#if CONFIG_IS_ENABLED(USB_KEYBOARD)
+int usb_kbd_remove_for_test(void);
+#else
+static inline int usb_kbd_remove_for_test(void) { return 0; }
+#endif
+
 #endif /*_USB_H_ */
diff --git a/test/test-main.c b/test/test-main.c
index 3fa6f6e32ec..bfe9f707dc1 100644
--- a/test/test-main.c
+++ b/test/test-main.c
@@ -12,6 +12,7 @@
 #include <net.h>
 #include <of_live.h>
 #include <os.h>
+#include <usb.h>
 #include <dm/ofnode.h>
 #include <dm/root.h>
 #include <dm/test.h>
@@ -289,6 +290,43 @@ static int test_pre_run(struct unit_test_state *uts, struct unit_test *test)
 {
 	ut_assertok(event_init());
 
+	/*
+	 * Remove any USB keyboard, so that we can add and remove USB devices
+	 * in tests.
+	 *
+	 * For UT_TESTF_DM tests, the old driver model state is saved and
+	 * restored across each test. Within in each test there is therefore a
+	 * new driver model state, which means that any USB keyboard device in
+	 * stdio points to the old state.
+	 *
+	 * This is fine in most cases. But if a non-UT_TESTF_DM test starts up
+	 * USB (thus creating a stdio record pointing to the USB keyboard
+	 * device) then when the test finishes, the new driver model state is
+	 * freed, meaning that there is now a stale pointer in stdio.
+	 *
+	 * This means that any future UT_TESTF_DM test which uses stdin will
+	 * cause the console system to call tstc() on the stale device pointer,
+	 * causing a crash.
+	 *
+	 * We don't want to fix this by enabling UT_TESTF_DM for all tests as
+	 * this causes other problems. For example, bootflow_efi relies on
+	 * U-Boot going through a proper init - without that we don't have the
+	 * TCG measurement working and get an error
+	 * 'tcg2 measurement fails(0x8000000000000007)'. Once we tidy up how EFI
+	 * runs tests (e.g. get rid of all the restarting of U-Boot) we could
+	 * potentially make the bootstd tests set UT_TESTF_DM, but other tests
+	 * might do the same thing.
+	 *
+	 * We could add a test flag to declare that USB is being used, but that
+	 * seems unnecessary, at least for now. We could detect USB being used
+	 * in a test, but there is no obvious drawback to clearing out stale
+	 * pointers always.
+	 *
+	 * So just remove any USB keyboards from the console tables. This allows
+	 * UT_TESTF_DM and non-UT_TESTF_DM tests to coexist happily.
+	 */
+	usb_kbd_remove_for_test();
+
 	if (test->flags & UT_TESTF_DM)
 		ut_assertok(dm_test_pre_run(uts));
 
-- 
2.34.1

[PATCH v2 11/39] test: mbr: Adjust test to use lower-case hex

[Simon Glass]

Switch to lower-case hex which is more commonly used in U-Boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/cmd/mbr.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
index 235b363290e..4c884af10c7 100644
--- a/test/cmd/mbr.c
+++ b/test/cmd/mbr.c
@@ -50,7 +50,7 @@ static char * mbr_parts_tail = "'";
 000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
 */
-static unsigned mbr_cmp_start = 0x1B8;
+static unsigned int mbr_cmp_start = 0x1b8;
 static unsigned mbr_cmp_size = 0x48;
 static unsigned char mbr_parts_ref_p1[] = {
                                                 0x78, 0x56, 0x34, 0x12, 0x00, 0x00, 0x80, 0x05,
@@ -257,7 +257,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	mbr_wa = map_to_sysmem(mbr_wbuf);
 	ebr_wa = map_to_sysmem(ebr_wbuf);
 	ra = map_to_sysmem(rbuf);
-	ebr_blk = (ulong)0xB00000 / 0x200;
+	ebr_blk = (ulong)0xb00000 / 0x200;
 
 	/* Make sure mmc6 exists */
 	ut_asserteq(6, blk_get_device_by_str("mmc", "6", &mmc_dev_desc));
@@ -268,7 +268,8 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(ut_check_console_end(uts));
 
 	/* Make sure mmc6 is 12+ MiB in size */
-	ut_assertok(run_commandf("mmc read 0x%lx 0x%lx 1", ra, (ulong)0xBFFE00 / 0x200));
+	ut_assertok(run_commandf("mmc read 0x%lx 0x%lx 1", ra,
+				 (ulong)0xbffe00 / 0x200));
 
 	/* Test one MBR partition */
 	init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
-- 
2.34.1

[PATCH v2 12/39] test: mbr: Adjust test to drop 0x

[Simon Glass]

U-Boot commands typically don't need 0x to specify hex, since they use
hex by default. Adding 0x in this test is confusing since it suggests
that it is necessary. Drop it from the file.

Also use the %#x construct to get the 0x when needed.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/cmd/mbr.c | 74 +++++++++++++++++++++++++-------------------------
 1 file changed, 37 insertions(+), 37 deletions(-)

diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
index 4c884af10c7..1a35ba3351c 100644
--- a/test/cmd/mbr.c
+++ b/test/cmd/mbr.c
@@ -268,19 +268,19 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(ut_check_console_end(uts));
 
 	/* Make sure mmc6 is 12+ MiB in size */
-	ut_assertok(run_commandf("mmc read 0x%lx 0x%lx 1", ra,
+	ut_assertok(run_commandf("mmc read %lx %lx 1", ra,
 				 (ulong)0xbffe00 / 0x200));
 
 	/* Test one MBR partition */
 	init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
 	ut_assertok(build_mbr_parts(mbr_parts_buf, sizeof(mbr_parts_buf), 1));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0 1", mbr_wa));
+	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0x%lx 1", ebr_wa, ebr_blk));
+	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
@@ -289,7 +289,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("mbr verify mmc 6"));
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(ut_check_console_end(uts));
 	/*
@@ -300,23 +300,23 @@ static int mbr_test_run(struct unit_test_state *uts)
 	000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
 	*/
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	for (unsigned i = 0; i < mbr_cmp_size; i++) {
 		ut_assertf(rbuf[mbr_cmp_start + i] == mbr_parts_ref_p1[i],
-			"1P MBR+0x%04X: expected 0x%02X, actual: 0x%02X\n",
+			"1P MBR+0x%04X: expected %#02X, actual: %#02X\n",
 			mbr_cmp_start + i, mbr_parts_ref_p1[i], rbuf[mbr_cmp_start + i]);
 	}
 
 	/* Test two MBR partitions */
 	init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
 	ut_assertok(build_mbr_parts(mbr_parts_buf, sizeof(mbr_parts_buf), 2));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0 1", mbr_wa));
+	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0x%lx 1", ebr_wa, ebr_blk));
+	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
@@ -325,7 +325,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("mbr verify mmc 6"));
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(ut_check_console_end(uts));
 	/*
@@ -336,23 +336,23 @@ static int mbr_test_run(struct unit_test_state *uts)
 	000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
 	*/
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	for (unsigned i = 0; i < mbr_cmp_size; i++) {
 		ut_assertf(rbuf[mbr_cmp_start + i] == mbr_parts_ref_p2[i],
-			"2P MBR+0x%04X: expected 0x%02X, actual: 0x%02X\n",
+			"2P MBR+0x%04X: expected %#02X, actual: %#02X\n",
 			mbr_cmp_start + i, mbr_parts_ref_p2[i], rbuf[mbr_cmp_start + i]);
 	}
 
 	/* Test three MBR partitions */
 	init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
 	ut_assertok(build_mbr_parts(mbr_parts_buf, sizeof(mbr_parts_buf), 3));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0 1", mbr_wa));
+	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0x%lx 1", ebr_wa, ebr_blk));
+	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
@@ -361,7 +361,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("mbr verify mmc 6"));
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(ut_check_console_end(uts));
 	/*
@@ -372,23 +372,23 @@ static int mbr_test_run(struct unit_test_state *uts)
 	000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
 	*/
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	for (unsigned i = 0; i < mbr_cmp_size; i++) {
 		ut_assertf(rbuf[mbr_cmp_start + i] == mbr_parts_ref_p3[i],
-			"3P MBR+0x%04X: expected 0x%02X, actual: 0x%02X\n",
+			"3P MBR+0x%04X: expected %#02X, actual: %#02X\n",
 			mbr_cmp_start + i, mbr_parts_ref_p3[i], rbuf[mbr_cmp_start + i]);
 	}
 
 	/* Test four MBR partitions */
 	init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
 	ut_assertok(build_mbr_parts(mbr_parts_buf, sizeof(mbr_parts_buf), 4));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0 1", mbr_wa));
+	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0x%lx 1", ebr_wa, ebr_blk));
+	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
@@ -397,7 +397,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("mbr verify mmc 6"));
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(ut_check_console_end(uts));
 	/*
@@ -408,23 +408,23 @@ static int mbr_test_run(struct unit_test_state *uts)
 	000001f0  26 01 0e 87 06 01 00 58  00 00 00 08 00 00 55 aa  |&......X......U.|
 	*/
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	for (unsigned i = 0; i < mbr_cmp_size; i++) {
 		ut_assertf(rbuf[mbr_cmp_start + i] == mbr_parts_ref_p4[i],
-			"4P MBR+0x%04X: expected 0x%02X, actual: 0x%02X\n",
+			"4P MBR+0x%04X: expected %#02X, actual: %#02X\n",
 			mbr_cmp_start + i, mbr_parts_ref_p4[i], rbuf[mbr_cmp_start + i]);
 	}
 
 	/* Test five MBR partitions */
 	init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
 	ut_assertok(build_mbr_parts(mbr_parts_buf, sizeof(mbr_parts_buf), 5));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0 1", mbr_wa));
+	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
-	ut_assertok(run_commandf("write mmc 6:0 0x%lx 0x%lx 1", ebr_wa, ebr_blk));
+	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
@@ -441,10 +441,10 @@ static int mbr_test_run(struct unit_test_state *uts)
 	000001f0  26 01 05 a7 26 01 00 58  00 00 00 10 00 00 55 aa  |&...&..X......U.|
 	*/
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0 1", ra));
+	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
 	for (unsigned i = 0; i < mbr_cmp_size; i++) {
 		ut_assertf(rbuf[mbr_cmp_start + i] == mbr_parts_ref_p5[i],
-			"5P MBR+0x%04X: expected 0x%02X, actual: 0x%02X\n",
+			"5P MBR+0x%04X: expected %#02X, actual: %#02X\n",
 			mbr_cmp_start + i, mbr_parts_ref_p5[i], rbuf[mbr_cmp_start + i]);
 	}
 	/*
@@ -455,10 +455,10 @@ static int mbr_test_run(struct unit_test_state *uts)
 	00b001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
 	*/
 	memset(rbuf, 0, sizeof(rbuf));
-	ut_assertok(run_commandf("read mmc 6:0 0x%lx 0x%lx 1", ra, ebr_blk));
+	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
 	for (unsigned i = 0; i < ebr_cmp_size; i++) {
 		ut_assertf(rbuf[ebr_cmp_start + i] == ebr_parts_ref_p5[i],
-			"5P EBR+0x%04X: expected 0x%02X, actual: 0x%02X\n",
+			"5P EBR+0x%04X: expected %#02X, actual: %#02X\n",
 			ebr_cmp_start + i, ebr_parts_ref_p5[i], rbuf[ebr_cmp_start + i]);
 	}
 
-- 
2.34.1

[PATCH v2 13/39] sandbox: Change the range used for memory-mapping tags

[Simon Glass]

Sandbox keeps a table of addresses which map to pointers which are
outside its emulated DRAM. The current range from 10000000 conflicts
with the PCI range, meaning that if PCI mapping is on, that particular
address can be decoded by PCI instead of the table.

Fix this by moving the range up to the top of memory. Update the docs
while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 arch/sandbox/cpu/state.c     |  9 +++++----
 doc/arch/sandbox/sandbox.rst | 21 ++++++++++++---------
 2 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/arch/sandbox/cpu/state.c b/arch/sandbox/cpu/state.c
index a9ca79e76d2..49236db99c2 100644
--- a/arch/sandbox/cpu/state.c
+++ b/arch/sandbox/cpu/state.c
@@ -373,12 +373,13 @@ void state_reset_for_test(struct sandbox_state *state)
 	memset(state->spi, '\0', sizeof(state->spi));
 
 	/*
-	 * Set up the memory tag list. Use the top of emulated SDRAM for the
-	 * first tag number, since that address offset is outside the legal
-	 * range, and can be assumed to be a tag.
+	 * Set up the memory tag list. We could use the top of emulated SDRAM
+	 * for the first tag number, since that address offset is outside the
+	 * legal SDRAM range, but PCI can have address there. So use a very
+	 * large address instead
 	 */
 	INIT_LIST_HEAD(&state->mapmem_head);
-	state->next_tag = state->ram_size;
+	state->next_tag = 0xff000000;
 }
 
 bool autoboot_keyed(void)
diff --git a/doc/arch/sandbox/sandbox.rst b/doc/arch/sandbox/sandbox.rst
index 5f8db126657..1515f93c84b 100644
--- a/doc/arch/sandbox/sandbox.rst
+++ b/doc/arch/sandbox/sandbox.rst
@@ -655,14 +655,17 @@ Memory Map
 Sandbox has its own emulated memory starting at 0. Here are some of the things
 that are mapped into that memory:
 
-=======   ========================   ===============================
+========  ========================   ===============================
 Addr      Config                     Usage
-=======   ========================   ===============================
-    100   CONFIG_SYS_FDT_LOAD_ADDR   Device tree
-   b000   CONFIG_BLOBLIST_ADDR       Blob list
-  10000   CFG_MALLOC_F_ADDR          Early memory allocation
-  f0000   CONFIG_PRE_CON_BUF_ADDR    Pre-console buffer
- 100000   CONFIG_TRACE_EARLY_ADDR    Early trace buffer (if enabled). Also used
+========  ========================   ===============================
+     100  CONFIG_SYS_FDT_LOAD_ADDR   Device tree
+    b000  CONFIG_BLOBLIST_ADDR       Blob list
+   10000  CFG_MALLOC_F_ADDR          Early memory allocation
+   f0000  CONFIG_PRE_CON_BUF_ADDR    Pre-console buffer
+  100000  CONFIG_TRACE_EARLY_ADDR    Early trace buffer (if enabled). Also used
                                      as the SPL load buffer in spl_test_load().
- 200000   CONFIG_TEXT_BASE           Load buffer for U-Boot (sandbox_spl only)
-=======   ========================   ===============================
+  200000  CONFIG_TEXT_BASE           Load buffer for U-Boot (sandbox_spl only)
+10000000                             PCI address space (see test.dts)
+
+ff000000                             Memory-mapping tags start here
+========  ========================   ===============================
-- 
2.34.1

[PATCH v2 14/39] sandbox: Update cpu to use logging

[Simon Glass]

Use log_debug() instead of including the function name in the string.
Add one more debug for PCI.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 arch/sandbox/cpu/cpu.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/arch/sandbox/cpu/cpu.c b/arch/sandbox/cpu/cpu.c
index 0ed85b354cf..d1500dcf32d 100644
--- a/arch/sandbox/cpu/cpu.c
+++ b/arch/sandbox/cpu/cpu.c
@@ -109,8 +109,8 @@ void *phys_to_virt(phys_addr_t paddr)
 	state = state_get_current();
 	list_for_each_entry(mentry, &state->mapmem_head, sibling_node) {
 		if (mentry->tag == paddr) {
-			debug("%s: Used map from %lx to %p\n", __func__,
-			      (ulong)paddr, mentry->ptr);
+			log_debug("Used map from %lx to %p\n", (ulong)paddr,
+				  mentry->ptr);
 			return mentry->ptr;
 		}
 	}
@@ -130,11 +130,12 @@ struct sandbox_mapmem_entry *find_tag(const void *ptr)
 
 	list_for_each_entry(mentry, &state->mapmem_head, sibling_node) {
 		if (mentry->ptr == ptr) {
-			debug("%s: Used map from %p to %lx\n", __func__, ptr,
-			      mentry->tag);
+			log_debug("Used map from %p to %lx\n", ptr,
+				  mentry->tag);
 			return mentry;
 		}
 	}
+
 	return NULL;
 }
 
@@ -156,7 +157,7 @@ phys_addr_t virt_to_phys(void *ptr)
 		       __func__, ptr, (ulong)gd->ram_size);
 		os_abort();
 	}
-	debug("%s: Used map from %p to %lx\n", __func__, ptr, mentry->tag);
+	log_debug("Used map from %p to %lx\n", ptr, mentry->tag);
 
 	return mentry->tag;
 }
@@ -174,6 +175,7 @@ void *map_physmem(phys_addr_t paddr, unsigned long len, unsigned long flags)
 			       __func__, (uint)paddr, len, plen);
 		}
 		map_len = len;
+		log_debug("pci map %lx -> %p\n", (ulong)paddr, ptr);
 		return ptr;
 	}
 #endif
@@ -218,8 +220,8 @@ phys_addr_t map_to_sysmem(const void *ptr)
 		mentry->tag = state->next_tag++;
 		mentry->ptr = (void *)ptr;
 		list_add_tail(&mentry->sibling_node, &state->mapmem_head);
-		debug("%s: Added map from %p to %lx\n", __func__, ptr,
-		      (ulong)mentry->tag);
+		log_debug("Added map from %p to %lx\n", ptr,
+			  (ulong)mentry->tag);
 	}
 
 	/*
-- 
2.34.1

[PATCH v2 15/39] sandbox: Unmap old tags

[Simon Glass]

So far unmapping has not been implemented. This means that if one test
maps a pointer to an address with map_sysmem(), then a second test can
use that same pointer, by mapping the address back to a pointer with
map_to_sysmem(). This is not really desirable, even if it doesn't
cause any problems at the moment.

Implement unmapping, to clean this up.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 arch/sandbox/cpu/cpu.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/arch/sandbox/cpu/cpu.c b/arch/sandbox/cpu/cpu.c
index d1500dcf32d..ac00be16cd7 100644
--- a/arch/sandbox/cpu/cpu.c
+++ b/arch/sandbox/cpu/cpu.c
@@ -185,12 +185,28 @@ void *map_physmem(phys_addr_t paddr, unsigned long len, unsigned long flags)
 
 void unmap_physmem(const void *ptr, unsigned long flags)
 {
+	struct sandbox_mapmem_entry *mentry;
+
 #ifdef CONFIG_PCI
 	if (map_dev) {
 		pci_unmap_physmem(ptr, map_len, map_dev);
 		map_dev = NULL;
 	}
 #endif
+
+	/* If it is in emulated RAM, we didn't create a tag, so nothing to do */
+	if (is_in_sandbox_mem(ptr))
+		return;
+
+	mentry = find_tag(ptr);
+	if (mentry) {
+		list_del(&mentry->sibling_node);
+		log_debug("Removed map from %p to %lx\n", ptr,
+			  (ulong)mentry->tag);
+		free(mentry);
+	} else {
+		log_warning("Address not mapped: %p\n", ptr);
+	}
 }
 
 phys_addr_t map_to_sysmem(const void *ptr)
-- 
2.34.1

[PATCH v2 16/39] sandbox: Add some debugging to pci_io

[Simon Glass]

Add a little debugging to this driver. Convert the existing debugging to
use logging.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 arch/sandbox/lib/pci_io.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/sandbox/lib/pci_io.c b/arch/sandbox/lib/pci_io.c
index 6040eacb594..5eff7c7d65d 100644
--- a/arch/sandbox/lib/pci_io.c
+++ b/arch/sandbox/lib/pci_io.c
@@ -8,6 +8,8 @@
  * IO space access commands.
  */
 
+#define LOG_CATEGORY	UCLASS_PCI
+
 #include <command.h>
 #include <dm.h>
 #include <log.h>
@@ -31,10 +33,11 @@ int pci_map_physmem(phys_addr_t paddr, unsigned long *lenp,
 		if (ret)
 			continue;
 		*devp = dev;
+		log_debug("addr=%lx, dev=%s\n", (ulong)paddr, dev->name);
 		return 0;
 	}
 
-	debug("%s: failed: addr=%pap\n", __func__, &paddr);
+	log_debug("%s: failed: addr=%pap\n", __func__, &paddr);
 	return -ENOSYS;
 }
 
@@ -66,7 +69,7 @@ static int pci_io_read(unsigned int addr, ulong *valuep, pci_size_t size)
 		}
 	}
 
-	debug("%s: failed: addr=%x\n", __func__, addr);
+	log_debug("%s: failed: addr=%x\n", __func__, addr);
 	return -ENOSYS;
 }
 
@@ -87,7 +90,7 @@ static int pci_io_write(unsigned int addr, ulong value, pci_size_t size)
 		}
 	}
 
-	debug("%s: failed: addr=%x, value=%lx\n", __func__, addr, value);
+	log_debug("%s: failed: addr=%x, value=%lx\n", __func__, addr, value);
 	return -ENOSYS;
 }
 
-- 
2.34.1

[PATCH v2 17/39] sandbox: Implement reference counting for address mapping

[Simon Glass]

An address may be mapped twice and unmapped twice. Delete the mapping
only when the last user unmaps it.

Fix a missing comment while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 arch/sandbox/cpu/cpu.c           | 14 ++++++++++----
 arch/sandbox/include/asm/state.h |  3 +++
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/arch/sandbox/cpu/cpu.c b/arch/sandbox/cpu/cpu.c
index ac00be16cd7..7a4621f7e40 100644
--- a/arch/sandbox/cpu/cpu.c
+++ b/arch/sandbox/cpu/cpu.c
@@ -111,6 +111,7 @@ void *phys_to_virt(phys_addr_t paddr)
 		if (mentry->tag == paddr) {
 			log_debug("Used map from %lx to %p\n", (ulong)paddr,
 				  mentry->ptr);
+			mentry->refcnt++;
 			return mentry->ptr;
 		}
 	}
@@ -200,10 +201,12 @@ void unmap_physmem(const void *ptr, unsigned long flags)
 
 	mentry = find_tag(ptr);
 	if (mentry) {
-		list_del(&mentry->sibling_node);
-		log_debug("Removed map from %p to %lx\n", ptr,
-			  (ulong)mentry->tag);
-		free(mentry);
+		if (!--mentry->refcnt) {
+			list_del(&mentry->sibling_node);
+			log_debug("Removed map from %p to %lx\n", ptr,
+				  (ulong)mentry->tag);
+			free(mentry);
+		}
 	} else {
 		log_warning("Address not mapped: %p\n", ptr);
 	}
@@ -235,11 +238,14 @@ phys_addr_t map_to_sysmem(const void *ptr)
 		}
 		mentry->tag = state->next_tag++;
 		mentry->ptr = (void *)ptr;
+		mentry->refcnt = 0;
 		list_add_tail(&mentry->sibling_node, &state->mapmem_head);
 		log_debug("Added map from %p to %lx\n", ptr,
 			  (ulong)mentry->tag);
 	}
 
+	mentry->refcnt++;
+
 	/*
 	 * Return the tag as the address to use. A later call to map_sysmem()
 	 * will return ptr
diff --git a/arch/sandbox/include/asm/state.h b/arch/sandbox/include/asm/state.h
index c84a1f7060f..8213e61c3ce 100644
--- a/arch/sandbox/include/asm/state.h
+++ b/arch/sandbox/include/asm/state.h
@@ -53,10 +53,13 @@ struct sandbox_wdt_info {
  * be returned, just as it would for a normal sandbox address.
  *
  * @tag: Address tag (a value which U-Boot uses to refer to the address)
+ * @refcnt: Number of references to this tag
  * @ptr: Associated pointer for that tag
+ * @sibling_node: Next node
  */
 struct sandbox_mapmem_entry {
 	ulong tag;
+	uint refcnt;
 	void *ptr;
 	struct list_head sibling_node;
 };
-- 
2.34.1

[PATCH v2 18/39] mmc: Use map_sysmem() with buffers in the mmc command

[Simon Glass]

The current implementation casts an address to a pointer. Make it more
sandbox-friendly by using map_sysmem().

Rename the variable to 'ptr' since it is a pointer, not an address.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 cmd/mmc.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/cmd/mmc.c b/cmd/mmc.c
index 7244a90f4dc..ff7ca3b371e 100644
--- a/cmd/mmc.c
+++ b/cmd/mmc.c
@@ -8,6 +8,7 @@
 #include <command.h>
 #include <console.h>
 #include <display_options.h>
+#include <mapmem.h>
 #include <memalign.h>
 #include <mmc.h>
 #include <part.h>
@@ -349,12 +350,12 @@ static int do_mmc_read(struct cmd_tbl *cmdtp, int flag,
 {
 	struct mmc *mmc;
 	u32 blk, cnt, n;
-	void *addr;
+	void *ptr;
 
 	if (argc != 4)
 		return CMD_RET_USAGE;
 
-	addr = (void *)hextoul(argv[1], NULL);
+	ptr = map_sysmem(hextoul(argv[1], NULL), 0);
 	blk = hextoul(argv[2], NULL);
 	cnt = hextoul(argv[3], NULL);
 
@@ -365,8 +366,9 @@ static int do_mmc_read(struct cmd_tbl *cmdtp, int flag,
 	printf("\nMMC read: dev # %d, block # %d, count %d ... ",
 	       curr_device, blk, cnt);
 
-	n = blk_dread(mmc_get_blk_desc(mmc), blk, cnt, addr);
+	n = blk_dread(mmc_get_blk_desc(mmc), blk, cnt, ptr);
 	printf("%d blocks read: %s\n", n, (n == cnt) ? "OK" : "ERROR");
+	unmap_sysmem(ptr);
 
 	return (n == cnt) ? CMD_RET_SUCCESS : CMD_RET_FAILURE;
 }
@@ -442,12 +444,12 @@ static int do_mmc_write(struct cmd_tbl *cmdtp, int flag,
 {
 	struct mmc *mmc;
 	u32 blk, cnt, n;
-	void *addr;
+	void *ptr;
 
 	if (argc != 4)
 		return CMD_RET_USAGE;
 
-	addr = (void *)hextoul(argv[1], NULL);
+	ptr = map_sysmem(hextoul(argv[1], NULL), 0);
 	blk = hextoul(argv[2], NULL);
 	cnt = hextoul(argv[3], NULL);
 
@@ -462,8 +464,9 @@ static int do_mmc_write(struct cmd_tbl *cmdtp, int flag,
 		printf("Error: card is write protected!\n");
 		return CMD_RET_FAILURE;
 	}
-	n = blk_dwrite(mmc_get_blk_desc(mmc), blk, cnt, addr);
+	n = blk_dwrite(mmc_get_blk_desc(mmc), blk, cnt, ptr);
 	printf("%d blocks written: %s\n", n, (n == cnt) ? "OK" : "ERROR");
+	unmap_sysmem(ptr);
 
 	return (n == cnt) ? CMD_RET_SUCCESS : CMD_RET_FAILURE;
 }
-- 
2.34.1

[PATCH v2 19/39] read: Use map_sysmem() with buffers in the read command

[Simon Glass]

The current implementation casts an address to a pointer. Make it more
sandbox-friendly by using map_sysmem().

Rename the variable to 'ptr' since it is a pointer, not an address.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 cmd/read.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/cmd/read.c b/cmd/read.c
index af54bd17654..8e21f004423 100644
--- a/cmd/read.c
+++ b/cmd/read.c
@@ -20,7 +20,7 @@ do_rw(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 	struct disk_partition part_info;
 	ulong offset, limit;
 	uint blk, cnt, res;
-	void *addr;
+	void *ptr;
 	int part;
 
 	if (argc != 6) {
@@ -33,7 +33,7 @@ do_rw(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 	if (part < 0)
 		return 1;
 
-	addr = map_sysmem(hextoul(argv[3], NULL), 0);
+	ptr = map_sysmem(hextoul(argv[3], NULL), 0);
 	blk = hextoul(argv[4], NULL);
 	cnt = hextoul(argv[5], NULL);
 
@@ -48,13 +48,15 @@ do_rw(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 
 	if (cnt + blk > limit) {
 		printf("%s out of range\n", cmdtp->name);
+		unmap_sysmem(ptr);
 		return 1;
 	}
 
 	if (IS_ENABLED(CONFIG_CMD_WRITE) && !strcmp(cmdtp->name, "write"))
-		res = blk_dwrite(dev_desc, offset + blk, cnt, addr);
+		res = blk_dwrite(dev_desc, offset + blk, cnt, ptr);
 	else
-		res = blk_dread(dev_desc, offset + blk, cnt, addr);
+		res = blk_dread(dev_desc, offset + blk, cnt, ptr);
+	unmap_sysmem(ptr);
 
 	if (res != cnt) {
 		printf("%s error\n", cmdtp->name);
-- 
2.34.1

[PATCH v2 20/39] cmd: Fix memory-mapping in cmp command

[Simon Glass]

This unmaps a different address from what was mapped. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 cmd/mem.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/cmd/mem.c b/cmd/mem.c
index 274348068c2..4d6fde28531 100644
--- a/cmd/mem.c
+++ b/cmd/mem.c
@@ -245,7 +245,7 @@ static int do_mem_cmp(struct cmd_tbl *cmdtp, int flag, int argc,
 	int	size;
 	int     rcode = 0;
 	const char *type;
-	const void *buf1, *buf2, *base;
+	const void *buf1, *buf2, *base, *ptr1, *ptr2;
 	ulong word1, word2;  /* 64-bit if MEM_SUPPORT_64BIT_DATA */
 
 	if (argc != 4)
@@ -270,22 +270,22 @@ static int do_mem_cmp(struct cmd_tbl *cmdtp, int flag, int argc,
 	bytes = size * count;
 	base = buf1 = map_sysmem(addr1, bytes);
 	buf2 = map_sysmem(addr2, bytes);
-	for (ngood = 0; ngood < count; ++ngood) {
+	for (ngood = 0, ptr1 = buf1, ptr2 = buf2; ngood < count; ++ngood) {
 		if (size == 4) {
-			word1 = *(u32 *)buf1;
-			word2 = *(u32 *)buf2;
+			word1 = *(u32 *)ptr1;
+			word2 = *(u32 *)ptr2;
 		} else if (MEM_SUPPORT_64BIT_DATA && size == 8) {
-			word1 = *(ulong *)buf1;
-			word2 = *(ulong *)buf2;
+			word1 = *(ulong *)ptr1;
+			word2 = *(ulong *)ptr2;
 		} else if (size == 2) {
-			word1 = *(u16 *)buf1;
-			word2 = *(u16 *)buf2;
+			word1 = *(u16 *)ptr1;
+			word2 = *(u16 *)ptr2;
 		} else {
-			word1 = *(u8 *)buf1;
-			word2 = *(u8 *)buf2;
+			word1 = *(u8 *)ptr1;
+			word2 = *(u8 *)ptr2;
 		}
 		if (word1 != word2) {
-			ulong offset = buf1 - base;
+			ulong offset = ptr1 - base;
 			printf("%s at 0x%08lx (%#0*lx) != %s at 0x%08lx (%#0*lx)\n",
 				type, (ulong)(addr1 + offset), size, word1,
 				type, (ulong)(addr2 + offset), size, word2);
@@ -293,8 +293,8 @@ static int do_mem_cmp(struct cmd_tbl *cmdtp, int flag, int argc,
 			break;
 		}
 
-		buf1 += size;
-		buf2 += size;
+		ptr1 += size;
+		ptr2 += size;
 
 		/* reset watchdog from time to time */
 		if ((ngood % (64 << 10)) == 0)
-- 
2.34.1

[PATCH v2 21/39] test: mbr: Unmap the buffers after use

[Simon Glass]

This tests maps some local variables into sandbox's address space. Make
sure to unmap them afterwards.

Note that the normal approach with sandbox is to use a fixed memory
address in the RAM, to avoid needing to create a map for transient local
variables.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/cmd/mbr.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
index 1a35ba3351c..8a4d61b2928 100644
--- a/test/cmd/mbr.c
+++ b/test/cmd/mbr.c
@@ -461,6 +461,9 @@ static int mbr_test_run(struct unit_test_state *uts)
 			"5P EBR+0x%04X: expected %#02X, actual: %#02X\n",
 			ebr_cmp_start + i, ebr_parts_ref_p5[i], rbuf[ebr_cmp_start + i]);
 	}
+	unmap_sysmem(mbr_wbuf);
+	unmap_sysmem(ebr_wbuf);
+	unmap_sysmem(rbuf);
 
 	return 0;
 }
-- 
2.34.1

[PATCH v2 22/39] test: mbr: Use a constant for the block size

[Simon Glass]

It isn't that important to factor out constants in tests, but in this
case we have 0x200 and 512 used. The commands don't use the constant
as they use a block count ('1'). It doesn't create more code to use a
constant, so create one.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/cmd/mbr.c | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
index 8a4d61b2928..bab92c3ba00 100644
--- a/test/cmd/mbr.c
+++ b/test/cmd/mbr.c
@@ -18,6 +18,9 @@
 #include <test/ut.h>
 
 DECLARE_GLOBAL_DATA_PTR;
+
+#define BLKSZ	0x200	/* block size */
+
 /*
  * Requirements for running test manually:
  * mmc6.img - File size needs to be at least 12 MiB
@@ -228,7 +231,7 @@ static unsigned build_mbr_parts(char *buf, size_t buf_size, unsigned num_parts)
 static int mbr_test_run(struct unit_test_state *uts)
 {
 	struct blk_desc *mmc_dev_desc;
-	unsigned char mbr_wbuf[512], ebr_wbuf[512], rbuf[512];
+	unsigned char mbr_wbuf[BLKSZ], ebr_wbuf[BLKSZ], rbuf[BLKSZ];
 	char mbr_parts_buf[256];
 	ulong mbr_wa, ebr_wa, ra, ebr_blk, mbr_parts_max;
 	struct udevice *dev;
@@ -257,7 +260,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	mbr_wa = map_to_sysmem(mbr_wbuf);
 	ebr_wa = map_to_sysmem(ebr_wbuf);
 	ra = map_to_sysmem(rbuf);
-	ebr_blk = (ulong)0xb00000 / 0x200;
+	ebr_blk = (ulong)0xb00000 / BLKSZ;
 
 	/* Make sure mmc6 exists */
 	ut_asserteq(6, blk_get_device_by_str("mmc", "6", &mmc_dev_desc));
@@ -269,7 +272,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 
 	/* Make sure mmc6 is 12+ MiB in size */
 	ut_assertok(run_commandf("mmc read %lx %lx 1", ra,
-				 (ulong)0xbffe00 / 0x200));
+				 (ulong)0xbffe00 / BLKSZ));
 
 	/* Test one MBR partition */
 	init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
@@ -277,11 +280,11 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
-	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
 	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
 	ut_assertok(run_commandf("mbr write mmc 6"));
@@ -290,7 +293,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(ut_check_console_end(uts));
 	/*
 	000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
@@ -313,11 +316,11 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
-	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
 	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
 	ut_assertok(run_commandf("mbr write mmc 6"));
@@ -326,7 +329,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(ut_check_console_end(uts));
 	/*
 	000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
@@ -349,11 +352,11 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
-	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
 	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
 	ut_assertok(run_commandf("mbr write mmc 6"));
@@ -362,7 +365,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(ut_check_console_end(uts));
 	/*
 	000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
@@ -385,11 +388,11 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
-	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
 	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
 	ut_assertok(run_commandf("mbr write mmc 6"));
@@ -398,7 +401,7 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assert_nextline("MBR: verify success!");
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(ut_check_console_end(uts));
 	/*
 	000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
@@ -421,11 +424,11 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
-	ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
 	ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
 	memset(rbuf, 0, sizeof(rbuf));
 	ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
-	ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
+	ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
 	ut_assertok(console_record_reset_enable());
 	ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
 	ut_assertf(0 == run_commandf("mbr write mmc 6"), "Invalid partitions string: %s\n", mbr_parts_buf);
-- 
2.34.1

[PATCH v2 23/39] test: mbr: Use RAM for the buffers

[Simon Glass]

The normal approach with sandbox is to use a fixed memory address in the
RAM, to avoid needing to create a map for transient local variables.

Update this test to use this approach.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/cmd/mbr.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
index bab92c3ba00..81ca6f15b2a 100644
--- a/test/cmd/mbr.c
+++ b/test/cmd/mbr.c
@@ -231,9 +231,11 @@ static unsigned build_mbr_parts(char *buf, size_t buf_size, unsigned num_parts)
 static int mbr_test_run(struct unit_test_state *uts)
 {
 	struct blk_desc *mmc_dev_desc;
-	unsigned char mbr_wbuf[BLKSZ], ebr_wbuf[BLKSZ], rbuf[BLKSZ];
+	unsigned char *mbr_wbuf, *ebr_wbuf, *rbuf;
 	char mbr_parts_buf[256];
-	ulong mbr_wa, ebr_wa, ra, ebr_blk, mbr_parts_max;
+	ulong addr = 0x1000;  /* start address for buffers */
+	ulong mbr_wa = addr, ebr_wa = addr + BLKSZ, ra = addr + BLKSZ * 2;
+	ulong ebr_blk, mbr_parts_max;
 	struct udevice *dev;
 	ofnode root, node;
 
@@ -257,9 +259,9 @@ static int mbr_test_run(struct unit_test_state *uts)
 	ut_assertf(sizeof(mbr_parts_buf) >= mbr_parts_max, "Buffer avail: %ld; buffer req: %ld\n",
 		sizeof(mbr_parts_buf), mbr_parts_max);
 
-	mbr_wa = map_to_sysmem(mbr_wbuf);
-	ebr_wa = map_to_sysmem(ebr_wbuf);
-	ra = map_to_sysmem(rbuf);
+	mbr_wbuf = map_sysmem(mbr_wa, BLKSZ);
+	ebr_wbuf = map_sysmem(ebr_wa, BLKSZ);
+	rbuf = map_sysmem(ra, BLKSZ);
 	ebr_blk = (ulong)0xb00000 / BLKSZ;
 
 	/* Make sure mmc6 exists */
-- 
2.34.1

[PATCH v2 24/39] test: mbr: Drop a duplicate test

[Simon Glass]

The test currently runs twice as it is declared twice. Unwind this.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 test/cmd/mbr.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
index 81ca6f15b2a..f1fc44051df 100644
--- a/test/cmd/mbr.c
+++ b/test/cmd/mbr.c
@@ -483,10 +483,3 @@ int do_ut_mbr(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
 
 	return cmd_ut_category("mbr", "mbr_test_", tests, n_ents, argc, argv);
 }
-
-static int dm_test_cmd_mbr(struct unit_test_state *uts)
-{
-	return mbr_test_run(uts);
-}
-
-DM_TEST(dm_test_cmd_mbr, UT_TESTF_SCAN_FDT | UT_TESTF_CONSOLE_REC);
-- 
2.34.1

[PATCH v2 25/39] efi: Use puts() in cout so that console recording works

[Simon Glass]

At present EFI output to the console uses fputs() which bypasses the
console-recording feature. This makes it impossible for tests to check
the output of an EFI app.

There doesn't seem to be any need to do this bypass, so adjust it to
simply use the puts() function.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
---

Changes in v2:
- Fix 'use' typo

 lib/efi_loader/efi_console.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/efi_loader/efi_console.c b/lib/efi_loader/efi_console.c
index c944c10b216..cea50c748aa 100644
--- a/lib/efi_loader/efi_console.c
+++ b/lib/efi_loader/efi_console.c
@@ -181,7 +181,7 @@ static efi_status_t EFIAPI efi_cout_output_string(
 	}
 	pos = buf;
 	utf16_utf8_strcpy(&pos, string);
-	fputs(stdout, buf);
+	puts(buf);
 	free(buf);
 
 	/*
-- 
2.34.1

[PATCH v2 26/39] efi_loader: Put back copyright message

[Simon Glass]

This was lost in a later commit, so add it back.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: dd860b90ff4 ("efi_loader: print boot device and file path in")
---

(no changes since v1)

 lib/efi_loader/helloworld.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/efi_loader/helloworld.c b/lib/efi_loader/helloworld.c
index bd72822c0b7..586177de0c8 100644
--- a/lib/efi_loader/helloworld.c
+++ b/lib/efi_loader/helloworld.c
@@ -2,6 +2,9 @@
 /*
  * Hello world EFI application
  *
+ * Copyright (c) 2016 Google, Inc
+ * Written by Simon Glass <sjg@chromium.org>
+ *
  * Copyright 2020, Heinrich Schuchardt <xypron.glpk@gmx.de>
  *
  * This test program is used to test the invocation of an EFI application.
-- 
2.34.1

[PATCH v2 27/39] efi_loader: Rename and move CMD_BOOTEFI_HELLO_COMPILE

[Simon Glass]

This is not actually a command so the name is confusing. Use
BOOTEFI_HELLO_COMPILE instead. Put it in the efi_loader directory
with the other such config options.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 arch/arm/lib/Makefile            |  2 +-
 arch/riscv/lib/Makefile          |  2 +-
 arch/x86/lib/Makefile            |  2 +-
 cmd/Kconfig                      | 14 +-------------
 configs/octeontx2_95xx_defconfig |  2 +-
 configs/octeontx2_96xx_defconfig |  2 +-
 configs/octeontx_81xx_defconfig  |  2 +-
 configs/octeontx_83xx_defconfig  |  2 +-
 doc/develop/uefi/uefi.rst        |  2 +-
 lib/efi_loader/Kconfig           | 12 ++++++++++++
 lib/efi_loader/Makefile          |  2 +-
 test/py/tests/test_efi_fit.py    |  2 +-
 test/py/tests/test_efi_loader.py |  2 +-
 13 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile
index 67275fba616..406797f998c 100644
--- a/arch/arm/lib/Makefile
+++ b/arch/arm/lib/Makefile
@@ -130,7 +130,7 @@ CFLAGS_REMOVE_$(EFI_CRT0) := $(CFLAGS_NON_EFI)
 CFLAGS_$(EFI_RELOC) := $(CFLAGS_EFI)
 CFLAGS_REMOVE_$(EFI_RELOC) := $(CFLAGS_NON_EFI)
 
-extra-$(CONFIG_CMD_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)
+extra-$(CONFIG_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)
 # TODO: As of v2019.01 the relocation code for the EFI application cannot
 # be built on ARMv7-M.
 ifndef CONFIG_CPU_V7M
diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile
index 65dc49f6fa5..3bd90c7a354 100644
--- a/arch/riscv/lib/Makefile
+++ b/arch/riscv/lib/Makefile
@@ -36,7 +36,7 @@ CFLAGS_REMOVE_$(EFI_CRT0) := $(CFLAGS_NON_EFI)
 CFLAGS_$(EFI_RELOC) := $(CFLAGS_EFI)
 CFLAGS_REMOVE_$(EFI_RELOC) := $(CFLAGS_NON_EFI)
 
-extra-$(CONFIG_CMD_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)
+extra-$(CONFIG_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)
 extra-$(CONFIG_CMD_BOOTEFI_SELFTEST) += $(EFI_CRT0) $(EFI_RELOC)
 extra-$(CONFIG_EFI) += $(EFI_CRT0) $(EFI_RELOC)
 
diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
index 8fc35e1b51e..70dccd7f983 100644
--- a/arch/x86/lib/Makefile
+++ b/arch/x86/lib/Makefile
@@ -97,7 +97,7 @@ endif
 else
 
 ifndef CONFIG_SPL_BUILD
-ifneq ($(CONFIG_CMD_BOOTEFI_SELFTEST)$(CONFIG_CMD_BOOTEFI_HELLO_COMPILE),)
+ifneq ($(CONFIG_CMD_BOOTEFI_SELFTEST)$(CONFIG_BOOTEFI_HELLO_COMPILE),)
 extra-y += $(EFI_CRT0) $(EFI_RELOC)
 endif
 endif
diff --git a/cmd/Kconfig b/cmd/Kconfig
index 978f44eda42..fcf6bb73f88 100644
--- a/cmd/Kconfig
+++ b/cmd/Kconfig
@@ -431,21 +431,9 @@ config CMD_BOOTEFI_BOOTMGR
 	  This subcommand will allow you to select the UEFI binary to be booted
 	  via UEFI variables Boot####, BootOrder, and BootNext.
 
-config CMD_BOOTEFI_HELLO_COMPILE
-	bool "Compile a standard EFI hello world binary for testing"
-	default y
-	help
-	  This compiles a standard EFI hello world application with U-Boot so
-	  that it can be used with the test/py testing framework. This is useful
-	  for testing that EFI is working at a basic level, and for bringing
-	  up EFI support on a new architecture.
-
-	  No additional space will be required in the resulting U-Boot binary
-	  when this option is enabled.
-
 config CMD_BOOTEFI_HELLO
 	bool "Allow booting a standard EFI hello world for testing"
-	depends on CMD_BOOTEFI_BINARY && CMD_BOOTEFI_HELLO_COMPILE
+	depends on CMD_BOOTEFI_BINARY && BOOTEFI_HELLO_COMPILE
 	default y if CMD_BOOTEFI_SELFTEST
 	help
 	  This adds a standard EFI hello world application to U-Boot so that
diff --git a/configs/octeontx2_95xx_defconfig b/configs/octeontx2_95xx_defconfig
index c5dc4f4dfa6..23c313375ac 100644
--- a/configs/octeontx2_95xx_defconfig
+++ b/configs/octeontx2_95xx_defconfig
@@ -38,7 +38,7 @@ CONFIG_SYS_PBSIZE=1050
 CONFIG_BOARD_EARLY_INIT_R=y
 CONFIG_HUSH_PARSER=y
 CONFIG_SYS_PROMPT="Marvell> "
-# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
+# CONFIG_BOOTEFI_HELLO_COMPILE is not set
 CONFIG_CMD_MD5SUM=y
 CONFIG_MD5SUM_VERIFY=y
 CONFIG_CMD_MX_CYCLIC=y
diff --git a/configs/octeontx2_96xx_defconfig b/configs/octeontx2_96xx_defconfig
index ad61b80300f..197e72acd1f 100644
--- a/configs/octeontx2_96xx_defconfig
+++ b/configs/octeontx2_96xx_defconfig
@@ -38,7 +38,7 @@ CONFIG_SYS_PBSIZE=1050
 CONFIG_BOARD_EARLY_INIT_R=y
 CONFIG_HUSH_PARSER=y
 CONFIG_SYS_PROMPT="Marvell> "
-# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
+# CONFIG_BOOTEFI_HELLO_COMPILE is not set
 CONFIG_CMD_MD5SUM=y
 CONFIG_MD5SUM_VERIFY=y
 CONFIG_CMD_MX_CYCLIC=y
diff --git a/configs/octeontx_81xx_defconfig b/configs/octeontx_81xx_defconfig
index 1d39bce6abd..b501d653c27 100644
--- a/configs/octeontx_81xx_defconfig
+++ b/configs/octeontx_81xx_defconfig
@@ -39,7 +39,7 @@ CONFIG_SYS_PBSIZE=1050
 CONFIG_BOARD_EARLY_INIT_R=y
 CONFIG_HUSH_PARSER=y
 CONFIG_SYS_PROMPT="Marvell> "
-# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
+# CONFIG_BOOTEFI_HELLO_COMPILE is not set
 CONFIG_CMD_MD5SUM=y
 CONFIG_MD5SUM_VERIFY=y
 CONFIG_CMD_MX_CYCLIC=y
diff --git a/configs/octeontx_83xx_defconfig b/configs/octeontx_83xx_defconfig
index ba9fc5f9553..4a537f5cbbe 100644
--- a/configs/octeontx_83xx_defconfig
+++ b/configs/octeontx_83xx_defconfig
@@ -37,7 +37,7 @@ CONFIG_SYS_PBSIZE=1050
 CONFIG_BOARD_EARLY_INIT_R=y
 CONFIG_HUSH_PARSER=y
 CONFIG_SYS_PROMPT="Marvell> "
-# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
+# CONFIG_BOOTEFI_HELLO_COMPILE is not set
 CONFIG_CMD_MD5SUM=y
 CONFIG_MD5SUM_VERIFY=y
 CONFIG_CMD_MX_CYCLIC=y
diff --git a/doc/develop/uefi/uefi.rst b/doc/develop/uefi/uefi.rst
index d450b12bf80..9aeee23a32e 100644
--- a/doc/develop/uefi/uefi.rst
+++ b/doc/develop/uefi/uefi.rst
@@ -693,7 +693,7 @@ Executing the built in hello world application
 
 A hello world UEFI application can be built with::
 
-    CONFIG_CMD_BOOTEFI_HELLO_COMPILE=y
+    CONFIG_BOOTEFI_HELLO_COMPILE=y
 
 It can be embedded into the U-Boot binary with::
 
diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
index 1179c31bb13..ab2c1c44364 100644
--- a/lib/efi_loader/Kconfig
+++ b/lib/efi_loader/Kconfig
@@ -516,4 +516,16 @@ config EFI_HTTP_BOOT
 	  Enabling this option adds EFI HTTP Boot support. It allows to
 	  directly boot from network.
 
+config BOOTEFI_HELLO_COMPILE
+	bool "Compile a standard EFI hello world binary for testing"
+	default y
+	help
+	  This compiles a standard EFI hello world application with U-Boot so
+	  that it can be used with the test/py testing framework. This is useful
+	  for testing that EFI is working at a basic level, and for bringing
+	  up EFI support on a new architecture.
+
+	  No additional space will be required in the resulting U-Boot binary
+	  when this option is enabled.
+
 endif
diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
index 2af6f2066b5..27dbd9e760d 100644
--- a/lib/efi_loader/Makefile
+++ b/lib/efi_loader/Makefile
@@ -27,7 +27,7 @@ always += boothart.efi
 targets += boothart.o
 endif
 
-ifneq ($(CONFIG_CMD_BOOTEFI_HELLO_COMPILE),)
+ifneq ($(CONFIG_BOOTEFI_HELLO_COMPILE),)
 always += helloworld.efi
 targets += helloworld.o
 endif
diff --git a/test/py/tests/test_efi_fit.py b/test/py/tests/test_efi_fit.py
index 0ad483500f8..550058a30fd 100644
--- a/test/py/tests/test_efi_fit.py
+++ b/test/py/tests/test_efi_fit.py
@@ -119,7 +119,7 @@ FDT_DATA = '''
 '''
 
 @pytest.mark.buildconfigspec('bootm_efi')
-@pytest.mark.buildconfigspec('cmd_bootefi_hello_compile')
+@pytest.mark.buildconfigspec('BOOTEFI_HELLO_COMPILE')
 @pytest.mark.buildconfigspec('fit')
 @pytest.mark.notbuildconfigspec('generate_acpi_table')
 @pytest.mark.requiredtool('dtc')
diff --git a/test/py/tests/test_efi_loader.py b/test/py/tests/test_efi_loader.py
index 85473a9049b..ef1f1f1afb3 100644
--- a/test/py/tests/test_efi_loader.py
+++ b/test/py/tests/test_efi_loader.py
@@ -148,7 +148,7 @@ def fetch_tftp_file(u_boot_console, env_conf):
     return addr
 
 @pytest.mark.buildconfigspec('of_control')
-@pytest.mark.buildconfigspec('cmd_bootefi_hello_compile')
+@pytest.mark.buildconfigspec('BOOTEFI_HELLO_COMPILE')
 def test_efi_helloworld_net(u_boot_console):
     """Run the helloworld.efi binary via TFTP.
 
-- 
2.34.1

[PATCH v2 28/39] efi_loader: Shorten the app rules

[Simon Glass]

We have quite a few apps now, so create a way to specify them as a list
rather than repeating the same rules again and again.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 lib/efi_loader/Makefile | 31 +++++++++----------------------
 1 file changed, 9 insertions(+), 22 deletions(-)

diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
index 27dbd9e760d..660368b9d8f 100644
--- a/lib/efi_loader/Makefile
+++ b/lib/efi_loader/Makefile
@@ -22,29 +22,13 @@ CFLAGS_REMOVE_dtbdump.o := $(CFLAGS_NON_EFI)
 CFLAGS_initrddump.o := $(CFLAGS_EFI) -Os -ffreestanding
 CFLAGS_REMOVE_initrddump.o := $(CFLAGS_NON_EFI)
 
-ifdef CONFIG_RISCV
-always += boothart.efi
-targets += boothart.o
-endif
-
-ifneq ($(CONFIG_BOOTEFI_HELLO_COMPILE),)
-always += helloworld.efi
-targets += helloworld.o
-endif
-
-ifneq ($(CONFIG_GENERATE_SMBIOS_TABLE),)
-always += smbiosdump.efi
-targets += smbiosdump.o
-endif
-
+# These are the apps that are built
+apps-$(CONFIG_RISCV) += boothart
+apps-$(CONFIG_BOOTEFI_HELLO_COMPILE) += helloworld
+apps-$(CONFIG_GENERATE_SMBIOS_TABLE) += smbiosdump
+apps-$(CONFIG_EFI_LOAD_FILE2_INITRD) += initrddump
 ifeq ($(CONFIG_GENERATE_ACPI_TABLE),)
-always += dtbdump.efi
-targets += dtbdump.o
-endif
-
-ifdef CONFIG_EFI_LOAD_FILE2_INITRD
-always += initrddump.efi
-targets += initrddump.o
+apps-y += dtbdump
 endif
 
 obj-$(CONFIG_CMD_BOOTEFI_HELLO) += helloworld_efi.o
@@ -95,3 +79,6 @@ obj-$(CONFIG_EFI_ECPT) += efi_conformance.o
 
 EFI_VAR_SEED_FILE := $(subst $\",,$(CONFIG_EFI_VAR_SEED_FILE))
 $(obj)/efi_var_seed.o: $(srctree)/$(EFI_VAR_SEED_FILE)
+
+always += $(foreach f,$(apps-y),$(f).efi)
+targets += $(foreach f,$(apps-y),$(f).o)
-- 
2.34.1

[PATCH v2 29/39] efi_loader: Shorten the app rules further

[Simon Glass]

Add a way to factor out the CFLAGS changes for each app, since they are
all the same.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 lib/efi_loader/Makefile | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
index 660368b9d8f..00d18966f9e 100644
--- a/lib/efi_loader/Makefile
+++ b/lib/efi_loader/Makefile
@@ -11,16 +11,6 @@ asflags-y += -I.
 CFLAGS_efi_boottime.o += \
   -DFW_VERSION="0x$(VERSION)" \
   -DFW_PATCHLEVEL="0x$(PATCHLEVEL)"
-CFLAGS_boothart.o := $(CFLAGS_EFI) -Os -ffreestanding
-CFLAGS_REMOVE_boothart.o := $(CFLAGS_NON_EFI)
-CFLAGS_helloworld.o := $(CFLAGS_EFI) -Os -ffreestanding
-CFLAGS_REMOVE_helloworld.o := $(CFLAGS_NON_EFI)
-CFLAGS_smbiosdump.o := $(CFLAGS_EFI) -Os -ffreestanding
-CFLAGS_REMOVE_smbiosdump.o := $(CFLAGS_NON_EFI)
-CFLAGS_dtbdump.o := $(CFLAGS_EFI) -Os -ffreestanding
-CFLAGS_REMOVE_dtbdump.o := $(CFLAGS_NON_EFI)
-CFLAGS_initrddump.o := $(CFLAGS_EFI) -Os -ffreestanding
-CFLAGS_REMOVE_initrddump.o := $(CFLAGS_NON_EFI)
 
 # These are the apps that are built
 apps-$(CONFIG_RISCV) += boothart
@@ -80,5 +70,10 @@ obj-$(CONFIG_EFI_ECPT) += efi_conformance.o
 EFI_VAR_SEED_FILE := $(subst $\",,$(CONFIG_EFI_VAR_SEED_FILE))
 $(obj)/efi_var_seed.o: $(srctree)/$(EFI_VAR_SEED_FILE)
 
+# Set the C flags to add and remove for each app
+$(foreach f,$(apps-y),\
+	$(eval CFLAGS_$(f).o := $(CFLAGS_EFI) -Os -ffreestanding)\
+	$(eval CFLAGS_REMOVE_$(f).o := $(CFLAGS_NON_EFI)))
+
 always += $(foreach f,$(apps-y),$(f).efi)
 targets += $(foreach f,$(apps-y),$(f).o)
-- 
2.34.1

[PATCH v2 30/39] efi: Show the vendor in helloworld

[Simon Glass]

Show the vendor name so it is clear that this app has been built by
U-Boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 lib/efi_loader/helloworld.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/efi_loader/helloworld.c b/lib/efi_loader/helloworld.c
index 586177de0c8..f120249126d 100644
--- a/lib/efi_loader/helloworld.c
+++ b/lib/efi_loader/helloworld.c
@@ -237,6 +237,9 @@ efi_status_t EFIAPI efi_main(efi_handle_t handle,
 			(con_out, u"Missing device path for device handle\r\n");
 		goto out;
 	}
+	con_out->output_string(con_out, u"Vendor: ");
+	con_out->output_string(con_out, systab->fw_vendor);
+	con_out->output_string(con_out, u"\n");
 	con_out->output_string(con_out, u"Boot device: ");
 	ret = print_device_path(device_path, device_path_to_text);
 	if (ret != EFI_SUCCESS)
-- 
2.34.1

[PATCH v2 31/39] Revert "bootdev: avoid infinite probe loop"

[Simon Glass]

This turns out to be insufficient to fix the problem, since when
bootdev_next_prio() exits, the caller has no idea that this really
is the end. Nor is it, since there may be other devices which should
be checked.

The caller iterates which calls iter_incr() which calls
bootdev_next_prio() again, which finds the same device and the loop
continues.

We never did create a test for this[1], which makes it hard to be
sure which problem was fixed.

The original code had the virtual of staying in the loop looking for
a bootdev, so let's go back to that and try to fix this another way.

This reverts commit 9d92c418acfb7576e12e2bd53fed294bb9543724.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 boot/bootdev-uclass.c | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)

diff --git a/boot/bootdev-uclass.c b/boot/bootdev-uclass.c
index 7c7bba088c9..15a8a3555c6 100644
--- a/boot/bootdev-uclass.c
+++ b/boot/bootdev-uclass.c
@@ -632,7 +632,7 @@ int bootdev_next_label(struct bootflow_iter *iter, struct udevice **devp,
 
 int bootdev_next_prio(struct bootflow_iter *iter, struct udevice **devp)
 {
-	struct udevice *dev = *devp, *last_dev = NULL;
+	struct udevice *dev = *devp;
 	bool found;
 	int ret;
 
@@ -682,19 +682,9 @@ int bootdev_next_prio(struct bootflow_iter *iter, struct udevice **devp)
 			}
 		} else {
 			ret = device_probe(dev);
-			if (!ret)
-				last_dev = dev;
 			if (ret) {
-				log_warning("Device '%s' failed to probe\n",
+				log_debug("Device '%s' failed to probe\n",
 					  dev->name);
-				if (last_dev == dev) {
-					/*
-					 * We have already tried this device
-					 * and it failed to probe. Give up.
-					 */
-					return log_msg_ret("probe", ret);
-				}
-				last_dev = dev;
 				dev = NULL;
 			}
 		}
-- 
2.34.1

[PATCH v2 32/39] bootstd: Make bootdev_next_prio() continue after failure

[Simon Glass]

When a device fails to probe, the next device should be tried, until
either we find a suitable device or run out of devices. A device
should never be tried twice.

When we run out of devices of a particular priority, the hunter should
be used to generate devices of the next priority. Only if all attempts
fail should this function return an error.

Update the function to use the latent 'found' boolean to determine
whether another loop iteration is warranted, rather than setting 'dev'
to NULL, which creates confusion, suggesting that no devices have been
scanned and the whole process is starting from the beginning.

Note that the upcoming bootflow_efi() test is used to test this
behaviour.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/17
---

(no changes since v1)

 boot/bootdev-uclass.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/boot/bootdev-uclass.c b/boot/bootdev-uclass.c
index 15a8a3555c6..807f8dfb064 100644
--- a/boot/bootdev-uclass.c
+++ b/boot/bootdev-uclass.c
@@ -640,6 +640,7 @@ int bootdev_next_prio(struct bootflow_iter *iter, struct udevice **devp)
 	*devp = NULL;
 	log_debug("next prio %d: dev=%p/%s\n", iter->cur_prio, dev,
 		  dev ? dev->name : "none");
+	found = false;
 	do {
 		/*
 		 * Don't probe devices here since they may not be of the
@@ -682,13 +683,13 @@ int bootdev_next_prio(struct bootflow_iter *iter, struct udevice **devp)
 			}
 		} else {
 			ret = device_probe(dev);
-			if (ret) {
+			if (ret)
 				log_debug("Device '%s' failed to probe\n",
 					  dev->name);
-				dev = NULL;
-			}
+			else
+				found = true;
 		}
-	} while (!dev);
+	} while (!found);
 
 	*devp = dev;
 
-- 
2.34.1

[PATCH v2 33/39] efi: Use the same filename for all sandbox builds

[Simon Glass]

Sandbox is not a real architecture, but within U-Boot it is real enough.
We should not need to pretend it is x86 or ARM anywhere in the code.

Also we want to be able to locate the sandbox app using a single
filename, 'bootsbox.efi', to avoid needing tests to produce different
files on each host architecture.

Drop the confusing use of host architecture and just let sandbox be
sandbox.

Fixes: 3a0654ecd0d ("efi_loader: correctly identify binary name")

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 include/efi_default_filename.h | 25 ++-----------------------
 1 file changed, 2 insertions(+), 23 deletions(-)

diff --git a/include/efi_default_filename.h b/include/efi_default_filename.h
index 77932984b55..6806c0daa88 100644
--- a/include/efi_default_filename.h
+++ b/include/efi_default_filename.h
@@ -5,7 +5,6 @@
  * file name is defined in this include.
  *
  * Copyright (c) 2022, Heinrich Schuchardt <xypron.glpk@gmx.de>
- * Copyright (c) 2022, Linaro Limited
  */
 
 #ifndef _EFI_DEFAULT_FILENAME_H
@@ -16,26 +15,8 @@
 #undef BOOTEFI_NAME
 
 #ifdef CONFIG_SANDBOX
-
-#if HOST_ARCH == HOST_ARCH_X86_64
-#define BOOTEFI_NAME "BOOTX64.EFI"
-#elif HOST_ARCH == HOST_ARCH_X86
-#define BOOTEFI_NAME "BOOTIA32.EFI"
-#elif HOST_ARCH == HOST_ARCH_AARCH64
-#define BOOTEFI_NAME "BOOTAA64.EFI"
-#elif HOST_ARCH == HOST_ARCH_ARM
-#define BOOTEFI_NAME "BOOTARM.EFI"
-#elif HOST_ARCH == HOST_ARCH_RISCV32
-#define BOOTEFI_NAME "BOOTRISCV32.EFI"
-#elif HOST_ARCH == HOST_ARCH_RISCV64
-#define BOOTEFI_NAME "BOOTRISCV64.EFI"
-#else
-#error Unsupported UEFI architecture
-#endif
-
-#else
-
-#if defined(CONFIG_ARM64)
+#define BOOTEFI_NAME "BOOTSBOX.EFI"
+#elif defined(CONFIG_ARM64)
 #define BOOTEFI_NAME "BOOTAA64.EFI"
 #elif defined(CONFIG_ARM)
 #define BOOTEFI_NAME "BOOTARM.EFI"
@@ -52,5 +33,3 @@
 #endif
 
 #endif
-
-#endif
-- 
2.34.1

[PATCH v2 34/39] bootstd: Add debugging for efi bootmeth

[Simon Glass]

Add a little debugging so we can see what is happening.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 boot/bootmeth_efi.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/boot/bootmeth_efi.c b/boot/bootmeth_efi.c
index 39232eb2e25..99887a89f02 100644
--- a/boot/bootmeth_efi.c
+++ b/boot/bootmeth_efi.c
@@ -163,8 +163,10 @@ static int distro_efi_try_bootflow_files(struct udevice *dev,
 	int ret, seq;
 
 	/* We require a partition table */
-	if (!bflow->part)
+	if (!bflow->part) {
+		log_debug("no partitions\n");
 		return -ENOENT;
+	}
 
 	strcpy(fname, EFI_DIRNAME);
 	strcat(fname, BOOTEFI_NAME);
@@ -172,8 +174,10 @@ static int distro_efi_try_bootflow_files(struct udevice *dev,
 	if (bflow->blk)
 		 desc = dev_get_uclass_plat(bflow->blk);
 	ret = bootmeth_try_file(bflow, desc, NULL, fname);
-	if (ret)
+	if (ret) {
+		log_debug("no file\n");
 		return log_msg_ret("try", ret);
+	}
 
 	/* Since we can access the file, let's call it ready */
 	bflow->state = BOOTFLOWST_READY;
@@ -308,6 +312,8 @@ static int distro_efi_read_bootflow(struct udevice *dev, struct bootflow *bflow)
 {
 	int ret;
 
+	log_debug("dev='%s', part=%d\n", bflow->dev->name, bflow->part);
+
 	/*
 	 * bootmeth_efi doesn't allocate any buffer neither for blk nor net device
 	 * set flag to avoid freeing static buffer.
@@ -333,6 +339,7 @@ static int distro_efi_boot(struct udevice *dev, struct bootflow *bflow)
 	ulong kernel, fdt;
 	int ret;
 
+	log_debug("boot\n");
 	kernel = env_get_hex("kernel_addr_r", 0);
 	if (!bootmeth_uses_network(bflow)) {
 		ret = efiload_read_file(bflow, kernel);
-- 
2.34.1

[PATCH v2 35/39] efi: Disable ANSI output for tests

[Simon Glass]

We don't want ANSI characters written in tests since it is a pain to
check the output with ut_assert_nextline() et al.

Provide a way to tests to request that ANSI characters not be sent.

Add a proper function comment while we are here, to encourage others.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 include/efi_loader.h         | 21 ++++++++++++++++++++-
 lib/efi_loader/efi_console.c | 26 +++++++++++++++++---------
 2 files changed, 37 insertions(+), 10 deletions(-)

diff --git a/include/efi_loader.h b/include/efi_loader.h
index f84852e384f..82b90ee0f1d 100644
--- a/include/efi_loader.h
+++ b/include/efi_loader.h
@@ -531,8 +531,27 @@ efi_status_t efi_bootmgr_delete_boot_option(u16 boot_index);
 efi_status_t efi_bootmgr_run(void *fdt);
 /* search the boot option index in BootOrder */
 bool efi_search_bootorder(u16 *bootorder, efi_uintn_t num, u32 target, u32 *index);
-/* Set up console modes */
+
+/**
+ * efi_setup_console_size() - update the mode table.
+ *
+ * By default the only mode available is 80x25. If the console has at least 50
+ * lines, enable mode 80x50. If we can query the console size and it is neither
+ * 80x25 nor 80x50, set it as an additional mode.
+ */
 void efi_setup_console_size(void);
+
+/**
+ * efi_console_set_ansi() - Set whether ANSI characters should be emitted
+ *
+ * These characters mess up tests which use ut_assert_nextline(). Call this
+ * function to tell efi_loader not to emit these characters when starting up the
+ * terminal
+ *
+ * @allow_ansi: Allow emitting ANSI characters
+ */
+void efi_console_set_ansi(bool allow_ansi);
+
 /* Set up load options from environment variable */
 efi_status_t efi_env_set_load_options(efi_handle_t handle, const char *env_var,
 				      u16 **load_options);
diff --git a/lib/efi_loader/efi_console.c b/lib/efi_loader/efi_console.c
index cea50c748aa..569fc9199bc 100644
--- a/lib/efi_loader/efi_console.c
+++ b/lib/efi_loader/efi_console.c
@@ -30,6 +30,17 @@ struct cout_mode {
 
 __maybe_unused static struct efi_object uart_obj;
 
+/*
+ * suppress emission of ANSI codes for use by unit tests. Leave it as 0 for the
+ * default behaviour
+ */
+static bool no_ansi;
+
+void efi_console_set_ansi(bool allow_ansi)
+{
+	no_ansi = !allow_ansi;
+}
+
 static struct cout_mode efi_cout_modes[] = {
 	/* EFI Mode 0 is 80x25 and always present */
 	{
@@ -348,13 +359,6 @@ static int __maybe_unused query_vidconsole(int *rows, int *cols)
 	return 0;
 }
 
-/**
- * efi_setup_console_size() - update the mode table.
- *
- * By default the only mode available is 80x25. If the console has at least 50
- * lines, enable mode 80x50. If we can query the console size and it is neither
- * 80x25 nor 80x50, set it as an additional mode.
- */
 void efi_setup_console_size(void)
 {
 	int rows = 25, cols = 80;
@@ -362,8 +366,12 @@ void efi_setup_console_size(void)
 
 	if (IS_ENABLED(CONFIG_VIDEO))
 		ret = query_vidconsole(&rows, &cols);
-	if (ret)
-		ret = query_console_serial(&rows, &cols);
+	if (ret) {
+		if (no_ansi)
+			ret = 0;
+		else
+			ret = query_console_serial(&rows, &cols);
+	}
 	if (ret)
 		return;
 
-- 
2.34.1

[PATCH v2 36/39] efi: Add a test app

[Simon Glass]

Add a simple app to use for testing. This is intended to do whatever it
needs to for testing purposes. For now it just prints a message and
exits boot services.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 lib/efi_loader/Kconfig   | 10 ++++++
 lib/efi_loader/Makefile  |  1 +
 lib/efi_loader/testapp.c | 68 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 79 insertions(+)
 create mode 100644 lib/efi_loader/testapp.c

diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
index ab2c1c44364..4de05c6f2d6 100644
--- a/lib/efi_loader/Kconfig
+++ b/lib/efi_loader/Kconfig
@@ -528,4 +528,14 @@ config BOOTEFI_HELLO_COMPILE
 	  No additional space will be required in the resulting U-Boot binary
 	  when this option is enabled.
 
+config BOOTEFI_TESTAPP_COMPILE
+	bool "Compile an EFI test app for testing"
+	default y
+	help
+	  This compiles an app designed for testing. It is packed into an image
+	  by the test.py testing frame in the setup_efi_image() function.
+
+	  No additional space will be required in the resulting U-Boot binary
+	  when this option is enabled.
+
 endif
diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
index 00d18966f9e..87131ab911d 100644
--- a/lib/efi_loader/Makefile
+++ b/lib/efi_loader/Makefile
@@ -20,6 +20,7 @@ apps-$(CONFIG_EFI_LOAD_FILE2_INITRD) += initrddump
 ifeq ($(CONFIG_GENERATE_ACPI_TABLE),)
 apps-y += dtbdump
 endif
+apps-$(CONFIG_BOOTEFI_TESTAPP_COMPILE) += testapp
 
 obj-$(CONFIG_CMD_BOOTEFI_HELLO) += helloworld_efi.o
 obj-$(CONFIG_EFI_BOOTMGR) += efi_bootmgr.o
diff --git a/lib/efi_loader/testapp.c b/lib/efi_loader/testapp.c
new file mode 100644
index 00000000000..feb444c92e9
--- /dev/null
+++ b/lib/efi_loader/testapp.c
@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Hello world EFI application
+ *
+ * Copyright 2024 Google LLC
+ * Written by Simon Glass <sjg@chromium.org>
+ *
+ * This test program is used to test the invocation of an EFI application.
+ * It writes a few messages to the console and then exits boot services
+ */
+
+#include <efi_api.h>
+
+static const efi_guid_t loaded_image_guid = EFI_LOADED_IMAGE_PROTOCOL_GUID;
+
+static struct efi_system_table *systable;
+static struct efi_boot_services *boottime;
+static struct efi_simple_text_output_protocol *con_out;
+
+/**
+ * efi_main() - entry point of the EFI application.
+ *
+ * @handle:	handle of the loaded image
+ * @systab:	system table
+ * Return:	status code
+ */
+efi_status_t EFIAPI efi_main(efi_handle_t handle,
+			     struct efi_system_table *systab)
+{
+	struct efi_loaded_image *loaded_image;
+	efi_status_t ret;
+	efi_uintn_t map_size;
+	efi_uintn_t map_key;
+	efi_uintn_t desc_size;
+	u32 desc_version;
+
+	systable = systab;
+	boottime = systable->boottime;
+	con_out = systable->con_out;
+
+	/* Get the loaded image protocol */
+	ret = boottime->open_protocol(handle, &loaded_image_guid,
+				      (void **)&loaded_image, NULL, NULL,
+				      EFI_OPEN_PROTOCOL_GET_PROTOCOL);
+	if (ret != EFI_SUCCESS) {
+		con_out->output_string
+			(con_out, u"Cannot open loaded image protocol\r\n");
+		goto out;
+	}
+
+	/* UEFI requires CR LF */
+	con_out->output_string(con_out, u"U-Boot test app for EFI_LOADER\r\n");
+
+out:
+	map_size = 0;
+	ret = boottime->get_memory_map(&map_size, NULL, &map_key, &desc_size,
+				       &desc_version);
+	con_out->output_string(con_out, u"Exiting boot sevices\n");
+
+	/* exit boot services so that this part of U-Boot can be tested */
+	boottime->exit_boot_services(handle, map_key);
+
+	/* now exit for real */
+	ret = boottime->exit(handle, ret, 0, NULL);
+
+	/* We should never arrive here */
+	return ret;
+}
-- 
2.34.1

[PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Simon Glass]

While sandbox supports virtio it cannot support actually using the block
devices to read files, since there is nothing on the other end of the
'virtqueue'.

A recent change makes EFI probe all block devices, whether used or not.
This is apparently required by EFI, although it violates U-Boot's
lazy-init principle.

We cannot just drop the virtio devices as they are used in sandbox tests.

So for now just add a special case to work around this.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 lib/efi_loader/efi_disk.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
index 93a9a5ac025..2e1d37848fc 100644
--- a/lib/efi_loader/efi_disk.c
+++ b/lib/efi_loader/efi_disk.c
@@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
 efi_status_t efi_disks_register(void)
 {
 	struct udevice *dev;
+	struct uclass *uc;
 
-	uclass_foreach_dev_probe(UCLASS_BLK, dev) {
+	uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
+		/*
+		 * The virtio block-device hangs on sandbox when accessed since
+		 * there is nothing listening to the mailbox
+		 */
+		if (IS_ENABLED(CONFIG_SANDBOX)) {
+			struct blk_desc *desc = dev_get_uclass_plat(dev);
+
+			if (desc->uclass_id == UCLASS_VIRTIO)
+				continue;
+		}
+		device_probe(dev);
 	}
 
 	return EFI_SUCCESS;
-- 
2.34.1

[PATCH v2 38/39] test: Set up an image suitable for EFI testing

[Simon Glass]

Create a new disk for use with tests, which contains the new 'testapp'
EFI app specifically intended for testing the EFI loader.

Attach it to the USB device, since most testing is currently done with
mmc.

Initially this image will be used to test the EFI bootmeth.

Fix a stale comment in prep_mmc_bootdev() while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 arch/sandbox/dts/test.dts           |   2 +-
 test/boot/bootdev.c                 |  18 +++++++++-
 test/boot/bootflow.c                |   2 +-
 test/py/tests/bootstd/flash1.img.xz | Bin 0 -> 5016 bytes
 test/py/tests/test_ut.py            |  52 ++++++++++++++++++++++++----
 5 files changed, 65 insertions(+), 9 deletions(-)
 create mode 100644 test/py/tests/bootstd/flash1.img.xz

diff --git a/arch/sandbox/dts/test.dts b/arch/sandbox/dts/test.dts
index 5fb5eac862e..a99de6e62ce 100644
--- a/arch/sandbox/dts/test.dts
+++ b/arch/sandbox/dts/test.dts
@@ -1507,7 +1507,7 @@
 				flash-stick@1 {
 					reg = <1>;
 					compatible = "sandbox,usb-flash";
-					sandbox,filepath = "testflash1.bin";
+					sandbox,filepath = "flash1.img";
 				};
 
 				flash-stick@2 {
diff --git a/test/boot/bootdev.c b/test/boot/bootdev.c
index de16a51956d..254fcd26a5c 100644
--- a/test/boot/bootdev.c
+++ b/test/boot/bootdev.c
@@ -216,6 +216,10 @@ static int bootdev_test_order(struct unit_test_state *uts)
 	/* Use the environment variable to override it */
 	ut_assertok(env_set("boot_targets", "mmc1 mmc2 usb"));
 	ut_assertok(bootflow_scan_first(NULL, NULL, &iter, 0, &bflow));
+
+	/* get the usb device which has a backing file (flash1.img) */
+	ut_asserteq(0, bootflow_scan_next(&iter, &bflow));
+
 	ut_asserteq(-ENODEV, bootflow_scan_next(&iter, &bflow));
 	ut_asserteq(5, iter.num_devs);
 	ut_asserteq_str("mmc1.bootdev", iter.dev_used[0]->name);
@@ -255,7 +259,11 @@ static int bootdev_test_order(struct unit_test_state *uts)
 	ut_assertok(bootflow_scan_first(NULL, NULL, &iter, 0, &bflow));
 	ut_asserteq(2, iter.num_devs);
 
-	/* Now scan past mmc1 and make sure that the 3 USB devices show up */
+	/*
+	 * Now scan past mmc1 and make sure that the 3 USB devices show up. The
+	 * first one has a backing file so returns success
+	 */
+	ut_asserteq(0, bootflow_scan_next(&iter, &bflow));
 	ut_asserteq(-ENODEV, bootflow_scan_next(&iter, &bflow));
 	ut_asserteq(6, iter.num_devs);
 	ut_asserteq_str("mmc2.bootdev", iter.dev_used[0]->name);
@@ -318,6 +326,10 @@ static int bootdev_test_prio(struct unit_test_state *uts)
 	/* 3 MMC and 3 USB bootdevs: MMC should come before USB */
 	console_record_reset_enable();
 	ut_assertok(bootflow_scan_first(NULL, NULL, &iter, 0, &bflow));
+
+	/* get the usb device which has a backing file (flash1.img) */
+	ut_asserteq(0, bootflow_scan_next(&iter, &bflow));
+
 	ut_asserteq(-ENODEV, bootflow_scan_next(&iter, &bflow));
 	ut_asserteq(6, iter.num_devs);
 	ut_asserteq_str("mmc2.bootdev", iter.dev_used[0]->name);
@@ -335,6 +347,10 @@ static int bootdev_test_prio(struct unit_test_state *uts)
 	bootflow_iter_uninit(&iter);
 	ut_assertok(bootflow_scan_first(NULL, NULL, &iter, BOOTFLOWIF_HUNT,
 					&bflow));
+
+	/* get the usb device which has a backing file (flash1.img) */
+	ut_asserteq(0, bootflow_scan_next(&iter, &bflow));
+
 	ut_asserteq(-ENODEV, bootflow_scan_next(&iter, &bflow));
 	ut_asserteq(7, iter.num_devs);
 	ut_asserteq_str("usb_mass_storage.lun0.bootdev",
diff --git a/test/boot/bootflow.c b/test/boot/bootflow.c
index 8ea091a36af..6d85cf20dd5 100644
--- a/test/boot/bootflow.c
+++ b/test/boot/bootflow.c
@@ -537,7 +537,7 @@ static int prep_mmc_bootdev(struct unit_test_state *uts, const char *mmc_dev,
 
 	order[2] = mmc_dev;
 
-	/* Enable the mmc4 node since we need a second bootflow */
+	/* Enable the requested mmc node since we need a second bootflow */
 	root = oftree_root(oftree_default());
 	node = ofnode_find_subnode(root, mmc_dev);
 	ut_assert(ofnode_valid(node));
diff --git a/test/py/tests/bootstd/flash1.img.xz b/test/py/tests/bootstd/flash1.img.xz
new file mode 100644
index 0000000000000000000000000000000000000000..9148b91a20c1f0acb54449b124d1408961e92507
GIT binary patch
literal 5016
zcmeI0cTf{r8pRVz9+4n5AV?_EgQ1BO;n5K(kx-;pUwkwL0tBUqlz==yF+ikBDAExL
zQWT{~2Sq?y06~gW5eP+U$nMPkwKMCEzInSlv*Z7F=Fa`i`OZ1tr78#8*Z}|x3nSGR
z=>Wn&ZU6ufAUmH=qlqxW9033y>XG5m*pL}cy$rvQVYbMeTPGAi$9qqfMnd47Jp+<S
zIxLcX-gdy?;z*~}QgHUO>b8@aTLW9^ZgBe#gV-u~nri0|VMr5lA6KhSV}ds3x_d!R
zK70s*W-GN^=;}ze2DKy`8Bp8a3s-y^i!J>$OdpXOt?{Uc-H+4p?j<BRmf`kr@RkOb
z#nr%a+fVcMj7<BT<4=wb&7}i+1?+`%_9rcV-IGFt)xzJvuYasj+h}W$aV}-mF#cWV
z0q02?Tk;;}mhB#MFO491k+tL~bm}lePR6P^F?uU1qWM~zn8n@>UP+#@pmNRUwJ*{D
zGJt0<ec}SEe@BuyWsjlSn3!f77C4m?PYr2;FS!1eCM{1>cjopJaH2B8t_BV*T|(V?
zQE{)kaF4zAwh?({NItl!v80%xP5j+-km~m0FjHOI&3eSrr0=y}ILxn<Dj+V+FtA;?
zkfwXZQuFzB-RUPwyo#9fJZ3vFLqjRnRXx{Cjj$``T7)LJeniuGCW|XG2Di@RTK-7v
z%@LmQG%VY9Q*iQk{IX5!h|sU6x5>&A^gJ={@<k08%TNsA;LO8|NDKjWvI9|*L4$MB
z+7n%gV=Mx1tAg1*_I$;8%e5KZC}1lz<@sfc9Rl028JFGGi3IeoQhBA1Rw!e4KbgZ>
zcWW%yKY}Nl<I~z}t_$|PJU0<-g!)ivhK&n#A9YhnK-3N*Q`%=WA+6TycBy`{rYBep
zuUe97ouG<2ucIu+XU;2LR8g8yklNz3;<k<<Au8XakMkm_=cM5gt`}-7tJ^bCnR7tn
zBe3b$z#jhsLv05PN&NHU+npZ4f?Y<fTmv;LxJ0w1oHl!Nb^<(FgZXuRy)xXYp~*LD
z3L*PG^Lc*574Eh7oi!F-PcV5j<eaGzxJ8`ao#8ImP6c^lrB7Shz+u!o#+lun*$NN4
ze+u>1tu^^*#UCxf;bJ&NSo@TX5>5e+pT?chViS8Qddj*v2(zyvvS5|tWhS65&b1s#
ztq%uj(ECDcNKU@WQlPgiVOeS{T^}+v^G!^rLMD_i<okVaaHNxvN%DKEC_OJjtJ`fy
zKFE0k6@0_k?{_0d-|8&uGq1;2=W4>2e;x1$aOEia;LV&Z930_1Thek8ad0!1k*%M3
z_Q0A`8vzaWYYX7vQ=^y;COH>9zmbcNam0;YlMB3IVMi><k<II&(g=3_UBMw^HCFC+
z@)H?uigXDBXA74rM>ftHk8xY%(%}{iymL<`3XMZW+f8a2?SETgT0;aAgL=a#3!1KX
z^cLebMqp!W-R{zh?QA#i^JT<kvfv0(8ry`yg<(qzh1_zt_{3BP*1~C5OmS7q)YTXx
zPX4FbYU!lv4)?BM&@_$q$I|V>ju!<g<X?nd#FT-)ZWn*qLX7tafjq@`!P?D{S5IGJ
zuvsiPZc%MJ!b8(+vK$@C`6S1;#b%n~3cgfSaXr-V3RSy1Czll1NJTo9Lb$}O7Z-C-
zx?3&7@!8q&vQI%h9&~esojd+AF!$PiZ2Gn^%Tq%h_1PQ80p&ToqgoX4NUYYmH|n;P
ztuS}UI)h4dAc_9_X4%0uX8Ct>o5tn8#754Z(}^I`q<{*5Rey^jYGXyHJRh;{p1ikj
zc9Dqls^pvK0YX|_7#zPTf!*b82Bk?#z3fQBiu4q*qMiE6{UF>)MUAS7qBGw*k4&SI
zpAJ7@{r3j-*FNfpyz$Rntz!RCiAZAuN`Ei6|5Fa%9ZzPgnpH&FE1=r9f!*ynU>LRk
zs^i2fv8m7Ts$<41M5$$$O!aU5TsIe9FH*b*iXYp#Gs1FO^Y+cMC@x{|f&W>e{yEir
zH?4mW<^(Pg@=wM6Kq2~v;(m_kcZ=>Pupg}4KY{%zU`U3zZ->A?w~ha5Y7#LChpCq#
zjLs}(gwV(qHKH0PIq==G%UE&4)cV#zQp!hu3P)N#5~J*rP#)Mrnf2FEJB=>p2`NB0
zxrbCGW-cOk>(@E=<>XoI^r+-qsU_Hbj=2C;jaT42`)4Nl?$<AeJrorX|6Ggdh?op?
zoEJg?^i<Tb`^gBPlWi0Ad}h<6eb;Olmg82pe%EB`<o%Hrcp--3u;QdeNlf1~)~}X+
z2_`kvu^$BMA*_~WOoBRm2Y2qwq_--k{wDL%<F=&7bN=H4xn;G>A9Ao502gVwPUdyH
zk|c8N(S#6;Vbf5B1-#i5rvvuH#*!j4W%J!DHrzLL7Y}2FiA;&kE{gOSUX)eQM58e%
zxzCf&=qNe#7eNq+P!z(uHeIWDRQiY=vVq^N5#m=JF!q>=l3~T}x`dQCQkTnB7Jyp9
z9<Ke9SBS3!ukGU#4CI-@!yY10$Hu`~TkU;%B9<w_?}51x8UCnvkWast$>!$S*si%Y
z#K@3yE)2HKtuQf<(p%-MP;{u#ln=wpQ1YA#gF=REU4Ysw)eE4nJR61Dj5RfBZ~dM*
zSMp96hovw!v!`3u@q#5n+3D*c)52Q{z19NhVU|dnO=-{cW*eIX_2Q97jaK23vWBG7
z@m-4+V3|8nS|8n5(7fuqxU~m1hiT=)sP4+=FIXUT1|kn%EzpMCxRuaG>(_h94n9E@
z^-0~gvKr&93iu$m=JxNvH`Z>5eVq0gWqPV|6Dd^9zrpc%XLqFfRKYs#OhX;3>Jm;b
zh)!B^-RF<)Tr}SWQrPsyLGNqxRRqOIG#PiTgwEoM7^!y@5qBOR-Cf%U)jHUm7=x1n
zabX0NS!nxd?gq)JryA$B@Ibw^qlg)E=DN@e$6aC^s8+O<N*T+sZtE3VJS6CHR;Hfb
zj6!OYV>_{x@AiR~<_W*A<j@8rZy})!cUSg4dNp5pXk>_j#gJdl-^e|Hn)_#5WHWmA
z*>1210pl<x|1SJEm1hL6rCToRBa2%(;ktp6={I?DPoB?_h#?1Y%pMU_Pnn5X#zFHB
zkD8>k_~Sa)Oa|ASTX<>*J$=fQh1r%Sk4yV+%E8c2*ol{zZAGEORz}_=pJYbILT?~9
zx@Ww_ZY!Phs9}DZ;|(-K7u0?|<x*!^Q((Pp0U@L0j5%*kym>tedsc3u1!ajplM14D
z?)FvVYF1EAD}VwEmh!>y)l!<B_<wT8$jHNU{OnXe*r~#qv;d5N2DZJiLI6N8ou4Vd
g!6FL)+!7BD4?iLFVyT<d=5|Q;_y0ElgRR})0MlW=WB>pF

literal 0
HcmV?d00001

diff --git a/test/py/tests/test_ut.py b/test/py/tests/test_ut.py
index 39aa1035e34..f647e2f0af2 100644
--- a/test/py/tests/test_ut.py
+++ b/test/py/tests/test_ut.py
@@ -28,21 +28,22 @@ def mkdir_cond(dirname):
     if not os.path.exists(dirname):
         os.mkdir(dirname)
 
-def setup_image(cons, mmc_dev, part_type, second_part=False):
+def setup_image(cons, devnum, part_type, second_part=False, basename='mmc'):
     """Create a 20MB disk image with a single partition
 
     Args:
         cons (ConsoleBase): Console to use
-        mmc_dev (int): MMC device number to use, e.g. 1
+        devnum (int): Device number to use, e.g. 1
         part_type (int): Partition type, e.g. 0xc for FAT32
         second_part (bool): True to contain a small second partition
+        basename (str): Base name to use in the filename, e.g. 'mmc'
 
     Returns:
         tuple:
             str: Filename of MMC image
             str: Directory name of 'mnt' directory
     """
-    fname = os.path.join(cons.config.source_dir, f'mmc{mmc_dev}.img')
+    fname = os.path.join(cons.config.source_dir, f'{basename}{devnum}.img')
     mnt = os.path.join(cons.config.persistent_data_dir, 'mnt')
     mkdir_cond(mnt)
 
@@ -78,16 +79,17 @@ def mount_image(cons, fname, mnt, fstype):
     u_boot_utils.run_and_log(cons, f'sudo chown {getpass.getuser()} {mnt}')
     return loop
 
-def copy_prepared_image(cons, mmc_dev, fname):
+def copy_prepared_image(cons, devnum, fname, basename='mmc'):
     """Use a prepared image since we cannot create one
 
     Args:
         cons (ConsoleBase): Console touse
-        mmc_dev (int): MMC device number
+        devnum (int): device number
         fname (str): Filename of MMC image
+        basename (str): Base name to use in the filename, e.g. 'mmc'
     """
     infname = os.path.join(cons.config.source_dir,
-                           f'test/py/tests/bootstd/mmc{mmc_dev}.img.xz')
+                           f'test/py/tests/bootstd/{basename}{devnum}.img.xz')
     u_boot_utils.run_and_log(cons, ['sh', '-c', f'xz -dc {infname} >{fname}'])
 
 def setup_bootmenu_image(cons):
@@ -549,6 +551,43 @@ def test_ut_dm_init(u_boot_console):
     with open(fn, 'wb') as fh:
         fh.write(data)
 
+
+def setup_efi_image(cons):
+    """Create a 20MB disk image with an EFI app on it"""
+    devnum = 1
+    basename = 'flash'
+    fname, mnt = setup_image(cons, devnum, 0xc, second_part=True,
+                             basename=basename)
+
+    loop = None
+    mounted = False
+    complete = False
+    try:
+        loop = mount_image(cons, fname, mnt, 'ext4')
+        mounted = True
+        efi_dir = os.path.join(mnt, 'EFI')
+        mkdir_cond(efi_dir)
+        bootdir = os.path.join(efi_dir, 'BOOT')
+        mkdir_cond(bootdir)
+        efi_src = os.path.join(cons.config.build_dir,
+                               f'lib/efi_loader/testapp.efi')
+        efi_dst = os.path.join(bootdir, 'BOOTSBOX.EFI')
+        with open(efi_src, 'rb') as inf:
+            with open(efi_dst, 'wb') as outf:
+                outf.write(inf.read())
+    except ValueError as exc:
+        print(f'Falled to create image, failing back to prepared copy: {exc}')
+
+    finally:
+        if mounted:
+            u_boot_utils.run_and_log(cons, 'sudo umount --lazy %s' % mnt)
+        if loop:
+            u_boot_utils.run_and_log(cons, 'sudo losetup -d %s' % loop)
+
+    if not complete:
+        copy_prepared_image(cons, devnum, fname, basename)
+
+
 @pytest.mark.buildconfigspec('cmd_bootflow')
 @pytest.mark.buildconfigspec('sandbox')
 def test_ut_dm_init_bootstd(u_boot_console):
@@ -559,6 +598,7 @@ def test_ut_dm_init_bootstd(u_boot_console):
     setup_cedit_file(u_boot_console)
     setup_cros_image(u_boot_console)
     setup_android_image(u_boot_console)
+    setup_efi_image(u_boot_console)
 
     # Restart so that the new mmc1.img is picked up
     u_boot_console.restart_uboot()
-- 
2.34.1

[PATCH v2 39/39] efi: Add a test for the efi bootmeth

[Simon Glass]

Add a simple test of booting with the EFI bootmeth, which runs the app
and checks that it can call 'exit boot-services' (to check that all the
device-removal code doesn't break anything) and then exit back to
U-Boot.

This uses a disk image containing the testapp, ready for execution by
sandbox when needed.

Signed-off-by: Simon Glass <sjg@chromium.org>

---

Changes in v2:
- Add many new patches to resolve all the outstanding test issues

 test/boot/bootflow.c | 58 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/test/boot/bootflow.c b/test/boot/bootflow.c
index 6d85cf20dd5..8822dac8503 100644
--- a/test/boot/bootflow.c
+++ b/test/boot/bootflow.c
@@ -13,6 +13,7 @@
 #include <cli.h>
 #include <dm.h>
 #include <efi_default_filename.h>
+#include <efi_loader.h>
 #include <expo.h>
 #ifdef CONFIG_SANDBOX
 #include <asm/test.h>
@@ -31,6 +32,9 @@ extern U_BOOT_DRIVER(bootmeth_android);
 extern U_BOOT_DRIVER(bootmeth_cros);
 extern U_BOOT_DRIVER(bootmeth_2script);
 
+/* Use this as the vendor for EFI to tell the app to exit boot services */
+static u16 __efi_runtime_data test_vendor[] = u"U-Boot testing";
+
 static int inject_response(struct unit_test_state *uts)
 {
 	/*
@@ -1225,3 +1229,57 @@ static int bootflow_android(struct unit_test_state *uts)
 	return 0;
 }
 BOOTSTD_TEST(bootflow_android, UT_TESTF_CONSOLE_REC);
+
+/* Test EFI bootmeth */
+static int bootflow_efi(struct unit_test_state *uts)
+{
+	/* disable ethernet since the hunter will run dhcp */
+	test_set_eth_enable(false);
+
+	/* make USB scan without delays */
+	test_set_skip_delays(true);
+
+	bootstd_reset_usb();
+
+	/* Avoid outputting ANSI characters which mess with our asserts */
+	efi_console_set_ansi(false);
+
+	ut_assertok(bootstd_test_drop_bootdev_order(uts));
+	ut_assertok(run_command("bootflow scan", 0));
+	ut_assert_skip_to_line(
+		"Bus usb@1: scanning bus usb@1 for devices... 5 USB Device(s) found");
+
+	ut_assertok(run_command("bootflow list", 0));
+
+	ut_assert_nextlinen("Showing all");
+	ut_assert_nextlinen("Seq");
+	ut_assert_nextlinen("---");
+	ut_assert_nextlinen("  0  extlinux");
+	ut_assert_nextlinen(
+		"  1  efi          ready   usb_mass_    1  usb_mass_storage.lun0.boo /EFI/BOOT/BOOTSBOX.EFI");
+	ut_assert_nextlinen("---");
+	ut_assert_skip_to_line("(2 bootflows, 2 valid)");
+	ut_assert_console_end();
+
+	ut_assertok(run_command("bootflow select 1", 0));
+	ut_assert_console_end();
+
+	/* signal to helloworld to exit boot services */
+	systab.fw_vendor = test_vendor;
+
+	ut_asserteq(1, run_command("bootflow boot", 0));
+	ut_assert_nextline(
+		"** Booting bootflow 'usb_mass_storage.lun0.bootdev.part_1' with efi");
+	ut_assert_skip_to_line("Booting /\\EFI\\BOOT\\BOOTSBOX.EFI");
+
+	/* TODO: Why the \r ? */
+	ut_assert_nextline("U-Boot test app for EFI_LOADER\r");
+	ut_assert_nextline("Exiting boot sevices");
+	ut_assert_nextline("## Application failed, r = 5");
+	ut_assert_nextline("Boot failed (err=-22)");
+
+	ut_assert_console_end();
+
+	return 0;
+}
+BOOTSTD_TEST(bootflow_efi, UT_TESTF_CONSOLE_REC);
-- 
2.34.1

Re: [PATCH v2 25/39] efi: Use puts() in cout so that console recording works

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> At present EFI output to the console uses fputs() which bypasses the
> console-recording feature. This makes it impossible for tests to check
> the output of an EFI app.
>
> There doesn't seem to be any need to do this bypass, so adjust it to
> simply use the puts() function.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>

Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>

> ---
>
> Changes in v2:
> - Fix 'use' typo
>
>   lib/efi_loader/efi_console.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/efi_loader/efi_console.c b/lib/efi_loader/efi_console.c
> index c944c10b216..cea50c748aa 100644
> --- a/lib/efi_loader/efi_console.c
> +++ b/lib/efi_loader/efi_console.c
> @@ -181,7 +181,7 @@ static efi_status_t EFIAPI efi_cout_output_string(
>   	}
>   	pos = buf;
>   	utf16_utf8_strcpy(&pos, string);
> -	fputs(stdout, buf);
> +	puts(buf);
>   	free(buf);
>
>   	/*

Re: [PATCH v2 27/39] efi_loader: Rename and move CMD_BOOTEFI_HELLO_COMPILE

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> This is not actually a command so the name is confusing. Use
> BOOTEFI_HELLO_COMPILE instead. Put it in the efi_loader directory
> with the other such config options.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>   arch/arm/lib/Makefile            |  2 +-
>   arch/riscv/lib/Makefile          |  2 +-
>   arch/x86/lib/Makefile            |  2 +-
>   cmd/Kconfig                      | 14 +-------------
>   configs/octeontx2_95xx_defconfig |  2 +-
>   configs/octeontx2_96xx_defconfig |  2 +-
>   configs/octeontx_81xx_defconfig  |  2 +-
>   configs/octeontx_83xx_defconfig  |  2 +-
>   doc/develop/uefi/uefi.rst        |  2 +-
>   lib/efi_loader/Kconfig           | 12 ++++++++++++
>   lib/efi_loader/Makefile          |  2 +-
>   test/py/tests/test_efi_fit.py    |  2 +-
>   test/py/tests/test_efi_loader.py |  2 +-
>   13 files changed, 24 insertions(+), 24 deletions(-)
>
> diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile
> index 67275fba616..406797f998c 100644
> --- a/arch/arm/lib/Makefile
> +++ b/arch/arm/lib/Makefile
> @@ -130,7 +130,7 @@ CFLAGS_REMOVE_$(EFI_CRT0) := $(CFLAGS_NON_EFI)
>   CFLAGS_$(EFI_RELOC) := $(CFLAGS_EFI)
>   CFLAGS_REMOVE_$(EFI_RELOC) := $(CFLAGS_NON_EFI)
>
> -extra-$(CONFIG_CMD_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)
> +extra-$(CONFIG_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)

In scripts/Makefile.lib we have this line:

$(obj)/%_efi.so: $(obj)/%.o $(obj)/efi_crt0.o $(obj)/efi_reloc.o
$(obj)/efi_freestanding.o

This is enough to build efi_crt0.o and efi_reloc.o when needed.

Please, remove the extra-* lines for these objects.

>   # TODO: As of v2019.01 the relocation code for the EFI application cannot
>   # be built on ARMv7-M.
>   ifndef CONFIG_CPU_V7M
> diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile
> index 65dc49f6fa5..3bd90c7a354 100644
> --- a/arch/riscv/lib/Makefile
> +++ b/arch/riscv/lib/Makefile
> @@ -36,7 +36,7 @@ CFLAGS_REMOVE_$(EFI_CRT0) := $(CFLAGS_NON_EFI)
>   CFLAGS_$(EFI_RELOC) := $(CFLAGS_EFI)
>   CFLAGS_REMOVE_$(EFI_RELOC) := $(CFLAGS_NON_EFI)
>
> -extra-$(CONFIG_CMD_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)
> +extra-$(CONFIG_BOOTEFI_HELLO_COMPILE) += $(EFI_CRT0) $(EFI_RELOC)
>   extra-$(CONFIG_CMD_BOOTEFI_SELFTEST) += $(EFI_CRT0) $(EFI_RELOC)
>   extra-$(CONFIG_EFI) += $(EFI_CRT0) $(EFI_RELOC)

ditto

Best regards

Heinrich

>
> diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
> index 8fc35e1b51e..70dccd7f983 100644
> --- a/arch/x86/lib/Makefile
> +++ b/arch/x86/lib/Makefile
> @@ -97,7 +97,7 @@ endif
>   else
>
>   ifndef CONFIG_SPL_BUILD
> -ifneq ($(CONFIG_CMD_BOOTEFI_SELFTEST)$(CONFIG_CMD_BOOTEFI_HELLO_COMPILE),)
> +ifneq ($(CONFIG_CMD_BOOTEFI_SELFTEST)$(CONFIG_BOOTEFI_HELLO_COMPILE),)
>   extra-y += $(EFI_CRT0) $(EFI_RELOC)
>   endif
>   endif
> diff --git a/cmd/Kconfig b/cmd/Kconfig
> index 978f44eda42..fcf6bb73f88 100644
> --- a/cmd/Kconfig
> +++ b/cmd/Kconfig
> @@ -431,21 +431,9 @@ config CMD_BOOTEFI_BOOTMGR
>   	  This subcommand will allow you to select the UEFI binary to be booted
>   	  via UEFI variables Boot####, BootOrder, and BootNext.
>
> -config CMD_BOOTEFI_HELLO_COMPILE
> -	bool "Compile a standard EFI hello world binary for testing"
> -	default y
> -	help
> -	  This compiles a standard EFI hello world application with U-Boot so
> -	  that it can be used with the test/py testing framework. This is useful
> -	  for testing that EFI is working at a basic level, and for bringing
> -	  up EFI support on a new architecture.
> -
> -	  No additional space will be required in the resulting U-Boot binary
> -	  when this option is enabled.
> -
>   config CMD_BOOTEFI_HELLO
>   	bool "Allow booting a standard EFI hello world for testing"
> -	depends on CMD_BOOTEFI_BINARY && CMD_BOOTEFI_HELLO_COMPILE
> +	depends on CMD_BOOTEFI_BINARY && BOOTEFI_HELLO_COMPILE
>   	default y if CMD_BOOTEFI_SELFTEST
>   	help
>   	  This adds a standard EFI hello world application to U-Boot so that
> diff --git a/configs/octeontx2_95xx_defconfig b/configs/octeontx2_95xx_defconfig
> index c5dc4f4dfa6..23c313375ac 100644
> --- a/configs/octeontx2_95xx_defconfig
> +++ b/configs/octeontx2_95xx_defconfig
> @@ -38,7 +38,7 @@ CONFIG_SYS_PBSIZE=1050
>   CONFIG_BOARD_EARLY_INIT_R=y
>   CONFIG_HUSH_PARSER=y
>   CONFIG_SYS_PROMPT="Marvell> "
> -# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
> +# CONFIG_BOOTEFI_HELLO_COMPILE is not set
>   CONFIG_CMD_MD5SUM=y
>   CONFIG_MD5SUM_VERIFY=y
>   CONFIG_CMD_MX_CYCLIC=y
> diff --git a/configs/octeontx2_96xx_defconfig b/configs/octeontx2_96xx_defconfig
> index ad61b80300f..197e72acd1f 100644
> --- a/configs/octeontx2_96xx_defconfig
> +++ b/configs/octeontx2_96xx_defconfig
> @@ -38,7 +38,7 @@ CONFIG_SYS_PBSIZE=1050
>   CONFIG_BOARD_EARLY_INIT_R=y
>   CONFIG_HUSH_PARSER=y
>   CONFIG_SYS_PROMPT="Marvell> "
> -# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
> +# CONFIG_BOOTEFI_HELLO_COMPILE is not set
>   CONFIG_CMD_MD5SUM=y
>   CONFIG_MD5SUM_VERIFY=y
>   CONFIG_CMD_MX_CYCLIC=y
> diff --git a/configs/octeontx_81xx_defconfig b/configs/octeontx_81xx_defconfig
> index 1d39bce6abd..b501d653c27 100644
> --- a/configs/octeontx_81xx_defconfig
> +++ b/configs/octeontx_81xx_defconfig
> @@ -39,7 +39,7 @@ CONFIG_SYS_PBSIZE=1050
>   CONFIG_BOARD_EARLY_INIT_R=y
>   CONFIG_HUSH_PARSER=y
>   CONFIG_SYS_PROMPT="Marvell> "
> -# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
> +# CONFIG_BOOTEFI_HELLO_COMPILE is not set
>   CONFIG_CMD_MD5SUM=y
>   CONFIG_MD5SUM_VERIFY=y
>   CONFIG_CMD_MX_CYCLIC=y
> diff --git a/configs/octeontx_83xx_defconfig b/configs/octeontx_83xx_defconfig
> index ba9fc5f9553..4a537f5cbbe 100644
> --- a/configs/octeontx_83xx_defconfig
> +++ b/configs/octeontx_83xx_defconfig
> @@ -37,7 +37,7 @@ CONFIG_SYS_PBSIZE=1050
>   CONFIG_BOARD_EARLY_INIT_R=y
>   CONFIG_HUSH_PARSER=y
>   CONFIG_SYS_PROMPT="Marvell> "
> -# CONFIG_CMD_BOOTEFI_HELLO_COMPILE is not set
> +# CONFIG_BOOTEFI_HELLO_COMPILE is not set
>   CONFIG_CMD_MD5SUM=y
>   CONFIG_MD5SUM_VERIFY=y
>   CONFIG_CMD_MX_CYCLIC=y
> diff --git a/doc/develop/uefi/uefi.rst b/doc/develop/uefi/uefi.rst
> index d450b12bf80..9aeee23a32e 100644
> --- a/doc/develop/uefi/uefi.rst
> +++ b/doc/develop/uefi/uefi.rst
> @@ -693,7 +693,7 @@ Executing the built in hello world application
>
>   A hello world UEFI application can be built with::
>
> -    CONFIG_CMD_BOOTEFI_HELLO_COMPILE=y
> +    CONFIG_BOOTEFI_HELLO_COMPILE=y
>
>   It can be embedded into the U-Boot binary with::
>
> diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
> index 1179c31bb13..ab2c1c44364 100644
> --- a/lib/efi_loader/Kconfig
> +++ b/lib/efi_loader/Kconfig
> @@ -516,4 +516,16 @@ config EFI_HTTP_BOOT
>   	  Enabling this option adds EFI HTTP Boot support. It allows to
>   	  directly boot from network.
>
> +config BOOTEFI_HELLO_COMPILE
> +	bool "Compile a standard EFI hello world binary for testing"
> +	default y
> +	help
> +	  This compiles a standard EFI hello world application with U-Boot so
> +	  that it can be used with the test/py testing framework. This is useful
> +	  for testing that EFI is working at a basic level, and for bringing
> +	  up EFI support on a new architecture.
> +
> +	  No additional space will be required in the resulting U-Boot binary
> +	  when this option is enabled.
> +
>   endif
> diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
> index 2af6f2066b5..27dbd9e760d 100644
> --- a/lib/efi_loader/Makefile
> +++ b/lib/efi_loader/Makefile
> @@ -27,7 +27,7 @@ always += boothart.efi
>   targets += boothart.o
>   endif
>
> -ifneq ($(CONFIG_CMD_BOOTEFI_HELLO_COMPILE),)
> +ifneq ($(CONFIG_BOOTEFI_HELLO_COMPILE),)
>   always += helloworld.efi
>   targets += helloworld.o
>   endif
> diff --git a/test/py/tests/test_efi_fit.py b/test/py/tests/test_efi_fit.py
> index 0ad483500f8..550058a30fd 100644
> --- a/test/py/tests/test_efi_fit.py
> +++ b/test/py/tests/test_efi_fit.py
> @@ -119,7 +119,7 @@ FDT_DATA = '''
>   '''
>
>   @pytest.mark.buildconfigspec('bootm_efi')
> -@pytest.mark.buildconfigspec('cmd_bootefi_hello_compile')
> +@pytest.mark.buildconfigspec('BOOTEFI_HELLO_COMPILE')
>   @pytest.mark.buildconfigspec('fit')
>   @pytest.mark.notbuildconfigspec('generate_acpi_table')
>   @pytest.mark.requiredtool('dtc')
> diff --git a/test/py/tests/test_efi_loader.py b/test/py/tests/test_efi_loader.py
> index 85473a9049b..ef1f1f1afb3 100644
> --- a/test/py/tests/test_efi_loader.py
> +++ b/test/py/tests/test_efi_loader.py
> @@ -148,7 +148,7 @@ def fetch_tftp_file(u_boot_console, env_conf):
>       return addr
>
>   @pytest.mark.buildconfigspec('of_control')
> -@pytest.mark.buildconfigspec('cmd_bootefi_hello_compile')
> +@pytest.mark.buildconfigspec('BOOTEFI_HELLO_COMPILE')
>   def test_efi_helloworld_net(u_boot_console):
>       """Run the helloworld.efi binary via TFTP.
>

Re: [PATCH v2 28/39] efi_loader: Shorten the app rules

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> We have quite a few apps now, so create a way to specify them as a list
> rather than repeating the same rules again and again.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>

Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>

> ---
>
> (no changes since v1)
>
>   lib/efi_loader/Makefile | 31 +++++++++----------------------
>   1 file changed, 9 insertions(+), 22 deletions(-)
>
> diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
> index 27dbd9e760d..660368b9d8f 100644
> --- a/lib/efi_loader/Makefile
> +++ b/lib/efi_loader/Makefile
> @@ -22,29 +22,13 @@ CFLAGS_REMOVE_dtbdump.o := $(CFLAGS_NON_EFI)
>   CFLAGS_initrddump.o := $(CFLAGS_EFI) -Os -ffreestanding
>   CFLAGS_REMOVE_initrddump.o := $(CFLAGS_NON_EFI)
>
> -ifdef CONFIG_RISCV
> -always += boothart.efi
> -targets += boothart.o
> -endif
> -
> -ifneq ($(CONFIG_BOOTEFI_HELLO_COMPILE),)
> -always += helloworld.efi
> -targets += helloworld.o
> -endif
> -
> -ifneq ($(CONFIG_GENERATE_SMBIOS_TABLE),)
> -always += smbiosdump.efi
> -targets += smbiosdump.o
> -endif
> -
> +# These are the apps that are built
> +apps-$(CONFIG_RISCV) += boothart
> +apps-$(CONFIG_BOOTEFI_HELLO_COMPILE) += helloworld
> +apps-$(CONFIG_GENERATE_SMBIOS_TABLE) += smbiosdump
> +apps-$(CONFIG_EFI_LOAD_FILE2_INITRD) += initrddump
>   ifeq ($(CONFIG_GENERATE_ACPI_TABLE),)
> -always += dtbdump.efi
> -targets += dtbdump.o
> -endif
> -
> -ifdef CONFIG_EFI_LOAD_FILE2_INITRD
> -always += initrddump.efi
> -targets += initrddump.o
> +apps-y += dtbdump
>   endif
>
>   obj-$(CONFIG_CMD_BOOTEFI_HELLO) += helloworld_efi.o
> @@ -95,3 +79,6 @@ obj-$(CONFIG_EFI_ECPT) += efi_conformance.o
>
>   EFI_VAR_SEED_FILE := $(subst $\",,$(CONFIG_EFI_VAR_SEED_FILE))
>   $(obj)/efi_var_seed.o: $(srctree)/$(EFI_VAR_SEED_FILE)
> +
> +always += $(foreach f,$(apps-y),$(f).efi)
> +targets += $(foreach f,$(apps-y),$(f).o)

Re: [PATCH v2 29/39] efi_loader: Shorten the app rules further

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> Add a way to factor out the CFLAGS changes for each app, since they are
> all the same.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>

The same could be done in lib/efi_selftest/Makefile.

Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>


> ---
>
> (no changes since v1)
>
>   lib/efi_loader/Makefile | 15 +++++----------
>   1 file changed, 5 insertions(+), 10 deletions(-)
>
> diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
> index 660368b9d8f..00d18966f9e 100644
> --- a/lib/efi_loader/Makefile
> +++ b/lib/efi_loader/Makefile
> @@ -11,16 +11,6 @@ asflags-y += -I.
>   CFLAGS_efi_boottime.o += \
>     -DFW_VERSION="0x$(VERSION)" \
>     -DFW_PATCHLEVEL="0x$(PATCHLEVEL)"
> -CFLAGS_boothart.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_boothart.o := $(CFLAGS_NON_EFI)
> -CFLAGS_helloworld.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_helloworld.o := $(CFLAGS_NON_EFI)
> -CFLAGS_smbiosdump.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_smbiosdump.o := $(CFLAGS_NON_EFI)
> -CFLAGS_dtbdump.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_dtbdump.o := $(CFLAGS_NON_EFI)
> -CFLAGS_initrddump.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_initrddump.o := $(CFLAGS_NON_EFI)
>
>   # These are the apps that are built
>   apps-$(CONFIG_RISCV) += boothart
> @@ -80,5 +70,10 @@ obj-$(CONFIG_EFI_ECPT) += efi_conformance.o
>   EFI_VAR_SEED_FILE := $(subst $\",,$(CONFIG_EFI_VAR_SEED_FILE))
>   $(obj)/efi_var_seed.o: $(srctree)/$(EFI_VAR_SEED_FILE)
>
> +# Set the C flags to add and remove for each app
> +$(foreach f,$(apps-y),\
> +	$(eval CFLAGS_$(f).o := $(CFLAGS_EFI) -Os -ffreestanding)\
> +	$(eval CFLAGS_REMOVE_$(f).o := $(CFLAGS_NON_EFI)))
> +
>   always += $(foreach f,$(apps-y),$(f).efi)
>   targets += $(foreach f,$(apps-y),$(f).o)

Re: [PATCH v2 30/39] efi: Show the vendor in helloworld

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> Show the vendor name so it is clear that this app has been built by
> U-Boot.

As described in the UEFI specification the system table is provided by
the UEFI firmware to the entry-point of EFI binaries. So on a Laptop you
might see "Xiaomi" or "Huawei" displayed when running through the new
lines. "U-Boot" will only show up when running on U-Boot.

>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>   lib/efi_loader/helloworld.c | 3 +++
>   1 file changed, 3 insertions(+)
>
> diff --git a/lib/efi_loader/helloworld.c b/lib/efi_loader/helloworld.c
> index 586177de0c8..f120249126d 100644
> --- a/lib/efi_loader/helloworld.c
> +++ b/lib/efi_loader/helloworld.c
> @@ -237,6 +237,9 @@ efi_status_t EFIAPI efi_main(efi_handle_t handle,
>   			(con_out, u"Missing device path for device handle\r\n");
>   		goto out;
>   	}
> +	con_out->output_string(con_out, u"Vendor: ");

%s/Vendor/Firmware vendor/

Best regards

Heinrich

> +	con_out->output_string(con_out, systab->fw_vendor);
> +	con_out->output_string(con_out, u"\n");
>   	con_out->output_string(con_out, u"Boot device: ");
>   	ret = print_device_path(device_path, device_path_to_text);
>   	if (ret != EFI_SUCCESS)

Re: [PATCH v2 31/39] Revert "bootdev: avoid infinite probe loop"

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> This turns out to be insufficient to fix the problem, since when
> bootdev_next_prio() exits, the caller has no idea that this really
> is the end. Nor is it, since there may be other devices which should
> be checked.

Please, describe how the original problem will be addressed.

@Caleb:

Do you have instruction how to reproduce the problem?

>
> The caller iterates which calls iter_incr() which calls
> bootdev_next_prio() again, which finds the same device and the loop
> continues.
>
> We never did create a test for this[1], which makes it hard to be
> sure which problem was fixed.
>
> The original code had the virtual of staying in the loop looking for

%s/virtual/virtue/

Best regards

Heinrich

> a bootdev, so let's go back to that and try to fix this another way.
>
> This reverts commit 9d92c418acfb7576e12e2bd53fed294bb9543724.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>   boot/bootdev-uclass.c | 14 ++------------
>   1 file changed, 2 insertions(+), 12 deletions(-)
>
> diff --git a/boot/bootdev-uclass.c b/boot/bootdev-uclass.c
> index 7c7bba088c9..15a8a3555c6 100644
> --- a/boot/bootdev-uclass.c
> +++ b/boot/bootdev-uclass.c
> @@ -632,7 +632,7 @@ int bootdev_next_label(struct bootflow_iter *iter, struct udevice **devp,
>
>   int bootdev_next_prio(struct bootflow_iter *iter, struct udevice **devp)
>   {
> -	struct udevice *dev = *devp, *last_dev = NULL;
> +	struct udevice *dev = *devp;
>   	bool found;
>   	int ret;
>
> @@ -682,19 +682,9 @@ int bootdev_next_prio(struct bootflow_iter *iter, struct udevice **devp)
>   			}
>   		} else {
>   			ret = device_probe(dev);
> -			if (!ret)
> -				last_dev = dev;
>   			if (ret) {
> -				log_warning("Device '%s' failed to probe\n",
> +				log_debug("Device '%s' failed to probe\n",
>   					  dev->name);
> -				if (last_dev == dev) {
> -					/*
> -					 * We have already tried this device
> -					 * and it failed to probe. Give up.
> -					 */
> -					return log_msg_ret("probe", ret);
> -				}
> -				last_dev = dev;
>   				dev = NULL;
>   			}
>   		}

Re: [PATCH v2 36/39] efi: Add a test app

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> Add a simple app to use for testing. This is intended to do whatever it
> needs to for testing purposes. For now it just prints a message and
> exits boot services.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>   lib/efi_loader/Kconfig   | 10 ++++++
>   lib/efi_loader/Makefile  |  1 +
>   lib/efi_loader/testapp.c | 68 ++++++++++++++++++++++++++++++++++++++++
>   3 files changed, 79 insertions(+)
>   create mode 100644 lib/efi_loader/testapp.c
>
> diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
> index ab2c1c44364..4de05c6f2d6 100644
> --- a/lib/efi_loader/Kconfig
> +++ b/lib/efi_loader/Kconfig
> @@ -528,4 +528,14 @@ config BOOTEFI_HELLO_COMPILE
>   	  No additional space will be required in the resulting U-Boot binary
>   	  when this option is enabled.
>
> +config BOOTEFI_TESTAPP_COMPILE
> +	bool "Compile an EFI test app for testing"
> +	default y
> +	help
> +	  This compiles an app designed for testing. It is packed into an image
> +	  by the test.py testing frame in the setup_efi_image() function.
> +
> +	  No additional space will be required in the resulting U-Boot binary
> +	  when this option is enabled.
> +
>   endif
> diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
> index 00d18966f9e..87131ab911d 100644
> --- a/lib/efi_loader/Makefile
> +++ b/lib/efi_loader/Makefile
> @@ -20,6 +20,7 @@ apps-$(CONFIG_EFI_LOAD_FILE2_INITRD) += initrddump
>   ifeq ($(CONFIG_GENERATE_ACPI_TABLE),)
>   apps-y += dtbdump
>   endif
> +apps-$(CONFIG_BOOTEFI_TESTAPP_COMPILE) += testapp
>
>   obj-$(CONFIG_CMD_BOOTEFI_HELLO) += helloworld_efi.o
>   obj-$(CONFIG_EFI_BOOTMGR) += efi_bootmgr.o
> diff --git a/lib/efi_loader/testapp.c b/lib/efi_loader/testapp.c
> new file mode 100644
> index 00000000000..feb444c92e9
> --- /dev/null
> +++ b/lib/efi_loader/testapp.c
> @@ -0,0 +1,68 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Hello world EFI application
> + *
> + * Copyright 2024 Google LLC
> + * Written by Simon Glass <sjg@chromium.org>
> + *
> + * This test program is used to test the invocation of an EFI application.
> + * It writes a few messages to the console and then exits boot services
> + */
> +
> +#include <efi_api.h>
> +
> +static const efi_guid_t loaded_image_guid = EFI_LOADED_IMAGE_PROTOCOL_GUID;
> +
> +static struct efi_system_table *systable;
> +static struct efi_boot_services *boottime;
> +static struct efi_simple_text_output_protocol *con_out;
> +
> +/**
> + * efi_main() - entry point of the EFI application.
> + *
> + * @handle:	handle of the loaded image
> + * @systab:	system table
> + * Return:	status code
> + */
> +efi_status_t EFIAPI efi_main(efi_handle_t handle,
> +			     struct efi_system_table *systab)
> +{
> +	struct efi_loaded_image *loaded_image;
> +	efi_status_t ret;
> +	efi_uintn_t map_size;
> +	efi_uintn_t map_key;
> +	efi_uintn_t desc_size;
> +	u32 desc_version;
> +
> +	systable = systab;
> +	boottime = systable->boottime;
> +	con_out = systable->con_out;
> +
> +	/* Get the loaded image protocol */
> +	ret = boottime->open_protocol(handle, &loaded_image_guid,
> +				      (void **)&loaded_image, NULL, NULL,
> +				      EFI_OPEN_PROTOCOL_GET_PROTOCOL);
> +	if (ret != EFI_SUCCESS) {
> +		con_out->output_string
> +			(con_out, u"Cannot open loaded image protocol\r\n");
> +		goto out;
> +	}
> +
> +	/* UEFI requires CR LF */
> +	con_out->output_string(con_out, u"U-Boot test app for EFI_LOADER\r\n");
> +
> +out:
> +	map_size = 0;
> +	ret = boottime->get_memory_map(&map_size, NULL, &map_key, &desc_size,
> +				       &desc_version);
> +	con_out->output_string(con_out, u"Exiting boot sevices\n");
> +
> +	/* exit boot services so that this part of U-Boot can be tested */
> +	boottime->exit_boot_services(handle, map_key);
> +
> +	/* now exit for real */
> +	ret = boottime->exit(handle, ret, 0, NULL);

Please, have a look at chapter 7.4.6,
"EFI_BOOT_SERVICES.ExitBootServices" of the UEFI specification.

After ExitBootServices() you cannot return anywhere. Boot services are
not available anymore. You can only invoke the UEFI runtime services
which include ResetSystem().

Best regards

Heinrich

> +
> +	/* We should never arrive here */
> +	return ret;
> +}

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> While sandbox supports virtio it cannot support actually using the block
> devices to read files, since there is nothing on the other end of the
> 'virtqueue'.
>
> A recent change makes EFI probe all block devices, whether used or not.
> This is apparently required by EFI, although it violates U-Boot's
> lazy-init principle.
>
> We cannot just drop the virtio devices as they are used in sandbox tests.
>
> So for now just add a special case to work around this.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
>   1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> index 93a9a5ac025..2e1d37848fc 100644
> --- a/lib/efi_loader/efi_disk.c
> +++ b/lib/efi_loader/efi_disk.c
> @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
>   efi_status_t efi_disks_register(void)
>   {
>   	struct udevice *dev;
> +	struct uclass *uc;
>
> -	uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> +	uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> +		/*
> +		 * The virtio block-device hangs on sandbox when accessed since
> +		 * there is nothing listening to the mailbox
> +		 */
> +		if (IS_ENABLED(CONFIG_SANDBOX)) {
> +			struct blk_desc *desc = dev_get_uclass_plat(dev);
> +
> +			if (desc->uclass_id == UCLASS_VIRTIO)
> +				continue;

We should avoid depending on the sandbox everywhere.

Please, fix the problem in the sandbox driver.

If you cannot fix it, run the tests involving virtio on QEMU instead of
the sandbox.

Best regards

Heinrich

> +		}
> +		device_probe(dev);
>   	}
>
>   	return EFI_SUCCESS;

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 2134 bytes --]

On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> On 06.08.24 14:58, Simon Glass wrote:
> > While sandbox supports virtio it cannot support actually using the block
> > devices to read files, since there is nothing on the other end of the
> > 'virtqueue'.
> > 
> > A recent change makes EFI probe all block devices, whether used or not.
> > This is apparently required by EFI, although it violates U-Boot's
> > lazy-init principle.
> > 
> > We cannot just drop the virtio devices as they are used in sandbox tests.
> > 
> > So for now just add a special case to work around this.
> > 
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> > 
> > (no changes since v1)
> > 
> >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> >   1 file changed, 13 insertions(+), 1 deletion(-)
> > 
> > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > index 93a9a5ac025..2e1d37848fc 100644
> > --- a/lib/efi_loader/efi_disk.c
> > +++ b/lib/efi_loader/efi_disk.c
> > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> >   efi_status_t efi_disks_register(void)
> >   {
> >   	struct udevice *dev;
> > +	struct uclass *uc;
> > 
> > -	uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > +	uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > +		/*
> > +		 * The virtio block-device hangs on sandbox when accessed since
> > +		 * there is nothing listening to the mailbox
> > +		 */
> > +		if (IS_ENABLED(CONFIG_SANDBOX)) {
> > +			struct blk_desc *desc = dev_get_uclass_plat(dev);
> > +
> > +			if (desc->uclass_id == UCLASS_VIRTIO)
> > +				continue;
> 
> We should avoid depending on the sandbox everywhere.
> 
> Please, fix the problem in the sandbox driver.
> 
> If you cannot fix it, run the tests involving virtio on QEMU instead of
> the sandbox.

This is an area we go back-and-forth on but, yes, IMHO, if we can't
easily provide a virtio device for sandbox, QEMU is right there and what
this is for, so I see sandbox as more useful as build rather than
runtime checking in this case.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH v2 06/39] bootstd: Create a function to reset USB

[Heinrich Schuchardt]

On 06.08.24 14:58, Simon Glass wrote:
> Set up a function for this, since it needs to be used from multiple test
> files.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>   test/boot/bootdev.c        | 19 ++++++-------------
>   test/boot/bootstd_common.c | 12 ++++++++++++
>   test/boot/bootstd_common.h |  8 ++++++++
>   3 files changed, 26 insertions(+), 13 deletions(-)
>
> diff --git a/test/boot/bootdev.c b/test/boot/bootdev.c
> index 1bf5929c396..de16a51956d 100644
> --- a/test/boot/bootdev.c
> +++ b/test/boot/bootdev.c
> @@ -16,13 +16,6 @@
>   #include <test/ut.h>
>   #include "bootstd_common.h"
>
> -/* Allow reseting the USB-started flag */
> -#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
> -extern bool usb_started;
> -#else
> -#include <usb.h>
> -#endif
> -
>   /* Check 'bootdev list' command */
>   static int bootdev_test_cmd_list(struct unit_test_state *uts)
>   {
> @@ -201,7 +194,7 @@ static int bootdev_test_order(struct unit_test_state *uts)
>   	test_set_skip_delays(true);
>
>   	/* Start up USB which gives us three additional bootdevs */
> -	usb_started = false;
> +	bootstd_reset_usb();
>   	ut_assertok(run_command("usb start", 0));
>
>   	/*
> @@ -317,7 +310,7 @@ static int bootdev_test_prio(struct unit_test_state *uts)
>   	test_set_eth_enable(false);
>
>   	/* Start up USB which gives us three additional bootdevs */
> -	usb_started = false;
> +	bootstd_reset_usb();
>   	ut_assertok(run_command("usb start", 0));
>
>   	ut_assertok(bootstd_test_drop_bootdev_order(uts));
> @@ -357,7 +350,7 @@ static int bootdev_test_hunter(struct unit_test_state *uts)
>   {
>   	struct bootstd_priv *std;
>
> -	usb_started = false;
> +	bootstd_reset_usb();
>   	test_set_skip_delays(true);
>
>   	/* get access to the used hunters */
> @@ -398,7 +391,7 @@ static int bootdev_test_cmd_hunt(struct unit_test_state *uts)
>   	struct bootstd_priv *std;
>
>   	test_set_skip_delays(true);
> -	usb_started = false;
> +	bootstd_reset_usb();
>
>   	/* get access to the used hunters */
>   	ut_assertok(bootstd_get_priv(&std));
> @@ -527,7 +520,7 @@ BOOTSTD_TEST(bootdev_test_bootable, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
>   /* Check hunting for bootdev of a particular priority */
>   static int bootdev_test_hunt_prio(struct unit_test_state *uts)
>   {
> -	usb_started = false;
> +	bootstd_reset_usb();
>   	test_set_skip_delays(true);
>
>   	console_record_reset_enable();
> @@ -556,7 +549,7 @@ static int bootdev_test_hunt_label(struct unit_test_state *uts)
>   	struct bootstd_priv *std;
>   	int mflags;
>
> -	usb_started = false;
> +	bootstd_reset_usb();
>
>   	/* get access to the used hunters */
>   	ut_assertok(bootstd_get_priv(&std));
> diff --git a/test/boot/bootstd_common.c b/test/boot/bootstd_common.c
> index e50539500a0..ff0aff4bbe7 100644
> --- a/test/boot/bootstd_common.c
> +++ b/test/boot/bootstd_common.c
> @@ -20,6 +20,13 @@
>   /* tracks whether bootstd_setup_for_tests() has been run yet */
>   bool vbe_setup_done;
>
> +/* Allow resetting the USB-started flag */
> +#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
> +extern bool usb_started;
> +#else
> +#include <usb.h>
> +#endif

Why can't you always include usb.h which defines usb_started?
Is there a bug in usb.h?

Best regards

Heinrich

> +
>   /* set up MMC for VBE tests */
>   int bootstd_setup_for_tests(void)
>   {
> @@ -88,6 +95,11 @@ int bootstd_test_check_mmc_hunter(struct unit_test_state *uts)
>   	return 0;
>   }
>
> +void bootstd_reset_usb(void)
> +{
> +	usb_started = false;
> +}
> +
>   int do_ut_bootstd(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
>   {
>   	struct unit_test *tests = UNIT_TEST_SUITE_START(bootstd_test);
> diff --git a/test/boot/bootstd_common.h b/test/boot/bootstd_common.h
> index 4a126e43ff4..e29036c897c 100644
> --- a/test/boot/bootstd_common.h
> +++ b/test/boot/bootstd_common.h
> @@ -53,4 +53,12 @@ int bootstd_setup_for_tests(void);
>    */
>   int bootstd_test_check_mmc_hunter(struct unit_test_state *uts);
>
> +/**
> + * bootstd_reset_usb() - Reset the USB subsystem
> + *
> + * Resets USB so that it can be started (and scanning) again. This is useful in
> + * tests which need to use USB.
> + */
> +void bootstd_reset_usb(void);
> +
>   #endif

Re: [PATCH v2 29/39] efi_loader: Shorten the app rules further

[Ilias Apalodimas]

On Tue, 6 Aug 2024 at 15:59, Simon Glass <sjg@chromium.org> wrote:
>
> Add a way to factor out the CFLAGS changes for each app, since they are
> all the same.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>  lib/efi_loader/Makefile | 15 +++++----------
>  1 file changed, 5 insertions(+), 10 deletions(-)
>
> diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
> index 660368b9d8f..00d18966f9e 100644
> --- a/lib/efi_loader/Makefile
> +++ b/lib/efi_loader/Makefile
> @@ -11,16 +11,6 @@ asflags-y += -I.
>  CFLAGS_efi_boottime.o += \
>    -DFW_VERSION="0x$(VERSION)" \
>    -DFW_PATCHLEVEL="0x$(PATCHLEVEL)"
> -CFLAGS_boothart.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_boothart.o := $(CFLAGS_NON_EFI)
> -CFLAGS_helloworld.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_helloworld.o := $(CFLAGS_NON_EFI)
> -CFLAGS_smbiosdump.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_smbiosdump.o := $(CFLAGS_NON_EFI)
> -CFLAGS_dtbdump.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_dtbdump.o := $(CFLAGS_NON_EFI)
> -CFLAGS_initrddump.o := $(CFLAGS_EFI) -Os -ffreestanding
> -CFLAGS_REMOVE_initrddump.o := $(CFLAGS_NON_EFI)
>
>  # These are the apps that are built
>  apps-$(CONFIG_RISCV) += boothart
> @@ -80,5 +70,10 @@ obj-$(CONFIG_EFI_ECPT) += efi_conformance.o
>  EFI_VAR_SEED_FILE := $(subst $\",,$(CONFIG_EFI_VAR_SEED_FILE))
>  $(obj)/efi_var_seed.o: $(srctree)/$(EFI_VAR_SEED_FILE)
>
> +# Set the C flags to add and remove for each app
> +$(foreach f,$(apps-y),\
> +       $(eval CFLAGS_$(f).o := $(CFLAGS_EFI) -Os -ffreestanding)\
> +       $(eval CFLAGS_REMOVE_$(f).o := $(CFLAGS_NON_EFI)))
> +
>  always += $(foreach f,$(apps-y),$(f).efi)
>  targets += $(foreach f,$(apps-y),$(f).o)
> --
> 2.34.1
>

Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>

Re: [PATCH v2 36/39] efi: Add a test app

[Simon Glass]

Hi Heinrich,

On Tue, 6 Aug 2024 at 19:47, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 06.08.24 14:58, Simon Glass wrote:
> > Add a simple app to use for testing. This is intended to do whatever it
> > needs to for testing purposes. For now it just prints a message and
> > exits boot services.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> >
> > (no changes since v1)
> >
> >   lib/efi_loader/Kconfig   | 10 ++++++
> >   lib/efi_loader/Makefile  |  1 +
> >   lib/efi_loader/testapp.c | 68 ++++++++++++++++++++++++++++++++++++++++
> >   3 files changed, 79 insertions(+)
> >   create mode 100644 lib/efi_loader/testapp.c
> >
> > diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
> > index ab2c1c44364..4de05c6f2d6 100644
> > --- a/lib/efi_loader/Kconfig
> > +++ b/lib/efi_loader/Kconfig
> > @@ -528,4 +528,14 @@ config BOOTEFI_HELLO_COMPILE
> >         No additional space will be required in the resulting U-Boot binary
> >         when this option is enabled.
> >
> > +config BOOTEFI_TESTAPP_COMPILE
> > +     bool "Compile an EFI test app for testing"
> > +     default y
> > +     help
> > +       This compiles an app designed for testing. It is packed into an image
> > +       by the test.py testing frame in the setup_efi_image() function.
> > +
> > +       No additional space will be required in the resulting U-Boot binary
> > +       when this option is enabled.
> > +
> >   endif
> > diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
> > index 00d18966f9e..87131ab911d 100644
> > --- a/lib/efi_loader/Makefile
> > +++ b/lib/efi_loader/Makefile
> > @@ -20,6 +20,7 @@ apps-$(CONFIG_EFI_LOAD_FILE2_INITRD) += initrddump
> >   ifeq ($(CONFIG_GENERATE_ACPI_TABLE),)
> >   apps-y += dtbdump
> >   endif
> > +apps-$(CONFIG_BOOTEFI_TESTAPP_COMPILE) += testapp
> >
> >   obj-$(CONFIG_CMD_BOOTEFI_HELLO) += helloworld_efi.o
> >   obj-$(CONFIG_EFI_BOOTMGR) += efi_bootmgr.o
> > diff --git a/lib/efi_loader/testapp.c b/lib/efi_loader/testapp.c
> > new file mode 100644
> > index 00000000000..feb444c92e9
> > --- /dev/null
> > +++ b/lib/efi_loader/testapp.c
> > @@ -0,0 +1,68 @@
> > +// SPDX-License-Identifier: GPL-2.0+
> > +/*
> > + * Hello world EFI application
> > + *
> > + * Copyright 2024 Google LLC
> > + * Written by Simon Glass <sjg@chromium.org>
> > + *
> > + * This test program is used to test the invocation of an EFI application.
> > + * It writes a few messages to the console and then exits boot services
> > + */
> > +
> > +#include <efi_api.h>
> > +
> > +static const efi_guid_t loaded_image_guid = EFI_LOADED_IMAGE_PROTOCOL_GUID;
> > +
> > +static struct efi_system_table *systable;
> > +static struct efi_boot_services *boottime;
> > +static struct efi_simple_text_output_protocol *con_out;
> > +
> > +/**
> > + * efi_main() - entry point of the EFI application.
> > + *
> > + * @handle:  handle of the loaded image
> > + * @systab:  system table
> > + * Return:   status code
> > + */
> > +efi_status_t EFIAPI efi_main(efi_handle_t handle,
> > +                          struct efi_system_table *systab)
> > +{
> > +     struct efi_loaded_image *loaded_image;
> > +     efi_status_t ret;
> > +     efi_uintn_t map_size;
> > +     efi_uintn_t map_key;
> > +     efi_uintn_t desc_size;
> > +     u32 desc_version;
> > +
> > +     systable = systab;
> > +     boottime = systable->boottime;
> > +     con_out = systable->con_out;
> > +
> > +     /* Get the loaded image protocol */
> > +     ret = boottime->open_protocol(handle, &loaded_image_guid,
> > +                                   (void **)&loaded_image, NULL, NULL,
> > +                                   EFI_OPEN_PROTOCOL_GET_PROTOCOL);
> > +     if (ret != EFI_SUCCESS) {
> > +             con_out->output_string
> > +                     (con_out, u"Cannot open loaded image protocol\r\n");
> > +             goto out;
> > +     }
> > +
> > +     /* UEFI requires CR LF */
> > +     con_out->output_string(con_out, u"U-Boot test app for EFI_LOADER\r\n");
> > +
> > +out:
> > +     map_size = 0;
> > +     ret = boottime->get_memory_map(&map_size, NULL, &map_key, &desc_size,
> > +                                    &desc_version);
> > +     con_out->output_string(con_out, u"Exiting boot sevices\n");
> > +
> > +     /* exit boot services so that this part of U-Boot can be tested */
> > +     boottime->exit_boot_services(handle, map_key);
> > +
> > +     /* now exit for real */
> > +     ret = boottime->exit(handle, ret, 0, NULL);
>
> Please, have a look at chapter 7.4.6,
> "EFI_BOOT_SERVICES.ExitBootServices" of the UEFI specification.
>
> After ExitBootServices() you cannot return anywhere. Boot services are
> not available anymore. You can only invoke the UEFI runtime services
> which include ResetSystem().

Yes, understood, but this is a test, so returning is needed so that
the test can check the output is correct.

Regards,
Simon

Re: [PATCH v2 06/39] bootstd: Create a function to reset USB

[Simon Glass]

Hi Heinrich,

On Tue, 6 Aug 2024 at 19:56, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 06.08.24 14:58, Simon Glass wrote:
> > Set up a function for this, since it needs to be used from multiple test
> > files.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> >
> > (no changes since v1)
> >
> >   test/boot/bootdev.c        | 19 ++++++-------------
> >   test/boot/bootstd_common.c | 12 ++++++++++++
> >   test/boot/bootstd_common.h |  8 ++++++++
> >   3 files changed, 26 insertions(+), 13 deletions(-)
> >
> > diff --git a/test/boot/bootdev.c b/test/boot/bootdev.c
> > index 1bf5929c396..de16a51956d 100644
> > --- a/test/boot/bootdev.c
> > +++ b/test/boot/bootdev.c
> > @@ -16,13 +16,6 @@
> >   #include <test/ut.h>
> >   #include "bootstd_common.h"
> >
> > -/* Allow reseting the USB-started flag */
> > -#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
> > -extern bool usb_started;
> > -#else
> > -#include <usb.h>
> > -#endif
> > -
> >   /* Check 'bootdev list' command */
> >   static int bootdev_test_cmd_list(struct unit_test_state *uts)
> >   {
> > @@ -201,7 +194,7 @@ static int bootdev_test_order(struct unit_test_state *uts)
> >       test_set_skip_delays(true);
> >
> >       /* Start up USB which gives us three additional bootdevs */
> > -     usb_started = false;
> > +     bootstd_reset_usb();
> >       ut_assertok(run_command("usb start", 0));
> >
> >       /*
> > @@ -317,7 +310,7 @@ static int bootdev_test_prio(struct unit_test_state *uts)
> >       test_set_eth_enable(false);
> >
> >       /* Start up USB which gives us three additional bootdevs */
> > -     usb_started = false;
> > +     bootstd_reset_usb();
> >       ut_assertok(run_command("usb start", 0));
> >
> >       ut_assertok(bootstd_test_drop_bootdev_order(uts));
> > @@ -357,7 +350,7 @@ static int bootdev_test_hunter(struct unit_test_state *uts)
> >   {
> >       struct bootstd_priv *std;
> >
> > -     usb_started = false;
> > +     bootstd_reset_usb();
> >       test_set_skip_delays(true);
> >
> >       /* get access to the used hunters */
> > @@ -398,7 +391,7 @@ static int bootdev_test_cmd_hunt(struct unit_test_state *uts)
> >       struct bootstd_priv *std;
> >
> >       test_set_skip_delays(true);
> > -     usb_started = false;
> > +     bootstd_reset_usb();
> >
> >       /* get access to the used hunters */
> >       ut_assertok(bootstd_get_priv(&std));
> > @@ -527,7 +520,7 @@ BOOTSTD_TEST(bootdev_test_bootable, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
> >   /* Check hunting for bootdev of a particular priority */
> >   static int bootdev_test_hunt_prio(struct unit_test_state *uts)
> >   {
> > -     usb_started = false;
> > +     bootstd_reset_usb();
> >       test_set_skip_delays(true);
> >
> >       console_record_reset_enable();
> > @@ -556,7 +549,7 @@ static int bootdev_test_hunt_label(struct unit_test_state *uts)
> >       struct bootstd_priv *std;
> >       int mflags;
> >
> > -     usb_started = false;
> > +     bootstd_reset_usb();
> >
> >       /* get access to the used hunters */
> >       ut_assertok(bootstd_get_priv(&std));
> > diff --git a/test/boot/bootstd_common.c b/test/boot/bootstd_common.c
> > index e50539500a0..ff0aff4bbe7 100644
> > --- a/test/boot/bootstd_common.c
> > +++ b/test/boot/bootstd_common.c
> > @@ -20,6 +20,13 @@
> >   /* tracks whether bootstd_setup_for_tests() has been run yet */
> >   bool vbe_setup_done;
> >
> > +/* Allow resetting the USB-started flag */
> > +#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
> > +extern bool usb_started;
> > +#else
> > +#include <usb.h>
> > +#endif
>
> Why can't you always include usb.h which defines usb_started?
> Is there a bug in usb.h?

I can, but even in tests it is not great to access globals. Having it
in a function lets us call out these particular settings better, which
has aided debugging, at least for me. These tests are special in that
they are not DM tests, but do stop USB.

Regards,
Simon

Re: [PATCH v2 21/39] test: mbr: Unmap the buffers after use

[Ilias Apalodimas]

Hi Simon

On Tue, 6 Aug 2024 at 15:59, Simon Glass <sjg@chromium.org> wrote:
>
> This tests maps some local variables into sandbox's address space. Make
> sure to unmap them afterwards.
>
> Note that the normal approach with sandbox is to use a fixed memory
> address in the RAM, to avoid needing to create a map for transient local
> variables.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>  test/cmd/mbr.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
> index 1a35ba3351c..8a4d61b2928 100644
> --- a/test/cmd/mbr.c
> +++ b/test/cmd/mbr.c
> @@ -461,6 +461,9 @@ static int mbr_test_run(struct unit_test_state *uts)
>                         "5P EBR+0x%04X: expected %#02X, actual: %#02X\n",
>                         ebr_cmp_start + i, ebr_parts_ref_p5[i], rbuf[ebr_cmp_start + i]);
>         }
> +       unmap_sysmem(mbr_wbuf);
> +       unmap_sysmem(ebr_wbuf);
> +       unmap_sysmem(rbuf);

Seems like a Fixes: tag is needed?

Thanks
/Ilias
>
>         return 0;
>  }
> --
> 2.34.1
>

Re: [PATCH v2 22/39] test: mbr: Use a constant for the block size

[Ilias Apalodimas]

Hi Simon

On Tue, 6 Aug 2024 at 15:59, Simon Glass <sjg@chromium.org> wrote:
>
> It isn't that important to factor out constants in tests, but in this
> case we have 0x200 and 512 used. The commands don't use the constant
> as they use a block count ('1'). It doesn't create more code to use a
> constant, so create one.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>  test/cmd/mbr.c | 37 ++++++++++++++++++++-----------------
>  1 file changed, 20 insertions(+), 17 deletions(-)
>
> diff --git a/test/cmd/mbr.c b/test/cmd/mbr.c
> index 8a4d61b2928..bab92c3ba00 100644
> --- a/test/cmd/mbr.c
> +++ b/test/cmd/mbr.c
> @@ -18,6 +18,9 @@
>  #include <test/ut.h>
>
>  DECLARE_GLOBAL_DATA_PTR;
> +
> +#define BLKSZ  0x200   /* block size */

We already have definitions for that. I prefer SZ_512

Thanks
/Ilias
> +
>  /*
>   * Requirements for running test manually:
>   * mmc6.img - File size needs to be at least 12 MiB
> @@ -228,7 +231,7 @@ static unsigned build_mbr_parts(char *buf, size_t buf_size, unsigned num_parts)
>  static int mbr_test_run(struct unit_test_state *uts)
>  {
>         struct blk_desc *mmc_dev_desc;
> -       unsigned char mbr_wbuf[512], ebr_wbuf[512], rbuf[512];
> +       unsigned char mbr_wbuf[BLKSZ], ebr_wbuf[BLKSZ], rbuf[BLKSZ];
>         char mbr_parts_buf[256];
>         ulong mbr_wa, ebr_wa, ra, ebr_blk, mbr_parts_max;
>         struct udevice *dev;
> @@ -257,7 +260,7 @@ static int mbr_test_run(struct unit_test_state *uts)
>         mbr_wa = map_to_sysmem(mbr_wbuf);
>         ebr_wa = map_to_sysmem(ebr_wbuf);
>         ra = map_to_sysmem(rbuf);
> -       ebr_blk = (ulong)0xb00000 / 0x200;
> +       ebr_blk = (ulong)0xb00000 / BLKSZ;
>
>         /* Make sure mmc6 exists */
>         ut_asserteq(6, blk_get_device_by_str("mmc", "6", &mmc_dev_desc));
> @@ -269,7 +272,7 @@ static int mbr_test_run(struct unit_test_state *uts)
>
>         /* Make sure mmc6 is 12+ MiB in size */
>         ut_assertok(run_commandf("mmc read %lx %lx 1", ra,
> -                                (ulong)0xbffe00 / 0x200));
> +                                (ulong)0xbffe00 / BLKSZ));
>
>         /* Test one MBR partition */
>         init_write_buffers(mbr_wbuf, sizeof(mbr_wbuf), ebr_wbuf, sizeof(ebr_wbuf), __LINE__);
> @@ -277,11 +280,11 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
> -       ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
>         ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(console_record_reset_enable());
>         ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
>         ut_assertok(run_commandf("mbr write mmc 6"));
> @@ -290,7 +293,7 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assert_nextline("MBR: verify success!");
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(ut_check_console_end(uts));
>         /*
>         000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
> @@ -313,11 +316,11 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
> -       ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
>         ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(console_record_reset_enable());
>         ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
>         ut_assertok(run_commandf("mbr write mmc 6"));
> @@ -326,7 +329,7 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assert_nextline("MBR: verify success!");
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(ut_check_console_end(uts));
>         /*
>         000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
> @@ -349,11 +352,11 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
> -       ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
>         ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(console_record_reset_enable());
>         ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
>         ut_assertok(run_commandf("mbr write mmc 6"));
> @@ -362,7 +365,7 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assert_nextline("MBR: verify success!");
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(ut_check_console_end(uts));
>         /*
>         000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
> @@ -385,11 +388,11 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
> -       ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
>         ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(console_record_reset_enable());
>         ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
>         ut_assertok(run_commandf("mbr write mmc 6"));
> @@ -398,7 +401,7 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assert_nextline("MBR: verify success!");
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(ut_check_console_end(uts));
>         /*
>         000001b0  00 00 00 00 00 00 00 00  78 56 34 12 00 00 80 05  |........xV4.....|
> @@ -421,11 +424,11 @@ static int mbr_test_run(struct unit_test_state *uts)
>         ut_assertok(run_commandf("write mmc 6:0 %lx 0 1", mbr_wa));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx 0 1", ra));
> -       ut_assertok(memcmp(mbr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(mbr_wbuf, rbuf, BLKSZ));
>         ut_assertok(run_commandf("write mmc 6:0 %lx %lx 1", ebr_wa, ebr_blk));
>         memset(rbuf, 0, sizeof(rbuf));
>         ut_assertok(run_commandf("read mmc 6:0 %lx %lx 1", ra, ebr_blk));
> -       ut_assertok(memcmp(ebr_wbuf, rbuf, 512));
> +       ut_assertok(memcmp(ebr_wbuf, rbuf, BLKSZ));
>         ut_assertok(console_record_reset_enable());
>         ut_assertf(0 == run_commandf(mbr_parts_buf), "Invalid partitions string: %s\n", mbr_parts_buf);
>         ut_assertf(0 == run_commandf("mbr write mmc 6"), "Invalid partitions string: %s\n", mbr_parts_buf);
> --
> 2.34.1
>

Re: [PATCH v2 33/39] efi: Use the same filename for all sandbox builds

[Ilias Apalodimas]

Hi Simon,

On Tue, 6 Aug 2024 at 15:59, Simon Glass <sjg@chromium.org> wrote:
>
> Sandbox is not a real architecture, but within U-Boot it is real enough.
> We should not need to pretend it is x86 or ARM anywhere in the code.
>
> Also we want to be able to locate the sandbox app using a single
> filename, 'bootsbox.efi', to avoid needing tests to produce different
> files on each host architecture.
>
> Drop the confusing use of host architecture and just let sandbox be
> sandbox.
>
> Fixes: 3a0654ecd0d ("efi_loader: correctly identify binary name")
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>  include/efi_default_filename.h | 25 ++-----------------------
>  1 file changed, 2 insertions(+), 23 deletions(-)
>
> diff --git a/include/efi_default_filename.h b/include/efi_default_filename.h
> index 77932984b55..6806c0daa88 100644
> --- a/include/efi_default_filename.h
> +++ b/include/efi_default_filename.h
> @@ -5,7 +5,6 @@
>   * file name is defined in this include.
>   *
>   * Copyright (c) 2022, Heinrich Schuchardt <xypron.glpk@gmx.de>
> - * Copyright (c) 2022, Linaro Limited

Any reasons why you removed this?

Thanks
/Ilias
>   */
>
>  #ifndef _EFI_DEFAULT_FILENAME_H
> @@ -16,26 +15,8 @@
>  #undef BOOTEFI_NAME
>
>  #ifdef CONFIG_SANDBOX
> -
> -#if HOST_ARCH == HOST_ARCH_X86_64
> -#define BOOTEFI_NAME "BOOTX64.EFI"
> -#elif HOST_ARCH == HOST_ARCH_X86
> -#define BOOTEFI_NAME "BOOTIA32.EFI"
> -#elif HOST_ARCH == HOST_ARCH_AARCH64
> -#define BOOTEFI_NAME "BOOTAA64.EFI"
> -#elif HOST_ARCH == HOST_ARCH_ARM
> -#define BOOTEFI_NAME "BOOTARM.EFI"
> -#elif HOST_ARCH == HOST_ARCH_RISCV32
> -#define BOOTEFI_NAME "BOOTRISCV32.EFI"
> -#elif HOST_ARCH == HOST_ARCH_RISCV64
> -#define BOOTEFI_NAME "BOOTRISCV64.EFI"
> -#else
> -#error Unsupported UEFI architecture
> -#endif
> -
> -#else
> -
> -#if defined(CONFIG_ARM64)
> +#define BOOTEFI_NAME "BOOTSBOX.EFI"
> +#elif defined(CONFIG_ARM64)
>  #define BOOTEFI_NAME "BOOTAA64.EFI"
>  #elif defined(CONFIG_ARM)
>  #define BOOTEFI_NAME "BOOTARM.EFI"
> @@ -52,5 +33,3 @@
>  #endif
>
>  #endif
> -
> -#endif
> --
> 2.34.1
>

Re: [PATCH v2 19/39] read: Use map_sysmem() with buffers in the read command

[Ilias Apalodimas]

On Tue, 6 Aug 2024 at 15:59, Simon Glass <sjg@chromium.org> wrote:
>
> The current implementation casts an address to a pointer. Make it more
> sandbox-friendly by using map_sysmem().

Isn't the code already using map_sysmem? All I see is some fixes for
unmapping it and a rename

>
> Rename the variable to 'ptr' since it is a pointer, not an address.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> (no changes since v1)
>
>  cmd/read.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/cmd/read.c b/cmd/read.c
> index af54bd17654..8e21f004423 100644
> --- a/cmd/read.c
> +++ b/cmd/read.c
> @@ -20,7 +20,7 @@ do_rw(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
>         struct disk_partition part_info;
>         ulong offset, limit;
>         uint blk, cnt, res;
> -       void *addr;
> +       void *ptr;
>         int part;
>
>         if (argc != 6) {
> @@ -33,7 +33,7 @@ do_rw(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
>         if (part < 0)
>                 return 1;
>
> -       addr = map_sysmem(hextoul(argv[3], NULL), 0);
> +       ptr = map_sysmem(hextoul(argv[3], NULL), 0);
>         blk = hextoul(argv[4], NULL);
>         cnt = hextoul(argv[5], NULL);
>
> @@ -48,13 +48,15 @@ do_rw(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
>
>         if (cnt + blk > limit) {
>                 printf("%s out of range\n", cmdtp->name);
> +               unmap_sysmem(ptr);
>                 return 1;
>         }
>
>         if (IS_ENABLED(CONFIG_CMD_WRITE) && !strcmp(cmdtp->name, "write"))
> -               res = blk_dwrite(dev_desc, offset + blk, cnt, addr);
> +               res = blk_dwrite(dev_desc, offset + blk, cnt, ptr);
>         else
> -               res = blk_dread(dev_desc, offset + blk, cnt, addr);
> +               res = blk_dread(dev_desc, offset + blk, cnt, ptr);
> +       unmap_sysmem(ptr);
>
>         if (res != cnt) {
>                 printf("%s error\n", cmdtp->name);
> --
> 2.34.1
>

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Simon Glass]

Hi Heinrick, Tom,

On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > On 06.08.24 14:58, Simon Glass wrote:
> > > While sandbox supports virtio it cannot support actually using the block
> > > devices to read files, since there is nothing on the other end of the
> > > 'virtqueue'.
> > >
> > > A recent change makes EFI probe all block devices, whether used or not.
> > > This is apparently required by EFI, although it violates U-Boot's
> > > lazy-init principle.
> > >
> > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > >
> > > So for now just add a special case to work around this.
> > >
> > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > ---
> > >
> > > (no changes since v1)
> > >
> > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > index 93a9a5ac025..2e1d37848fc 100644
> > > --- a/lib/efi_loader/efi_disk.c
> > > +++ b/lib/efi_loader/efi_disk.c
> > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > >   efi_status_t efi_disks_register(void)
> > >   {
> > >     struct udevice *dev;
> > > +   struct uclass *uc;
> > >
> > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > +           /*
> > > +            * The virtio block-device hangs on sandbox when accessed since
> > > +            * there is nothing listening to the mailbox
> > > +            */
> > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > +
> > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > +                           continue;
> >
> > We should avoid depending on the sandbox everywhere.
> >
> > Please, fix the problem in the sandbox driver.
> >
> > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > the sandbox.

Which test? All of the EFI tests fail due to this problem. The test
actually has nothing to do with virtio, it is just that EFI goes and
probes every single block device, since [1].

>
> This is an area we go back-and-forth on but, yes, IMHO, if we can't
> easily provide a virtio device for sandbox, QEMU is right there and what
> this is for, so I see sandbox as more useful as build rather than
> runtime checking in this case.

The best solution would be to implement a simple emulator, like we do
in other places for sandbox. At present virtio_sandbox_notify() is
empty.

I don't mind working on that, but would like to get a temporary
solution here so this test can land.

Talking about virtio for QEMU is missing the point of this test, which
is after all a test of booting an EFI app. I do wish more people would
see the value in these unit tests. There is a talk at [2] which shows
how emulators are used in Zephyr.

Regards,
Simon

[1] d5391bf02b9 ("efi_loader: ensure all block devices are probed")
[2] https://www.youtube.com/watch?v=usXCAXR2G_c

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 3866 bytes --]

On Thu, Aug 08, 2024 at 12:44:05PM -0600, Simon Glass wrote:
> Hi Heinrick, Tom,
> 
> On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > > On 06.08.24 14:58, Simon Glass wrote:
> > > > While sandbox supports virtio it cannot support actually using the block
> > > > devices to read files, since there is nothing on the other end of the
> > > > 'virtqueue'.
> > > >
> > > > A recent change makes EFI probe all block devices, whether used or not.
> > > > This is apparently required by EFI, although it violates U-Boot's
> > > > lazy-init principle.
> > > >
> > > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > > >
> > > > So for now just add a special case to work around this.
> > > >
> > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > ---
> > > >
> > > > (no changes since v1)
> > > >
> > > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > > index 93a9a5ac025..2e1d37848fc 100644
> > > > --- a/lib/efi_loader/efi_disk.c
> > > > +++ b/lib/efi_loader/efi_disk.c
> > > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > > >   efi_status_t efi_disks_register(void)
> > > >   {
> > > >     struct udevice *dev;
> > > > +   struct uclass *uc;
> > > >
> > > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > > +           /*
> > > > +            * The virtio block-device hangs on sandbox when accessed since
> > > > +            * there is nothing listening to the mailbox
> > > > +            */
> > > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > > +
> > > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > > +                           continue;
> > >
> > > We should avoid depending on the sandbox everywhere.
> > >
> > > Please, fix the problem in the sandbox driver.
> > >
> > > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > > the sandbox.
> 
> Which test? All of the EFI tests fail due to this problem. The test
> actually has nothing to do with virtio, it is just that EFI goes and
> probes every single block device, since [1].

Aren't we running "the tests" on other platforms such as QEMU today?

> > This is an area we go back-and-forth on but, yes, IMHO, if we can't
> > easily provide a virtio device for sandbox, QEMU is right there and what
> > this is for, so I see sandbox as more useful as build rather than
> > runtime checking in this case.
> 
> The best solution would be to implement a simple emulator, like we do
> in other places for sandbox. At present virtio_sandbox_notify() is
> empty.
> 
> I don't mind working on that, but would like to get a temporary
> solution here so this test can land.
> 
> Talking about virtio for QEMU is missing the point of this test, which
> is after all a test of booting an EFI app. I do wish more people would
> see the value in these unit tests. There is a talk at [2] which shows
> how emulators are used in Zephyr.

So that talk is interesting, yes. So, yes, implement the bus driver for
sandbox for virtio, and until then we shouldn't have the tests running
on sandbox? Or am I still missing something?

But I also still say that given that we as a project are more resource
constrained than Zephyr, for things that are QEMU-centric, there's
already a wealth of information on debugging QEMU since it too is
software. There's only so many hours in the day after all.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH v2 06/39] bootstd: Create a function to reset USB

[Heinrich Schuchardt]

On 07.08.24 16:36, Simon Glass wrote:
> Hi Heinrich,
>
> On Tue, 6 Aug 2024 at 19:56, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> On 06.08.24 14:58, Simon Glass wrote:
>>> Set up a function for this, since it needs to be used from multiple test
>>> files.
>>>
>>> Signed-off-by: Simon Glass <sjg@chromium.org>
>>> ---
>>>
>>> (no changes since v1)
>>>
>>>    test/boot/bootdev.c        | 19 ++++++-------------
>>>    test/boot/bootstd_common.c | 12 ++++++++++++
>>>    test/boot/bootstd_common.h |  8 ++++++++
>>>    3 files changed, 26 insertions(+), 13 deletions(-)
>>>
>>> diff --git a/test/boot/bootdev.c b/test/boot/bootdev.c
>>> index 1bf5929c396..de16a51956d 100644
>>> --- a/test/boot/bootdev.c
>>> +++ b/test/boot/bootdev.c
>>> @@ -16,13 +16,6 @@
>>>    #include <test/ut.h>
>>>    #include "bootstd_common.h"
>>>
>>> -/* Allow reseting the USB-started flag */
>>> -#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
>>> -extern bool usb_started;
>>> -#else
>>> -#include <usb.h>
>>> -#endif
>>> -
>>>    /* Check 'bootdev list' command */
>>>    static int bootdev_test_cmd_list(struct unit_test_state *uts)
>>>    {
>>> @@ -201,7 +194,7 @@ static int bootdev_test_order(struct unit_test_state *uts)
>>>        test_set_skip_delays(true);
>>>
>>>        /* Start up USB which gives us three additional bootdevs */
>>> -     usb_started = false;
>>> +     bootstd_reset_usb();
>>>        ut_assertok(run_command("usb start", 0));
>>>
>>>        /*
>>> @@ -317,7 +310,7 @@ static int bootdev_test_prio(struct unit_test_state *uts)
>>>        test_set_eth_enable(false);
>>>
>>>        /* Start up USB which gives us three additional bootdevs */
>>> -     usb_started = false;
>>> +     bootstd_reset_usb();
>>>        ut_assertok(run_command("usb start", 0));
>>>
>>>        ut_assertok(bootstd_test_drop_bootdev_order(uts));
>>> @@ -357,7 +350,7 @@ static int bootdev_test_hunter(struct unit_test_state *uts)
>>>    {
>>>        struct bootstd_priv *std;
>>>
>>> -     usb_started = false;
>>> +     bootstd_reset_usb();
>>>        test_set_skip_delays(true);
>>>
>>>        /* get access to the used hunters */
>>> @@ -398,7 +391,7 @@ static int bootdev_test_cmd_hunt(struct unit_test_state *uts)
>>>        struct bootstd_priv *std;
>>>
>>>        test_set_skip_delays(true);
>>> -     usb_started = false;
>>> +     bootstd_reset_usb();
>>>
>>>        /* get access to the used hunters */
>>>        ut_assertok(bootstd_get_priv(&std));
>>> @@ -527,7 +520,7 @@ BOOTSTD_TEST(bootdev_test_bootable, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
>>>    /* Check hunting for bootdev of a particular priority */
>>>    static int bootdev_test_hunt_prio(struct unit_test_state *uts)
>>>    {
>>> -     usb_started = false;
>>> +     bootstd_reset_usb();
>>>        test_set_skip_delays(true);
>>>
>>>        console_record_reset_enable();
>>> @@ -556,7 +549,7 @@ static int bootdev_test_hunt_label(struct unit_test_state *uts)
>>>        struct bootstd_priv *std;
>>>        int mflags;
>>>
>>> -     usb_started = false;
>>> +     bootstd_reset_usb();
>>>
>>>        /* get access to the used hunters */
>>>        ut_assertok(bootstd_get_priv(&std));
>>> diff --git a/test/boot/bootstd_common.c b/test/boot/bootstd_common.c
>>> index e50539500a0..ff0aff4bbe7 100644
>>> --- a/test/boot/bootstd_common.c
>>> +++ b/test/boot/bootstd_common.c
>>> @@ -20,6 +20,13 @@
>>>    /* tracks whether bootstd_setup_for_tests() has been run yet */
>>>    bool vbe_setup_done;
>>>
>>> +/* Allow resetting the USB-started flag */
>>> +#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
>>> +extern bool usb_started;
>>> +#else
>>> +#include <usb.h>
>>> +#endif
>>
>> Why can't you always include usb.h which defines usb_started?
>> Is there a bug in usb.h?
>
> I can, but even in tests it is not great to access globals. Having it
> in a function lets us call out these particular settings better, which
> has aided debugging, at least for me. These tests are special in that
> they are not DM tests, but do stop USB.

I was just wondering if we can can rid of the #if here. That should not
stop you from creating a function to access the variable.

Best regards

Heinrich

Re: [PATCH v2 36/39] efi: Add a test app

[Heinrich Schuchardt]

On 07.08.24 16:36, Simon Glass wrote:
> Hi Heinrich,
>
> On Tue, 6 Aug 2024 at 19:47, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> On 06.08.24 14:58, Simon Glass wrote:
>>> Add a simple app to use for testing. This is intended to do whatever it
>>> needs to for testing purposes. For now it just prints a message and
>>> exits boot services.
>>>
>>> Signed-off-by: Simon Glass <sjg@chromium.org>
>>> ---
>>>
>>> (no changes since v1)
>>>
>>>    lib/efi_loader/Kconfig   | 10 ++++++
>>>    lib/efi_loader/Makefile  |  1 +
>>>    lib/efi_loader/testapp.c | 68 ++++++++++++++++++++++++++++++++++++++++
>>>    3 files changed, 79 insertions(+)
>>>    create mode 100644 lib/efi_loader/testapp.c
>>>
>>> diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
>>> index ab2c1c44364..4de05c6f2d6 100644
>>> --- a/lib/efi_loader/Kconfig
>>> +++ b/lib/efi_loader/Kconfig
>>> @@ -528,4 +528,14 @@ config BOOTEFI_HELLO_COMPILE
>>>          No additional space will be required in the resulting U-Boot binary
>>>          when this option is enabled.
>>>
>>> +config BOOTEFI_TESTAPP_COMPILE
>>> +     bool "Compile an EFI test app for testing"
>>> +     default y
>>> +     help
>>> +       This compiles an app designed for testing. It is packed into an image
>>> +       by the test.py testing frame in the setup_efi_image() function.
>>> +
>>> +       No additional space will be required in the resulting U-Boot binary
>>> +       when this option is enabled.
>>> +
>>>    endif
>>> diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
>>> index 00d18966f9e..87131ab911d 100644
>>> --- a/lib/efi_loader/Makefile
>>> +++ b/lib/efi_loader/Makefile
>>> @@ -20,6 +20,7 @@ apps-$(CONFIG_EFI_LOAD_FILE2_INITRD) += initrddump
>>>    ifeq ($(CONFIG_GENERATE_ACPI_TABLE),)
>>>    apps-y += dtbdump
>>>    endif
>>> +apps-$(CONFIG_BOOTEFI_TESTAPP_COMPILE) += testapp
>>>
>>>    obj-$(CONFIG_CMD_BOOTEFI_HELLO) += helloworld_efi.o
>>>    obj-$(CONFIG_EFI_BOOTMGR) += efi_bootmgr.o
>>> diff --git a/lib/efi_loader/testapp.c b/lib/efi_loader/testapp.c
>>> new file mode 100644
>>> index 00000000000..feb444c92e9
>>> --- /dev/null
>>> +++ b/lib/efi_loader/testapp.c
>>> @@ -0,0 +1,68 @@
>>> +// SPDX-License-Identifier: GPL-2.0+
>>> +/*
>>> + * Hello world EFI application
>>> + *
>>> + * Copyright 2024 Google LLC
>>> + * Written by Simon Glass <sjg@chromium.org>
>>> + *
>>> + * This test program is used to test the invocation of an EFI application.
>>> + * It writes a few messages to the console and then exits boot services
>>> + */
>>> +
>>> +#include <efi_api.h>
>>> +
>>> +static const efi_guid_t loaded_image_guid = EFI_LOADED_IMAGE_PROTOCOL_GUID;
>>> +
>>> +static struct efi_system_table *systable;
>>> +static struct efi_boot_services *boottime;
>>> +static struct efi_simple_text_output_protocol *con_out;
>>> +
>>> +/**
>>> + * efi_main() - entry point of the EFI application.
>>> + *
>>> + * @handle:  handle of the loaded image
>>> + * @systab:  system table
>>> + * Return:   status code
>>> + */
>>> +efi_status_t EFIAPI efi_main(efi_handle_t handle,
>>> +                          struct efi_system_table *systab)
>>> +{
>>> +     struct efi_loaded_image *loaded_image;
>>> +     efi_status_t ret;
>>> +     efi_uintn_t map_size;
>>> +     efi_uintn_t map_key;
>>> +     efi_uintn_t desc_size;
>>> +     u32 desc_version;
>>> +
>>> +     systable = systab;
>>> +     boottime = systable->boottime;
>>> +     con_out = systable->con_out;
>>> +
>>> +     /* Get the loaded image protocol */
>>> +     ret = boottime->open_protocol(handle, &loaded_image_guid,
>>> +                                   (void **)&loaded_image, NULL, NULL,
>>> +                                   EFI_OPEN_PROTOCOL_GET_PROTOCOL);
>>> +     if (ret != EFI_SUCCESS) {
>>> +             con_out->output_string
>>> +                     (con_out, u"Cannot open loaded image protocol\r\n");
>>> +             goto out;
>>> +     }
>>> +
>>> +     /* UEFI requires CR LF */
>>> +     con_out->output_string(con_out, u"U-Boot test app for EFI_LOADER\r\n");
>>> +
>>> +out:
>>> +     map_size = 0;
>>> +     ret = boottime->get_memory_map(&map_size, NULL, &map_key, &desc_size,
>>> +                                    &desc_version);
>>> +     con_out->output_string(con_out, u"Exiting boot sevices\n");
>>> +
>>> +     /* exit boot services so that this part of U-Boot can be tested */
>>> +     boottime->exit_boot_services(handle, map_key);
>>> +
>>> +     /* now exit for real */
>>> +     ret = boottime->exit(handle, ret, 0, NULL);
>>
>> Please, have a look at chapter 7.4.6,
>> "EFI_BOOT_SERVICES.ExitBootServices" of the UEFI specification.
>>
>> After ExitBootServices() you cannot return anywhere. Boot services are
>> not available anymore. You can only invoke the UEFI runtime services
>> which include ResetSystem().
>
> Yes, understood, but this is a test, so returning is needed so that
> the test can check the output is correct.
>

ExitBootServices() calls dm_remove_devices_flags(DM_REMOVE_ACTIVE_ALL).
You cannot print anything afterwards as all devices are gone.

But you can check if a reboot occurs. This is enough to verify that a
ResetSystem() call placed after ExitBootServices() was executed.

We already have

* lib/efi_selftest/efi_selftest_exitbootservices.c
* lib/efi_selftest/efi_selftest_variables_runtime.c
* lib/efi_selftest/efi_selftest_set_virtual_address_map.c

which test different aspects of ExitBootServices() on the sandbox.

There is no need to duplicate these.

Best regards

Heinrich

Re: [PATCH v2 36/39] efi: Add a test app

[Simon Glass]

Hi Heinrich,

On Thu, 8 Aug 2024 at 15:17, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 07.08.24 16:36, Simon Glass wrote:
> > Hi Heinrich,
> >
> > On Tue, 6 Aug 2024 at 19:47, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> On 06.08.24 14:58, Simon Glass wrote:
> >>> Add a simple app to use for testing. This is intended to do whatever it
> >>> needs to for testing purposes. For now it just prints a message and
> >>> exits boot services.
> >>>
> >>> Signed-off-by: Simon Glass <sjg@chromium.org>
> >>> ---
> >>>
> >>> (no changes since v1)
> >>>
> >>>    lib/efi_loader/Kconfig   | 10 ++++++
> >>>    lib/efi_loader/Makefile  |  1 +
> >>>    lib/efi_loader/testapp.c | 68 ++++++++++++++++++++++++++++++++++++++++
> >>>    3 files changed, 79 insertions(+)
> >>>    create mode 100644 lib/efi_loader/testapp.c
> >>>
> >>> diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
> >>> index ab2c1c44364..4de05c6f2d6 100644
> >>> --- a/lib/efi_loader/Kconfig
> >>> +++ b/lib/efi_loader/Kconfig
> >>> @@ -528,4 +528,14 @@ config BOOTEFI_HELLO_COMPILE
> >>>          No additional space will be required in the resulting U-Boot binary
> >>>          when this option is enabled.
> >>>
> >>> +config BOOTEFI_TESTAPP_COMPILE
> >>> +     bool "Compile an EFI test app for testing"
> >>> +     default y
> >>> +     help
> >>> +       This compiles an app designed for testing. It is packed into an image
> >>> +       by the test.py testing frame in the setup_efi_image() function.
> >>> +
> >>> +       No additional space will be required in the resulting U-Boot binary
> >>> +       when this option is enabled.
> >>> +
> >>>    endif
> >>> diff --git a/lib/efi_loader/Makefile b/lib/efi_loader/Makefile
> >>> index 00d18966f9e..87131ab911d 100644
> >>> --- a/lib/efi_loader/Makefile
> >>> +++ b/lib/efi_loader/Makefile
> >>> @@ -20,6 +20,7 @@ apps-$(CONFIG_EFI_LOAD_FILE2_INITRD) += initrddump
> >>>    ifeq ($(CONFIG_GENERATE_ACPI_TABLE),)
> >>>    apps-y += dtbdump
> >>>    endif
> >>> +apps-$(CONFIG_BOOTEFI_TESTAPP_COMPILE) += testapp
> >>>
> >>>    obj-$(CONFIG_CMD_BOOTEFI_HELLO) += helloworld_efi.o
> >>>    obj-$(CONFIG_EFI_BOOTMGR) += efi_bootmgr.o
> >>> diff --git a/lib/efi_loader/testapp.c b/lib/efi_loader/testapp.c
> >>> new file mode 100644
> >>> index 00000000000..feb444c92e9
> >>> --- /dev/null
> >>> +++ b/lib/efi_loader/testapp.c
> >>> @@ -0,0 +1,68 @@
> >>> +// SPDX-License-Identifier: GPL-2.0+
> >>> +/*
> >>> + * Hello world EFI application
> >>> + *
> >>> + * Copyright 2024 Google LLC
> >>> + * Written by Simon Glass <sjg@chromium.org>
> >>> + *
> >>> + * This test program is used to test the invocation of an EFI application.
> >>> + * It writes a few messages to the console and then exits boot services
> >>> + */
> >>> +
> >>> +#include <efi_api.h>
> >>> +
> >>> +static const efi_guid_t loaded_image_guid = EFI_LOADED_IMAGE_PROTOCOL_GUID;
> >>> +
> >>> +static struct efi_system_table *systable;
> >>> +static struct efi_boot_services *boottime;
> >>> +static struct efi_simple_text_output_protocol *con_out;
> >>> +
> >>> +/**
> >>> + * efi_main() - entry point of the EFI application.
> >>> + *
> >>> + * @handle:  handle of the loaded image
> >>> + * @systab:  system table
> >>> + * Return:   status code
> >>> + */
> >>> +efi_status_t EFIAPI efi_main(efi_handle_t handle,
> >>> +                          struct efi_system_table *systab)
> >>> +{
> >>> +     struct efi_loaded_image *loaded_image;
> >>> +     efi_status_t ret;
> >>> +     efi_uintn_t map_size;
> >>> +     efi_uintn_t map_key;
> >>> +     efi_uintn_t desc_size;
> >>> +     u32 desc_version;
> >>> +
> >>> +     systable = systab;
> >>> +     boottime = systable->boottime;
> >>> +     con_out = systable->con_out;
> >>> +
> >>> +     /* Get the loaded image protocol */
> >>> +     ret = boottime->open_protocol(handle, &loaded_image_guid,
> >>> +                                   (void **)&loaded_image, NULL, NULL,
> >>> +                                   EFI_OPEN_PROTOCOL_GET_PROTOCOL);
> >>> +     if (ret != EFI_SUCCESS) {
> >>> +             con_out->output_string
> >>> +                     (con_out, u"Cannot open loaded image protocol\r\n");
> >>> +             goto out;
> >>> +     }
> >>> +
> >>> +     /* UEFI requires CR LF */
> >>> +     con_out->output_string(con_out, u"U-Boot test app for EFI_LOADER\r\n");
> >>> +
> >>> +out:
> >>> +     map_size = 0;
> >>> +     ret = boottime->get_memory_map(&map_size, NULL, &map_key, &desc_size,
> >>> +                                    &desc_version);
> >>> +     con_out->output_string(con_out, u"Exiting boot sevices\n");
> >>> +
> >>> +     /* exit boot services so that this part of U-Boot can be tested */
> >>> +     boottime->exit_boot_services(handle, map_key);
> >>> +
> >>> +     /* now exit for real */
> >>> +     ret = boottime->exit(handle, ret, 0, NULL);
> >>
> >> Please, have a look at chapter 7.4.6,
> >> "EFI_BOOT_SERVICES.ExitBootServices" of the UEFI specification.
> >>
> >> After ExitBootServices() you cannot return anywhere. Boot services are
> >> not available anymore. You can only invoke the UEFI runtime services
> >> which include ResetSystem().
> >
> > Yes, understood, but this is a test, so returning is needed so that
> > the test can check the output is correct.
> >
>
> ExitBootServices() calls dm_remove_devices_flags(DM_REMOVE_ACTIVE_ALL).
> You cannot print anything afterwards as all devices are gone.

That only removes activate devices, i.e. those which use DMA, etc. So
for example the serial port will still be present.

>
> But you can check if a reboot occurs. This is enough to verify that a
> ResetSystem() call placed after ExitBootServices() was executed.
>
> We already have
>
> * lib/efi_selftest/efi_selftest_exitbootservices.c
> * lib/efi_selftest/efi_selftest_variables_runtime.c
> * lib/efi_selftest/efi_selftest_set_virtual_address_map.c
>
> which test different aspects of ExitBootServices() on the sandbox.
>
> There is no need to duplicate these.

Ilias asked that I not put my stuff into HelloWorld, which is why I
added a new app. I can then use it how I please!

Actually none of the examples you list does what my new test does,
which is to boot an app from a bootflow. This was the subject of a
nasty bug, so I do what to add a test for it.

I don't want to reboot, since then it cannot be a unit test. I may
have mentioned this before...but the EFI approach to testing is quite
poor in this respect. We need to test specific regions of the code and
avoid these large, slow and complicated tests.

Regards,
Simon

Re: [PATCH v2 06/39] bootstd: Create a function to reset USB

[Simon Glass]

Hi Heinrich,

On Thu, 8 Aug 2024 at 15:12, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 07.08.24 16:36, Simon Glass wrote:
> > Hi Heinrich,
> >
> > On Tue, 6 Aug 2024 at 19:56, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> On 06.08.24 14:58, Simon Glass wrote:
> >>> Set up a function for this, since it needs to be used from multiple test
> >>> files.
> >>>
> >>> Signed-off-by: Simon Glass <sjg@chromium.org>
> >>> ---
> >>>
> >>> (no changes since v1)
> >>>
> >>>    test/boot/bootdev.c        | 19 ++++++-------------
> >>>    test/boot/bootstd_common.c | 12 ++++++++++++
> >>>    test/boot/bootstd_common.h |  8 ++++++++
> >>>    3 files changed, 26 insertions(+), 13 deletions(-)
> >>>
> >>> diff --git a/test/boot/bootdev.c b/test/boot/bootdev.c
> >>> index 1bf5929c396..de16a51956d 100644
> >>> --- a/test/boot/bootdev.c
> >>> +++ b/test/boot/bootdev.c
> >>> @@ -16,13 +16,6 @@
> >>>    #include <test/ut.h>
> >>>    #include "bootstd_common.h"
> >>>
> >>> -/* Allow reseting the USB-started flag */
> >>> -#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
> >>> -extern bool usb_started;
> >>> -#else
> >>> -#include <usb.h>
> >>> -#endif
> >>> -
> >>>    /* Check 'bootdev list' command */
> >>>    static int bootdev_test_cmd_list(struct unit_test_state *uts)
> >>>    {
> >>> @@ -201,7 +194,7 @@ static int bootdev_test_order(struct unit_test_state *uts)
> >>>        test_set_skip_delays(true);
> >>>
> >>>        /* Start up USB which gives us three additional bootdevs */
> >>> -     usb_started = false;
> >>> +     bootstd_reset_usb();
> >>>        ut_assertok(run_command("usb start", 0));
> >>>
> >>>        /*
> >>> @@ -317,7 +310,7 @@ static int bootdev_test_prio(struct unit_test_state *uts)
> >>>        test_set_eth_enable(false);
> >>>
> >>>        /* Start up USB which gives us three additional bootdevs */
> >>> -     usb_started = false;
> >>> +     bootstd_reset_usb();
> >>>        ut_assertok(run_command("usb start", 0));
> >>>
> >>>        ut_assertok(bootstd_test_drop_bootdev_order(uts));
> >>> @@ -357,7 +350,7 @@ static int bootdev_test_hunter(struct unit_test_state *uts)
> >>>    {
> >>>        struct bootstd_priv *std;
> >>>
> >>> -     usb_started = false;
> >>> +     bootstd_reset_usb();
> >>>        test_set_skip_delays(true);
> >>>
> >>>        /* get access to the used hunters */
> >>> @@ -398,7 +391,7 @@ static int bootdev_test_cmd_hunt(struct unit_test_state *uts)
> >>>        struct bootstd_priv *std;
> >>>
> >>>        test_set_skip_delays(true);
> >>> -     usb_started = false;
> >>> +     bootstd_reset_usb();
> >>>
> >>>        /* get access to the used hunters */
> >>>        ut_assertok(bootstd_get_priv(&std));
> >>> @@ -527,7 +520,7 @@ BOOTSTD_TEST(bootdev_test_bootable, UT_TESTF_DM | UT_TESTF_SCAN_FDT);
> >>>    /* Check hunting for bootdev of a particular priority */
> >>>    static int bootdev_test_hunt_prio(struct unit_test_state *uts)
> >>>    {
> >>> -     usb_started = false;
> >>> +     bootstd_reset_usb();
> >>>        test_set_skip_delays(true);
> >>>
> >>>        console_record_reset_enable();
> >>> @@ -556,7 +549,7 @@ static int bootdev_test_hunt_label(struct unit_test_state *uts)
> >>>        struct bootstd_priv *std;
> >>>        int mflags;
> >>>
> >>> -     usb_started = false;
> >>> +     bootstd_reset_usb();
> >>>
> >>>        /* get access to the used hunters */
> >>>        ut_assertok(bootstd_get_priv(&std));
> >>> diff --git a/test/boot/bootstd_common.c b/test/boot/bootstd_common.c
> >>> index e50539500a0..ff0aff4bbe7 100644
> >>> --- a/test/boot/bootstd_common.c
> >>> +++ b/test/boot/bootstd_common.c
> >>> @@ -20,6 +20,13 @@
> >>>    /* tracks whether bootstd_setup_for_tests() has been run yet */
> >>>    bool vbe_setup_done;
> >>>
> >>> +/* Allow resetting the USB-started flag */
> >>> +#if defined(CONFIG_USB_HOST) || defined(CONFIG_USB_GADGET)
> >>> +extern bool usb_started;
> >>> +#else
> >>> +#include <usb.h>
> >>> +#endif
> >>
> >> Why can't you always include usb.h which defines usb_started?
> >> Is there a bug in usb.h?
> >
> > I can, but even in tests it is not great to access globals. Having it
> > in a function lets us call out these particular settings better, which
> > has aided debugging, at least for me. These tests are special in that
> > they are not DM tests, but do stop USB.
>
> I was just wondering if we can can rid of the #if here. That should not
> stop you from creating a function to access the variable.

Ah, I see. Yes, it is an artifact of it being built for boards without
USB, but now we only build for sandbox, so I can drop that.

Regards,
Simon

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Simon Glass]

Hi Tom,

On Thu, 8 Aug 2024 at 14:06, Tom Rini <trini@konsulko.com> wrote:
>
> On Thu, Aug 08, 2024 at 12:44:05PM -0600, Simon Glass wrote:
> > Hi Heinrick, Tom,
> >
> > On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > > > On 06.08.24 14:58, Simon Glass wrote:
> > > > > While sandbox supports virtio it cannot support actually using the block
> > > > > devices to read files, since there is nothing on the other end of the
> > > > > 'virtqueue'.
> > > > >
> > > > > A recent change makes EFI probe all block devices, whether used or not.
> > > > > This is apparently required by EFI, although it violates U-Boot's
> > > > > lazy-init principle.
> > > > >
> > > > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > > > >
> > > > > So for now just add a special case to work around this.
> > > > >
> > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > ---
> > > > >
> > > > > (no changes since v1)
> > > > >
> > > > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > > > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > > > index 93a9a5ac025..2e1d37848fc 100644
> > > > > --- a/lib/efi_loader/efi_disk.c
> > > > > +++ b/lib/efi_loader/efi_disk.c
> > > > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > > > >   efi_status_t efi_disks_register(void)
> > > > >   {
> > > > >     struct udevice *dev;
> > > > > +   struct uclass *uc;
> > > > >
> > > > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > > > +           /*
> > > > > +            * The virtio block-device hangs on sandbox when accessed since
> > > > > +            * there is nothing listening to the mailbox
> > > > > +            */
> > > > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > > > +
> > > > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > > > +                           continue;
> > > >
> > > > We should avoid depending on the sandbox everywhere.
> > > >
> > > > Please, fix the problem in the sandbox driver.
> > > >
> > > > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > > > the sandbox.
> >
> > Which test? All of the EFI tests fail due to this problem. The test
> > actually has nothing to do with virtio, it is just that EFI goes and
> > probes every single block device, since [1].
>
> Aren't we running "the tests" on other platforms such as QEMU today?
>
> > > This is an area we go back-and-forth on but, yes, IMHO, if we can't
> > > easily provide a virtio device for sandbox, QEMU is right there and what
> > > this is for, so I see sandbox as more useful as build rather than
> > > runtime checking in this case.
> >
> > The best solution would be to implement a simple emulator, like we do
> > in other places for sandbox. At present virtio_sandbox_notify() is
> > empty.
> >
> > I don't mind working on that, but would like to get a temporary
> > solution here so this test can land.
> >
> > Talking about virtio for QEMU is missing the point of this test, which
> > is after all a test of booting an EFI app. I do wish more people would
> > see the value in these unit tests. There is a talk at [2] which shows
> > how emulators are used in Zephyr.
>
> So that talk is interesting, yes. So, yes, implement the bus driver for
> sandbox for virtio, and until then we shouldn't have the tests running
> on sandbox? Or am I still missing something?

Sure, but perhaps there is a way to get this test landed without doing
that work right away?

I'm not sure if we actually need d5391bf02b9 ("efi_loader: ensure all
block devices are probed"). It seems to fix a real problem, though.

>
> But I also still say that given that we as a project are more resource
> constrained than Zephyr, for things that are QEMU-centric, there's
> already a wealth of information on debugging QEMU since it too is
> software. There's only so many hours in the day after all.

One of Zephyr's challenges is that it relied on QEMU for almost all
testing for a long time. As a result it takes a huge amount of CPU
power to run tests - last I checked it was something like an hour on a
64-core machine. U-Boot's unit tests ('ut all') run in about 12
seconds on my machine. I could go on for hours about the different
types of tests and the benefits of one versus the other, but the
cheapest answer is not necessarily just to do a 'happy path' test and
call it good.

For this particular test, I do want to have tests for each bootmeth,
to the point of running the target, so far as possible. It turns out
that we can launch an EFI app on sandbox, so connecting it up to
bootstd seems valuable to me.

I've filed an issue for the virtio improvements.

Regards,
Simon

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 6266 bytes --]

On Sun, Aug 11, 2024 at 08:50:21AM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Thu, 8 Aug 2024 at 14:06, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Thu, Aug 08, 2024 at 12:44:05PM -0600, Simon Glass wrote:
> > > Hi Heinrick, Tom,
> > >
> > > On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > > > > On 06.08.24 14:58, Simon Glass wrote:
> > > > > > While sandbox supports virtio it cannot support actually using the block
> > > > > > devices to read files, since there is nothing on the other end of the
> > > > > > 'virtqueue'.
> > > > > >
> > > > > > A recent change makes EFI probe all block devices, whether used or not.
> > > > > > This is apparently required by EFI, although it violates U-Boot's
> > > > > > lazy-init principle.
> > > > > >
> > > > > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > > > > >
> > > > > > So for now just add a special case to work around this.
> > > > > >
> > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > ---
> > > > > >
> > > > > > (no changes since v1)
> > > > > >
> > > > > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > > > > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > > > > index 93a9a5ac025..2e1d37848fc 100644
> > > > > > --- a/lib/efi_loader/efi_disk.c
> > > > > > +++ b/lib/efi_loader/efi_disk.c
> > > > > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > > > > >   efi_status_t efi_disks_register(void)
> > > > > >   {
> > > > > >     struct udevice *dev;
> > > > > > +   struct uclass *uc;
> > > > > >
> > > > > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > > > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > > > > +           /*
> > > > > > +            * The virtio block-device hangs on sandbox when accessed since
> > > > > > +            * there is nothing listening to the mailbox
> > > > > > +            */
> > > > > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > > > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > > > > +
> > > > > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > > > > +                           continue;
> > > > >
> > > > > We should avoid depending on the sandbox everywhere.
> > > > >
> > > > > Please, fix the problem in the sandbox driver.
> > > > >
> > > > > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > > > > the sandbox.
> > >
> > > Which test? All of the EFI tests fail due to this problem. The test
> > > actually has nothing to do with virtio, it is just that EFI goes and
> > > probes every single block device, since [1].
> >
> > Aren't we running "the tests" on other platforms such as QEMU today?
> >
> > > > This is an area we go back-and-forth on but, yes, IMHO, if we can't
> > > > easily provide a virtio device for sandbox, QEMU is right there and what
> > > > this is for, so I see sandbox as more useful as build rather than
> > > > runtime checking in this case.
> > >
> > > The best solution would be to implement a simple emulator, like we do
> > > in other places for sandbox. At present virtio_sandbox_notify() is
> > > empty.
> > >
> > > I don't mind working on that, but would like to get a temporary
> > > solution here so this test can land.
> > >
> > > Talking about virtio for QEMU is missing the point of this test, which
> > > is after all a test of booting an EFI app. I do wish more people would
> > > see the value in these unit tests. There is a talk at [2] which shows
> > > how emulators are used in Zephyr.
> >
> > So that talk is interesting, yes. So, yes, implement the bus driver for
> > sandbox for virtio, and until then we shouldn't have the tests running
> > on sandbox? Or am I still missing something?
> 
> Sure, but perhaps there is a way to get this test landed without doing
> that work right away?

Maybe? I guess I'm missing how the problem isn't a problem with the
sandbox emulation of it.

> I'm not sure if we actually need d5391bf02b9 ("efi_loader: ensure all
> block devices are probed"). It seems to fix a real problem, though.

That commit seems fairly intentional and I kinda recall that being one
of those challenge points, conceptually. In order for the EFI_LOADER to
do what it needs to do correctly, it needs to know what all exists. This
is contrary to the usual U-Boot practice of not probing something until
it's specifically needed.

> > But I also still say that given that we as a project are more resource
> > constrained than Zephyr, for things that are QEMU-centric, there's
> > already a wealth of information on debugging QEMU since it too is
> > software. There's only so many hours in the day after all.
> 
> One of Zephyr's challenges is that it relied on QEMU for almost all
> testing for a long time. As a result it takes a huge amount of CPU
> power to run tests - last I checked it was something like an hour on a
> 64-core machine. U-Boot's unit tests ('ut all') run in about 12
> seconds on my machine. I could go on for hours about the different
> types of tests and the benefits of one versus the other, but the
> cheapest answer is not necessarily just to do a 'happy path' test and
> call it good.

Everything has it's place, yes. For us, sandbox tests take 15-20 minutes
per run in CI and most QEMU platforms finish up in about a minute. And
of course, it would be real nice if we could easily build those
platforms in CI with CONFIG_UNIT_TEST enabled, but I don't think that
would cause the run time to balloon up (nor are they the cause of the
overall time on sandbox, that would be filesystem tests).

> For this particular test, I do want to have tests for each bootmeth,
> to the point of running the target, so far as possible. It turns out
> that we can launch an EFI app on sandbox, so connecting it up to
> bootstd seems valuable to me.
> 
> I've filed an issue for the virtio improvements.

OK, thanks.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Simon Glass]

Hi Tom,

On Wed, 14 Aug 2024 at 11:56, Tom Rini <trini@konsulko.com> wrote:
>
> On Sun, Aug 11, 2024 at 08:50:21AM -0600, Simon Glass wrote:
> > Hi Tom,
> >
> > On Thu, 8 Aug 2024 at 14:06, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Thu, Aug 08, 2024 at 12:44:05PM -0600, Simon Glass wrote:
> > > > Hi Heinrick, Tom,
> > > >
> > > > On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > > > > > On 06.08.24 14:58, Simon Glass wrote:
> > > > > > > While sandbox supports virtio it cannot support actually using the block
> > > > > > > devices to read files, since there is nothing on the other end of the
> > > > > > > 'virtqueue'.
> > > > > > >
> > > > > > > A recent change makes EFI probe all block devices, whether used or not.
> > > > > > > This is apparently required by EFI, although it violates U-Boot's
> > > > > > > lazy-init principle.
> > > > > > >
> > > > > > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > > > > > >
> > > > > > > So for now just add a special case to work around this.
> > > > > > >
> > > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > ---
> > > > > > >
> > > > > > > (no changes since v1)
> > > > > > >
> > > > > > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > > > > > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > > > > > >
> > > > > > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > > > > > index 93a9a5ac025..2e1d37848fc 100644
> > > > > > > --- a/lib/efi_loader/efi_disk.c
> > > > > > > +++ b/lib/efi_loader/efi_disk.c
> > > > > > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > > > > > >   efi_status_t efi_disks_register(void)
> > > > > > >   {
> > > > > > >     struct udevice *dev;
> > > > > > > +   struct uclass *uc;
> > > > > > >
> > > > > > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > > > > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > > > > > +           /*
> > > > > > > +            * The virtio block-device hangs on sandbox when accessed since
> > > > > > > +            * there is nothing listening to the mailbox
> > > > > > > +            */
> > > > > > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > > > > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > > > > > +
> > > > > > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > > > > > +                           continue;
> > > > > >
> > > > > > We should avoid depending on the sandbox everywhere.
> > > > > >
> > > > > > Please, fix the problem in the sandbox driver.
> > > > > >
> > > > > > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > > > > > the sandbox.
> > > >
> > > > Which test? All of the EFI tests fail due to this problem. The test
> > > > actually has nothing to do with virtio, it is just that EFI goes and
> > > > probes every single block device, since [1].
> > >
> > > Aren't we running "the tests" on other platforms such as QEMU today?
> > >
> > > > > This is an area we go back-and-forth on but, yes, IMHO, if we can't
> > > > > easily provide a virtio device for sandbox, QEMU is right there and what
> > > > > this is for, so I see sandbox as more useful as build rather than
> > > > > runtime checking in this case.
> > > >
> > > > The best solution would be to implement a simple emulator, like we do
> > > > in other places for sandbox. At present virtio_sandbox_notify() is
> > > > empty.
> > > >
> > > > I don't mind working on that, but would like to get a temporary
> > > > solution here so this test can land.
> > > >
> > > > Talking about virtio for QEMU is missing the point of this test, which
> > > > is after all a test of booting an EFI app. I do wish more people would
> > > > see the value in these unit tests. There is a talk at [2] which shows
> > > > how emulators are used in Zephyr.
> > >
> > > So that talk is interesting, yes. So, yes, implement the bus driver for
> > > sandbox for virtio, and until then we shouldn't have the tests running
> > > on sandbox? Or am I still missing something?
> >
> > Sure, but perhaps there is a way to get this test landed without doing
> > that work right away?
>
> Maybe? I guess I'm missing how the problem isn't a problem with the
> sandbox emulation of it.

Basically the virtio block device is probed by EFI (for no useful
purpose), but doesn't actually work.

>
> > I'm not sure if we actually need d5391bf02b9 ("efi_loader: ensure all
> > block devices are probed"). It seems to fix a real problem, though.
>
> That commit seems fairly intentional and I kinda recall that being one
> of those challenge points, conceptually. In order for the EFI_LOADER to
> do what it needs to do correctly, it needs to know what all exists. This
> is contrary to the usual U-Boot practice of not probing something until
> it's specifically needed.

Indeed, but we have to live with it.

>
> > > But I also still say that given that we as a project are more resource
> > > constrained than Zephyr, for things that are QEMU-centric, there's
> > > already a wealth of information on debugging QEMU since it too is
> > > software. There's only so many hours in the day after all.
> >
> > One of Zephyr's challenges is that it relied on QEMU for almost all
> > testing for a long time. As a result it takes a huge amount of CPU
> > power to run tests - last I checked it was something like an hour on a
> > 64-core machine. U-Boot's unit tests ('ut all') run in about 12
> > seconds on my machine. I could go on for hours about the different
> > types of tests and the benefits of one versus the other, but the
> > cheapest answer is not necessarily just to do a 'happy path' test and
> > call it good.
>
> Everything has it's place, yes. For us, sandbox tests take 15-20 minutes
> per run in CI and most QEMU platforms finish up in about a minute. And
> of course, it would be real nice if we could easily build those
> platforms in CI with CONFIG_UNIT_TEST enabled, but I don't think that
> would cause the run time to balloon up (nor are they the cause of the
> overall time on sandbox, that would be filesystem tests).

Right, I normally use 'make qcheck' which skips the FS test and some
other slow ones

Some notes from a little bit of digging: There are almost 1000 sandbox
tests (17-20mins), but qemu_arm only runs 62 (2mins). With 'make
qcheck' it runs about 2500 tests in about 4 minutes. I just got 'make
pcheck' going again and that is a little faster (2.5 mins). Binman
tests run in parallel if you use 'pip install concurrencytest'

>
> > For this particular test, I do want to have tests for each bootmeth,
> > to the point of running the target, so far as possible. It turns out
> > that we can launch an EFI app on sandbox, so connecting it up to
> > bootstd seems valuable to me.
> >
> > I've filed an issue for the virtio improvements.
>
> OK, thanks.

Regards,
Simon

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 7614 bytes --]

On Thu, Aug 15, 2024 at 09:33:18PM +0100, Simon Glass wrote:
> Hi Tom,
> 
> On Wed, 14 Aug 2024 at 11:56, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Sun, Aug 11, 2024 at 08:50:21AM -0600, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Thu, 8 Aug 2024 at 14:06, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Thu, Aug 08, 2024 at 12:44:05PM -0600, Simon Glass wrote:
> > > > > Hi Heinrick, Tom,
> > > > >
> > > > > On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > > > > > > On 06.08.24 14:58, Simon Glass wrote:
> > > > > > > > While sandbox supports virtio it cannot support actually using the block
> > > > > > > > devices to read files, since there is nothing on the other end of the
> > > > > > > > 'virtqueue'.
> > > > > > > >
> > > > > > > > A recent change makes EFI probe all block devices, whether used or not.
> > > > > > > > This is apparently required by EFI, although it violates U-Boot's
> > > > > > > > lazy-init principle.
> > > > > > > >
> > > > > > > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > > > > > > >
> > > > > > > > So for now just add a special case to work around this.
> > > > > > > >
> > > > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > > ---
> > > > > > > >
> > > > > > > > (no changes since v1)
> > > > > > > >
> > > > > > > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > > > > > > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > > > > > > >
> > > > > > > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > > > > > > index 93a9a5ac025..2e1d37848fc 100644
> > > > > > > > --- a/lib/efi_loader/efi_disk.c
> > > > > > > > +++ b/lib/efi_loader/efi_disk.c
> > > > > > > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > > > > > > >   efi_status_t efi_disks_register(void)
> > > > > > > >   {
> > > > > > > >     struct udevice *dev;
> > > > > > > > +   struct uclass *uc;
> > > > > > > >
> > > > > > > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > > > > > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > > > > > > +           /*
> > > > > > > > +            * The virtio block-device hangs on sandbox when accessed since
> > > > > > > > +            * there is nothing listening to the mailbox
> > > > > > > > +            */
> > > > > > > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > > > > > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > > > > > > +
> > > > > > > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > > > > > > +                           continue;
> > > > > > >
> > > > > > > We should avoid depending on the sandbox everywhere.
> > > > > > >
> > > > > > > Please, fix the problem in the sandbox driver.
> > > > > > >
> > > > > > > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > > > > > > the sandbox.
> > > > >
> > > > > Which test? All of the EFI tests fail due to this problem. The test
> > > > > actually has nothing to do with virtio, it is just that EFI goes and
> > > > > probes every single block device, since [1].
> > > >
> > > > Aren't we running "the tests" on other platforms such as QEMU today?
> > > >
> > > > > > This is an area we go back-and-forth on but, yes, IMHO, if we can't
> > > > > > easily provide a virtio device for sandbox, QEMU is right there and what
> > > > > > this is for, so I see sandbox as more useful as build rather than
> > > > > > runtime checking in this case.
> > > > >
> > > > > The best solution would be to implement a simple emulator, like we do
> > > > > in other places for sandbox. At present virtio_sandbox_notify() is
> > > > > empty.
> > > > >
> > > > > I don't mind working on that, but would like to get a temporary
> > > > > solution here so this test can land.
> > > > >
> > > > > Talking about virtio for QEMU is missing the point of this test, which
> > > > > is after all a test of booting an EFI app. I do wish more people would
> > > > > see the value in these unit tests. There is a talk at [2] which shows
> > > > > how emulators are used in Zephyr.
> > > >
> > > > So that talk is interesting, yes. So, yes, implement the bus driver for
> > > > sandbox for virtio, and until then we shouldn't have the tests running
> > > > on sandbox? Or am I still missing something?
> > >
> > > Sure, but perhaps there is a way to get this test landed without doing
> > > that work right away?
> >
> > Maybe? I guess I'm missing how the problem isn't a problem with the
> > sandbox emulation of it.
> 
> Basically the virtio block device is probed by EFI (for no useful
> purpose), but doesn't actually work.
> 
> >
> > > I'm not sure if we actually need d5391bf02b9 ("efi_loader: ensure all
> > > block devices are probed"). It seems to fix a real problem, though.
> >
> > That commit seems fairly intentional and I kinda recall that being one
> > of those challenge points, conceptually. In order for the EFI_LOADER to
> > do what it needs to do correctly, it needs to know what all exists. This
> > is contrary to the usual U-Boot practice of not probing something until
> > it's specifically needed.
> 
> Indeed, but we have to live with it.

Which gets back to the question I was asking, is the sandbox virtio
driver deficient here? It sounds like that doesn't work, and that's the
problem.

> > > > But I also still say that given that we as a project are more resource
> > > > constrained than Zephyr, for things that are QEMU-centric, there's
> > > > already a wealth of information on debugging QEMU since it too is
> > > > software. There's only so many hours in the day after all.
> > >
> > > One of Zephyr's challenges is that it relied on QEMU for almost all
> > > testing for a long time. As a result it takes a huge amount of CPU
> > > power to run tests - last I checked it was something like an hour on a
> > > 64-core machine. U-Boot's unit tests ('ut all') run in about 12
> > > seconds on my machine. I could go on for hours about the different
> > > types of tests and the benefits of one versus the other, but the
> > > cheapest answer is not necessarily just to do a 'happy path' test and
> > > call it good.
> >
> > Everything has it's place, yes. For us, sandbox tests take 15-20 minutes
> > per run in CI and most QEMU platforms finish up in about a minute. And
> > of course, it would be real nice if we could easily build those
> > platforms in CI with CONFIG_UNIT_TEST enabled, but I don't think that
> > would cause the run time to balloon up (nor are they the cause of the
> > overall time on sandbox, that would be filesystem tests).
> 
> Right, I normally use 'make qcheck' which skips the FS test and some
> other slow ones
> 
> Some notes from a little bit of digging: There are almost 1000 sandbox
> tests (17-20mins), but qemu_arm only runs 62 (2mins). With 'make
> qcheck' it runs about 2500 tests in about 4 minutes. I just got 'make
> pcheck' going again and that is a little faster (2.5 mins). Binman
> tests run in parallel if you use 'pip install concurrencytest'

Yes, I find it frustrating that I only tend to run ~50 tests on real
hardware each iteration and skip ~400. Of course, 250 of them are skips
because they only support sandbox. I guess I need to find time to dig in
to UNIT_TEST compiling on more platforms again.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Simon Glass]

Hi Tom,

On Thu, 15 Aug 2024 at 16:56, Tom Rini <trini@konsulko.com> wrote:
>
> On Thu, Aug 15, 2024 at 09:33:18PM +0100, Simon Glass wrote:
> > Hi Tom,
> >
> > On Wed, 14 Aug 2024 at 11:56, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Sun, Aug 11, 2024 at 08:50:21AM -0600, Simon Glass wrote:
> > > > Hi Tom,
> > > >
> > > > On Thu, 8 Aug 2024 at 14:06, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Thu, Aug 08, 2024 at 12:44:05PM -0600, Simon Glass wrote:
> > > > > > Hi Heinrick, Tom,
> > > > > >
> > > > > > On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > > > > > > > On 06.08.24 14:58, Simon Glass wrote:
> > > > > > > > > While sandbox supports virtio it cannot support actually using the block
> > > > > > > > > devices to read files, since there is nothing on the other end of the
> > > > > > > > > 'virtqueue'.
> > > > > > > > >
> > > > > > > > > A recent change makes EFI probe all block devices, whether used or not.
> > > > > > > > > This is apparently required by EFI, although it violates U-Boot's
> > > > > > > > > lazy-init principle.
> > > > > > > > >
> > > > > > > > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > > > > > > > >
> > > > > > > > > So for now just add a special case to work around this.
> > > > > > > > >
> > > > > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > > > ---
> > > > > > > > >
> > > > > > > > > (no changes since v1)
> > > > > > > > >
> > > > > > > > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > > > > > > > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > > > > > > > >
> > > > > > > > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > > > > > > > index 93a9a5ac025..2e1d37848fc 100644
> > > > > > > > > --- a/lib/efi_loader/efi_disk.c
> > > > > > > > > +++ b/lib/efi_loader/efi_disk.c
> > > > > > > > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > > > > > > > >   efi_status_t efi_disks_register(void)
> > > > > > > > >   {
> > > > > > > > >     struct udevice *dev;
> > > > > > > > > +   struct uclass *uc;
> > > > > > > > >
> > > > > > > > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > > > > > > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > > > > > > > +           /*
> > > > > > > > > +            * The virtio block-device hangs on sandbox when accessed since
> > > > > > > > > +            * there is nothing listening to the mailbox
> > > > > > > > > +            */
> > > > > > > > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > > > > > > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > > > > > > > +
> > > > > > > > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > > > > > > > +                           continue;
> > > > > > > >
> > > > > > > > We should avoid depending on the sandbox everywhere.
> > > > > > > >
> > > > > > > > Please, fix the problem in the sandbox driver.
> > > > > > > >
> > > > > > > > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > > > > > > > the sandbox.
> > > > > >
> > > > > > Which test? All of the EFI tests fail due to this problem. The test
> > > > > > actually has nothing to do with virtio, it is just that EFI goes and
> > > > > > probes every single block device, since [1].
> > > > >
> > > > > Aren't we running "the tests" on other platforms such as QEMU today?
> > > > >
> > > > > > > This is an area we go back-and-forth on but, yes, IMHO, if we can't
> > > > > > > easily provide a virtio device for sandbox, QEMU is right there and what
> > > > > > > this is for, so I see sandbox as more useful as build rather than
> > > > > > > runtime checking in this case.
> > > > > >
> > > > > > The best solution would be to implement a simple emulator, like we do
> > > > > > in other places for sandbox. At present virtio_sandbox_notify() is
> > > > > > empty.
> > > > > >
> > > > > > I don't mind working on that, but would like to get a temporary
> > > > > > solution here so this test can land.
> > > > > >
> > > > > > Talking about virtio for QEMU is missing the point of this test, which
> > > > > > is after all a test of booting an EFI app. I do wish more people would
> > > > > > see the value in these unit tests. There is a talk at [2] which shows
> > > > > > how emulators are used in Zephyr.
> > > > >
> > > > > So that talk is interesting, yes. So, yes, implement the bus driver for
> > > > > sandbox for virtio, and until then we shouldn't have the tests running
> > > > > on sandbox? Or am I still missing something?
> > > >
> > > > Sure, but perhaps there is a way to get this test landed without doing
> > > > that work right away?
> > >
> > > Maybe? I guess I'm missing how the problem isn't a problem with the
> > > sandbox emulation of it.
> >
> > Basically the virtio block device is probed by EFI (for no useful
> > purpose), but doesn't actually work.
> >
> > >
> > > > I'm not sure if we actually need d5391bf02b9 ("efi_loader: ensure all
> > > > block devices are probed"). It seems to fix a real problem, though.
> > >
> > > That commit seems fairly intentional and I kinda recall that being one
> > > of those challenge points, conceptually. In order for the EFI_LOADER to
> > > do what it needs to do correctly, it needs to know what all exists. This
> > > is contrary to the usual U-Boot practice of not probing something until
> > > it's specifically needed.
> >
> > Indeed, but we have to live with it.
>
> Which gets back to the question I was asking, is the sandbox virtio
> driver deficient here? It sounds like that doesn't work, and that's the
> problem.

Well until the EFI change there was no need to have a virtio block
device. It just isn't implemented, and that hasn't mattered until now.

>
> > > > > But I also still say that given that we as a project are more resource
> > > > > constrained than Zephyr, for things that are QEMU-centric, there's
> > > > > already a wealth of information on debugging QEMU since it too is
> > > > > software. There's only so many hours in the day after all.
> > > >
> > > > One of Zephyr's challenges is that it relied on QEMU for almost all
> > > > testing for a long time. As a result it takes a huge amount of CPU
> > > > power to run tests - last I checked it was something like an hour on a
> > > > 64-core machine. U-Boot's unit tests ('ut all') run in about 12
> > > > seconds on my machine. I could go on for hours about the different
> > > > types of tests and the benefits of one versus the other, but the
> > > > cheapest answer is not necessarily just to do a 'happy path' test and
> > > > call it good.
> > >
> > > Everything has it's place, yes. For us, sandbox tests take 15-20 minutes
> > > per run in CI and most QEMU platforms finish up in about a minute. And
> > > of course, it would be real nice if we could easily build those
> > > platforms in CI with CONFIG_UNIT_TEST enabled, but I don't think that
> > > would cause the run time to balloon up (nor are they the cause of the
> > > overall time on sandbox, that would be filesystem tests).
> >
> > Right, I normally use 'make qcheck' which skips the FS test and some
> > other slow ones
> >
> > Some notes from a little bit of digging: There are almost 1000 sandbox
> > tests (17-20mins), but qemu_arm only runs 62 (2mins). With 'make
> > qcheck' it runs about 2500 tests in about 4 minutes. I just got 'make
> > pcheck' going again and that is a little faster (2.5 mins). Binman
> > tests run in parallel if you use 'pip install concurrencytest'
>
> Yes, I find it frustrating that I only tend to run ~50 tests on real
> hardware each iteration and skip ~400. Of course, 250 of them are skips
> because they only support sandbox. I guess I need to find time to dig in
> to UNIT_TEST compiling on more platforms again.

Yes...I just tried on snow and found that quite a lot of tests run
which cannot pass, such as bloblist. So that needs tidying up at some
point.

I hope somehow we can make progress on the lab side...I don't see any
interest from the Labgrid people so far.

Regards,
Simon

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Simon Glass]

Hi Tom,

On Thu, 15 Aug 2024 at 14:33, Simon Glass <sjg@chromium.org> wrote:
>
> Hi Tom,
>
> On Wed, 14 Aug 2024 at 11:56, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Sun, Aug 11, 2024 at 08:50:21AM -0600, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Thu, 8 Aug 2024 at 14:06, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Thu, Aug 08, 2024 at 12:44:05PM -0600, Simon Glass wrote:
> > > > > Hi Heinrick, Tom,
> > > > >
> > > > > On Tue, 6 Aug 2024 at 19:56, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Wed, Aug 07, 2024 at 03:47:21AM +0200, Heinrich Schuchardt wrote:
> > > > > > > On 06.08.24 14:58, Simon Glass wrote:
> > > > > > > > While sandbox supports virtio it cannot support actually using the block
> > > > > > > > devices to read files, since there is nothing on the other end of the
> > > > > > > > 'virtqueue'.
> > > > > > > >
> > > > > > > > A recent change makes EFI probe all block devices, whether used or not.
> > > > > > > > This is apparently required by EFI, although it violates U-Boot's
> > > > > > > > lazy-init principle.
> > > > > > > >
> > > > > > > > We cannot just drop the virtio devices as they are used in sandbox tests.
> > > > > > > >
> > > > > > > > So for now just add a special case to work around this.
> > > > > > > >
> > > > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > > ---
> > > > > > > >
> > > > > > > > (no changes since v1)
> > > > > > > >
> > > > > > > >   lib/efi_loader/efi_disk.c | 14 +++++++++++++-
> > > > > > > >   1 file changed, 13 insertions(+), 1 deletion(-)
> > > > > > > >
> > > > > > > > diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c
> > > > > > > > index 93a9a5ac025..2e1d37848fc 100644
> > > > > > > > --- a/lib/efi_loader/efi_disk.c
> > > > > > > > +++ b/lib/efi_loader/efi_disk.c
> > > > > > > > @@ -838,8 +838,20 @@ efi_status_t efi_disk_get_device_name(const efi_handle_t handle, char *buf, int
> > > > > > > >   efi_status_t efi_disks_register(void)
> > > > > > > >   {
> > > > > > > >     struct udevice *dev;
> > > > > > > > +   struct uclass *uc;
> > > > > > > >
> > > > > > > > -   uclass_foreach_dev_probe(UCLASS_BLK, dev) {
> > > > > > > > +   uclass_id_foreach_dev(UCLASS_BLK, dev, uc) {
> > > > > > > > +           /*
> > > > > > > > +            * The virtio block-device hangs on sandbox when accessed since
> > > > > > > > +            * there is nothing listening to the mailbox
> > > > > > > > +            */
> > > > > > > > +           if (IS_ENABLED(CONFIG_SANDBOX)) {
> > > > > > > > +                   struct blk_desc *desc = dev_get_uclass_plat(dev);
> > > > > > > > +
> > > > > > > > +                   if (desc->uclass_id == UCLASS_VIRTIO)
> > > > > > > > +                           continue;
> > > > > > >
> > > > > > > We should avoid depending on the sandbox everywhere.
> > > > > > >
> > > > > > > Please, fix the problem in the sandbox driver.
> > > > > > >
> > > > > > > If you cannot fix it, run the tests involving virtio on QEMU instead of
> > > > > > > the sandbox.
> > > > >
> > > > > Which test? All of the EFI tests fail due to this problem. The test
> > > > > actually has nothing to do with virtio, it is just that EFI goes and
> > > > > probes every single block device, since [1].
> > > >
> > > > Aren't we running "the tests" on other platforms such as QEMU today?
> > > >
> > > > > > This is an area we go back-and-forth on but, yes, IMHO, if we can't
> > > > > > easily provide a virtio device for sandbox, QEMU is right there and what
> > > > > > this is for, so I see sandbox as more useful as build rather than
> > > > > > runtime checking in this case.
> > > > >
> > > > > The best solution would be to implement a simple emulator, like we do
> > > > > in other places for sandbox. At present virtio_sandbox_notify() is
> > > > > empty.
> > > > >
> > > > > I don't mind working on that, but would like to get a temporary
> > > > > solution here so this test can land.
> > > > >
> > > > > Talking about virtio for QEMU is missing the point of this test, which
> > > > > is after all a test of booting an EFI app. I do wish more people would
> > > > > see the value in these unit tests. There is a talk at [2] which shows
> > > > > how emulators are used in Zephyr.
> > > >
> > > > So that talk is interesting, yes. So, yes, implement the bus driver for
> > > > sandbox for virtio, and until then we shouldn't have the tests running
> > > > on sandbox? Or am I still missing something?
> > >
> > > Sure, but perhaps there is a way to get this test landed without doing
> > > that work right away?
> >
> > Maybe? I guess I'm missing how the problem isn't a problem with the
> > sandbox emulation of it.
>
> Basically the virtio block device is probed by EFI (for no useful
> purpose), but doesn't actually work.
>
> >
> > > I'm not sure if we actually need d5391bf02b9 ("efi_loader: ensure all
> > > block devices are probed"). It seems to fix a real problem, though.
> >
> > That commit seems fairly intentional and I kinda recall that being one
> > of those challenge points, conceptually. In order for the EFI_LOADER to
> > do what it needs to do correctly, it needs to know what all exists. This
> > is contrary to the usual U-Boot practice of not probing something until
> > it's specifically needed.
>
> Indeed, but we have to live with it.
>
> >
> > > > But I also still say that given that we as a project are more resource
> > > > constrained than Zephyr, for things that are QEMU-centric, there's
> > > > already a wealth of information on debugging QEMU since it too is
> > > > software. There's only so many hours in the day after all.
> > >
> > > One of Zephyr's challenges is that it relied on QEMU for almost all
> > > testing for a long time. As a result it takes a huge amount of CPU
> > > power to run tests - last I checked it was something like an hour on a
> > > 64-core machine. U-Boot's unit tests ('ut all') run in about 12
> > > seconds on my machine. I could go on for hours about the different
> > > types of tests and the benefits of one versus the other, but the
> > > cheapest answer is not necessarily just to do a 'happy path' test and
> > > call it good.
> >
> > Everything has it's place, yes. For us, sandbox tests take 15-20 minutes
> > per run in CI and most QEMU platforms finish up in about a minute. And
> > of course, it would be real nice if we could easily build those
> > platforms in CI with CONFIG_UNIT_TEST enabled, but I don't think that
> > would cause the run time to balloon up (nor are they the cause of the
> > overall time on sandbox, that would be filesystem tests).
>
> Right, I normally use 'make qcheck' which skips the FS test and some
> other slow ones
>
> Some notes from a little bit of digging: There are almost 1000 sandbox
> tests (17-20mins), but qemu_arm only runs 62 (2mins). With 'make
> qcheck' it runs about 2500 tests in about 4 minutes. I just got 'make
> pcheck' going again and that is a little faster (2.5 mins). Binman
> tests run in parallel if you use 'pip install concurrencytest'

While on this topic I forgot to mention that the sandbox tests (~800
of them) actually run in about 10 seconds. Try 'ut all' (although
you'll need my test-fix series to stop it crashing).

The reason pytest is so slow with sandbox is that it talks to U-Boot
through a pty. Stephen Warren did something clever to stop it being
horrible, but it is still very slow.


>
> >
> > > For this particular test, I do want to have tests for each bootmeth,
> > > to the point of running the target, so far as possible. It turns out
> > > that we can launch an EFI app on sandbox, so connecting it up to
> > > bootstd seems valuable to me.
> > >
> > > I've filed an issue for the virtio improvements.
> >
> > OK, thanks.

Regards,
Simon

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]

On Fri, Aug 16, 2024 at 05:53:59PM -0600, Simon Glass wrote:
[snip]
> > Some notes from a little bit of digging: There are almost 1000 sandbox
> > tests (17-20mins), but qemu_arm only runs 62 (2mins). With 'make
> > qcheck' it runs about 2500 tests in about 4 minutes. I just got 'make
> > pcheck' going again and that is a little faster (2.5 mins). Binman
> > tests run in parallel if you use 'pip install concurrencytest'
> 
> While on this topic I forgot to mention that the sandbox tests (~800
> of them) actually run in about 10 seconds. Try 'ut all' (although
> you'll need my test-fix series to stop it crashing).
> 
> The reason pytest is so slow with sandbox is that it talks to U-Boot
> through a pty. Stephen Warren did something clever to stop it being
> horrible, but it is still very slow.

Seeing about addressing this more would probably be very helpful then as
most of the runs (by user, I mean) happen via pytest and not directly
running sandbox I strongly suspect.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH v2 37/39] efi: Avoid using sandbox virtio devices

[Simon Glass]

Hi Tom,

On Thu, 22 Aug 2024 at 09:13, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Aug 16, 2024 at 05:53:59PM -0600, Simon Glass wrote:
> [snip]
> > > Some notes from a little bit of digging: There are almost 1000 sandbox
> > > tests (17-20mins), but qemu_arm only runs 62 (2mins). With 'make
> > > qcheck' it runs about 2500 tests in about 4 minutes. I just got 'make
> > > pcheck' going again and that is a little faster (2.5 mins). Binman
> > > tests run in parallel if you use 'pip install concurrencytest'
> >
> > While on this topic I forgot to mention that the sandbox tests (~800
> > of them) actually run in about 10 seconds. Try 'ut all' (although
> > you'll need my test-fix series to stop it crashing).
> >
> > The reason pytest is so slow with sandbox is that it talks to U-Boot
> > through a pty. Stephen Warren did something clever to stop it being
> > horrible, but it is still very slow.
>
> Seeing about addressing this more would probably be very helpful then as
> most of the runs (by user, I mean) happen via pytest and not directly
> running sandbox I strongly suspect.

Yes, there is a long thread about performance[1] which makes for
interesting reading these days. We do indeed have >1000 tests although
it took more than 2 years. The Python approach is indeed quite slow,
although half as slow as using the real 'expect' library.

I do have one idea that might help, so I filed [2].

Regards,
Simon

[1] https://patchwork.ozlabs.org/project/uboot/patch/1447570381-1361-1-git-send-email-swarren@wwwdotorg.org/
[2] https://source.denx.de/u-boot/u-boot/-/issues/39