[U-Boot] bootefi failures on armv7

Ilias Apalodimas
2019-04-10T16:57:04Z

Hi Alexander, Heinrich,

(Resending since i was too quick on the trigger forgot to cc the public
mailing list. Sorry for the mess)

I can't get an armv7 board (STM32MP157C-DK2) to boot with bootefi on
specific kernel/uboot configurations.
u-boot version: U-Boot 2019.04-rc3
kernel version: 5.0.0-rc3

What works:
- bootefi + 'dtb=' on kernel cmdline regardless of kernel config
- bootefi + fdtcontroladdr if CONFIG_ARM_LPAE=y on my kernel config

What doesn't work:
bootefi + fdtcontroladdr if CONFIG_ARM_LPAE is not set.

Once this config is selected the kernel hangs on a BUG_ON() in arm_pte_alloc()

Adding a dump_stack() there throws the relevant information.
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc3-29427-g769f1f8f9b56-dirty #127
[ 0.000000] Hardware name: STM32 (Device Tree Support)
[ 0.000000] [<c03123ec>] (unwind_backtrace) from [<c030ce0c>] (show_stack+0x10/0x14)
[ 0.000000] [<c030ce0c>] (show_stack) from [<c0e27250>] (dump_stack+0x8c/0xa0)
[ 0.000000] [<c0e27250>] (dump_stack) from [<c1507e9c>] (arm_pte_alloc+0x74/0xb8)
[ 0.000000] [<c1507e9c>] (arm_pte_alloc) from [<c15081ec>] (__create_mapping+0x30c/0x36c)
[ 0.000000] [<c15081ec>] (__create_mapping) from [<c15089f8>] (paging_init+0x234/0x648)
[ 0.000000] [<c15089f8>] (paging_init) from [<c1504950>] (setup_arch+0x660/0xcac)
[ 0.000000] [<c1504950>] (setup_arch) from [<c1500a4c>] (start_kernel+0x70/0x458)
[ 0.000000] [<c1500a4c>] (start_kernel) from [<00000000>] ( (null))

Kernel's EFI map:
[ 0.000000] efi: 0x0000c0000000-0x0000c1ffffff [Boot Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2000000-0x0000c2860fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2861000-0x0000c7cfffff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7d00000-0x0000c7efffff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7f00000-0x0000c7f0bfff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7f0c000-0x0000dc705fff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc706000-0x0000dc709fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc70a000-0x0000dcf6afff [Loader Code | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6b000-0x0000dcf6bfff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6c000-0x0000dcf72fff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf73000-0x0000dcf73fff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf74000-0x0000dcf75fff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf76000-0x0000dff80fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff81000-0x0000dff81fff [Runtime Code |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff82000-0x0000dfffffff [Loader Data | | | | | | | | |WB| | | ]

I've found some hacky workarounds on that and i'd like some input on what's
actually wrong

- Workaround 1, add a bogus bootefi cmd

instead of
load mmc 0:5 $kernel_addr_r efi/BOOT/zImage;
bootefi $kernel_addr_r $fdtcontroladdr

doing
bootefi $kernel_addr_r $fdtcontroladdr
load mmc 0:5 $kernel_addr_r efi/BOOT/zImage
bootefi $kernel_addr_r $fdtcontroladdr' works

The first bootefi command will fail since there's nothing loaded on
that address yet. What happens is that u-boot adds an extra
memory_map, via efi_add_memory_map() and the kernel boots

Kernel's EFI map:
[ 0.000000] efi: Processing EFI memory map:
[ 0.000000] efi: 0x0000c0000000-0x0000c1ffffff [Boot Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2000000-0x0000c2860fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2861000-0x0000c7cf3fff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7cf4000-0x0000c7ef3fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7ef4000-0x0000c7f0bfff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7f0c000-0x0000dc704fff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc705000-0x0000dc708fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc709000-0x0000dcf69fff [Loader Code | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6a000-0x0000dcf6cfff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6d000-0x0000dcf6dfff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6e000-0x0000dcf74fff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf75000-0x0000dcf75fff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf76000-0x0000dff80fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff81000-0x0000dff81fff [Runtime Code |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff82000-0x0000dfffffff [Loader Data | | | | | | | | |WB| | | ]

- Workaround 2, copy fdt one page before the 127mb limit

diff --git a/cmd/bootefi.c b/cmd/bootefi.c
index 3619a20e6433..a0b6156845da 100644
--- a/cmd/bootefi.c
+++ b/cmd/bootefi.c
@@ -108,11 +108,13 @@ static efi_status_t copy_fdt(void **fdtp)
* Safe fdt location is at 127 MiB.
* On the sandbox convert from the sandbox address space.
*/
- new_fdt_addr = (uintptr_t)map_sysmem(fdt_ram_start + 0x7f00000 +
- fdt_size, 0);
+ //new_fdt_addr = (uintptr_t)map_sysmem(fdt_ram_start + 0x7f00000 +
+ //fdt_size, 0);
+ new_fdt_addr = (uintptr_t)map_sysmem(fdt_ram_start + 0x7f00000, fdt_size);
ret = efi_allocate_pages(EFI_ALLOCATE_MAX_ADDRESS,
EFI_RUNTIME_SERVICES_DATA, fdt_pages,
&new_fdt_addr);
+ printf("FDTADDR: %llx pages %lu sz %x\n", new_fdt_addr, fdt_pages, fdt_size);
if (ret != EFI_SUCCESS) {
/* If we can't put it there, put it somewhere */
new_fdt_addr = (ulong)memalign(EFI_PAGE_SIZE, fdt_size);

This loads the fdt on c7ef4000 (which is more than a page). Changing
the address from 0x7f00000 to 7EFF000, on the original code, works as
well

Kernel's EFI map (with the patch) :
[ 0.000000] efi: Processing EFI memory map:
[ 0.000000] efi: 0x0000c0000000-0x0000c1ffffff [Boot Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2000000-0x0000c2860fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2861000-0x0000c7cf3fff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7cf4000-0x0000c7ef3fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7ef4000-0x0000c7efffff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7f00000-0x0000dc705fff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc706000-0x0000dc709fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc70a000-0x0000dcf6afff [Loader Code | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6b000-0x0000dcf6bfff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6c000-0x0000dcf72fff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf73000-0x0000dcf73fff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf74000-0x0000dcf75fff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf76000-0x0000dff80fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff81000-0x0000dff81fff [Runtime Code |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff82000-0x0000dfffffff [Loader Data | | | | | | | | |WB| | | ]

- Workaround 3, align fdt on 1mb boundries

diff --git a/cmd/bootefi.c b/cmd/bootefi.c
index 3619a20e6433..37e849fad773 100644
--- a/cmd/bootefi.c
+++ b/cmd/bootefi.c
@@ -101,7 +101,8 @@ static efi_status_t copy_fdt(void **fdtp)
* needs to be expanded later.
*/
fdt = *fdtp;
- fdt_pages = efi_size_in_pages(fdt_totalsize(fdt) + 0x3000);
+ //fdt_pages = efi_size_in_pages(fdt_totalsize(fdt) + 0x3000);
+ fdt_pages = efi_size_in_pages(0x100000);
fdt_size = fdt_pages << EFI_PAGE_SHIFT;

/*

Kernel's EFI map:
[ 0.000000] efi: Processing EFI memory map:
[ 0.000000] efi: 0x0000c0000000-0x0000c1ffffff [Boot Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2000000-0x0000c2860fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c2861000-0x0000c7cfffff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7d00000-0x0000c7efffff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c7f00000-0x0000c7ffffff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000c8000000-0x0000dc705fff [Conventional Memory| | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc706000-0x0000dc709fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dc70a000-0x0000dcf6afff [Loader Code | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6b000-0x0000dcf6bfff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf6c000-0x0000dcf72fff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf73000-0x0000dcf73fff [Runtime Data |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf74000-0x0000dcf75fff [Reserved | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dcf76000-0x0000dff80fff [Loader Data | | | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff81000-0x0000dff81fff [Runtime Code |RUN| | | | | | | |WB| | | ]
[ 0.000000] efi: 0x0000dff82000-0x0000dfffffff [Loader Data | | | | | | | | |WB| | | ]

Thanks
/Ilias