Grub doesn’t load devicetree

Grub doesn’t load devicetree

[Tomaž Zaman]

[-- Attachment #1: Type: text/plain, Size: 1560 bytes --]

I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot get Grub to load a different device tree than the one that’s provided to it by the EFI Stub.

=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
=> bootefi $kernel_addr_r $fdt_addr_r

At his point, I get a couple of errors like “smmu node not found”.

However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.

And this is where I encounter the issue. Because I would like to use a different DTB for booting Linux than the one that’s provided by the EFI configuration table, I set

devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
linux /boot/vmlinuz-6.6.3-arm64
initrd /boot/initrd.img-6.6.3-arm64

In this case, the devicetree instruction gets ignored completely, and before I see kernel loading, the message “Using FDT from the EFI configuration table"

Is there a way for me to override/replace the device tree that u-boot passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot command, but in that case, I get "EFI stub: ERROR: Failed to handle fs_proto” because kernel when booting doesn’t appear to have access to the same filesystem that Grub does.

Thank you!


(this is my first time using *any* mailing list, so apologies if this is the wrong address to send this question to)

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]


[-- Attachment #2: 18EB2FEE485A4DD7A1CE716CE14FCBC7.png --]
[-- Type: image/png, Size: 5014 bytes --]

Re: Grub doesn’t load devicetree

[Peter Robinson]

On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si> wrote:
>
> I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot get Grub to load a different device tree than the one that’s provided to it by the EFI Stub.
>
> => fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
> => fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> => bootefi $kernel_addr_r $fdt_addr_r
>
> At his point, I get a couple of errors like “smmu node not found”.
>
> However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.
>
> And this is where I encounter the issue. Because I would like to use a different DTB for booting Linux than the one that’s provided by the EFI configuration table, I set
>
> devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> linux /boot/vmlinuz-6.6.3-arm64
> initrd /boot/initrd.img-6.6.3-arm64
>
> In this case, the devicetree instruction gets ignored completely, and before I see kernel loading, the message “Using FDT from the EFI configuration table"
>
> Is there a way for me to override/replace the device tree that u-boot passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot command, but in that case, I get "EFI stub: ERROR: Failed to handle fs_proto” because kernel when booting doesn’t appear to have access to the same filesystem that Grub does.

Why don't you load a new one into memory with u-boot and then grub
will already have the one you want to use and grun doesn't need to do
anything. Something like this on the U-Boot command line would load
the DT and grub and start to boot:

load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
bootefi ${kernel_addr_r} ${fdt_addr_r}

You'll need to adjust for storage, partitions, file paths etc.

> Thank you!
>
>
> (this is my first time using *any* mailing list, so apologies if this is the wrong address to send this question to)
>
> Tomaž Zaman, CEO
> Mono Technologies Inc.
> +386 30 651 645
>
> [cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]
>

Re: Grub doesn’t load devicetree

[Tomaž Zaman]

[-- Attachment #1: Type: text/plain, Size: 3061 bytes --]

Doesn’t work - at least not correctly:

=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
31793 bytes read in 4 ms (7.6 MiB/s)
=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
3978688 bytes read in 168 ms (22.6 MiB/s)
=> setenv fman_ucode
=> bootefi $kernel_addr_r $fdt_addr_r
Failed to load EFI variables
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
Booting /\EFI\boot\grubaa64.efi
Welcome to GRUB!

Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and doesn’t boot with all the hardware support I need it to.

On the other hand, if I run the same kernel/devicetree/initramfs with booti, bypassing the Grub completely, it works fine. The problem is, I need Grub2 for his project.

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:98510E94234546F0B7D19E62D33FDFCC]

On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com>, wrote:
On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si> wrote:

I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot get Grub to load a different device tree than the one that’s provided to it by the EFI Stub.

=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
=> bootefi $kernel_addr_r $fdt_addr_r

At his point, I get a couple of errors like “smmu node not found”.

However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.

And this is where I encounter the issue. Because I would like to use a different DTB for booting Linux than the one that’s provided by the EFI configuration table, I set

devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
linux /boot/vmlinuz-6.6.3-arm64
initrd /boot/initrd.img-6.6.3-arm64

In this case, the devicetree instruction gets ignored completely, and before I see kernel loading, the message “Using FDT from the EFI configuration table"

Is there a way for me to override/replace the device tree that u-boot passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot command, but in that case, I get "EFI stub: ERROR: Failed to handle fs_proto” because kernel when booting doesn’t appear to have access to the same filesystem that Grub does.

Why don't you load a new one into memory with u-boot and then grub
will already have the one you want to use and grun doesn't need to do
anything. Something like this on the U-Boot command line would load
the DT and grub and start to boot:

load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
bootefi ${kernel_addr_r} ${fdt_addr_r}

You'll need to adjust for storage, partitions, file paths etc.

Thank you!


(this is my first time using *any* mailing list, so apologies if this is the wrong address to send this question to)

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]


[-- Attachment #2: 98510E94234546F0B7D19E62D33FDFCC.png --]
[-- Type: image/png, Size: 5014 bytes --]

Re: Grub doesn’t load devicetree

[Peter Robinson]

[-- Attachment #1: Type: text/plain, Size: 3345 bytes --]

On Thu, 29 Aug 2024 at 10:31, Tomaž Zaman <tomaz@mono.si> wrote:

> Doesn’t work - at least not correctly:
>
> => fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> 31793 bytes read in 4 ms (7.6 MiB/s)
> => fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
> 3978688 bytes read in 168 ms (22.6 MiB/s)
> => setenv fman_ucode
> => bootefi $kernel_addr_r $fdt_addr_r
> Failed to load EFI variables
> WARNING failed to get smmu node: FDT_ERR_NOTFOUND
> WARNING failed to get smmu node: FDT_ERR_NOTFOUND
> Booting /\EFI\boot\grubaa64.efi
> Welcome to GRUB!
>
> Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and
> doesn’t boot with all the hardware support I need it to.
>
> On the other hand, if I run the same kernel/devicetree/initramfs with
> booti, bypassing the Grub completely, it works fine. The problem is, I need
> Grub2 for his project.
>

What if you boot the kernel directly using bootefi rather than booti?


> *Tomaž Zaman, *CEO
> Mono Technologies Inc.
> +386 30 651 645
>
>
>
> On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com>,
> wrote:
>
> On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si> wrote:
>
>
> I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot
> get Grub to load a different device tree than the one that’s provided to it
> by the EFI Stub.
>
> => fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
> => fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> => bootefi $kernel_addr_r $fdt_addr_r
>
> At his point, I get a couple of errors like “smmu node not found”.
>
> However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use
> "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.
>
> And this is where I encounter the issue. Because I would like to use a
> different DTB for booting Linux than the one that’s provided by the EFI
> configuration table, I set
>
> devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> linux /boot/vmlinuz-6.6.3-arm64
> initrd /boot/initrd.img-6.6.3-arm64
>
> In this case, the devicetree instruction gets ignored completely, and
> before I see kernel loading, the message “Using FDT from the EFI
> configuration table"
>
> Is there a way for me to override/replace the device tree that u-boot
> passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot
> command, but in that case, I get "EFI stub: ERROR: Failed to handle
> fs_proto” because kernel when booting doesn’t appear to have access to the
> same filesystem that Grub does.
>
>
> Why don't you load a new one into memory with u-boot and then grub
> will already have the one you want to use and grun doesn't need to do
> anything. Something like this on the U-Boot command line would load
> the DT and grub and start to boot:
>
> load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
> load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
> bootefi ${kernel_addr_r} ${fdt_addr_r}
>
> You'll need to adjust for storage, partitions, file paths etc.
>
> Thank you!
>
>
> (this is my first time using *any* mailing list, so apologies if this is
> the wrong address to send this question to)
>
> Tomaž Zaman, CEO
> Mono Technologies Inc.
> +386 30 651 645
>
> [cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]
>
>

[-- Attachment #2: 98510E94234546F0B7D19E62D33FDFCC.png --]
[-- Type: image/png, Size: 5014 bytes --]

Re: Grub doesn’t load devicetree

[Tomaž Zaman]

[-- Attachment #1: Type: text/plain, Size: 3504 bytes --]

I get this:

No UEFI binary known at 0x81000000

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:C1D504EAD21B41379FD3EEB7B3129E94]

On 29 Aug 2024 at 11:38 +0200, Peter Robinson <pbrobinson@gmail.com>, wrote:


On Thu, 29 Aug 2024 at 10:31, Tomaž Zaman <tomaz@mono.si<mailto:tomaz@mono.si>> wrote:
Doesn’t work - at least not correctly:

=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
31793 bytes read in 4 ms (7.6 MiB/s)
=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
3978688 bytes read in 168 ms (22.6 MiB/s)
=> setenv fman_ucode
=> bootefi $kernel_addr_r $fdt_addr_r
Failed to load EFI variables
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
Booting /\EFI\boot\grubaa64.efi
Welcome to GRUB!

Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and doesn’t boot with all the hardware support I need it to.

On the other hand, if I run the same kernel/devicetree/initramfs with booti, bypassing the Grub completely, it works fine. The problem is, I need Grub2 for his project.

What if you boot the kernel directly using bootefi rather than booti?

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:ii_1919d7f0aabad8457221]

On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com<mailto:pbrobinson@gmail.com>>, wrote:
On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si<mailto:tomaz@mono.si>> wrote:

I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot get Grub to load a different device tree than the one that’s provided to it by the EFI Stub.

=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
=> bootefi $kernel_addr_r $fdt_addr_r

At his point, I get a couple of errors like “smmu node not found”.

However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.

And this is where I encounter the issue. Because I would like to use a different DTB for booting Linux than the one that’s provided by the EFI configuration table, I set

devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
linux /boot/vmlinuz-6.6.3-arm64
initrd /boot/initrd.img-6.6.3-arm64

In this case, the devicetree instruction gets ignored completely, and before I see kernel loading, the message “Using FDT from the EFI configuration table"

Is there a way for me to override/replace the device tree that u-boot passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot command, but in that case, I get "EFI stub: ERROR: Failed to handle fs_proto” because kernel when booting doesn’t appear to have access to the same filesystem that Grub does.

Why don't you load a new one into memory with u-boot and then grub
will already have the one you want to use and grun doesn't need to do
anything. Something like this on the U-Boot command line would load
the DT and grub and start to boot:

load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
bootefi ${kernel_addr_r} ${fdt_addr_r}

You'll need to adjust for storage, partitions, file paths etc.

Thank you!


(this is my first time using *any* mailing list, so apologies if this is the wrong address to send this question to)

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]


[-- Attachment #2: C1D504EAD21B41379FD3EEB7B3129E94.png --]
[-- Type: image/png, Size: 5014 bytes --]

[-- Attachment #3: 98510E94234546F0B7D19E62D33FDFCC.png --]
[-- Type: image/png, Size: 5014 bytes --]

Re: Grub doesn’t load devicetree

[Peter Robinson]

[-- Attachment #1: Type: text/plain, Size: 3938 bytes --]

On Thu, 29 Aug 2024 at 10:40, Tomaž Zaman <tomaz@mono.si> wrote:

> I get this:
>
> No UEFI binary known at 0x81000000
>

Are you building the kernel with all the appropriate UEFI shim bits?
Installing the EFI kernel pieces etc?

Sorry, I can't help with kernel build process bits, you'll need to google
that.


> *Tomaž Zaman, *CEO
> Mono Technologies Inc.
> +386 30 651 645
>
>
>
> On 29 Aug 2024 at 11:38 +0200, Peter Robinson <pbrobinson@gmail.com>,
> wrote:
>
>
>
> On Thu, 29 Aug 2024 at 10:31, Tomaž Zaman <tomaz@mono.si> wrote:
>
>> Doesn’t work - at least not correctly:
>>
>> => fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
>> 31793 bytes read in 4 ms (7.6 MiB/s)
>> => fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
>> 3978688 bytes read in 168 ms (22.6 MiB/s)
>> => setenv fman_ucode
>> => bootefi $kernel_addr_r $fdt_addr_r
>> Failed to load EFI variables
>> WARNING failed to get smmu node: FDT_ERR_NOTFOUND
>> WARNING failed to get smmu node: FDT_ERR_NOTFOUND
>> Booting /\EFI\boot\grubaa64.efi
>> Welcome to GRUB!
>>
>> Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and
>> doesn’t boot with all the hardware support I need it to.
>>
>> On the other hand, if I run the same kernel/devicetree/initramfs with
>> booti, bypassing the Grub completely, it works fine. The problem is, I need
>> Grub2 for his project.
>>
>
> What if you boot the kernel directly using bootefi rather than booti?
>
>
>> *Tomaž Zaman, *CEO
>> Mono Technologies Inc.
>> +386 30 651 645
>>
>>
>>
>> On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com>,
>> wrote:
>>
>> On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si> wrote:
>>
>>
>> I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot
>> get Grub to load a different device tree than the one that’s provided to it
>> by the EFI Stub.
>>
>> => fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
>> => fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
>> => bootefi $kernel_addr_r $fdt_addr_r
>>
>> At his point, I get a couple of errors like “smmu node not found”.
>>
>> However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use
>> "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.
>>
>> And this is where I encounter the issue. Because I would like to use a
>> different DTB for booting Linux than the one that’s provided by the EFI
>> configuration table, I set
>>
>> devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
>> linux /boot/vmlinuz-6.6.3-arm64
>> initrd /boot/initrd.img-6.6.3-arm64
>>
>> In this case, the devicetree instruction gets ignored completely, and
>> before I see kernel loading, the message “Using FDT from the EFI
>> configuration table"
>>
>> Is there a way for me to override/replace the device tree that u-boot
>> passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot
>> command, but in that case, I get "EFI stub: ERROR: Failed to handle
>> fs_proto” because kernel when booting doesn’t appear to have access to the
>> same filesystem that Grub does.
>>
>>
>> Why don't you load a new one into memory with u-boot and then grub
>> will already have the one you want to use and grun doesn't need to do
>> anything. Something like this on the U-Boot command line would load
>> the DT and grub and start to boot:
>>
>> load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
>> load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
>> bootefi ${kernel_addr_r} ${fdt_addr_r}
>>
>> You'll need to adjust for storage, partitions, file paths etc.
>>
>> Thank you!
>>
>>
>> (this is my first time using *any* mailing list, so apologies if this is
>> the wrong address to send this question to)
>>
>> Tomaž Zaman, CEO
>> Mono Technologies Inc.
>> +386 30 651 645
>>
>> [cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]
>>
>>

[-- Attachment #2: C1D504EAD21B41379FD3EEB7B3129E94.png --]
[-- Type: image/png, Size: 5014 bytes --]

[-- Attachment #3: 98510E94234546F0B7D19E62D33FDFCC.png --]
[-- Type: image/png, Size: 5014 bytes --]

Re: Grub doesn’t load devicetree

[Tomaž Zaman]

[-- Attachment #1: Type: text/plain, Size: 4133 bytes --]

Thanks for the help, kernel is configured with all the EFI necessities turned on (CONFIG_EFI=y, CONFIG_EFI_STUB=y).

I’ll dig deeper.

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:7E23C6DC89DF4381B59E228581293ABA]

On 29 Aug 2024 at 11:44 +0200, Peter Robinson <pbrobinson@gmail.com>, wrote:


On Thu, 29 Aug 2024 at 10:40, Tomaž Zaman <tomaz@mono.si<mailto:tomaz@mono.si>> wrote:
I get this:

No UEFI binary known at 0x81000000

Are you building the kernel with all the appropriate UEFI shim bits? Installing the EFI kernel pieces etc?

Sorry, I can't help with kernel build process bits, you'll need to google that.

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:ii_1919d84802f123364e41]

On 29 Aug 2024 at 11:38 +0200, Peter Robinson <pbrobinson@gmail.com<mailto:pbrobinson@gmail.com>>, wrote:


On Thu, 29 Aug 2024 at 10:31, Tomaž Zaman <tomaz@mono.si<mailto:tomaz@mono.si>> wrote:
Doesn’t work - at least not correctly:

=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
31793 bytes read in 4 ms (7.6 MiB/s)
=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
3978688 bytes read in 168 ms (22.6 MiB/s)
=> setenv fman_ucode
=> bootefi $kernel_addr_r $fdt_addr_r
Failed to load EFI variables
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
Booting /\EFI\boot\grubaa64.efi
Welcome to GRUB!

Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and doesn’t boot with all the hardware support I need it to.

On the other hand, if I run the same kernel/devicetree/initramfs with booti, bypassing the Grub completely, it works fine. The problem is, I need Grub2 for his project.

What if you boot the kernel directly using bootefi rather than booti?

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:ii_1919d84802fad8457222]

On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com<mailto:pbrobinson@gmail.com>>, wrote:
On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si<mailto:tomaz@mono.si>> wrote:

I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot get Grub to load a different device tree than the one that’s provided to it by the EFI Stub.

=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
=> bootefi $kernel_addr_r $fdt_addr_r

At his point, I get a couple of errors like “smmu node not found”.

However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.

And this is where I encounter the issue. Because I would like to use a different DTB for booting Linux than the one that’s provided by the EFI configuration table, I set

devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
linux /boot/vmlinuz-6.6.3-arm64
initrd /boot/initrd.img-6.6.3-arm64

In this case, the devicetree instruction gets ignored completely, and before I see kernel loading, the message “Using FDT from the EFI configuration table"

Is there a way for me to override/replace the device tree that u-boot passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot command, but in that case, I get "EFI stub: ERROR: Failed to handle fs_proto” because kernel when booting doesn’t appear to have access to the same filesystem that Grub does.

Why don't you load a new one into memory with u-boot and then grub
will already have the one you want to use and grun doesn't need to do
anything. Something like this on the U-Boot command line would load
the DT and grub and start to boot:

load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
bootefi ${kernel_addr_r} ${fdt_addr_r}

You'll need to adjust for storage, partitions, file paths etc.

Thank you!


(this is my first time using *any* mailing list, so apologies if this is the wrong address to send this question to)

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]


[-- Attachment #2: 7E23C6DC89DF4381B59E228581293ABA.png --]
[-- Type: image/png, Size: 5014 bytes --]

[-- Attachment #3: C1D504EAD21B41379FD3EEB7B3129E94.png --]
[-- Type: image/png, Size: 5014 bytes --]

[-- Attachment #4: 98510E94234546F0B7D19E62D33FDFCC.png --]
[-- Type: image/png, Size: 5014 bytes --]

Re: Grub doesn’t load devicetree

[Simon Glass]

Hi Tomaž,

On Thu, 29 Aug 2024 at 03:44, Peter Robinson <pbrobinson@gmail.com> wrote:
>
> On Thu, 29 Aug 2024 at 10:40, Tomaž Zaman <tomaz@mono.si> wrote:
>
> > I get this:
> >
> > No UEFI binary known at 0x81000000
> >

That is the UEFI hack which we need to clean up at some point.

When you load an image, it calls efi_set_bootdev() so EFI knows where
it came from. You could check that the call is happening in your case.

Can U-Boot use the same DT as Linux?

>
> Are you building the kernel with all the appropriate UEFI shim bits?
> Installing the EFI kernel pieces etc?
>
> Sorry, I can't help with kernel build process bits, you'll need to google
> that.
>
>
> > *Tomaž Zaman, *CEO
> > Mono Technologies Inc.
> > +386 30 651 645
> >
> >
> >
> > On 29 Aug 2024 at 11:38 +0200, Peter Robinson <pbrobinson@gmail.com>,
> > wrote:
> >
> >
> >
> > On Thu, 29 Aug 2024 at 10:31, Tomaž Zaman <tomaz@mono.si> wrote:
> >
> >> Doesn’t work - at least not correctly:
> >>
> >> => fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> >> 31793 bytes read in 4 ms (7.6 MiB/s)
> >> => fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
> >> 3978688 bytes read in 168 ms (22.6 MiB/s)
> >> => setenv fman_ucode
> >> => bootefi $kernel_addr_r $fdt_addr_r
> >> Failed to load EFI variables
> >> WARNING failed to get smmu node: FDT_ERR_NOTFOUND
> >> WARNING failed to get smmu node: FDT_ERR_NOTFOUND
> >> Booting /\EFI\boot\grubaa64.efi
> >> Welcome to GRUB!
> >>
> >> Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and
> >> doesn’t boot with all the hardware support I need it to.
> >>
> >> On the other hand, if I run the same kernel/devicetree/initramfs with
> >> booti, bypassing the Grub completely, it works fine. The problem is, I need
> >> Grub2 for his project.
> >>
> >
> > What if you boot the kernel directly using bootefi rather than booti?
> >
> >
> >> *Tomaž Zaman, *CEO
> >> Mono Technologies Inc.
> >> +386 30 651 645
> >>
> >>
> >>
> >> On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com>,
> >> wrote:
> >>
> >> On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si> wrote:
> >>
> >>
> >> I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot
> >> get Grub to load a different device tree than the one that’s provided to it
> >> by the EFI Stub.
> >>
> >> => fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
> >> => fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> >> => bootefi $kernel_addr_r $fdt_addr_r
> >>
> >> At his point, I get a couple of errors like “smmu node not found”.
> >>
> >> However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use
> >> "fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.
> >>
> >> And this is where I encounter the issue. Because I would like to use a
> >> different DTB for booting Linux than the one that’s provided by the EFI
> >> configuration table, I set
> >>
> >> devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
> >> linux /boot/vmlinuz-6.6.3-arm64
> >> initrd /boot/initrd.img-6.6.3-arm64
> >>
> >> In this case, the devicetree instruction gets ignored completely, and
> >> before I see kernel loading, the message “Using FDT from the EFI
> >> configuration table"
> >>
> >> Is there a way for me to override/replace the device tree that u-boot
> >> passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot
> >> command, but in that case, I get "EFI stub: ERROR: Failed to handle
> >> fs_proto” because kernel when booting doesn’t appear to have access to the
> >> same filesystem that Grub does.
> >>
> >>
> >> Why don't you load a new one into memory with u-boot and then grub
> >> will already have the one you want to use and grun doesn't need to do
> >> anything. Something like this on the U-Boot command line would load
> >> the DT and grub and start to boot:
> >>
> >> load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
> >> load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
> >> bootefi ${kernel_addr_r} ${fdt_addr_r}
> >>
> >> You'll need to adjust for storage, partitions, file paths etc.
> >>
> >> Thank you!
> >>
> >>
> >> (this is my first time using *any* mailing list, so apologies if this is
> >> the wrong address to send this question to)
> >>
> >> Tomaž Zaman, CEO
> >> Mono Technologies Inc.
> >> +386 30 651 645
> >>
> >> [cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]
> >>
> >>

Regards,
Simon

Re: Grub doesn’t load devicetree

[Tomaž Zaman]

[-- Attachment #1: Type: text/plain, Size: 4481 bytes --]

This is exactly what I’m testing now.

The board I’m working on has an SDK device tree which deletes the smmu node that’s defined in another dts file, and the lack of that node makes uboot throw errors, but if it’s there, then kernel doesn’t load correclty.

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:9597EA2EB3714038B2BF48BCAAC0C1F8]

On 29 Aug 2024 at 14:18 +0200, Simon Glass <sjg@chromium.org>, wrote:
Hi Tomaž,

On Thu, 29 Aug 2024 at 03:44, Peter Robinson <pbrobinson@gmail.com> wrote:

On Thu, 29 Aug 2024 at 10:40, Tomaž Zaman <tomaz@mono.si> wrote:

I get this:

No UEFI binary known at 0x81000000


That is the UEFI hack which we need to clean up at some point.

When you load an image, it calls efi_set_bootdev() so EFI knows where
it came from. You could check that the call is happening in your case.

Can U-Boot use the same DT as Linux?


Are you building the kernel with all the appropriate UEFI shim bits?
Installing the EFI kernel pieces etc?

Sorry, I can't help with kernel build process bits, you'll need to google
that.


*Tomaž Zaman, *CEO
Mono Technologies Inc.
+386 30 651 645



On 29 Aug 2024 at 11:38 +0200, Peter Robinson <pbrobinson@gmail.com>,
wrote:



On Thu, 29 Aug 2024 at 10:31, Tomaž Zaman <tomaz@mono.si> wrote:

Doesn’t work - at least not correctly:

=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
31793 bytes read in 4 ms (7.6 MiB/s)
=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
3978688 bytes read in 168 ms (22.6 MiB/s)
=> setenv fman_ucode
=> bootefi $kernel_addr_r $fdt_addr_r
Failed to load EFI variables
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
Booting /\EFI\boot\grubaa64.efi
Welcome to GRUB!

Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and
doesn’t boot with all the hardware support I need it to.

On the other hand, if I run the same kernel/devicetree/initramfs with
booti, bypassing the Grub completely, it works fine. The problem is, I need
Grub2 for his project.


What if you boot the kernel directly using bootefi rather than booti?


*Tomaž Zaman, *CEO
Mono Technologies Inc.
+386 30 651 645



On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com>,
wrote:

On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si> wrote:


I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot
get Grub to load a different device tree than the one that’s provided to it
by the EFI Stub.

=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
=> bootefi $kernel_addr_r $fdt_addr_r

At his point, I get a couple of errors like “smmu node not found”.

However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use
"fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.

And this is where I encounter the issue. Because I would like to use a
different DTB for booting Linux than the one that’s provided by the EFI
configuration table, I set

devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
linux /boot/vmlinuz-6.6.3-arm64
initrd /boot/initrd.img-6.6.3-arm64

In this case, the devicetree instruction gets ignored completely, and
before I see kernel loading, the message “Using FDT from the EFI
configuration table"

Is there a way for me to override/replace the device tree that u-boot
passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot
command, but in that case, I get "EFI stub: ERROR: Failed to handle
fs_proto” because kernel when booting doesn’t appear to have access to the
same filesystem that Grub does.


Why don't you load a new one into memory with u-boot and then grub
will already have the one you want to use and grun doesn't need to do
anything. Something like this on the U-Boot command line would load
the DT and grub and start to boot:

load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
bootefi ${kernel_addr_r} ${fdt_addr_r}

You'll need to adjust for storage, partitions, file paths etc.

Thank you!


(this is my first time using *any* mailing list, so apologies if this is
the wrong address to send this question to)

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]



Regards,
Simon

[-- Attachment #2: 9597EA2EB3714038B2BF48BCAAC0C1F8.png --]
[-- Type: image/png, Size: 5014 bytes --]

Re: Grub doesn’t load devicetree

[Tomaž Zaman]

[-- Attachment #1: Type: text/plain, Size: 5930 bytes --]

Update on this, it’s not the smmu node that is problematic, regardless if I add or remove it, the drivers for the DPAA architecture aren’t working:

[ 5.756406] cpu 1: ! CRITICAL FM Error [CPU01, drivers/net/ethernet/freescale/sdk_fman/Peripherals/FM/fm.c:3311 FM_Config]: Invalid Value;
[ 5.756412] cpu 1:
[ 5.768943] cpu 1:
[ 5.773220] cpu 1: ! MAJOR FM Error [CPU01, drivers/net/ethernet/freescale/sdk_fman/src/wrapper/lnxwrp_fm.c:1136 InitFmDev]: Invalid Handle;
[ 5.773223] cpu 1: FM
[ 5.785917] cpu 1:
[ 5.790388] fsl-fman: probe of 1a00000.fman failed with error -5
[ 5.796469] Freescale FM module, FMD API version 21.1.0
[ 5.801866] fsl-fman-port: probe of 1a82000.port failed with error -5
[ 5.808360] fsl-fman-port: probe of 1a83000.port failed with error -5
[ 5.814858] fsl-fman-port: probe of 1a84000.port failed with error -5
[ 5.821351] fsl-fman-port: probe of 1a85000.port failed with error -5
[ 5.827846] fsl-fman-port: probe of 1a86000.port failed with error -5
[ 5.834349] fsl-fman-port: probe of 1a87000.port failed with error -5

interestingly, they’re working fine if I bypass the EFI stub and load the kernel with booti (and give it the exact same device tree)

Any pointers as to where to look for a probable cause of this?


Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:0BC8C304A1D64EE6B1E9C0F91B0A388F]

On 29 Aug 2024 at 15:25 +0200, Tomaz Zaman <tomaz@mono.si>, wrote:
This is exactly what I’m testing now.

The board I’m working on has an SDK device tree which deletes the smmu node that’s defined in another dts file, and the lack of that node makes uboot throw errors, but if it’s there, then kernel doesn’t load correclty.

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:9597EA2EB3714038B2BF48BCAAC0C1F8]

On 29 Aug 2024 at 14:18 +0200, Simon Glass <sjg@chromium.org>, wrote:
Hi Tomaž,

On Thu, 29 Aug 2024 at 03:44, Peter Robinson <pbrobinson@gmail.com> wrote:

On Thu, 29 Aug 2024 at 10:40, Tomaž Zaman <tomaz@mono.si> wrote:

I get this:

No UEFI binary known at 0x81000000


That is the UEFI hack which we need to clean up at some point.

When you load an image, it calls efi_set_bootdev() so EFI knows where
it came from. You could check that the call is happening in your case.

Can U-Boot use the same DT as Linux?


Are you building the kernel with all the appropriate UEFI shim bits?
Installing the EFI kernel pieces etc?

Sorry, I can't help with kernel build process bits, you'll need to google
that.


*Tomaž Zaman, *CEO
Mono Technologies Inc.
+386 30 651 645



On 29 Aug 2024 at 11:38 +0200, Peter Robinson <pbrobinson@gmail.com>,
wrote:



On Thu, 29 Aug 2024 at 10:31, Tomaž Zaman <tomaz@mono.si> wrote:

Doesn’t work - at least not correctly:

=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
31793 bytes read in 4 ms (7.6 MiB/s)
=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
3978688 bytes read in 168 ms (22.6 MiB/s)
=> setenv fman_ucode
=> bootefi $kernel_addr_r $fdt_addr_r
Failed to load EFI variables
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
Booting /\EFI\boot\grubaa64.efi
Welcome to GRUB!

Because of those FDT_ERR_NOTFOUND kernel reports a bunch of errors and
doesn’t boot with all the hardware support I need it to.

On the other hand, if I run the same kernel/devicetree/initramfs with
booti, bypassing the Grub completely, it works fine. The problem is, I need
Grub2 for his project.


What if you boot the kernel directly using bootefi rather than booti?


*Tomaž Zaman, *CEO
Mono Technologies Inc.
+386 30 651 645



On 29 Aug 2024 at 10:02 +0200, Peter Robinson <pbrobinson@gmail.com>,
wrote:

On Thu, 29 Aug 2024 at 05:14, Tomaž Zaman <tomaz@mono.si> wrote:


I’m working on an arm64 board with an NXP LS1046A CPU and I simply cannot
get Grub to load a different device tree than the one that’s provided to it
by the EFI Stub.

=> fatload mmc 0:1 $kernel_addr_r /EFI/boot/grubaa64.efi
=> fatload mmc 0:1 $fdt_addr_r /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
=> bootefi $kernel_addr_r $fdt_addr_r

At his point, I get a couple of errors like “smmu node not found”.

However, if I, instead of "fsl-ls1046a-rdb-sdk.dtb” use
"fsl-ls1046a-rdb.dtb”, then I get to Grub without any errors.

And this is where I encounter the issue. Because I would like to use a
different DTB for booting Linux than the one that’s provided by the EFI
configuration table, I set

devicetree /boot/dtb/fsl-ls1046a-rdb-sdk.dtb
linux /boot/vmlinuz-6.6.3-arm64
initrd /boot/initrd.img-6.6.3-arm64

In this case, the devicetree instruction gets ignored completely, and
before I see kernel loading, the message “Using FDT from the EFI
configuration table"

Is there a way for me to override/replace the device tree that u-boot
passed to Grub? I tried passing the “dtb=“ parameter to the kernel boot
command, but in that case, I get "EFI stub: ERROR: Failed to handle
fs_proto” because kernel when booting doesn’t appear to have access to the
same filesystem that Grub does.


Why don't you load a new one into memory with u-boot and then grub
will already have the one you want to use and grun doesn't need to do
anything. Something like this on the U-Boot command line would load
the DT and grub and start to boot:

load mmc 0:1 ${kernel_addr_r} EFI/BOOT/BOOTAA64.EFI
load mmc 0:1 ${fdt_addr_r} dtb/${fdtfile}
bootefi ${kernel_addr_r} ${fdt_addr_r}

You'll need to adjust for storage, partitions, file paths etc.

Thank you!


(this is my first time using *any* mailing list, so apologies if this is
the wrong address to send this question to)

Tomaž Zaman, CEO
Mono Technologies Inc.
+386 30 651 645

[cid:18EB2FEE485A4DD7A1CE716CE14FCBC7]



Regards,
Simon

[-- Attachment #2: 0BC8C304A1D64EE6B1E9C0F91B0A388F.png --]
[-- Type: image/png, Size: 5014 bytes --]

[-- Attachment #3: 9597EA2EB3714038B2BF48BCAAC0C1F8.png --]
[-- Type: image/png, Size: 5014 bytes --]