RFC: grub-efi native tools

RFC: grub-efi native tools

[Darren Hart]

While working towards supporting efi boot in live images, I have been
refactoring bootimg.bbclass, syslinux.bbclass, and adding recipes and
classes for grub-efi.

I'd like the seasoned OE experts' opinions on a question of
implementation. grub-efi seems to require a final step after the normal
build to assemble the efi image, specifically:

grub-mkimage -p / -d ./grub-core/ -O i386-efi -o ./bootia32.efi \
             boot linux fat serial part_msdos normal

grub-mkimage is a tool built during the grub-efi do_compile step, so it
needs to be native in order to run the above command. So I'm left with a
few options:

1) Use BBCLASSEXTEND="native" and make grub-efi depend on
   grub-efi-native so the native grub-mkimage is available.
   - I'm not sure how to avoid a circular dependency there,
     other than making an explicit grub-efi-native_1.99.bb.
   - No component of the grub-efi build needs to be installed
     on the target rootfs - so perhaps the the target arch version
     isn't needed at all...

2) Only build grub-efi-native.
   - The above grub-mkimage command fails with some incorrect elf format
     error messages on the grub kernel.img file. A similar error was
     reported on the grub mailing list when compiling with cygwin.
     Some work may be required to fix GRUB here.
   - Installing output of -native recipes onto the target seems to be a
     bit at odds with the existing mechanisms (it doesn't find
     bootia32.efi when placed in ${STAGING_LIBDIR}/grub by a -native
     build, for example).

3) Tweak the build so that only the grub-* utilities are built natively.
   - The Linux kernel build does this for some of the internal
     tooling required for the build (Kconfig, etc.).
   - If we did want to install the tools on the target, we wouldn't have
     the target arch binaries available. Perhaps both could be built.

So there are a number of ways to go about this. If someone can leverage
some experience with OE to indicate which of these would be met with the
least resistance, both in terms of implementation as well as upstream
acceptance, I would appreciate it.

Thanks,

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: grub-efi native tools

[Koen Kooi]

[-- Attachment #1: Type: text/plain, Size: 2607 bytes --]


Op 23 nov. 2011, om 08:01 heeft Darren Hart het volgende geschreven:

> While working towards supporting efi boot in live images, I have been
> refactoring bootimg.bbclass, syslinux.bbclass, and adding recipes and
> classes for grub-efi.
> 
> I'd like the seasoned OE experts' opinions on a question of
> implementation. grub-efi seems to require a final step after the normal
> build to assemble the efi image, specifically:
> 
> grub-mkimage -p / -d ./grub-core/ -O i386-efi -o ./bootia32.efi \
>             boot linux fat serial part_msdos normal
> 
> grub-mkimage is a tool built during the grub-efi do_compile step, so it
> needs to be native in order to run the above command. So I'm left with a
> few options:
> 
> 1) Use BBCLASSEXTEND="native" and make grub-efi depend on
>   grub-efi-native so the native grub-mkimage is available.
>   - I'm not sure how to avoid a circular dependency there,
>     other than making an explicit grub-efi-native_1.99.bb.

One way:

DEPENDS = "grub-efi-native"
DEPENDS_virtclass-native = ""

But the proper way is IMAGE_DEPENDS = "grub-efi-native" in the image classes that need it.


>   - No component of the grub-efi build needs to be installed
>     on the target rootfs - so perhaps the the target arch version
>     isn't needed at all...

BBCLASSEXTEND is quite cheap.

> 2) Only build grub-efi-native.
>   - The above grub-mkimage command fails with some incorrect elf format
>     error messages on the grub kernel.img file. A similar error was
>     reported on the grub mailing list when compiling with cygwin.
>     Some work may be required to fix GRUB here.
>   - Installing output of -native recipes onto the target seems to be a
>     bit at odds with the existing mechanisms (it doesn't find
>     bootia32.efi when placed in ${STAGING_LIBDIR}/grub by a -native
>     build, for example).
> 
> 3) Tweak the build so that only the grub-* utilities are built natively.
>   - The Linux kernel build does this for some of the internal
>     tooling required for the build (Kconfig, etc.).
>   - If we did want to install the tools on the target, we wouldn't have
>     the target arch binaries available. Perhaps both could be built.
> 
> So there are a number of ways to go about this. If someone can leverage
> some experience with OE to indicate which of these would be met with the
> least resistance, both in terms of implementation as well as upstream
> acceptance, I would appreciate it.

Have a look at u-boot-mkimage, there's a native and target version available.

regards,

Koen

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 169 bytes --]

Re: RFC: grub-efi native tools

[Darren Hart]

On 11/22/2011 11:36 PM, Koen Kooi wrote:
> 
> Op 23 nov. 2011, om 08:01 heeft Darren Hart het volgende geschreven:
> 
>> While working towards supporting efi boot in live images, I have been
>> refactoring bootimg.bbclass, syslinux.bbclass, and adding recipes and
>> classes for grub-efi.
>>
>> I'd like the seasoned OE experts' opinions on a question of
>> implementation. grub-efi seems to require a final step after the normal
>> build to assemble the efi image, specifically:
>>
>> grub-mkimage -p / -d ./grub-core/ -O i386-efi -o ./bootia32.efi \
>>             boot linux fat serial part_msdos normal
>>
>> grub-mkimage is a tool built during the grub-efi do_compile step, so it
>> needs to be native in order to run the above command. So I'm left with a
>> few options:
>>
>> 1) Use BBCLASSEXTEND="native" and make grub-efi depend on
>>   grub-efi-native so the native grub-mkimage is available.
>>   - I'm not sure how to avoid a circular dependency there,
>>     other than making an explicit grub-efi-native_1.99.bb.
> 
> One way:
> 
> DEPENDS = "grub-efi-native"
> DEPENDS_virtclass-native = ""

Ah, nice. I presume I can also zero out a function for the virtclass-native?

do_compile_append() {
	... non-native stuff ...
}

do_compile_append_virtclass-native() {
	:
}

> 
> But the proper way is IMAGE_DEPENDS = "grub-efi-native" in the image classes that need it.

Hrm, but the MACHINE is what will determine if it's needed. Specifically
if MACHINE_FEATURES contains "efi" and PREFERREF_EFI_LOADER (or some
such) is set to "grub-efi", then it's needed.


> 
> 
>>   - No component of the grub-efi build needs to be installed
>>     on the target rootfs - so perhaps the the target arch version
>>     isn't needed at all...
> 
> BBCLASSEXTEND is quite cheap.
> 
>> 2) Only build grub-efi-native.
>>   - The above grub-mkimage command fails with some incorrect elf format
>>     error messages on the grub kernel.img file. A similar error was
>>     reported on the grub mailing list when compiling with cygwin.
>>     Some work may be required to fix GRUB here.
>>   - Installing output of -native recipes onto the target seems to be a
>>     bit at odds with the existing mechanisms (it doesn't find
>>     bootia32.efi when placed in ${STAGING_LIBDIR}/grub by a -native
>>     build, for example).
>>
>> 3) Tweak the build so that only the grub-* utilities are built natively.
>>   - The Linux kernel build does this for some of the internal
>>     tooling required for the build (Kconfig, etc.).
>>   - If we did want to install the tools on the target, we wouldn't have
>>     the target arch binaries available. Perhaps both could be built.
>>
>> So there are a number of ways to go about this. If someone can leverage
>> some experience with OE to indicate which of these would be met with the
>> least resistance, both in terms of implementation as well as upstream
>> acceptance, I would appreciate it.
> 
> Have a look at u-boot-mkimage, there's a native and target version available.
> 

That looked really promising. Poring over the grub make system not only
made my eyes bleed, but also yielded no means to build only the util
directory. I'm hesitant to add a dependency that require GRUB be
configured and built twice - definitely goes against the goal of
improving build time.

I'll send a note to the grub guys and see if they have a solution.

Thanks Koen, very helpful stuff.

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: grub-efi native tools

[Paul Eggleton]

On Wednesday 23 November 2011 00:01:58 Darren Hart wrote:
> On 11/22/2011 11:36 PM, Koen Kooi wrote:
> > One way:
> > 
> > DEPENDS = "grub-efi-native"
> > DEPENDS_virtclass-native = ""
> 
> Ah, nice. I presume I can also zero out a function for the virtclass-native?
> 
> do_compile_append() {
> 	... non-native stuff ...
> }
> 
> do_compile_append_virtclass-native() {
> 
> }

Unfortunately no, the second will not wipe out the first since appends are 
cumulative. The non-native append would have to be qualified with an override 
(or its content wrapped in a conditional statement).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: RFC: grub-efi native tools

[Darren Hart]

On 11/23/2011 12:10 AM, Paul Eggleton wrote:
> On Wednesday 23 November 2011 00:01:58 Darren Hart wrote:
>> On 11/22/2011 11:36 PM, Koen Kooi wrote:
>>> One way:
>>>
>>> DEPENDS = "grub-efi-native"
>>> DEPENDS_virtclass-native = ""
>>
>> Ah, nice. I presume I can also zero out a function for the virtclass-native?
>>
>> do_compile_append() {
>> 	... non-native stuff ...
>> }
>>
>> do_compile_append_virtclass-native() {
>>
>> }
> 
> Unfortunately no, the second will not wipe out the first since appends are 
> cumulative. The non-native append would have to be qualified with an override 
> (or its content wrapped in a conditional statement).

Right, duh. Thanks for clarifying that.


-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: grub-efi native tools

[Darren Hart]

On 11/22/2011 11:01 PM, Darren Hart wrote:

> 2) Only build grub-efi-native.
>    - The above grub-mkimage command fails with some incorrect elf format
>      error messages on the grub kernel.img file. A similar error was
>      reported on the grub mailing list when compiling with cygwin.
>      Some work may be required to fix GRUB here.

This turned out to be the result of TARGET_ARCH changing from the real
target arch to the BUILD_ARCH via the native.bbclass. I can force a
correct build by hard coding my target. So GRUB is off the hook.

If I cannot access the real TARGET_ARCH when building natively, then
perhaps I should not be trying to assemble a target image during this
step. I could install the native binaries and all the supporting files
in the native sysroot and assemble the target bootia32.efi binary either
as part of the bootimg functions (via a grub-efi.bbclass which depends
on the grub-efi-native do_sysroot)...

This seems pretty round about, and I would prefer for a grub-efi recipe
to deploy (to tmp/deploy/images) a bootia32.efi file. This means that
the grub utilities need to be built native, but grub needs to be
configured with --target=i586 (even when my BUILD_ARCH is x86_64). This
seems to not fit well with how the tooling is setup.

Is there perhaps a better approach?

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel