[Buildroot] Build error when building two GRUB2 configurations (race condition)

[Buildroot] Build error when building two GRUB2 configurations (race condition)

[Stefan Agner]

Hello,

In Home Assistant OS we use the capability to build two GRUB2 binaries
by enabling these two configurations:

BR2_TARGET_GRUB2_I386_EFI=y  
BR2_TARGET_GRUB2_X86_64_EFI=y

Every now and then a from scratch build seems to fail with the following
error:

config.status: creating config-util.h
In file included from ../include/grub/disk.h:***,
                 from ../include/grub/file.h:26,
                 from ../grub-core/kern/emu/hostfs.c:23:
./config.h:38:10: fatal error: ./config-util.h: No such file or
directory
   38 | #include <config-util.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.

At least in this instance it seems to be the second configuration
x86_64-efi which fails (as the previous >>> grub2 2.04 Building i386-efi
succeeds).

The full build log can be found here:
https://pipelines.actions.githubusercontent.com/serviceHosts/dff1d65b-5367-4f4f-a0ee-c2bf0f874fbd/_apis/pipelines/1/runs/8778/signedlogcontent/14?urlExpires=2022-06-25T16%3A32%3A02.7601449Z&urlSigningMethod=HMACV1&urlSignature=c9ayjKpOOIoTexbMMXYB8A1G6UwmGfhBwTdtxTE3wmI%3D

I haven't dig into it really, maybe someone with some familiarity of the
GRUB2 (multi-platform) build system has some idea?

Best regards,
Stefan
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Build error when building two GRUB2 configurations (race condition)

[Yann E. MORIN]

Stefan, All,

On 2022-06-25 09:44 -0700, Stefan Agner spake thusly:
> In Home Assistant OS we use the capability to build two GRUB2 binaries
> by enabling these two configurations:
> 
> BR2_TARGET_GRUB2_I386_EFI=y  
> BR2_TARGET_GRUB2_X86_64_EFI=y
> 
> Every now and then a from scratch build seems to fail with the following
> error:
> 
> config.status: creating config-util.h
> In file included from ../include/grub/disk.h:***,
>                  from ../include/grub/file.h:26,
>                  from ../grub-core/kern/emu/hostfs.c:23:
> ./config.h:38:10: fatal error: ./config-util.h: No such file or
> directory
>    38 | #include <config-util.h>
>       |          ^~~~~~~~~~~~~~~
> compilation terminated.

I had the same issue even with a single grub2 config enabled. This is a
grub2 issue, and it was supposedly fixed with upstream commit
42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h) and we
do have this patch backported as:
boot/grub2/0150-Makefile-Make-libgrub.pp-depend-on-config-util.h.patch

We currently have grub2 2.04, almost three years old now; we currently
carry 149 backported patches. This is getting insane. We should update.

grub 2.06 has been out for a year now, but there are already a lot of
commits applied in the tree.

I think we should just bite the bullet and bump to the current HEAD of
the repository.

We have (indirect) runtime testing for grub2, so we can at least check
if bumping is not breaing those (TestIso9660Grub2Hybrid is a two-grub2
build configuration):
    support/testing/tests/boot/test_edk2.py
    support/testing/tests/fs/test_iso9660.py

New runtime tests to explicitly test grub2, added before we bump, then a
bump, would be nice. Hint, hint. ;-)

> At least in this instance it seems to be the second configuration
> x86_64-efi which fails (as the previous >>> grub2 2.04 Building i386-efi
> succeeds).

This is a race condition. As all good race conditions, who wins and who
lose is totally arbitrary... :-/

> The full build log can be found here:
> https://pipelines.actions.githubusercontent.com/serviceHosts/dff1d65b-5367-4f4f-a0ee-c2bf0f874fbd/_apis/pipelines/1/runs/8778/signedlogcontent/14?urlExpires=2022-06-25T16%3A32%3A02.7601449Z&urlSigningMethod=HMACV1&urlSignature=c9ayjKpOOIoTexbMMXYB8A1G6UwmGfhBwTdtxTE3wmI%3D
> 
> I haven't dig into it really, maybe someone with some familiarity of the
> GRUB2 (multi-platform) build system has some idea?

Their Makefiles are, err... interesting. I did have a look a while back
to try and solve that exact issue, and was relieved to see that they
supposedly had a patch already. Once I found it was not enough, I did
not dare look back at the Makefiles again, sadly...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Build error when building two GRUB2 configurations (race condition)

[Stefan Agner]

Hi Yann,

Thanks for your answer!

On 2022-06-25 10:53, Yann E. MORIN wrote:
> Stefan, All,
> 
> On 2022-06-25 09:44 -0700, Stefan Agner spake thusly:
>> In Home Assistant OS we use the capability to build two GRUB2 binaries
>> by enabling these two configurations:
>>
>> BR2_TARGET_GRUB2_I386_EFI=y
>> BR2_TARGET_GRUB2_X86_64_EFI=y
>>
>> Every now and then a from scratch build seems to fail with the following
>> error:
>>
>> config.status: creating config-util.h
>> In file included from ../include/grub/disk.h:***,
>>                  from ../include/grub/file.h:26,
>>                  from ../grub-core/kern/emu/hostfs.c:23:
>> ./config.h:38:10: fatal error: ./config-util.h: No such file or
>> directory
>>    38 | #include <config-util.h>
>>       |          ^~~~~~~~~~~~~~~
>> compilation terminated.
> 
> I had the same issue even with a single grub2 config enabled. This is a
> grub2 issue, and it was supposedly fixed with upstream commit
> 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h) and we
> do have this patch backported as:
> boot/grub2/0150-Makefile-Make-libgrub.pp-depend-on-config-util.h.patch

Hm, yeah this seems to be applied properly.

> 
> We currently have grub2 2.04, almost three years old now; we currently
> carry 149 backported patches. This is getting insane. We should update.
> 
> grub 2.06 has been out for a year now, but there are already a lot of
> commits applied in the tree.
> 
> I think we should just bite the bullet and bump to the current HEAD of
> the repository.
> 
> We have (indirect) runtime testing for grub2, so we can at least check
> if bumping is not breaing those (TestIso9660Grub2Hybrid is a two-grub2
> build configuration):
>     support/testing/tests/boot/test_edk2.py
>     support/testing/tests/fs/test_iso9660.py
> 
> New runtime tests to explicitly test grub2, added before we bump, then a
> bump, would be nice. Hint, hint. ;-)
> 
>> At least in this instance it seems to be the second configuration
>> x86_64-efi which fails (as the previous >>> grub2 2.04 Building i386-efi
>> succeeds).
> 
> This is a race condition. As all good race conditions, who wins and who
> lose is totally arbitrary... :-/
> 
>> The full build log can be found here:
>> https://pipelines.actions.githubusercontent.com/serviceHosts/dff1d65b-5367-4f4f-a0ee-c2bf0f874fbd/_apis/pipelines/1/runs/8778/signedlogcontent/14?urlExpires=2022-06-25T16%3A32%3A02.7601449Z&urlSigningMethod=HMACV1&urlSignature=c9ayjKpOOIoTexbMMXYB8A1G6UwmGfhBwTdtxTE3wmI%3D
>>
>> I haven't dig into it really, maybe someone with some familiarity of the
>> GRUB2 (multi-platform) build system has some idea?
> 
> Their Makefiles are, err... interesting. I did have a look a while back
> to try and solve that exact issue, and was relieved to see that they
> supposedly had a patch already. Once I found it was not enough, I did
> not dare look back at the Makefiles again, sadly...


Just run another build, and it had the same outcome. Also this time, it
was the second build, but could be random still I guess.

From what I can tell config-util.h is required in the all target, and
this is the Makefile part which makes sure it gets created.

config-util.h: stamp-h1
        @test -f $@ || rm -f stamp-h1
        @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) stamp-h1

stamp-h1: $(srcdir)/config-util.h.in $(top_builddir)/config.status
        @rm -f stamp-h1
        cd $(top_builddir) && $(SHELL) ./config.status config-util.h
$(srcdir)/config-util.h.in:  $(am__configure_deps) 
        ($(am__cd) $(top_srcdir) && $(AUTOHEADER))
        rm -f stamp-h1
        touch $@

I wonder if it somehow picks up stamp-h1 from the previous build, or
otherwise races in this section. As this is happening on CI I can't
really inspect the state. I'll try to reproduce locally.

--
Stefan


> Regards,
> Yann E. MORIN.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Build error when building two GRUB2 configurations (race condition)

[Stefan Agner]

On 2022-06-25 19:53, Yann E. MORIN wrote:
> Stefan, All,
> 
> On 2022-06-25 09:44 -0700, Stefan Agner spake thusly:
>> In Home Assistant OS we use the capability to build two GRUB2 binaries
>> by enabling these two configurations:
>>
>> BR2_TARGET_GRUB2_I386_EFI=y
>> BR2_TARGET_GRUB2_X86_64_EFI=y
>>
>> Every now and then a from scratch build seems to fail with the following
>> error:
>>
>> config.status: creating config-util.h
>> In file included from ../include/grub/disk.h:***,
>>                  from ../include/grub/file.h:26,
>>                  from ../grub-core/kern/emu/hostfs.c:23:
>> ./config.h:38:10: fatal error: ./config-util.h: No such file or
>> directory
>>    38 | #include <config-util.h>
>>       |          ^~~~~~~~~~~~~~~
>> compilation terminated.
> 
> I had the same issue even with a single grub2 config enabled. This is a
> grub2 issue, and it was supposedly fixed with upstream commit
> 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h) and we
> do have this patch backported as:
> boot/grub2/0150-Makefile-Make-libgrub.pp-depend-on-config-util.h.patch

I've tracked this down today.

It seems a very similar fix is required for grub_fstest.pp. I sent a
patch upstream to the GRUB mailing list and to the buildroot mailing
list:
https://lists.buildroot.org/pipermail/buildroot/2022-August/648479.html

--
Stefan
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot