Re: [Buildroot] [PATCH 1/1] Makefile: fix rule order for legal-info

["Yann E. MORIN" <yann.morin.1998@free.fr>]

James, All,

On 2023-02-12 03:12 -0700, James Hilliard spake thusly:
> On Sun, Feb 12, 2023 at 2:19 AM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > On 2023-02-11 11:43 -0700, James Hilliard spake thusly:
> > > This command relies on the clean/prepare operations being in a
> > > specific order.
[--SNIP--]
> > However, in the legal-info case, the rule is definitely not
> > parallel-safe, and we do really want it and its dependencies *not* to
> > execute in parallel in fact.
> >
> > Indeed, all PKG-legal-info rules will emit a little blurb that is
> > appended to the manisfest.csv, with commands like:
> >     echo 'PKG,VERSION,blablsablab' >> manifest.csv
> >
> > So, if we run that in parallel, this is going to explode [1].
> Hmm, might just need a flock operation when doing the manifest.csv
> file echo append.

Except we probably want, no, sorry: we do want, that the manifest to be
reproducible (and alphabeticaly sorted), which would not be guaranteed
with an flock.

> We do something like that for downloads already:
> https://github.com/buildroot/buildroot/blob/2022.11.1/package/pkg-download.mk#L113

Careful there: the flock is to protect against a concurrent Buildroot
build, not an internal protection (although it also serves as an
internal protection, when two packages have the same dl dir, like mesa3d
and mesa3d-headers).

> > So, I believe in this case, we do not want to use :: rules, but really
> > ensure that legal-info and all PKG-legal-info rules are never run in
> > parallel, *and* that they are executed in the order they are listed in
> > the prerequisites.
> 
> Well legal-info can take a while to run so it might make sense to make
> it work in parallel.

I understand that, but I still think legal-info should produce a
reproducible output, and that can't be guaranteed if all we change is
using an flock around the manifests.

Also, legal-info is probably run after a complete build, so the overhead
of downloading, extracting, and patching, ahs already been paid during
the build.

We talked about legal-info during the dev-days, and one idea that
floated was that we need to generate an SBOM of the build, and the
closest we have now is legal-info. The SBOM should always be generated,
and I believe legal-info should awlays be generated, so the overhead
point would be moot (but would be replaced by the overhead of running
legal-info for every builds...) More needs to be dug on that topic...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot