Re: [PATCH] bitbake.conf: add BB_NUMBER_THREADS to BB_HASHEXCLUDE_COMMON

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Mon, 2022-02-28 at 15:06 +0100, Rasmus Villemoes wrote:
> On 28/02/2022 14.41, Richard Purdie wrote:
> > On Mon, 2022-02-28 at 09:42 +0100, Rasmus Villemoes wrote:
> > > The imx-gpu-sdk recipe in the meta-imx layer references
> > > ${BB_NUMBER_THREADS} in its do_compile function. Changing
> > > BB_NUMBER_THREADS between bitbake invocations leads to the well-known
> > > 
> > >   When reparsing ...meta-imx/meta-sdk/recipes-graphics/imx-gpu-sdk/imx-gpu-sdk_5.8.0.bb:do_compile, the basehash value changed from 69be88cf220840ff2203e11cfe65681880b0bf9b88db67d50c1ba772b883bd18 to 5e6d5029fac8d7856ada4c2eca359568298f82cdb64567d7dd4deda503d9f83a. The metadata is not deterministic and this needs to be fixed.
> > > 
> > > And I'm not the first to hit this problem with that recipe:
> > > https://community.nxp.com/t5/i-MX-Processors/imx-gpu-sdk-compile-error-IMX8MP-IMX8MM-IMX8MQ/td-p/1217864
> > > 
> > > This happens because BB_NUMBER_THREADS is in BB_HASHCONFIG_IGNORE_VARS,
> > > so changing it does not cause the recipe to be reparsed, but it is not
> > > included in BB_HASHEXCLUDE_COMMON and thus
> > > BB_BASEHASH_IGNORE_VARS. This is inconsistent with and in contrast to
> > > both PARALLEL_MAKE and OMP_NUM_THREADS, the latter of which even has
> > > ${BB_NUMBER_THREADS} as default value.
> > 
> > Technically imx-gpu-sdk is incorrect. BB_NUMBER_THREADS is the number of tasks
> > bitbake should run. PARALLEL_MAKE is what is used for parallelism in do_compile.
> > 
> > I appreciate that has -j in but you can use: ${@oe.utils.parallel_make(d)} to
> > obtain the value that recipe needs.
> 
> Oh, I'm not saying that that recipe isn't buggy, but is still seems
> awfully user-unfriendly and inconsistent to treat BB_NUMBER_THREADS
> different from PARALLEL_MAKE wrt. hash computations.

The reasoning is that nothing should be using BB_NUMBER_THREADS like that. Your
patch would just mask it even more :/. Ideally we'd have something to tell
people they were misusing it but that is harder to do.

> But now I've raised the issue, and I don't care much what happens next;
> I now know the root cause of the error I saw and rarely build using
> meta-imx in the first place.

Just to check, are the meta-imx people aware of it and able to fix the real
problem?

Cheers,

Richard