[PATCH] base.bbclass: Add sstate-cache dependency to GCCVERSION

[PATCH] base.bbclass: Add sstate-cache dependency to GCCVERSION

[Tomasz Dziendzielski]

If we have the build with different gcc versions in the same workspace
it might happen that nativesdk recipe will not detect the change of gcc
and the package will be taken from sstate-cache. This will lead to
do_package_qa failure due to binaries requiring symbols that are not
present in the older libstdc++.

Example error:
| ERROR: nativesdk-mgen-1.0-r0 do_package_qa: QA Issue:
| /opt/poky/3.2.3/sysroots/x86_64-pokysdk-linux/usr/lib/libssh2pp.so.0.1
| contained in package nativesdk-mgen requires
| libstdc++.so.6(GLIBCXX_3.4.11)(64bit), but no providers found in
| RDEPENDS_nativesdk-mgen? [file-rdeps]

Add vardeps dependency to GCCVERSION to make sure the package is rebuild
with correct gcc version.

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com>
---
 meta/classes/base.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index cc02de5f77..da2dc05bba 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -148,6 +148,7 @@ do_fetch[dirs] = "${DL_DIR}"
 do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
 do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
 do_fetch[vardeps] += "SRCREV"
+do_fetch[vardeps] += "GCCVERSION"
 do_fetch[network] = "1"
 python base_do_fetch() {
 
-- 
2.36.1

Re: [OE-core] [PATCH] base.bbclass: Add sstate-cache dependency to GCCVERSION

[Richard Purdie]

On Wed, 2022-06-29 at 15:08 +0200, Tomasz Dziendzielski wrote:
> If we have the build with different gcc versions in the same workspace
> it might happen that nativesdk recipe will not detect the change of gcc
> and the package will be taken from sstate-cache. This will lead to
> do_package_qa failure due to binaries requiring symbols that are not
> present in the older libstdc++.
> 
> Example error:
> > ERROR: nativesdk-mgen-1.0-r0 do_package_qa: QA Issue:
> > /opt/poky/3.2.3/sysroots/x86_64-pokysdk-linux/usr/lib/libssh2pp.so.0.1
> > contained in package nativesdk-mgen requires
> > libstdc++.so.6(GLIBCXX_3.4.11)(64bit), but no providers found in
> > RDEPENDS_nativesdk-mgen? [file-rdeps]
> 
> Add vardeps dependency to GCCVERSION to make sure the package is rebuild
> with correct gcc version.
> 
> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com>
> ---
>  meta/classes/base.bbclass | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index cc02de5f77..da2dc05bba 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -148,6 +148,7 @@ do_fetch[dirs] = "${DL_DIR}"
>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
>  do_fetch[vardeps] += "SRCREV"
> +do_fetch[vardeps] += "GCCVERSION"
>  do_fetch[network] = "1"
>  python base_do_fetch() {
> 


We're definitely not doing that, it is incorrect on many different
levels (e.g. fetching is not dependent on the target compiler version
just for starters). You also just made all native recipes rebuild for
the target GCC version too, which again, is just wrong.

Building with a different target gcc version should mean all the sstate
checksums change. I'm guessing what has happened in your case is that a
TMPDIR as reused after changing gcc version but something didn't
rebuild, probably as ${S} == ${B} and hence ${B} couldn't be cleaned.
We should really track down where the corruption came in and improve
the build output isolation of wherever that came from.

Worse is that your "fix" above probably won't even solve the problem as
if you repeat your build workflow where this broke originally, I'd
strongly suspect it will still break with that change above too.

Cheers,

Richard

Re: [OE-core] [PATCH] base.bbclass: Add sstate-cache dependency to GCCVERSION

[Tomasz Dziendzielski]

[-- Attachment #1: Type: text/plain, Size: 3184 bytes --]

Would setting dependency to SDKGCCVERSION be more acceptable? That way
native recipes will not depend on target gcc version.

>I'm guessing what has happened in your case is that a
>TMPDIR as reused after changing gcc version but something didn't
>rebuild, probably as ${S} == ${B} and hence ${B} couldn't be cleaned

The problem is not that it didn't rerun the make or didn't clean the ${B}
or ${S}. After changing GCCVERSION the nativesdk-mgen is taken from
sstate-cache.

Best regards,
Tomasz Dziendzielski


śr., 29 cze 2022 o 16:23 Richard Purdie <richard.purdie@linuxfoundation.org>
napisał(a):

> On Wed, 2022-06-29 at 15:08 +0200, Tomasz Dziendzielski wrote:
> > If we have the build with different gcc versions in the same workspace
> > it might happen that nativesdk recipe will not detect the change of gcc
> > and the package will be taken from sstate-cache. This will lead to
> > do_package_qa failure due to binaries requiring symbols that are not
> > present in the older libstdc++.
> >
> > Example error:
> > > ERROR: nativesdk-mgen-1.0-r0 do_package_qa: QA Issue:
> > > /opt/poky/3.2.3/sysroots/x86_64-pokysdk-linux/usr/lib/libssh2pp.so.0.1
> > > contained in package nativesdk-mgen requires
> > > libstdc++.so.6(GLIBCXX_3.4.11)(64bit), but no providers found in
> > > RDEPENDS_nativesdk-mgen? [file-rdeps]
> >
> > Add vardeps dependency to GCCVERSION to make sure the package is rebuild
> > with correct gcc version.
> >
> > Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> > Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com>
> > ---
> >  meta/classes/base.bbclass | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> > index cc02de5f77..da2dc05bba 100644
> > --- a/meta/classes/base.bbclass
> > +++ b/meta/classes/base.bbclass
> > @@ -148,6 +148,7 @@ do_fetch[dirs] = "${DL_DIR}"
> >  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
> >  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
> >  do_fetch[vardeps] += "SRCREV"
> > +do_fetch[vardeps] += "GCCVERSION"
> >  do_fetch[network] = "1"
> >  python base_do_fetch() {
> >
>
>
> We're definitely not doing that, it is incorrect on many different
> levels (e.g. fetching is not dependent on the target compiler version
> just for starters). You also just made all native recipes rebuild for
> the target GCC version too, which again, is just wrong.
>
> Building with a different target gcc version should mean all the sstate
> checksums change. I'm guessing what has happened in your case is that a
> TMPDIR as reused after changing gcc version but something didn't
> rebuild, probably as ${S} == ${B} and hence ${B} couldn't be cleaned.
> We should really track down where the corruption came in and improve
> the build output isolation of wherever that came from.
>
> Worse is that your "fix" above probably won't even solve the problem as
> if you repeat your build workflow where this broke originally, I'd
> strongly suspect it will still break with that change above too.
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 4063 bytes --]

Re: [OE-core] [PATCH] base.bbclass: Add sstate-cache dependency to GCCVERSION

[Richard Purdie]

On Thu, 2022-06-30 at 16:14 +0200, Tomasz Dziendzielski wrote:
> Would setting dependency to SDKGCCVERSION be more acceptable?

No, it wouldn't. It doesn't address the manjority of my concerns.

>  That way native recipes will not depend on target gcc version.

The native recipes shouldn't depend on the SDK gcc version either.

> > I'm guessing what has happened in your case is that a
> > TMPDIR as reused after changing gcc version but something didn't
> > rebuild, probably as ${S} == ${B} and hence ${B} couldn't be
> > cleaned
> 
> The problem is not that it didn't rerun the make or didn't clean the
> ${B} or ${S}. After changing GCCVERSION the nativesdk-mgen is taken
> from sstate-cache.

I don't believe that is the issue. I don't have "nativesdk-mgen", but
taking nativesdk-sed as an example:

$ bitbake nativesdk-sed -S none

(to dump the signatures for that recipe)

$ bitbake-dumpsig tmp/stamps/aarch64-nativesdk-pokysdk-linux/nativesdk-sed/4.8-r0.do_prepare_recipe_sysroot.sigdata.*

shows:

Hash for dependent task /media/build1/poky/meta/recipes-devtools/gcc/gcc-crosssdk_12.1.bb:do_populate_sysroot is 4a58e102e0d4e2652a7f960e728b5ae3465a03a9fe6c26135ab6b4dc5b89b951
Hash for dependent task virtual:nativesdk:/media/build1/poky/meta/recipes-devtools/gcc/gcc-runtime_12.1.bb:do_populate_sysroot is 3bcf5017659d63c6e744140dde8db8a9e37a412a38924ab5f1031337dec44198

amongst other things.

i.e. the nativesdk recipes *clearly* do have a dependency on the gcc
version used for nativesdk.

Cheers,

Richard