[PATCH] multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for native

[PATCH] multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for native

[Mike Crowe]

Ensure that RECIPE_SYSROOT is the same for -native recipes whether
multilib.conf is included or not.

Without this change task signatures for -native recipes change when
switching between MACHINEs that require multilib.conf and those that
don't.

This fix was one of the ones suggested by Khem Raj in
http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html

Add test_sstate_multilib_or_not_native_samesigs test case to
sstatetests.py to ensure that this stays fixed.

Signed-off-by: Mike Crowe <mac@mcrowe.com>
---
 meta/conf/multilib.conf                     |  1 +
 meta/lib/oeqa/selftest/cases/sstatetests.py | 40 +++++++++++++++++++++
 2 files changed, 41 insertions(+)

diff --git a/meta/conf/multilib.conf b/meta/conf/multilib.conf
index cfed3fbbd0..58f2ac5c86 100644
--- a/meta/conf/multilib.conf
+++ b/meta/conf/multilib.conf
@@ -9,6 +9,7 @@ MULTILIBS ??= "multilib:lib32"
 STAGING_DIR_HOST = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
 STAGING_DIR_TARGET = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
 RECIPE_SYSROOT = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
+RECIPE_SYSROOT_class-native = "${WORKDIR}/recipe-sysroot"
 
 INHERIT += "multilib_global"
 
diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py
index 6757a0ec68..9adb511960 100644
--- a/meta/lib/oeqa/selftest/cases/sstatetests.py
+++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
@@ -446,6 +446,46 @@ BB_SIGNATURE_HANDLER = "OEBasicHash"
         self.assertCountEqual(files1, files2)
 
 
+    def test_sstate_multilib_or_not_native_samesigs(self):
+        """The sstate checksums of two native recipes (and their dependencies)
+        where the target is using multilib in one but not the other
+        should be the same. We use the qemux86copy machine to test
+        this.
+        """
+
+        self.write_config("""
+TMPDIR = \"${TOPDIR}/tmp-sstatesamehash\"
+TCLIBCAPPEND = \"\"
+MACHINE = \"qemux86\"
+require conf/multilib.conf
+MULTILIBS = "multilib:lib32"
+DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
+BB_SIGNATURE_HANDLER = "OEBasicHash"
+""")
+        self.track_for_cleanup(self.topdir + "/tmp-sstatesamehash")
+        bitbake("binutils-native  -S none")
+        self.write_config("""
+TMPDIR = \"${TOPDIR}/tmp-sstatesamehash2\"
+TCLIBCAPPEND = \"\"
+MACHINE = \"qemux86copy\"
+BB_SIGNATURE_HANDLER = "OEBasicHash"
+""")
+        self.track_for_cleanup(self.topdir + "/tmp-sstatesamehash2")
+        bitbake("binutils-native -S none")
+
+        def get_files(d):
+            f = []
+            for root, dirs, files in os.walk(d):
+                for name in files:
+                    f.append(os.path.join(root, name))
+            return f
+        files1 = get_files(self.topdir + "/tmp-sstatesamehash/stamps")
+        files2 = get_files(self.topdir + "/tmp-sstatesamehash2/stamps")
+        files2 = [x.replace("tmp-sstatesamehash2", "tmp-sstatesamehash") for x in files2]
+        self.maxDiff = None
+        self.assertCountEqual(files1, files2)
+
+
     def test_sstate_noop_samesigs(self):
         """
         The sstate checksums of two builds with these variables changed or
-- 
2.24.0

Re: [PATCH] multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for native

[ChenQi]

On 12/17/2019 04:02 PM, Mike Crowe via Openembedded-core wrote:
> Ensure that RECIPE_SYSROOT is the same for -native recipes whether
> multilib.conf is included or not.
>
> Without this change task signatures for -native recipes change when
> switching between MACHINEs that require multilib.conf and those that
> don't.
>
> This fix was one of the ones suggested by Khem Raj in
> http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html
>
> Add test_sstate_multilib_or_not_native_samesigs test case to
> sstatetests.py to ensure that this stays fixed.
>
> Signed-off-by: Mike Crowe <mac@mcrowe.com>
> ---
>   meta/conf/multilib.conf                     |  1 +
>   meta/lib/oeqa/selftest/cases/sstatetests.py | 40 +++++++++++++++++++++
>   2 files changed, 41 insertions(+)
>
> diff --git a/meta/conf/multilib.conf b/meta/conf/multilib.conf
> index cfed3fbbd0..58f2ac5c86 100644
> --- a/meta/conf/multilib.conf
> +++ b/meta/conf/multilib.conf
> @@ -9,6 +9,7 @@ MULTILIBS ??= "multilib:lib32"
>   STAGING_DIR_HOST = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
>   STAGING_DIR_TARGET = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
>   RECIPE_SYSROOT = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
> +RECIPE_SYSROOT_class-native = "${WORKDIR}/recipe-sysroot"

How about just removing MLPREFIX?

Regards,
Chen Qi

>   
>   INHERIT += "multilib_global"
>   
> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py
> index 6757a0ec68..9adb511960 100644
> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
> @@ -446,6 +446,46 @@ BB_SIGNATURE_HANDLER = "OEBasicHash"
>           self.assertCountEqual(files1, files2)
>   
>   
> +    def test_sstate_multilib_or_not_native_samesigs(self):
> +        """The sstate checksums of two native recipes (and their dependencies)
> +        where the target is using multilib in one but not the other
> +        should be the same. We use the qemux86copy machine to test
> +        this.
> +        """
> +
> +        self.write_config("""
> +TMPDIR = \"${TOPDIR}/tmp-sstatesamehash\"
> +TCLIBCAPPEND = \"\"
> +MACHINE = \"qemux86\"
> +require conf/multilib.conf
> +MULTILIBS = "multilib:lib32"
> +DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
> +BB_SIGNATURE_HANDLER = "OEBasicHash"
> +""")
> +        self.track_for_cleanup(self.topdir + "/tmp-sstatesamehash")
> +        bitbake("binutils-native  -S none")
> +        self.write_config("""
> +TMPDIR = \"${TOPDIR}/tmp-sstatesamehash2\"
> +TCLIBCAPPEND = \"\"
> +MACHINE = \"qemux86copy\"
> +BB_SIGNATURE_HANDLER = "OEBasicHash"
> +""")
> +        self.track_for_cleanup(self.topdir + "/tmp-sstatesamehash2")
> +        bitbake("binutils-native -S none")
> +
> +        def get_files(d):
> +            f = []
> +            for root, dirs, files in os.walk(d):
> +                for name in files:
> +                    f.append(os.path.join(root, name))
> +            return f
> +        files1 = get_files(self.topdir + "/tmp-sstatesamehash/stamps")
> +        files2 = get_files(self.topdir + "/tmp-sstatesamehash2/stamps")
> +        files2 = [x.replace("tmp-sstatesamehash2", "tmp-sstatesamehash") for x in files2]
> +        self.maxDiff = None
> +        self.assertCountEqual(files1, files2)
> +
> +
>       def test_sstate_noop_samesigs(self):
>           """
>           The sstate checksums of two builds with these variables changed or

Re: [PATCH] multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for native

[Mike Crowe]

On Tuesday 17 December 2019 at 16:56:11 +0800, ChenQi wrote:
> On 12/17/2019 04:02 PM, Mike Crowe via Openembedded-core wrote:
> > Ensure that RECIPE_SYSROOT is the same for -native recipes whether
> > multilib.conf is included or not.
> > 
> > Without this change task signatures for -native recipes change when
> > switching between MACHINEs that require multilib.conf and those that
> > don't.
> > 
> > This fix was one of the ones suggested by Khem Raj in
> > http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html
> > 
> > Add test_sstate_multilib_or_not_native_samesigs test case to
> > sstatetests.py to ensure that this stays fixed.
> > 
> > Signed-off-by: Mike Crowe <mac@mcrowe.com>
> > ---
> >   meta/conf/multilib.conf                     |  1 +
> >   meta/lib/oeqa/selftest/cases/sstatetests.py | 40 +++++++++++++++++++++
> >   2 files changed, 41 insertions(+)
> > 
> > diff --git a/meta/conf/multilib.conf b/meta/conf/multilib.conf
> > index cfed3fbbd0..58f2ac5c86 100644
> > --- a/meta/conf/multilib.conf
> > +++ b/meta/conf/multilib.conf
> > @@ -9,6 +9,7 @@ MULTILIBS ??= "multilib:lib32"
> >   STAGING_DIR_HOST = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
> >   STAGING_DIR_TARGET = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
> >   RECIPE_SYSROOT = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
> > +RECIPE_SYSROOT_class-native = "${WORKDIR}/recipe-sysroot"
> 
> How about just removing MLPREFIX?

I'm afraid that I don't understand what you are suggesting. The above
change does remove MLPREFIX for -native only.

Thanks.

Mike.

Re: [PATCH] multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for native

[ChenQi]

On 12/17/2019 09:55 PM, Mike Crowe wrote:
> On Tuesday 17 December 2019 at 16:56:11 +0800, ChenQi wrote:
>> On 12/17/2019 04:02 PM, Mike Crowe via Openembedded-core wrote:
>>> Ensure that RECIPE_SYSROOT is the same for -native recipes whether
>>> multilib.conf is included or not.
>>>
>>> Without this change task signatures for -native recipes change when
>>> switching between MACHINEs that require multilib.conf and those that
>>> don't.
>>>
>>> This fix was one of the ones suggested by Khem Raj in
>>> http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html
>>>
>>> Add test_sstate_multilib_or_not_native_samesigs test case to
>>> sstatetests.py to ensure that this stays fixed.
>>>
>>> Signed-off-by: Mike Crowe <mac@mcrowe.com>
>>> ---
>>>    meta/conf/multilib.conf                     |  1 +
>>>    meta/lib/oeqa/selftest/cases/sstatetests.py | 40 +++++++++++++++++++++
>>>    2 files changed, 41 insertions(+)
>>>
>>> diff --git a/meta/conf/multilib.conf b/meta/conf/multilib.conf
>>> index cfed3fbbd0..58f2ac5c86 100644
>>> --- a/meta/conf/multilib.conf
>>> +++ b/meta/conf/multilib.conf
>>> @@ -9,6 +9,7 @@ MULTILIBS ??= "multilib:lib32"
>>>    STAGING_DIR_HOST = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
>>>    STAGING_DIR_TARGET = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
>>>    RECIPE_SYSROOT = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
>>> +RECIPE_SYSROOT_class-native = "${WORKDIR}/recipe-sysroot"
>> How about just removing MLPREFIX?
> I'm afraid that I don't understand what you are suggesting. The above
> change does remove MLPREFIX for -native only.
>
> Thanks.
>
> Mike.
>
I mean removing all these MLPREFIX, so that STAGING_DIR_HOST, 
STAGING_DIR_TARGET and RECIPE_SYSROOT will all be ${WORKDIR}/recipe-sysroot.

Best Regards,

Chen Qi

Re: [PATCH] multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for native

[Mike Crowe]

On Wednesday 18 December 2019 at 09:19:20 +0800, ChenQi wrote:
> On 12/17/2019 09:55 PM, Mike Crowe wrote:
> > On Tuesday 17 December 2019 at 16:56:11 +0800, ChenQi wrote:
> > > On 12/17/2019 04:02 PM, Mike Crowe via Openembedded-core wrote:
> > > > Ensure that RECIPE_SYSROOT is the same for -native recipes whether
> > > > multilib.conf is included or not.
> > > > 
> > > > Without this change task signatures for -native recipes change when
> > > > switching between MACHINEs that require multilib.conf and those that
> > > > don't.
> > > > 
> > > > This fix was one of the ones suggested by Khem Raj in
> > > > http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html
> > > > 
> > > > Add test_sstate_multilib_or_not_native_samesigs test case to
> > > > sstatetests.py to ensure that this stays fixed.
> > > > 
> > > > Signed-off-by: Mike Crowe <mac@mcrowe.com>
> > > > ---
> > > >    meta/conf/multilib.conf                     |  1 +
> > > >    meta/lib/oeqa/selftest/cases/sstatetests.py | 40 +++++++++++++++++++++
> > > >    2 files changed, 41 insertions(+)
> > > > 
> > > > diff --git a/meta/conf/multilib.conf b/meta/conf/multilib.conf
> > > > index cfed3fbbd0..58f2ac5c86 100644
> > > > --- a/meta/conf/multilib.conf
> > > > +++ b/meta/conf/multilib.conf
> > > > @@ -9,6 +9,7 @@ MULTILIBS ??= "multilib:lib32"
> > > >    STAGING_DIR_HOST = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
> > > >    STAGING_DIR_TARGET = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
> > > >    RECIPE_SYSROOT = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
> > > > +RECIPE_SYSROOT_class-native = "${WORKDIR}/recipe-sysroot"
> > > How about just removing MLPREFIX?
> >
> > I'm afraid that I don't understand what you are suggesting. The above
> > change does remove MLPREFIX for -native only.
>
> I mean removing all these MLPREFIX, so that STAGING_DIR_HOST,
> STAGING_DIR_TARGET and RECIPE_SYSROOT will all be
> ${WORKDIR}/recipe-sysroot.

The prefix is necessary for RECIPE_SYSROOT because without it a single
recipe can't depend on multiple multilib variants of the same recipe. We do
this because our bootloader requires both a 32-bit and a 64-bit compiler.
When I changed RECIPE_SYSROOT to always be "${WORKDIR}/recipe-sysroot" I
get failures like:

 do_prepare_recipe_sysroot: The file /usr/include/mtd/mtd-user.h is installed by both linux-libc-headers and lib32-linux-libc-headers, aborting

Thanks.

Mike.