[PATCH 1/3] coreutils: add openssl PACKAGECONFIG

[PATCH 1/3] coreutils: add openssl PACKAGECONFIG

[Dan McGregor]

From: Daniel McGregor <daniel.mcgregor@vecima.com>

coreutils-native will pick up openssl on the host if it's GPL
compatible (version >= 3), which causes uninative failures with hosts
that don't have openssl3.

Add a PACKAGECONFIG entry for openssl so it can be enabled, but isn't
by default.

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
---
 meta/recipes-core/coreutils/coreutils_9.1.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/coreutils/coreutils_9.1.bb b/meta/recipes-core/coreutils/coreutils_9.1.bb
index 55663c77136..4807eefd04d 100644
--- a/meta/recipes-core/coreutils/coreutils_9.1.bb
+++ b/meta/recipes-core/coreutils/coreutils_9.1.bb
@@ -46,6 +46,7 @@ PACKAGECONFIG[acl] = "--enable-acl,--disable-acl,acl,"
 PACKAGECONFIG[xattr] = "--enable-xattr,--disable-xattr,attr,"
 PACKAGECONFIG[single-binary] = "--enable-single-binary,--disable-single-binary,,"
 PACKAGECONFIG[selinux] = "--with-selinux,--without-selinux,libselinux"
+PACKAGECONFIG[openssl] = "--with-openssl=yes,--with-openssl=no,openssl"
 
 # [ df mktemp nice printenv base64 gets a special treatment and is not included in this
 bindir_progs = "arch basename chcon cksum comm csplit cut dir dircolors dirname du \
-- 
2.37.3

[PATCH 2/3] gcc: set the default target arch

[Dan McGregor]

From: Daniel McGregor <daniel.mcgregor@vecima.com>

The default x86-64 architecture for target gcc (ie, the one in poky
build appliances) is native. Since we have a variety of build systems
it will occasionally produce instructions that don't work on all of
our development system.

Instead, set gcc's default architecture to the one specified in
TUNE_CC_ARCH, that guarantees that gcc-runtime and any binaries
produced are compatible with the target machine type.

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
---
 meta/recipes-devtools/gcc/gcc-common.inc | 10 ++++++++++
 meta/recipes-devtools/gcc/gcc-target.inc |  2 +-
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-common.inc b/meta/recipes-devtools/gcc/gcc-common.inc
index 2abc0e355d7..d3b36937bf4 100644
--- a/meta/recipes-devtools/gcc/gcc-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-common.inc
@@ -32,6 +32,16 @@ def get_gcc_float_setting(bb, d):
 
 get_gcc_float_setting[vardepvalue] = "${@get_gcc_float_setting(bb, d)}"
 
+def get_gcc_x86_64_arch_setting(bb, d):
+    import re
+    march = re.match(r'^.*-march=([^\s]*)', d.getVar('TUNE_CCARGS'))
+    if march:
+        return "--with-arch=%s " % march.group(1)
+    # The earliest supported x86-64 CPU
+    return "--with-arch=core2"
+
+get_gcc_x86_64_arch_setting[vardepvalue] = "${@get_gcc_x86_64_arch_setting(bb, d)}"
+
 def get_gcc_mips_plt_setting(bb, d):
     if d.getVar('TRANSLATED_TARGET_ARCH') in [ 'mips', 'mipsel' ] and bb.utils.contains('DISTRO_FEATURES', 'mplt', True, False, d):
         return "--with-mips-plt"
diff --git a/meta/recipes-devtools/gcc/gcc-target.inc b/meta/recipes-devtools/gcc/gcc-target.inc
index cc65e995c30..7dac3ef422c 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -19,7 +19,7 @@ EXTRA_OECONF:append:armv6:class-target = " --with-arch=armv6${ARMFPARCHEXT}"
 EXTRA_OECONF:append:armv7a:class-target = " --with-arch=armv7-a${ARMFPARCHEXT}"
 EXTRA_OECONF:append:armv7ve:class-target = " --with-arch=armv7ve${ARMFPARCHEXT}"
 EXTRA_OECONF:append:arc:class-target = " --with-cpu=${TUNE_PKGARCH}"
-EXTRA_OECONF:append:x86-64:class-target = " --with-arch=native"
+EXTRA_OECONF:append:x86-64:class-target = " ${@get_gcc_x86_64_arch_setting(bb, d)}"
 
 # libcc1 requres gcc_cv_objdump when cross build, but gcc_cv_objdump is
 # set in subdir gcc, so subdir libcc1 can't use it, export it here to
-- 
2.37.3

Re: [OE-core] [PATCH 2/3] gcc: set the default target arch

[Ross Burton]

On 28 Sep 2022, at 23:13, Dan McGregor via lists.openembedded.org <danismostlikely=gmail.com@lists.openembedded.org> wrote:
> 
> From: Daniel McGregor <daniel.mcgregor@vecima.com>
> 
> The default x86-64 architecture for target gcc (ie, the one in poky
> build appliances) is native. Since we have a variety of build systems
> it will occasionally produce instructions that don't work on all of
> our development system.
> 
> Instead, set gcc's default architecture to the one specified in
> TUNE_CC_ARCH, that guarantees that gcc-runtime and any binaries
> produced are compatible with the target machine type.

Can you explain the use-case here?  These are target binaries, so why is the development system relevant?

Ross

Re: [OE-core] [PATCH 2/3] gcc: set the default target arch

[Ross Burton]

> On 29 Sep 2022, at 16:31, Daniel McGregor <daniel.mcgregor@vecima.com> wrote:
> 
> Sure, I'm actually talking about host binaries, built inside our equivalent of a poky build appliance. The most obvious example is pseudo-native built on our newest build servers. GCC decides to use instructions (in particular BMI2 instructions) that don't exist on our older build machines. Since we use a common shared state and uninative it was becoming pretty common for old machines to pull shared state built on a much newer one, leading to build failures do to invalid opcode exceptions.

Ah, so you build a container with Yocto and then do your Yocto builds inside that container.  Got it, thanks.

Ross

[PATCH 3/3] systemd: generate manpages indexes

[Dan McGregor]

From: Daniel McGregor <daniel.mcgregor@vecima.com>

Now that python3-lxml is in core, use it to generate
systemd.directives and systemd.index manual pages.

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
---
 meta/recipes-core/systemd/systemd_251.4.bb | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/systemd/systemd_251.4.bb b/meta/recipes-core/systemd/systemd_251.4.bb
index 910ea71bf64..b4ad7e45eb0 100644
--- a/meta/recipes-core/systemd/systemd_251.4.bb
+++ b/meta/recipes-core/systemd/systemd_251.4.bb
@@ -159,7 +159,10 @@ PACKAGECONFIG[localed] = "-Dlocaled=true,-Dlocaled=false"
 PACKAGECONFIG[logind] = "-Dlogind=true,-Dlogind=false"
 PACKAGECONFIG[lz4] = "-Dlz4=true,-Dlz4=false,lz4"
 PACKAGECONFIG[machined] = "-Dmachined=true,-Dmachined=false"
-PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,libxslt-native xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
+PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,python3-lxml-native xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
+# Ensure we're seeing the XML catalog from the build system and not the host.
+# Otherwise, the build may fail generating the manpages.
+export XML_CATALOG_FILES="${STAGING_ETCDIR_NATIVE}/xml/catalog"
 PACKAGECONFIG[microhttpd] = "-Dmicrohttpd=true,-Dmicrohttpd=false,libmicrohttpd"
 PACKAGECONFIG[myhostname] = "-Dnss-myhostname=true,-Dnss-myhostname=false,,libnss-myhostname"
 PACKAGECONFIG[networkd] = "-Dnetworkd=true,-Dnetworkd=false"
-- 
2.37.3

Re: [OE-core] [PATCH 3/3] systemd: generate manpages indexes

[Richard Purdie]

On Wed, 2022-09-28 at 16:13 -0600, Dan McGregor wrote:
> From: Daniel McGregor <daniel.mcgregor@vecima.com>
> 
> Now that python3-lxml is in core, use it to generate
> systemd.directives and systemd.index manual pages.
> 
> Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
> ---
>  meta/recipes-core/systemd/systemd_251.4.bb | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-core/systemd/systemd_251.4.bb b/meta/recipes-core/systemd/systemd_251.4.bb
> index 910ea71bf64..b4ad7e45eb0 100644
> --- a/meta/recipes-core/systemd/systemd_251.4.bb
> +++ b/meta/recipes-core/systemd/systemd_251.4.bb
> @@ -159,7 +159,10 @@ PACKAGECONFIG[localed] = "-Dlocaled=true,-Dlocaled=false"
>  PACKAGECONFIG[logind] = "-Dlogind=true,-Dlogind=false"
>  PACKAGECONFIG[lz4] = "-Dlz4=true,-Dlz4=false,lz4"
>  PACKAGECONFIG[machined] = "-Dmachined=true,-Dmachined=false"
> -PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,libxslt-native xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
> +PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,python3-lxml-native xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
> +# Ensure we're seeing the XML catalog from the build system and not the host.
> +# Otherwise, the build may fail generating the manpages.
> +export XML_CATALOG_FILES="${STAGING_ETCDIR_NATIVE}/xml/catalog"

Wasn't there a fix that means this isn't required now?

https://git.yoctoproject.org/poky/commit/?id=544c3255b1b1d2be7d7c77f0d64cc4a11a947887
https://git.yoctoproject.org/poky/commit/?id=fbc8f3d395711db00cae32d2502588ea9b61b1da

Cheers,

Richard

Re: [OE-core] [PATCH 3/3] systemd: generate manpages indexes

[Dan McGregor]

On Thu, 29 Sep 2022, Richard Purdie wrote:

> On Wed, 2022-09-28 at 16:13 -0600, Dan McGregor wrote:
>> From: Daniel McGregor <daniel.mcgregor@vecima.com>
>>
>> Now that python3-lxml is in core, use it to generate
>> systemd.directives and systemd.index manual pages.
>>
>> Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
>> ---
>>  meta/recipes-core/systemd/systemd_251.4.bb | 5 ++++-
>>  1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/systemd/systemd_251.4.bb b/meta/recipes-core/systemd/systemd_251.4.bb
>> index 910ea71bf64..b4ad7e45eb0 100644
>> --- a/meta/recipes-core/systemd/systemd_251.4.bb
>> +++ b/meta/recipes-core/systemd/systemd_251.4.bb
>> @@ -159,7 +159,10 @@ PACKAGECONFIG[localed] = "-Dlocaled=true,-Dlocaled=false"
>>  PACKAGECONFIG[logind] = "-Dlogind=true,-Dlogind=false"
>>  PACKAGECONFIG[lz4] = "-Dlz4=true,-Dlz4=false,lz4"
>>  PACKAGECONFIG[machined] = "-Dmachined=true,-Dmachined=false"
>> -PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,libxslt-native xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
>> +PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,python3-lxml-native xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
>> +# Ensure we're seeing the XML catalog from the build system and not the host.
>> +# Otherwise, the build may fail generating the manpages.
>> +export XML_CATALOG_FILES="${STAGING_ETCDIR_NATIVE}/xml/catalog"
>
> Wasn't there a fix that means this isn't required now?
>
> https://git.yoctoproject.org/poky/commit/?id=544c3255b1b1d2be7d7c77f0d64cc4a11a947887
> https://git.yoctoproject.org/poky/commit/?id=fbc8f3d395711db00cae32d2502588ea9b61b1da

Could be, I've been sitting on this change for *years*, but haven't 
submitted it because python3-lxml wasn't in core. Last time I tested 
without setting XML_CATALOG_FILES it failed, but now I see that I also 
removed libxslt-native from the PACKAGECONFIG depends list. I'll test 
with libxslt-native and without setting XML_CATALOG_FILES and probably 
send a v2.

>
> Cheers,
>
> Richard
>

Re: [OE-core] [PATCH 3/3] systemd: generate manpages indexes

[Dan McGregor]

On Thu, 29 Sep 2022, Dan McGregor wrote:

> On Thu, 29 Sep 2022, Richard Purdie wrote:
>
>> On Wed, 2022-09-28 at 16:13 -0600, Dan McGregor wrote:
>>> From: Daniel McGregor <daniel.mcgregor@vecima.com>
>>> 
>>> Now that python3-lxml is in core, use it to generate
>>> systemd.directives and systemd.index manual pages.
>>> 
>>> Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
>>> ---
>>>  meta/recipes-core/systemd/systemd_251.4.bb | 5 ++++-
>>>  1 file changed, 4 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/meta/recipes-core/systemd/systemd_251.4.bb 
>>> b/meta/recipes-core/systemd/systemd_251.4.bb
>>> index 910ea71bf64..b4ad7e45eb0 100644
>>> --- a/meta/recipes-core/systemd/systemd_251.4.bb
>>> +++ b/meta/recipes-core/systemd/systemd_251.4.bb
>>> @@ -159,7 +159,10 @@ PACKAGECONFIG[localed] = 
>>> "-Dlocaled=true,-Dlocaled=false"
>>>  PACKAGECONFIG[logind] = "-Dlogind=true,-Dlogind=false"
>>>  PACKAGECONFIG[lz4] = "-Dlz4=true,-Dlz4=false,lz4"
>>>  PACKAGECONFIG[machined] = "-Dmachined=true,-Dmachined=false"
>>> -PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,libxslt-native 
>>> xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
>>> +PACKAGECONFIG[manpages] = "-Dman=true,-Dman=false,python3-lxml-native 
>>> xmlto-native docbook-xml-dtd4-native docbook-xsl-stylesheets-native"
>>> +# Ensure we're seeing the XML catalog from the build system and not the 
>>> host.
>>> +# Otherwise, the build may fail generating the manpages.
>>> +export XML_CATALOG_FILES="${STAGING_ETCDIR_NATIVE}/xml/catalog"
>> 
>> Wasn't there a fix that means this isn't required now?
>> 
>> https://git.yoctoproject.org/poky/commit/?id=544c3255b1b1d2be7d7c77f0d64cc4a11a947887
>> https://git.yoctoproject.org/poky/commit/?id=fbc8f3d395711db00cae32d2502588ea9b61b1da
>
> Could be, I've been sitting on this change for *years*, but haven't submitted 
> it because python3-lxml wasn't in core. Last time I tested without setting 
> XML_CATALOG_FILES it failed, but now I see that I also removed libxslt-native 
> from the PACKAGECONFIG depends list. I'll test with libxslt-native and 
> without setting XML_CATALOG_FILES and probably send a v2.
>

Looking further, setting XML_CATALOG_FILES *is* required, because 
python3-lxml wraps the libxml2 libraries directly, not the command-line 
tools. For now I'd like to keep setting the variable, but we should 
probably find a way to handle the catalog path in the library itself 
rather than only wrapping the executables. Or finding a way to patch 
python3-lxml to use the build path's xml catalogue.