Re: [OE-core] [PATCH v3] rpm: Allow setting platform macro settings externally

[Alexandre Belloni <alexandre.belloni@bootlin.com>]

Most of the builds are breaking with:

ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/genericx86-alt/build/build/tmp/work/core2-32-poky-linux/rpm/1_4.18.1-r0/temp/log.do_install.3923248
Log data follows:
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-core/dbus/dbus_1.14.8.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-core/glibc/glibc_2.37.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/elfutils/elfutils_0.189.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/file/file_5.44.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/gcc/gcc-cross_13.1.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/gcc/gcc-runtime_13.1.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/libtool/libtool-cross_2.4.7.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/libtool/libtool-native_2.4.7.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/lua/lua_5.4.6.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/python/python3_3.11.4.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-extended/xz/xz_5.4.3.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-extended/zstd/zstd_1.5.5.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-support/libgcrypt/libgcrypt_1.10.2.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-support/popt/popt_1.19.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-support/sqlite/sqlite3_3.42.0.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-core/gettext/gettext_0.21.1.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/autoconf/autoconf_2.72c.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/automake/automake_1.16.5.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-devtools/python/python3_3.11.4.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['dbus', 'glibc', 'elfutils', 'file', 'gcc-cross-i686', 'gcc-runtime', 'libtool-cross', 'libtool-native', 'lua', 'python3', 'quilt-native', 'bzip2', 'xz', 'zstd', 'libgcrypt', 'popt', 'sqlite3', 'gettext-native', 'autoconf-native', 'automake-native', 'patch-native', 'pkgconfig-native', 'pseudo-native', 'python3-native', 'opkg-utils', 'zlib', 'binutils-cross-i686', 'texinfo-dummy-native', 'readline', 'linux-libc-headers', 'libgcc', 'xz-native', 'libmpc-native', 'gmp-native', 'zstd-native', 'mpfr-native', 'flex-native', 'gnu-config-native', 'zlib-native', 'gettext-minimal-native', 'attr-native', 'base-passwd', 'glib-2.0', 'libx11', 'base-files', 'shadow-sysroot', 'shadow', 'expat', 'systemd', 'shadow-native', 'libsm', 'm4-native', 'openssl-native', 'libffi-native', 'gdbm-native', 'ncurses-native', 'bzip2-native', 'libtirpc-native', 'sqlite3-native', 'libnsl2-native', 'util-linux-libuuid-native', 'libedit-native', 'expat-native', 'libffi', 'libxcrypt', 'ncurses', 'libnsl2', 'openssl', 'util-linux-libuuid', 'libedit', 'libtirpc', 'gdbm', 'curl', 'libmicrohttpd', 'libarchive', 'libgpg-error', 'libcap', 'libpcre2', 'util-linux', 'bash-completion', 'xtrans', 'xorgproto', 'util-macros', 'libxcb', 'libpam', 'attr', 'kmod', 'libseccomp', 'libxkbcommon', 'acl', 'libice', 'perl-native', 'cmake-native', 'libidn2', 'gnutls', 'libcap-ng', 'libpthread-stubs', 'libxau', 'libxdmcp', 'xcb-proto', 'cracklib', 'wayland-protocols', 'libxml2', 'wayland', 'xkeyboard-config', 'make-native', 'libunistring', 'libtasn1', 'gmp', 'nettle']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing python function autotools_aclocals
| DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i686-linux', 'common']
| DEBUG: Python function autotools_aclocals finished
| DEBUG: Executing shell function do_install
| /home/pokybuild/yocto-worker/genericx86-alt/build/build/tmp/work/core2-32-poky-linux/rpm/1_4.18.1-r0/temp/run.do_install.3923248: line 163: export: `=': not a valid identifier
| WARNING: /home/pokybuild/yocto-worker/genericx86-alt/build/build/tmp/work/core2-32-poky-linux/rpm/1_4.18.1-r0/temp/run.do_install.3923248:163 exit 1 from 'export RPM_CUSTOM_ARCH = "genericx86"'
| WARNING: Backtrace (BB generated script):
| 	#1: do_install, /home/pokybuild/yocto-worker/genericx86-alt/build/build/tmp/work/core2-32-poky-linux/rpm/1_4.18.1-r0/temp/run.do_install.3923248, line 163
| 	#2: main, /home/pokybuild/yocto-worker/genericx86-alt/build/build/tmp/work/core2-32-poky-linux/rpm/1_4.18.1-r0/temp/run.do_install.3923248, line 220
NOTE: recipe rpm-1_4.18.1-r0: task do_install: Failed

On 27/07/2023 09:43:07+0200, Zoltan Boszormenyi wrote:
> Feed platform settings to installplatform externally. Based on the patch
> submitted under https://github.com/rpm-software-management/rpm/pull/2585
> 
> Patch against INSTALL was backported for rpm 4.18.1
> 
> Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
> ---
>  ...g-platform-macro-settings-externally.patch | 107 ++++++++++++++++++
>  meta/recipes-devtools/rpm/rpm_4.18.1.bb       |  20 ++++
>  2 files changed, 127 insertions(+)
>  create mode 100644 meta/recipes-devtools/rpm/files/0001-Allow-setting-platform-macro-settings-externally.patch
> 
> diff --git a/meta/recipes-devtools/rpm/files/0001-Allow-setting-platform-macro-settings-externally.patch b/meta/recipes-devtools/rpm/files/0001-Allow-setting-platform-macro-settings-externally.patch
> new file mode 100644
> index 0000000000..86d357a4e8
> --- /dev/null
> +++ b/meta/recipes-devtools/rpm/files/0001-Allow-setting-platform-macro-settings-externally.patch
> @@ -0,0 +1,107 @@
> +From 12ae0bf5d1540695af0a3499fc88698c48bc20d3 Mon Sep 17 00:00:00 2001
> +From: =?UTF-8?q?Zolt=C3=A1n=20B=C3=B6sz=C3=B6rm=C3=A9nyi?=
> + <zboszor@gmail.com>
> +Date: Tue, 25 Jul 2023 10:56:44 +0200
> +Subject: [PATCH] Allow setting platform macro settings externally
> +MIME-Version: 1.0
> +Content-Type: text/plain; charset=UTF-8
> +Content-Transfer-Encoding: 8bit
> +
> +By default, rpm installs a series of default platforms based on
> +the CPU architecture names in subdirectories called
> +
> +    /usr/lib/platform/<arch>-linux
> +
> +This is enough for regular Linux distributions. However, some
> +distributions may use more specific platform names that refer to
> +particular computer systems, like SBCs or specific CPU tuning when
> +compiling.
> +
> +If the platform subdirectory does not exist in /usr/lib/platform
> +then rpmbuild does not work.
> +
> +Allow creating such custom platform subdirectory with feeding
> +the necessary data using external variables: RPM_CUSTOM_ARCH,
> +RPM_CUSTOM_ISANAME, RPM_CUSTOM_ISABITS, RPM_CUSTOM_CANONARCH
> +and RPM_CUSTOM_CANONCOLOR
> +
> +Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
> +Upstream-Status: Submitted [https://github.com/rpm-software-management/rpm/pull/2585]
> +---
> + INSTALL         | 32 ++++++++++++++++++++++++++++++++
> + installplatform |  9 ++++++++-
> + 2 files changed, 40 insertions(+), 1 deletion(-)
> +
> +diff --git a/INSTALL b/INSTALL
> +index 13d0d8a83..35ad10793 100644
> +--- a/INSTALL
> ++++ b/INSTALL
> +@@ -148,6 +148,38 @@ and then install with:
> + 
> +     make install
> + 
> ++By default, rpm installs a series of default platforms based on the CPU
> ++architecture names in subdirectories called
> ++
> ++    /usr/lib/platform/<arch>-linux
> ++
> ++This is enough for regular Linux distributions. However, some distributions
> ++may use more specific platform names that refer to particular computer
> ++systems, like SBCs or specific CPU tuning when compiling. Examples of such
> ++platform names are: "genericx86_64", "intel_skylake_64", "raspberrypi_armv7",
> ++"raspberrypi_armv8", etc.
> ++
> ++If the platform name is put into /etc/rpm/platform, then rpmbuild uses it
> ++and the only macros file rpmbuild looks for is
> ++
> ++    /usr/lib/platform/`cat /etc/rpm/platform`-linux/macros
> ++
> ++If this file does not exist, many rpm macros will not have their expected
> ++values set and e.g. %configure will fail when trying to run rpmbuild.
> ++
> ++To allow creating the macros file for such a custom platform, the shell
> ++variables listed below must be set. If RPM_CUSTOM_ARCH is not set, the rest
> ++is ignored.
> ++
> ++    export RPM_CUSTOM_ARCH=genericx86_64
> ++    export RPM_CUSTOM_ISANAME=x86
> ++    export RPM_CUSTOM_ISABITS=64
> ++    export RPM_CUSTOM_CANONARCH=x86_64
> ++    export RPM_CUSTOM_CANONCOLOR=0 # to use /usr/lib for %_libdir
> ++    export RPM_CUSTOM_CANONCOLOR=3 # to use /usr/lib64 for %_libdir
> ++
> ++    make install # this also installs /usr/lib/platform/genericx86_64-linux/macros
> ++
> + Rpm comes with an automated self-test suite. The test-suite relies heavily
> + on fakechroot (https://github.com/dex4er/fakechroot/) and cannot be executed
> + without it. Provided that fakechroot was found during configure,
> +diff --git a/installplatform b/installplatform
> +index a5ad7c5b8..59f57697b 100755
> +--- a/installplatform
> ++++ b/installplatform
> +@@ -11,7 +11,7 @@ VENDOR="${4}"
> + OS="${5}"
> + RPMRC_GNU="${6}"
> + 
> +-for ARCH in noarch `grep ^arch_canon $RPMRC | cut -d: -f2`; do
> ++for ARCH in noarch `grep ^arch_canon $RPMRC | cut -d: -f2` ${RPM_CUSTOM_ARCH:+custom}; do
> +   RPMRC_OPTFLAGS="`sed -n 's/^optflags: '$ARCH' //p' $RPMRC`"
> +   RPMRC_OPTFLAGS="`echo $RPMRC_OPTFLAGS | sed -e 's, ,\ ,g'`"
> +   case $RPMRC_OPTFLAGS in
> +@@ -30,6 +30,13 @@ for ARCH in noarch `grep ^arch_canon $RPMRC | cut -d: -f2`; do
> +   CANONCOLOR=
> +   FILTER=cat
> +   case "${ARCH}" in
> ++    custom)
> ++	ARCH=$RPM_CUSTOM_ARCH
> ++	ISANAME=$RPM_CUSTOM_ISANAME
> ++	ISABITS=$RPM_CUSTOM_ISABITS
> ++	CANONARCH=$RPM_CUSTOM_CANONARCH
> ++	CANONCOLOR=$RPM_CUSTOM_CANONCOLOR
> ++    ;;
> +     sparc64*) 
> + 	ISANAME=sparc
> + 	ISABITS=64
> +-- 
> +2.41.0
> +
> diff --git a/meta/recipes-devtools/rpm/rpm_4.18.1.bb b/meta/recipes-devtools/rpm/rpm_4.18.1.bb
> index 95a9e92f96..97dd843e56 100644
> --- a/meta/recipes-devtools/rpm/rpm_4.18.1.bb
> +++ b/meta/recipes-devtools/rpm/rpm_4.18.1.bb
> @@ -40,6 +40,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm;branch=rpm-4.18.x;protoc
>             file://0001-python-Use-Py_hash_t-instead-of-long-in-hdr_hash.patch \
>             file://fix-declaration.patch \
>             file://ea3187cfcf9cac87e5bc5e7db79b0338da9e355e.patch \
> +           file://0001-Allow-setting-platform-macro-settings-externally.patch \
>             "
>  
>  PE = "1"
> @@ -128,10 +129,29 @@ do_install:append:class-nativesdk() {
>  	EOF
>  }
>  
> +def rpm_isaname(d):
> +    import re
> +    arch = d.getVar('TARGET_ARCH')
> +    if re.match("^i.86$", arch) or re.match("^x86.*64$", arch):
> +        return "x86"
> +    # Add more platform tweaks for ISANAME as needed
> +    return arch
> +
> +# Add the custom BSP platform "macros" file
> +do_install:prepend:class-target() {
> +    export RPM_CUSTOM_ARCH = "${MACHINE_ARCH}"
> +    export RPM_CUSTOM_ISANAME = "${@rpm_isaname(d)}"
> +    export RPM_CUSTOM_ISABITS = "${SITEINFO_BITS}"
> +    export RPM_CUSTOM_CANONARCH = "${TARGET_ARCH}"
> +    # CANONCOLOR determines whether /usr/lib or /usr/lib64 is used for a 64-bit platform
> +    export RPM_CUSTOM_CANONCOLOR = "${@bb.utils.contains('DISTRO_FEATURES', 'multilib', '3', '0', d)}"
> +}
> +
>  # Rpm's make install creates var/tmp which clashes with base-files packaging
>  do_install:append:class-target() {
>      rm -rf ${D}/var
>  }
> +
>  do_install:append:class-nativesdk() {
>      rm -rf ${D}${SDKPATHNATIVE}/var
>      # Ensure find-debuginfo is located correctly inside SDK
> -- 
> 2.41.0
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184916): https://lists.openembedded.org/g/openembedded-core/message/184916
> Mute This Topic: https://lists.openembedded.org/mt/100386513/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com