* [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
@ 2023-07-24 20:46 Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 2/5] package.bbclass: " Piotr Łobacz
` (4 more replies)
0 siblings, 5 replies; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-24 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Piotr Łobacz
Extend OPKGBUILDCMD variable, with additional parameters, depending
on target distro features, in order to support ACLs and xattr.
With fix pushed to the opkg-devel:
https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
opkg-build is able to create tar archives with ACLs and xattr.
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
meta/classes-global/package_ipk.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
index b4b7bc9ac2..a0f106e4ad 100644
--- a/meta/classes-global/package_ipk.bbclass
+++ b/meta/classes-global/package_ipk.bbclass
@@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
# Program to be used to build opkg packages
-OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
+OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [OE-Core][PATCH v10][master-next 2/5] package.bbclass: add support for ACLs and xattr
2023-07-24 20:46 [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
@ 2023-07-24 20:46 ` Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 3/5] opkg-utils: add acl and xattr support Piotr Łobacz
` (3 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-24 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Piotr Łobacz
Extend `tar` command, with additional parameters, depending
on choosen package class and target distro features, in order
to support ACLs and xattr.
Currently only `package_ipk` supports fully ACLs and xattr.
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
meta/classes-global/package.bbclass | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/meta/classes-global/package.bbclass b/meta/classes-global/package.bbclass
index e8055a9cdc..6e5d0dd4dc 100644
--- a/meta/classes-global/package.bbclass
+++ b/meta/classes-global/package.bbclass
@@ -342,8 +342,13 @@ python perform_packagecopy () {
# Start by package population by taking a copy of the installed
# files to operate on
- # Preserve sparse files and hard links
- cmd = 'tar --exclude=./sysroot-only -cf - -C %s -p -S . | tar -xf - -C %s' % (dest, dvar)
+ # Preserve sparse files, hard links, ACLs and extended attributes
+ # TODO: for the moment only ipk packages are supporting ACLs and extended attributes
+ # we need to add support for other package systems as well, but that doesn't bother
+ # tar from creating archives with acl and/or xattr support
+ acl = bb.utils.contains('DISTRO_FEATURES', 'acl', '--acls', '', d)
+ xattr = bb.utils.contains('DISTRO_FEATURES', 'xattr', '--xattrs', '', d)
+ cmd = f'tar {acl} {xattr} --numeric-owner --exclude=./sysroot-only -cf - -C {dest} -p -S . | tar {acl} {xattr} -xf - -C {dvar}'
subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
# replace RPATHs for the nativesdk binaries, to make them relocatable
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [OE-Core][PATCH v10][master-next 3/5] opkg-utils: add acl and xattr support
2023-07-24 20:46 [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 2/5] package.bbclass: " Piotr Łobacz
@ 2023-07-24 20:46 ` Piotr Łobacz
2023-08-04 15:27 ` Khem Raj
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 4/5] opkg: add options to enable support for acl and xattr Piotr Łobacz
` (2 subsequent siblings)
4 siblings, 1 reply; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-24 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Piotr Łobacz
Add support for tar archives created with --acls and/or --xattrs options,
PAX header format.
GNU tar and libarchive already supports ACLs and extended attributes.
We can now add this support as well to opkg-build script in order to use
fsetattr or setcap inside do_install command and end up with a file in
an image with the relevant ACLs and xattrs.
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
...kg-build-Add-acls-and-xattrs-support.patch | 164 ++++++++++++++++++
.../opkg-utils/opkg-utils_0.6.2.bb | 1 +
2 files changed, 165 insertions(+)
create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
new file mode 100644
index 0000000000..0874d3f75c
--- /dev/null
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
@@ -0,0 +1,164 @@
+From 5a5901f703bfac7376cfef3d4734c37400db03f1 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
+Date: Wed, 5 Jul 2023 10:31:13 +0200
+Subject: [PATCH] opkg-build: Add acls and xattrs support
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+Add support for tar archives created with --acls and/or --xattrs options,
+PAX header format.
+
+GNU tar and libarchive already supports ACLs and extended attributes.
+We can now add this support as well to opkg-build script in order to use
+fsetattr or setcap inside do_install command and end up with a file in
+an image with the relevant ACLs and xattrs.
+
+Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8]
+
+[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097
+[2] https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA
+
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ opkg-build | 76 +++++++++++++++++++++++++++++++++++-------------------
+ 1 file changed, 50 insertions(+), 26 deletions(-)
+
+diff --git a/opkg-build b/opkg-build
+index a9e45d4..b3127e0 100755
+--- a/opkg-build
++++ b/opkg-build
+@@ -145,6 +145,7 @@ You probably want to chown these to a system user: " >&2
+ ###
+ # opkg-build "main"
+ ###
++attributesargs=""
+ ogargs=""
+ outer=ar
+ noclean=0
+@@ -153,22 +154,6 @@ compressor=gzip
+ zipargs="-9n"
+ compressorargs=""
+
+-# Determine if tar supports the --format argument by checking the help output.
+-#
+-# This is needed because:
+-# - Busybox tar doesn't support '--format'
+-# - On some Linux distros, tar now defaults to posix format if '--format'
+-# isn't explicitly specified
+-# - Opkg doesn't currently support posix format archives
+-#
+-# It's easier to check for mention of the '--format' option than to detect the
+-# tar implementation and maintain a list of which support '--format'.
+-tarformat=""
+-if tar --help 2>&1 | grep -- "--format" > /dev/null;
+-then
+- tarformat="--format=gnu"
+-fi
+-
+ compressor_ext() {
+ case $1 in
+ gzip|pigz)
+@@ -197,13 +182,17 @@ compressor_ext() {
+ : <<=cut
+ =head1 SYNOPSIS
+
+-B<opkg-build> [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
++B<opkg-build> [B<-A>] [B<-X>] [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
+
+ =cut
+
+-usage="Usage: $0 [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
+-while getopts "a:cCg:ho:vOZ:" opt; do
++usage="Usage: $0 [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
++while getopts "Aa:cCg:ho:vOXZ:" opt; do
+ case $opt in
++ A ) attributesargs="--acls"
++ ;;
++ X ) attributesargs="$attributesargs --xattrs"
++ ;;
+ o ) owner=$OPTARG
+ ogargs="--owner=$owner"
+ ;;
+@@ -232,6 +221,31 @@ while getopts "a:cCg:ho:vOZ:" opt; do
+ esac
+ done
+
++# Determine if tar supports the --format argument by checking the help output.
++#
++# This is needed because:
++# - Busybox tar doesn't support '--format'
++# - On some Linux distros, tar now defaults to posix format if '--format'
++# isn't explicitly specified
++# - Opkg doesn't currently support posix format archives
++#
++# It's easier to check for mention of the '--format' option than to detect the
++# tar implementation and maintain a list of which support '--format'.
++tarformat=""
++if tar --help 2>&1 | grep -- "--format" > /dev/null;
++then
++ # For ACLs or xattr support, gnu format will not work
++ # we need to set posix format instead
++ if [ ! -z "$attributesargs" ] ; then
++ tarformat="--format=posix"
++ else
++ tarformat="--format=gnu"
++ fi
++elif [ ! -z "$attributesargs" ] ; then
++ echo "*** Error: Attributes: $attributesargs, doesn't' work, without posix format, which is not supported by tar command." >&2
++ exit 1
++fi
++
+ cext=$(compressor_ext $compressor)
+
+ # pgzip requires -T to avoid timestamps on the gzip archive
+@@ -301,21 +315,31 @@ fi
+ tmp_dir=$dest_dir/IPKG_BUILD.$$
+ mkdir $tmp_dir
+
+-build_date="${SOURCE_DATE_EPOCH:-$(date +%s)}"
+-
+-mtime_args=""
++mtime_args="--mtime=@${SOURCE_DATE_EPOCH:-$(date +%s)}"
+ # --clamp-mtime requires tar > 1.28. Only use it if SOURCE_DATE_EPOCH is set, to avoid having a generic case dependency on tar > 1.28.
+ # this setting will make sure files generated at build time have consistent mtimes, for reproducible builds.
+ if [ ! -z "$SOURCE_DATE_EPOCH" ]; then
+- mtime_args="--mtime=@$build_date --clamp-mtime"
++ mtime_args="$mtime_args --clamp-mtime"
++fi
++
++# Notice, that if you create an archive in POSIX format (see section GNU tar and POSIX tar) and the environment variable POSIXLY_CORRECT is set,
++# then the two archives created using the same options on the same set of files will not be byte-to-byte equivalent even with the above option.
++# This is because the posix default for extended header names includes the PID of the tar process, which is different at each run. To produce
++# byte-to-byte equivalent archives in this case, either unset POSIXLY_CORRECT, or use the following option:
++#
++# --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0
++#
++# https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html
++if [ $tarformat == "--format=posix" ]; then
++ mtime_args="$mtime_args --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0"
+ fi
+
+ export LANG=C
+ export LC_ALL=C
+ ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list )
+ ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print | sort > $tmp_dir/file_list )
+-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
+-( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
++( cd $pkg_dir && tar $attributesargs $ogargs $tsortargs --numeric-owner --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
++( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
+ rm $tmp_dir/file_list
+ rm $tmp_dir/control_list
+
+@@ -331,7 +355,7 @@ rm -f $pkg_file
+ if [ "$outer" = "ar" ] ; then
+ ( cd $tmp_dir && ar -crfD $pkg_file ./debian-binary ./control.tar.gz ./data.tar.$cext )
+ else
+- ( cd $tmp_dir && tar -c $tsortargs --mtime=@$build_date $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
++ ( cd $tmp_dir && tar -c $tsortargs $mtime_args $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
+ fi
+
+ rm $tmp_dir/debian-binary $tmp_dir/data.tar.$cext $tmp_dir/control.tar.gz
+--
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
index eb88b9b734..d5ce2cfbe2 100644
--- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
@@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu
SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \
file://0001-update-alternatives-correctly-match-priority.patch \
+ file://0002-opkg-build-Add-acls-and-xattrs-support.patch \
"
SRCREV = "67994e62dc598282830385da75ba9b1abbbda941"
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [OE-Core][PATCH v10][master-next 4/5] opkg: add options to enable support for acl and xattr
2023-07-24 20:46 [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 2/5] package.bbclass: " Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 3/5] opkg-utils: add acl and xattr support Piotr Łobacz
@ 2023-07-24 20:46 ` Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 5/5] opkg: set locale from system environment variables Piotr Łobacz
2023-07-24 20:50 ` ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
4 siblings, 0 replies; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-24 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Piotr Łobacz
The libarchive library, which is being used by opkg, supports ACLs
and xattr already.
More informations can be read at this link:
https://github.com/libarchive/libarchive/pull/691
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
...-to-enable-support-for-acl-and-xattr.patch | 70 +++++++++++++++++++
meta/recipes-devtools/opkg/opkg_0.6.2.bb | 5 +-
2 files changed, 74 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch
diff --git a/meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch b/meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch
new file mode 100644
index 0000000000..d6cb1d79fb
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch
@@ -0,0 +1,70 @@
+From 1c935e994bd572d9fff436f660ac1a060a434df0 Mon Sep 17 00:00:00 2001
+From: Maciej Liszewski <m.liszewski@welotec.com>
+Date: Tue, 4 Jul 2023 22:01:58 +0200
+Subject: [PATCH] Add options to enable support for acl and xattr
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+The libarchive library, which is being used by opkg, supports ACLs
+and xattr already.
+
+More informations can be read at this link:
+https://github.com/libarchive/libarchive/pull/691
+
+Upstream-Status: Accepted [https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA]
+
+[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097
+
+Signed-off-by: Maciej Liszewski <m.liszewski@welotec.com>
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ configure.ac | 12 ++++++++++++
+ libopkg/opkg_archive.c | 8 ++++++++
+ 2 files changed, 20 insertions(+)
+
+diff --git a/configure.ac b/configure.ac
+index 389a818..46949cd 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -158,6 +158,18 @@ return OPENSSL_VERSION_NUMBER; ],
+ AC_SUBST(OPENSSL_LIBS)
+ fi
+
++# check for ACL support
++AC_ARG_WITH([acl], [AS_HELP_STRING([--with-acl], [Enable ACL support])])
++if test "x$with_acl" = "xyes"; then
++ AC_DEFINE([ENABLE_ACL], [1], [Enable ACL support])
++fi
++
++# check for xattr support
++AC_ARG_WITH([xattr], [AS_HELP_STRING([--with-xattr], [Enable xattr support])])
++if test "x$with_xattr" = "xyes"; then
++ AC_DEFINE([ENABLE_XATTR], [1], [Enable xattr support])
++fi
++
+ # check for libsolv solver
+ AC_ARG_WITH(libsolv, AC_HELP_STRING([--with-libsolv], [Use libsolv solver support.
+ ]), [], [with_libsolv="no"])
+diff --git a/libopkg/opkg_archive.c b/libopkg/opkg_archive.c
+index 03a4afb..8dd902d 100644
+--- a/libopkg/opkg_archive.c
++++ b/libopkg/opkg_archive.c
+@@ -912,6 +912,14 @@ struct opkg_ar *ar_open_pkg_data_archive(const char *filename)
+ ar->extract_flags = ARCHIVE_EXTRACT_OWNER | ARCHIVE_EXTRACT_PERM |
+ ARCHIVE_EXTRACT_TIME | ARCHIVE_EXTRACT_UNLINK | ARCHIVE_EXTRACT_NO_OVERWRITE;
+
++#ifdef ENABLE_ACL
++ ar->extract_flags |= ARCHIVE_EXTRACT_ACL;
++#endif
++
++#ifdef ENABLE_XATTR
++ ar->extract_flags |= ARCHIVE_EXTRACT_FFLAGS | ARCHIVE_EXTRACT_XATTR;
++#endif
++
+ if (opkg_config->ignore_uid)
+ ar->extract_flags &= ~ARCHIVE_EXTRACT_OWNER;
+
+--
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.6.2.bb b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
index 46be137354..d7dc6ab715 100644
--- a/meta/recipes-devtools/opkg/opkg_0.6.2.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
@@ -15,6 +15,7 @@ PE = "1"
SRC_URI = "http://downloads.yoctoproject.org/releases/${BPN}/${BPN}-${PV}.tar.gz \
file://opkg.conf \
file://0001-opkg_conf-create-opkg.lock-in-run-instead-of-var-run.patch \
+ file://0002-Add-options-to-enable-support-for-acl-and-xattr.patch \
file://run-ptest \
"
@@ -30,8 +31,10 @@ inherit autotools pkgconfig ptest
target_localstatedir := "${localstatedir}"
OPKGLIBDIR ??= "${target_localstatedir}/lib"
-PACKAGECONFIG ??= "libsolv"
+PACKAGECONFIG ??= "libsolv ${@bb.utils.filter('DISTRO_FEATURES', 'acl xattr', d)}"
+PACKAGECONFIG[acl] = "--with-acl,--without-acl"
+PACKAGECONFIG[xattr] = "--with-xattr,--without-xattr"
PACKAGECONFIG[gpg] = "--enable-gpg,--disable-gpg,\
gnupg gpgme libgpg-error,\
${@ "gnupg" if ("native" in d.getVar("PN")) else "gnupg-gpg"}\
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [OE-Core][PATCH v10][master-next 5/5] opkg: set locale from system environment variables
2023-07-24 20:46 [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
` (2 preceding siblings ...)
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 4/5] opkg: add options to enable support for acl and xattr Piotr Łobacz
@ 2023-07-24 20:46 ` Piotr Łobacz
2023-07-24 20:50 ` ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
4 siblings, 0 replies; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-24 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Piotr Łobacz
A C program inherits its locale environment variables when it starts up.
This happens automatically. However, these variables do not automatically
control the locale used by the library functions, because ISO C says that
all programs start by default in the standard ‘C’ locale.
Fixes warnings:
Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84)
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
...le-from-system-environment-variables.patch | 48 +++++++++++++++++++
meta/recipes-devtools/opkg/opkg_0.6.2.bb | 1 +
2 files changed, 49 insertions(+)
create mode 100644 meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch
diff --git a/meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch b/meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch
new file mode 100644
index 0000000000..71240ec8fd
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch
@@ -0,0 +1,48 @@
+From 712895b1914bf63ee4d669863bfd106814329076 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
+Date: Wed, 19 Jul 2023 21:26:09 +0200
+Subject: [PATCH] opkg: set locale from system environment variables
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+A C program inherits its locale environment variables when it starts up.
+This happens automatically. However, these variables do not automatically
+control the locale used by the library functions, because ISO C says that
+all programs start by default in the standard ‘C’ locale.
+
+Fixes warnings:
+Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84)
+
+Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/16kgZfJ26mQ]
+
+[1] https://www.gnu.org/software/libc/manual/html_node/Setting-the-Locale.html
+
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ src/opkg.c | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/src/opkg.c b/src/opkg.c
+index 544c58a..0c729ff 100644
+--- a/src/opkg.c
++++ b/src/opkg.c
+@@ -27,6 +27,7 @@
+ #include <stdio.h>
+ #include <getopt.h>
+ #include <stdlib.h>
++#include <locale.h>
+
+ #include "opkg_conf.h"
+ #include "opkg_cmd.h"
+@@ -408,6 +409,7 @@ int main(int argc, char *argv[])
+ if (opkg_conf_init())
+ goto err0;
+
++ setlocale(LC_ALL, "");
+ opkg_config->verbosity = NOTICE;
+
+ opts = args_parse(argc, argv);
+--
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.6.2.bb b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
index d7dc6ab715..3b5d51d74a 100644
--- a/meta/recipes-devtools/opkg/opkg_0.6.2.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
@@ -16,6 +16,7 @@ SRC_URI = "http://downloads.yoctoproject.org/releases/${BPN}/${BPN}-${PV}.tar.gz
file://opkg.conf \
file://0001-opkg_conf-create-opkg.lock-in-run-instead-of-var-run.patch \
file://0002-Add-options-to-enable-support-for-acl-and-xattr.patch \
+ file://0003-opkg-set-locale-from-system-environment-variables.patch \
file://run-ptest \
"
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
2023-07-24 20:46 [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
` (3 preceding siblings ...)
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 5/5] opkg: set locale from system environment variables Piotr Łobacz
@ 2023-07-24 20:50 ` Piotr Łobacz
2023-07-25 9:28 ` Alexandre Belloni
4 siblings, 1 reply; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-24 20:50 UTC (permalink / raw)
To: openembedded-core@lists.openembedded.org; +Cc: alexandre.belloni@bootlin.com
Hello Alexandre,
this is a final patchest which works for me, even with reproducible builds, but additionally, with posix format of tar archives.
I hope that this time it will work for autobuild as well.
BR
Piotr
Od: Piotr Łobacz <p.lobacz@welotec.com>
Wysłane: poniedziałek, 24 lipca 2023 22:46
Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
DW: Piotr Łobacz <p.lobacz@welotec.com>
Temat: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
Extend OPKGBUILDCMD variable, with additional parameters, depending
on target distro features, in order to support ACLs and xattr.
With fix pushed to the opkg-devel:
https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
opkg-build is able to create tar archives with ACLs and xattr.
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
meta/classes-global/package_ipk.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
index b4b7bc9ac2..a0f106e4ad 100644
--- a/meta/classes-global/package_ipk.bbclass
+++ b/meta/classes-global/package_ipk.bbclass
@@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
# Program to be used to build opkg packages
-OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
+OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
2023-07-24 20:50 ` ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
@ 2023-07-25 9:28 ` Alexandre Belloni
2023-07-25 9:40 ` ODP: " Piotr Łobacz
[not found] ` <177512EFFC81235D.7114@lists.openembedded.org>
0 siblings, 2 replies; 14+ messages in thread
From: Alexandre Belloni @ 2023-07-25 9:28 UTC (permalink / raw)
To: Piotr Łobacz; +Cc: openembedded-core@lists.openembedded.org
Hello Piotr,
On 24/07/2023 20:50:50+0000, Piotr Łobacz wrote:
>
> Hello Alexandre,
> this is a final patchest which works for me, even with reproducible builds, but additionally, with posix format of tar archives.
>
> I hope that this time it will work for autobuild as well.
We are almost there, there are still reproducibility failures which I
guess where hidding in all the other other ones:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3251/steps/12/logs/stdio
This time, diffoscope ran:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230724-5av30jl8/packages/diff-html/
>
> BR
> Piotr
>
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: poniedziałek, 24 lipca 2023 22:46
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
>
> With fix pushed to the opkg-devel:
> https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
> opkg-build is able to create tar archives with ACLs and xattr.
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
> meta/classes-global/package_ipk.bbclass | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
> PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>
> # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>
> OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
> OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> --
> 2.34.1
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184818): https://lists.openembedded.org/g/openembedded-core/message/184818
> Mute This Topic: https://lists.openembedded.org/mt/100337925/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 14+ messages in thread
* ODP: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
2023-07-25 9:28 ` Alexandre Belloni
@ 2023-07-25 9:40 ` Piotr Łobacz
[not found] ` <177512EFFC81235D.7114@lists.openembedded.org>
1 sibling, 0 replies; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-25 9:40 UTC (permalink / raw)
To: Alexandre Belloni; +Cc: openembedded-core@lists.openembedded.org
Hmmm, this is odd because yesterday I have checked this and the additional parameters added to tar have fixed the issue for me.
But I have added them only for posix tar format, because for gnu format there is no need.
BTW. only these packages are different?
AssertionError: The following ipk packages are different and not in exclusion list:
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/acpid-src_2.0.34-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/avahi-src_0.8-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/bind-src_9.18.16-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/debugedit-src_5.0-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/flac-src_1.4.3-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-dev_1.6.6-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-ptest_1.6.6-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-src_1.6.6-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libmnl-src_1.0.5-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libuuid-src_2.38.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/log4cplus-src_2.1.0-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/lttng-tools-src_2.13.9-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-numpy-src_1.25.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-dev_3.44.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-src_3.44.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject_3.44.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/util-linux-ptest_2.38.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/util-linux-src_2.38.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/webkitgtk-src_2.40.2-r0_core2-64.ipk
Od: Alexandre Belloni <alexandre.belloni@bootlin.com>
Wysłane: wtorek, 25 lipca 2023 11:28
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Temat: Re: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
Hello Piotr,
On 24/07/2023 20:50:50+0000, Piotr Łobacz wrote:
>
> Hello Alexandre,
> this is a final patchest which works for me, even with reproducible builds, but additionally, with posix format of tar archives.
>
> I hope that this time it will work for autobuild as well.
We are almost there, there are still reproducibility failures which I
guess where hidding in all the other other ones:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3251/steps/12/logs/stdio
This time, diffoscope ran:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230724-5av30jl8/packages/diff-html/
>
> BR
> Piotr
>
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: poniedziałek, 24 lipca 2023 22:46
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
>
> With fix pushed to the opkg-devel:
> https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
> opkg-build is able to create tar archives with ACLs and xattr.
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
> meta/classes-global/package_ipk.bbclass | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
> PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>
> # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>
> OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
> OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> --
> 2.34.1
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184818): https://lists.openembedded.org/g/openembedded-core/message/184818
> Mute This Topic: https://lists.openembedded.org/mt/100337925/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com/
^ permalink raw reply [flat|nested] 14+ messages in thread
* ODP: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
[not found] ` <177512EFFC81235D.7114@lists.openembedded.org>
@ 2023-07-25 9:57 ` Piotr Łobacz
2023-07-25 10:28 ` Alexander Kanavin
0 siblings, 1 reply; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-25 9:57 UTC (permalink / raw)
To: Alexandre Belloni, Piotr Łobacz
Cc: openembedded-core@lists.openembedded.org
[-- Attachment #1: Type: text/plain, Size: 8387 bytes --]
Yeah, I have just checked it for acpid-src packages and diffoscope worked without any errors.
Question, is this test being run with acl or/and xattr? Because from what I remember it was not. Can you check?
If this will confirm, then something is still wrong with the gnu format, because right now all posix - meaning with acl/xattr support - packages are working for me.
BR
Piotr
________________________________
Od: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> w imieniu użytkownika Piotr Łobacz via lists.openembedded.org <p.lobacz=welotec.com@lists.openembedded.org>
Wysłane: wtorek, 25 lipca 2023 11:40
Do: Alexandre Belloni <alexandre.belloni@bootlin.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Temat: ODP: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
Hmmm, this is odd because yesterday I have checked this and the additional parameters added to tar have fixed the issue for me.
But I have added them only for posix tar format, because for gnu format there is no need.
BTW. only these packages are different?
AssertionError: The following ipk packages are different and not in exclusion list:
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/acpid-src_2.0.34-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/avahi-src_0.8-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/bind-src_9.18.16-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/debugedit-src_5.0-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/flac-src_1.4.3-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-dev_1.6.6-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-ptest_1.6.6-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-src_1.6.6-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libmnl-src_1.0.5-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libuuid-src_2.38.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/log4cplus-src_2.1.0-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/lttng-tools-src_2.13.9-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-numpy-src_1.25.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-dev_3.44.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-src_3.44.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject_3.44.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/util-linux-ptest_2.38.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/util-linux-src_2.38.1-r0_core2-64.ipk
/home/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/webkitgtk-src_2.40.2-r0_core2-64.ipk
Od: Alexandre Belloni <alexandre.belloni@bootlin.com>
Wysłane: wtorek, 25 lipca 2023 11:28
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Temat: Re: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
Hello Piotr,
On 24/07/2023 20:50:50+0000, Piotr Łobacz wrote:
>
> Hello Alexandre,
> this is a final patchest which works for me, even with reproducible builds, but additionally, with posix format of tar archives.
>
> I hope that this time it will work for autobuild as well.
We are almost there, there are still reproducibility failures which I
guess where hidding in all the other other ones:
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F117%2Fbuilds%2F3251%2Fsteps%2F12%2Flogs%2Fstdio&data=05%7C01%7Cp.lobacz%40welotec.com%7C591c597fa0a04e4ecb3f08db8cf340eb%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638258748639032081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XVprHSXzRygzFvdXHDDhEKLrqC%2FSN%2FC3Q8%2FyqNF2jE0%3D&reserved=0<https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3251/steps/12/logs/stdio>
This time, diffoscope ran:
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yocto.io%2Fpub%2Frepro-fail%2Foe-reproducible-20230724-5av30jl8%2Fpackages%2Fdiff-html%2F&data=05%7C01%7Cp.lobacz%40welotec.com%7C591c597fa0a04e4ecb3f08db8cf340eb%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638258748639032081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3ml4mQptghh889Urap%2BC5y%2BZKhzHXc7rV2anf7bQas4%3D&reserved=0<https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230724-5av30jl8/packages/diff-html/>
>
> BR
> Piotr
>
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: poniedziałek, 24 lipca 2023 22:46
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
>
> With fix pushed to the opkg-devel:
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fopkg-devel%2Fc%2FdYNHrLjDwg8&data=05%7C01%7Cp.lobacz%40welotec.com%7C591c597fa0a04e4ecb3f08db8cf340eb%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638258748639032081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=IWhy2Cj%2BWUcAlyUY9doZltLivjF9%2FMwHsfgNP%2BGtoLI%3D&reserved=0<https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8>
> opkg-build is able to create tar archives with ACLs and xattr.
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
> meta/classes-global/package_ipk.bbclass | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
> PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>
> # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>
> OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
> OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> --
> 2.34.1
>
>
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbootlin.com%2F&data=05%7C01%7Cp.lobacz%40welotec.com%7C591c597fa0a04e4ecb3f08db8cf340eb%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638258748639032081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=N0DPKQWxIBeTfnxUMZGJrpJjFD%2B4fMje%2FDvn37%2FJGpE%3D&reserved=0<https://bootlin.com/>
[-- Attachment #2: Type: text/html, Size: 11306 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
2023-07-25 9:57 ` Piotr Łobacz
@ 2023-07-25 10:28 ` Alexander Kanavin
2023-07-25 15:22 ` ODP: " Piotr Łobacz
[not found] ` <177525976001C949.23395@lists.openembedded.org>
0 siblings, 2 replies; 14+ messages in thread
From: Alexander Kanavin @ 2023-07-25 10:28 UTC (permalink / raw)
To: Piotr Łobacz
Cc: Alexandre Belloni, openembedded-core@lists.openembedded.org
On Tue, 25 Jul 2023 at 11:58, Piotr Łobacz <p.lobacz@welotec.com> wrote:
>
> Yeah, I have just checked it for acpid-src packages and diffoscope worked without any errors.
>
> Question, is this test being run with acl or/and xattr? Because from what I remember it was not. Can you check?
> If this will confirm, then something is still wrong with the gnu format, because right now all posix - meaning with acl/xattr support - packages are working for me.
Hello Piotr,
may I suggest: you need to learn to reproduce the autobuilder failures
locally. Integrators are stretched very thin, they juggle a ton of
patches (other than the ones you want to get in), and the expectation
is that when they tell you something fails, it is on you to do further
investigation. Here's how you do it:
- start by finding the failed build in the list of their definitions:
https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json
You can see that it's this one:
"reproducible" : {
"MACHINE" : "qemux86-64",
"SDKMACHINE" : "x86_64",
"step1" : {
"shortname" : "Reproducible Selftest",
"EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc;
OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail/
DISPLAY=:1 oe-selftest -r reproducible"],
"ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
}
What that means is you need to set up a plain poky build (no extra
layers, no custom configs specific to your organization), then run
'oe-selftest -r reproducible'. That is however a heavy test as it
builds everything, so we need to narrow it down to just the failing
pieces.
- the failure log shows that reproducibility is tested via
meta/lib/oeqa/selftest/cases/reproducible.py. If you open that file,
you can see OEQA_REPRODUCIBLE_TEST_TARGET narrows down the targets
used to test reproducibility, and you can also set
OEQA_REPRODUCIBLE_TEST_PACKAGE similarly.
- now you can run oe-selftest -r reproducible. Once it finishes and
confirms the failure, head to build-st/reproducibleA and
build-st/reproducibleB, to see where the difference first appears.
You can also replicate reproducibility test by manually setting up two
different build directories with identical configs.
Alex
^ permalink raw reply [flat|nested] 14+ messages in thread
* ODP: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
2023-07-25 10:28 ` Alexander Kanavin
@ 2023-07-25 15:22 ` Piotr Łobacz
[not found] ` <177525976001C949.23395@lists.openembedded.org>
1 sibling, 0 replies; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-25 15:22 UTC (permalink / raw)
To: Alexander Kanavin
Cc: Alexandre Belloni, openembedded-core@lists.openembedded.org
[-- Attachment #1: Type: text/plain, Size: 5487 bytes --]
Hi Alex,
at the beginning I would like to thank you for your suggestions, and I want to mention that this is what I wanted to do that from the very beginning,
but I could not find any good HowTo or manual how to do that. Right now, I have this poky-contrib:abelloni/master-next branch downloaded,
which also has the patches given by me.
Now, I'm trying to run it using these commands:
. ./oe-init-build-env build
and than:
oe-selftest -r reproducible
but for some reason it produces me an error:
raceback (most recent call last):
File "/home/plobacz/workspace/poky-contrib/scripts/oe-selftest", line 60, in <module>
ret = main()
File "/home/plobacz/workspace/poky-contrib/scripts/oe-selftest", line 47, in main
results = args.func(logger, args)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/selftest/context.py", line 406, in run
self._process_args(logger, args)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/selftest/context.py", line 252, in _process_args
bbvars = get_bb_vars()
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 248, in get_bb_vars
bbenv = get_bb_env(target, postconfig=postconfig)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 244, in get_bb_env
return bitbake("-e", postconfig=postconfig).output
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 234, in bitbake
return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 212, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake -e' returned non-zero exit status 1:
ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:
chrpath
But:
echo $PATH
/home/plobacz/workspace/poky-contrib/scripts:/home/plobacz/workspace/poky-contrib/bitbake/bin:/home/plobacz/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
Maybe I'm doing it somehow in wrong order?
BR
Piotr
________________________________
Od: Alexander Kanavin <alex.kanavin@gmail.com>
Wysłane: wtorek, 25 lipca 2023 12:28
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Temat: Re: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
On Tue, 25 Jul 2023 at 11:58, Piotr Łobacz <p.lobacz@welotec.com> wrote:
>
> Yeah, I have just checked it for acpid-src packages and diffoscope worked without any errors.
>
> Question, is this test being run with acl or/and xattr? Because from what I remember it was not. Can you check?
> If this will confirm, then something is still wrong with the gnu format, because right now all posix - meaning with acl/xattr support - packages are working for me.
Hello Piotr,
may I suggest: you need to learn to reproduce the autobuilder failures
locally. Integrators are stretched very thin, they juggle a ton of
patches (other than the ones you want to get in), and the expectation
is that when they tell you something fails, it is on you to do further
investigation. Here's how you do it:
- start by finding the failed build in the list of their definitions:
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.yoctoproject.org%2Fyocto-autobuilder-helper%2Ftree%2Fconfig.json&data=05%7C01%7Cp.lobacz%40welotec.com%7C29e17992f2b148e9206c08db8cf9e54c%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638258777149507734%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3czxntfPZq68MvQxKQtJT6Rk5la4O0cca9osxYGcVqI%3D&reserved=0<https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json>
You can see that it's this one:
"reproducible" : {
"MACHINE" : "qemux86-64",
"SDKMACHINE" : "x86_64",
"step1" : {
"shortname" : "Reproducible Selftest",
"EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc;
OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail/
DISPLAY=:1 oe-selftest -r reproducible"],
"ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
}
What that means is you need to set up a plain poky build (no extra
layers, no custom configs specific to your organization), then run
'oe-selftest -r reproducible'. That is however a heavy test as it
builds everything, so we need to narrow it down to just the failing
pieces.
- the failure log shows that reproducibility is tested via
meta/lib/oeqa/selftest/cases/reproducible.py. If you open that file,
you can see OEQA_REPRODUCIBLE_TEST_TARGET narrows down the targets
used to test reproducibility, and you can also set
OEQA_REPRODUCIBLE_TEST_PACKAGE similarly.
- now you can run oe-selftest -r reproducible. Once it finishes and
confirms the failure, head to build-st/reproducibleA and
build-st/reproducibleB, to see where the difference first appears.
You can also replicate reproducibility test by manually setting up two
different build directories with identical configs.
Alex
H
[-- Attachment #2: Type: text/html, Size: 14232 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* ODP: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
[not found] ` <177525976001C949.23395@lists.openembedded.org>
@ 2023-07-25 15:28 ` Piotr Łobacz
0 siblings, 0 replies; 14+ messages in thread
From: Piotr Łobacz @ 2023-07-25 15:28 UTC (permalink / raw)
To: Alexander Kanavin, Piotr Łobacz
Cc: Alexandre Belloni, openembedded-core@lists.openembedded.org
[-- Attachment #1: Type: text/plain, Size: 6180 bytes --]
Oh jesus I'm a morron I haven't noticed that I'm missing chrpath
Do not bother this messages:P
Cheers,
Piotr
________________________________
Od: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> w imieniu użytkownika Piotr Łobacz via lists.openembedded.org <p.lobacz=welotec.com@lists.openembedded.org>
Wysłane: wtorek, 25 lipca 2023 17:22
Do: Alexander Kanavin <alex.kanavin@gmail.com>
DW: Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Temat: ODP: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
Hi Alex,
at the beginning I would like to thank you for your suggestions, and I want to mention that this is what I wanted to do that from the very beginning,
but I could not find any good HowTo or manual how to do that. Right now, I have this poky-contrib:abelloni/master-next branch downloaded,
which also has the patches given by me.
Now, I'm trying to run it using these commands:
. ./oe-init-build-env build
and than:
oe-selftest -r reproducible
but for some reason it produces me an error:
raceback (most recent call last):
File "/home/plobacz/workspace/poky-contrib/scripts/oe-selftest", line 60, in <module>
ret = main()
File "/home/plobacz/workspace/poky-contrib/scripts/oe-selftest", line 47, in main
results = args.func(logger, args)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/selftest/context.py", line 406, in run
self._process_args(logger, args)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/selftest/context.py", line 252, in _process_args
bbvars = get_bb_vars()
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 248, in get_bb_vars
bbenv = get_bb_env(target, postconfig=postconfig)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 244, in get_bb_env
return bitbake("-e", postconfig=postconfig).output
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 234, in bitbake
return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
File "/home/plobacz/workspace/poky-contrib/meta/lib/oeqa/utils/commands.py", line 212, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake -e' returned non-zero exit status 1:
ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:
chrpath
But:
echo $PATH
/home/plobacz/workspace/poky-contrib/scripts:/home/plobacz/workspace/poky-contrib/bitbake/bin:/home/plobacz/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
Maybe I'm doing it somehow in wrong order?
BR
Piotr
________________________________
Od: Alexander Kanavin <alex.kanavin@gmail.com>
Wysłane: wtorek, 25 lipca 2023 12:28
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Temat: Re: ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
On Tue, 25 Jul 2023 at 11:58, Piotr Łobacz <p.lobacz@welotec.com> wrote:
>
> Yeah, I have just checked it for acpid-src packages and diffoscope worked without any errors.
>
> Question, is this test being run with acl or/and xattr? Because from what I remember it was not. Can you check?
> If this will confirm, then something is still wrong with the gnu format, because right now all posix - meaning with acl/xattr support - packages are working for me.
Hello Piotr,
may I suggest: you need to learn to reproduce the autobuilder failures
locally. Integrators are stretched very thin, they juggle a ton of
patches (other than the ones you want to get in), and the expectation
is that when they tell you something fails, it is on you to do further
investigation. Here's how you do it:
- start by finding the failed build in the list of their definitions:
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.yoctoproject.org%2Fyocto-autobuilder-helper%2Ftree%2Fconfig.json&data=05%7C01%7Cp.lobacz%40welotec.com%7C29e17992f2b148e9206c08db8cf9e54c%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638258777149507734%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3czxntfPZq68MvQxKQtJT6Rk5la4O0cca9osxYGcVqI%3D&reserved=0<https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json>
You can see that it's this one:
"reproducible" : {
"MACHINE" : "qemux86-64",
"SDKMACHINE" : "x86_64",
"step1" : {
"shortname" : "Reproducible Selftest",
"EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc;
OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail/
DISPLAY=:1 oe-selftest -r reproducible"],
"ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
}
What that means is you need to set up a plain poky build (no extra
layers, no custom configs specific to your organization), then run
'oe-selftest -r reproducible'. That is however a heavy test as it
builds everything, so we need to narrow it down to just the failing
pieces.
- the failure log shows that reproducibility is tested via
meta/lib/oeqa/selftest/cases/reproducible.py. If you open that file,
you can see OEQA_REPRODUCIBLE_TEST_TARGET narrows down the targets
used to test reproducibility, and you can also set
OEQA_REPRODUCIBLE_TEST_PACKAGE similarly.
- now you can run oe-selftest -r reproducible. Once it finishes and
confirms the failure, head to build-st/reproducibleA and
build-st/reproducibleB, to see where the difference first appears.
You can also replicate reproducibility test by manually setting up two
different build directories with identical configs.
Alex
H
[-- Attachment #2: Type: text/html, Size: 17045 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-Core][PATCH v10][master-next 3/5] opkg-utils: add acl and xattr support
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 3/5] opkg-utils: add acl and xattr support Piotr Łobacz
@ 2023-08-04 15:27 ` Khem Raj
2023-08-05 6:08 ` Khem Raj
0 siblings, 1 reply; 14+ messages in thread
From: Khem Raj @ 2023-08-04 15:27 UTC (permalink / raw)
To: Piotr Łobacz, openembedded-core
[-- Attachment #1.1.1: Type: text/plain, Size: 12831 bytes --]
On 7/24/23 1:46 PM, Piotr Łobacz wrote:
> Add support for tar archives created with --acls and/or --xattrs options,
> PAX header format.
>
> GNU tar and libarchive already supports ACLs and extended attributes.
> We can now add this support as well to opkg-build script in order to use
> fsetattr or setcap inside do_install command and end up with a file in
> an image with the relevant ACLs and xattrs.
this patch series is regressing ipk backends with errors like below
I am using archlinux for by build system if that matters.
ERROR: linux-libc-headers-6.4-r0 do_package_write_ipk: Fatal errors
occurred in subprocesses:
Command
'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools"
opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dbg
/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto'
returned non-zero exit status 1.
Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc '
Command
'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools"
opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dev
/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto'
returned non-zero exit status 1.
Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc '
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
> ...kg-build-Add-acls-and-xattrs-support.patch | 164 ++++++++++++++++++
> .../opkg-utils/opkg-utils_0.6.2.bb | 1 +
> 2 files changed, 165 insertions(+)
> create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
>
> diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
> new file mode 100644
> index 0000000000..0874d3f75c
> --- /dev/null
> +++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
> @@ -0,0 +1,164 @@
> +From 5a5901f703bfac7376cfef3d4734c37400db03f1 Mon Sep 17 00:00:00 2001
> +From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
> +Date: Wed, 5 Jul 2023 10:31:13 +0200
> +Subject: [PATCH] opkg-build: Add acls and xattrs support
> +MIME-Version: 1.0
> +Content-Type: text/plain; charset=UTF-8
> +Content-Transfer-Encoding: 8bit
> +
> +Add support for tar archives created with --acls and/or --xattrs options,
> +PAX header format.
> +
> +GNU tar and libarchive already supports ACLs and extended attributes.
> +We can now add this support as well to opkg-build script in order to use
> +fsetattr or setcap inside do_install command and end up with a file in
> +an image with the relevant ACLs and xattrs.
> +
> +Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8]
> +
> +[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097
> +[2] https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA
> +
> +Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> +---
> + opkg-build | 76 +++++++++++++++++++++++++++++++++++-------------------
> + 1 file changed, 50 insertions(+), 26 deletions(-)
> +
> +diff --git a/opkg-build b/opkg-build
> +index a9e45d4..b3127e0 100755
> +--- a/opkg-build
> ++++ b/opkg-build
> +@@ -145,6 +145,7 @@ You probably want to chown these to a system user: " >&2
> + ###
> + # opkg-build "main"
> + ###
> ++attributesargs=""
> + ogargs=""
> + outer=ar
> + noclean=0
> +@@ -153,22 +154,6 @@ compressor=gzip
> + zipargs="-9n"
> + compressorargs=""
> +
> +-# Determine if tar supports the --format argument by checking the help output.
> +-#
> +-# This is needed because:
> +-# - Busybox tar doesn't support '--format'
> +-# - On some Linux distros, tar now defaults to posix format if '--format'
> +-# isn't explicitly specified
> +-# - Opkg doesn't currently support posix format archives
> +-#
> +-# It's easier to check for mention of the '--format' option than to detect the
> +-# tar implementation and maintain a list of which support '--format'.
> +-tarformat=""
> +-if tar --help 2>&1 | grep -- "--format" > /dev/null;
> +-then
> +- tarformat="--format=gnu"
> +-fi
> +-
> + compressor_ext() {
> + case $1 in
> + gzip|pigz)
> +@@ -197,13 +182,17 @@ compressor_ext() {
> + : <<=cut
> + =head1 SYNOPSIS
> +
> +-B<opkg-build> [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
> ++B<opkg-build> [B<-A>] [B<-X>] [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
> +
> + =cut
> +
> +-usage="Usage: $0 [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
> +-while getopts "a:cCg:ho:vOZ:" opt; do
> ++usage="Usage: $0 [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
> ++while getopts "Aa:cCg:ho:vOXZ:" opt; do
> + case $opt in
> ++ A ) attributesargs="--acls"
> ++ ;;
> ++ X ) attributesargs="$attributesargs --xattrs"
> ++ ;;
> + o ) owner=$OPTARG
> + ogargs="--owner=$owner"
> + ;;
> +@@ -232,6 +221,31 @@ while getopts "a:cCg:ho:vOZ:" opt; do
> + esac
> + done
> +
> ++# Determine if tar supports the --format argument by checking the help output.
> ++#
> ++# This is needed because:
> ++# - Busybox tar doesn't support '--format'
> ++# - On some Linux distros, tar now defaults to posix format if '--format'
> ++# isn't explicitly specified
> ++# - Opkg doesn't currently support posix format archives
> ++#
> ++# It's easier to check for mention of the '--format' option than to detect the
> ++# tar implementation and maintain a list of which support '--format'.
> ++tarformat=""
> ++if tar --help 2>&1 | grep -- "--format" > /dev/null;
> ++then
> ++ # For ACLs or xattr support, gnu format will not work
> ++ # we need to set posix format instead
> ++ if [ ! -z "$attributesargs" ] ; then
> ++ tarformat="--format=posix"
> ++ else
> ++ tarformat="--format=gnu"
> ++ fi
> ++elif [ ! -z "$attributesargs" ] ; then
> ++ echo "*** Error: Attributes: $attributesargs, doesn't' work, without posix format, which is not supported by tar command." >&2
> ++ exit 1
> ++fi
> ++
> + cext=$(compressor_ext $compressor)
> +
> + # pgzip requires -T to avoid timestamps on the gzip archive
> +@@ -301,21 +315,31 @@ fi
> + tmp_dir=$dest_dir/IPKG_BUILD.$$
> + mkdir $tmp_dir
> +
> +-build_date="${SOURCE_DATE_EPOCH:-$(date +%s)}"
> +-
> +-mtime_args=""
> ++mtime_args="--mtime=@${SOURCE_DATE_EPOCH:-$(date +%s)}"
> + # --clamp-mtime requires tar > 1.28. Only use it if SOURCE_DATE_EPOCH is set, to avoid having a generic case dependency on tar > 1.28.
> + # this setting will make sure files generated at build time have consistent mtimes, for reproducible builds.
> + if [ ! -z "$SOURCE_DATE_EPOCH" ]; then
> +- mtime_args="--mtime=@$build_date --clamp-mtime"
> ++ mtime_args="$mtime_args --clamp-mtime"
> ++fi
> ++
> ++# Notice, that if you create an archive in POSIX format (see section GNU tar and POSIX tar) and the environment variable POSIXLY_CORRECT is set,
> ++# then the two archives created using the same options on the same set of files will not be byte-to-byte equivalent even with the above option.
> ++# This is because the posix default for extended header names includes the PID of the tar process, which is different at each run. To produce
> ++# byte-to-byte equivalent archives in this case, either unset POSIXLY_CORRECT, or use the following option:
> ++#
> ++# --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0
> ++#
> ++# https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html
> ++if [ $tarformat == "--format=posix" ]; then
> ++ mtime_args="$mtime_args --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0"
> + fi
> +
> + export LANG=C
> + export LC_ALL=C
> + ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list )
> + ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print | sort > $tmp_dir/file_list )
> +-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
> +-( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
> ++( cd $pkg_dir && tar $attributesargs $ogargs $tsortargs --numeric-owner --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
> ++( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
> + rm $tmp_dir/file_list
> + rm $tmp_dir/control_list
> +
> +@@ -331,7 +355,7 @@ rm -f $pkg_file
> + if [ "$outer" = "ar" ] ; then
> + ( cd $tmp_dir && ar -crfD $pkg_file ./debian-binary ./control.tar.gz ./data.tar.$cext )
> + else
> +- ( cd $tmp_dir && tar -c $tsortargs --mtime=@$build_date $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
> ++ ( cd $tmp_dir && tar -c $tsortargs $mtime_args $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
> + fi
> +
> + rm $tmp_dir/debian-binary $tmp_dir/data.tar.$cext $tmp_dir/control.tar.gz
> +--
> +2.34.1
> +
> diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
> index eb88b9b734..d5ce2cfbe2 100644
> --- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
> +++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
> @@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu
>
> SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \
> file://0001-update-alternatives-correctly-match-priority.patch \
> + file://0002-opkg-build-Add-acls-and-xattrs-support.patch \
> "
> SRCREV = "67994e62dc598282830385da75ba9b1abbbda941"
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184815): https://lists.openembedded.org/g/openembedded-core/message/184815
> Mute This Topic: https://lists.openembedded.org/mt/100337842/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 2613 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-Core][PATCH v10][master-next 3/5] opkg-utils: add acl and xattr support
2023-08-04 15:27 ` Khem Raj
@ 2023-08-05 6:08 ` Khem Raj
0 siblings, 0 replies; 14+ messages in thread
From: Khem Raj @ 2023-08-05 6:08 UTC (permalink / raw)
To: Piotr Łobacz, openembedded-core
On Fri, Aug 4, 2023 at 8:27 AM Khem Raj <raj.khem@gmail.com> wrote:
>
> On 7/24/23 1:46 PM, Piotr Łobacz wrote:
> > Add support for tar archives created with --acls and/or --xattrs options,
> > PAX header format.
> >
> > GNU tar and libarchive already supports ACLs and extended attributes.
> > We can now add this support as well to opkg-build script in order to use
> > fsetattr or setcap inside do_install command and end up with a file in
> > an image with the relevant ACLs and xattrs.
>
> this patch series is regressing ipk backends with errors like below
> I am using archlinux for by build system if that matters.
>
> ERROR: linux-libc-headers-6.4-r0 do_package_write_ipk: Fatal errors
> occurred in subprocesses:
> Command
> 'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools"
> opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dbg
> /mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto'
> returned non-zero exit status 1.
> Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc '
> Command
> 'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools"
> opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dev
> /mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto'
> returned non-zero exit status 1.
> Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc '
>
I think this is more related to building on hosts with glibc 2.38
rather than opkg issue. So this patch might not
be the cause of this problem but maybe something in pseudo or some such layer.
> >
> > Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> > ---
> > ...kg-build-Add-acls-and-xattrs-support.patch | 164 ++++++++++++++++++
> > .../opkg-utils/opkg-utils_0.6.2.bb | 1 +
> > 2 files changed, 165 insertions(+)
> > create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
> >
> > diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
> > new file mode 100644
> > index 0000000000..0874d3f75c
> > --- /dev/null
> > +++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
> > @@ -0,0 +1,164 @@
> > +From 5a5901f703bfac7376cfef3d4734c37400db03f1 Mon Sep 17 00:00:00 2001
> > +From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
> > +Date: Wed, 5 Jul 2023 10:31:13 +0200
> > +Subject: [PATCH] opkg-build: Add acls and xattrs support
> > +MIME-Version: 1.0
> > +Content-Type: text/plain; charset=UTF-8
> > +Content-Transfer-Encoding: 8bit
> > +
> > +Add support for tar archives created with --acls and/or --xattrs options,
> > +PAX header format.
> > +
> > +GNU tar and libarchive already supports ACLs and extended attributes.
> > +We can now add this support as well to opkg-build script in order to use
> > +fsetattr or setcap inside do_install command and end up with a file in
> > +an image with the relevant ACLs and xattrs.
> > +
> > +Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8]
> > +
> > +[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097
> > +[2] https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA
> > +
> > +Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> > +---
> > + opkg-build | 76 +++++++++++++++++++++++++++++++++++-------------------
> > + 1 file changed, 50 insertions(+), 26 deletions(-)
> > +
> > +diff --git a/opkg-build b/opkg-build
> > +index a9e45d4..b3127e0 100755
> > +--- a/opkg-build
> > ++++ b/opkg-build
> > +@@ -145,6 +145,7 @@ You probably want to chown these to a system user: " >&2
> > + ###
> > + # opkg-build "main"
> > + ###
> > ++attributesargs=""
> > + ogargs=""
> > + outer=ar
> > + noclean=0
> > +@@ -153,22 +154,6 @@ compressor=gzip
> > + zipargs="-9n"
> > + compressorargs=""
> > +
> > +-# Determine if tar supports the --format argument by checking the help output.
> > +-#
> > +-# This is needed because:
> > +-# - Busybox tar doesn't support '--format'
> > +-# - On some Linux distros, tar now defaults to posix format if '--format'
> > +-# isn't explicitly specified
> > +-# - Opkg doesn't currently support posix format archives
> > +-#
> > +-# It's easier to check for mention of the '--format' option than to detect the
> > +-# tar implementation and maintain a list of which support '--format'.
> > +-tarformat=""
> > +-if tar --help 2>&1 | grep -- "--format" > /dev/null;
> > +-then
> > +- tarformat="--format=gnu"
> > +-fi
> > +-
> > + compressor_ext() {
> > + case $1 in
> > + gzip|pigz)
> > +@@ -197,13 +182,17 @@ compressor_ext() {
> > + : <<=cut
> > + =head1 SYNOPSIS
> > +
> > +-B<opkg-build> [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
> > ++B<opkg-build> [B<-A>] [B<-X>] [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
> > +
> > + =cut
> > +
> > +-usage="Usage: $0 [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
> > +-while getopts "a:cCg:ho:vOZ:" opt; do
> > ++usage="Usage: $0 [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
> > ++while getopts "Aa:cCg:ho:vOXZ:" opt; do
> > + case $opt in
> > ++ A ) attributesargs="--acls"
> > ++ ;;
> > ++ X ) attributesargs="$attributesargs --xattrs"
> > ++ ;;
> > + o ) owner=$OPTARG
> > + ogargs="--owner=$owner"
> > + ;;
> > +@@ -232,6 +221,31 @@ while getopts "a:cCg:ho:vOZ:" opt; do
> > + esac
> > + done
> > +
> > ++# Determine if tar supports the --format argument by checking the help output.
> > ++#
> > ++# This is needed because:
> > ++# - Busybox tar doesn't support '--format'
> > ++# - On some Linux distros, tar now defaults to posix format if '--format'
> > ++# isn't explicitly specified
> > ++# - Opkg doesn't currently support posix format archives
> > ++#
> > ++# It's easier to check for mention of the '--format' option than to detect the
> > ++# tar implementation and maintain a list of which support '--format'.
> > ++tarformat=""
> > ++if tar --help 2>&1 | grep -- "--format" > /dev/null;
> > ++then
> > ++ # For ACLs or xattr support, gnu format will not work
> > ++ # we need to set posix format instead
> > ++ if [ ! -z "$attributesargs" ] ; then
> > ++ tarformat="--format=posix"
> > ++ else
> > ++ tarformat="--format=gnu"
> > ++ fi
> > ++elif [ ! -z "$attributesargs" ] ; then
> > ++ echo "*** Error: Attributes: $attributesargs, doesn't' work, without posix format, which is not supported by tar command." >&2
> > ++ exit 1
> > ++fi
> > ++
> > + cext=$(compressor_ext $compressor)
> > +
> > + # pgzip requires -T to avoid timestamps on the gzip archive
> > +@@ -301,21 +315,31 @@ fi
> > + tmp_dir=$dest_dir/IPKG_BUILD.$$
> > + mkdir $tmp_dir
> > +
> > +-build_date="${SOURCE_DATE_EPOCH:-$(date +%s)}"
> > +-
> > +-mtime_args=""
> > ++mtime_args="--mtime=@${SOURCE_DATE_EPOCH:-$(date +%s)}"
> > + # --clamp-mtime requires tar > 1.28. Only use it if SOURCE_DATE_EPOCH is set, to avoid having a generic case dependency on tar > 1.28.
> > + # this setting will make sure files generated at build time have consistent mtimes, for reproducible builds.
> > + if [ ! -z "$SOURCE_DATE_EPOCH" ]; then
> > +- mtime_args="--mtime=@$build_date --clamp-mtime"
> > ++ mtime_args="$mtime_args --clamp-mtime"
> > ++fi
> > ++
> > ++# Notice, that if you create an archive in POSIX format (see section GNU tar and POSIX tar) and the environment variable POSIXLY_CORRECT is set,
> > ++# then the two archives created using the same options on the same set of files will not be byte-to-byte equivalent even with the above option.
> > ++# This is because the posix default for extended header names includes the PID of the tar process, which is different at each run. To produce
> > ++# byte-to-byte equivalent archives in this case, either unset POSIXLY_CORRECT, or use the following option:
> > ++#
> > ++# --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0
> > ++#
> > ++# https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html
> > ++if [ $tarformat == "--format=posix" ]; then
> > ++ mtime_args="$mtime_args --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0"
> > + fi
> > +
> > + export LANG=C
> > + export LC_ALL=C
> > + ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list )
> > + ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print | sort > $tmp_dir/file_list )
> > +-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
> > +-( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
> > ++( cd $pkg_dir && tar $attributesargs $ogargs $tsortargs --numeric-owner --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
> > ++( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
> > + rm $tmp_dir/file_list
> > + rm $tmp_dir/control_list
> > +
> > +@@ -331,7 +355,7 @@ rm -f $pkg_file
> > + if [ "$outer" = "ar" ] ; then
> > + ( cd $tmp_dir && ar -crfD $pkg_file ./debian-binary ./control.tar.gz ./data.tar.$cext )
> > + else
> > +- ( cd $tmp_dir && tar -c $tsortargs --mtime=@$build_date $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
> > ++ ( cd $tmp_dir && tar -c $tsortargs $mtime_args $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
> > + fi
> > +
> > + rm $tmp_dir/debian-binary $tmp_dir/data.tar.$cext $tmp_dir/control.tar.gz
> > +--
> > +2.34.1
> > +
> > diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
> > index eb88b9b734..d5ce2cfbe2 100644
> > --- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
> > +++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
> > @@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu
> >
> > SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \
> > file://0001-update-alternatives-correctly-match-priority.patch \
> > + file://0002-opkg-build-Add-acls-and-xattrs-support.patch \
> > "
> > SRCREV = "67994e62dc598282830385da75ba9b1abbbda941"
> >
> >
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#184815): https://lists.openembedded.org/g/openembedded-core/message/184815
> > Mute This Topic: https://lists.openembedded.org/mt/100337842/1997914
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2023-08-05 6:09 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-07-24 20:46 [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 2/5] package.bbclass: " Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 3/5] opkg-utils: add acl and xattr support Piotr Łobacz
2023-08-04 15:27 ` Khem Raj
2023-08-05 6:08 ` Khem Raj
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 4/5] opkg: add options to enable support for acl and xattr Piotr Łobacz
2023-07-24 20:46 ` [OE-Core][PATCH v10][master-next 5/5] opkg: set locale from system environment variables Piotr Łobacz
2023-07-24 20:50 ` ODP: [OE-Core][PATCH v10][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr Piotr Łobacz
2023-07-25 9:28 ` Alexandre Belloni
2023-07-25 9:40 ` ODP: " Piotr Łobacz
[not found] ` <177512EFFC81235D.7114@lists.openembedded.org>
2023-07-25 9:57 ` Piotr Łobacz
2023-07-25 10:28 ` Alexander Kanavin
2023-07-25 15:22 ` ODP: " Piotr Łobacz
[not found] ` <177525976001C949.23395@lists.openembedded.org>
2023-07-25 15:28 ` Piotr Łobacz
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox