[OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

Extend OPKGBUILDCMD variable, with additional parameters, depending
on target distro features, in order to support ACLs and xattr.

With fix pushed to the opkg-devel:
https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
opkg-build is able to create tar archives with ACLs and xattr.

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 meta/classes-global/package_ipk.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
index b4b7bc9ac2..a0f106e4ad 100644
--- a/meta/classes-global/package_ipk.bbclass
+++ b/meta/classes-global/package_ipk.bbclass
@@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
 PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
 
 # Program to be used to build opkg packages
-OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
+OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
 
 OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
 OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
-- 
2.34.1

[OE-Core][PATCH v11][master-next 2/5] package.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

Extend `tar` command, with additional parameters, depending
on choosen package class and target distro features, in order
to support ACLs and xattr.

Currently only `package_ipk` supports fully ACLs and xattr.

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 meta/classes-global/package.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/classes-global/package.bbclass b/meta/classes-global/package.bbclass
index e8055a9cdc..6e5d0dd4dc 100644
--- a/meta/classes-global/package.bbclass
+++ b/meta/classes-global/package.bbclass
@@ -342,8 +342,13 @@ python perform_packagecopy () {
 
     # Start by package population by taking a copy of the installed
     # files to operate on
-    # Preserve sparse files and hard links
-    cmd = 'tar --exclude=./sysroot-only -cf - -C %s -p -S . | tar -xf - -C %s' % (dest, dvar)
+    # Preserve sparse files, hard links, ACLs and extended attributes
+    # TODO: for the moment only ipk packages are supporting ACLs and extended attributes
+    # we need to add support for other package systems as well, but that doesn't bother
+    # tar from creating archives with acl and/or xattr support
+    acl = bb.utils.contains('DISTRO_FEATURES', 'acl', '--acls', '', d)
+    xattr = bb.utils.contains('DISTRO_FEATURES', 'xattr', '--xattrs', '', d)
+    cmd = f'tar {acl} {xattr} --numeric-owner --exclude=./sysroot-only -cf - -C {dest} -p -S . | tar {acl} {xattr} -xf - -C {dvar}'
     subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 
     # replace RPATHs for the nativesdk binaries, to make them relocatable
-- 
2.34.1

[OE-Core][PATCH v11][master-next 3/5] opkg-utils: add acl and xattr support

[Piotr Łobacz]

Add support for tar archives created with --acls and/or --xattrs options,
PAX header format.

GNU tar and libarchive already supports ACLs and extended attributes.
We can now add this support as well to opkg-build script in order to use
fsetattr or setcap inside do_install command and end up with a file in
an image with the relevant ACLs and xattrs.

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 ...kg-build-Add-acls-and-xattrs-support.patch | 164 ++++++++++++++++++
 .../opkg-utils/opkg-utils_0.6.2.bb            |   1 +
 2 files changed, 165 insertions(+)
 create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch

diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
new file mode 100644
index 0000000000..7e88c1754c
--- /dev/null
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch
@@ -0,0 +1,164 @@
+From 03931040018a0e3cc34e4c93a625f3671ff1a980 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
+Date: Wed, 5 Jul 2023 10:31:13 +0200
+Subject: [PATCH] opkg-build: Add acls and xattrs support
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+Add support for tar archives created with --acls and/or --xattrs options,
+PAX header format.
+
+GNU tar and libarchive already supports ACLs and extended attributes.
+We can now add this support as well to opkg-build script in order to use
+fsetattr or setcap inside do_install command and end up with a file in
+an image with the relevant ACLs and xattrs.
+
+Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8]
+
+[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097
+[2] https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA
+
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ opkg-build | 76 +++++++++++++++++++++++++++++++++++-------------------
+ 1 file changed, 50 insertions(+), 26 deletions(-)
+
+diff --git a/opkg-build b/opkg-build
+index a9e45d4..47ac1a8 100755
+--- a/opkg-build
++++ b/opkg-build
+@@ -145,6 +145,7 @@ You probably want to chown these to a system user: " >&2
+ ###
+ # opkg-build "main"
+ ###
++attributesargs=""
+ ogargs=""
+ outer=ar
+ noclean=0
+@@ -153,22 +154,6 @@ compressor=gzip
+ zipargs="-9n"
+ compressorargs=""
+ 
+-# Determine if tar supports the --format argument by checking the help output.
+-#
+-# This is needed because:
+-#    - Busybox tar doesn't support '--format'
+-#    - On some Linux distros, tar now defaults to posix format if '--format'
+-#      isn't explicitly specified
+-#    - Opkg doesn't currently support posix format archives
+-#
+-# It's easier to check for mention of the '--format' option than to detect the
+-# tar implementation and maintain a list of which support '--format'.
+-tarformat=""
+-if tar --help 2>&1 | grep -- "--format" > /dev/null;
+-then
+-    tarformat="--format=gnu"
+-fi
+-
+ compressor_ext() {
+     case $1 in
+ 	gzip|pigz)
+@@ -197,13 +182,17 @@ compressor_ext() {
+ : <<=cut
+ =head1 SYNOPSIS
+ 
+-B<opkg-build> [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
++B<opkg-build> [B<-A>] [B<-X>] [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>]
+ 
+ =cut
+ 
+-usage="Usage: $0 [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
+-while getopts "a:cCg:ho:vOZ:" opt; do
++usage="Usage: $0 [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]"
++while getopts "Aa:cCg:ho:vOXZ:" opt; do
+     case $opt in
++        A ) attributesargs="--acls"
++            ;;
++        X ) attributesargs="$attributesargs --xattrs"
++            ;;
+ 	o ) owner=$OPTARG
+ 	    ogargs="--owner=$owner"
+ 	    ;;
+@@ -232,6 +221,31 @@ while getopts "a:cCg:ho:vOZ:" opt; do
+     esac
+ done
+ 
++# Determine if tar supports the --format argument by checking the help output.
++#
++# This is needed because:
++#    - Busybox tar doesn't support '--format'
++#    - On some Linux distros, tar now defaults to posix format if '--format'
++#      isn't explicitly specified
++#    - Opkg doesn't currently support posix format archives
++#
++# It's easier to check for mention of the '--format' option than to detect the
++# tar implementation and maintain a list of which support '--format'.
++tarformat=""
++if tar --help 2>&1 | grep -- "--format" > /dev/null;
++then
++    # For ACLs or xattr support, gnu format will not work
++    # we need to set posix format instead
++    if [ ! -z "$attributesargs" ] ; then
++	    tarformat="--format=posix"
++    else
++	    tarformat="--format=gnu"
++    fi
++elif [ ! -z "$attributesargs" ] ; then
++	echo "*** Error: Attributes: $attributesargs, doesn't' work, without posix format, which is not supported by tar command." >&2
++	exit 1
++fi
++
+ cext=$(compressor_ext $compressor)
+ 
+ # pgzip requires -T to avoid timestamps on the gzip archive
+@@ -301,21 +315,31 @@ fi
+ tmp_dir=$dest_dir/IPKG_BUILD.$$
+ mkdir $tmp_dir
+ 
+-build_date="${SOURCE_DATE_EPOCH:-$(date +%s)}"
+-
+-mtime_args=""
++mtime_args="--mtime=@${SOURCE_DATE_EPOCH:-$(date +%s)}"
+ # --clamp-mtime requires tar > 1.28. Only use it if SOURCE_DATE_EPOCH is set, to avoid having a generic case dependency on tar > 1.28.
+ # this setting will make sure files generated at build time have consistent mtimes, for reproducible builds.
+ if [ ! -z "$SOURCE_DATE_EPOCH"  ]; then
+-    mtime_args="--mtime=@$build_date --clamp-mtime"
++    mtime_args="$mtime_args --clamp-mtime"
++fi
++
++# Notice, that if you create an archive in POSIX format (see section GNU tar and POSIX tar) and the environment variable POSIXLY_CORRECT is set,
++# then the two archives created using the same options on the same set of files will not be byte-to-byte equivalent even with the above option.
++# This is because the posix default for extended header names includes the PID of the tar process, which is different at each run. To produce
++# byte-to-byte equivalent archives in this case, either unset POSIXLY_CORRECT, or use the following option:
++#
++# --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0
++#
++# https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html
++if [[ "$tarformat" == "--format=posix" ]]; then
++    mtime_args="$mtime_args --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0"
+ fi
+ 
+ export LANG=C
+ export LC_ALL=C
+ ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list )
+ ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print  | sort > $tmp_dir/file_list )
+-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
+-( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
++( cd $pkg_dir && tar $attributesargs $ogargs $tsortargs --numeric-owner --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
++( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
+ rm $tmp_dir/file_list
+ rm $tmp_dir/control_list
+ 
+@@ -331,7 +355,7 @@ rm -f $pkg_file
+ if [ "$outer" = "ar" ] ; then
+   ( cd $tmp_dir && ar -crfD $pkg_file ./debian-binary ./control.tar.gz ./data.tar.$cext )
+ else
+-  ( cd $tmp_dir && tar -c $tsortargs --mtime=@$build_date $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
++  ( cd $tmp_dir && tar -c $tsortargs $mtime_args $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file )
+ fi
+ 
+ rm $tmp_dir/debian-binary $tmp_dir/data.tar.$cext $tmp_dir/control.tar.gz
+-- 
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
index eb88b9b734..d5ce2cfbe2 100644
--- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb
@@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu
 
 SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \
            file://0001-update-alternatives-correctly-match-priority.patch \
+           file://0002-opkg-build-Add-acls-and-xattrs-support.patch \
            "
 SRCREV = "67994e62dc598282830385da75ba9b1abbbda941"
 
-- 
2.34.1

[OE-Core][PATCH v11][master-next 4/5] opkg: add options to enable support for acl and xattr

[Piotr Łobacz]

The libarchive library, which is being used by opkg, supports ACLs
and xattr already.

More informations can be read at this link:
https://github.com/libarchive/libarchive/pull/691

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 ...-to-enable-support-for-acl-and-xattr.patch | 70 +++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.6.2.bb      |  5 +-
 2 files changed, 74 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch

diff --git a/meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch b/meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch
new file mode 100644
index 0000000000..d6cb1d79fb
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/0002-Add-options-to-enable-support-for-acl-and-xattr.patch
@@ -0,0 +1,70 @@
+From 1c935e994bd572d9fff436f660ac1a060a434df0 Mon Sep 17 00:00:00 2001
+From: Maciej Liszewski <m.liszewski@welotec.com>
+Date: Tue, 4 Jul 2023 22:01:58 +0200
+Subject: [PATCH] Add options to enable support for acl and xattr
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+The libarchive library, which is being used by opkg, supports ACLs
+and xattr already.
+
+More informations can be read at this link:
+https://github.com/libarchive/libarchive/pull/691
+
+Upstream-Status: Accepted [https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA]
+
+[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097
+
+Signed-off-by: Maciej Liszewski <m.liszewski@welotec.com>
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ configure.ac           | 12 ++++++++++++
+ libopkg/opkg_archive.c |  8 ++++++++
+ 2 files changed, 20 insertions(+)
+
+diff --git a/configure.ac b/configure.ac
+index 389a818..46949cd 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -158,6 +158,18 @@ return OPENSSL_VERSION_NUMBER; ],
+   AC_SUBST(OPENSSL_LIBS)
+ fi
+ 
++# check for ACL support
++AC_ARG_WITH([acl], [AS_HELP_STRING([--with-acl], [Enable ACL support])])
++if test "x$with_acl" = "xyes"; then
++  AC_DEFINE([ENABLE_ACL], [1], [Enable ACL support])
++fi
++
++# check for xattr support
++AC_ARG_WITH([xattr], [AS_HELP_STRING([--with-xattr], [Enable xattr support])])
++if test "x$with_xattr" = "xyes"; then
++  AC_DEFINE([ENABLE_XATTR], [1], [Enable xattr support])
++fi
++
+ # check for libsolv solver
+ AC_ARG_WITH(libsolv, AC_HELP_STRING([--with-libsolv], [Use libsolv solver support.
+   ]), [], [with_libsolv="no"])
+diff --git a/libopkg/opkg_archive.c b/libopkg/opkg_archive.c
+index 03a4afb..8dd902d 100644
+--- a/libopkg/opkg_archive.c
++++ b/libopkg/opkg_archive.c
+@@ -912,6 +912,14 @@ struct opkg_ar *ar_open_pkg_data_archive(const char *filename)
+     ar->extract_flags = ARCHIVE_EXTRACT_OWNER | ARCHIVE_EXTRACT_PERM |
+         ARCHIVE_EXTRACT_TIME | ARCHIVE_EXTRACT_UNLINK | ARCHIVE_EXTRACT_NO_OVERWRITE;
+ 
++#ifdef ENABLE_ACL
++    ar->extract_flags |= ARCHIVE_EXTRACT_ACL;
++#endif
++
++#ifdef ENABLE_XATTR
++    ar->extract_flags |= ARCHIVE_EXTRACT_FFLAGS | ARCHIVE_EXTRACT_XATTR;
++#endif
++
+     if (opkg_config->ignore_uid)
+         ar->extract_flags &= ~ARCHIVE_EXTRACT_OWNER;
+ 
+-- 
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.6.2.bb b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
index 46be137354..d7dc6ab715 100644
--- a/meta/recipes-devtools/opkg/opkg_0.6.2.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
@@ -15,6 +15,7 @@ PE = "1"
 SRC_URI = "http://downloads.yoctoproject.org/releases/${BPN}/${BPN}-${PV}.tar.gz \
            file://opkg.conf \
            file://0001-opkg_conf-create-opkg.lock-in-run-instead-of-var-run.patch \
+           file://0002-Add-options-to-enable-support-for-acl-and-xattr.patch \
            file://run-ptest \
            "
 
@@ -30,8 +31,10 @@ inherit autotools pkgconfig ptest
 target_localstatedir := "${localstatedir}"
 OPKGLIBDIR ??= "${target_localstatedir}/lib"
 
-PACKAGECONFIG ??= "libsolv"
+PACKAGECONFIG ??= "libsolv ${@bb.utils.filter('DISTRO_FEATURES', 'acl xattr', d)}"
 
+PACKAGECONFIG[acl] = "--with-acl,--without-acl"
+PACKAGECONFIG[xattr] = "--with-xattr,--without-xattr"
 PACKAGECONFIG[gpg] = "--enable-gpg,--disable-gpg,\
     gnupg gpgme libgpg-error,\
     ${@ "gnupg" if ("native" in d.getVar("PN")) else "gnupg-gpg"}\
-- 
2.34.1

[OE-Core][PATCH v11][master-next 5/5] opkg: set locale from system environment variables

[Piotr Łobacz]

A C program inherits its locale environment variables when it starts up.
This happens automatically. However, these variables do not automatically
control the locale used by the library functions, because ISO C says that
all programs start by default in the standard ‘C’ locale.

Fixes warnings:
Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84)

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 ...le-from-system-environment-variables.patch | 48 +++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.6.2.bb      |  1 +
 2 files changed, 49 insertions(+)
 create mode 100644 meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch

diff --git a/meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch b/meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch
new file mode 100644
index 0000000000..71240ec8fd
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/0003-opkg-set-locale-from-system-environment-variables.patch
@@ -0,0 +1,48 @@
+From 712895b1914bf63ee4d669863bfd106814329076 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
+Date: Wed, 19 Jul 2023 21:26:09 +0200
+Subject: [PATCH] opkg: set locale from system environment variables
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+A C program inherits its locale environment variables when it starts up.
+This happens automatically. However, these variables do not automatically
+control the locale used by the library functions, because ISO C says that
+all programs start by default in the standard ‘C’ locale.
+
+Fixes warnings:
+Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84)
+
+Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/16kgZfJ26mQ]
+
+[1] https://www.gnu.org/software/libc/manual/html_node/Setting-the-Locale.html
+
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ src/opkg.c | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/src/opkg.c b/src/opkg.c
+index 544c58a..0c729ff 100644
+--- a/src/opkg.c
++++ b/src/opkg.c
+@@ -27,6 +27,7 @@
+ #include <stdio.h>
+ #include <getopt.h>
+ #include <stdlib.h>
++#include <locale.h>
+ 
+ #include "opkg_conf.h"
+ #include "opkg_cmd.h"
+@@ -408,6 +409,7 @@ int main(int argc, char *argv[])
+     if (opkg_conf_init())
+         goto err0;
+ 
++    setlocale(LC_ALL, "");
+     opkg_config->verbosity = NOTICE;
+ 
+     opts = args_parse(argc, argv);
+-- 
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.6.2.bb b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
index d7dc6ab715..3b5d51d74a 100644
--- a/meta/recipes-devtools/opkg/opkg_0.6.2.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.6.2.bb
@@ -16,6 +16,7 @@ SRC_URI = "http://downloads.yoctoproject.org/releases/${BPN}/${BPN}-${PV}.tar.gz
            file://opkg.conf \
            file://0001-opkg_conf-create-opkg.lock-in-run-instead-of-var-run.patch \
            file://0002-Add-options-to-enable-support-for-acl-and-xattr.patch \
+           file://0003-opkg-set-locale-from-system-environment-variables.patch \
            file://run-ptest \
            "
 
-- 
2.34.1

ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

Hi Alexandre, Alex,
Thx for supporting me, I have finally discovered the issue regarding this packages differences.
It occurred that I was badly comparing two strings for tarformat comparison. Now it is fixed and should finally work.

This has been additionally tested with oe-selftest -r reproducible.

BR
Piotr

Od: Piotr Łobacz <p.lobacz@welotec.com>
Wysłane: środa, 26 lipca 2023 11:22
Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
DW: Piotr Łobacz <p.lobacz@welotec.com>
Temat: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr 
 
Extend OPKGBUILDCMD variable, with additional parameters, depending
on target distro features, in order to support ACLs and xattr.

With fix pushed to the opkg-devel:
https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
opkg-build is able to create tar archives with ACLs and xattr.

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 meta/classes-global/package_ipk.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
index b4b7bc9ac2..a0f106e4ad 100644
--- a/meta/classes-global/package_ipk.bbclass
+++ b/meta/classes-global/package_ipk.bbclass
@@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
 PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
 
 # Program to be used to build opkg packages
-OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
+OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
 
 OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
 OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
-- 
2.34.1

Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Alexandre Belloni]

Hello,

I sent this to the autobuilders and I got the same reproducibility
failure as earlier:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263/steps/13/logs/stdio

I'm not sure why you wouldn't see those.

On 26/07/2023 09:27:53+0000, Piotr Łobacz wrote:
> 
> Hi Alexandre, Alex,
> Thx for supporting me, I have finally discovered the issue regarding this packages differences.
> It occurred that I was badly comparing two strings for tarformat comparison. Now it is fixed and should finally work.
> 
> This has been additionally tested with oe-selftest -r reproducible.
> 
> BR
> Piotr
> 
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: środa, 26 lipca 2023 11:22
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr 
>  
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
> 
> With fix pushed to the opkg-devel:
> https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
> opkg-build is able to create tar archives with ACLs and xattr.
> 
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
>  meta/classes-global/package_ipk.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
>  PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>  
>  # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>  
>  OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
>  OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> -- 
> 2.34.1

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184875): https://lists.openembedded.org/g/openembedded-core/message/184875
> Mute This Topic: https://lists.openembedded.org/mt/100367408/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

Hi Alexandre,

> Hello,
>
> I sent this to the autobuilders and I got the same reproducibility
> failure as earlier:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263/steps/13/logs/stdio>data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=C6mYlLwOmYosyl1ZIS8vKQf2ailFqMqBXeamF%2FnSE7E%3D&reserved=0
>
> I'm not sure why you wouldn't see those.

Nor do I. I'm running it once again on our remote CI server, maybe I missed something. I'll answer you when I will know something more.

BR
Piotr

On 26/07/2023 09:27:53+0000, Piotr Łobacz wrote:
>
> Hi Alexandre, Alex,
> Thx for supporting me, I have finally discovered the issue regarding this packages differences.
> It occurred that I was badly comparing two strings for tarformat comparison. Now it is fixed and should finally work.
>
> This has been additionally tested with oe-selftest -r reproducible.
>
> BR
> Piotr
>
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: środa, 26 lipca 2023 11:22
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
>
> With fix pushed to the opkg-devel:
> https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8
> opkg-build is able to create tar archives with ACLs and xattr.
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
>  meta/classes-global/package_ipk.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
>  PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>
>  # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>
>  OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
>  OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> --
> 2.34.1

>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184875): https://lists.openembedded.org/g/openembedded-core/message/184875
> Mute This Topic: https://lists.openembedded.org/mt/100367408/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com/

ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

[-- Attachment #1: Type: text/plain, Size: 9962 bytes --]

Hello again,
Funny thing, I have run the same revision as you in here https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263
and I have different failures, but I have them:


File "/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/meta/lib/oeqa/selftest/cases/reproducible.py", line 327, in test_reproducible_builds
    self.fail('\n'.join(fails))
AssertionError: The following ipk packages are different and not in exclusion list:
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/acpid-src_2.0.34-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/avahi-src_0.8-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/bind-src_9.18.16-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/debugedit-src_5.0-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/flac-src_1.4.3-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-dev_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-ptest_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-src_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libmnl-src_1.0.5-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libuuid-src_2.38.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/log4cplus-src_2.1.0-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/lttng-tools-src_2.13.9-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-numpy-src_1.25.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-dev_3.44.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-src_3.44.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject_3.44.1-r0_core2-64.ipk

2023-07-29 00:18:31,843 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-07-29 00:18:31,843 - oe-selftest - INFO - Ran 2 tests in 51978.070s
2023-07-29 00:18:31,843 - oe-selftest - INFO - FAILED
2023-07-29 00:18:31,843 - oe-selftest - INFO -  (failures=1)
2023-07-29 00:18:35,545 - oe-selftest - INFO - RESULTS:
2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.DiffoscopeTests.test_diffoscope: PASSED (35.32s)
2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (51940.60s)
2023-07-29 00:18:35,948 - oe-selftest - INFO - SUMMARY:
2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest () - Ran 2 tests in 51978.070s
2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=1, skipped=0, failures=1, errors=0)

due to this tests result difference, my suspicion at the moment is only the possibility of changing the format between gnu and posix depending on DISTRO_FEATURES having acl and/or xattr.

I will force now tar format to have only posix format and run another test. We will be in touch.

BR
Piotr

________________________________
Od: Alexandre Belloni <alexandre.belloni@bootlin.com>
Wysłane: czwartek, 27 lipca 2023 16:18
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

Hello,

I sent this to the autobuilders and I got the same reproducibility
failure as earlier:

https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F117%2Fbuilds%2F3263%2Fsteps%2F13%2Flogs%2Fstdio&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=C6mYlLwOmYosyl1ZIS8vKQf2ailFqMqBXeamF%2FnSE7E%3D&reserved=0<https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263/steps/13/logs/stdio>

I'm not sure why you wouldn't see those.

On 26/07/2023 09:27:53+0000, Piotr Łobacz wrote:
>
> Hi Alexandre, Alex,
> Thx for supporting me, I have finally discovered the issue regarding this packages differences.
> It occurred that I was badly comparing two strings for tarformat comparison. Now it is fixed and should finally work.
>
> This has been additionally tested with oe-selftest -r reproducible.
>
> BR
> Piotr
>
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: środa, 26 lipca 2023 11:22
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
>
> With fix pushed to the opkg-devel:
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fopkg-devel%2Fc%2FdYNHrLjDwg8&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aZziODWaDuhvajNYsPmX47BT2z2CLK8NpgFeF80X1Jg%3D&reserved=0<https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8>
> opkg-build is able to create tar archives with ACLs and xattr.
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
>  meta/classes-global/package_ipk.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
>  PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>
>  # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>
>  OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
>  OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> --
> 2.34.1

>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184875): https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F184875&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KQ4jb1%2B0nyhGquOBb6Ca1JnSh1stDIg%2FEltszh9Z8Rk%3D&reserved=0<https://lists.openembedded.org/g/openembedded-core/message/184875>
> Mute This Topic: https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openembedded.org%2Fmt%2F100367408%2F3617179&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=AghjaI2zcxq0sYRMiBCDuHqx4dro5gh6MigROTfilxc%3D&reserved=0<https://lists.openembedded.org/mt/100367408/3617179>
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openembedded.org%2Fg%2Fopenembedded-core%2Funsub&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RBq5q4%2BLxdxmoglWkVKE%2BKAYWRaJ1Jrt1qu6EugIWU0%3D&reserved=0<https://lists.openembedded.org/g/openembedded-core/unsub> [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbootlin.com%2F&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QdR%2FSw2mI6zSaDoLdARAe4Q9CTcPPR5Jazo1mllODi4%3D&reserved=0<https://bootlin.com/>

[-- Attachment #2: Type: text/html, Size: 14343 bytes --]

Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

[-- Attachment #1: Type: text/plain, Size: 9180 bytes --]

OK, I have finally discovered the root cause. First of all there is another bug in yocto, for all these 19 packages.

I have compared the full timestamps with 'ls —full-time’ command in both directories (meaning reproducibleA and reproducibleB) of packages-split/acpid-src and they differ even just before packages creation…

Wysyłane z aplikacji Outlook dla systemu iOS<https://aka.ms/o0ukef>
________________________________
Od: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> w imieniu użytkownika Piotr Łobacz via lists.openembedded.org <p.lobacz=welotec.com@lists.openembedded.org>
Wysłane: Saturday, July 29, 2023 12:58:51 AM
Do: Alexandre Belloni <alexandre.belloni@bootlin.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

Hello again,
Funny thing, I have run the same revision as you in here https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263
and I have different failures, but I have them:


File "/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/meta/lib/oeqa/selftest/cases/reproducible.py", line 327, in test_reproducible_builds
    self.fail('\n'.join(fails))
AssertionError: The following ipk packages are different and not in exclusion list:
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/acpid-src_2.0.34-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/avahi-src_0.8-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/bind-src_9.18.16-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/debugedit-src_5.0-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/flac-src_1.4.3-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-dev_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-ptest_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-src_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libmnl-src_1.0.5-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libuuid-src_2.38.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/log4cplus-src_2.1.0-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/lttng-tools-src_2.13.9-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-numpy-src_1.25.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-dev_3.44.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-src_3.44.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject_3.44.1-r0_core2-64.ipk

2023-07-29 00:18:31,843 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-07-29 00:18:31,843 - oe-selftest - INFO - Ran 2 tests in 51978.070s
2023-07-29 00:18:31,843 - oe-selftest - INFO - FAILED
2023-07-29 00:18:31,843 - oe-selftest - INFO -  (failures=1)
2023-07-29 00:18:35,545 - oe-selftest - INFO - RESULTS:
2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.DiffoscopeTests.test_diffoscope: PASSED (35.32s)
2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (51940.60s)
2023-07-29 00:18:35,948 - oe-selftest - INFO - SUMMARY:
2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest () - Ran 2 tests in 51978.070s
2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=1, skipped=0, failures=1, errors=0)

due to this tests result difference, my suspicion at the moment is only the possibility of changing the format between gnu and posix depending on DISTRO_FEATURES having acl and/or xattr.

I will force now tar format to have only posix format and run another test. We will be in touch.

BR
Piotr

________________________________
Od: Alexandre Belloni <alexandre.belloni@bootlin.com>
Wysłane: czwartek, 27 lipca 2023 16:18
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

Hello,

I sent this to the autobuilders and I got the same reproducibility
failure as earlier:

https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F117%2Fbuilds%2F3263%2Fsteps%2F13%2Flogs%2Fstdio&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=C6mYlLwOmYosyl1ZIS8vKQf2ailFqMqBXeamF%2FnSE7E%3D&reserved=0<https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263/steps/13/logs/stdio>

I'm not sure why you wouldn't see those.

On 26/07/2023 09:27:53+0000, Piotr Łobacz wrote:
>
> Hi Alexandre, Alex,
> Thx for supporting me, I have finally discovered the issue regarding this packages differences.
> It occurred that I was badly comparing two strings for tarformat comparison. Now it is fixed and should finally work.
>
> This has been additionally tested with oe-selftest -r reproducible.
>
> BR
> Piotr
>
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: środa, 26 lipca 2023 11:22
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
>
> With fix pushed to the opkg-devel:
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fopkg-devel%2Fc%2FdYNHrLjDwg8&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aZziODWaDuhvajNYsPmX47BT2z2CLK8NpgFeF80X1Jg%3D&reserved=0<https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8>
> opkg-build is able to create tar archives with ACLs and xattr.
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
>  meta/classes-global/package_ipk.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
>  PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>
>  # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>
>  OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
>  OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> --
> 2.34.1

>
>
>


--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbootlin.com%2F&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QdR%2FSw2mI6zSaDoLdARAe4Q9CTcPPR5Jazo1mllODi4%3D&reserved=0<https://bootlin.com/>

[-- Attachment #2: Type: text/html, Size: 15048 bytes --]

Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

[-- Attachment #1: Type: text/plain, Size: 10087 bytes --]

I’m sorry for splitting this message but it has happend by accident…

Returning to the topic my assumption is that for some reason in reproducibleA, these miliseconds are cutted and in case of reproducibleB they are not.

This was obviously working for opkg-build in case of gnu format which is cutting it also but for posix format it stores and thus error occurs.

My question is where can I find the code responsible for moving/coping data into these packages-split directories?

BR
Piotr
________________________________
Od: Piotr Łobacz <p.lobacz@welotec.com>
Wysłano: poniedziałek, lipca 31, 2023 9:03 PM
Do: Alexandre Belloni <alexandre.belloni@bootlin.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

OK, I have finally discovered the root cause. First of all there is another bug in yocto, for all these 19 packages.

I have compared the full timestamps with 'ls —full-time’ command in both directories (meaning reproducibleA and reproducibleB) of packages-split/acpid-src and they differ even just before packages creation…

Wysyłane z aplikacji Outlook dla systemu iOS<https://aka.ms/o0ukef>
________________________________
Od: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> w imieniu użytkownika Piotr Łobacz via lists.openembedded.org <p.lobacz=welotec.com@lists.openembedded.org>
Wysłane: Saturday, July 29, 2023 12:58:51 AM
Do: Alexandre Belloni <alexandre.belloni@bootlin.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

Hello again,
Funny thing, I have run the same revision as you in here https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263
and I have different failures, but I have them:


File "/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/meta/lib/oeqa/selftest/cases/reproducible.py", line 327, in test_reproducible_builds
    self.fail('\n'.join(fails))
AssertionError: The following ipk packages are different and not in exclusion list:
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/acpid-src_2.0.34-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/avahi-src_0.8-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/bind-src_9.18.16-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/debugedit-src_5.0-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/flac-src_1.4.3-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-dev_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-ptest_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-src_1.6.6-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libmnl-src_1.0.5-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libuuid-src_2.38.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/log4cplus-src_2.1.0-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/lttng-tools-src_2.13.9-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-numpy-src_1.25.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-dev_3.44.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-src_3.44.1-r0_core2-64.ipk
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject_3.44.1-r0_core2-64.ipk

2023-07-29 00:18:31,843 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-07-29 00:18:31,843 - oe-selftest - INFO - Ran 2 tests in 51978.070s
2023-07-29 00:18:31,843 - oe-selftest - INFO - FAILED
2023-07-29 00:18:31,843 - oe-selftest - INFO -  (failures=1)
2023-07-29 00:18:35,545 - oe-selftest - INFO - RESULTS:
2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.DiffoscopeTests.test_diffoscope: PASSED (35.32s)
2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (51940.60s)
2023-07-29 00:18:35,948 - oe-selftest - INFO - SUMMARY:
2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest () - Ran 2 tests in 51978.070s
2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=1, skipped=0, failures=1, errors=0)

due to this tests result difference, my suspicion at the moment is only the possibility of changing the format between gnu and posix depending on DISTRO_FEATURES having acl and/or xattr.

I will force now tar format to have only posix format and run another test. We will be in touch.

BR
Piotr

________________________________
Od: Alexandre Belloni <alexandre.belloni@bootlin.com>
Wysłane: czwartek, 27 lipca 2023 16:18
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

Hello,

I sent this to the autobuilders and I got the same reproducibility
failure as earlier:

https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F117%2Fbuilds%2F3263%2Fsteps%2F13%2Flogs%2Fstdio&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=C6mYlLwOmYosyl1ZIS8vKQf2ailFqMqBXeamF%2FnSE7E%3D&reserved=0<https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263/steps/13/logs/stdio>

I'm not sure why you wouldn't see those.

On 26/07/2023 09:27:53+0000, Piotr Łobacz wrote:
>
> Hi Alexandre, Alex,
> Thx for supporting me, I have finally discovered the issue regarding this packages differences.
> It occurred that I was badly comparing two strings for tarformat comparison. Now it is fixed and should finally work.
>
> This has been additionally tested with oe-selftest -r reproducible.
>
> BR
> Piotr
>
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłane: środa, 26 lipca 2023 11:22
> Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> DW: Piotr Łobacz <p.lobacz@welotec.com>
> Temat: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Extend OPKGBUILDCMD variable, with additional parameters, depending
> on target distro features, in order to support ACLs and xattr.
>
> With fix pushed to the opkg-devel:
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fopkg-devel%2Fc%2FdYNHrLjDwg8&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aZziODWaDuhvajNYsPmX47BT2z2CLK8NpgFeF80X1Jg%3D&reserved=0<https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8>
> opkg-build is able to create tar archives with ACLs and xattr.
>
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
>  meta/classes-global/package_ipk.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> index b4b7bc9ac2..a0f106e4ad 100644
> --- a/meta/classes-global/package_ipk.bbclass
> +++ b/meta/classes-global/package_ipk.bbclass
> @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
>  PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
>
>  # Program to be used to build opkg packages
> -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
>
>  OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
>  OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> --
> 2.34.1

>
>
>


--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbootlin.com%2F&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QdR%2FSw2mI6zSaDoLdARAe4Q9CTcPPR5Jazo1mllODi4%3D&reserved=0<https://bootlin.com/>

[-- Attachment #2: Type: text/html, Size: 16509 bytes --]

Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Joshua Watt]

On Mon, Jul 31, 2023 at 1:10 PM Piotr Łobacz <p.lobacz@welotec.com> wrote:
>
> I’m sorry for splitting this message but it has happend by accident…
>
> Returning to the topic my assumption is that for some reason in reproducibleA, these miliseconds are cutted and in case of reproducibleB they are not.
>
> This was obviously working for opkg-build in case of gnu format which is cutting it also but for posix format it stores and thus error occurs.
>
> My question is where can I find the code responsible for moving/coping data into these packages-split directories?

I suspect this is because reproducibleA is allowed to restore from
sstate where the milliseconds are not preserved, while reproducibleB
doesn't restore from sstate.

I'm not sure how you would store the milliseconds in sstate and make
sure it was consistent, but that is probably where you need to look.
Start in sstate.bbclass

>
> BR
> Piotr
> ________________________________
> Od: Piotr Łobacz <p.lobacz@welotec.com>
> Wysłano: poniedziałek, lipca 31, 2023 9:03 PM
> Do: Alexandre Belloni <alexandre.belloni@bootlin.com>
> DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
> Temat: Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> OK, I have finally discovered the root cause. First of all there is another bug in yocto, for all these 19 packages.
>
> I have compared the full timestamps with 'ls —full-time’ command in both directories (meaning reproducibleA and reproducibleB) of packages-split/acpid-src and they differ even just before packages creation…
>
> Wysyłane z aplikacji Outlook dla systemu iOS
> ________________________________
> Od: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> w imieniu użytkownika Piotr Łobacz via lists.openembedded.org <p.lobacz=welotec.com@lists.openembedded.org>
> Wysłane: Saturday, July 29, 2023 12:58:51 AM
> Do: Alexandre Belloni <alexandre.belloni@bootlin.com>
> DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
> Temat: ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Hello again,
> Funny thing, I have run the same revision as you in here https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3263
> and I have different failures, but I have them:
>
>
> File "/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/meta/lib/oeqa/selftest/cases/reproducible.py", line 327, in test_reproducible_builds
>     self.fail('\n'.join(fails))
> AssertionError: The following ipk packages are different and not in exclusion list:
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/acpid-src_2.0.34-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/avahi-src_0.8-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/bind-src_9.18.16-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/debugedit-src_5.0-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/flac-src_1.4.3-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-dev_1.6.6-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-ptest_1.6.6-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libjson-glib-1.0-src_1.6.6-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libmnl-src_1.0.5-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/libuuid-src_2.38.1-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/log4cplus-src_2.1.0-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/lttng-tools-src_2.13.9-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-numpy-src_1.25.1-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-dev_3.44.1-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject-src_3.44.1-r0_core2-64.ipk
> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/deploy/ipk/./core2-64/python3-pygobject_3.44.1-r0_core2-64.ipk
>
> 2023-07-29 00:18:31,843 - oe-selftest - INFO - ----------------------------------------------------------------------
> 2023-07-29 00:18:31,843 - oe-selftest - INFO - Ran 2 tests in 51978.070s
> 2023-07-29 00:18:31,843 - oe-selftest - INFO - FAILED
> 2023-07-29 00:18:31,843 - oe-selftest - INFO -  (failures=1)
> 2023-07-29 00:18:35,545 - oe-selftest - INFO - RESULTS:
> 2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.DiffoscopeTests.test_diffoscope: PASSED (35.32s)
> 2023-07-29 00:18:35,546 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (51940.60s)
> 2023-07-29 00:18:35,948 - oe-selftest - INFO - SUMMARY:
> 2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest () - Ran 2 tests in 51978.070s
> 2023-07-29 00:18:35,949 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=1, skipped=0, failures=1, errors=0)
>
> due to this tests result difference, my suspicion at the moment is only the possibility of changing the format between gnu and posix depending on DISTRO_FEATURES having acl and/or xattr.
>
> I will force now tar format to have only posix format and run another test. We will be in touch.
>
> BR
> Piotr
>
> ________________________________
> Od: Alexandre Belloni <alexandre.belloni@bootlin.com>
> Wysłane: czwartek, 27 lipca 2023 16:18
> Do: Piotr Łobacz <p.lobacz@welotec.com>
> DW: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
> Temat: Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
>
> Hello,
>
> I sent this to the autobuilders and I got the same reproducibility
> failure as earlier:
>
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F117%2Fbuilds%2F3263%2Fsteps%2F13%2Flogs%2Fstdio&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=C6mYlLwOmYosyl1ZIS8vKQf2ailFqMqBXeamF%2FnSE7E%3D&reserved=0
>
> I'm not sure why you wouldn't see those.
>
> On 26/07/2023 09:27:53+0000, Piotr Łobacz wrote:
> >
> > Hi Alexandre, Alex,
> > Thx for supporting me, I have finally discovered the issue regarding this packages differences.
> > It occurred that I was badly comparing two strings for tarformat comparison. Now it is fixed and should finally work.
> >
> > This has been additionally tested with oe-selftest -r reproducible.
> >
> > BR
> > Piotr
> >
> > Od: Piotr Łobacz <p.lobacz@welotec.com>
> > Wysłane: środa, 26 lipca 2023 11:22
> > Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
> > DW: Piotr Łobacz <p.lobacz@welotec.com>
> > Temat: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr
> >
> > Extend OPKGBUILDCMD variable, with additional parameters, depending
> > on target distro features, in order to support ACLs and xattr.
> >
> > With fix pushed to the opkg-devel:
> > https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fg%2Fopkg-devel%2Fc%2FdYNHrLjDwg8&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aZziODWaDuhvajNYsPmX47BT2z2CLK8NpgFeF80X1Jg%3D&reserved=0
> > opkg-build is able to create tar archives with ACLs and xattr.
> >
> > Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> > ---
> >  meta/classes-global/package_ipk.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/classes-global/package_ipk.bbclass b/meta/classes-global/package_ipk.bbclass
> > index b4b7bc9ac2..a0f106e4ad 100644
> > --- a/meta/classes-global/package_ipk.bbclass
> > +++ b/meta/classes-global/package_ipk.bbclass
> > @@ -15,7 +15,7 @@ IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
> >  PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
> >
> >  # Program to be used to build opkg packages
> > -OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
> > +OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}" ${@bb.utils.contains('DISTRO_FEATURES', 'acl', '-A', '', d)} ${@bb.utils.contains('DISTRO_FEATURES', 'xattr', '-X', '', d)}'
> >
> >  OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
> >  OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
> > --
> > 2.34.1
>
> >
> >
> >
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbootlin.com%2F&data=05%7C01%7Cp.lobacz%40welotec.com%7C8fb759c0e1ab49b0c69408db8eac5547%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638260643038719160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QdR%2FSw2mI6zSaDoLdARAe4Q9CTcPPR5Jazo1mllODi4%3D&reserved=0
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#185163): https://lists.openembedded.org/g/openembedded-core/message/185163
> Mute This Topic: https://lists.openembedded.org/mt/100468421/3616693
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Richard Purdie]

On Mon, 2023-07-31 at 14:25 -0600, Joshua Watt wrote:
> On Mon, Jul 31, 2023 at 1:10 PM Piotr Łobacz <p.lobacz@welotec.com> wrote:
> > 
> > I’m sorry for splitting this message but it has happend by accident…
> > 
> > Returning to the topic my assumption is that for some reason in reproducibleA, these miliseconds are cutted and in case of reproducibleB they are not.
> > 
> > This was obviously working for opkg-build in case of gnu format which is cutting it also but for posix format it stores and thus error occurs.
> > 
> > My question is where can I find the code responsible for moving/coping data into these packages-split directories?
> 
> I suspect this is because reproducibleA is allowed to restore from
> sstate where the milliseconds are not preserved, while reproducibleB
> doesn't restore from sstate.
> 
> I'm not sure how you would store the milliseconds in sstate and make
> sure it was consistent, but that is probably where you need to look.
> Start in sstate.bbclass

What is likely happening is that we have two cases:

a) do_package is run and do_package_write_ipk uses the files directly
from disk which has the millisecond data. do_package writes an sstate
tarball using tar in non-posix mode.

b) do_package_setscene runs instead of do_package. This restores the
tarball written into sstate in a). The millisecond data is therefore
lost. do_package_write_ipk would then be written without it.

I'd guess you need to make the sstate archives of do_package store the
acl and extended attribute data and when that is done, the millisecond
timestamp data will probably be saved too? I'm not sure this is going
to be a generally useful thing in the general case though and zeroing
the millisecond portion would really be easier in general if we could.
Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.

Cheers,

Richard

Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

[-- Attachment #1: Type: text/plain, Size: 2474 bytes --]

Big thx for quick response Richard.

I will look into it.

BR
Piotr
________________________________
Od: Richard Purdie <richard.purdie@linuxfoundation.org>
Wysłane: Monday, July 31, 2023 11:19:39 PM
Do: Joshua Watt <JPEWhacker@gmail.com>; Piotr Łobacz <p.lobacz@welotec.com>
DW: Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

On Mon, 2023-07-31 at 14:25 -0600, Joshua Watt wrote:
> On Mon, Jul 31, 2023 at 1:10 PM Piotr Łobacz <p.lobacz@welotec.com> wrote:
> >
> > I’m sorry for splitting this message but it has happend by accident…
> >
> > Returning to the topic my assumption is that for some reason in reproducibleA, these miliseconds are cutted and in case of reproducibleB they are not.
> >
> > This was obviously working for opkg-build in case of gnu format which is cutting it also but for posix format it stores and thus error occurs.
> >
> > My question is where can I find the code responsible for moving/coping data into these packages-split directories?
>
> I suspect this is because reproducibleA is allowed to restore from
> sstate where the milliseconds are not preserved, while reproducibleB
> doesn't restore from sstate.
>
> I'm not sure how you would store the milliseconds in sstate and make
> sure it was consistent, but that is probably where you need to look.
> Start in sstate.bbclass

What is likely happening is that we have two cases:

a) do_package is run and do_package_write_ipk uses the files directly
from disk which has the millisecond data. do_package writes an sstate
tarball using tar in non-posix mode.

b) do_package_setscene runs instead of do_package. This restores the
tarball written into sstate in a). The millisecond data is therefore
lost. do_package_write_ipk would then be written without it.

I'd guess you need to make the sstate archives of do_package store the
acl and extended attribute data and when that is done, the millisecond
timestamp data will probably be saved too? I'm not sure this is going
to be a generally useful thing in the general case though and zeroing
the millisecond portion would really be easier in general if we could.
Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.

Cheers,

Richard





[-- Attachment #2: Type: text/html, Size: 3655 bytes --]

ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

> I'd guess you need to make the sstate archives of do_package store the
> acl and extended attribute data and when that is done, the millisecond
> timestamp data will probably be saved too? I'm not sure this is going
> to be a generally useful thing in the general case though and zeroing
> the millisecond portion would really be easier in general if we could.
> Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.

This is what we were talking about on tar mailing list, to remove this precision for posix format.
Additionally in case of acl and extended attribute data they need to be also added to this sstate
archives, which still means that it needs to be extended with posix format.

BR
Piotr

Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1711 bytes --]

Just FYI:

Not sure if it's still relevant, but last time I was comparing some posix
archives from different host OSes there were some differences other than
timestamps I believe, this was 2.5 years ago showing differences between
tar 1.29 on ubuntu 18.04 and 1.33 on tumbleweed:

https://git.openembedded.org/openembedded-core/commit/?id=3ecea58f2a3382d9f4b410d6ad7089111334cb6f

Cheers,

On Mon, Jul 31, 2023 at 11:28 PM Piotr Łobacz <p.lobacz@welotec.com> wrote:

>
>
> > I'd guess you need to make the sstate archives of do_package store the
> > acl and extended attribute data and when that is done, the millisecond
> > timestamp data will probably be saved too? I'm not sure this is going
> > to be a generally useful thing in the general case though and zeroing
> > the millisecond portion would really be easier in general if we could.
> > Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.
>
> This is what we were talking about on tar mailing list, to remove this
> precision for posix format.
> Additionally in case of acl and extended attribute data they need to be
> also added to this sstate
> archives, which still means that it needs to be extended with posix format.
>
> BR
> Piotr
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#185173):
> https://lists.openembedded.org/g/openembedded-core/message/185173
> Mute This Topic: https://lists.openembedded.org/mt/100471398/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 2713 bytes --]

ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

Hi Martin, thx for the link,
actually, this error:

$ diffoscope rootfs.tumbleweed.posix.tar rootfs.ubuntu.posix.tar
 016a4c00: 2e2f 7573 722f 6269 6e2f 5061 7848 6561  ./usr/bin/PaxHea
-016a4c10: 6465 7273 2f63 6861 7474 722e 6532 6673  ders/chattr.e2fs
-016a4c20: 7072 6f67 7300 0000 0000 0000 0000 0000  progs...........
+016a4c10: 6465 7273 2e32 322f 6368 6174 7472 2e65  ders.22/chattr.e
+016a4c20: 3266 7370 726f 6773 0000 0000 0000 0000  2fsprogs........
...
 03937000: 2e2f 7573 722f 6269 6e2f 5061 7848 6561  ./usr/bin/PaxHea
-03937010: 6465 7273 2f63 6f6e 7461 696e 6572 642d  ders/containerd-
-03937020: 6374 7200 0000 0000 0000 0000 0000 0000  ctr.............
+03937010: 6465 7273 2e32 322f 636f 6e74 6169 6e65  ders.22/containe
+03937020: 7264 2d63 7472 0000 0000 0000 0000 0000  rd-ctr..........

for posix, is due to missing params "--pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0"
which can be read in here https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html
in the last paragraph and I do know that this information is well hidden :P

And we have currently a talk about that on bug-tar mailing list...

Unfortunately the issue is unrelevant, because I already know that... the problem is because as Richard
pointed it out correctly:

> What is likely happening is that we have two cases:
>
> a) do_package is run and do_package_write_ipk uses the files directly
> from disk which has the millisecond data. do_package writes an sstate
> tarball using tar in non-posix mode.
> 
> b) do_package_setscene runs instead of do_package. This restores the
> tarball written into sstate in a). The millisecond data is therefore
> lost. do_package_write_ipk would then be written without it.
>
> I'd guess you need to make the sstate archives of do_package store the
> acl and extended attribute data and when that is done, the millisecond
> timestamp data will probably be saved too? I'm not sure this is going
> to be a generally useful thing in the general case though and zeroing
> the millisecond portion would really be easier in general if we could.
> Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.

And this is exactly what is happening between reproducibleA and reproducibleB,
which we can observe in both directories e.g. acpid:

plobacz@latitude:/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib$ ls --full-time /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/packages-split/acpid-src/usr/src/debug/acpid/2.0.34-r0/
razem 192
-rw-r--r-- 2 plobacz plobacz 10934 2019-07-31 14:21:00.000000000 +0200 acpid.c
-rw-r--r-- 2 plobacz plobacz  1670 2018-04-11 01:51:39.000000000 +0200 acpid.h
-rw-r--r-- 2 plobacz plobacz   878 2009-04-29 16:37:13.000000000 +0200 acpi_genetlink.h
-rw-r--r-- 2 plobacz plobacz  7339 2012-02-16 21:24:42.000000000 +0100 acpi_ids.c
-rw-r--r-- 2 plobacz plobacz  1118 2012-02-16 21:24:42.000000000 +0100 acpi_ids.h
-rw-r--r-- 2 plobacz plobacz  5264 2015-07-29 00:32:35.000000000 +0200 acpi_listen.c
-rw-r--r-- 2 plobacz plobacz  4583 2013-08-16 00:49:18.000000000 +0200 connection_list.c
-rw-r--r-- 2 plobacz plobacz  2725 2013-08-15 22:24:07.000000000 +0200 connection_list.h
-rw-r--r-- 2 plobacz plobacz 17402 2018-03-29 17:07:23.000000000 +0200 event.c
-rw-r--r-- 2 plobacz plobacz  1288 2013-12-19 03:42:46.000000000 +0100 event.h
-rw-r--r-- 2 plobacz plobacz  1656 2009-04-29 16:37:13.000000000 +0200 genetlink.h
-rw-r--r-- 2 plobacz plobacz  4943 2021-08-29 18:46:00.000000000 +0200 inotify_handler.c
-rw-r--r-- 2 plobacz plobacz  1094 2009-11-09 23:03:27.000000000 +0100 inotify_handler.h
-rw-r--r-- 2 plobacz plobacz 18544 2022-08-15 16:38:03.000000000 +0200 input_layer.c
-rw-r--r-- 2 plobacz plobacz  1257 2009-11-10 00:16:59.000000000 +0100 input_layer.h
drwxr-xr-x 2 plobacz plobacz  4096 2022-09-16 03:24:42.000000000 +0200 kacpimon
-rw-r--r-- 2 plobacz plobacz 13746 2021-09-20 03:31:56.000000000 +0200 libnetlink.c
-rw-r--r-- 2 plobacz plobacz  3028 2021-09-20 03:30:25.000000000 +0200 libnetlink.h
-rw-r--r-- 2 plobacz plobacz  1401 2018-03-29 17:08:05.000000000 +0200 log.c
-rw-r--r-- 2 plobacz plobacz  1269 2018-03-29 17:08:02.000000000 +0200 log.h
-rw-r--r-- 2 plobacz plobacz  6155 2021-09-20 03:32:03.000000000 +0200 netlink.c
-rw-r--r-- 2 plobacz plobacz  1066 2009-04-29 16:37:13.000000000 +0200 netlink.h
-rw-r--r-- 2 plobacz plobacz  4876 2015-07-29 00:32:35.000000000 +0200 proc.c
-rw-r--r-- 2 plobacz plobacz  1066 2012-02-16 21:25:02.000000000 +0100 proc.h
-rw-r--r-- 2 plobacz plobacz  5018 2022-09-16 03:24:42.000000000 +0200 sock.c
-rw-r--r-- 2 plobacz plobacz  1198 2012-02-16 21:25:02.000000000 +0100 sock.h
-rw-r--r-- 2 plobacz plobacz  3007 2016-01-27 17:13:18.000000000 +0100 ud_socket.c
-rw-r--r-- 2 plobacz plobacz   352 2012-03-21 00:54:18.000000000 +0100 ud_socket.h
plobacz@latitude:/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib$ ls --full-time /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/packages-split/acpid-src/usr/src/debug/acpid/2.0.34-r0/
razem 192
-rw-r--r-- 3 plobacz plobacz 10934 2019-07-31 14:21:00.861236271 +0200 acpid.c
-rw-r--r-- 3 plobacz plobacz  1670 2018-04-11 01:51:39.467499913 +0200 acpid.h
-rw-r--r-- 3 plobacz plobacz   878 2009-04-29 16:37:13.000000000 +0200 acpi_genetlink.h
-rw-r--r-- 3 plobacz plobacz  7339 2012-02-16 21:24:42.515701450 +0100 acpi_ids.c
-rw-r--r-- 3 plobacz plobacz  1118 2012-02-16 21:24:42.544701444 +0100 acpi_ids.h
-rw-r--r-- 3 plobacz plobacz  5264 2015-07-29 00:32:35.499948465 +0200 acpi_listen.c
-rw-r--r-- 3 plobacz plobacz  4583 2013-08-16 00:49:18.764134072 +0200 connection_list.c
-rw-r--r-- 3 plobacz plobacz  2725 2013-08-15 22:24:07.562169686 +0200 connection_list.h
-rw-r--r-- 3 plobacz plobacz 17402 2018-03-29 17:07:23.716843024 +0200 event.c
-rw-r--r-- 3 plobacz plobacz  1288 2013-12-19 03:42:46.529591162 +0100 event.h
-rw-r--r-- 3 plobacz plobacz  1656 2009-04-29 16:37:13.000000000 +0200 genetlink.h
-rw-r--r-- 3 plobacz plobacz  4943 2021-08-29 18:46:00.558827438 +0200 inotify_handler.c
-rw-r--r-- 3 plobacz plobacz  1094 2009-11-09 23:03:27.000000000 +0100 inotify_handler.h
-rw-r--r-- 3 plobacz plobacz 18544 2022-08-15 16:38:03.122942190 +0200 input_layer.c
-rw-r--r-- 3 plobacz plobacz  1257 2009-11-10 00:16:59.000000000 +0100 input_layer.h
drwxr-xr-x 2 plobacz plobacz  4096 2022-09-16 03:24:42.000000000 +0200 kacpimon
-rw-r--r-- 3 plobacz plobacz 13746 2021-09-20 03:31:56.196781952 +0200 libnetlink.c
-rw-r--r-- 3 plobacz plobacz  3028 2021-09-20 03:30:25.930632950 +0200 libnetlink.h
-rw-r--r-- 3 plobacz plobacz  1401 2018-03-29 17:08:05.842574608 +0200 log.c
-rw-r--r-- 3 plobacz plobacz  1269 2018-03-29 17:08:02.566518016 +0200 log.h
-rw-r--r-- 3 plobacz plobacz  6155 2021-09-20 03:32:03.418793923 +0200 netlink.c
-rw-r--r-- 3 plobacz plobacz  1066 2009-04-29 16:37:13.000000000 +0200 netlink.h
-rw-r--r-- 3 plobacz plobacz  4876 2015-07-29 00:32:35.453948235 +0200 proc.c
-rw-r--r-- 3 plobacz plobacz  1066 2012-02-16 21:25:02.924701352 +0100 proc.h
-rw-r--r-- 3 plobacz plobacz  5018 2022-09-16 03:24:42.000000000 +0200 sock.c
-rw-r--r-- 3 plobacz plobacz  1198 2012-02-16 21:25:02.925701357 +0100 sock.h
-rw-r--r-- 3 plobacz plobacz  3007 2016-01-27 17:13:18.836936981 +0100 ud_socket.c
-rw-r--r-- 3 plobacz plobacz   352 2012-03-21 00:54:18.929518037 +0100 ud_socket.h
plobacz@latitude:/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib$

And you see that for some reason for reproducibleA which uses sstate-cache miliseconds are lost,
due to exactly what Richard has written. Now I need to check one more thing how and were is that
exactly happening.

BR
Piotr

Od: Martin Jansa <martin.jansa@gmail.com>
Wysłane: wtorek, 1 sierpnia 2023 15:04
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: Richard Purdie <richard.purdie@linuxfoundation.org>; Joshua Watt <JPEWhacker@gmail.com>; Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr 
 
Just FYI: 

Not sure if it's still relevant, but last time I was comparing some posix archives from different host OSes there were some differences other than timestamps I believe, this was 2.5 years ago showing differences between tar 1.29 on ubuntu 18.04 and 1.33 on tumbleweed: 

https://git.openembedded.org/openembedded-core/commit/?id=3ecea58f2a3382d9f4b410d6ad7089111334cb6f

Cheers,

On Mon, Jul 31, 2023 at 11:28 PM Piotr Łobacz <p.lobacz@welotec.com> wrote:


> I'd guess you need to make the sstate archives of do_package store the
> acl and extended attribute data and when that is done, the millisecond
> timestamp data will probably be saved too? I'm not sure this is going
> to be a generally useful thing in the general case though and zeroing
> the millisecond portion would really be easier in general if we could.
> Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.

This is what we were talking about on tar mailing list, to remove this precision for posix format.
Additionally in case of acl and extended attribute data they need to be also added to this sstate
archives, which still means that it needs to be extended with posix format.

BR
Piotr




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#185173): https://lists.openembedded.org/g/openembedded-core/message/185173
Mute This Topic: https://lists.openembedded.org/mt/100471398/3617156
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Martin.Jansa@gmail.com]
-=-=-=-=-=-=-=-=-=-=-=-

ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

Hi Richard,
you were right. I have made another test with wiping out whole sstate-cache for acpid and rerunning the whole reproducible test.
It occurs that the package /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build/sstate-cache/a7/41/sstate:acpid:core2-64-poky-linux:2.0.34:r0:core2-64:11:a741afe7c0bfc218a844a546735ddc4dcaf578bff5d6a3762b1bdedf0819fc9b_package.tar.zst was already containing wrong timestamps.

> I'd guess you need to make the sstate archives of do_package store the
> acl and extended attribute data and when that is done, the millisecond
> timestamp data will probably be saved too?

So I'm guessing that commands which are producing this package_name_package.tar.zst are producing it in gnu format. Question is where can I find it?
Is it being produced in this sstate.bbclass or elsewhere?

BR
Piotr

Od: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> w imieniu użytkownika Piotr Łobacz via lists.openembedded.org <p.lobacz=welotec.com@lists.openembedded.org>
Wysłane: wtorek, 1 sierpnia 2023 15:27
Do: Martin Jansa <martin.jansa@gmail.com>
DW: Richard Purdie <richard.purdie@linuxfoundation.org>; Joshua Watt <JPEWhacker@gmail.com>; Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

Hi Martin, thx for the link,
actually, this error:

$ diffoscope rootfs.tumbleweed.posix.tar rootfs.ubuntu.posix.tar
 016a4c00: 2e2f 7573 722f 6269 6e2f 5061 7848 6561  ./usr/bin/PaxHea
-016a4c10: 6465 7273 2f63 6861 7474 722e 6532 6673  ders/chattr.e2fs
-016a4c20: 7072 6f67 7300 0000 0000 0000 0000 0000  progs...........
+016a4c10: 6465 7273 2e32 322f 6368 6174 7472 2e65  ders.22/chattr.e
+016a4c20: 3266 7370 726f 6773 0000 0000 0000 0000  2fsprogs........
...
 03937000: 2e2f 7573 722f 6269 6e2f 5061 7848 6561  ./usr/bin/PaxHea
-03937010: 6465 7273 2f63 6f6e 7461 696e 6572 642d  ders/containerd-
-03937020: 6374 7200 0000 0000 0000 0000 0000 0000  ctr.............
+03937010: 6465 7273 2e32 322f 636f 6e74 6169 6e65  ders.22/containe
+03937020: 7264 2d63 7472 0000 0000 0000 0000 0000  rd-ctr..........

for posix, is due to missing params "--pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0"
which can be read in here https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html
in the last paragraph and I do know that this information is well hidden :P

And we have currently a talk about that on bug-tar mailing list...

Unfortunately the issue is unrelevant, because I already know that... the problem is because as Richard
pointed it out correctly:

> What is likely happening is that we have two cases:
>
> a) do_package is run and do_package_write_ipk uses the files directly
> from disk which has the millisecond data. do_package writes an sstate
> tarball using tar in non-posix mode.
>
> b) do_package_setscene runs instead of do_package. This restores the
> tarball written into sstate in a). The millisecond data is therefore
> lost. do_package_write_ipk would then be written without it.
>
> I'd guess you need to make the sstate archives of do_package store the
> acl and extended attribute data and when that is done, the millisecond
> timestamp data will probably be saved too? I'm not sure this is going
> to be a generally useful thing in the general case though and zeroing
> the millisecond portion would really be easier in general if we could.
> Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.

And this is exactly what is happening between reproducibleA and reproducibleB,
which we can observe in both directories e.g. acpid:

plobacz@latitude:/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib$ ls --full-time /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/packages-split/acpid-src/usr/src/debug/acpid/2.0.34-r0/
razem 192
-rw-r--r-- 2 plobacz plobacz 10934 2019-07-31 14:21:00.000000000 +0200 acpid.c
-rw-r--r-- 2 plobacz plobacz  1670 2018-04-11 01:51:39.000000000 +0200 acpid.h
-rw-r--r-- 2 plobacz plobacz   878 2009-04-29 16:37:13.000000000 +0200 acpi_genetlink.h
-rw-r--r-- 2 plobacz plobacz  7339 2012-02-16 21:24:42.000000000 +0100 acpi_ids.c
-rw-r--r-- 2 plobacz plobacz  1118 2012-02-16 21:24:42.000000000 +0100 acpi_ids.h
-rw-r--r-- 2 plobacz plobacz  5264 2015-07-29 00:32:35.000000000 +0200 acpi_listen.c
-rw-r--r-- 2 plobacz plobacz  4583 2013-08-16 00:49:18.000000000 +0200 connection_list.c
-rw-r--r-- 2 plobacz plobacz  2725 2013-08-15 22:24:07.000000000 +0200 connection_list.h
-rw-r--r-- 2 plobacz plobacz 17402 2018-03-29 17:07:23.000000000 +0200 event.c
-rw-r--r-- 2 plobacz plobacz  1288 2013-12-19 03:42:46.000000000 +0100 event.h
-rw-r--r-- 2 plobacz plobacz  1656 2009-04-29 16:37:13.000000000 +0200 genetlink.h
-rw-r--r-- 2 plobacz plobacz  4943 2021-08-29 18:46:00.000000000 +0200 inotify_handler.c
-rw-r--r-- 2 plobacz plobacz  1094 2009-11-09 23:03:27.000000000 +0100 inotify_handler.h
-rw-r--r-- 2 plobacz plobacz 18544 2022-08-15 16:38:03.000000000 +0200 input_layer.c
-rw-r--r-- 2 plobacz plobacz  1257 2009-11-10 00:16:59.000000000 +0100 input_layer.h
drwxr-xr-x 2 plobacz plobacz  4096 2022-09-16 03:24:42.000000000 +0200 kacpimon
-rw-r--r-- 2 plobacz plobacz 13746 2021-09-20 03:31:56.000000000 +0200 libnetlink.c
-rw-r--r-- 2 plobacz plobacz  3028 2021-09-20 03:30:25.000000000 +0200 libnetlink.h
-rw-r--r-- 2 plobacz plobacz  1401 2018-03-29 17:08:05.000000000 +0200 log.c
-rw-r--r-- 2 plobacz plobacz  1269 2018-03-29 17:08:02.000000000 +0200 log.h
-rw-r--r-- 2 plobacz plobacz  6155 2021-09-20 03:32:03.000000000 +0200 netlink.c
-rw-r--r-- 2 plobacz plobacz  1066 2009-04-29 16:37:13.000000000 +0200 netlink.h
-rw-r--r-- 2 plobacz plobacz  4876 2015-07-29 00:32:35.000000000 +0200 proc.c
-rw-r--r-- 2 plobacz plobacz  1066 2012-02-16 21:25:02.000000000 +0100 proc.h
-rw-r--r-- 2 plobacz plobacz  5018 2022-09-16 03:24:42.000000000 +0200 sock.c
-rw-r--r-- 2 plobacz plobacz  1198 2012-02-16 21:25:02.000000000 +0100 sock.h
-rw-r--r-- 2 plobacz plobacz  3007 2016-01-27 17:13:18.000000000 +0100 ud_socket.c
-rw-r--r-- 2 plobacz plobacz   352 2012-03-21 00:54:18.000000000 +0100 ud_socket.h
plobacz@latitude:/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib$ ls --full-time /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleB/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/packages-split/acpid-src/usr/src/debug/acpid/2.0.34-r0/
razem 192
-rw-r--r-- 3 plobacz plobacz 10934 2019-07-31 14:21:00.861236271 +0200 acpid.c
-rw-r--r-- 3 plobacz plobacz  1670 2018-04-11 01:51:39.467499913 +0200 acpid.h
-rw-r--r-- 3 plobacz plobacz   878 2009-04-29 16:37:13.000000000 +0200 acpi_genetlink.h
-rw-r--r-- 3 plobacz plobacz  7339 2012-02-16 21:24:42.515701450 +0100 acpi_ids.c
-rw-r--r-- 3 plobacz plobacz  1118 2012-02-16 21:24:42.544701444 +0100 acpi_ids.h
-rw-r--r-- 3 plobacz plobacz  5264 2015-07-29 00:32:35.499948465 +0200 acpi_listen.c
-rw-r--r-- 3 plobacz plobacz  4583 2013-08-16 00:49:18.764134072 +0200 connection_list.c
-rw-r--r-- 3 plobacz plobacz  2725 2013-08-15 22:24:07.562169686 +0200 connection_list.h
-rw-r--r-- 3 plobacz plobacz 17402 2018-03-29 17:07:23.716843024 +0200 event.c
-rw-r--r-- 3 plobacz plobacz  1288 2013-12-19 03:42:46.529591162 +0100 event.h
-rw-r--r-- 3 plobacz plobacz  1656 2009-04-29 16:37:13.000000000 +0200 genetlink.h
-rw-r--r-- 3 plobacz plobacz  4943 2021-08-29 18:46:00.558827438 +0200 inotify_handler.c
-rw-r--r-- 3 plobacz plobacz  1094 2009-11-09 23:03:27.000000000 +0100 inotify_handler.h
-rw-r--r-- 3 plobacz plobacz 18544 2022-08-15 16:38:03.122942190 +0200 input_layer.c
-rw-r--r-- 3 plobacz plobacz  1257 2009-11-10 00:16:59.000000000 +0100 input_layer.h
drwxr-xr-x 2 plobacz plobacz  4096 2022-09-16 03:24:42.000000000 +0200 kacpimon
-rw-r--r-- 3 plobacz plobacz 13746 2021-09-20 03:31:56.196781952 +0200 libnetlink.c
-rw-r--r-- 3 plobacz plobacz  3028 2021-09-20 03:30:25.930632950 +0200 libnetlink.h
-rw-r--r-- 3 plobacz plobacz  1401 2018-03-29 17:08:05.842574608 +0200 log.c
-rw-r--r-- 3 plobacz plobacz  1269 2018-03-29 17:08:02.566518016 +0200 log.h
-rw-r--r-- 3 plobacz plobacz  6155 2021-09-20 03:32:03.418793923 +0200 netlink.c
-rw-r--r-- 3 plobacz plobacz  1066 2009-04-29 16:37:13.000000000 +0200 netlink.h
-rw-r--r-- 3 plobacz plobacz  4876 2015-07-29 00:32:35.453948235 +0200 proc.c
-rw-r--r-- 3 plobacz plobacz  1066 2012-02-16 21:25:02.924701352 +0100 proc.h
-rw-r--r-- 3 plobacz plobacz  5018 2022-09-16 03:24:42.000000000 +0200 sock.c
-rw-r--r-- 3 plobacz plobacz  1198 2012-02-16 21:25:02.925701357 +0100 sock.h
-rw-r--r-- 3 plobacz plobacz  3007 2016-01-27 17:13:18.836936981 +0100 ud_socket.c
-rw-r--r-- 3 plobacz plobacz   352 2012-03-21 00:54:18.929518037 +0100 ud_socket.h
plobacz@latitude:/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib$

And you see that for some reason for reproducibleA which uses sstate-cache miliseconds are lost,
due to exactly what Richard has written. Now I need to check one more thing how and were is that
exactly happening.

BR
Piotr

Od: Martin Jansa <martin.jansa@gmail.com>
Wysłane: wtorek, 1 sierpnia 2023 15:04
Do: Piotr Łobacz <p.lobacz@welotec.com>
DW: Richard Purdie <richard.purdie@linuxfoundation.org>; Joshua Watt <JPEWhacker@gmail.com>; Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

Just FYI:

Not sure if it's still relevant, but last time I was comparing some posix archives from different host OSes there were some differences other than timestamps I believe, this was 2.5 years ago showing differences between tar 1.29 on ubuntu 18.04 and 1.33 on tumbleweed:

https://git.openembedded.org/openembedded-core/commit/?id=3ecea58f2a3382d9f4b410d6ad7089111334cb6f

Cheers,

On Mon, Jul 31, 2023 at 11:28 PM Piotr Łobacz <p.lobacz@welotec.com> wrote:


> I'd guess you need to make the sstate archives of do_package store the
> acl and extended attribute data and when that is done, the millisecond
> timestamp data will probably be saved too? I'm not sure this is going
> to be a generally useful thing in the general case though and zeroing
> the millisecond portion would really be easier in general if we could.
> Certainly, SOURCE_DATE_EPOCH doesn't have millisecond precision.

This is what we were talking about on tar mailing list, to remove this precision for posix format.
Additionally in case of acl and extended attribute data they need to be also added to this sstate
archives, which still means that it needs to be extended with posix format.

BR
Piotr

Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Richard Purdie]

On Tue, 2023-08-01 at 15:16 +0000, Piotr Łobacz wrote:
> Hi Richard,
> you were right. I have made another test with wiping out whole sstate-cache for acpid and rerunning the whole reproducible test.
> It occurs that the package /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build/sstate-cache/a7/41/sstate:acpid:core2-64-poky-linux:2.0.34:r0:core2-64:11:a741afe7c0bfc218a844a546735ddc4dcaf578bff5d6a3762b1bdedf0819fc9b_package.tar.zst was already containing wrong timestamps.
> 
> > I'd guess you need to make the sstate archives of do_package store the
> > acl and extended attribute data and when that is done, the millisecond
> > timestamp data will probably be saved too?
> 
> So I'm guessing that commands which are producing this package_name_package.tar.zst are producing it in gnu format. Question is where can I find it?
> Is it being produced in this sstate.bbclass or elsewhere?

https://git.yoctoproject.org/poky/tree/meta/classes-global/sstate.bbclass#n859

tar -I "$ZSTD" $OPT -f $TFILE *

or

tar -I "$ZSTD" $OPT --file=$TFILE --files-from=/dev/null

Cheers,

Richard

ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

[-- Attachment #1: Type: text/plain, Size: 2858 bytes --]

That is what I was thinking for the first time, but by default tar is being run with posix,
so, it is not possible that these commands are responsible for this...

My guess is about do_package from package.bbclass and it calls this createCopy function which calls copyhardlinktree.
And to my supprise there is also tar command, but it is written there:

        # Need to copy directories only with tar first since cp will error if two
        # writers try and create a directory at the same time

and afterwards is this cp command with --preserve=xattrs and according to cp manual we can also preserve timestamps.

Correct me if my thinking is wrong.

I'm giving it a shot and will inform you about progress.

BR
Piotr
________________________________
Od: Richard Purdie <richard.purdie@linuxfoundation.org>
Wysłane: wtorek, 1 sierpnia 2023 17:41
Do: Piotr Łobacz <p.lobacz@welotec.com>; Martin Jansa <martin.jansa@gmail.com>
DW: Joshua Watt <JPEWhacker@gmail.com>; Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

On Tue, 2023-08-01 at 15:16 +0000, Piotr Łobacz wrote:
> Hi Richard,
> you were right. I have made another test with wiping out whole sstate-cache for acpid and rerunning the whole reproducible test.
> It occurs that the package /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build/sstate-cache/a7/41/sstate:acpid:core2-64-poky-linux:2.0.34:r0:core2-64:11:a741afe7c0bfc218a844a546735ddc4dcaf578bff5d6a3762b1bdedf0819fc9b_package.tar.zst was already containing wrong timestamps.
>
> > I'd guess you need to make the sstate archives of do_package store the
> > acl and extended attribute data and when that is done, the millisecond
> > timestamp data will probably be saved too?
>
> So I'm guessing that commands which are producing this package_name_package.tar.zst are producing it in gnu format. Question is where can I find it?
> Is it being produced in this sstate.bbclass or elsewhere?

https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.yoctoproject.org%2Fpoky%2Ftree%2Fmeta%2Fclasses-global%2Fsstate.bbclass%23n859&data=05%7C01%7Cp.lobacz%40welotec.com%7C6ede1a2d1d444ff526c208db92a5cf36%7C25111a7f1d5a4c51a4ca7f8e44011b39%7C0%7C0%7C638265013099817702%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1JiMQn00kK9Kddz6%2FMhPVKThB3crvKNTz9%2FrTOt14hM%3D&reserved=0<https://git.yoctoproject.org/poky/tree/meta/classes-global/sstate.bbclass#n859>

tar -I "$ZSTD" $OPT -f $TFILE *

or

tar -I "$ZSTD" $OPT --file=$TFILE --files-from=/dev/null

Cheers,

Richard


[-- Attachment #2: Type: text/html, Size: 6745 bytes --]

ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

[Piotr Łobacz]

Hi Richard,
back again, you were absolutely right I have added in here https://git.yoctoproject.org/poky/tree/meta/classes-global/sstate.bbclass#n847
stupid check `ls -R --full-time "${SSTATE_BUILDDIR}" >> /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/sstate_pkg.log`
and than run again the test.

Now, I'm sorry for this huge spam but I want to show what happens:

${SSTATE_PKG}=/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build/sstate-cache/81/96/sstate:acpid:core2-64-poky-linux:2.0.34:r0:core2-64:11:8196f1f3e3094158d888b6ead4b56624c52fa363d1133c447bfc88398e5f7fc6_package.tar.zst ${SSTATE_BUILDDIR}=/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/
/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/:
total 12
drwxr-xr-x 5 root root 4096 2022-09-16 01:24:42.000000000 +0000 package
drwxr-xr-x 9 root root 4096 2022-09-16 01:24:42.000000000 +0000 packages-split
drwxr-xr-x 7 root root 4096 2022-09-16 01:24:42.000000000 +0000 pkgdata

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package:
total 12
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 etc
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 lib
drwxr-xr-x 6 root root 4096 2022-09-16 01:24:42.000000000 +0000 usr

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/etc:
total 8
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpi
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 init.d

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/etc/acpi:
total 4
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 events

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/etc/acpi/events:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/etc/init.d:
total 4
-rwxr-xr-x 2 root root 728 2022-09-16 01:24:42.000000000 +0000 acpid

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/lib:
total 4
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 systemd

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/lib/systemd:
total 8
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 system
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 system-preset

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/lib/systemd/system:
total 4
-rw-r--r-- 2 root root 145 2022-09-16 01:24:42.000000000 +0000 acpid.service

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/lib/systemd/system-preset:
total 4
-rw-r--r-- 2 root root 21 2022-09-16 01:24:42.000000000 +0000 98-acpid.preset

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr:
total 16
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 bin
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 sbin
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 share
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 src

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/bin:
total 44
-rwxr-xr-x 2 root root 43008 2022-09-16 01:24:42.000000000 +0000 acpi_listen

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/sbin:
total 288
-rwxr-xr-x 2 root root 193320 2022-09-16 01:24:42.000000000 +0000 acpid
-rwxr-xr-x 2 root root  94856 2022-09-16 01:24:42.000000000 +0000 kacpimon

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/share:
total 8
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 doc
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 man

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/share/doc:
total 4
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/share/doc/acpid:
total 60
-rw-r--r-- 2 root root 27404 2022-09-16 01:24:42.000000000 +0000 Changelog
-rw-r--r-- 2 root root 17982 2022-09-16 01:24:42.000000000 +0000 COPYING
-rw-r--r-- 2 root root  3069 2022-09-16 01:24:42.000000000 +0000 README
-rw-r--r-- 2 root root  3639 2022-09-16 01:24:42.000000000 +0000 TESTPLAN
-rw-r--r-- 2 root root  1289 2022-09-16 01:24:42.000000000 +0000 TODO

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/share/man:
total 4
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 man8

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/share/man/man8:
total 20
-rw-r--r-- 2 root root 9927 2022-09-16 01:24:42.000000000 +0000 acpid.8
-rw-r--r-- 2 root root 1296 2022-09-16 01:24:42.000000000 +0000 acpi_listen.8
-rw-r--r-- 2 root root 2280 2022-09-16 01:24:42.000000000 +0000 kacpimon.8

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/src:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 debug

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/src/debug:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/src/debug/acpid:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 2.0.34-r0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/src/debug/acpid/2.0.34-r0:
total 192
-rw-r--r-- 3 root root 10934 2019-07-31 12:21:00.861236271 +0000 acpid.c
-rw-r--r-- 3 root root  1670 2018-04-10 23:51:39.467499913 +0000 acpid.h
-rw-r--r-- 3 root root   878 2009-04-29 14:37:13.000000000 +0000 acpi_genetlink.h
-rw-r--r-- 3 root root  7339 2012-02-16 20:24:42.515701450 +0000 acpi_ids.c
-rw-r--r-- 3 root root  1118 2012-02-16 20:24:42.544701444 +0000 acpi_ids.h
-rw-r--r-- 3 root root  5264 2015-07-28 22:32:35.499948465 +0000 acpi_listen.c
-rw-r--r-- 3 root root  4583 2013-08-15 22:49:18.764134072 +0000 connection_list.c
-rw-r--r-- 3 root root  2725 2013-08-15 20:24:07.562169686 +0000 connection_list.h
-rw-r--r-- 3 root root 17402 2018-03-29 15:07:23.716843024 +0000 event.c
-rw-r--r-- 3 root root  1288 2013-12-19 02:42:46.529591162 +0000 event.h
-rw-r--r-- 3 root root  1656 2009-04-29 14:37:13.000000000 +0000 genetlink.h
-rw-r--r-- 3 root root  4943 2021-08-29 16:46:00.558827438 +0000 inotify_handler.c
-rw-r--r-- 3 root root  1094 2009-11-09 22:03:27.000000000 +0000 inotify_handler.h
-rw-r--r-- 3 root root 18544 2022-08-15 14:38:03.122942190 +0000 input_layer.c
-rw-r--r-- 3 root root  1257 2009-11-09 23:16:59.000000000 +0000 input_layer.h
drwxr-xr-x 2 root root  4096 2022-09-16 01:24:42.000000000 +0000 kacpimon
-rw-r--r-- 3 root root 13746 2021-09-20 01:31:56.196781952 +0000 libnetlink.c
-rw-r--r-- 3 root root  3028 2021-09-20 01:30:25.930632950 +0000 libnetlink.h
-rw-r--r-- 3 root root  1401 2018-03-29 15:08:05.842574608 +0000 log.c
-rw-r--r-- 3 root root  1269 2018-03-29 15:08:02.566518016 +0000 log.h
-rw-r--r-- 3 root root  6155 2021-09-20 01:32:03.418793923 +0000 netlink.c
-rw-r--r-- 3 root root  1066 2009-04-29 14:37:13.000000000 +0000 netlink.h
-rw-r--r-- 3 root root  4876 2015-07-28 22:32:35.453948235 +0000 proc.c
-rw-r--r-- 3 root root  1066 2012-02-16 20:25:02.924701352 +0000 proc.h
-rw-r--r-- 3 root root  5018 2022-09-16 01:24:42.000000000 +0000 sock.c
-rw-r--r-- 3 root root  1198 2012-02-16 20:25:02.925701357 +0000 sock.h
-rw-r--r-- 3 root root  3007 2016-01-27 16:13:18.836936981 +0000 ud_socket.c
-rw-r--r-- 3 root root   352 2012-03-20 23:54:18.929518037 +0000 ud_socket.h

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/package/usr/src/debug/acpid/2.0.34-r0/kacpimon:
total 80
-rw-r--r-- 3 root root   878 2007-07-20 06:30:46.000000000 +0000 acpi_genetlink.h
-rw-r--r-- 3 root root  7172 2012-02-17 04:09:31.971838002 +0000 acpi_ids.c
-rw-r--r-- 3 root root  1118 2012-02-17 04:02:39.842852687 +0000 acpi_ids.h
-rw-r--r-- 3 root root  2788 2018-11-07 00:17:33.622466168 +0000 connection_list.c
-rw-r--r-- 3 root root  1851 2013-08-15 20:23:36.896169812 +0000 connection_list.h
-rw-r--r-- 3 root root  1656 2008-11-10 00:45:55.000000000 +0000 genetlink.h
-rw-r--r-- 3 root root  4075 2012-02-17 04:10:45.463835383 +0000 input_layer.c
-rw-r--r-- 3 root root  1151 2008-11-13 23:40:17.000000000 +0000 input_layer.h
-rw-r--r-- 3 root root  4650 2012-02-17 04:13:20.288829866 +0000 kacpimon.c
-rw-r--r-- 3 root root  1180 2008-12-14 22:09:24.000000000 +0000 kacpimon.h
-rw-r--r-- 3 root root 13720 2021-09-20 01:32:49.077869766 +0000 libnetlink.c
-rw-r--r-- 3 root root  3051 2021-09-20 01:32:38.182851644 +0000 libnetlink.h
-rw-r--r-- 3 root root  5304 2012-02-17 04:15:01.295826266 +0000 netlink.c
-rw-r--r-- 3 root root  1066 2008-11-13 23:40:06.000000000 +0000 netlink.h

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split:
total 32
drwxrwxrwx 5 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid
drwxrwxrwx 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid-dbg
drwxrwxrwx 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid-dev
drwxrwxrwx 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid-doc
drwxrwxrwx 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid-locale
-rw-r--r-- 1 root root   16 2022-09-16 01:24:42.000000000 +0000 acpid.shlibdeps
drwxrwxrwx 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid-src
drwxrwxrwx 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid-staticdev

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid:
total 12
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 etc
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 lib
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 usr

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/etc:
total 8
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpi
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 init.d

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/etc/acpi:
total 4
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 events

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/etc/acpi/events:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/etc/init.d:
total 4
-rwxr-xr-x 2 root root 728 2022-09-16 01:24:42.000000000 +0000 acpid

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/lib:
total 4
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 systemd

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/lib/systemd:
total 8
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 system
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 system-preset

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/lib/systemd/system:
total 4
-rw-r--r-- 2 root root 145 2022-09-16 01:24:42.000000000 +0000 acpid.service

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/lib/systemd/system-preset:
total 4
-rw-r--r-- 2 root root 21 2022-09-16 01:24:42.000000000 +0000 98-acpid.preset

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/usr:
total 8
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 bin
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 sbin

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/usr/bin:
total 44
-rwxr-xr-x 2 root root 43008 2022-09-16 01:24:42.000000000 +0000 acpi_listen

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid/usr/sbin:
total 288
-rwxr-xr-x 2 root root 193320 2022-09-16 01:24:42.000000000 +0000 acpid
-rwxr-xr-x 2 root root  94856 2022-09-16 01:24:42.000000000 +0000 kacpimon

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-dbg:
total 4
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 usr

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-dbg/usr:
total 8
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 bin
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 sbin

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-dbg/usr/bin:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-dbg/usr/sbin:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-dev:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-doc:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 usr

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-doc/usr:
total 4
drwxr-xr-x 4 root root 4096 2022-09-16 01:24:42.000000000 +0000 share

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-doc/usr/share:
total 8
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 doc
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 man

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-doc/usr/share/doc:
total 4
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-doc/usr/share/doc/acpid:
total 60
-rw-r--r-- 2 root root 27404 2022-09-16 01:24:42.000000000 +0000 Changelog
-rw-r--r-- 2 root root 17982 2022-09-16 01:24:42.000000000 +0000 COPYING
-rw-r--r-- 2 root root  3069 2022-09-16 01:24:42.000000000 +0000 README
-rw-r--r-- 2 root root  3639 2022-09-16 01:24:42.000000000 +0000 TESTPLAN
-rw-r--r-- 2 root root  1289 2022-09-16 01:24:42.000000000 +0000 TODO

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-doc/usr/share/man:
total 4
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 man8

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-doc/usr/share/man/man8:
total 20
-rw-r--r-- 2 root root 9927 2022-09-16 01:24:42.000000000 +0000 acpid.8
-rw-r--r-- 2 root root 1296 2022-09-16 01:24:42.000000000 +0000 acpi_listen.8
-rw-r--r-- 2 root root 2280 2022-09-16 01:24:42.000000000 +0000 kacpimon.8

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-locale:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-src:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 usr

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-src/usr:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 src

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-src/usr/src:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 debug

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-src/usr/src/debug:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 acpid

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-src/usr/src/debug/acpid:
total 4
drwxr-xr-x 3 root root 4096 2022-09-16 01:24:42.000000000 +0000 2.0.34-r0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-src/usr/src/debug/acpid/2.0.34-r0:
total 192
-rw-r--r-- 3 root root 10934 2019-07-31 12:21:00.861236271 +0000 acpid.c
-rw-r--r-- 3 root root  1670 2018-04-10 23:51:39.467499913 +0000 acpid.h
-rw-r--r-- 3 root root   878 2009-04-29 14:37:13.000000000 +0000 acpi_genetlink.h
-rw-r--r-- 3 root root  7339 2012-02-16 20:24:42.515701450 +0000 acpi_ids.c
-rw-r--r-- 3 root root  1118 2012-02-16 20:24:42.544701444 +0000 acpi_ids.h
-rw-r--r-- 3 root root  5264 2015-07-28 22:32:35.499948465 +0000 acpi_listen.c
-rw-r--r-- 3 root root  4583 2013-08-15 22:49:18.764134072 +0000 connection_list.c
-rw-r--r-- 3 root root  2725 2013-08-15 20:24:07.562169686 +0000 connection_list.h
-rw-r--r-- 3 root root 17402 2018-03-29 15:07:23.716843024 +0000 event.c
-rw-r--r-- 3 root root  1288 2013-12-19 02:42:46.529591162 +0000 event.h
-rw-r--r-- 3 root root  1656 2009-04-29 14:37:13.000000000 +0000 genetlink.h
-rw-r--r-- 3 root root  4943 2021-08-29 16:46:00.558827438 +0000 inotify_handler.c
-rw-r--r-- 3 root root  1094 2009-11-09 22:03:27.000000000 +0000 inotify_handler.h
-rw-r--r-- 3 root root 18544 2022-08-15 14:38:03.122942190 +0000 input_layer.c
-rw-r--r-- 3 root root  1257 2009-11-09 23:16:59.000000000 +0000 input_layer.h
drwxr-xr-x 2 root root  4096 2022-09-16 01:24:42.000000000 +0000 kacpimon
-rw-r--r-- 3 root root 13746 2021-09-20 01:31:56.196781952 +0000 libnetlink.c
-rw-r--r-- 3 root root  3028 2021-09-20 01:30:25.930632950 +0000 libnetlink.h
-rw-r--r-- 3 root root  1401 2018-03-29 15:08:05.842574608 +0000 log.c
-rw-r--r-- 3 root root  1269 2018-03-29 15:08:02.566518016 +0000 log.h
-rw-r--r-- 3 root root  6155 2021-09-20 01:32:03.418793923 +0000 netlink.c
-rw-r--r-- 3 root root  1066 2009-04-29 14:37:13.000000000 +0000 netlink.h
-rw-r--r-- 3 root root  4876 2015-07-28 22:32:35.453948235 +0000 proc.c
-rw-r--r-- 3 root root  1066 2012-02-16 20:25:02.924701352 +0000 proc.h
-rw-r--r-- 3 root root  5018 2022-09-16 01:24:42.000000000 +0000 sock.c
-rw-r--r-- 3 root root  1198 2012-02-16 20:25:02.925701357 +0000 sock.h
-rw-r--r-- 3 root root  3007 2016-01-27 16:13:18.836936981 +0000 ud_socket.c
-rw-r--r-- 3 root root   352 2012-03-20 23:54:18.929518037 +0000 ud_socket.h

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-src/usr/src/debug/acpid/2.0.34-r0/kacpimon:
total 80
-rw-r--r-- 3 root root   878 2007-07-20 06:30:46.000000000 +0000 acpi_genetlink.h
-rw-r--r-- 3 root root  7172 2012-02-17 04:09:31.971838002 +0000 acpi_ids.c
-rw-r--r-- 3 root root  1118 2012-02-17 04:02:39.842852687 +0000 acpi_ids.h
-rw-r--r-- 3 root root  2788 2018-11-07 00:17:33.622466168 +0000 connection_list.c
-rw-r--r-- 3 root root  1851 2013-08-15 20:23:36.896169812 +0000 connection_list.h
-rw-r--r-- 3 root root  1656 2008-11-10 00:45:55.000000000 +0000 genetlink.h
-rw-r--r-- 3 root root  4075 2012-02-17 04:10:45.463835383 +0000 input_layer.c
-rw-r--r-- 3 root root  1151 2008-11-13 23:40:17.000000000 +0000 input_layer.h
-rw-r--r-- 3 root root  4650 2012-02-17 04:13:20.288829866 +0000 kacpimon.c
-rw-r--r-- 3 root root  1180 2008-12-14 22:09:24.000000000 +0000 kacpimon.h
-rw-r--r-- 3 root root 13720 2021-09-20 01:32:49.077869766 +0000 libnetlink.c
-rw-r--r-- 3 root root  3051 2021-09-20 01:32:38.182851644 +0000 libnetlink.h
-rw-r--r-- 3 root root  5304 2012-02-17 04:15:01.295826266 +0000 netlink.c
-rw-r--r-- 3 root root  1066 2008-11-13 23:40:06.000000000 +0000 netlink.h

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/packages-split/acpid-staticdev:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/pkgdata:
total 24
-rw-r--r-- 1 root root   85 2022-09-16 01:24:42.000000000 +0000 acpid
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 extended
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 runtime
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 runtime-reverse
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 runtime-rprovides
drwxr-xr-x 2 root root 4096 2022-09-16 01:24:42.000000000 +0000 shlibs2

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/pkgdata/extended:
total 28
-rw-r--r-- 1 root root  121 2022-09-16 01:24:42.000000000 +0000 acpid-dbg.json.zstd
-rw-r--r-- 1 root root   42 2022-09-16 01:24:42.000000000 +0000 acpid-dev.json.zstd
-rw-r--r-- 1 root root  193 2022-09-16 01:24:42.000000000 +0000 acpid-doc.json.zstd
-rw-r--r-- 1 root root 1138 2022-09-16 01:24:42.000000000 +0000 acpid.json.zstd
-rw-r--r-- 1 root root   42 2022-09-16 01:24:42.000000000 +0000 acpid-locale.json.zstd
-rw-r--r-- 1 root root  478 2022-09-16 01:24:42.000000000 +0000 acpid-src.json.zstd
-rw-r--r-- 1 root root   42 2022-09-16 01:24:42.000000000 +0000 acpid-staticdev.json.zstd

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/pkgdata/runtime:
total 28
-rw-r--r-- 1 root root 3728 2022-09-16 01:24:42.000000000 +0000 acpid
-rw-r--r-- 1 root root  967 2022-09-16 01:24:42.000000000 +0000 acpid-dbg
-rw-r--r-- 1 root root    0 2022-09-16 01:24:42.000000000 +0000 acpid-dbg.packaged
-rw-r--r-- 1 root root 1164 2022-09-16 01:24:42.000000000 +0000 acpid-dev
-rw-r--r-- 1 root root    0 2022-09-16 01:24:42.000000000 +0000 acpid-dev.packaged
-rw-r--r-- 1 root root 1119 2022-09-16 01:24:42.000000000 +0000 acpid-doc
-rw-r--r-- 1 root root    0 2022-09-16 01:24:42.000000000 +0000 acpid-doc.packaged
-rw-r--r-- 1 root root  651 2022-09-16 01:24:42.000000000 +0000 acpid-locale
-rw-r--r-- 1 root root    0 2022-09-16 01:24:42.000000000 +0000 acpid.packaged
-rw-r--r-- 1 root root 2995 2022-09-16 01:24:42.000000000 +0000 acpid-src
-rw-r--r-- 1 root root    0 2022-09-16 01:24:42.000000000 +0000 acpid-src.packaged
-rw-r--r-- 1 root root  907 2022-09-16 01:24:42.000000000 +0000 acpid-staticdev

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/pkgdata/runtime-reverse:
total 0
lrwxrwxrwx 1 root root 16 2022-09-16 01:24:42.000000000 +0000 acpid -> ../runtime/acpid
lrwxrwxrwx 1 root root 20 2022-09-16 01:24:42.000000000 +0000 acpid-dbg -> ../runtime/acpid-dbg
lrwxrwxrwx 1 root root 20 2022-09-16 01:24:42.000000000 +0000 acpid-dev -> ../runtime/acpid-dev
lrwxrwxrwx 1 root root 20 2022-09-16 01:24:42.000000000 +0000 acpid-doc -> ../runtime/acpid-doc
lrwxrwxrwx 1 root root 20 2022-09-16 01:24:42.000000000 +0000 acpid-src -> ../runtime/acpid-src

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/pkgdata/runtime-rprovides:
total 0

/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/reproducibleA/tmp/work/core2-64-poky-linux/acpid/2.0.34-r0/sstate-build-package/pkgdata/shlibs2:
total 0

As you can see all data have correct timestamps and when I extract this file /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build/sstate-cache/81/96/sstate:acpid:core2-64-poky-linux:2.0.34:r0:core2-64:11:8196f1f3e3094158d888b6ead4b56624c52fa363d1133c447bfc88398e5f7fc6_package.tar.zst the disapper e.g.

./package/usr/src/debug/acpid/2.0.34-r0:
razem 192
-rw-r--r-- 2 plobacz plobacz 10934 2019-07-31 14:21:00.000000000 +0200 acpid.c
-rw-r--r-- 2 plobacz plobacz  1670 2018-04-11 01:51:39.000000000 +0200 acpid.h
-rw-r--r-- 2 plobacz plobacz   878 2009-04-29 16:37:13.000000000 +0200 acpi_genetlink.h
-rw-r--r-- 2 plobacz plobacz  7339 2012-02-16 21:24:42.000000000 +0100 acpi_ids.c
-rw-r--r-- 2 plobacz plobacz  1118 2012-02-16 21:24:42.000000000 +0100 acpi_ids.h
-rw-r--r-- 2 plobacz plobacz  5264 2015-07-29 00:32:35.000000000 +0200 acpi_listen.c
-rw-r--r-- 2 plobacz plobacz  4583 2013-08-16 00:49:18.000000000 +0200 connection_list.c
-rw-r--r-- 2 plobacz plobacz  2725 2013-08-15 22:24:07.000000000 +0200 connection_list.h
-rw-r--r-- 2 plobacz plobacz 17402 2018-03-29 17:07:23.000000000 +0200 event.c
-rw-r--r-- 2 plobacz plobacz  1288 2013-12-19 03:42:46.000000000 +0100 event.h
-rw-r--r-- 2 plobacz plobacz  1656 2009-04-29 16:37:13.000000000 +0200 genetlink.h
-rw-r--r-- 2 plobacz plobacz  4943 2021-08-29 18:46:00.000000000 +0200 inotify_handler.c
-rw-r--r-- 2 plobacz plobacz  1094 2009-11-09 23:03:27.000000000 +0100 inotify_handler.h
-rw-r--r-- 2 plobacz plobacz 18544 2022-08-15 16:38:03.000000000 +0200 input_layer.c
-rw-r--r-- 2 plobacz plobacz  1257 2009-11-10 00:16:59.000000000 +0100 input_layer.h
drwxr-xr-x 2 plobacz plobacz  4096 2022-09-16 03:24:42.000000000 +0200 kacpimon
-rw-r--r-- 2 plobacz plobacz 13746 2021-09-20 03:31:56.000000000 +0200 libnetlink.c
-rw-r--r-- 2 plobacz plobacz  3028 2021-09-20 03:30:25.000000000 +0200 libnetlink.h
-rw-r--r-- 2 plobacz plobacz  1401 2018-03-29 17:08:05.000000000 +0200 log.c
-rw-r--r-- 2 plobacz plobacz  1269 2018-03-29 17:08:02.000000000 +0200 log.h
-rw-r--r-- 2 plobacz plobacz  6155 2021-09-20 03:32:03.000000000 +0200 netlink.c
-rw-r--r-- 2 plobacz plobacz  1066 2009-04-29 16:37:13.000000000 +0200 netlink.h
-rw-r--r-- 2 plobacz plobacz  4876 2015-07-29 00:32:35.000000000 +0200 proc.c
-rw-r--r-- 2 plobacz plobacz  1066 2012-02-16 21:25:02.000000000 +0100 proc.h
-rw-r--r-- 2 plobacz plobacz  5018 2022-09-16 03:24:42.000000000 +0200 sock.c
-rw-r--r-- 2 plobacz plobacz  1198 2012-02-16 21:25:02.000000000 +0100 sock.h
-rw-r--r-- 2 plobacz plobacz  3007 2016-01-27 17:13:18.000000000 +0100 ud_socket.c
-rw-r--r-- 2 plobacz plobacz   352 2012-03-21 00:54:18.000000000 +0100 ud_socket.h

This is rather unexpected, because by default tar is in posix format. I need to talk about it with tar people, but hell yeah finally we got it :-)

BR
Piotr

Od: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> w imieniu użytkownika Piotr Łobacz via lists.openembedded.org <p.lobacz=welotec.com@lists.openembedded.org>
Wysłane: wtorek, 1 sierpnia 2023 18:00
Do: Richard Purdie <richard.purdie@linuxfoundation.org>; Martin Jansa <martin.jansa@gmail.com>
DW: Joshua Watt <JPEWhacker@gmail.com>; Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: ODP: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

That is what I was thinking for the first time, but by default tar is being run with posix,
so, it is not possible that these commands are responsible for this...

My guess is about do_package from package.bbclass and it calls this createCopy function which calls copyhardlinktree.
And to my supprise there is also tar command, but it is written there:

        # Need to copy directories only with tar first since cp will error if two
        # writers try and create a directory at the same time

and afterwards is this cp command with --preserve=xattrs and according to cp manual we can also preserve timestamps.

Correct me if my thinking is wrong.

I'm giving it a shot and will inform you about progress.

BR
Piotr

Od: Richard Purdie <richard.purdie@linuxfoundation.org>
Wysłane: wtorek, 1 sierpnia 2023 17:41
Do: Piotr Łobacz <p.lobacz@welotec.com>; Martin Jansa <martin.jansa@gmail.com>
DW: Joshua Watt <JPEWhacker@gmail.com>; Alexandre Belloni <alexandre.belloni@bootlin.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>; Alex Stewart <alex.stewart@ni.com>
Temat: Re: ODP: [OE-Core][PATCH v11][master-next 1/5] package_ipk.bbclass: add support for ACLs and xattr

On Tue, 2023-08-01 at 15:16 +0000, Piotr Łobacz wrote:
> Hi Richard,
> you were right. I have made another test with wiping out whole sstate-cache for acpid and rerunning the whole reproducible test.
> It occurs that the package /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build/sstate-cache/a7/41/sstate:acpid:core2-64-poky-linux:2.0.34:r0:core2-64:11:a741afe7c0bfc218a844a546735ddc4dcaf578bff5d6a3762b1bdedf0819fc9b_package.tar.zst was already containing wrong timestamps.
>
> > I'd guess you need to make the sstate archives of do_package store the
> > acl and extended attribute data and when that is done, the millisecond
> > timestamp data will probably be saved too?
>
> So I'm guessing that commands which are producing this package_name_package.tar.zst are producing it in gnu format. Question is where can I find it?
> Is it being produced in this sstate.bbclass or elsewhere?

https://git.yoctoproject.org/poky/tree/meta/classes-global/sstate.bbclass#n859

tar -I "$ZSTD" $OPT -f $TFILE *

or

tar -I "$ZSTD" $OPT --file=$TFILE --files-from=/dev/null

Cheers,

Richard