[OE-core][kirkstone][PATCH v2 1/4] package.bbclase: Add ACLs and xattrs to perform_packagecopy

[OE-core][kirkstone][PATCH v2 1/4] package.bbclase: Add ACLs and xattrs to perform_packagecopy

[Piotr Łobacz]

This is necessary in order to preserve all the extended attributes
set in do_install also in PKGD directory, otherwise they are
dropped and not stored in created package(s).

For the momemnt it is only supported by OPKG package manager
as it has support for libarchive and tar implementations, which
supports ACLs and xattrs.

Fixes: [YOCTO #9858]

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 meta/classes/package.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 2950218145..b0334d9190 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -863,8 +863,8 @@ python perform_packagecopy () {
 
     # Start by package population by taking a copy of the installed
     # files to operate on
-    # Preserve sparse files and hard links
-    cmd = 'tar --exclude=./sysroot-only -cf - -C %s -p -S . | tar -xf - -C %s' % (dest, dvar)
+    # Preserve sparse files, hard links, ACLs and extended attributes
+    cmd = 'tar --acls --xattrs --exclude=./sysroot-only -cf - -C %s -p -S . | tar --acls --xattrs -xf - -C %s' % (dest, dvar)
     subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 
     # replace RPATHs for the nativesdk binaries, to make them relocatable
-- 
2.34.1

[OE-core][kirkstone][PATCH v2 2/4] libarchive: remove acls, xattr for target only

[Piotr Łobacz]

Libarchive is being used by OPKG package manager as default
API for extracting tar files. This fix allows us to extract
ipks packages with preserved ACLs and xattrs.

Fixes: [YOCTO #15091]

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 meta/recipes-extended/libarchive/libarchive_3.6.2.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-extended/libarchive/libarchive_3.6.2.bb b/meta/recipes-extended/libarchive/libarchive_3.6.2.bb
index acc84de9da..15b1a88ce9 100644
--- a/meta/recipes-extended/libarchive/libarchive_3.6.2.bb
+++ b/meta/recipes-extended/libarchive/libarchive_3.6.2.bb
@@ -9,7 +9,7 @@ DEPENDS = "e2fsprogs-native"
 
 PACKAGECONFIG ?= "zlib bz2 xz lzo zstd"
 
-PACKAGECONFIG:append:class-target = "\
+PACKAGECONFIG:append = "\
 	${@bb.utils.filter('DISTRO_FEATURES', 'acl xattr', d)} \
 "
 
-- 
2.34.1

[OE-core][kirkstone][PATCH v2 3/4] opkg_archive: support ACLs, extended flags and attributes

[Piotr Łobacz]

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 ...t-ACLs-extended-flags-and-attributes.patch | 41 +++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.5.0.bb      |  1 +
 2 files changed, 42 insertions(+)
 create mode 100644 meta/recipes-devtools/opkg/opkg/0002-opkg_archive-support-ACLs-extended-flags-and-attributes.patch

diff --git a/meta/recipes-devtools/opkg/opkg/0002-opkg_archive-support-ACLs-extended-flags-and-attributes.patch b/meta/recipes-devtools/opkg/opkg/0002-opkg_archive-support-ACLs-extended-flags-and-attributes.patch
new file mode 100644
index 0000000000..9a90795245
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/0002-opkg_archive-support-ACLs-extended-flags-and-attributes.patch
@@ -0,0 +1,41 @@
+From 3b433f1df6c80bc380a367b86fdc149eb33e2674 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
+Date: Mon, 3 Apr 2023 03:12:17 +0200
+Subject: [PATCH] opkg_archive: support ACLs, extended flags and attributes
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+Libarchive is supporting extracting ACLS, extended flags
+and extended attributes, so from now we can turn on this
+extract flags.
+
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ libopkg/opkg_archive.c | 6 ++++--
+ 1 file changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/libopkg/opkg_archive.c b/libopkg/opkg_archive.c
+index 03a4afb..632bef8 100644
+--- a/libopkg/opkg_archive.c
++++ b/libopkg/opkg_archive.c
+@@ -905,12 +905,14 @@ struct opkg_ar *ar_open_pkg_data_archive(const char *filename)
+ 
+     /** Flags:
+      *
+-     * TODO: Do we want to support ACLs, extended flags and extended
++     * TODO: This should be done with some verification from configure,
++     * if we want to support ACLs, extended flags and extended
+      * attributes? (ARCHIVE_EXTRACT_ACL, ARCHIVE_EXTRACT_FFLAGS,
+      * ARCHIVE_EXTRACT_XATTR).
+      */
+     ar->extract_flags = ARCHIVE_EXTRACT_OWNER | ARCHIVE_EXTRACT_PERM |
+-        ARCHIVE_EXTRACT_TIME | ARCHIVE_EXTRACT_UNLINK | ARCHIVE_EXTRACT_NO_OVERWRITE;
++        ARCHIVE_EXTRACT_TIME | ARCHIVE_EXTRACT_UNLINK | ARCHIVE_EXTRACT_NO_OVERWRITE |
++        ARCHIVE_EXTRACT_ACL | ARCHIVE_EXTRACT_FFLAGS | ARCHIVE_EXTRACT_XATTR;
+ 
+     if (opkg_config->ignore_uid)
+         ar->extract_flags &= ~ARCHIVE_EXTRACT_OWNER;
+-- 
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.5.0.bb b/meta/recipes-devtools/opkg/opkg_0.5.0.bb
index 7bddaa3016..8ce2cd7863 100644
--- a/meta/recipes-devtools/opkg/opkg_0.5.0.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.5.0.bb
@@ -15,6 +15,7 @@ PE = "1"
 SRC_URI = "http://downloads.yoctoproject.org/releases/${BPN}/${BPN}-${PV}.tar.gz \
            file://opkg.conf \
            file://0001-opkg_conf-create-opkg.lock-in-run-instead-of-var-run.patch \
+           file://0002-opkg_archive-support-ACLs-extended-flags-and-attributes.patch
            file://run-ptest \
 "
 
-- 
2.34.1

[OE-core][kirkstone][PATCH v2 4/4] opkg-build: support ACLs and extended attributes

[Piotr Łobacz]

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 ...support-ACLs-and-extended-attributes.patch | 52 +++++++++++++++++++
 .../opkg-utils/opkg-utils_0.5.0.bb            |  1 +
 2 files changed, 53 insertions(+)
 create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-support-ACLs-and-extended-attributes.patch

diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-support-ACLs-and-extended-attributes.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-support-ACLs-and-extended-attributes.patch
new file mode 100644
index 0000000000..0744453ce9
--- /dev/null
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-support-ACLs-and-extended-attributes.patch
@@ -0,0 +1,52 @@
+From 61dcbe59656e0100dd13b980c530bf64ece78b37 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com>
+Date: Sun, 2 Apr 2023 01:10:05 +0200
+Subject: [PATCH] opkg-build: support ACLs and extended attributes
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+Add support for tar archives created with --acls, --xattrs options and POSIX header format.
+
+GNU tar and libarchive already supports ACLs and extended attributes. We can now
+add this support as well to opkg-build script in order to use setfacl or setcap
+inside do_install command and end up with a file in an image with the relevant
+extended file attributes.
+
+Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
+---
+ opkg-build | 5 ++---
+ 1 file changed, 2 insertions(+), 3 deletions(-)
+
+diff --git a/opkg-build b/opkg-build
+index a9e45d4..eb764a5 100755
+--- a/opkg-build
++++ b/opkg-build
+@@ -159,14 +159,13 @@ compressorargs=""
+ #    - Busybox tar doesn't support '--format'
+ #    - On some Linux distros, tar now defaults to posix format if '--format'
+ #      isn't explicitly specified
+-#    - Opkg doesn't currently support posix format archives
+ #
+ # It's easier to check for mention of the '--format' option than to detect the
+ # tar implementation and maintain a list of which support '--format'.
+ tarformat=""
+ if tar --help 2>&1 | grep -- "--format" > /dev/null;
+ then
+-    tarformat="--format=gnu"
++    tarformat="--format=posix"
+ fi
+ 
+ compressor_ext() {
+@@ -314,7 +313,7 @@ export LANG=C
+ export LC_ALL=C
+ ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list )
+ ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print  | sort > $tmp_dir/file_list )
+-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
++( cd $pkg_dir && tar --xattrs --acls $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
+ ( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
+ rm $tmp_dir/file_list
+ rm $tmp_dir/control_list
+-- 
+2.34.1
+
diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.5.0.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.5.0.bb
index b27e3ded33..9349ba06d2 100644
--- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.5.0.bb
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.5.0.bb
@@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu
 
 SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \
            file://0001-update-alternatives-correctly-match-priority.patch \
+           file://0002-opkg-build-support-ACLs-and-extended-attributes.patch \
            "
 SRCREV = "9239541f14a2529b9d01c0a253ab11afa2822dab"
 
-- 
2.34.1

Re: [OE-core][kirkstone][PATCH v2 1/4] package.bbclase: Add ACLs and xattrs to perform_packagecopy

[Richard Purdie]

On Mon, 2023-04-03 at 14:24 +0200, Piotr Łobacz wrote:
> This is necessary in order to preserve all the extended attributes
> set in do_install also in PKGD directory, otherwise they are
> dropped and not stored in created package(s).
> 
> For the momemnt it is only supported by OPKG package manager
> as it has support for libarchive and tar implementations, which
> supports ACLs and xattrs.
> 
> Fixes: [YOCTO #9858]
> 
> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
> ---
>  meta/classes/package.bbclass | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index 2950218145..b0334d9190 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -863,8 +863,8 @@ python perform_packagecopy () {
>  
>      # Start by package population by taking a copy of the installed
>      # files to operate on
> -    # Preserve sparse files and hard links
> -    cmd = 'tar --exclude=./sysroot-only -cf - -C %s -p -S . | tar -xf - -C %s' % (dest, dvar)
> +    # Preserve sparse files, hard links, ACLs and extended attributes
> +    cmd = 'tar --acls --xattrs --exclude=./sysroot-only -cf - -C %s -p -S . | tar --acls --xattrs -xf - -C %s' % (dest, dvar)
>      subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
>  
>      # replace RPATHs for the nativesdk binaries, to make them relocatable

Just to be clear, new feature development work happens on master, not
on our LTS stable release branches. That means these can't merge for
kirkstone and they don't apply to master so whilst I appreciate you
sharing them, they're unlikely to move forward without someone picking
them up and working on master with them.

Cheers,

Richard