[master][PATCH v2 1/2] image_types: Add 7-Zip support in conversion types and commands

[master][PATCH v2 1/2] image_types: Add 7-Zip support in conversion types and commands

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=3434; bh=eNxjqWZU+tl0AZ9YUgtOaR4TIY8Bfpl2P5DPwepueXQ=;

	b=wZMUkEkKc6ThvsmW3sPpIhapOvyXPbXHUBUcKuOPSIqiWwX2XmUI5wb7i3tH9dPr

	oZ6iXMS7mEaEMjbYwm3M38lCFbxOYwa1xlnJytVTglO/jcnUOVHoLb52SafjQUwxoq5

	GJ+IWY63T/eA364H8gmAkDBRcla5luP0tWv4CiRtWL0eEvk65y2e2+9iKzuWdWlilg3

	ArvG8gvjRJkeN8Mc1D80H9kTALxV/TIoiflaHJBBr+kxNKHXQb9+/TNN/dRxaywwvun

	RUm7Iji0qYe6gbtyAK5gfM9KRvZ667MbkWFMOiDZ2p4c1bKIMFvmVc3W+z0mRbI4G11

	X16sWBsOCA==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

Add 7-Zip support in conversion types and commands. Default arguments of =
7-Zip are compression level: 9, method: BZip2, extension: zip. All these =
are overridable from local.con for any other use case in user side. First=
 goal was to make an alternative zip conversion which can operate faster =
in multi-thread systems (on many CPU core), normal zip is just a single t=
hread compression. 7-Zip can do a zip compression in multi-thread process=
ing, so it is fine to use and speed up it.

Signed-off-by: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>
---
 meta/classes-recipe/image_types.bbclass | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recip=
e/image_types.bbclass
index 764e6a5574..f70c71b9d4 100644
--- a/meta/classes-recipe/image_types.bbclass
+++ b/meta/classes-recipe/image_types.bbclass
@@ -59,6 +59,10 @@ XZ_INTEGRITY_CHECK ?=3D "crc32"
=20
 ZIP_COMPRESSION_LEVEL ?=3D "-9"
=20
+7ZIP_COMPRESSION_LEVEL ?=3D "9"
+7ZIP_COMPRESSION_METHOD ?=3D "BZip2"
+7ZIP_EXTENSION ?=3D "zip"
+
 ZSTD_COMPRESSION_LEVEL ?=3D "-3"
=20
 JFFS2_SUM_EXTRA_ARGS ?=3D ""
@@ -304,7 +308,7 @@ IMAGE_TYPES:append:x86-64 =3D " hddimg iso"
 # CONVERSION_CMD/DEPENDS.
 COMPRESSIONTYPES ?=3D ""
=20
-CONVERSIONTYPES =3D "gz bz2 lzma xz lz4 lzo zip zst sum md5sum sha1sum s=
ha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow=
2 base64 gzsync zsync ${COMPRESSIONTYPES}"
+CONVERSIONTYPES =3D "gz bz2 lzma xz lz4 lzo zip 7zip zst sum md5sum sha1=
sum sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi=
 qcow2 base64 gzsync zsync ${COMPRESSIONTYPES}"
 CONVERSION_CMD:lzma =3D "lzma -k -f -7 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}=
.${type}"
 CONVERSION_CMD:gz =3D "gzip -f -9 -n -c --rsyncable ${IMAGE_NAME}${IMAGE=
_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.gz"
 CONVERSION_CMD:bz2 =3D "pbzip2 -f -k ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$=
{type}"
@@ -312,6 +316,7 @@ CONVERSION_CMD:xz =3D "xz -f -k -c ${XZ_COMPRESSION_L=
EVEL} ${XZ_DEFAULTS} --check=3D
 CONVERSION_CMD:lz4 =3D "lz4 -9 -z -l ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$=
{type} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.lz4"
 CONVERSION_CMD:lzo =3D "lzop -9 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type=
}"
 CONVERSION_CMD:zip =3D "zip ${ZIP_COMPRESSION_LEVEL} ${IMAGE_NAME}${IMAG=
E_NAME_SUFFIX}.${type}.zip ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
+CONVERSION_CMD:7zip =3D "7za a -mx=3D${7ZIP_COMPRESSION_LEVEL} -mm=3D${7=
ZIP_COMPRESSION_METHOD} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.${7ZIP_=
EXTENSION} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
 CONVERSION_CMD:zst =3D "zstd -f -k -T0 -c ${ZSTD_COMPRESSION_LEVEL} ${IM=
AGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}=
.${type}.zst"
 CONVERSION_CMD:sum =3D "sumtool -i ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${t=
ype} -o ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.sum ${JFFS2_SUM_EXTRA_A=
RGS}"
 CONVERSION_CMD:md5sum =3D "md5sum ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${ty=
pe} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.md5sum"
@@ -337,6 +342,7 @@ CONVERSION_DEPENDS_xz =3D "xz-native"
 CONVERSION_DEPENDS_lz4 =3D "lz4-native"
 CONVERSION_DEPENDS_lzo =3D "lzop-native"
 CONVERSION_DEPENDS_zip =3D "zip-native"
+CONVERSION_DEPENDS_7zip =3D "p7zip-native"
 CONVERSION_DEPENDS_zst =3D "zstd-native"
 CONVERSION_DEPENDS_sum =3D "mtd-utils-native"
 CONVERSION_DEPENDS_bmap =3D "bmap-tools-native"
--=20
2.38.1.windows.1

[master][PATCH v2 2/2] image_types: Set .7z extension for 7-Zip conversion command

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=666; bh=okipRjAsLt/gRy2A6q/lcxg6ztIVro0gzrU1Q3UX7lM=;

	b=OdOcPP+8x0PqdYPB0jCiuEiAdOyGkAraGNaGHbERYjGY38mUGQPcqIC8TmBv4yt6

	LW30dqJZZQgC0aVBO9EGpGv1cGwTK+G9TyvCKDqOI0mpAzch1+tIzEWtcb/LnnG11Rd

	LwXLHtgxm2sO+sRoy6StBGR8sEb+NxUsgzuBG0PHVXfgtdZ7IbQe5OM53nuHMnWEltd

	lrkZ5MoQfaHF1vx3L1QdbuNSWdo0xFzMvjlNxk4ONBmMLgLMg2sqIrtmNHSQhHhP3Lh

	Qh9+W4mSb9ai2ZIiK1pYdKU8tBOWyVKcT9Oe2whspZdqNPLAyWD2Jk1EHO/jhpulG4a

	MyHaAzuaew==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

Signed-off-by: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>
---
 meta/classes-recipe/image_types.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recip=
e/image_types.bbclass
index f70c71b9d4..ae00728a74 100644
--- a/meta/classes-recipe/image_types.bbclass
+++ b/meta/classes-recipe/image_types.bbclass
@@ -61,7 +61,7 @@ ZIP_COMPRESSION_LEVEL ?=3D "-9"
=20
 7ZIP_COMPRESSION_LEVEL ?=3D "9"
 7ZIP_COMPRESSION_METHOD ?=3D "BZip2"
-7ZIP_EXTENSION ?=3D "zip"
+7ZIP_EXTENSION ?=3D "7z"
=20
 ZSTD_COMPRESSION_LEVEL ?=3D "-3"
=20
--=20
2.38.1.windows.1

Re: [OE-core] [master][PATCH v2 2/2] image_types: Set .7z extension for 7-Zip conversion command

[Quentin Schulz]

Hi Benjamin,

On 11/7/22 13:37, Livius wrote:
> From: Benjamin Szőke <egyszeregy@freemail.hu>
> 
> Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
> ---
>   meta/classes-recipe/image_types.bbclass | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass
> index f70c71b9d4..ae00728a74 100644
> --- a/meta/classes-recipe/image_types.bbclass
> +++ b/meta/classes-recipe/image_types.bbclass
> @@ -61,7 +61,7 @@ ZIP_COMPRESSION_LEVEL ?= "-9"
>   
>   7ZIP_COMPRESSION_LEVEL ?= "9"
>   7ZIP_COMPRESSION_METHOD ?= "BZip2"
> -7ZIP_EXTENSION ?= "zip"
> +7ZIP_EXTENSION ?= "7z"
>   

We're almost there :)

There's no need for a second commit since the issue is introduced in 
your first commit. Instead, you should squash them together so the issue 
does not exist in the first place.

You can do so by doing:
git reset --soft HEAD^
git commit --amend --no-edit

This can also be done via git rebase:
git rebase --interactive HEAD^^

then modify "pick" for "squash" for the last commit, save and exit. 
You'll be prompted for commit log modification while rebasing, save and 
exit after your changes have been made (if any necessary).

You should only have one commit left, the first one, with the 
7ZIP_EXTENSION fix applied.

Don't forget to send as a v3.

Cheers,
Quentin

Re: [master][PATCH v2 2/2] image_types: Set .7z extension for 7-Zip conversion command

[Livius]

[-- Attachment #1: Type: text/plain, Size: 79 bytes --]

It is done: https://lists.openembedded.org/g/openembedded-core/message/172841

[-- Attachment #2: Type: text/html, Size: 195 bytes --]

Re: [OE-core] [master][PATCH v2 1/2] image_types: Add 7-Zip support in conversion types and commands

[Alexandre Belloni]

Hello,

On 07/11/2022 13:37:32+0100, Livius wrote:
> From: Benjamin Szőke <egyszeregy@freemail.hu>
> 
> Add 7-Zip support in conversion types and commands. Default arguments of 7-Zip are compression level: 9, method: BZip2, extension: zip. All these are overridable from local.con for any other use case in user side. First goal was to make an alternative zip conversion which can operate faster in multi-thread systems (on many CPU core), normal zip is just a single thread compression. 7-Zip can do a zip compression in multi-thread processing, so it is fine to use and speed up it.
> 

Could you also wrap the commit message?

Thanks!

> Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
> ---
>  meta/classes-recipe/image_types.bbclass | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass
> index 764e6a5574..f70c71b9d4 100644
> --- a/meta/classes-recipe/image_types.bbclass
> +++ b/meta/classes-recipe/image_types.bbclass
> @@ -59,6 +59,10 @@ XZ_INTEGRITY_CHECK ?= "crc32"
>  
>  ZIP_COMPRESSION_LEVEL ?= "-9"
>  
> +7ZIP_COMPRESSION_LEVEL ?= "9"
> +7ZIP_COMPRESSION_METHOD ?= "BZip2"
> +7ZIP_EXTENSION ?= "zip"
> +
>  ZSTD_COMPRESSION_LEVEL ?= "-3"
>  
>  JFFS2_SUM_EXTRA_ARGS ?= ""
> @@ -304,7 +308,7 @@ IMAGE_TYPES:append:x86-64 = " hddimg iso"
>  # CONVERSION_CMD/DEPENDS.
>  COMPRESSIONTYPES ?= ""
>  
> -CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip zst sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 base64 gzsync zsync ${COMPRESSIONTYPES}"
> +CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip 7zip zst sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 base64 gzsync zsync ${COMPRESSIONTYPES}"
>  CONVERSION_CMD:lzma = "lzma -k -f -7 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
>  CONVERSION_CMD:gz = "gzip -f -9 -n -c --rsyncable ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.gz"
>  CONVERSION_CMD:bz2 = "pbzip2 -f -k ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
> @@ -312,6 +316,7 @@ CONVERSION_CMD:xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_DEFAULTS} --check=
>  CONVERSION_CMD:lz4 = "lz4 -9 -z -l ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.lz4"
>  CONVERSION_CMD:lzo = "lzop -9 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
>  CONVERSION_CMD:zip = "zip ${ZIP_COMPRESSION_LEVEL} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.zip ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
> +CONVERSION_CMD:7zip = "7za a -mx=${7ZIP_COMPRESSION_LEVEL} -mm=${7ZIP_COMPRESSION_METHOD} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.${7ZIP_EXTENSION} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
>  CONVERSION_CMD:zst = "zstd -f -k -T0 -c ${ZSTD_COMPRESSION_LEVEL} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.zst"
>  CONVERSION_CMD:sum = "sumtool -i ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} -o ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
>  CONVERSION_CMD:md5sum = "md5sum ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.md5sum"
> @@ -337,6 +342,7 @@ CONVERSION_DEPENDS_xz = "xz-native"
>  CONVERSION_DEPENDS_lz4 = "lz4-native"
>  CONVERSION_DEPENDS_lzo = "lzop-native"
>  CONVERSION_DEPENDS_zip = "zip-native"
> +CONVERSION_DEPENDS_7zip = "p7zip-native"
>  CONVERSION_DEPENDS_zst = "zstd-native"
>  CONVERSION_DEPENDS_sum = "mtd-utils-native"
>  CONVERSION_DEPENDS_bmap = "bmap-tools-native"
> -- 
> 2.38.1.windows.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#172837): https://lists.openembedded.org/g/openembedded-core/message/172837
> Mute This Topic: https://lists.openembedded.org/mt/94864714/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com