* [PATCH] Add pigz as gzip-native alternative.
@ 2012-03-13 14:52 Björn Stenberg
2012-03-13 15:05 ` Paul Eggleton
2012-03-19 16:32 ` [PATCH] Add pigz as gzip-native alternative Saul Wold
0 siblings, 2 replies; 15+ messages in thread
From: Björn Stenberg @ 2012-03-13 14:52 UTC (permalink / raw)
To: openembedded-core
pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.
Upstream-status: Pending
Signed-off-by: Björn Stenberg <bjst@enea.com>
---
meta/recipes-extended/pigz/files/ldflags.patch | 11 +++++++++++
meta/recipes-extended/pigz/pigz.inc | 21 +++++++++++++++++++++
meta/recipes-extended/pigz/pigz_2.2.4.bb | 9 +++++++++
3 files changed, 41 insertions(+), 0 deletions(-)
create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
create mode 100644 meta/recipes-extended/pigz/pigz.inc
create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
diff --git a/meta/recipes-extended/pigz/files/ldflags.patch b/meta/recipes-extended/pigz/files/ldflags.patch
new file mode 100644
index 0000000..534d280
--- /dev/null
+++ b/meta/recipes-extended/pigz/files/ldflags.patch
@@ -0,0 +1,11 @@
+--- pigz-2.2.4/Makefile.orig 2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.4/Makefile 2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+
+ pigz: pigz.o yarn.o
+- $(CC) -o pigz pigz.o yarn.o -lpthread -lz
++ $(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ ln -f pigz unpigz
+
+ pigz.o: pigz.c yarn.h
diff --git a/meta/recipes-extended/pigz/pigz.inc b/meta/recipes-extended/pigz/pigz.inc
new file mode 100644
index 0000000..b501d7a
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz.inc
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+ file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+ install -d ${D}${bindir}
+ install ${B}/pigz ${D}${bindir}/gzip
+ install ${B}/unpigz ${D}${bindir}/gunzip
+}
diff --git a/meta/recipes-extended/pigz/pigz_2.2.4.bb b/meta/recipes-extended/pigz/pigz_2.2.4.bb
new file mode 100644
index 0000000..598bbe3
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz_2.2.4.bb
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
--
1.7.5.4
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-13 14:52 [PATCH] Add pigz as gzip-native alternative Björn Stenberg
@ 2012-03-13 15:05 ` Paul Eggleton
2012-03-14 15:59 ` Björn Stenberg
2012-03-19 16:32 ` [PATCH] Add pigz as gzip-native alternative Saul Wold
1 sibling, 1 reply; 15+ messages in thread
From: Paul Eggleton @ 2012-03-13 15:05 UTC (permalink / raw)
To: openembedded-core
Hi Björn,
On Tuesday 13 March 2012 15:52:51 Björn Stenberg wrote:
> pigz, which stands for parallel implementation of gzip, is a fully
> functional replacement for gzip that exploits multiple processors and
> multiple cores to the hilt when compressing data. pigz was written by Mark
> Adler, and uses the zlib and pthread libraries.
>
> This recipe adds pigz as an alternative provider of gzip-native. Pigz is
> much faster than gzip on multi-core machines.
>
> Upstream-status: Pending
This needs to be moved to the inner ldflags.patch, and..
> Signed-off-by: Björn Stenberg <bjst@enea.com>
this should be copied to there. The patch should also start with an
explanation of what the patch does (in this case it's trivial, but still worth
having in there.)
Thanks,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH] Add pigz as gzip-native alternative.
2012-03-13 15:05 ` Paul Eggleton
@ 2012-03-14 15:59 ` Björn Stenberg
2012-03-14 16:27 ` Paul Eggleton
0 siblings, 1 reply; 15+ messages in thread
From: Björn Stenberg @ 2012-03-14 15:59 UTC (permalink / raw)
To: openembedded-core
pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
This recipe adds pigz as an alternative gzip-native implementation only.
Signed-off-by: Björn Stenberg <bjst@enea.com>
---
meta/recipes-extended/pigz/files/ldflags.patch | 17 +++++++++++++++++
meta/recipes-extended/pigz/pigz.inc | 21 +++++++++++++++++++++
meta/recipes-extended/pigz/pigz_2.2.4.bb | 9 +++++++++
3 files changed, 47 insertions(+), 0 deletions(-)
create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
create mode 100644 meta/recipes-extended/pigz/pigz.inc
create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
diff --git a/meta/recipes-extended/pigz/files/ldflags.patch b/meta/recipes-extended/pigz/files/ldflags.patch
new file mode 100644
index 0000000..928c3a5
--- /dev/null
+++ b/meta/recipes-extended/pigz/files/ldflags.patch
@@ -0,0 +1,17 @@
+Link using the bitbake LDFLAGS.
+
+Signed-off-by: Björn Stenberg <bjst@enea.com>
+
+Upstream-Status: Pending
+
+--- pigz-2.2.4/Makefile.orig 2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.4/Makefile 2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+
+ pigz: pigz.o yarn.o
+- $(CC) -o pigz pigz.o yarn.o -lpthread -lz
++ $(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ ln -f pigz unpigz
+
+ pigz.o: pigz.c yarn.h
diff --git a/meta/recipes-extended/pigz/pigz.inc b/meta/recipes-extended/pigz/pigz.inc
new file mode 100644
index 0000000..b501d7a
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz.inc
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+ file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+ install -d ${D}${bindir}
+ install ${B}/pigz ${D}${bindir}/gzip
+ install ${B}/unpigz ${D}${bindir}/gunzip
+}
diff --git a/meta/recipes-extended/pigz/pigz_2.2.4.bb b/meta/recipes-extended/pigz/pigz_2.2.4.bb
new file mode 100644
index 0000000..598bbe3
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz_2.2.4.bb
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
--
1.7.5.4
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-14 15:59 ` Björn Stenberg
@ 2012-03-14 16:27 ` Paul Eggleton
2012-03-15 8:25 ` Björn Stenberg
0 siblings, 1 reply; 15+ messages in thread
From: Paul Eggleton @ 2012-03-14 16:27 UTC (permalink / raw)
To: Björn Stenberg; +Cc: openembedded-core
On Wednesday 14 March 2012 16:59:31 Björn Stenberg wrote:
> pigz, which stands for parallel implementation of gzip, is a fully
> functional replacement for gzip that exploits multiple processors and
> multiple cores to the hilt when compressing data. pigz was written by Mark
> Adler, and uses the zlib and pthread libraries.
>
> This recipe adds pigz as an alternative gzip-native implementation only.
>...
> diff --git a/meta/recipes-extended/pigz/pigz.inc
> b/meta/recipes-extended/pigz/pigz.inc new file mode 100644
> index 0000000..b501d7a
> --- /dev/null
> +++ b/meta/recipes-extended/pigz/pigz.inc
> @@ -0,0 +1,21 @@
> +SUMMARY = "A parallel implementation of gzip"
> +DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a
> \ +fully functional replacement for gzip that exploits multiple processors
> and \ +multiple cores to the hilt when compressing data. pigz was written
> by Mark \ +Adler, and uses the zlib and pthread libraries."
> +HOMEPAGE = "http://zlib.net/pigz/"
> +SECTION = "console/utils"
> +LICENSE = "Zlib"
> +
> +SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
> + file://ldflags.patch"
> +
> +PROVIDES += "gzip-native"
Sorry for not picking up on this earlier, but I think this needs to be
PROVIDES_virtclass-native, otherwise the target recipe will also provide gzip-
native.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH] Add pigz as gzip-native alternative.
2012-03-14 16:27 ` Paul Eggleton
@ 2012-03-15 8:25 ` Björn Stenberg
2012-03-19 14:46 ` Koen Kooi
0 siblings, 1 reply; 15+ messages in thread
From: Björn Stenberg @ 2012-03-15 8:25 UTC (permalink / raw)
To: openembedded-core
pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
This recipe adds pigz as an alternative gzip-native implementation only.
Signed-off-by: Björn Stenberg <bjst@enea.com>
---
meta/recipes-extended/pigz/files/ldflags.patch | 17 +++++++++++++++++
meta/recipes-extended/pigz/pigz.inc | 21 +++++++++++++++++++++
meta/recipes-extended/pigz/pigz_2.2.4.bb | 9 +++++++++
3 files changed, 47 insertions(+), 0 deletions(-)
create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
create mode 100644 meta/recipes-extended/pigz/pigz.inc
create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
diff --git a/meta/recipes-extended/pigz/files/ldflags.patch b/meta/recipes-extended/pigz/files/ldflags.patch
new file mode 100644
index 0000000..928c3a5
--- /dev/null
+++ b/meta/recipes-extended/pigz/files/ldflags.patch
@@ -0,0 +1,17 @@
+Link using the bitbake LDFLAGS.
+
+Signed-off-by: Björn Stenberg <bjst@enea.com>
+
+Upstream-Status: Pending
+
+--- pigz-2.2.4/Makefile.orig 2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.4/Makefile 2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+
+ pigz: pigz.o yarn.o
+- $(CC) -o pigz pigz.o yarn.o -lpthread -lz
++ $(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ ln -f pigz unpigz
+
+ pigz.o: pigz.c yarn.h
diff --git a/meta/recipes-extended/pigz/pigz.inc b/meta/recipes-extended/pigz/pigz.inc
new file mode 100644
index 0000000..acf1b71
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz.inc
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+ file://ldflags.patch"
+
+PROVIDES_virtclass-native += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+ install -d ${D}${bindir}
+ install ${B}/pigz ${D}${bindir}/gzip
+ install ${B}/unpigz ${D}${bindir}/gunzip
+}
diff --git a/meta/recipes-extended/pigz/pigz_2.2.4.bb b/meta/recipes-extended/pigz/pigz_2.2.4.bb
new file mode 100644
index 0000000..598bbe3
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz_2.2.4.bb
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
--
1.7.5.4
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-15 8:25 ` Björn Stenberg
@ 2012-03-19 14:46 ` Koen Kooi
2012-03-19 15:58 ` Björn Stenberg
0 siblings, 1 reply; 15+ messages in thread
From: Koen Kooi @ 2012-03-19 14:46 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Op 15 mrt. 2012, om 09:25 heeft Björn Stenberg het volgende geschreven:
>
> +++ b/meta/recipes-extended/pigz/pigz.inc
> @@ -0,0 +1,21 @@
> +SUMMARY = "A parallel implementation of gzip"
> +DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
> +fully functional replacement for gzip that exploits multiple processors and \
> +multiple cores to the hilt when compressing data. pigz was written by Mark \
> +Adler, and uses the zlib and pthread libraries."
> +HOMEPAGE = "http://zlib.net/pigz/"
> +SECTION = "console/utils"
> +LICENSE = "Zlib"
> +
> +SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
> + file://ldflags.patch"
> +
> +PROVIDES_virtclass-native += "gzip-native"
> +
> +DEPENDS_virtclass-native = "zlib-native"
> +
> +do_install () {
> + install -d ${D}${bindir}
> + install ${B}/pigz ${D}${bindir}/gzip
> + install ${B}/unpigz ${D}${bindir}/gunzip
> +}
This is either missing update-alternatives entries or it is missing RREPLACES/RCONFLICTS with gzip/busybox/etc. Or both.
regards,
Koen
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-19 14:46 ` Koen Kooi
@ 2012-03-19 15:58 ` Björn Stenberg
2012-03-19 17:56 ` Koen Kooi
0 siblings, 1 reply; 15+ messages in thread
From: Björn Stenberg @ 2012-03-19 15:58 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Koen Kooi wrote:
> Op 15 mrt. 2012, om 09:25 heeft Björn Stenberg het volgende geschreven:
> > +PROVIDES_virtclass-native += "gzip-native"
>
> This is either missing update-alternatives entries or it is missing
> RREPLACES/RCONFLICTS with gzip/busybox/etc. Or both.
My intention with the patch was to provide a native-only recipe for pigz, to improve distro build speed.
I'm not sure there is much interest in using pigz on target.
--
Björn
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-19 15:58 ` Björn Stenberg
@ 2012-03-19 17:56 ` Koen Kooi
2012-03-20 14:12 ` Björn Stenberg
0 siblings, 1 reply; 15+ messages in thread
From: Koen Kooi @ 2012-03-19 17:56 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Op 19 mrt. 2012, om 16:58 heeft Björn Stenberg het volgende geschreven:
> Koen Kooi wrote:
>> Op 15 mrt. 2012, om 09:25 heeft Björn Stenberg het volgende geschreven:
>>> +PROVIDES_virtclass-native += "gzip-native"
>>
>> This is either missing update-alternatives entries or it is missing
>> RREPLACES/RCONFLICTS with gzip/busybox/etc. Or both.
>
> My intention with the patch was to provide a native-only recipe for pigz, to improve distro build speed.
But it went in as a regular recipe and hence needs to play well with others.
>
> I'm not sure there is much interest in using pigz on target.
>
> --
> Björn
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-19 17:56 ` Koen Kooi
@ 2012-03-20 14:12 ` Björn Stenberg
2012-03-20 14:16 ` Koen Kooi
0 siblings, 1 reply; 15+ messages in thread
From: Björn Stenberg @ 2012-03-20 14:12 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Koen Kooi wrote:
> > My intention with the patch was to provide a native-only recipe for pigz,
> > to improve distro build speed.
>
> But it went in as a regular recipe and hence needs to play well with others.
I thought I made it native-only by only providing gzip-native?
--
Björn
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-20 14:12 ` Björn Stenberg
@ 2012-03-20 14:16 ` Koen Kooi
2012-03-21 14:29 ` [PATCH] pigz: Add update-altneratives support since this otherwise conflicts with busybox/gzip Richard Purdie
0 siblings, 1 reply; 15+ messages in thread
From: Koen Kooi @ 2012-03-20 14:16 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Op 20 mrt. 2012, om 15:12 heeft Björn Stenberg het volgende geschreven:
> Koen Kooi wrote:
>>> My intention with the patch was to provide a native-only recipe for pigz,
>>> to improve distro build speed.
>>
>> But it went in as a regular recipe and hence needs to play well with others.
>
> I thought I made it native-only by only providing gzip-native?
try 'bitbake pigz' :)
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH] pigz: Add update-altneratives support since this otherwise conflicts with busybox/gzip
2012-03-20 14:16 ` Koen Kooi
@ 2012-03-21 14:29 ` Richard Purdie
0 siblings, 0 replies; 15+ messages in thread
From: Richard Purdie @ 2012-03-21 14:29 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
diff --git a/meta/recipes-extended/pigz/pigz.inc b/meta/recipes-extended/pigz/pigz.inc
index 5fda8ca..e1cbc8b 100644
--- a/meta/recipes-extended/pigz/pigz.inc
+++ b/meta/recipes-extended/pigz/pigz.inc
@@ -15,7 +15,18 @@ PROVIDES_virtclass-native += "gzip-native"
DEPENDS = "zlib"
do_install () {
- install -d ${D}${bindir}
- install ${B}/pigz ${D}${bindir}/gzip
- install ${B}/unpigz ${D}${bindir}/gunzip
+ if [ "${PN}" = "${BPN}" ] ; then
+ # Install files into /bin (FHS), which is typical place for gzip
+ install -d ${D}${base_bindir}
+ install ${B}/pigz ${D}${base_bindir}/gzip
+ install ${B}/unpigz ${D}${base_bindir}/gunzip
+ else
+ install -d ${D}${bindir}
+ install ${B}/pigz ${D}${bindir}/gzip
+ install ${B}/unpigz ${D}${bindir}/gunzip
+ fi
}
+
+ALTERNATIVE_LINKS = "${base_bindir}/gzip ${base_bindir}/gunzip"
+ALTERNATIVE_PRIORITY = "80"
+
diff --git a/meta/recipes-extended/pigz/pigz_2.2.4.bb b/meta/recipes-extended/pigz/pigz_2.2.4.bb
index eaf00c6..e76c49c 100644
--- a/meta/recipes-extended/pigz/pigz_2.2.4.bb
+++ b/meta/recipes-extended/pigz/pigz_2.2.4.bb
@@ -1,6 +1,6 @@
require pigz.inc
-PR = "r1"
+PR = "r2"
NATIVE_PACKAGE_PATH_SUFFIX = "/${PN}"
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-13 14:52 [PATCH] Add pigz as gzip-native alternative Björn Stenberg
2012-03-13 15:05 ` Paul Eggleton
@ 2012-03-19 16:32 ` Saul Wold
1 sibling, 0 replies; 15+ messages in thread
From: Saul Wold @ 2012-03-19 16:32 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
On 03/13/2012 07:52 AM, Björn Stenberg wrote:
> pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
>
> This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.
>
> Upstream-status: Pending
> Signed-off-by: Björn Stenberg<bjst@enea.com>
> ---
> meta/recipes-extended/pigz/files/ldflags.patch | 11 +++++++++++
> meta/recipes-extended/pigz/pigz.inc | 21 +++++++++++++++++++++
> meta/recipes-extended/pigz/pigz_2.2.4.bb | 9 +++++++++
> 3 files changed, 41 insertions(+), 0 deletions(-)
> create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
> create mode 100644 meta/recipes-extended/pigz/pigz.inc
> create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
>
> diff --git a/meta/recipes-extended/pigz/files/ldflags.patch b/meta/recipes-extended/pigz/files/ldflags.patch
> new file mode 100644
> index 0000000..534d280
> --- /dev/null
> +++ b/meta/recipes-extended/pigz/files/ldflags.patch
> @@ -0,0 +1,11 @@
> +--- pigz-2.2.4/Makefile.orig 2012-01-01 02:46:58.000000000 +0100
> ++++ pigz-2.2.4/Makefile 2012-03-09 08:47:40.025094863 +0100
> +@@ -2,7 +2,7 @@
> + CFLAGS=-O3 -Wall -Wextra
> +
> + pigz: pigz.o yarn.o
> +- $(CC) -o pigz pigz.o yarn.o -lpthread -lz
> ++ $(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
> + ln -f pigz unpigz
> +
> + pigz.o: pigz.c yarn.h
> diff --git a/meta/recipes-extended/pigz/pigz.inc b/meta/recipes-extended/pigz/pigz.inc
> new file mode 100644
> index 0000000..b501d7a
> --- /dev/null
> +++ b/meta/recipes-extended/pigz/pigz.inc
> @@ -0,0 +1,21 @@
> +SUMMARY = "A parallel implementation of gzip"
> +DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
> +fully functional replacement for gzip that exploits multiple processors and \
> +multiple cores to the hilt when compressing data. pigz was written by Mark \
> +Adler, and uses the zlib and pthread libraries."
> +HOMEPAGE = "http://zlib.net/pigz/"
> +SECTION = "console/utils"
> +LICENSE = "Zlib"
> +
> +SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
> + file://ldflags.patch"
> +
> +PROVIDES += "gzip-native"
> +
> +DEPENDS_virtclass-native = "zlib-native"
> +
> +do_install () {
> + install -d ${D}${bindir}
> + install ${B}/pigz ${D}${bindir}/gzip
> + install ${B}/unpigz ${D}${bindir}/gunzip
> +}
> diff --git a/meta/recipes-extended/pigz/pigz_2.2.4.bb b/meta/recipes-extended/pigz/pigz_2.2.4.bb
> new file mode 100644
> index 0000000..598bbe3
> --- /dev/null
> +++ b/meta/recipes-extended/pigz/pigz_2.2.4.bb
> @@ -0,0 +1,9 @@
> +require pigz.inc
> +
> +PR = "r0"
> +
> +BBCLASSEXTEND = "native"
> +
> +SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
> +SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
> +LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
Merged and enabled in OE-Core
Thanks
Sau!
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH] Add pigz as gzip-native alternative.
@ 2012-03-12 15:47 Björn Stenberg
2012-03-12 16:11 ` McClintock Matthew-B29882
0 siblings, 1 reply; 15+ messages in thread
From: Björn Stenberg @ 2012-03-12 15:47 UTC (permalink / raw)
To: poky
pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.
Signed-off-by: Björn Stenberg <bjst@enea.com>
---
meta/recipes-extended/pigz/files/ldflags.patch | 11 +++++++++++
meta/recipes-extended/pigz/pigz.inc | 21 +++++++++++++++++++++
meta/recipes-extended/pigz/pigz_2.2.4.bb | 9 +++++++++
3 files changed, 41 insertions(+), 0 deletions(-)
create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
create mode 100644 meta/recipes-extended/pigz/pigz.inc
create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
diff --git a/meta/recipes-extended/pigz/files/ldflags.patch b/meta/recipes-extended/pigz/files/ldflags.patch
new file mode 100644
index 0000000..534d280
--- /dev/null
+++ b/meta/recipes-extended/pigz/files/ldflags.patch
@@ -0,0 +1,11 @@
+--- pigz-2.2.3/Makefile.orig 2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.3/Makefile 2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+
+ pigz: pigz.o yarn.o
+- $(CC) -o pigz pigz.o yarn.o -lpthread -lz
++ $(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ ln -f pigz unpigz
+
+ pigz.o: pigz.c yarn.h
diff --git a/meta/recipes-extended/pigz/pigz.inc b/meta/recipes-extended/pigz/pigz.inc
new file mode 100644
index 0000000..b501d7a
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz.inc
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+ file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+ install -d ${D}${bindir}
+ install ${B}/pigz ${D}${bindir}/gzip
+ install ${B}/unpigz ${D}${bindir}/gunzip
+}
diff --git a/meta/recipes-extended/pigz/pigz_2.2.4.bb b/meta/recipes-extended/pigz/pigz_2.2.4.bb
new file mode 100644
index 0000000..598bbe3
--- /dev/null
+++ b/meta/recipes-extended/pigz/pigz_2.2.4.bb
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
--
1.7.5.4
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-12 15:47 Björn Stenberg
@ 2012-03-12 16:11 ` McClintock Matthew-B29882
2012-03-13 8:59 ` Björn Stenberg
0 siblings, 1 reply; 15+ messages in thread
From: McClintock Matthew-B29882 @ 2012-03-12 16:11 UTC (permalink / raw)
To: Björn Stenberg; +Cc: poky@yoctoproject.org
I thought about doing this - can you share any performance numbers?
-M
On Mon, Mar 12, 2012 at 10:47 AM, Björn Stenberg <bjst@enea.com> wrote:
> pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
>
> This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.
>
> Signed-off-by: Björn Stenberg <bjst@enea.com>
> ---
> meta/recipes-extended/pigz/files/ldflags.patch | 11 +++++++++++
> meta/recipes-extended/pigz/pigz.inc | 21 +++++++++++++++++++++
> meta/recipes-extended/pigz/pigz_2.2.4.bb | 9 +++++++++
> 3 files changed, 41 insertions(+), 0 deletions(-)
> create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
> create mode 100644 meta/recipes-extended/pigz/pigz.inc
> create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
>
> diff --git a/meta/recipes-extended/pigz/files/ldflags.patch b/meta/recipes-extended/pigz/files/ldflags.patch
> new file mode 100644
> index 0000000..534d280
> --- /dev/null
> +++ b/meta/recipes-extended/pigz/files/ldflags.patch
> @@ -0,0 +1,11 @@
> +--- pigz-2.2.3/Makefile.orig 2012-01-01 02:46:58.000000000 +0100
> ++++ pigz-2.2.3/Makefile 2012-03-09 08:47:40.025094863 +0100
> +@@ -2,7 +2,7 @@
> + CFLAGS=-O3 -Wall -Wextra
> +
> + pigz: pigz.o yarn.o
> +- $(CC) -o pigz pigz.o yarn.o -lpthread -lz
> ++ $(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
> + ln -f pigz unpigz
> +
> + pigz.o: pigz.c yarn.h
> diff --git a/meta/recipes-extended/pigz/pigz.inc b/meta/recipes-extended/pigz/pigz.inc
> new file mode 100644
> index 0000000..b501d7a
> --- /dev/null
> +++ b/meta/recipes-extended/pigz/pigz.inc
> @@ -0,0 +1,21 @@
> +SUMMARY = "A parallel implementation of gzip"
> +DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
> +fully functional replacement for gzip that exploits multiple processors and \
> +multiple cores to the hilt when compressing data. pigz was written by Mark \
> +Adler, and uses the zlib and pthread libraries."
> +HOMEPAGE = "http://zlib.net/pigz/"
> +SECTION = "console/utils"
> +LICENSE = "Zlib"
> +
> +SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
> + file://ldflags.patch"
> +
> +PROVIDES += "gzip-native"
> +
> +DEPENDS_virtclass-native = "zlib-native"
> +
> +do_install () {
> + install -d ${D}${bindir}
> + install ${B}/pigz ${D}${bindir}/gzip
> + install ${B}/unpigz ${D}${bindir}/gunzip
> +}
> diff --git a/meta/recipes-extended/pigz/pigz_2.2.4.bb b/meta/recipes-extended/pigz/pigz_2.2.4.bb
> new file mode 100644
> index 0000000..598bbe3
> --- /dev/null
> +++ b/meta/recipes-extended/pigz/pigz_2.2.4.bb
> @@ -0,0 +1,9 @@
> +require pigz.inc
> +
> +PR = "r0"
> +
> +BBCLASSEXTEND = "native"
> +
> +SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
> +SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
> +LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
> --
> 1.7.5.4
>
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH] Add pigz as gzip-native alternative.
2012-03-12 16:11 ` McClintock Matthew-B29882
@ 2012-03-13 8:59 ` Björn Stenberg
0 siblings, 0 replies; 15+ messages in thread
From: Björn Stenberg @ 2012-03-13 8:59 UTC (permalink / raw)
To: McClintock Matthew-B29882; +Cc: poky@yoctoproject.org
McClintock Matthew-B29882 wrote:
> I thought about doing this - can you share any performance numbers?
pigz seems to scale fairly well. I compared the time it takes to compress the final rootfs tarball in do_rootfs for one of my images.
gzip:
real 1m33.523s
user 1m33.146s
sys 0m0.368s
pigz:
real 0m2.239s
user 1m40.554s
sys 0m0.716s
This is on a 64-core machine.
--
Björn
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2012-03-21 14:38 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-03-13 14:52 [PATCH] Add pigz as gzip-native alternative Björn Stenberg
2012-03-13 15:05 ` Paul Eggleton
2012-03-14 15:59 ` Björn Stenberg
2012-03-14 16:27 ` Paul Eggleton
2012-03-15 8:25 ` Björn Stenberg
2012-03-19 14:46 ` Koen Kooi
2012-03-19 15:58 ` Björn Stenberg
2012-03-19 17:56 ` Koen Kooi
2012-03-20 14:12 ` Björn Stenberg
2012-03-20 14:16 ` Koen Kooi
2012-03-21 14:29 ` [PATCH] pigz: Add update-altneratives support since this otherwise conflicts with busybox/gzip Richard Purdie
2012-03-19 16:32 ` [PATCH] Add pigz as gzip-native alternative Saul Wold
-- strict thread matches above, loose matches on Subject: below --
2012-03-12 15:47 Björn Stenberg
2012-03-12 16:11 ` McClintock Matthew-B29882
2012-03-13 8:59 ` Björn Stenberg
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.