From: Alexander Kanavin <alexander.kanavin@linux.intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCHv2 0/2] Multi-threaded RPM support
Date: Thu, 8 Jun 2017 17:42:08 +0300 [thread overview]
Message-ID: <cover.1496932793.git.alexander.kanavin@linux.intel.com> (raw)
Changes in version 2: use openmp for multi-processing instead of NSPR library, as
requested by upstream. Add a link to the upstream submission to patches.
These two patches add multi-threaded features to RPM to speed up the do_package_write_rpm task.
Specifically:
1) Multi-threaded xz compressor is used instead of default single-threaded gz. This has the most
dramatic effect when a recipe produces a smaller number of large-sized packages.
2) Packages creation is run in multiple threads via thread pools. This is most beneficial when
a recipe produces a large amount of small packages.
Some not very scientific benchmarks for time and .rpm sizes
(time is measured for do_package_write_rpm tasks only):
webkitgtk
before: 9m12s 1550M
after: 1m40s 858M
glibc-locale
before: 2m52s 125M
after: 30s 56M
glibc
before: 46s 54M
after: 13s 38M
perl
before: 1m09s 63M
after: 45s 42M
python3
before: 30s 38M
after: 18s 24M
The following changes since commit 576821ea0a7558b626ccc87e9ae0e9ee40864956:
bitbake: bitbake-layers: check layer dependencies before adding (2017-06-06 19:52:51 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib akanavin/parallel-rpm
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=akanavin/parallel-rpm
Alexander Kanavin (2):
package_rpm.bbclass: use multithreaded xz compression
rpm: run binary package generation via thread pools
meta/classes/package_rpm.bbclass | 2 +
...y-package-building-into-a-separate-functi.patch | 83 ++++++++
...-binary-package-creation-via-thread-pools.patch | 125 ++++++++++++
...c-make-operations-over-string-pools-threa.patch | 207 ++++++++++++++++++++
...c-remove-static-local-variables-from-buil.patch | 216 +++++++++++++++++++++
meta/recipes-devtools/rpm/rpm_git.bb | 4 +
6 files changed, 637 insertions(+)
create mode 100644 meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
create mode 100644 meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
create mode 100644 meta/recipes-devtools/rpm/files/0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch
create mode 100644 meta/recipes-devtools/rpm/files/0004-build-pack.c-remove-static-local-variables-from-buil.patch
--
2.11.0
next reply other threads:[~2017-06-08 14:42 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-08 14:42 Alexander Kanavin [this message]
2017-06-08 14:42 ` [PATCHv2 1/2] package_rpm.bbclass: use multithreaded xz compression Alexander Kanavin
2017-06-08 14:42 ` [PATCHv2 2/2] rpm: run binary package generation via thread pools Alexander Kanavin
2017-06-08 16:55 ` Mark Hatle
2017-06-08 17:23 ` Alexander Kanavin
2017-06-08 20:29 ` Mark Hatle
2017-06-09 9:02 ` Alexander Kanavin
2017-06-09 11:18 ` Alexander Kanavin
2017-06-08 21:15 ` [PATCHv2 0/2] Multi-threaded RPM support Leonardo Sandoval
2017-06-09 9:13 ` Alexander Kanavin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1496932793.git.alexander.kanavin@linux.intel.com \
--to=alexander.kanavin@linux.intel.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.