From: Alexander Kanavin <alexander.kanavin@linux.intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 0/2] Multi-threaded RPM support
Date: Thu, 1 Jun 2017 18:15:57 +0300 [thread overview]
Message-ID: <cover.1496329361.git.alexander.kanavin@linux.intel.com> (raw)
These two patches add multi-threaded features to RPM to speed up the do_package_write_rpm task.
Specifically:
1) Multi-threaded xz compressor is used instead of default single-threaded gz. This has the most
dramatic effect when a recipe produces a smaller number of large-sized packages.
2) Packages creation is run in multiple threads via thread pools. This is most beneficial when
a recipe produces a large amount of small packages.
Some not very scientific benchmarks for time and .rpm sizes
(time is measured for do_package_write_rpm tasks only):
webkitgtk
before: 9m22s 1550M
after: 1m50s 858M
glibc-locale
before: 3m2s 125M
after: 40s 56M
glibc
before: 56s 54M
after: 23s 38M
perl
before: 1m19s 63M
after: 55s 42M
python3
before: 40s 38M
after: 28s 24M
The following changes since commit bd063fa288b49b6e3ea77982d0ccc46610feb1ad:
tune-mips32*.inc: use consistent comments across all three .inc files (2017-05-30 10:15:20 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib akanavin/parallel-rpm
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=akanavin/parallel-rpm
Alexander Kanavin (2):
package_rpm.bbclass: use multithreaded xz compression
rpm: run binary package generation via thread pools
meta/classes/package_rpm.bbclass | 2 +
...y-package-building-into-a-separate-functi.patch | 83 ++++++++
...-binary-package-creation-via-thread-pools.patch | 137 +++++++++++++
...c-make-operations-over-string-pools-threa.patch | 209 +++++++++++++++++++
...c-remove-static-local-variables-from-buil.patch | 227 +++++++++++++++++++++
meta/recipes-devtools/rpm/rpm_git.bb | 4 +
6 files changed, 662 insertions(+)
create mode 100644 meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
create mode 100644 meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
create mode 100644 meta/recipes-devtools/rpm/files/0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch
create mode 100644 meta/recipes-devtools/rpm/files/0004-build-pack.c-remove-static-local-variables-from-buil.patch
--
2.11.0
next reply other threads:[~2017-06-01 15:17 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-01 15:15 Alexander Kanavin [this message]
2017-06-01 15:15 ` [PATCH 1/2] package_rpm.bbclass: use multithreaded xz compression Alexander Kanavin
2017-06-01 17:11 ` Mark Hatle
2017-06-01 17:44 ` Alexander Kanavin
2017-06-01 19:02 ` Andre McCurdy
2017-06-02 11:40 ` Alexander Kanavin
2017-06-02 18:33 ` Andre McCurdy
2017-06-01 15:15 ` [PATCH 2/2] rpm: run binary package generation via thread pools Alexander Kanavin
2017-06-01 16:02 ` Alexander Kanavin
2017-06-02 12:44 ` Alexander Kanavin
2017-06-01 16:52 ` [PATCH 0/2] Multi-threaded RPM support Leonardo Sandoval
2017-06-01 17:13 ` Mark Hatle
2017-06-01 17:37 ` Alexander Kanavin
2017-06-02 21:41 ` Leonardo Sandoval
2017-06-02 22:20 ` Richard Purdie
2017-06-04 14:15 ` Richard Purdie
2017-06-05 12:14 ` Alexander Kanavin
2017-06-05 12:41 ` Alexander Kanavin
2017-06-05 13:01 ` Burton, Ross
2017-06-05 13:15 ` Alexander Kanavin
2017-06-05 13:21 ` Joshua Lock
2017-06-05 13:34 ` Alexander Kanavin
2017-06-05 13:47 ` Davis, Michael
2017-06-05 14:00 ` Alexander Kanavin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1496329361.git.alexander.kanavin@linux.intel.com \
--to=alexander.kanavin@linux.intel.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox