From: "Richard Purdie" <richard.purdie@linuxfoundation.org>
To: "Yu, Mingli" <mingli.yu@windriver.com>,
openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] bitbake.conf: set the default pigz thread
Date: Tue, 11 Aug 2020 08:57:14 +0100 [thread overview]
Message-ID: <2803933a96d0b791c8cad542bbfec3c3dd67eefd.camel@linuxfoundation.org> (raw)
In-Reply-To: <20200811072258.4074290-1-mingli.yu@windriver.com>
On Tue, 2020-08-11 at 15:22 +0800, Yu, Mingli wrote:
> From: Mingli Yu <mingli.yu@windriver.com>
>
> pigz, which stands for parallel implementation of gzip, is a
> fully functional replacement for gzip that exploits multiple
> processors and multiple cores to the hilt when compressing data.
>
> do_package uses pigz to compress data to speed up the time
> if there is pigz available on the build server.
>
> But for some big package such as clang, there comes below
> error during do_package.
> gzip: stdout: Cannot allocate memory
> tar: TOPDIR/sstate-cache/20/80/sstate:clang:cortexa57-wrs-linux:10.0.1:r0:cortexa57:3:20808f8a746b00ec9470a81057c8595d878cfe8c565cfbcbbaa3ea870528ee34_package.tgz.QF5dss8M: Wrote only 8192 of 10240 bytes
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
>
> So set the default pigz thread and the user also can customize
> the pigz thread such as via PIGZ = "-p 3" or PIGZ_pn-clang = "-p 3"
> for the individual package in local.conf to avoid eating so much
> memory.
>
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
> meta/conf/bitbake.conf | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 353caacef9..bc87e78d78 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -804,6 +804,9 @@ XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
> XZ_DEFAULTS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
> XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
>
> +# Default parallelism and resource usage for pigz
> +export PIGZ = "-p ${@oe.utils.cpu_count(at_least=2)}"
Why not just set this in the cases you need it in and leave the
defaults alone? You can just export it in the tasks which need it too.
This change actually makes the sstate number of cpus specific which we
definitely don't want.
Cheers,
Richard
next prev parent reply other threads:[~2020-08-11 7:57 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-11 7:22 [PATCH] bitbake.conf: set the default pigz thread Yu, Mingli
2020-08-11 7:57 ` Richard Purdie [this message]
2020-08-11 8:09 ` [OE-core] " Yu, Mingli
2020-08-11 8:19 ` Richard Purdie
2020-08-11 8:19 ` Yu, Mingli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2803933a96d0b791c8cad542bbfec3c3dd67eefd.camel@linuxfoundation.org \
--to=richard.purdie@linuxfoundation.org \
--cc=mingli.yu@windriver.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox