From: Adrian Bunk <bunk@stusta.de>
To: Mike Looijmans <mike.looijmans@topic.nl>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [PATCH] classes/populate_sdk_base: Implement xz compression options
Date: Thu, 19 Mar 2020 13:37:36 +0200 [thread overview]
Message-ID: <20200319113736.GA28081@localhost> (raw)
In-Reply-To: <20200318122156.23599-1-mike.looijmans@topic.nl>
On Wed, Mar 18, 2020 at 01:21:56PM +0100, Mike Looijmans wrote:
> Building an SDK on a machine with 8GB RAM resulted in excessive swapping
> due to the xz compressor using ~20GB of memory. This is because xz is
> being called with "-T 0 -9".
>
> To allow tuning the compression versus memory usage, introduce a variable
> named SDK_XZ_OPTIONS that defaults to a more sane default:
> SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${XZ_COMPRESSION_LEVEL}"
> Thus, inherit any XZ tuning already done, and allow users to specify
> overrides for this task in their config by supplying SDK_XZ_OPTIONS.
> Since XZ_COMPRESSION_LEVEL defaults to -9 this does not change the standard
> behavior.
>...
> +SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${XZ_COMPRESSION_LEVEL}"
A problem is that XZ_COMPRESSION_LEVEL compression would now be used for
unrelated usecases, and lowering the compression for being able to boot
on low-end targets (<= 64 MB RAM) shouldn't impact the SDK.
>...
> - d.setVar('SDK_ARCHIVE_CMD', 'cd ${SDK_OUTPUT}/${SDKPATH}; tar ${SDKTAROPTS} -cf - . | xz -T 0 -9 > ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}')
> + d.setVar('SDK_ARCHIVE_CMD', 'cd ${SDK_OUTPUT}/${SDKPATH}; tar ${SDKTAROPTS} -cf - . | xz ${SDK_XZ_OPTIONS} > ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}')
>...
Replacing "-T 0" with ${XZ_DEFAULTS} should solve your problem.
This already takes both cpus and memory into account.
cu
Adrian
next prev parent reply other threads:[~2020-03-19 11:37 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-18 12:21 [PATCH] classes/populate_sdk_base: Implement xz compression options Mike Looijmans
2020-03-19 11:37 ` Adrian Bunk [this message]
2020-03-20 14:48 ` Mike Looijmans
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200319113736.GA28081@localhost \
--to=bunk@stusta.de \
--cc=mike.looijmans@topic.nl \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox