Openembedded Core Discussions
 help / color / mirror / Atom feed
From: "André Draszik" <git@andred.net>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH v2] bitbake.conf: omit XZ threads and RAM from sstate signatures
Date: Mon, 24 Feb 2020 12:59:55 +0000	[thread overview]
Message-ID: <20200224125955.26771-1-git@andred.net> (raw)

The number of threads used, and the amount of memory allowed
to be used, should not affect sstate signatures, as they
don't affect the result.

Otherwise, it becomes impossible to re-use sstate from
automated builders on developer's machines (as the former
might execute bitbake with certain constraints different
compared to developer's machines).

This is in particular a problem with the opkg package writing
backend, as the OPKGBUILDCMD depends on XZ_DEFAULTS. Without
the vardepexclude, there is no re-use possible of the
package_write_ipk sstate.

The reason XZ_DEFAULTS as a whole isn't being whitelisted as
such here is that some arguments can affect the outcome of
the xz operation, so the approach taken is to only
whitelist the safe flags memlimit and # of threads.

Signed-off-by: André Draszik <git@andred.net>

---
v2: remove Gerrit Change-Id line
---
 meta/conf/bitbake.conf | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 954c06b313..d5a1a2471d 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -794,7 +794,10 @@ BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
 PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"
 
 # Default parallelism and resource usage for xz
-XZ_DEFAULTS ?= "--memlimit=50% --threads=${@oe.utils.cpu_count()}"
+XZ_MEMLIMIT ?= "50%"
+XZ_THREADS ?= "${@oe.utils.cpu_count()}"
+XZ_DEFAULTS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
+XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
 
 ##################################################################
 # Magic Cookie for SANITY CHECK
-- 
2.23.0.rc1



             reply	other threads:[~2020-02-24 12:59 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-24 12:59 André Draszik [this message]
2020-02-24 13:40 ` [PATCH v2] bitbake.conf: omit XZ threads and RAM from sstate signatures Adrian Bunk
2020-02-24 14:21   ` André Draszik
2020-02-24 14:31     ` Adrian Bunk
2020-02-24 14:58       ` André Draszik
2020-02-24 15:10         ` Adrian Bunk
2020-02-25 11:16           ` reproducible builds involving xz (was: Re: [PATCH v2] bitbake.conf: omit XZ threads and RAM from sstate signatures) André Draszik
2020-02-25 11:23             ` Richard Purdie
2020-02-24 16:44   ` [PATCH v2] bitbake.conf: omit XZ threads and RAM from sstate signatures Richard Purdie
2020-02-24 17:12     ` Adrian Bunk
2020-02-24 17:14     ` André Draszik
2020-02-24 17:32       ` Richard Purdie
2020-02-24 22:00         ` Adrian Bunk
2020-02-25  9:16           ` André Draszik
2020-02-25  9:54             ` Adrian Bunk
2020-02-26 15:26     ` xz threads / memlimit behaviour (was: Re: [PATCH v2] bitbake.conf: omit XZ threads and RAM from sstate signatures) André Draszik

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200224125955.26771-1-git@andred.net \
    --to=git@andred.net \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox