[PATCH 0/4] Support symbolic links in paths in PSEUDO_IGNORE

public inbox for openembedded-core@lists.openembedded.org
 help / color / mirror / Atom feed

* [PATCH 0/4] Support symbolic links in paths in PSEUDO_IGNORE_PATHS
@ 2020-11-25 13:30 Peter Kjellerstedt
  2020-11-25 13:30 ` [PATCH 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot() Peter Kjellerstedt
                   ` (3 more replies)
  0 siblings, 4 replies; 7+ messages in thread
From: Peter Kjellerstedt @ 2020-11-25 13:30 UTC (permalink / raw)
  To: openembedded-core

The changes in this patch series aim to rectify a problem with pseudo
and its support for ignoring paths. When pseudo compares a path to the
paths specified in PSEUDO_IGNORE_PATHS, it uses a path that has been
canonicalized. However, it does not canonicalize the paths in
PSEUDO_IGNORE_PATHS itself and unless they are canonicalized, they
will not match as intended. Thus the paths in PSEUDO_IGNORE_PATHS
needs to be canonicalized. These patches do that by adding a new
PSEUDO_IGNORE_REAL_PATHS variable, which contains the paths from
PSEUDO_IGNORE_PATHS after having run them through os.path.realpath().

There is also one patch that adds two patches to pseudo to clean up
pseudo_client_ignore_path_chroot(), and they also plug a memory leak.
The patches were brought about as I initially intended to do the
canonicalization in pseudo itself in this function.

I have not tested the change to wic as we do not use it, though I do
not expect it to be problematic.

//Peter

The following changes since commit 3ecf5d9692fec97b37af6a4e6c747a4e2c2ca292:

  uninative: Don't use single sstate for pseudo-native (2020-11-24 15:53:07 +0000)

are available in the Git repository at:

  git://push.yoctoproject.org/poky-contrib pkj/pseudo-ignore

Peter Kjellerstedt (4):
  pseudo: Simplify pseudo_client_ignore_path_chroot()
  bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS
  bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS
  wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS

 meta/classes/image_types_wic.bbclass          |  2 +-
 meta/conf/bitbake.conf                        | 10 +--
 meta/conf/machine/include/tune-thunderx.inc   |  2 +-
 ...ssen-indentation-of-pseudo_client_ig.patch | 69 +++++++++++++++++++
 ...mplify-pseudo_client_ignore_path_chr.patch | 50 ++++++++++++++
 meta/recipes-devtools/pseudo/pseudo_git.bb    |  2 +
 scripts/contrib/bb-perf/bb-matrix-plot.sh     |  8 ++-
 scripts/lib/wic/partition.py                  |  2 +-
 8 files changed, 135 insertions(+), 10 deletions(-)
 create mode 100644 meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch
 create mode 100644 meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot()
  2020-11-25 13:30 [PATCH 0/4] Support symbolic links in paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
@ 2020-11-25 13:30 ` Peter Kjellerstedt
  2020-11-25 13:30 ` [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS Peter Kjellerstedt
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 7+ messages in thread
From: Peter Kjellerstedt @ 2020-11-25 13:30 UTC (permalink / raw)
  To: openembedded-core

This also plugs a memory leak in pseudo_client_ignore_path_chroot().

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 ...ssen-indentation-of-pseudo_client_ig.patch | 69 +++++++++++++++++++
 ...mplify-pseudo_client_ignore_path_chr.patch | 50 ++++++++++++++
 meta/recipes-devtools/pseudo/pseudo_git.bb    |  2 +
 3 files changed, 121 insertions(+)
 create mode 100644 meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch
 create mode 100644 meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch

diff --git a/meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch b/meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch
new file mode 100644
index 0000000000..e4a5356f5c
--- /dev/null
+++ b/meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch
@@ -0,0 +1,69 @@
+From 28c760542eecd7c5b35ea88aa2b14d62afbda961 Mon Sep 17 00:00:00 2001
+From: Peter Kjellerstedt <pkj@axis.com>
+Date: Sat, 21 Nov 2020 17:22:38 +0100
+Subject: [PATCH] pseudo_client: Lessen indentation of
+ pseudo_client_ignore_path_chroot()
+
+Change-Id: I739b18efb7a95ce2d2d907d0faf7df539ab9af1c
+---
+ pseudo_client.c | 45 +++++++++++++++++++++++++--------------------
+ 1 file changed, 25 insertions(+), 20 deletions(-)
+
+diff --git a/pseudo_client.c b/pseudo_client.c
+index 116d926..a8bc3dc 100644
+--- a/pseudo_client.c
++++ b/pseudo_client.c
+@@ -1527,28 +1527,33 @@ int pseudo_client_ignore_fd(int fd) {
+ 
+ int pseudo_client_ignore_path_chroot(const char *path, int ignore_chroot) {
+ 	char *env;
+-	if (path) {
+-		if (ignore_chroot && pseudo_chroot && strncmp(path, pseudo_chroot, pseudo_chroot_len) == 0)
+-			return 0;
+-		env = pseudo_get_value("PSEUDO_IGNORE_PATHS");
+-		if (env) {
+-			char *p = env;
+-        	        while (*p) {
+-				char *next = strchr(p, ',');
+-				if (!next)
+-				    next = strchr(p, '\0');
+-				if ((next - p) && !strncmp(path, p, next - p)) {
+-		 			pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n", path);
+-					return 1;
+-				}
+-				if (next && *next != '\0')
+-					p = next+1;
+-				else
+-					break;
+-			}
+-			free(env);
++
++	if (!path)
++		return 0;
++
++	if (ignore_chroot && pseudo_chroot && strncmp(path, pseudo_chroot, pseudo_chroot_len) == 0)
++		return 0;
++
++	env = pseudo_get_value("PSEUDO_IGNORE_PATHS");
++	if (!env)
++		return 0;
++
++	char *p = env;
++	while (*p) {
++		char *next = strchr(p, ',');
++		if (!next)
++			next = strchr(p, '\0');
++		if ((next - p) && !strncmp(path, p, next - p)) {
++ 			pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n", path);
++			return 1;
+ 		}
++		if (next && *next != '\0')
++			p = next+1;
++		else
++			break;
+ 	}
++	free(env);
++
+ 	return 0;
+ }
+ 
diff --git a/meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch b/meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch
new file mode 100644
index 0000000000..be3ab25bd3
--- /dev/null
+++ b/meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch
@@ -0,0 +1,50 @@
+From a1d61d68777373a50ae23b9dd83b428abe2f748d Mon Sep 17 00:00:00 2001
+From: Peter Kjellerstedt <pkj@axis.com>
+Date: Sat, 21 Nov 2020 17:30:33 +0100
+Subject: [PATCH] pseudo_client: Simplify pseudo_client_ignore_path_chroot()
+
+This also plugs a memory leak by making sure env is freed.
+
+Change-Id: Ia8635fd2c6b1e85919e4743713a85e0b52c28fac
+---
+ pseudo_client.c | 21 ++++++++++-----------
+ 1 file changed, 10 insertions(+), 11 deletions(-)
+
+diff --git a/pseudo_client.c b/pseudo_client.c
+index a8bc3dc..7dc0345 100644
+--- a/pseudo_client.c
++++ b/pseudo_client.c
+@@ -1538,23 +1538,22 @@ int pseudo_client_ignore_path_chroot(const char *path, int ignore_chroot) {
+ 	if (!env)
+ 		return 0;
+ 
++	int ret = 0;
+ 	char *p = env;
+-	while (*p) {
++	while (p) {
+ 		char *next = strchr(p, ',');
+-		if (!next)
+-			next = strchr(p, '\0');
+-		if ((next - p) && !strncmp(path, p, next - p)) {
+- 			pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n", path);
+-			return 1;
+-		}
+-		if (next && *next != '\0')
+-			p = next+1;
+-		else
++		if (next)
++			*next++ = '\0';
++		if (!strncmp(path, p, strlen(p))) {
++			pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n", path);
++			ret = 1;
+ 			break;
++		}
++		p = next;
+ 	}
+ 	free(env);
+ 
+-	return 0;
++	return ret;
+ }
+ 
+ int pseudo_client_ignore_path(const char *path) {
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 2e13fec540..50933c4bc1 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -4,6 +4,8 @@ SRC_URI = "git://git.yoctoproject.org/pseudo;branch=oe-core \
            file://0001-configure-Prune-PIE-flags.patch \
            file://fallback-passwd \
            file://fallback-group \
+           file://0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch \
+           file://0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch \
            "
 
 SRCREV = "cca0d7f15b7197095cd587420d31b187620c3093"

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS
  2020-11-25 13:30 [PATCH 0/4] Support symbolic links in paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
  2020-11-25 13:30 ` [PATCH 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot() Peter Kjellerstedt
@ 2020-11-25 13:30 ` Peter Kjellerstedt
  2020-11-25 13:32   ` [OE-core] " Martin Jansa
  2020-11-25 13:30 ` [PATCH 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
  2020-11-25 13:30 ` [PATCH 4/4] wic: Pass canonicalized " Peter Kjellerstedt
  3 siblings, 1 reply; 7+ messages in thread
From: Peter Kjellerstedt @ 2020-11-25 13:30 UTC (permalink / raw)
  To: openembedded-core

Instead of ignoring ${COREBASE}/meta in PSEUDO_IGNORE_PATHS (which may
or may not ignore all layers depending on how they are named and placed
under ${COREBASE}), ignore all layers.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/conf/bitbake.conf                      | 2 +-
 meta/conf/machine/include/tune-thunderx.inc | 2 +-
 scripts/contrib/bb-perf/bb-matrix-plot.sh   | 8 +++++---
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0d38eac094..9742fe4fe2 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -685,7 +685,7 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
-PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
+PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
 
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
diff --git a/meta/conf/machine/include/tune-thunderx.inc b/meta/conf/machine/include/tune-thunderx.inc
index d1aaf4891e..dadcadedc6 100644
--- a/meta/conf/machine/include/tune-thunderx.inc
+++ b/meta/conf/machine/include/tune-thunderx.inc
@@ -11,7 +11,7 @@ ARMPKGARCH_tune-thunderx ?= "thunderx"
 ARMPKGARCH_tune-thunderx_be ?= "thunderx"
 
 TUNE_FEATURES_tune-thunderx = "${TUNE_FEATURES_tune-aarch64} thunderx"
-TUNE_FEATURES_tune-thunderx_be = "${TUNE_FEATURES_tune-thunderx} bigendian"
+TUNE_FEATURES_tune-thunderx_be = "${TUNE_FEATURES_tune-aarch64_be} thunderx"
 BASE_LIB_tune-thunderx = "lib64"
 BASE_LIB_tune-thunderx_be = "lib64"
 
diff --git a/scripts/contrib/bb-perf/bb-matrix-plot.sh b/scripts/contrib/bb-perf/bb-matrix-plot.sh
index e7bd129e9e..450afcb559 100755
--- a/scripts/contrib/bb-perf/bb-matrix-plot.sh
+++ b/scripts/contrib/bb-perf/bb-matrix-plot.sh
@@ -90,10 +90,12 @@ fi
 # Determine the dgrid3d mesh dimensions size
 MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sed 's/^0*//' | sort -n | uniq | head -n1)
 MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sed 's/^0*//' | sort -n | uniq | tail -n1)
-BB_CNT=$[${MAX} - $MIN + 1]
+#BB_CNT=$[${MAX} - $MIN + 1]
+BB_CNT=$[(${MAX} - $MIN)/4 + 1]
 MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sed 's/^0*//' | sort -n | uniq | head -n1)
 MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sed 's/^0*//' | sort -n | uniq | tail -n1)
-PM_CNT=$[${MAX} - $MIN + 1]
+#PM_CNT=$[${MAX} - $MIN + 1]
+PM_CNT=$[(${MAX} - $MIN)/4 + 1]
 
 
 (cat <<EOF
@@ -118,7 +120,7 @@ set output "$PLOT_BASENAME-pm.png"
 replot
 
 set view 60,30
-set term wxt size $SIZE
+set term qt size $SIZE
 replot
 EOF
 ) | gnuplot --persist

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS
  2020-11-25 13:30 ` [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS Peter Kjellerstedt
@ 2020-11-25 13:32   ` Martin Jansa
  2020-11-25 13:47     ` Peter Kjellerstedt
  0 siblings, 1 reply; 7+ messages in thread
From: Martin Jansa @ 2020-11-25 13:32 UTC (permalink / raw)
  To: Peter Kjellerstedt; +Cc: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 3909 bytes --]

Looks like some unrelated changes sneaked into this one.

On Wed, Nov 25, 2020 at 2:31 PM Peter Kjellerstedt <
peter.kjellerstedt@axis.com> wrote:

> Instead of ignoring ${COREBASE}/meta in PSEUDO_IGNORE_PATHS (which may
> or may not ignore all layers depending on how they are named and placed
> under ${COREBASE}), ignore all layers.
>
> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> ---
>  meta/conf/bitbake.conf                      | 2 +-
>  meta/conf/machine/include/tune-thunderx.inc | 2 +-
>  scripts/contrib/bb-perf/bb-matrix-plot.sh   | 8 +++++---
>  3 files changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 0d38eac094..9742fe4fe2 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -685,7 +685,7 @@ SRC_URI = ""
>  PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
>  PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
>  PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
> -PSEUDO_IGNORE_PATHS =
> "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
> +PSEUDO_IGNORE_PATHS =
> "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
>
>  export PSEUDO_DISABLED = "1"
>  #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
> diff --git a/meta/conf/machine/include/tune-thunderx.inc
> b/meta/conf/machine/include/tune-thunderx.inc
> index d1aaf4891e..dadcadedc6 100644
> --- a/meta/conf/machine/include/tune-thunderx.inc
> +++ b/meta/conf/machine/include/tune-thunderx.inc
> @@ -11,7 +11,7 @@ ARMPKGARCH_tune-thunderx ?= "thunderx"
>  ARMPKGARCH_tune-thunderx_be ?= "thunderx"
>
>  TUNE_FEATURES_tune-thunderx = "${TUNE_FEATURES_tune-aarch64} thunderx"
> -TUNE_FEATURES_tune-thunderx_be = "${TUNE_FEATURES_tune-thunderx}
> bigendian"
> +TUNE_FEATURES_tune-thunderx_be = "${TUNE_FEATURES_tune-aarch64_be}
> thunderx"
>  BASE_LIB_tune-thunderx = "lib64"
>  BASE_LIB_tune-thunderx_be = "lib64"
>
> diff --git a/scripts/contrib/bb-perf/bb-matrix-plot.sh
> b/scripts/contrib/bb-perf/bb-matrix-plot.sh
> index e7bd129e9e..450afcb559 100755
> --- a/scripts/contrib/bb-perf/bb-matrix-plot.sh
> +++ b/scripts/contrib/bb-perf/bb-matrix-plot.sh
> @@ -90,10 +90,12 @@ fi
>  # Determine the dgrid3d mesh dimensions size
>  MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sed 's/^0*//' | sort -n |
> uniq | head -n1)
>  MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sed 's/^0*//' | sort -n |
> uniq | tail -n1)
> -BB_CNT=$[${MAX} - $MIN + 1]
> +#BB_CNT=$[${MAX} - $MIN + 1]
> +BB_CNT=$[(${MAX} - $MIN)/4 + 1]
>  MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sed 's/^0*//' | sort -n |
> uniq | head -n1)
>  MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sed 's/^0*//' | sort -n |
> uniq | tail -n1)
> -PM_CNT=$[${MAX} - $MIN + 1]
> +#PM_CNT=$[${MAX} - $MIN + 1]
> +PM_CNT=$[(${MAX} - $MIN)/4 + 1]
>
>
>  (cat <<EOF
> @@ -118,7 +120,7 @@ set output "$PLOT_BASENAME-pm.png"
>  replot
>
>  set view 60,30
> -set term wxt size $SIZE
> +set term qt size $SIZE
>  replot
>  EOF
>  ) | gnuplot --persist
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 4697 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS
  2020-11-25 13:32   ` [OE-core] " Martin Jansa
@ 2020-11-25 13:47     ` Peter Kjellerstedt
  0 siblings, 0 replies; 7+ messages in thread
From: Peter Kjellerstedt @ 2020-11-25 13:47 UTC (permalink / raw)
  To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 4199 bytes --]

But duh. Ok, patch v2 incoming…

//Peter

From: Martin Jansa <martin.jansa@gmail.com>
Sent: den 25 november 2020 14:32
To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Cc: Patches and discussions about the oe-core layer <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS

Looks like some unrelated changes sneaked into this one.

On Wed, Nov 25, 2020 at 2:31 PM Peter Kjellerstedt <peter.kjellerstedt@axis.com<mailto:peter.kjellerstedt@axis.com>> wrote:
Instead of ignoring ${COREBASE}/meta in PSEUDO_IGNORE_PATHS (which may
or may not ignore all layers depending on how they are named and placed
under ${COREBASE}), ignore all layers.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com<mailto:peter.kjellerstedt@axis.com>>
---
 meta/conf/bitbake.conf                      | 2 +-
 meta/conf/machine/include/tune-thunderx.inc | 2 +-
 scripts/contrib/bb-perf/bb-matrix-plot.sh   | 8 +++++---
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0d38eac094..9742fe4fe2 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -685,7 +685,7 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
-PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
+PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"

 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
diff --git a/meta/conf/machine/include/tune-thunderx.inc b/meta/conf/machine/include/tune-thunderx.inc
index d1aaf4891e..dadcadedc6 100644
--- a/meta/conf/machine/include/tune-thunderx.inc
+++ b/meta/conf/machine/include/tune-thunderx.inc
@@ -11,7 +11,7 @@ ARMPKGARCH_tune-thunderx ?= "thunderx"
 ARMPKGARCH_tune-thunderx_be ?= "thunderx"

 TUNE_FEATURES_tune-thunderx = "${TUNE_FEATURES_tune-aarch64} thunderx"
-TUNE_FEATURES_tune-thunderx_be = "${TUNE_FEATURES_tune-thunderx} bigendian"
+TUNE_FEATURES_tune-thunderx_be = "${TUNE_FEATURES_tune-aarch64_be} thunderx"
 BASE_LIB_tune-thunderx = "lib64"
 BASE_LIB_tune-thunderx_be = "lib64"

diff --git a/scripts/contrib/bb-perf/bb-matrix-plot.sh b/scripts/contrib/bb-perf/bb-matrix-plot.sh
index e7bd129e9e..450afcb559 100755
--- a/scripts/contrib/bb-perf/bb-matrix-plot.sh
+++ b/scripts/contrib/bb-perf/bb-matrix-plot.sh
@@ -90,10 +90,12 @@ fi
 # Determine the dgrid3d mesh dimensions size
 MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sed 's/^0*//' | sort -n | uniq | head -n1)
 MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sed 's/^0*//' | sort -n | uniq | tail -n1)
-BB_CNT=$[${MAX} - $MIN + 1]
+#BB_CNT=$[${MAX} - $MIN + 1]
+BB_CNT=$[(${MAX} - $MIN)/4 + 1]
 MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sed 's/^0*//' | sort -n | uniq | head -n1)
 MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sed 's/^0*//' | sort -n | uniq | tail -n1)
-PM_CNT=$[${MAX} - $MIN + 1]
+#PM_CNT=$[${MAX} - $MIN + 1]
+PM_CNT=$[(${MAX} - $MIN)/4 + 1]


 (cat <<EOF
@@ -118,7 +120,7 @@ set output "$PLOT_BASENAME-pm.png"
 replot

 set view 60,30
-set term wxt size $SIZE
+set term qt size $SIZE
 replot
 EOF
 ) | gnuplot --persist



[-- Attachment #2: Type: text/html, Size: 7849 bytes --]

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS
  2020-11-25 13:30 [PATCH 0/4] Support symbolic links in paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
  2020-11-25 13:30 ` [PATCH 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot() Peter Kjellerstedt
  2020-11-25 13:30 ` [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS Peter Kjellerstedt
@ 2020-11-25 13:30 ` Peter Kjellerstedt
  2020-11-25 13:30 ` [PATCH 4/4] wic: Pass canonicalized " Peter Kjellerstedt
  3 siblings, 0 replies; 7+ messages in thread
From: Peter Kjellerstedt @ 2020-11-25 13:30 UTC (permalink / raw)
  To: openembedded-core

Introduce PSEUDO_IGNORE_REAL_PATHS and make it contain the canonicalized
paths from PSEUDO_IGNORE_PATHS, obtained by passing the latter to
oe.path.to_real_paths(). This is needed since pseudo's
pseudo_client_ignore_path_chroot() will compare the ignored paths to
paths that have been canonicalized.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/conf/bitbake.conf | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 9742fe4fe2..4862095a1b 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -685,15 +685,16 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
+PSEUDO_IGNORE_REAL_PATHS = "${@','.join(os.path.realpath(path) for path in (d.getVar('PSEUDO_IGNORE_PATHS') or '').split(','))}"
 PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
 
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
 #export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
 #export PSEUDO_LIBDIR = "${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native} PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native} PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=1"
 FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
-FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=0"
+FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=0"
 FAKEROOTNOENV = "PSEUDO_UNLOAD=1"
 FAKEROOTDIRS = "${PSEUDO_LOCALSTATEDIR}"
 PREFERRED_PROVIDER_virtual/fakeroot-native ?= "pseudo-native"
@@ -884,7 +885,8 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI
     BB_WORKERCONTEXT BB_LIMITEDDEPS BB_UNIHASH extend_recipe_sysroot DEPLOY_DIR \
     SSTATE_HASHEQUIV_METHOD SSTATE_HASHEQUIV_REPORT_TASKDATA \
     SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES"
-BB_HASHBASE_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_IGNORE_PATHS BUILDHISTORY_DIR SSTATE_DIR "
+BB_HASHBASE_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_IGNORE_PATHS \
+    PSEUDO_IGNORE_REAL_PATHS BUILDHISTORY_DIR SSTATE_DIR "
 BB_HASHCONFIG_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID \
     SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_EXTRAWHITE DISABLE_SANITY_CHECKS \
     PARALLEL_MAKE BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 4/4] wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS
  2020-11-25 13:30 [PATCH 0/4] Support symbolic links in paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
                   ` (2 preceding siblings ...)
  2020-11-25 13:30 ` [PATCH 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
@ 2020-11-25 13:30 ` Peter Kjellerstedt
  3 siblings, 0 replies; 7+ messages in thread
From: Peter Kjellerstedt @ 2020-11-25 13:30 UTC (permalink / raw)
  To: openembedded-core

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/image_types_wic.bbclass | 2 +-
 scripts/lib/wic/partition.py         | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/image_types_wic.bbclass b/meta/classes/image_types_wic.bbclass
index 286e0f5d54..9c1605c29c 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -5,7 +5,7 @@ WICVARS ?= "\
            IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
            ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS \
            KERNEL_IMAGETYPE MACHINE INITRAMFS_IMAGE INITRAMFS_IMAGE_BUNDLE INITRAMFS_LINK_NAME APPEND \
-           ASSUME_PROVIDED PSEUDO_IGNORE_PATHS"
+           ASSUME_PROVIDED PSEUDO_IGNORE_REAL_PATHS"
 
 inherit ${@bb.utils.contains('INITRAMFS_IMAGE_BUNDLE', '1', 'kernel-artifact-names', '', d)}
 
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index ebe250b00d..e910fd3fa0 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -211,7 +211,7 @@ class Partition():
             pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir
             pseudo += "export PSEUDO_PASSWD=%s;" % rootfs_dir
             pseudo += "export PSEUDO_NOSYMLINKEXP=1;"
-            pseudo += "export PSEUDO_IGNORE_PATHS=%s;" % (rootfs + "," + (get_bitbake_var("PSEUDO_IGNORE_PATHS") or ""))
+            pseudo += "export PSEUDO_IGNORE_PATHS=%s;" % (os.path.realpath(rootfs) + "," + (get_bitbake_var("PSEUDO_IGNORE_REAL_PATHS") or ""))
             pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
         else:
             pseudo = None

^ permalink raw reply related	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2020-11-25 13:47 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-11-25 13:30 [PATCH 0/4] Support symbolic links in paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
2020-11-25 13:30 ` [PATCH 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot() Peter Kjellerstedt
2020-11-25 13:30 ` [PATCH 2/4] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS Peter Kjellerstedt
2020-11-25 13:32   ` [OE-core] " Martin Jansa
2020-11-25 13:47     ` Peter Kjellerstedt
2020-11-25 13:30 ` [PATCH 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS Peter Kjellerstedt
2020-11-25 13:30 ` [PATCH 4/4] wic: Pass canonicalized " Peter Kjellerstedt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox