Openembedded Core Discussions
 help / color / mirror / Atom feed
* [OE-core][langdale 00/26] Patch review
@ 2023-04-06 16:01 Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 01/26] xdg-utils: Add a patch for CVE-2020-27748 Steve Sakoman
                   ` (25 more replies)
  0 siblings, 26 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

Please review this set of patches for langdale and have comments back by
end of day Monday.

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/5152

The following changes since commit 6c9ca2f2de98c84d11beccbfde7c1e26cb25dfb6:

  scripts/lib/buildstats: handle top-level build_stats not being complete (2023-03-30 04:10:26 -1000)

are available in the Git repository at:

  https://git.openembedded.org/openembedded-core-contrib stable/langdale-nut
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/langdale-nut

Andrew Geissler (1):
  filemap.py: enforce maximum of 4kb block size

Arturo Buzarra (1):
  run-postinsts: Set dependency for ldconfig to avoid boot issues

Enrico Jörns (7):
  oeqa/selftest/cases/runqemu: update imports
  oeqa/targetcontrol: fix misspelled RuntimeError
  oeqa/targetcontrol: do not set dump_host_cmds redundantly
  oeqa/targetcontrol: remove unused imports
  oeqa/utils/commands: fix usage of undefined EPIPE
  oeqa/utils/commands: remove unused imports
  oeqa/utils/qemurunner: replace hard-coded user 'root' in debug output

Fawzi KHABER (2):
  package.bbclass: check packages name conflict in do_package
  oeqa/selftest/cases/package.py: adding unittest for package rename
    conflicts

Jose Quaresma (3):
  oeqs/selftest: OESelftestTestContext: replace the os.environ after
    subprocess.check_output
  oeqa/selftest: OESelftestTestContext: convert relative to full path
    when newbuilddir is provided
  oeqa/selftest/reproducible: Split different packages from missing
    packages output

Khem Raj (1):
  Revert "runqemu: Add workaround for APIC hang on pre 4.15 kernels on
    qemux86"

Martin Jansa (3):
  selftest: runqemu: better check for ROOTFS: in the log
  selftest: runqemu: use better error message when asserts fail
  runqemu: respect IMAGE_LINK_NAME

Michael Halstead (1):
  uninative: Upgrade to 3.9 to include glibc 2.37

Mingli Yu (1):
  report-error: catch Nothing PROVIDES error

Richard Purdie (2):
  xdg-utils: Add a patch for CVE-2020-27748
  xdg-utils: Fix CVE number

Ross Burton (2):
  tzdata: upgrade to 2023c
  oeqa/runtime: clean up deprecated backslash expansion

Soumya (1):
  shadow: Fix can not print full login timeout message

Tobias Hagelborn (1):
  lib/oe/gpg_sign.py: Avoid race when creating .sig files in detach_sign

 .../packagenameconflict.bb                    |  10 +
 meta/classes-global/package.bbclass           |   9 +
 meta/classes-recipe/testimage.bbclass         |   2 +-
 meta/classes/report-error.bbclass             |  27 +-
 meta/conf/distro/include/yocto-uninative.inc  |  10 +-
 meta/lib/oe/gpg_sign.py                       |  27 +-
 meta/lib/oeqa/runtime/cases/apt.py            |   4 +-
 meta/lib/oeqa/runtime/cases/buildcpio.py      |   7 +-
 meta/lib/oeqa/runtime/cases/systemd.py        |   2 +-
 meta/lib/oeqa/selftest/cases/package.py       |   7 +
 meta/lib/oeqa/selftest/cases/reproducible.py  |  10 +-
 meta/lib/oeqa/selftest/cases/runqemu.py       |  16 +-
 meta/lib/oeqa/selftest/context.py             |  20 +-
 meta/lib/oeqa/targetcontrol.py                |   8 +-
 meta/lib/oeqa/utils/commands.py               |   6 +-
 meta/lib/oeqa/utils/qemurunner.py             |   4 +-
 .../run-postinsts/run-postinsts.service       |   2 +-
 .../0001-Fix-can-not-print-full-login.patch   |  41 +++
 meta/recipes-extended/shadow/shadow.inc       |   1 +
 meta/recipes-extended/timezone/timezone.inc   |   6 +-
 .../timezone/tzcode-native.bb                 |   2 -
 ...0001-Fix-C23-related-conformance-bug.patch | 301 ------------------
 .../xdg-utils/xdg-utils/CVE-2022-4055.patch   | 145 +++++++++
 .../xdg-utils/xdg-utils_1.1.3.bb              |   1 +
 scripts/lib/wic/filemap.py                    |   7 +
 scripts/runqemu                               |  74 +++--
 26 files changed, 371 insertions(+), 378 deletions(-)
 create mode 100644 meta-selftest/recipes-test/packagenameconflict/packagenameconflict.bb
 create mode 100644 meta/recipes-extended/shadow/files/0001-Fix-can-not-print-full-login.patch
 delete mode 100644 meta/recipes-extended/timezone/tzcode/0001-Fix-C23-related-conformance-bug.patch
 create mode 100644 meta/recipes-extended/xdg-utils/xdg-utils/CVE-2022-4055.patch

-- 
2.34.1



^ permalink raw reply	[flat|nested] 27+ messages in thread

* [OE-core][langdale 01/26] xdg-utils: Add a patch for CVE-2020-27748
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 02/26] xdg-utils: Fix CVE number Steve Sakoman
                   ` (24 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Take a patch submitted upstream for the issue while upstream decide what
to do. We don't use thunderbird integration so this isn't an issue for us.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b85b7714a44caa70beb2f115483ee52745aa1b97)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 .../xdg-utils/xdg-utils/CVE-2020-27748.patch  | 145 ++++++++++++++++++
 .../xdg-utils/xdg-utils_1.1.3.bb              |   1 +
 2 files changed, 146 insertions(+)
 create mode 100644 meta/recipes-extended/xdg-utils/xdg-utils/CVE-2020-27748.patch

diff --git a/meta/recipes-extended/xdg-utils/xdg-utils/CVE-2020-27748.patch b/meta/recipes-extended/xdg-utils/xdg-utils/CVE-2020-27748.patch
new file mode 100644
index 0000000000..ec3605e158
--- /dev/null
+++ b/meta/recipes-extended/xdg-utils/xdg-utils/CVE-2020-27748.patch
@@ -0,0 +1,145 @@
+xdg-email does not parse mailto uris properly for thunderbird
+
+When using thunderbird as mailto handler xdg-email translates mailto uris into an 'thunderbird -compose' argument. While to, cc and bcc values are properly enclosed in single quotes this is not the case for subject or body. This breaks functionality and allows to use all thunderbird -compose arguments within a mailto uri, e.g.
+
+xdg-email 'mailto:test@example.com?subject=Test,attachment=~/.thunderbird/profiles.ini,message=/home/test/test.txt'
+
+translates into
+
+thunderbird -compose to='test@example.com,',subject=Test,attachment=~/.thunderbird/profiles.ini,message=/home/test/test.txt
+
+with working attachment and message. (And, yes, ~ expands to the home directory.)
+
+Upstream-Status: Submitted [https://gitlab.freedesktop.org/xdg/xdg-utils/-/issues/205]
+
+Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
+
+CVE: CVE-2020-27748
+
+
+Index: xdg-utils-1.1.3/scripts/xdg-email.in
+===================================================================
+--- xdg-utils-1.1.3.orig/scripts/xdg-email.in
++++ xdg-utils-1.1.3/scripts/xdg-email.in
+@@ -30,53 +30,6 @@ _USAGE
+ 
+ #@xdg-utils-common@
+ 
+-run_thunderbird()
+-{
+-    local THUNDERBIRD MAILTO NEWMAILTO TO CC BCC SUBJECT BODY
+-    THUNDERBIRD="$1"
+-    MAILTO=$(echo "$2" | sed 's/^mailto://')
+-    echo "$MAILTO" | grep -qs "^?"
+-    if [ "$?" = "0" ] ; then
+-        MAILTO=$(echo "$MAILTO" | sed 's/^?//')
+-    else
+-        MAILTO=$(echo "$MAILTO" | sed 's/^/to=/' | sed 's/?/\&/')
+-    fi
+-
+-    MAILTO=$(echo "$MAILTO" | sed 's/&/\n/g')
+-    TO=$(/bin/echo -e $(echo "$MAILTO" | grep '^to=' | sed 's/^to=//;s/%\(..\)/\\x\1/g' | awk '{ printf "%s,",$0 }'))
+-    CC=$(/bin/echo -e $(echo "$MAILTO" | grep '^cc=' | sed 's/^cc=//;s/%\(..\)/\\x\1/g' | awk '{ printf "%s,",$0 }'))
+-    BCC=$(/bin/echo -e $(echo "$MAILTO" | grep '^bcc=' | sed 's/^bcc=//;s/%\(..\)/\\x\1/g' | awk '{ printf "%s,",$0 }'))
+-    SUBJECT=$(echo "$MAILTO" | grep '^subject=' | tail -n 1)
+-    BODY=$(echo "$MAILTO" | grep '^body=' | tail -n 1)
+-
+-    if [ -z "$TO" ] ; then
+-        NEWMAILTO=
+-    else
+-        NEWMAILTO="to='$TO'"
+-    fi
+-    if [ -n "$CC" ] ; then
+-        NEWMAILTO="${NEWMAILTO},cc='$CC'"
+-    fi
+-    if [ -n "$BCC" ] ; then
+-        NEWMAILTO="${NEWMAILTO},bcc='$BCC'"
+-    fi
+-    if [ -n "$SUBJECT" ] ; then
+-        NEWMAILTO="${NEWMAILTO},$SUBJECT"
+-    fi
+-    if [ -n "$BODY" ] ; then
+-        NEWMAILTO="${NEWMAILTO},$BODY"
+-    fi
+-
+-    NEWMAILTO=$(echo "$NEWMAILTO" | sed 's/^,//')
+-    DEBUG 1 "Running $THUNDERBIRD -compose \"$NEWMAILTO\""
+-    "$THUNDERBIRD" -compose "$NEWMAILTO"
+-    if [ $? -eq 0 ]; then
+-        exit_success
+-    else
+-        exit_failure_operation_failed
+-    fi
+-}
+-
+ open_kde()
+ {
+     if [ -n "$KDE_SESSION_VERSION" ] && [ "$KDE_SESSION_VERSION" -ge 5 ]; then
+@@ -130,15 +83,6 @@ open_kde()
+ 
+ open_gnome3()
+ {
+-    local client
+-    local desktop
+-    desktop=`xdg-mime query default "x-scheme-handler/mailto"`
+-    client=`desktop_file_to_binary "$desktop"`
+-    echo $client | grep -E 'thunderbird|icedove' > /dev/null 2>&1
+-    if [ $? -eq 0 ] ; then
+-        run_thunderbird "$client" "$1"
+-    fi
+-
+     if gio help open 2>/dev/null 1>&2; then
+         DEBUG 1 "Running gio open \"$1\""
+         gio open "$1"
+@@ -159,13 +103,6 @@ open_gnome3()
+ 
+ open_gnome()
+ {
+-    local client
+-    client=`gconftool-2 --get /desktop/gnome/url-handlers/mailto/command | cut -d ' ' -f 1` || ""
+-    echo $client | grep -E 'thunderbird|icedove' > /dev/null 2>&1
+-    if [ $? -eq 0 ] ; then
+-        run_thunderbird "$client" "$1"
+-    fi
+-
+     if gio help open 2>/dev/null 1>&2; then
+         DEBUG 1 "Running gio open \"$1\""
+         gio open "$1"
+@@ -231,15 +168,6 @@ open_flatpak()
+ 
+ open_generic()
+ {
+-    local client
+-    local desktop
+-    desktop=`xdg-mime query default "x-scheme-handler/mailto"`
+-    client=`desktop_file_to_binary "$desktop"`
+-    echo $client | grep -E 'thunderbird|icedove' > /dev/null 2>&1
+-    if [ $? -eq 0 ] ; then
+-        run_thunderbird "$client" "$1"
+-    fi
+-
+     xdg-open "$1"
+     local ret=$?
+ 
+@@ -364,21 +292,6 @@ while [ $# -gt 0 ] ; do
+         shift
+         ;;
+ 
+-      --attach)
+-        if [ -z "$1" ] ; then
+-            exit_failure_syntax "file argument missing for --attach option"
+-        fi
+-        check_input_file "$1"
+-        file=`readlink -f "$1"` # Normalize path
+-        if [ -z "$file" ] || [ ! -f "$file" ] ; then
+-            exit_failure_file_missing "file '$1' does not exist"
+-        fi
+-
+-        url_encode "$file"
+-        options="${options}attach=${result}&"
+-        shift
+-        ;;
+-
+       -*)
+         exit_failure_syntax "unexpected option '$parm'"
+         ;;
diff --git a/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb b/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb
index 73acf6b744..d95bcccd2a 100644
--- a/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb
+++ b/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb
@@ -21,6 +21,7 @@ SRC_URI = "https://portland.freedesktop.org/download/${BPN}-${PV}.tar.gz \
            file://0001-Reinstate-xdg-terminal.patch \
            file://0001-Don-t-build-the-in-script-manual.patch \
            file://1f199813e0eb0246f63b54e9e154970e609575af.patch \
+           file://CVE-2020-27748.patch \
           "
 
 SRC_URI[md5sum] = "902042508b626027a3709d105f0b63ff"
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 02/26] xdg-utils: Fix CVE number
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 01/26] xdg-utils: Add a patch for CVE-2020-27748 Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 03/26] tzdata: upgrade to 2023c Steve Sakoman
                   ` (23 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Richard Purdie <richard.purdie@linuxfoundation.org>

In the previous commit I somehow mixed up and used an incorrect CVE number.
Use the correct one.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b3e2729f686ff6e16e11590bcd701c057ae5f1e2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 .../xdg-utils/{CVE-2020-27748.patch => CVE-2022-4055.patch}     | 2 +-
 meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb              | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-extended/xdg-utils/xdg-utils/{CVE-2020-27748.patch => CVE-2022-4055.patch} (99%)

diff --git a/meta/recipes-extended/xdg-utils/xdg-utils/CVE-2020-27748.patch b/meta/recipes-extended/xdg-utils/xdg-utils/CVE-2022-4055.patch
similarity index 99%
rename from meta/recipes-extended/xdg-utils/xdg-utils/CVE-2020-27748.patch
rename to meta/recipes-extended/xdg-utils/xdg-utils/CVE-2022-4055.patch
index ec3605e158..b236030108 100644
--- a/meta/recipes-extended/xdg-utils/xdg-utils/CVE-2020-27748.patch
+++ b/meta/recipes-extended/xdg-utils/xdg-utils/CVE-2022-4055.patch
@@ -14,7 +14,7 @@ Upstream-Status: Submitted [https://gitlab.freedesktop.org/xdg/xdg-utils/-/issue
 
 Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
 
-CVE: CVE-2020-27748
+CVE: CVE-2022-4055
 
 
 Index: xdg-utils-1.1.3/scripts/xdg-email.in
diff --git a/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb b/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb
index d95bcccd2a..4d93180535 100644
--- a/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb
+++ b/meta/recipes-extended/xdg-utils/xdg-utils_1.1.3.bb
@@ -21,7 +21,7 @@ SRC_URI = "https://portland.freedesktop.org/download/${BPN}-${PV}.tar.gz \
            file://0001-Reinstate-xdg-terminal.patch \
            file://0001-Don-t-build-the-in-script-manual.patch \
            file://1f199813e0eb0246f63b54e9e154970e609575af.patch \
-           file://CVE-2020-27748.patch \
+           file://CVE-2022-4055.patch \
           "
 
 SRC_URI[md5sum] = "902042508b626027a3709d105f0b63ff"
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 03/26] tzdata: upgrade to 2023c
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 01/26] xdg-utils: Add a patch for CVE-2020-27748 Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 02/26] xdg-utils: Fix CVE number Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 04/26] uninative: Upgrade to 3.9 to include glibc 2.37 Steve Sakoman
                   ` (22 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Ross Burton <ross.burton@arm.com>

Drop a backport patch as it is now integrated.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 80d26d1da47dcd9213a7083d9493a7bce0897a57)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/recipes-extended/timezone/timezone.inc   |   6 +-
 .../timezone/tzcode-native.bb                 |   2 -
 ...0001-Fix-C23-related-conformance-bug.patch | 301 ------------------
 3 files changed, 3 insertions(+), 306 deletions(-)
 delete mode 100644 meta/recipes-extended/timezone/tzcode/0001-Fix-C23-related-conformance-bug.patch

diff --git a/meta/recipes-extended/timezone/timezone.inc b/meta/recipes-extended/timezone/timezone.inc
index eec7177228..14a1ce18f3 100644
--- a/meta/recipes-extended/timezone/timezone.inc
+++ b/meta/recipes-extended/timezone/timezone.inc
@@ -6,7 +6,7 @@ SECTION = "base"
 LICENSE = "PD & BSD-3-Clause"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=c679c9d6b02bc2757b3eaf8f53c43fba"
 
-PV = "2022g"
+PV = "2023c"
 
 SRC_URI =" http://www.iana.org/time-zones/repository/releases/tzcode${PV}.tar.gz;name=tzcode;subdir=tz \
            http://www.iana.org/time-zones/repository/releases/tzdata${PV}.tar.gz;name=tzdata;subdir=tz \
@@ -16,5 +16,5 @@ S = "${WORKDIR}/tz"
 
 UPSTREAM_CHECK_URI = "http://www.iana.org/time-zones"
 
-SRC_URI[tzcode.sha256sum] = "9610bb0b9656ff404c361a41f3286da53064b5469d84f00c9cb2314c8614da74"
-SRC_URI[tzdata.sha256sum] = "4491db8281ae94a84d939e427bdd83dc389f26764d27d9a5c52d782c16764478"
+SRC_URI[tzcode.sha256sum] = "46d17f2bb19ad73290f03a203006152e0fa0d7b11e5b71467c4a823811b214e7"
+SRC_URI[tzdata.sha256sum] = "3f510b5d1b4ae9bb38e485aa302a776b317fb3637bdb6404c4adf7b6cadd965c"
diff --git a/meta/recipes-extended/timezone/tzcode-native.bb b/meta/recipes-extended/timezone/tzcode-native.bb
index 6d52b3c422..d0b23a9d80 100644
--- a/meta/recipes-extended/timezone/tzcode-native.bb
+++ b/meta/recipes-extended/timezone/tzcode-native.bb
@@ -2,8 +2,6 @@ require timezone.inc
 
 SUMMARY = "tzcode, timezone zoneinfo utils -- zic, zdump, tzselect"
 
-SRC_URI += "file://0001-Fix-C23-related-conformance-bug.patch"
-
 inherit native
 
 EXTRA_OEMAKE += "cc='${CC}'"
diff --git a/meta/recipes-extended/timezone/tzcode/0001-Fix-C23-related-conformance-bug.patch b/meta/recipes-extended/timezone/tzcode/0001-Fix-C23-related-conformance-bug.patch
deleted file mode 100644
index c91ef93e95..0000000000
--- a/meta/recipes-extended/timezone/tzcode/0001-Fix-C23-related-conformance-bug.patch
+++ /dev/null
@@ -1,301 +0,0 @@
-From 509c5974398952618abdd17f39117b88e3f50057 Mon Sep 17 00:00:00 2001
-From: Paul Eggert <eggert@cs.ucla.edu>
-Date: Thu, 1 Dec 2022 10:28:04 -0800
-Subject: [PATCH] Fix C23-related conformance bug
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-Problem reported by Houge Langley for ‘gcc -std=gnu99’ in:
-https://bugs.gentoo.org/show_bug.cgi?id=883719
-* NEWS: Mention this.
-* date.c, localtime.c, private.h, zdump.c, zic.c:
-Use ATTRIBUTE_* at the start of function declarations,
-not later (such as after the keyword ‘static’).
-This is required for strict conformance to C23.
-
-Upstream-Status: Backport [https://github.com/eggert/tz/commit/9cfe9507fcc22cd4a0c4da486ea1c7f0de6b075f]
-
-NEWS change skipped to avoid conflicts.
-
-Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
----
- date.c      |  2 +-
- localtime.c |  4 ++--
- private.h   |  6 +++---
- zdump.c     | 12 ++++++------
- zic.c       | 34 +++++++++++++++++-----------------
- 5 files changed, 29 insertions(+), 29 deletions(-)
-
-diff --git a/date.c b/date.c
-index 11c5e5fe..97df6ab0 100644
---- a/date.c
-+++ b/date.c
-@@ -42,7 +42,7 @@ static void		display(const char *, time_t);
- static void		dogmt(void);
- static void		errensure(void);
- static void		timeout(FILE *, const char *, const struct tm *);
--static ATTRIBUTE_NORETURN void usage(void);
-+ATTRIBUTE_NORETURN static void usage(void);
- 
- int
- main(const int argc, char *argv[])
-diff --git a/localtime.c b/localtime.c
-index 1d22d351..3bf1b911 100644
---- a/localtime.c
-+++ b/localtime.c
-@@ -838,7 +838,7 @@ is_digit(char c)
- ** Return a pointer to that character.
- */
- 
--static ATTRIBUTE_REPRODUCIBLE const char *
-+ATTRIBUTE_REPRODUCIBLE static const char *
- getzname(register const char *strp)
- {
- 	register char	c;
-@@ -859,7 +859,7 @@ getzname(register const char *strp)
- ** We don't do any checking here; checking is done later in common-case code.
- */
- 
--static ATTRIBUTE_REPRODUCIBLE const char *
-+ATTRIBUTE_REPRODUCIBLE static const char *
- getqzname(register const char *strp, const int delim)
- {
- 	register int	c;
-diff --git a/private.h b/private.h
-index 7a73eff7..ae522986 100644
---- a/private.h
-+++ b/private.h
-@@ -628,7 +628,7 @@ char *asctime(struct tm const *);
- char *asctime_r(struct tm const *restrict, char *restrict);
- char *ctime(time_t const *);
- char *ctime_r(time_t const *, char *);
--double difftime(time_t, time_t) ATTRIBUTE_UNSEQUENCED;
-+ATTRIBUTE_UNSEQUENCED double difftime(time_t, time_t);
- size_t strftime(char *restrict, size_t, char const *restrict,
- 		struct tm const *restrict);
- # if HAVE_STRFTIME_L
-@@ -740,10 +740,10 @@ timezone_t tzalloc(char const *);
- void tzfree(timezone_t);
- # ifdef STD_INSPIRED
- #  if TZ_TIME_T || !defined posix2time_z
--time_t posix2time_z(timezone_t, time_t) ATTRIBUTE_REPRODUCIBLE;
-+ATTRIBUTE_REPRODUCIBLE time_t posix2time_z(timezone_t, time_t);
- #  endif
- #  if TZ_TIME_T || !defined time2posix_z
--time_t time2posix_z(timezone_t, time_t) ATTRIBUTE_REPRODUCIBLE;
-+ATTRIBUTE_REPRODUCIBLE time_t time2posix_z(timezone_t, time_t);
- #  endif
- # endif
- #endif
-diff --git a/zdump.c b/zdump.c
-index 7acb3e2d..3e482ba3 100644
---- a/zdump.c
-+++ b/zdump.c
-@@ -89,7 +89,7 @@ static bool	warned;
- static bool	errout;
- 
- static char const *abbr(struct tm const *);
--static intmax_t	delta(struct tm *, struct tm *) ATTRIBUTE_REPRODUCIBLE;
-+ATTRIBUTE_REPRODUCIBLE static intmax_t delta(struct tm *, struct tm *);
- static void dumptime(struct tm const *);
- static time_t hunt(timezone_t, time_t, time_t, bool);
- static void show(timezone_t, char *, time_t, bool);
-@@ -97,7 +97,7 @@ static void showextrema(timezone_t, char *, time_t, struct tm *, time_t);
- static void showtrans(char const *, struct tm const *, time_t, char const *,
- 		      char const *);
- static const char *tformat(void);
--static time_t yeartot(intmax_t) ATTRIBUTE_REPRODUCIBLE;
-+ATTRIBUTE_REPRODUCIBLE static time_t yeartot(intmax_t);
- 
- /* Is C an ASCII digit?  */
- static bool
-@@ -125,7 +125,7 @@ is_alpha(char a)
- 	}
- }
- 
--static ATTRIBUTE_NORETURN void
-+ATTRIBUTE_NORETURN static void
- size_overflow(void)
- {
-   fprintf(stderr, _("%s: size overflow\n"), progname);
-@@ -134,7 +134,7 @@ size_overflow(void)
- 
- /* Return A + B, exiting if the result would overflow either ptrdiff_t
-    or size_t.  */
--static ATTRIBUTE_REPRODUCIBLE ptrdiff_t
-+ATTRIBUTE_REPRODUCIBLE static ptrdiff_t
- sumsize(size_t a, size_t b)
- {
- #ifdef ckd_add
-@@ -151,7 +151,7 @@ sumsize(size_t a, size_t b)
- 
- /* Return a pointer to a newly allocated buffer of size SIZE, exiting
-    on failure.  SIZE should be nonzero.  */
--static void * ATTRIBUTE_MALLOC
-+ATTRIBUTE_MALLOC static void *
- xmalloc(size_t size)
- {
-   void *p = malloc(size);
-@@ -920,7 +920,7 @@ showextrema(timezone_t tz, char *zone, time_t lo, struct tm *lotmp, time_t hi)
- # include <stdarg.h>
- 
- /* A substitute for snprintf that is good enough for zdump.  */
--static int ATTRIBUTE_FORMAT((printf, 3, 4))
-+ATTRIBUTE_FORMAT((printf, 3, 4)) static int
- my_snprintf(char *s, size_t size, char const *format, ...)
- {
-   int n;
-diff --git a/zic.c b/zic.c
-index 892414af..f143fcef 100644
---- a/zic.c
-+++ b/zic.c
-@@ -459,20 +459,20 @@ static char		roll[TZ_MAX_LEAPS];
- ** Memory allocation.
- */
- 
--static ATTRIBUTE_NORETURN void
-+ATTRIBUTE_NORETURN static void
- memory_exhausted(const char *msg)
- {
- 	fprintf(stderr, _("%s: Memory exhausted: %s\n"), progname, msg);
- 	exit(EXIT_FAILURE);
- }
- 
--static ATTRIBUTE_NORETURN void
-+ATTRIBUTE_NORETURN static void
- size_overflow(void)
- {
-   memory_exhausted(_("size overflow"));
- }
- 
--static ATTRIBUTE_REPRODUCIBLE ptrdiff_t
-+ATTRIBUTE_REPRODUCIBLE static ptrdiff_t
- size_sum(size_t a, size_t b)
- {
- #ifdef ckd_add
-@@ -487,7 +487,7 @@ size_sum(size_t a, size_t b)
-   size_overflow();
- }
- 
--static ATTRIBUTE_REPRODUCIBLE ptrdiff_t
-+ATTRIBUTE_REPRODUCIBLE static ptrdiff_t
- size_product(ptrdiff_t nitems, ptrdiff_t itemsize)
- {
- #ifdef ckd_mul
-@@ -502,7 +502,7 @@ size_product(ptrdiff_t nitems, ptrdiff_t itemsize)
-   size_overflow();
- }
- 
--static ATTRIBUTE_REPRODUCIBLE ptrdiff_t
-+ATTRIBUTE_REPRODUCIBLE static ptrdiff_t
- align_to(ptrdiff_t size, ptrdiff_t alignment)
- {
-   ptrdiff_t lo_bits = alignment - 1, sum = size_sum(size, lo_bits);
-@@ -526,7 +526,7 @@ memcheck(void *ptr)
- 	return ptr;
- }
- 
--static void * ATTRIBUTE_MALLOC
-+ATTRIBUTE_MALLOC static void *
- emalloc(size_t size)
- {
-   return memcheck(malloc(size));
-@@ -538,7 +538,7 @@ erealloc(void *ptr, size_t size)
-   return memcheck(realloc(ptr, size));
- }
- 
--static char * ATTRIBUTE_MALLOC
-+ATTRIBUTE_MALLOC static char *
- estrdup(char const *str)
- {
-   return memcheck(strdup(str));
-@@ -608,7 +608,7 @@ eat(int fnum, lineno num)
- 	eats(fnum, num, 0, -1);
- }
- 
--static void ATTRIBUTE_FORMAT((printf, 1, 0))
-+ATTRIBUTE_FORMAT((printf, 1, 0)) static void
- verror(const char *const string, va_list args)
- {
- 	/*
-@@ -626,7 +626,7 @@ verror(const char *const string, va_list args)
- 	fprintf(stderr, "\n");
- }
- 
--static void ATTRIBUTE_FORMAT((printf, 1, 2))
-+ATTRIBUTE_FORMAT((printf, 1, 2)) static void
- error(const char *const string, ...)
- {
- 	va_list args;
-@@ -636,7 +636,7 @@ error(const char *const string, ...)
- 	errors = true;
- }
- 
--static void ATTRIBUTE_FORMAT((printf, 1, 2))
-+ATTRIBUTE_FORMAT((printf, 1, 2)) static void
- warning(const char *const string, ...)
- {
- 	va_list args;
-@@ -666,7 +666,7 @@ close_file(FILE *stream, char const *dir, char const *name,
-   }
- }
- 
--static ATTRIBUTE_NORETURN void
-+ATTRIBUTE_NORETURN static void
- usage(FILE *stream, int status)
- {
-   fprintf(stream,
-@@ -3597,7 +3597,7 @@ lowerit(char a)
- }
- 
- /* case-insensitive equality */
--static ATTRIBUTE_REPRODUCIBLE bool
-+ATTRIBUTE_REPRODUCIBLE static bool
- ciequal(register const char *ap, register const char *bp)
- {
- 	while (lowerit(*ap) == lowerit(*bp++))
-@@ -3606,7 +3606,7 @@ ciequal(register const char *ap, register const char *bp)
- 	return false;
- }
- 
--static ATTRIBUTE_REPRODUCIBLE bool
-+ATTRIBUTE_REPRODUCIBLE static bool
- itsabbr(register const char *abbr, register const char *word)
- {
- 	if (lowerit(*abbr) != lowerit(*word))
-@@ -3622,7 +3622,7 @@ itsabbr(register const char *abbr, register const char *word)
- 
- /* Return true if ABBR is an initial prefix of WORD, ignoring ASCII case.  */
- 
--static ATTRIBUTE_REPRODUCIBLE bool
-+ATTRIBUTE_REPRODUCIBLE static bool
- ciprefix(char const *abbr, char const *word)
- {
-   do
-@@ -3725,14 +3725,14 @@ getfields(char *cp, char **array, int arrayelts)
- 	return nsubs;
- }
- 
--static ATTRIBUTE_NORETURN void
-+ATTRIBUTE_NORETURN static void
- time_overflow(void)
- {
-   error(_("time overflow"));
-   exit(EXIT_FAILURE);
- }
- 
--static ATTRIBUTE_REPRODUCIBLE zic_t
-+ATTRIBUTE_REPRODUCIBLE static zic_t
- oadd(zic_t t1, zic_t t2)
- {
- #ifdef ckd_add
-@@ -3746,7 +3746,7 @@ oadd(zic_t t1, zic_t t2)
-   time_overflow();
- }
- 
--static ATTRIBUTE_REPRODUCIBLE zic_t
-+ATTRIBUTE_REPRODUCIBLE static zic_t
- tadd(zic_t t1, zic_t t2)
- {
- #ifdef ckd_add
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 04/26] uninative: Upgrade to 3.9 to include glibc 2.37
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (2 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 03/26] tzdata: upgrade to 2023c Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 05/26] filemap.py: enforce maximum of 4kb block size Steve Sakoman
                   ` (21 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Michael Halstead <mhalstead@linuxfoundation.org>

Update uninative to work with the new glibc 2.37.

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/conf/distro/include/yocto-uninative.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/conf/distro/include/yocto-uninative.inc b/meta/conf/distro/include/yocto-uninative.inc
index 51c0cdee43..8a5cab5360 100644
--- a/meta/conf/distro/include/yocto-uninative.inc
+++ b/meta/conf/distro/include/yocto-uninative.inc
@@ -6,10 +6,10 @@
 # to the distro running on the build machine.
 #
 
-UNINATIVE_MAXGLIBCVERSION = "2.36"
-UNINATIVE_VERSION = "3.8.1"
+UNINATIVE_MAXGLIBCVERSION = "2.37"
+UNINATIVE_VERSION = "3.9"
 
 UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/${UNINATIVE_VERSION}/"
-UNINATIVE_CHECKSUM[aarch64] ?= "3f15d420049b21854bd7a8843da0f42f17064559492c8b752d7a6f998ff5ea65"
-UNINATIVE_CHECKSUM[i686] ?= "a6dcf316d738ade2e5e463bd3b33a270b4bfc25bba41770ad5cbdc3b0e24044c"
-UNINATIVE_CHECKSUM[x86_64] ?= "5fab9a5c97fc73a21134e5a81f74498cbaecda75d56aab971c934e0b803bcc00"
+UNINATIVE_CHECKSUM[aarch64] ?= "de35708c95c34573af140da910132c3291ba4fd26ebf7b74b755ada432cdf07b"
+UNINATIVE_CHECKSUM[i686] ?= "adac07b08adb88eb26fc7fd87fee0cec9d5be167bf7c5ffd3a549a2a6699c29c"
+UNINATIVE_CHECKSUM[x86_64] ?= "3dd82c3fbdb59e87bf091c3eef555a05fae528eeda3083828f76cd4deaceca8b"
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 05/26] filemap.py: enforce maximum of 4kb block size
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (3 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 04/26] uninative: Upgrade to 3.9 to include glibc 2.37 Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 06/26] oeqs/selftest: OESelftestTestContext: replace the os.environ after subprocess.check_output Steve Sakoman
                   ` (20 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Andrew Geissler <geissonator@gmail.com>

The logic in this script validates that the length of data sections are
evenly divisible by the block size. On most systems the block size is
4KB and all is good. Some systems though, such as ppc64le, have a block
size larger then 4KB. For example on a POWER9 based ppc64le system, the
block size is 64KB.

This results in this script failing with errors like this when building
wic images:
|440, in _do_get_mapped_ranges
|     assert extent_len % self.block_size == 0
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| AssertionError

In this case the data section size was 268KB and the block size was
64KB, resulting in the above assert failure.

Resolves https://bugzilla.yoctoproject.org/show_bug.cgi?id=15075

Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1e23b803af6991fc20e4a4e88a0ef0541399e722)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 scripts/lib/wic/filemap.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 4d9da28172..85b39d5d74 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -46,6 +46,13 @@ def get_block_size(file_obj):
             bsize = stat.st_blksize
         else:
             raise IOError("Unable to determine block size")
+
+    # The logic in this script only supports a maximum of a 4KB
+    # block size
+    max_block_size = 4 * 1024
+    if bsize > max_block_size:
+        bsize = max_block_size
+
     return bsize
 
 class ErrorNotSupp(Exception):
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 06/26] oeqs/selftest: OESelftestTestContext: replace the os.environ after subprocess.check_output
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (4 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 05/26] filemap.py: enforce maximum of 4kb block size Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 07/26] oeqa/selftest: OESelftestTestContext: convert relative to full path when newbuilddir is provided Steve Sakoman
                   ` (19 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Jose Quaresma <quaresma.jose@gmail.com>

No function change but is needed to simplify the next patch.

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 193d8f0d6383f54c61f90d58eeb8f41580c2360a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/selftest/context.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/context.py b/meta/lib/oeqa/selftest/context.py
index 78c7a467e2..0d26a07438 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -86,17 +86,17 @@ class OESelftestTestContext(OETestContext):
         oe.path.copytree(builddir + "/cache", newbuilddir + "/cache")
         oe.path.copytree(selftestdir, newselftestdir)
 
+        subprocess.check_output("git init; git add *; git commit -a -m 'initial'", cwd=newselftestdir, shell=True)
+
+        # Tried to used bitbake-layers add/remove but it requires recipe parsing and hence is too slow
+        subprocess.check_output("sed %s/conf/bblayers.conf -i -e 's#%s#%s#g'" % (newbuilddir, selftestdir, newselftestdir), cwd=newbuilddir, shell=True)
+
         for e in os.environ:
             if builddir + "/" in os.environ[e]:
                 os.environ[e] = os.environ[e].replace(builddir + "/", newbuilddir + "/")
             if os.environ[e].endswith(builddir):
                 os.environ[e] = os.environ[e].replace(builddir, newbuilddir)
 
-        subprocess.check_output("git init; git add *; git commit -a -m 'initial'", cwd=newselftestdir, shell=True)
-
-        # Tried to used bitbake-layers add/remove but it requires recipe parsing and hence is too slow
-        subprocess.check_output("sed %s/conf/bblayers.conf -i -e 's#%s#%s#g'" % (newbuilddir, selftestdir, newselftestdir), cwd=newbuilddir, shell=True)
-
         os.chdir(newbuilddir)
 
         def patch_test(t):
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 07/26] oeqa/selftest: OESelftestTestContext: convert relative to full path when newbuilddir is provided
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (5 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 06/26] oeqs/selftest: OESelftestTestContext: replace the os.environ after subprocess.check_output Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 08/26] oeqa/selftest/reproducible: Split different packages from missing packages output Steve Sakoman
                   ` (18 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Jose Quaresma <quaresma.jose@gmail.com>

Relative paths in BBLAYERS only works when the new build dir are on the same
ascending directory node:
 .
 ├── build
 ├── build-st

It works because they share the same ascending relative directory node.

So use the full path when the argument newbuilddir is provided
to make the oe-selftest work everywere regardless of the location chosen.

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 2e022c1977bc1006c00a87e08a2dca5b69db4801)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/selftest/context.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/lib/oeqa/selftest/context.py b/meta/lib/oeqa/selftest/context.py
index 0d26a07438..0e3244a1c5 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -91,6 +91,16 @@ class OESelftestTestContext(OETestContext):
         # Tried to used bitbake-layers add/remove but it requires recipe parsing and hence is too slow
         subprocess.check_output("sed %s/conf/bblayers.conf -i -e 's#%s#%s#g'" % (newbuilddir, selftestdir, newselftestdir), cwd=newbuilddir, shell=True)
 
+        # Relative paths in BBLAYERS only works when the new build dir share the same ascending node
+        if self.newbuilddir:
+            bblayers = subprocess.check_output("bitbake-getvar --value BBLAYERS | tail -1", cwd=builddir, shell=True, text=True)
+            if '..' in bblayers:
+                bblayers_abspath = [os.path.abspath(path) for path in bblayers.split()]
+                with open("%s/conf/bblayers.conf" % newbuilddir, "a") as f:
+                    newbblayers = "# new bblayers to be used by selftest in the new build dir '%s'\n" % newbuilddir
+                    newbblayers += 'BBLAYERS = "%s"\n' % ' '.join(bblayers_abspath)
+                    f.write(newbblayers)
+
         for e in os.environ:
             if builddir + "/" in os.environ[e]:
                 os.environ[e] = os.environ[e].replace(builddir + "/", newbuilddir + "/")
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 08/26] oeqa/selftest/reproducible: Split different packages from missing packages output
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (6 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 07/26] oeqa/selftest: OESelftestTestContext: convert relative to full path when newbuilddir is provided Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 09/26] run-postinsts: Set dependency for ldconfig to avoid boot issues Steve Sakoman
                   ` (17 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Jose Quaresma <quaresma.jose@gmail.com>

When we provide sstate_targets recipes they are reported as missing
which is not quite correct because they are proveided to cut build/debugging time.

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit fd70a564d6946fa460638dd04ce2daecf4566cf3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index f4dd779842..cd7aa8aafa 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -292,9 +292,13 @@ class ReproducibleTests(OESelftestTestCase):
                         self.copy_file(d.reference, '/'.join([save_dir, 'packages-excluded', strip_topdir(d.reference)]))
                         self.copy_file(d.test, '/'.join([save_dir, 'packages-excluded', strip_topdir(d.test)]))
 
-                if result.missing or result.different:
-                    fails.append("The following %s packages are missing or different and not in exclusion list: %s" %
-                            (c, '\n'.join(r.test for r in (result.missing + result.different))))
+                if result.different:
+                    fails.append("The following %s packages are different and not in exclusion list:\n%s" %
+                            (c, '\n'.join(r.test for r in (result.different))))
+
+                if result.missing and len(self.sstate_targets) == 0:
+                    fails.append("The following %s packages are missing and not in exclusion list:\n%s" %
+                            (c, '\n'.join(r.test for r in (result.missing))))
 
         # Clean up empty directories
         if self.save_results:
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 09/26] run-postinsts: Set dependency for ldconfig to avoid boot issues
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (7 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 08/26] oeqa/selftest/reproducible: Split different packages from missing packages output Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 10/26] package.bbclass: check packages name conflict in do_package Steve Sakoman
                   ` (16 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Arturo Buzarra <arturo.buzarra@digi.com>

If a package with a postsints script requires ldconfig, the package class adds
a ldconfig postinst fragment to initialize it before. Systemd has its own
ldconfig.service to initialize it and sometimes if both services are running
at the same time in the first boot, the first one will work, but the second
one will fail with the following error:

    ldconfig[141]: /sbin/ldconfig: Renaming of /etc/ld.so.cache~ to /etc/ld.so.cache failed: No such file or directory

This commit adds a ordering dependency between them to make sure that only one
service is running at the same time.

Signed-off-by: Arturo Buzarra <arturo.buzarra@digi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4e9d812e127dc6743f52f4881e509e8e2e833afe)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 .../run-postinsts/run-postinsts/run-postinsts.service           | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts.service b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts.service
index 7f72f3388a..b6b81d5c1a 100644
--- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts.service
+++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts.service
@@ -1,7 +1,7 @@
 [Unit]
 Description=Run pending postinsts
 DefaultDependencies=no
-After=systemd-remount-fs.service systemd-tmpfiles-setup.service tmp.mount
+After=systemd-remount-fs.service systemd-tmpfiles-setup.service tmp.mount ldconfig.service
 Before=sysinit.target
 
 [Service]
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 10/26] package.bbclass: check packages name conflict in do_package
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (8 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 09/26] run-postinsts: Set dependency for ldconfig to avoid boot issues Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 11/26] oeqa/selftest/cases/package.py: adding unittest for package rename conflicts Steve Sakoman
                   ` (15 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Fawzi KHABER <fawzi.khaber@smile.fr>

It is possible to rename packages with the macro PKG:${PN} and result in
a package name conflict if the target name exists already.

Add a fatal check to prevent this issue to go unnoticed.

Fix [Yocto #12060]

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
Signed-off-by: Fawzi KHABER <fawzi.khaber@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f6cb63d3a871c4eef52d981131a56bf41aa1dcdf)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/classes-global/package.bbclass | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/meta/classes-global/package.bbclass b/meta/classes-global/package.bbclass
index 7a0a428b30..a47da14ea5 100644
--- a/meta/classes-global/package.bbclass
+++ b/meta/classes-global/package.bbclass
@@ -2449,6 +2449,15 @@ python do_package () {
 
     bb.build.exec_func("package_convert_pr_autoinc", d)
 
+    # Check for conflict between renamed packages and existing ones
+    # for each package in PACKAGES, check if it will be renamed to an existing one
+    for p in packages:
+        localdata = bb.data.createCopy(d)
+        localdata.setVar('OVERRIDES', p)
+        rename = localdata.getVar('PKG')
+        if (rename != None) and rename in packages:
+            bb.fatal('package "%s" is renamed to "%s" using PKG:%s, but package name already exists'%(p,rename,p))
+
     ###########################################################################
     # Optimisations
     ###########################################################################
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 11/26] oeqa/selftest/cases/package.py: adding unittest for package rename conflicts
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (9 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 10/26] package.bbclass: check packages name conflict in do_package Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 12/26] Revert "runqemu: Add workaround for APIC hang on pre 4.15 kernels on qemux86" Steve Sakoman
                   ` (14 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Fawzi KHABER <fawzi.khaber@smile.fr>

This Unittest tries to rename a package, using an already used name and
fails on do_package.

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
Signed-off-by: Fawzi KHABER <fawzi.khaber@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c3fe173d5196506d89aa464ba56aabcf581a60db)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 .../packagenameconflict/packagenameconflict.bb         | 10 ++++++++++
 meta/lib/oeqa/selftest/cases/package.py                |  7 +++++++
 2 files changed, 17 insertions(+)
 create mode 100644 meta-selftest/recipes-test/packagenameconflict/packagenameconflict.bb

diff --git a/meta-selftest/recipes-test/packagenameconflict/packagenameconflict.bb b/meta-selftest/recipes-test/packagenameconflict/packagenameconflict.bb
new file mode 100644
index 0000000000..5d19a4dd25
--- /dev/null
+++ b/meta-selftest/recipes-test/packagenameconflict/packagenameconflict.bb
@@ -0,0 +1,10 @@
+SUMMARY = "Test case that tries to rename a package to an existing one and fails"
+DESCRIPTION = "This generates a packaging error when a package is renamed to a pre-existing name"
+LICENSE = "MIT"
+
+# Add a new package ${PN}-renametest
+PACKAGES += "${PN}-renametest"
+# ... and try to rename the ${PN}-dev to the new ${PN}-renametest (conflict)
+PKG:${PN}-dev = "${PN}-renametest"
+
+EXCLUDE_FROM_WORLD = "1"
diff --git a/meta/lib/oeqa/selftest/cases/package.py b/meta/lib/oeqa/selftest/cases/package.py
index 2d1b48a15d..cc09a1442d 100644
--- a/meta/lib/oeqa/selftest/cases/package.py
+++ b/meta/lib/oeqa/selftest/cases/package.py
@@ -89,6 +89,13 @@ class VersionOrdering(OESelftestTestCase):
             self.assertEqual(status - 100, sort, "%s %s (%d) failed" % (ver1, ver2, sort))
 
 class PackageTests(OESelftestTestCase):
+    # Verify that a recipe cannot rename a package into an existing one
+    def test_package_name_conflict(self):
+        res = bitbake("packagenameconflict", ignore_status=True)
+        self.assertNotEqual(res.status, 0)
+        err = "package name already exists"
+        self.assertTrue(err in res.output)
+
     # Verify that a recipe which sets up hardlink files has those preserved into split packages
     # Also test file sparseness is preserved
     def test_preserve_sparse_hardlinks(self):
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 12/26] Revert "runqemu: Add workaround for APIC hang on pre 4.15 kernels on qemux86"
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (10 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 11/26] oeqa/selftest/cases/package.py: adding unittest for package rename conflicts Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 13/26] lib/oe/gpg_sign.py: Avoid race when creating .sig files in detach_sign Steve Sakoman
                   ` (13 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Khem Raj <raj.khem@gmail.com>

This reverts commit 82e67b82ea8e12aa0b7b9db1d84fec0436dec71b.

It was commited as part of https://bugzilla.yoctoproject.org/show_bug.cgi?id=12301
for kernels < 4.15, as of now oldest builder kernel we have is 4.15 on
ubuntu 18.04 so we should not require this workaround. Moreover, this
fixes an smp problem with qemux86 where no matter what -smp <x> option
is used, qemu always starts with single core.

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit fa8a7c0608fc800c48d0ff1cd832ad63c51eeab1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 scripts/runqemu | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/scripts/runqemu b/scripts/runqemu
index db35d83fa9..efc135e535 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -591,11 +591,6 @@ class BaseConfig(object):
 
         if os.access(dev_kvm, os.W_OK|os.R_OK):
             self.qemu_opt_script += ' -enable-kvm'
-            if self.get('MACHINE') == "qemux86":
-                # Workaround for broken APIC window on pre 4.15 host kernels which causes boot hangs
-                # See YOCTO #12301
-                # On 64 bit we use x2apic
-                self.kernel_cmdline_script += " clocksource=kvm-clock hpet=disable noapic nolapic"
         else:
             logger.error("You have no read or write permission on /dev/kvm.")
             logger.error("Please change the ownership of this file as described at:")
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 13/26] lib/oe/gpg_sign.py: Avoid race when creating .sig files in detach_sign
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (11 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 12/26] Revert "runqemu: Add workaround for APIC hang on pre 4.15 kernels on qemux86" Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 14/26] oeqa/selftest/cases/runqemu: update imports Steve Sakoman
                   ` (12 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Tobias Hagelborn <tobias.hagelborn@axis.com>

Move the signature file into place only after it is successfully signed.
This to avoid race and corrupted .sig files in cases multiple onging
builds write to a shared sstate-cache dir.

Signed-off-by: Tobias Hagelborn <tobiasha@axis.com>
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b4ec08ea9efebac262d43f47d95a356fe2829de9)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oe/gpg_sign.py | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 613dab8561..ede6186c84 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -5,11 +5,12 @@
 #
 
 """Helper module for GPG signing"""
-import os
 
 import bb
-import subprocess
+import os
 import shlex
+import subprocess
+import tempfile
 
 class LocalSigner(object):
     """Class for handling local (on the build host) signing"""
@@ -73,8 +74,6 @@ class LocalSigner(object):
             cmd += ['--homedir', self.gpg_path]
         if armor:
             cmd += ['--armor']
-        if output_suffix:
-            cmd += ['-o', input_file + "." + output_suffix]
         if use_sha256:
             cmd += ['--digest-algo', "SHA256"]
 
@@ -83,19 +82,27 @@ class LocalSigner(object):
         if self.gpg_version > (2,1,):
             cmd += ['--pinentry-mode', 'loopback']
 
-        cmd += [input_file]
-
         try:
             if passphrase_file:
                 with open(passphrase_file) as fobj:
                     passphrase = fobj.readline();
 
-            job = subprocess.Popen(cmd, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
-            (_, stderr) = job.communicate(passphrase.encode("utf-8"))
+            if not output_suffix:
+                output_suffix = 'asc' if armor else 'sig'
+            output_file = input_file + "." + output_suffix
+            with tempfile.TemporaryDirectory(dir=os.path.dirname(output_file)) as tmp_dir:
+                tmp_file = os.path.join(tmp_dir, os.path.basename(output_file))
+                cmd += ['-o', tmp_file]
+
+                cmd += [input_file]
+
+                job = subprocess.Popen(cmd, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+                (_, stderr) = job.communicate(passphrase.encode("utf-8"))
 
-            if job.returncode:
-                bb.fatal("GPG exited with code %d: %s" % (job.returncode, stderr.decode("utf-8")))
+                if job.returncode:
+                    bb.fatal("GPG exited with code %d: %s" % (job.returncode, stderr.decode("utf-8")))
 
+                os.rename(tmp_file, output_file)
         except IOError as e:
             bb.error("IO error (%s): %s" % (e.errno, e.strerror))
             raise Exception("Failed to sign '%s'" % input_file)
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 14/26] oeqa/selftest/cases/runqemu: update imports
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (12 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 13/26] lib/oe/gpg_sign.py: Avoid race when creating .sig files in detach_sign Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 15/26] oeqa/targetcontrol: fix misspelled RuntimeError Steve Sakoman
                   ` (11 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Enrico Jörns <ejo@pengutronix.de>

'tempfile' and 'oeqa.utils.command.runCmd' are unused while 'os' is
actually used.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 50f766233e83528eebeca92877277374c7193530)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/selftest/cases/runqemu.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py b/meta/lib/oeqa/selftest/cases/runqemu.py
index c1d277a095..72ca77b845 100644
--- a/meta/lib/oeqa/selftest/cases/runqemu.py
+++ b/meta/lib/oeqa/selftest/cases/runqemu.py
@@ -4,13 +4,13 @@
 # SPDX-License-Identifier: MIT
 #
 
+import os
 import re
-import tempfile
 import time
 import oe.types
 from oeqa.core.decorator import OETestTag
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, runqemu, get_bb_var, runCmd
+from oeqa.utils.commands import bitbake, runqemu, get_bb_var
 
 @OETestTag("runqemu")
 class RunqemuTests(OESelftestTestCase):
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 15/26] oeqa/targetcontrol: fix misspelled RuntimeError
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (13 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 14/26] oeqa/selftest/cases/runqemu: update imports Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 16/26] oeqa/targetcontrol: do not set dump_host_cmds redundantly Steve Sakoman
                   ` (10 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Enrico Jörns <ejo@pengutronix.de>

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 92ffc35052768c753a89b4839c70db87072437a2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/targetcontrol.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index 1fdff82889..b8d4ea8a70 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -205,7 +205,7 @@ class QemuTarget(BaseTarget):
             self.server_ip = self.runner.server_ip
             self.connection = SSHControl(ip=self.ip, logfile=self.sshlog)
         else:
-            raise RuntimError("%s - FAILED to re-start qemu - check the task log and the boot log" % self.pn)
+            raise RuntimeError("%s - FAILED to re-start qemu - check the task log and the boot log" % self.pn)
 
     def run_serial(self, command, timeout=60):
         return self.runner.run_serial(command, timeout=timeout)
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 16/26] oeqa/targetcontrol: do not set dump_host_cmds redundantly
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (14 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 15/26] oeqa/targetcontrol: fix misspelled RuntimeError Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 17/26] oeqa/targetcontrol: remove unused imports Steve Sakoman
                   ` (9 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Enrico Jörns <ejo@pengutronix.de>

This is already set above, thus simply use the set value from there.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2f6571ffb23b1a0b5bd6a1212fc108e536f04539)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/targetcontrol.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index b8d4ea8a70..bc6b91e994 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -145,7 +145,7 @@ class QemuTarget(BaseTarget):
                             boottime = int(d.getVar("TEST_QEMUBOOT_TIMEOUT")),
                             use_kvm = use_kvm,
                             dump_dir = dump_dir,
-                            dump_host_cmds = d.getVar("testimage_dump_host"),
+                            dump_host_cmds = dump_host_cmds,
                             logger = logger,
                             tmpfsdir = d.getVar("RUNQEMU_TMPFS_DIR"),
                             serial_ports = len(d.getVar("SERIAL_CONSOLES").split()))
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 17/26] oeqa/targetcontrol: remove unused imports
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (15 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 16/26] oeqa/targetcontrol: do not set dump_host_cmds redundantly Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 18/26] oeqa/utils/commands: fix usage of undefined EPIPE Steve Sakoman
                   ` (8 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Enrico Jörns <ejo@pengutronix.de>

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 46b42edbf814d15be3a18283018ede2d4f817bc6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/targetcontrol.py | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index bc6b91e994..f5f2ce2f12 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -7,18 +7,14 @@
 # This module is used by testimage.bbclass for setting up and controlling a target machine.
 
 import os
-import shutil
 import subprocess
 import bb
-import traceback
-import sys
 import logging
 from oeqa.utils.sshcontrol import SSHControl
 from oeqa.utils.qemurunner import QemuRunner
 from oeqa.utils.qemutinyrunner import QemuTinyRunner
 from oeqa.utils.dump import TargetDumper
 from oeqa.utils.dump import MonitorDumper
-from oeqa.controllers.testtargetloader import TestTargetLoader
 from abc import ABCMeta, abstractmethod
 
 class BaseTarget(object, metaclass=ABCMeta):
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 18/26] oeqa/utils/commands: fix usage of undefined EPIPE
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (16 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 17/26] oeqa/targetcontrol: remove unused imports Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 19/26] oeqa/utils/commands: remove unused imports Steve Sakoman
                   ` (7 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Enrico Jörns <ejo@pengutronix.de>

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1cb17e37e275794e26debed2d171c2394d8bb945)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/utils/commands.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index f733fcdf3c..c03ba52ef2 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -21,6 +21,7 @@ from oeqa.utils import CommandError
 from oeqa.utils import ftools
 import re
 import contextlib
+import errno
 # Export test doesn't require bb
 try:
     import bb
@@ -85,7 +86,7 @@ class Command(object):
             except OSError as ex:
                 # It's not an error when the command does not consume all
                 # of our data. subprocess.communicate() also ignores that.
-                if ex.errno != EPIPE:
+                if ex.errno != errno.EPIPE:
                     raise
 
         # We write in a separate thread because then we can read
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 19/26] oeqa/utils/commands: remove unused imports
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (17 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 18/26] oeqa/utils/commands: fix usage of undefined EPIPE Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 20/26] oeqa/utils/qemurunner: replace hard-coded user 'root' in debug output Steve Sakoman
                   ` (6 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Enrico Jörns <ejo@pengutronix.de>

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0c00b5cdd57c3d9c47d4780d6627c74221911b72)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/utils/commands.py | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index c03ba52ef2..473aa38d41 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -8,11 +8,8 @@
 # This module is mainly used by scripts/oe-selftest and modules under meta/oeqa/selftest
 # It provides a class and methods for running commands on the host in a convienent way for tests.
 
-
-
 import os
 import sys
-import signal
 import subprocess
 import threading
 import time
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 20/26] oeqa/utils/qemurunner: replace hard-coded user 'root' in debug output
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (18 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 19/26] oeqa/utils/commands: remove unused imports Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 21/26] selftest: runqemu: better check for ROOTFS: in the log Steve Sakoman
                   ` (5 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Enrico Jörns <ejo@pengutronix.de>

Since login user is configurable with boot_patterns['send_login_user'],
unconditionally using 'root' in the debug message can be confusing.

Also fix the debug message to say 'Logged in' instead of 'Logged'.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 900e3d42b918b5a33d8d952b3a8078fbe72ba98f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/utils/qemurunner.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 1aaba02bd8..a455b3b389 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -511,7 +511,7 @@ class QemuRunner:
             (status, output) = self.run_serial(self.boot_patterns['send_login_user'], raw=True, timeout=120)
             if re.search(self.boot_patterns['search_login_succeeded'], output):
                 self.logged = True
-                self.logger.debug("Logged as root in serial console")
+                self.logger.debug("Logged in as %s in serial console" % self.boot_patterns['send_login_user'].replace("\n", ""))
                 if netconf:
                     # configure guest networking
                     cmd = "ifconfig eth0 %s netmask %s up\n" % (self.ip, self.netmask)
@@ -522,7 +522,7 @@ class QemuRunner:
                         self.logger.debug("Couldn't configure guest networking")
             else:
                 self.logger.warning("Couldn't login into serial console"
-                            " as root using blank password")
+                            " as %s using blank password" % self.boot_patterns['send_login_user'].replace("\n", ""))
                 self.logger.warning("The output:\n%s" % output)
         except:
             self.logger.warning("Serial console failed while trying to login")
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 21/26] selftest: runqemu: better check for ROOTFS: in the log
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (19 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 20/26] oeqa/utils/qemurunner: replace hard-coded user 'root' in debug output Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 22/26] selftest: runqemu: use better error message when asserts fail Steve Sakoman
                   ` (4 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Martin Jansa <Martin.Jansa@gmail.com>

* it was searching for line like this:
  ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch1/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64-20230320081121.rootfs.ext4]
  but with IMAGE_NAME_SUFFIX changed to something else than default ".rootfs"
  or with my pending changes the line looks like this:
  ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch2/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-20230320085744.ext4]
  and test was failing.

* Check for whole line starting with ROOTFS: and ending just with .ext4

[YOCTO #12937]

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7a0f622f23aff2c4eeca0606e7682931eb53287a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/selftest/cases/runqemu.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py b/meta/lib/oeqa/selftest/cases/runqemu.py
index 72ca77b845..e72dc1036e 100644
--- a/meta/lib/oeqa/selftest/cases/runqemu.py
+++ b/meta/lib/oeqa/selftest/cases/runqemu.py
@@ -57,7 +57,8 @@ SYSLINUX_TIMEOUT = "10"
         cmd = "%s %s ext4" % (self.cmd_common, self.machine)
         with runqemu(self.recipe, ssh=False, launch_cmd=cmd) as qemu:
             with open(qemu.qemurunnerlog) as f:
-                self.assertIn('rootfs.ext4', f.read(), "Failed: %s" % cmd)
+                regexp = r'\nROOTFS: .*\.ext4]\n'
+                self.assertRegex(f.read(), regexp, "Failed to find '%s' in '%s' after running '%s'" % (regexp, qemu.qemurunnerlog, cmd))
 
     def test_boot_machine_iso(self):
         """Test runqemu machine iso"""
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 22/26] selftest: runqemu: use better error message when asserts fail
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (20 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 21/26] selftest: runqemu: better check for ROOTFS: in the log Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 23/26] oeqa/runtime: clean up deprecated backslash expansion Steve Sakoman
                   ` (3 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Martin Jansa <Martin.Jansa@gmail.com>

* It was showing whole log and that the runqemu command failed, but not
  where the log file is, nor why it thinks the runqemu failed

[YOCTO #12937]

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ad4b72e6524114a3bdee10cab71f194ea143cd24)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/selftest/cases/runqemu.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py b/meta/lib/oeqa/selftest/cases/runqemu.py
index e72dc1036e..d3eeee3b41 100644
--- a/meta/lib/oeqa/selftest/cases/runqemu.py
+++ b/meta/lib/oeqa/selftest/cases/runqemu.py
@@ -65,7 +65,8 @@ SYSLINUX_TIMEOUT = "10"
         cmd = "%s %s iso" % (self.cmd_common, self.machine)
         with runqemu(self.recipe, ssh=False, launch_cmd=cmd) as qemu:
             with open(qemu.qemurunnerlog) as f:
-                self.assertIn('media=cdrom', f.read(), "Failed: %s" % cmd)
+                text_in = 'media=cdrom'
+                self.assertIn(text_in, f.read(), "Failed to find '%s' in '%s' after running '%s'" % (text_in, qemu.qemurunnerlog, cmd))
 
     def test_boot_recipe_image(self):
         """Test runqemu recipe-image"""
@@ -80,14 +81,16 @@ SYSLINUX_TIMEOUT = "10"
         cmd = "%s %s wic.vmdk" % (self.cmd_common, self.recipe)
         with runqemu(self.recipe, ssh=False, launch_cmd=cmd) as qemu:
             with open(qemu.qemurunnerlog) as f:
-                self.assertIn('format=vmdk', f.read(), "Failed: %s" % cmd)
+                text_in = 'format=vmdk'
+                self.assertIn(text_in, f.read(), "Failed to find '%s' in '%s' after running '%s'" % (text_in, qemu.qemurunnerlog, cmd))
 
     def test_boot_recipe_image_vdi(self):
         """Test runqemu recipe-image vdi"""
         cmd = "%s %s wic.vdi" % (self.cmd_common, self.recipe)
         with runqemu(self.recipe, ssh=False, launch_cmd=cmd) as qemu:
             with open(qemu.qemurunnerlog) as f:
-                self.assertIn('format=vdi', f.read(), "Failed: %s" % cmd)
+                text_in = 'format=vdi'
+                self.assertIn(text_in, f.read(), "Failed to find '%s' in '%s' after running '%s'" % (text_in, qemu.qemurunnerlog, cmd))
 
     def test_boot_deploy(self):
         """Test runqemu deploy_dir_image"""
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 23/26] oeqa/runtime: clean up deprecated backslash expansion
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (21 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 22/26] selftest: runqemu: use better error message when asserts fail Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 24/26] shadow: Fix can not print full login timeout message Steve Sakoman
                   ` (2 subsequent siblings)
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Ross Burton <ross.burton@arm.com>

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 77085a05240c3f8226b9f2199c977f2555807789)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/oeqa/runtime/cases/apt.py       | 4 ++--
 meta/lib/oeqa/runtime/cases/buildcpio.py | 7 +++++--
 meta/lib/oeqa/runtime/cases/systemd.py   | 2 +-
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
index 4e09374add..8000645843 100644
--- a/meta/lib/oeqa/runtime/cases/apt.py
+++ b/meta/lib/oeqa/runtime/cases/apt.py
@@ -39,9 +39,9 @@ class AptRepoTest(AptTest):
         self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
 
     def setup_source_config_for_package_install_signed(self):
-        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
+        apt_get_source_server = 'http://%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
         apt_get_sourceslist_dir = '/etc/apt/'
-        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
+        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's|\[trusted=yes\] http://bogus_ip:bogus_port|%s|g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
 
     def cleanup_source_config_for_package_install(self):
         apt_get_sourceslist_dir = '/etc/apt/'
diff --git a/meta/lib/oeqa/runtime/cases/buildcpio.py b/meta/lib/oeqa/runtime/cases/buildcpio.py
index bd3b46d9ef..3728855d24 100644
--- a/meta/lib/oeqa/runtime/cases/buildcpio.py
+++ b/meta/lib/oeqa/runtime/cases/buildcpio.py
@@ -29,7 +29,10 @@ class BuildCpioTest(OERuntimeTestCase):
     @OEHasPackage(['autoconf'])
     def test_cpio(self):
         self.project.download_archive()
-        self.project.run_configure('--disable-maintainer-mode',
-                                   'sed -i -e "/char \*program_name/d" src/global.c;')
+        self.project.run_configure('--disable-maintainer-mode')
+        # This sed is needed until
+        # https://git.savannah.gnu.org/cgit/cpio.git/commit/src/global.c?id=641d3f489cf6238bb916368d4ba0d9325a235afb
+        # is in a release.
+        self.project._run(r'sed -i -e "/char \*program_name/d" %s/src/global.c' % self.project.targetdir)
         self.project.run_make()
         self.project.run_install()
diff --git a/meta/lib/oeqa/runtime/cases/systemd.py b/meta/lib/oeqa/runtime/cases/systemd.py
index 720b4b517a..37f295492d 100644
--- a/meta/lib/oeqa/runtime/cases/systemd.py
+++ b/meta/lib/oeqa/runtime/cases/systemd.py
@@ -154,7 +154,7 @@ class SystemdJournalTests(SystemdTest):
         """
 
         # The expression chain that uniquely identifies the time boot message.
-        expr_items=['Startup finished', 'kernel', 'userspace','\.$']
+        expr_items=['Startup finished', 'kernel', 'userspace', r'\.$']
         try:
             output = self.journalctl(args='-o cat --reverse')
         except AssertionError:
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 24/26] shadow: Fix can not print full login timeout message
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (22 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 23/26] oeqa/runtime: clean up deprecated backslash expansion Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 25/26] runqemu: respect IMAGE_LINK_NAME Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 26/26] report-error: catch Nothing PROVIDES error Steve Sakoman
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Soumya <soumya.sambu@windriver.com>

When we do not enter password during login, it is expected to display message "Login timed out after 60 seconds". But it prints only first few bytes(like "Login t") when write is immediately followed by exit.
Fix - Calling exit from new handler provides enough time to display full message.

Upstream-Status: Accepted [https://github.com/shadow-maint/shadow/commit/670cae834827a8f794e6f7464fa57790d911b63c]

Signed-off-by: Soumya <soumya.sambu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 644cfe9dcf351bfa6c67f4b4d1e7dec416a59021)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 .../0001-Fix-can-not-print-full-login.patch   | 41 +++++++++++++++++++
 meta/recipes-extended/shadow/shadow.inc       |  1 +
 2 files changed, 42 insertions(+)
 create mode 100644 meta/recipes-extended/shadow/files/0001-Fix-can-not-print-full-login.patch

diff --git a/meta/recipes-extended/shadow/files/0001-Fix-can-not-print-full-login.patch b/meta/recipes-extended/shadow/files/0001-Fix-can-not-print-full-login.patch
new file mode 100644
index 0000000000..37ba5f3dc2
--- /dev/null
+++ b/meta/recipes-extended/shadow/files/0001-Fix-can-not-print-full-login.patch
@@ -0,0 +1,41 @@
+commit 670cae834827a8f794e6f7464fa57790d911b63c
+Author: SoumyaWind <121475834+SoumyaWind@users.noreply.github.com>
+Date:   Tue Dec 27 17:40:17 2022 +0530
+
+    shadow: Fix can not print full login timeout message
+    
+    Login timed out message prints only first few bytes when write is immediately followed by exit.
+    Calling exit from new handler provides enough time to display full message.
+
+Upstream-Status: Accepted [https://github.com/shadow-maint/shadow/commit/670cae834827a8f794e6f7464fa57790d911b63c]
+
+diff --git a/src/login.c b/src/login.c
+index 116e2cb3..c55f4de0 100644
+--- a/src/login.c
++++ b/src/login.c
+@@ -120,6 +120,7 @@ static void get_pam_user (char **ptr_pam_user);
+ 
+ static void init_env (void);
+ static void alarm_handler (int);
++static void exit_handler (int);
+ 
+ /*
+  * usage - print login command usage and exit
+@@ -391,11 +392,16 @@ static void init_env (void)
+ #endif				/* !USE_PAM */
+ }
+ 
++static void exit_handler (unused int sig)
++{
++	_exit (0);
++}
+ 
+ static void alarm_handler (unused int sig)
+ {
+ 	write (STDERR_FILENO, tmsg, strlen (tmsg));
+-	_exit (0);
++	signal(SIGALRM, exit_handler);
++	alarm(2);
+ }
+ 
+ #ifdef USE_PAM
diff --git a/meta/recipes-extended/shadow/shadow.inc b/meta/recipes-extended/shadow/shadow.inc
index 5656a375b7..0ed220ac5c 100644
--- a/meta/recipes-extended/shadow/shadow.inc
+++ b/meta/recipes-extended/shadow/shadow.inc
@@ -15,6 +15,7 @@ SRC_URI = "${GITHUB_BASE_URI}/download/${PV}/${BP}.tar.gz \
            file://0001-shadow-use-relaxed-usernames.patch \
            ${@bb.utils.contains('PACKAGECONFIG', 'pam', '${PAM_SRC_URI}', '', d)} \
            file://useradd \
+	   file://0001-Fix-can-not-print-full-login.patch \
            "
 
 SRC_URI:append:class-target = " \
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 25/26] runqemu: respect IMAGE_LINK_NAME
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (23 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 24/26] shadow: Fix can not print full login timeout message Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  2023-04-06 16:01 ` [OE-core][langdale 26/26] report-error: catch Nothing PROVIDES error Steve Sakoman
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Martin Jansa <Martin.Jansa@gmail.com>

* when searching for qemuboot.conf
* don't assume that IMAGE_LINK_NAME is always
  <rootfs>-<machine> (with <rootfs>-<machine>.qemuboot.conf)

* runqemu: use IMAGE_LINK_NAME set by testimage.bbclass or query with bitbake -e

* testimage.bbclass was setting DEPLOY_DIR which I don't see used
  anywhere else, so I assume it was supposed to be DEPLOY_DIR_IMAGE as mentioned
  in corresponding runqemu code, do the same with IMAGE_LINK_NAME variable

* add virtual/kernel as bitbake -e target in run_bitbake_env to make
  sure IMAGE_LINK_NAME is defined (kernel-artifact-names.bbclass inherits
  image-artifact-names.bbclass as well)

* improve .qemuboot.conf search
  1st search for file matching the rootfs and only when not found
  try again with .rootfs suffix removed

[YOCTO #12937]

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 716eb55bb963db7b02d985849cb025898aabc855)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/classes-recipe/testimage.bbclass |  2 +-
 scripts/runqemu                       | 69 ++++++++++++++++++++-------
 2 files changed, 52 insertions(+), 19 deletions(-)

diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass
index df22bb2344..b48cd96575 100644
--- a/meta/classes-recipe/testimage.bbclass
+++ b/meta/classes-recipe/testimage.bbclass
@@ -98,7 +98,7 @@ TESTIMAGELOCK:qemuall = ""
 
 TESTIMAGE_DUMP_DIR ?= "${LOG_DIR}/runtime-hostdump/"
 
-TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR"
+TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR_IMAGE IMAGE_LINK_NAME"
 
 testimage_dump_target () {
     top -bn1
diff --git a/scripts/runqemu b/scripts/runqemu
index efc135e535..9a3c9d2ce4 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -381,13 +381,19 @@ class BaseConfig(object):
                     fst =  m.group(1)
             if fst:
                 self.check_arg_fstype(fst)
-                qb = re.sub('\.' + fst + "$", '', self.rootfs)
-                qb = '%s%s' % (re.sub('\.rootfs$', '', qb), '.qemuboot.conf')
+                qb = re.sub('\.' + fst + "$", '.qemuboot.conf', self.rootfs)
                 if os.path.exists(qb):
                     self.qemuboot = qb
                     self.qbconfload = True
                 else:
-                    logger.warning("%s doesn't exist" % qb)
+                    logger.warning("%s doesn't exist, will try to remove '.rootfs' from filename" % qb)
+                    # They to remove .rootfs (IMAGE_NAME_SUFFIX) as well
+                    qb = re.sub('\.rootfs.qemuboot.conf$', '.qemuboot.conf', qb)
+                    if os.path.exists(qb):
+                        self.qemuboot = qb
+                        self.qbconfload = True
+                    else:
+                        logger.warning("%s doesn't exist" % qb)
             else:
                 raise RunQemuError("Can't find FSTYPE from: %s" % p)
 
@@ -421,6 +427,7 @@ class BaseConfig(object):
         # are there other scenarios in which we need to support being
         # invoked by bitbake?
         deploy = self.get('DEPLOY_DIR_IMAGE')
+        image_link_name = self.get('IMAGE_LINK_NAME')
         bbchild = deploy and self.get('OE_TMPDIR')
         if bbchild:
             self.set_machine_deploy_dir(arg, deploy)
@@ -445,6 +452,12 @@ class BaseConfig(object):
         else:
             logger.error("%s not a directory valid DEPLOY_DIR_IMAGE" % deploy_dir_image)
             self.set("MACHINE", arg)
+        if not image_link_name:
+            s = re.search('^IMAGE_LINK_NAME="(.*)"', self.bitbake_e, re.M)
+            if s:
+                image_link_name = s.group(1)
+                self.set("IMAGE_LINK_NAME", image_link_name)
+                logger.debug('Using IMAGE_LINK_NAME = "%s"' % image_link_name)
 
     def set_dri_path(self):
         # As runqemu can be run within bitbake (when using testimage, for example),
@@ -557,11 +570,18 @@ class BaseConfig(object):
             self.check_arg_machine(unknown_arg)
 
         if not (self.get('DEPLOY_DIR_IMAGE') or self.qbconfload):
-            self.load_bitbake_env()
+            self.load_bitbake_env(target=self.rootfs)
             s = re.search('^DEPLOY_DIR_IMAGE="(.*)"', self.bitbake_e, re.M)
             if s:
                 self.set("DEPLOY_DIR_IMAGE", s.group(1))
 
+        if not self.get('IMAGE_LINK_NAME') and self.rootfs:
+            s = re.search('^IMAGE_LINK_NAME="(.*)"', self.bitbake_e, re.M)
+            if s:
+                image_link_name = s.group(1)
+                self.set("IMAGE_LINK_NAME", image_link_name)
+                logger.debug('Using IMAGE_LINK_NAME = "%s"' % image_link_name)
+
     def check_kvm(self):
         """Check kvm and kvm-host"""
         if not (self.kvm_enabled or self.vhost_enabled):
@@ -667,8 +687,8 @@ class BaseConfig(object):
 
         if self.rootfs and not os.path.exists(self.rootfs):
             # Lazy rootfs
-            self.rootfs = "%s/%s-%s.%s" % (self.get('DEPLOY_DIR_IMAGE'),
-                    self.rootfs, self.get('MACHINE'),
+            self.rootfs = "%s/%s.%s" % (self.get('DEPLOY_DIR_IMAGE'),
+                    self.get('IMAGE_LINK_NAME'),
                     self.fstype)
         elif not self.rootfs:
             cmd_name = '%s/%s*.%s' % (self.get('DEPLOY_DIR_IMAGE'), self.get('IMAGE_NAME'), self.fstype)
@@ -872,8 +892,10 @@ class BaseConfig(object):
                 machine = self.get('MACHINE')
                 if not machine:
                     machine = os.path.basename(deploy_dir_image)
-                self.qemuboot = "%s/%s-%s.qemuboot.conf" % (deploy_dir_image,
-                        self.rootfs, machine)
+                if not self.get('IMAGE_LINK_NAME'):
+                    raise RunQemuError("IMAGE_LINK_NAME wasn't set to find corresponding .qemuboot.conf file")
+                self.qemuboot = "%s/%s.qemuboot.conf" % (deploy_dir_image,
+                        self.get('IMAGE_LINK_NAME'))
             else:
                 cmd = 'ls -t %s/*.qemuboot.conf' %  deploy_dir_image
                 logger.debug('Running %s...' % cmd)
@@ -1569,7 +1591,7 @@ class BaseConfig(object):
 
         self.cleaned = True
 
-    def run_bitbake_env(self, mach=None):
+    def run_bitbake_env(self, mach=None, target=''):
         bitbake = shutil.which('bitbake')
         if not bitbake:
             return
@@ -1582,22 +1604,33 @@ class BaseConfig(object):
             multiconfig = "mc:%s" % multiconfig
 
         if mach:
-            cmd = 'MACHINE=%s bitbake -e %s' % (mach, multiconfig)
+            cmd = 'MACHINE=%s bitbake -e %s %s' % (mach, multiconfig, target)
         else:
-            cmd = 'bitbake -e %s' % multiconfig
+            cmd = 'bitbake -e %s %s' % (multiconfig, target)
 
         logger.info('Running %s...' % cmd)
-        return subprocess.check_output(cmd, shell=True).decode('utf-8')
+        try:
+            return subprocess.check_output(cmd, shell=True).decode('utf-8')
+        except subprocess.CalledProcessError as err:
+            logger.warning("Couldn't run '%s' to gather environment information, maybe the target wasn't an image name, will retry with virtual/kernel as a target:\n%s" % (cmd, err.output.decode('utf-8')))
+            # need something with IMAGE_NAME_SUFFIX/IMAGE_LINK_NAME defined (kernel also inherits image-artifact-names.bbclass)
+            target = 'virtual/kernel'
+            if mach:
+                cmd = 'MACHINE=%s bitbake -e %s %s' % (mach, multiconfig, target)
+            else:
+                cmd = 'bitbake -e %s %s' % (multiconfig, target)
+            try:
+                return subprocess.check_output(cmd, shell=True).decode('utf-8')
+            except subprocess.CalledProcessError as err:
+                logger.warning("Couldn't run '%s' to gather environment information, giving up with 'bitbake -e':\n%s" % (cmd, err.output.decode('utf-8')))
+                return ''
 
-    def load_bitbake_env(self, mach=None):
+
+    def load_bitbake_env(self, mach=None, target=None):
         if self.bitbake_e:
             return
 
-        try:
-            self.bitbake_e = self.run_bitbake_env(mach=mach)
-        except subprocess.CalledProcessError as err:
-            self.bitbake_e = ''
-            logger.warning("Couldn't run 'bitbake -e' to gather environment information:\n%s" % err.output.decode('utf-8'))
+        self.bitbake_e = self.run_bitbake_env(mach=mach, target=target)
 
     def validate_combos(self):
         if (self.fstype in self.vmtypes) and self.kernel:
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [OE-core][langdale 26/26] report-error: catch Nothing PROVIDES error
  2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
                   ` (24 preceding siblings ...)
  2023-04-06 16:01 ` [OE-core][langdale 25/26] runqemu: respect IMAGE_LINK_NAME Steve Sakoman
@ 2023-04-06 16:01 ` Steve Sakoman
  25 siblings, 0 replies; 27+ messages in thread
From: Steve Sakoman @ 2023-04-06 16:01 UTC (permalink / raw)
  To: openembedded-core

From: Mingli Yu <mingli.yu@windriver.com>

Make the report-error catch Nothing PROVIDES error and then
we can check it directly via error report web.

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a57d8f82b83554c821a83eacc02f9c73b263ff02)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/classes/report-error.bbclass | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/meta/classes/report-error.bbclass b/meta/classes/report-error.bbclass
index 2f692fbbcc..2b2ad56514 100644
--- a/meta/classes/report-error.bbclass
+++ b/meta/classes/report-error.bbclass
@@ -107,6 +107,31 @@ python errorreport_handler () {
             errorreport_savedata(e, jsondata, "error-report.txt")
             bb.utils.unlockfile(lock)
 
+        elif isinstance(e, bb.event.NoProvider):
+            bb.utils.mkdirhier(logpath)
+            data = {}
+            machine = e.data.getVar("MACHINE")
+            data['machine'] = machine
+            data['build_sys'] = e.data.getVar("BUILD_SYS")
+            data['nativelsb'] = nativelsb()
+            data['distro'] = e.data.getVar("DISTRO")
+            data['target_sys'] = e.data.getVar("TARGET_SYS")
+            data['failures'] = []
+            data['component'] = str(e._item)
+            data['branch_commit'] = str(oe.buildcfg.detect_branch(e.data)) + ": " + str(oe.buildcfg.detect_revision(e.data))
+            data['bitbake_version'] = e.data.getVar("BB_VERSION")
+            data['layer_version'] = get_layers_branch_rev(e.data)
+            data['local_conf'] = get_conf_data(e, 'local.conf')
+            data['auto_conf'] = get_conf_data(e, 'auto.conf')
+            taskdata={}
+            taskdata['log'] = str(e)
+            taskdata['package'] = str(e._item)
+            taskdata['task'] = "Nothing provides " + "'" + str(e._item) + "'"
+            data['failures'].append(taskdata)
+            lock = bb.utils.lockfile(datafile + '.lock')
+            errorreport_savedata(e, data, "error-report.txt")
+            bb.utils.unlockfile(lock)
+
         elif isinstance(e, bb.event.BuildCompleted):
             lock = bb.utils.lockfile(datafile + '.lock')
             jsondata = json.loads(errorreport_getdata(e))
@@ -120,4 +145,4 @@ python errorreport_handler () {
 }
 
 addhandler errorreport_handler
-errorreport_handler[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted bb.build.TaskFailed"
+errorreport_handler[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted bb.build.TaskFailed bb.event.NoProvider"
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2023-04-06 16:03 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-04-06 16:01 [OE-core][langdale 00/26] Patch review Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 01/26] xdg-utils: Add a patch for CVE-2020-27748 Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 02/26] xdg-utils: Fix CVE number Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 03/26] tzdata: upgrade to 2023c Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 04/26] uninative: Upgrade to 3.9 to include glibc 2.37 Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 05/26] filemap.py: enforce maximum of 4kb block size Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 06/26] oeqs/selftest: OESelftestTestContext: replace the os.environ after subprocess.check_output Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 07/26] oeqa/selftest: OESelftestTestContext: convert relative to full path when newbuilddir is provided Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 08/26] oeqa/selftest/reproducible: Split different packages from missing packages output Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 09/26] run-postinsts: Set dependency for ldconfig to avoid boot issues Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 10/26] package.bbclass: check packages name conflict in do_package Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 11/26] oeqa/selftest/cases/package.py: adding unittest for package rename conflicts Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 12/26] Revert "runqemu: Add workaround for APIC hang on pre 4.15 kernels on qemux86" Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 13/26] lib/oe/gpg_sign.py: Avoid race when creating .sig files in detach_sign Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 14/26] oeqa/selftest/cases/runqemu: update imports Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 15/26] oeqa/targetcontrol: fix misspelled RuntimeError Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 16/26] oeqa/targetcontrol: do not set dump_host_cmds redundantly Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 17/26] oeqa/targetcontrol: remove unused imports Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 18/26] oeqa/utils/commands: fix usage of undefined EPIPE Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 19/26] oeqa/utils/commands: remove unused imports Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 20/26] oeqa/utils/qemurunner: replace hard-coded user 'root' in debug output Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 21/26] selftest: runqemu: better check for ROOTFS: in the log Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 22/26] selftest: runqemu: use better error message when asserts fail Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 23/26] oeqa/runtime: clean up deprecated backslash expansion Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 24/26] shadow: Fix can not print full login timeout message Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 25/26] runqemu: respect IMAGE_LINK_NAME Steve Sakoman
2023-04-06 16:01 ` [OE-core][langdale 26/26] report-error: catch Nothing PROVIDES error Steve Sakoman

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox