[PATCH 1/7] glibc: Add make-native depends

[PATCH 1/7] glibc: Add make-native depends

[Richard Purdie]

glibc needs make >= 4 yet some of our build workers have older versions of
make. Add a make-native dependency to work around this until all our supported
distros have a recent version of make.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-core/glibc/glibc-initial.inc | 2 +-
 meta/recipes-core/glibc/glibc_2.28.bb     | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-initial.inc b/meta/recipes-core/glibc/glibc-initial.inc
index 28f006fc7f4..d3cfdcc4ccb 100644
--- a/meta/recipes-core/glibc/glibc-initial.inc
+++ b/meta/recipes-core/glibc/glibc-initial.inc
@@ -1,4 +1,4 @@
-DEPENDS = "linux-libc-headers virtual/${TARGET_PREFIX}gcc-initial libgcc-initial"
+DEPENDS = "linux-libc-headers virtual/${TARGET_PREFIX}gcc-initial libgcc-initial make-native"
 PROVIDES = "virtual/${TARGET_PREFIX}libc-initial"
 
 PACKAGES = ""
diff --git a/meta/recipes-core/glibc/glibc_2.28.bb b/meta/recipes-core/glibc/glibc_2.28.bb
index 906e6c2b622..95e333dd5cb 100644
--- a/meta/recipes-core/glibc/glibc_2.28.bb
+++ b/meta/recipes-core/glibc/glibc_2.28.bb
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
       file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
       file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
 
-DEPENDS += "gperf-native bison-native"
+DEPENDS += "gperf-native bison-native make-native"
 
 SRCREV ?= "3c03baca37fdcb52c3881e653ca392bba7a99c2b"
 
-- 
2.17.1

[PATCH 2/7] screen: Add virtual/crypt dependency

[Richard Purdie]

screen uses crypt() so add the missing DEPENDS triggered by glibc 2.28.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-extended/screen/screen_4.6.2.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-extended/screen/screen_4.6.2.bb b/meta/recipes-extended/screen/screen_4.6.2.bb
index 7a33c31d8cc..375dce949d0 100644
--- a/meta/recipes-extended/screen/screen_4.6.2.bb
+++ b/meta/recipes-extended/screen/screen_4.6.2.bb
@@ -11,7 +11,7 @@ LICENSE = "GPLv3+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
                     file://screen.h;endline=26;md5=3971142989289a8198a544220703c2bf"
 
-DEPENDS = "ncurses \
+DEPENDS = "ncurses virtual/crypt \
           ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'libpam', '', d)}"
 RDEPENDS_${PN} = "base-files"
 
-- 
2.17.1

[PATCH 3/7] xf86-video-intel: Fix for glibc

[Richard Purdie]

It fails to build wi9th glibc 2.28, add the missing required header inclusion.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 .../xorg-driver/xf86-video-intel/glibc.patch  | 25 +++++++++++++++++++
 .../xorg-driver/xf86-video-intel_git.bb       |  1 +
 2 files changed, 26 insertions(+)
 create mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch

diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch b/meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch
new file mode 100644
index 00000000000..ada9eb5e52c
--- /dev/null
+++ b/meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch
@@ -0,0 +1,25 @@
+Add a missing include needed for glibc 2.28 to avoid:
+
+| ../../git/tools/backlight_helper.c: In function 'main':
+| ../../git/tools/backlight_helper.c:54:34: error: implicit declaration of function 'major' [-Werror=implicit-function-declaration]
+|   if (fd < 0 || fstat(fd, &st) || major(st.st_dev))
+|                                   ^~~~~
+| ../../git/tools/backlight_helper.c:54:34: warning: nested extern declaration of 'major' [-Wnested-externs]
+| cc1: some warnings being treated as errors
+| Makefile:666: recipe for target 'backlight_helper.o' failed
+
+Upstream-Status: Pending
+RP 2018/8/12
+
+Index: git/tools/backlight_helper.c
+===================================================================
+--- git.orig/tools/backlight_helper.c
++++ git/tools/backlight_helper.c
+@@ -8,6 +8,7 @@
+ 
+ #include <sys/types.h>
+ #include <sys/stat.h>
++#include <sys/sysmacros.h>
+ 
+ #if MAJOR_IN_MKDEV
+ #include <sys/mkdev.h>
diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb b/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
index 5cc4eefc785..655fd4dc692 100644
--- a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
+++ b/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
@@ -15,6 +15,7 @@ S = "${WORKDIR}/git"
 
 SRC_URI = "git://anongit.freedesktop.org/xorg/driver/xf86-video-intel \
            file://0001-Add-Coffeelake-PCI-IDs-for-S-Skus.patch \
+           file://glibc.patch \
            "
 
 UPSTREAM_CHECK_GITTAGREGEX = "(?P<pver>\d+(\.\d+)+)"
-- 
2.17.1

Re: [PATCH 3/7] xf86-video-intel: Fix for glibc

[Khem Raj]

On Mon, Aug 13, 2018 at 7:29 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> It fails to build wi9th glibc 2.28, add the missing required header inclusion.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  .../xorg-driver/xf86-video-intel/glibc.patch  | 25 +++++++++++++++++++
>  .../xorg-driver/xf86-video-intel_git.bb       |  1 +
>  2 files changed, 26 insertions(+)
>  create mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch
>
> diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch b/meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch
> new file mode 100644
> index 00000000000..ada9eb5e52c
> --- /dev/null
> +++ b/meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch
> @@ -0,0 +1,25 @@
> +Add a missing include needed for glibc 2.28 to avoid:
> +
> +| ../../git/tools/backlight_helper.c: In function 'main':
> +| ../../git/tools/backlight_helper.c:54:34: error: implicit declaration of function 'major' [-Werror=implicit-function-declaration]
> +|   if (fd < 0 || fstat(fd, &st) || major(st.st_dev))
> +|                                   ^~~~~
> +| ../../git/tools/backlight_helper.c:54:34: warning: nested extern declaration of 'major' [-Wnested-externs]
> +| cc1: some warnings being treated as errors
> +| Makefile:666: recipe for target 'backlight_helper.o' failed
> +
> +Upstream-Status: Pending
> +RP 2018/8/12
> +
> +Index: git/tools/backlight_helper.c
> +===================================================================
> +--- git.orig/tools/backlight_helper.c
> ++++ git/tools/backlight_helper.c
> +@@ -8,6 +8,7 @@
> +
> + #include <sys/types.h>
> + #include <sys/stat.h>
> ++#include <sys/sysmacros.h>

for OE this is perfectly fine, however, I think if we checked for this
header in configure.ac with AC_CHECK_HEADERS and then did a
conditional include e..g

#ifdef HAVE_SYS_SYSMACROS_H
#include <sys/sysmacros.h>
#endif

will make it upstream-able since this header is relatively new in glibc

> +
> + #if MAJOR_IN_MKDEV
> + #include <sys/mkdev.h>
> diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb b/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
> index 5cc4eefc785..655fd4dc692 100644
> --- a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
> +++ b/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
> @@ -15,6 +15,7 @@ S = "${WORKDIR}/git"
>
>  SRC_URI = "git://anongit.freedesktop.org/xorg/driver/xf86-video-intel \
>             file://0001-Add-Coffeelake-PCI-IDs-for-S-Skus.patch \
> +           file://glibc.patch \
>             "
>
>  UPSTREAM_CHECK_GITTAGREGEX = "(?P<pver>\d+(\.\d+)+)"
> --
> 2.17.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

[PATCH 4/7] package: Fix file copying to preserve sparse files

[Richard Purdie]

We want to preserve sparse files when building the system, add the option
to tar to ensure we do this when copying files.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/package.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 8459d39b27e..a1169489522 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -649,7 +649,7 @@ python perform_packagecopy () {
     # Start by package population by taking a copy of the installed
     # files to operate on
     # Preserve sparse files and hard links
-    cmd = 'tar -cf - -C %s -p . | tar -xf - -C %s' % (dest, dvar)
+    cmd = 'tar -cf - -C %s -p -S . | tar -xf - -C %s' % (dest, dvar)
     subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 
     # replace RPATHs for the nativesdk binaries, to make them relocatable
-- 
2.17.1

[PATCH 5/7] selftest/package: Add test to ensure sparse files are preserved

[Richard Purdie]

Add a new element to the hardlink test to check we also preseve file
sparseness during the packing process. This should ensure we don't regress this
issue again.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 .../recipes-test/selftest-hardlink/selftest-hardlink.bb     | 1 +
 meta/lib/oeqa/selftest/cases/package.py                     | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb
index 8a0e6336aa8..ec330fa9fff 100644
--- a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb
+++ b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb
@@ -16,4 +16,5 @@ do_install () {
 	ln ${D}${bindir}/hello ${D}${bindir}/hello2
 	ln ${D}${bindir}/hello ${D}${bindir}/hello3
 	ln ${D}${bindir}/hello ${D}${bindir}/hello4
+	dd if=/dev/zero of=${D}${bindir}/sparsetest bs=1 count=0 seek=1M
 }
diff --git a/meta/lib/oeqa/selftest/cases/package.py b/meta/lib/oeqa/selftest/cases/package.py
index f288fcafdb4..4ff9b08c96a 100644
--- a/meta/lib/oeqa/selftest/cases/package.py
+++ b/meta/lib/oeqa/selftest/cases/package.py
@@ -88,6 +88,7 @@ class VersionOrdering(OESelftestTestCase):
 
 class PackageTests(OESelftestTestCase):
     # Verify that a recipe which sets up hardlink files has those preserved into split packages
+    # Also test file sparseness is preserved
     def test_preserve_hardlinks(self):
         result = bitbake("selftest-hardlink -c package")
 
@@ -97,3 +98,8 @@ class PackageTests(OESelftestTestCase):
         # Recipe creates 4 hardlinked files, there is a copy in package/ and a copy in packages-split/
         # so expect 8 in total.
         self.assertEqual(os.stat(dest + "/selftest-hardlink" + bindir + "/hello").st_nlink, 8)
+
+        # Test a sparse file remains sparse
+        sparsestat = os.stat(dest + "/selftest-hardlink" + bindir + "/sparsetest")
+        self.assertEqual(sparsestat.st_blocks, 0)
+        self.assertEqual(sparsestat.st_size, 1048576)
-- 
2.17.1

[PATCH 6/7] image: Add locale archive optimisation

[Richard Purdie]

Refactor the locale archive function from the SDK to also make it work during
general image creation. This reduces the size of the locales from 900MB to 220MB
in core-image-lsb-sdk.

The exception handling around subprocess was dropped as the standard subprocess
exception printing is better handled than the catchall exception.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/image.bbclass     |  2 +-
 meta/lib/oe/package_manager.py | 49 ++++++++++++++++++++++++++++++++++
 meta/lib/oe/sdk.py             | 45 ++-----------------------------
 3 files changed, 52 insertions(+), 44 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index c3e73676dca..c0a2714288d 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -22,7 +22,7 @@ POPULATE_SDK_POST_TARGET_COMMAND += "rootfs_sysroot_relativelinks; "
 
 LICENSE ?= "MIT"
 PACKAGES = ""
-DEPENDS += "${@' '.join(["%s-qemuwrapper-cross" % m for m in d.getVar("MULTILIB_VARIANTS").split()])} qemuwrapper-cross depmodwrapper-cross"
+DEPENDS += "${@' '.join(["%s-qemuwrapper-cross" % m for m in d.getVar("MULTILIB_VARIANTS").split()])} qemuwrapper-cross depmodwrapper-cross cross-localedef-native"
 RDEPENDS += "${PACKAGE_INSTALL} ${LINGUAS_INSTALL}"
 RRECOMMENDS += "${PACKAGE_INSTALL_ATTEMPTONLY}"
 PATH_prepend = "${@":".join(all_multilib_tune_values(d, 'STAGING_BINDIR_CROSS').split())}:"
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 64c8a912160..6011e873503 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -12,6 +12,7 @@ import oe.path
 import string
 from oe.gpg_sign import get_signer
 import hashlib
+import fnmatch
 
 # this can be used by all PM backends to create the index files in parallel
 def create_index(arg):
@@ -89,6 +90,47 @@ def failed_postinsts_warn(pkgs, log_path):
 If deferring to first boot wasn't the intent, then scriptlet failure may mean an issue in the recipe, or a regression elsewhere.
 Details of the failure are in %s.""" %(pkgs, log_path))
 
+def generate_locale_archive(d, rootfs, target_arch, localedir):
+    # Pretty sure we don't need this for locale archive generation but
+    # keeping it to be safe...
+    locale_arch_options = { \
+        "arm": ["--uint32-align=4", "--little-endian"],
+        "armeb": ["--uint32-align=4", "--big-endian"],
+        "aarch64": ["--uint32-align=4", "--little-endian"],
+        "aarch64_be": ["--uint32-align=4", "--big-endian"],
+        "sh4": ["--uint32-align=4", "--big-endian"],
+        "powerpc": ["--uint32-align=4", "--big-endian"],
+        "powerpc64": ["--uint32-align=4", "--big-endian"],
+        "mips": ["--uint32-align=4", "--big-endian"],
+        "mipsisa32r6": ["--uint32-align=4", "--big-endian"],
+        "mips64": ["--uint32-align=4", "--big-endian"],
+        "mipsisa64r6": ["--uint32-align=4", "--big-endian"],
+        "mipsel": ["--uint32-align=4", "--little-endian"],
+        "mipsisa32r6el": ["--uint32-align=4", "--little-endian"],
+        "mips64el": ["--uint32-align=4", "--little-endian"],
+        "mipsisa64r6el": ["--uint32-align=4", "--little-endian"],
+        "i586": ["--uint32-align=4", "--little-endian"],
+        "i686": ["--uint32-align=4", "--little-endian"],
+        "x86_64": ["--uint32-align=4", "--little-endian"]
+    }
+    if target_arch in locale_arch_options:
+        arch_options = locale_arch_options[target_arch]
+    else:
+        bb.error("locale_arch_options not found for target_arch=" + target_arch)
+        bb.fatal("unknown arch:" + target_arch + " for locale_arch_options")
+
+    # Need to set this so cross-localedef knows where the archive is
+    env = dict(os.environ)
+    env["LOCALEARCHIVE"] = oe.path.join(localedir, "locale-archive")
+
+    for name in os.listdir(localedir):
+        path = os.path.join(localedir, name)
+        if os.path.isdir(path):
+            cmd = ["cross-localedef", "--verbose"]
+            cmd += arch_options
+            cmd += ["--add-to-archive", path]
+            subprocess.check_output(cmd, env=env, stderr=subprocess.STDOUT)
+
 class Indexer(object, metaclass=ABCMeta):
     def __init__(self, d, deploy_dir):
         self.d = d
@@ -536,6 +578,13 @@ class PackageManager(object, metaclass=ABCMeta):
                          "'%s' returned %d:\n%s" %
                          (' '.join(cmd), e.returncode, e.output.decode("utf-8")))
 
+        target_arch = self.d.getVar('TARGET_ARCH')
+        localedir = oe.path.join(self.target_rootfs, self.d.getVar("libdir"), "locale")
+        if os.path.exists(localedir) and os.listdir(localedir):
+            generate_locale_archive(self.d, self.target_rootfs, target_arch, localedir)
+            # And now delete the binary locales
+            self.remove(fnmatch.filter(self.list_installed(), "glibc-binary-localedata-*"), False)
+
     def deploy_dir_lock(self):
         if self.deploy_dir is None:
             raise RuntimeError("deploy_dir is not set!")
diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
index 0d39ea8a91e..a579d9ae77f 100644
--- a/meta/lib/oe/sdk.py
+++ b/meta/lib/oe/sdk.py
@@ -7,51 +7,10 @@ import shutil
 import glob
 import traceback
 
-def generate_locale_archive(d, rootfs):
-    # Pretty sure we don't need this for SDK archive generation but
-    # keeping it to be safe...
+def generate_locale_archive_sdk(d, rootfs):
     target_arch = d.getVar('SDK_ARCH')
-    locale_arch_options = { \
-        "arm": ["--uint32-align=4", "--little-endian"],
-        "armeb": ["--uint32-align=4", "--big-endian"],
-        "aarch64": ["--uint32-align=4", "--little-endian"],
-        "aarch64_be": ["--uint32-align=4", "--big-endian"],
-        "sh4": ["--uint32-align=4", "--big-endian"],
-        "powerpc": ["--uint32-align=4", "--big-endian"],
-        "powerpc64": ["--uint32-align=4", "--big-endian"],
-        "mips": ["--uint32-align=4", "--big-endian"],
-        "mipsisa32r6": ["--uint32-align=4", "--big-endian"],
-        "mips64": ["--uint32-align=4", "--big-endian"],
-        "mipsisa64r6": ["--uint32-align=4", "--big-endian"],
-        "mipsel": ["--uint32-align=4", "--little-endian"],
-        "mipsisa32r6el": ["--uint32-align=4", "--little-endian"],
-        "mips64el": ["--uint32-align=4", "--little-endian"],
-        "mipsisa64r6el": ["--uint32-align=4", "--little-endian"],
-        "i586": ["--uint32-align=4", "--little-endian"],
-        "i686": ["--uint32-align=4", "--little-endian"],
-        "x86_64": ["--uint32-align=4", "--little-endian"]
-    }
-    if target_arch in locale_arch_options:
-        arch_options = locale_arch_options[target_arch]
-    else:
-        bb.error("locale_arch_options not found for target_arch=" + target_arch)
-        bb.fatal("unknown arch:" + target_arch + " for locale_arch_options")
-
     localedir = oe.path.join(rootfs, d.getVar("libdir_nativesdk"), "locale")
-    # Need to set this so cross-localedef knows where the archive is
-    env = dict(os.environ)
-    env["LOCALEARCHIVE"] = oe.path.join(localedir, "locale-archive")
-
-    for name in os.listdir(localedir):
-        path = os.path.join(localedir, name)
-        if os.path.isdir(path):
-            try:
-                cmd = ["cross-localedef", "--verbose"]
-                cmd += arch_options
-                cmd += ["--add-to-archive", path]
-                subprocess.check_output(cmd, env=env, stderr=subprocess.STDOUT)
-            except Exception as e:
-                bb.fatal("Cannot create locale archive: %s" % e.output)
+    generate_locale_archive(d, rootfs, target_arch, localedir)
 
 class Sdk(object, metaclass=ABCMeta):
     def __init__(self, d, manifest_dir):
-- 
2.17.1

Re: [PATCH 6/7] image: Add locale archive optimisation

[Khem Raj]

On Mon, Aug 13, 2018 at 7:29 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> Refactor the locale archive function from the SDK to also make it work during
> general image creation. This reduces the size of the locales from 900MB to 220MB
> in core-image-lsb-sdk.
>
> The exception handling around subprocess was dropped as the standard subprocess
> exception printing is better handled than the catchall exception.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/image.bbclass     |  2 +-
>  meta/lib/oe/package_manager.py | 49 ++++++++++++++++++++++++++++++++++
>  meta/lib/oe/sdk.py             | 45 ++-----------------------------
>  3 files changed, 52 insertions(+), 44 deletions(-)
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index c3e73676dca..c0a2714288d 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -22,7 +22,7 @@ POPULATE_SDK_POST_TARGET_COMMAND += "rootfs_sysroot_relativelinks; "
>
>  LICENSE ?= "MIT"
>  PACKAGES = ""
> -DEPENDS += "${@' '.join(["%s-qemuwrapper-cross" % m for m in d.getVar("MULTILIB_VARIANTS").split()])} qemuwrapper-cross depmodwrapper-cross"
> +DEPENDS += "${@' '.join(["%s-qemuwrapper-cross" % m for m in d.getVar("MULTILIB_VARIANTS").split()])} qemuwrapper-cross depmodwrapper-cross cross-localedef-native"
>  RDEPENDS += "${PACKAGE_INSTALL} ${LINGUAS_INSTALL}"
>  RRECOMMENDS += "${PACKAGE_INSTALL_ATTEMPTONLY}"
>  PATH_prepend = "${@":".join(all_multilib_tune_values(d, 'STAGING_BINDIR_CROSS').split())}:"
> diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
> index 64c8a912160..6011e873503 100644
> --- a/meta/lib/oe/package_manager.py
> +++ b/meta/lib/oe/package_manager.py
> @@ -12,6 +12,7 @@ import oe.path
>  import string
>  from oe.gpg_sign import get_signer
>  import hashlib
> +import fnmatch
>
>  # this can be used by all PM backends to create the index files in parallel
>  def create_index(arg):
> @@ -89,6 +90,47 @@ def failed_postinsts_warn(pkgs, log_path):
>  If deferring to first boot wasn't the intent, then scriptlet failure may mean an issue in the recipe, or a regression elsewhere.
>  Details of the failure are in %s.""" %(pkgs, log_path))
>
> +def generate_locale_archive(d, rootfs, target_arch, localedir):
> +    # Pretty sure we don't need this for locale archive generation but
> +    # keeping it to be safe...
> +    locale_arch_options = { \
> +        "arm": ["--uint32-align=4", "--little-endian"],
> +        "armeb": ["--uint32-align=4", "--big-endian"],
> +        "aarch64": ["--uint32-align=4", "--little-endian"],
> +        "aarch64_be": ["--uint32-align=4", "--big-endian"],
> +        "sh4": ["--uint32-align=4", "--big-endian"],
> +        "powerpc": ["--uint32-align=4", "--big-endian"],
> +        "powerpc64": ["--uint32-align=4", "--big-endian"],
> +        "mips": ["--uint32-align=4", "--big-endian"],
> +        "mipsisa32r6": ["--uint32-align=4", "--big-endian"],
> +        "mips64": ["--uint32-align=4", "--big-endian"],
> +        "mipsisa64r6": ["--uint32-align=4", "--big-endian"],
> +        "mipsel": ["--uint32-align=4", "--little-endian"],
> +        "mipsisa32r6el": ["--uint32-align=4", "--little-endian"],
> +        "mips64el": ["--uint32-align=4", "--little-endian"],
> +        "mipsisa64r6el": ["--uint32-align=4", "--little-endian"],
> +        "i586": ["--uint32-align=4", "--little-endian"],
> +        "i686": ["--uint32-align=4", "--little-endian"],
> +        "x86_64": ["--uint32-align=4", "--little-endian"]
> +    }

perhaps now we should also add risc-v here since its not limited to SDK alone

"riscv64": ["--uint32-align=4", "--little-endian"]
"riscv32": ["--uint32-align=4", "--little-endian"]

> +    if target_arch in locale_arch_options:
> +        arch_options = locale_arch_options[target_arch]
> +    else:
> +        bb.error("locale_arch_options not found for target_arch=" + target_arch)
> +        bb.fatal("unknown arch:" + target_arch + " for locale_arch_options")
> +
> +    # Need to set this so cross-localedef knows where the archive is
> +    env = dict(os.environ)
> +    env["LOCALEARCHIVE"] = oe.path.join(localedir, "locale-archive")
> +
> +    for name in os.listdir(localedir):
> +        path = os.path.join(localedir, name)
> +        if os.path.isdir(path):
> +            cmd = ["cross-localedef", "--verbose"]
> +            cmd += arch_options
> +            cmd += ["--add-to-archive", path]
> +            subprocess.check_output(cmd, env=env, stderr=subprocess.STDOUT)
> +
>  class Indexer(object, metaclass=ABCMeta):
>      def __init__(self, d, deploy_dir):
>          self.d = d
> @@ -536,6 +578,13 @@ class PackageManager(object, metaclass=ABCMeta):
>                           "'%s' returned %d:\n%s" %
>                           (' '.join(cmd), e.returncode, e.output.decode("utf-8")))
>
> +        target_arch = self.d.getVar('TARGET_ARCH')
> +        localedir = oe.path.join(self.target_rootfs, self.d.getVar("libdir"), "locale")
> +        if os.path.exists(localedir) and os.listdir(localedir):
> +            generate_locale_archive(self.d, self.target_rootfs, target_arch, localedir)
> +            # And now delete the binary locales
> +            self.remove(fnmatch.filter(self.list_installed(), "glibc-binary-localedata-*"), False)
> +
>      def deploy_dir_lock(self):
>          if self.deploy_dir is None:
>              raise RuntimeError("deploy_dir is not set!")
> diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
> index 0d39ea8a91e..a579d9ae77f 100644
> --- a/meta/lib/oe/sdk.py
> +++ b/meta/lib/oe/sdk.py
> @@ -7,51 +7,10 @@ import shutil
>  import glob
>  import traceback
>
> -def generate_locale_archive(d, rootfs):
> -    # Pretty sure we don't need this for SDK archive generation but
> -    # keeping it to be safe...
> +def generate_locale_archive_sdk(d, rootfs):
>      target_arch = d.getVar('SDK_ARCH')
> -    locale_arch_options = { \
> -        "arm": ["--uint32-align=4", "--little-endian"],
> -        "armeb": ["--uint32-align=4", "--big-endian"],
> -        "aarch64": ["--uint32-align=4", "--little-endian"],
> -        "aarch64_be": ["--uint32-align=4", "--big-endian"],
> -        "sh4": ["--uint32-align=4", "--big-endian"],
> -        "powerpc": ["--uint32-align=4", "--big-endian"],
> -        "powerpc64": ["--uint32-align=4", "--big-endian"],
> -        "mips": ["--uint32-align=4", "--big-endian"],
> -        "mipsisa32r6": ["--uint32-align=4", "--big-endian"],
> -        "mips64": ["--uint32-align=4", "--big-endian"],
> -        "mipsisa64r6": ["--uint32-align=4", "--big-endian"],
> -        "mipsel": ["--uint32-align=4", "--little-endian"],
> -        "mipsisa32r6el": ["--uint32-align=4", "--little-endian"],
> -        "mips64el": ["--uint32-align=4", "--little-endian"],
> -        "mipsisa64r6el": ["--uint32-align=4", "--little-endian"],
> -        "i586": ["--uint32-align=4", "--little-endian"],
> -        "i686": ["--uint32-align=4", "--little-endian"],
> -        "x86_64": ["--uint32-align=4", "--little-endian"]
> -    }
> -    if target_arch in locale_arch_options:
> -        arch_options = locale_arch_options[target_arch]
> -    else:
> -        bb.error("locale_arch_options not found for target_arch=" + target_arch)
> -        bb.fatal("unknown arch:" + target_arch + " for locale_arch_options")
> -
>      localedir = oe.path.join(rootfs, d.getVar("libdir_nativesdk"), "locale")
> -    # Need to set this so cross-localedef knows where the archive is
> -    env = dict(os.environ)
> -    env["LOCALEARCHIVE"] = oe.path.join(localedir, "locale-archive")
> -
> -    for name in os.listdir(localedir):
> -        path = os.path.join(localedir, name)
> -        if os.path.isdir(path):
> -            try:
> -                cmd = ["cross-localedef", "--verbose"]
> -                cmd += arch_options
> -                cmd += ["--add-to-archive", path]
> -                subprocess.check_output(cmd, env=env, stderr=subprocess.STDOUT)
> -            except Exception as e:
> -                bb.fatal("Cannot create locale archive: %s" % e.output)
> +    generate_locale_archive(d, rootfs, target_arch, localedir)
>
>  class Sdk(object, metaclass=ABCMeta):
>      def __init__(self, d, manifest_dir):
> --
> 2.17.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

[PATCH 7/7] glibc: Fix locale archive path patch

[Richard Purdie]

The locale code uses the archive location in two places, ensure both are
corrected to use the environment variable which avoids nasty build
failures when archiving locales in images.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 ...-archive-uses-a-hard-coded-locale-pa.patch | 54 +++++++++++++++----
 1 file changed, 45 insertions(+), 9 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc/0029-localedef-add-to-archive-uses-a-hard-coded-locale-pa.patch b/meta/recipes-core/glibc/glibc/0029-localedef-add-to-archive-uses-a-hard-coded-locale-pa.patch
index aa3a9e26546..974fecdd569 100644
--- a/meta/recipes-core/glibc/glibc/0029-localedef-add-to-archive-uses-a-hard-coded-locale-pa.patch
+++ b/meta/recipes-core/glibc/glibc/0029-localedef-add-to-archive-uses-a-hard-coded-locale-pa.patch
@@ -18,11 +18,50 @@ Signed-off-by: Khem Raj <raj.khem@gmail.com>
  locale/programs/locarchive.c | 13 ++++++++-----
  1 file changed, 8 insertions(+), 5 deletions(-)
 
-diff --git a/locale/programs/locarchive.c b/locale/programs/locarchive.c
-index ca332a345f..6b7ba9b2e3 100644
---- a/locale/programs/locarchive.c
-+++ b/locale/programs/locarchive.c
-@@ -569,10 +569,13 @@ open_archive (struct locarhandle *ah, bool readonly)
+Index: git/locale/programs/locarchive.c
+===================================================================
+--- git.orig/locale/programs/locarchive.c
++++ git/locale/programs/locarchive.c
+@@ -340,12 +340,24 @@ enlarge_archive (struct locarhandle *ah,
+   struct namehashent *oldnamehashtab;
+   struct locarhandle new_ah;
+   size_t prefix_len = output_prefix ? strlen (output_prefix) : 0;
+-  char archivefname[prefix_len + sizeof (ARCHIVE_NAME)];
+-  char fname[prefix_len + sizeof (ARCHIVE_NAME) + sizeof (".XXXXXX") - 1];
++  char *archivefname;
++  char *fname;
++  char *envarchive = getenv("LOCALEARCHIVE");
+ 
+-  if (output_prefix)
+-    memcpy (archivefname, output_prefix, prefix_len);
+-  strcpy (archivefname + prefix_len, ARCHIVE_NAME);
++  if (envarchive != NULL) 
++    {
++      archivefname = xmalloc(strlen(envarchive));
++      fname = xmalloc(strlen(envarchive) + sizeof (".XXXXXX") - 1);
++      strcpy (archivefname, envarchive);
++    }
++  else
++    {
++      archivefname = xmalloc(prefix_len + sizeof (ARCHIVE_NAME));
++      fname = xmalloc(prefix_len + sizeof (ARCHIVE_NAME) + sizeof (".XXXXXX") - 1);
++      if (output_prefix)
++        memcpy (archivefname, output_prefix, prefix_len);
++      strcpy (archivefname + prefix_len, ARCHIVE_NAME);
++    }
+   strcpy (stpcpy (fname, archivefname), ".XXXXXX");
+ 
+   /* Not all of the old file has to be mapped.  Change this now this
+@@ -551,6 +563,8 @@ enlarge_archive (struct locarhandle *ah,
+ 
+   /* Add the information for the new one.  */
+   *ah = new_ah;
++  free(archivefname);
++  free(fname);
+ }
+ 
+ 
+@@ -569,10 +583,13 @@ open_archive (struct locarhandle *ah, bo
    /* If ah has a non-NULL fname open that otherwise open the default.  */
    if (archivefname == NULL)
      {
@@ -40,7 +79,7 @@ index ca332a345f..6b7ba9b2e3 100644
      }
  
    while (1)
-@@ -585,7 +588,7 @@ open_archive (struct locarhandle *ah, bool readonly)
+@@ -585,7 +602,7 @@ open_archive (struct locarhandle *ah, bo
  	     the default locale archive we ignore the failure and
  	     list an empty archive, otherwise we print an error
  	     and exit.  */
@@ -49,6 +88,3 @@ index ca332a345f..6b7ba9b2e3 100644
  	    {
  	      if (readonly)
  		{
--- 
-2.18.0
-
-- 
2.17.1

✗ patchtest: failure for "glibc: Add make-native depends..." and 6 more

[Patchwork]

== Series Details ==

Series: "glibc: Add make-native depends..." and 6 more
Revision: 1
URL   : https://patchwork.openembedded.org/series/13483/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 3d493838f0)

* Issue             A patch file has been added, but does not have a Signed-off-by tag [test_signed_off_by_presence] 
  Suggested fix    Sign off the added patch file (meta/recipes-graphics/xorg-driver/xf86-video-intel/glibc.patch)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe