[PATCH 0/4] Fix the build failure when creating a multilib wic

[PATCH 0/4] Fix the build failure when creating a multilib wic

[Kevin Hao]

We have observed the following build failure when building lib32-core-image-minimal
for a qemux86-64 machine.
    DEBUG: | ERROR: Couldn't find correct bootimg_dir, exiting
    | ERROR: Couldn't find correct bootimg_dir, exiting
    DEBUG: | 

This patch series tries to fix this issue. You can use the following
steps to reproduce this issue:
 1. source .../git/poky/oe-init-build-env and set the MACHINE to
"qemux86-64"

 2. Add the following to conf/local.conf:
    IMAGE_FSTYPES = "wic"
    
    MULTILIBS = "multilib:lib32"
    require conf/multilib.conf
    DEFAULTTUNE_virtclass-multilib-lib32 = "x86"

 3. bitbake lib32-core-image-minimal

Kevin Hao (4):
  wic: bootimg-pcbios: Drop the unnecessary reference of STAGING_DATADIR
  wic: isoimage-isohybrid: Drop the unnecessary reference of
    STAGING_DATADIR
  bitbake.conf: Introduce variables used to access the other recipe
    specific sysroot directory in multilib case
  wic: Set the right bootimg_dir when building a multilib image

 meta/classes/image_types_wic.bbclass                 |  2 +-
 meta/conf/bitbake.conf                               |  4 ++++
 meta/conf/multilib.conf                              |  1 +
 scripts/lib/wic/plugins/source/bootimg-pcbios.py     |  7 +++----
 scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 13 ++++---------
 scripts/wic                                          |  2 +-
 6 files changed, 14 insertions(+), 15 deletions(-)

-- 
2.14.4

[PATCH 1/4] wic: bootimg-pcbios: Drop the unnecessary reference of STAGING_DATADIR

[Kevin Hao]

Actually the bootimg_dir is the value of STAGING_DATADIR, so we don't
need to check this directory twice times.

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
---
 scripts/lib/wic/plugins/source/bootimg-pcbios.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index d599112dd759..dc7b76cf22a7 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -45,11 +45,10 @@ class BootimgPcbiosPlugin(SourcePlugin):
     @classmethod
     def _get_bootimg_dir(cls, bootimg_dir, dirname):
         """
-        Check if dirname exists in default bootimg_dir or in STAGING_DIR.
+        Check if dirname exists in default bootimg_dir
         """
-        for result in (bootimg_dir, get_bitbake_var("STAGING_DATADIR")):
-            if os.path.exists("%s/%s" % (result, dirname)):
-                return result
+        if os.path.exists("%s/%s" % (bootimg_dir, dirname)):
+            return bootimg_dir
 
         raise WicError("Couldn't find correct bootimg_dir, exiting")
 
-- 
2.14.4

[PATCH 2/4] wic: isoimage-isohybrid: Drop the unnecessary reference of STAGING_DATADIR

[Kevin Hao]

Actually the bootimg_dir is the value of the STAGING_DATADIR, so we
can use it directly.

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
---
 scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index 170077c22c21..d9c7b55f03c5 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -360,11 +360,6 @@ class IsoImagePlugin(SourcePlugin):
             chmod_cmd = "chmod 644 %s" % bootimg
             exec_cmd(chmod_cmd)
 
-        # Prepare files for legacy boot
-        syslinux_dir = get_bitbake_var("STAGING_DATADIR")
-        if not syslinux_dir:
-            raise WicError("Couldn't find STAGING_DATADIR, exiting.")
-
         if os.path.exists("%s/isolinux" % isodir):
             shutil.rmtree("%s/isolinux" % isodir)
 
@@ -373,19 +368,19 @@ class IsoImagePlugin(SourcePlugin):
 
         cls.do_configure_syslinux(creator, cr_workdir)
 
-        install_cmd = "install -m 444 %s/syslinux/ldlinux.sys " % syslinux_dir
+        install_cmd = "install -m 444 %s/syslinux/ldlinux.sys " % bootimg_dir
         install_cmd += "%s/isolinux/ldlinux.sys" % isodir
         exec_cmd(install_cmd)
 
-        install_cmd = "install -m 444 %s/syslinux/isohdpfx.bin " % syslinux_dir
+        install_cmd = "install -m 444 %s/syslinux/isohdpfx.bin " % bootimg_dir
         install_cmd += "%s/isolinux/isohdpfx.bin" % isodir
         exec_cmd(install_cmd)
 
-        install_cmd = "install -m 644 %s/syslinux/isolinux.bin " % syslinux_dir
+        install_cmd = "install -m 644 %s/syslinux/isolinux.bin " % bootimg_dir
         install_cmd += "%s/isolinux/isolinux.bin" % isodir
         exec_cmd(install_cmd)
 
-        install_cmd = "install -m 644 %s/syslinux/ldlinux.c32 " % syslinux_dir
+        install_cmd = "install -m 644 %s/syslinux/ldlinux.c32 " % bootimg_dir
         install_cmd += "%s/isolinux/ldlinux.c32" % isodir
         exec_cmd(install_cmd)
 
-- 
2.14.4

[PATCH 3/4] bitbake.conf: Introduce variables used to access the other recipe specific sysroot directory in multilib case

[Kevin Hao]

When multilib is enabled, there would be two different recipe specific
sysroot directories:
  libxx-recipe-sysroot
  recipe-sysroot

We do need to access the files under the "recipe-sysroot" directory in
some cases. So introduces these variables for this purpose.

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
---
 meta/conf/bitbake.conf  | 4 ++++
 meta/conf/multilib.conf | 1 +
 2 files changed, 5 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 93aee1ae4628..b7e3e5ca26df 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -370,6 +370,7 @@ STAGING_DIR = "${TMPDIR}/sysroots"
 COMPONENTS_DIR = "${STAGING_DIR}-components"
 RECIPE_SYSROOT = "${WORKDIR}/recipe-sysroot"
 RECIPE_SYSROOT_NATIVE = "${WORKDIR}/recipe-sysroot-native"
+RECIPE_ALT_SYSROOT = "${RECIPE_SYSROOT}"
 
 STAGING_DIR_NATIVE = "${RECIPE_SYSROOT_NATIVE}"
 STAGING_BINDIR_NATIVE = "${STAGING_DIR_NATIVE}${bindir_native}"
@@ -394,6 +395,9 @@ STAGING_EXECPREFIXDIR = "${STAGING_DIR_HOST}${exec_prefix}"
 STAGING_LOADER_DIR = "${STAGING_DIR_HOST}/loader"
 STAGING_FIRMWARE_DIR = "${STAGING_DIR_HOST}/firmware"
 
+STAGING_ALT_DIR_HOST = "${RECIPE_ALT_SYSROOT}"
+STAGING_ALT_DATADIR = "${STAGING_ALT_DIR_HOST}${datadir}"
+
 STAGING_DIR_TARGET = "${RECIPE_SYSROOT}"
 
 # Setting DEPLOY_DIR outside of TMPDIR is helpful, when you are using
diff --git a/meta/conf/multilib.conf b/meta/conf/multilib.conf
index e74dec81a897..4dd824fdb273 100644
--- a/meta/conf/multilib.conf
+++ b/meta/conf/multilib.conf
@@ -9,6 +9,7 @@ MULTILIBS ??= "multilib:lib32"
 STAGING_DIR_HOST = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
 STAGING_DIR_TARGET = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
 RECIPE_SYSROOT = "${WORKDIR}/${MLPREFIX}recipe-sysroot"
+RECIPE_ALT_SYSROOT = "${WORKDIR}/recipe-sysroot"
 
 INHERIT += "multilib_global"
 
-- 
2.14.4

Re: [PATCH 3/4] bitbake.conf: Introduce variables used to access the other recipe specific sysroot directory in multilib case

[Richard Purdie]

On Sat, 2018-09-29 at 15:01 +0800, Kevin Hao wrote:
> When multilib is enabled, there would be two different recipe
> specific
> sysroot directories:
>   libxx-recipe-sysroot
>   recipe-sysroot
> 
> We do need to access the files under the "recipe-sysroot" directory
> in some cases. So introduces these variables for this purpose.
> 
> Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
> ---
>  meta/conf/bitbake.conf  | 4 ++++
>  meta/conf/multilib.conf | 1 +
>  2 files changed, 5 insertions(+)

This doesn't look like a good idea to me. With multilib its very easy
just to add in special case options and make it do the right thing
until something else needs 'fixing'. What is harder but works out
better in the long run is to figure out the real underlying problem and
the solution. This patch adds datadir but paves the way to add all the
other *dir variants.

Also, the name "RECIPE_ALT_SYSROOT" is horrible, its near impossible to
know what "ALT" means. Alternative to what?

So whilst I understand what this is trying to do, we need to come up
with a better way, sorry. I've tried thinking how to do it differently
but don't have a proposal right now, sorry.

Cheers,

Richard

[PATCH 4/4] wic: Set the right bootimg_dir when building a multilib image

[Kevin Hao]

When building a multilib image (such as lib32-xxx), we do want the
bootloader to be build in the 64bit mode. So it will be installed into
a directory like:
  .../lib32-core-image-minimal/1.0-r0/recipe-sysroot/usr/share/

In this case, we need to set the bootimg_dir to use the value of
STAGING_ALT_DATADIR.

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
---
 meta/classes/image_types_wic.bbclass | 2 +-
 scripts/wic                          | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/image_types_wic.bbclass b/meta/classes/image_types_wic.bbclass
index 5b40a9e919f2..241e52b3e9f2 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -3,7 +3,7 @@
 WICVARS ?= "\
            BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD IMAGE_BASENAME IMAGE_BOOT_FILES \
            IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
-           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
+           ROOTFS_SIZE STAGING_ALT_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
 
 WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
 WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
diff --git a/scripts/wic b/scripts/wic
index 7392bc4e7f49..067acaabf030 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -166,7 +166,7 @@ def wic_create_subcommand(options, usage_str):
 
         rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", options.image_name)
         kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE", options.image_name)
-        bootimg_dir = get_bitbake_var("STAGING_DATADIR", options.image_name)
+        bootimg_dir = get_bitbake_var("STAGING_ALT_DATADIR", options.image_name)
 
         native_sysroot = options.native_sysroot
         if options.vars_dir and not native_sysroot:
-- 
2.14.4

✗ patchtest: failure for Fix the build failure when creating a multilib wic

[Patchwork]

== Series Details ==

Series: Fix the build failure when creating a multilib wic
Revision: 1
URL   : https://patchwork.openembedded.org/series/14288/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch            [3/4] bitbake.conf: Introduce variables used to access the other recipe specific sysroot directory in multilib case
 Issue             Commit shortlog is too long [test_shortlog_length] 
  Suggested fix    Edit shortlog so that it is 90 characters or less (currently 109 characters)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe