[PATCH 0/6] Fixes for mutilib SDK

[PATCH 0/6] Fixes for mutilib SDK

[Robert Yang]

* This fixed:
  MACHINE = "qemux86-64"
  require conf/multilib.conf
  MULTILIBS = "multilib:lib32"
  DEFAULTTUNE_virtclass-multilib-lib32 = "x86"

  $ bitbake core-image-minimal -cpopulate_sdk
  Install poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh,
  then source environment-setup-core2-64-pokymllib32-linux or
  environment-setup-core2-64-poky-linux, both of them will compile hello.c
  well. (The 32bit was broken in the past)

  $ bitbake lib32-core-image-minimal -cpopulate_sdk
  Install poky-glibc-x86_64-lib32-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh,
  then source environment-setup-core2-64-poky-linux or
  environment-setup-core2-64-pokymllib32-linux, both of them will compile
  hello.c well (neither of them worked in the past)

  This also fixed the problem that not all dependencies mutilib packages
  are installed, please see:
  https://bugzilla.yoctoproject.org/show_bug.cgi?id=4408

Tested:
$ bitbake core-image-minimal core-image-sato -cpopulate_sdk && bitbake core-image-sato-sdk world core-image-minimal core-image-sato

// Robert

The following changes since commit f0189829498e30231d826c9f55aad73e622d076e:

  qemu: Update to upstream patches (2015-09-14 11:22:02 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/sdk
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/sdk

Robert Yang (6):
  toolchain-shar-extract.sh: remove checkbashism
  oe-pkgdata-util: avoid returning skipped packages
  package_manager.py: make rpm install mutilib pkgs corectly
  multilib.bbclass: install all bits toochains when populate mlprefix
    SDK
  populate_sdk_base.bbclass: fix SDKTARGETSYSROOT when populate
    mlprefix SDK
  meta-environment.bb: fix environment-setup* when populate mlprefix
    SDK

 meta/classes/multilib.bbclass              |   15 +++++
 meta/classes/populate_sdk_base.bbclass     |    6 +-
 meta/files/toolchain-shar-extract.sh       |    4 +-
 meta/lib/oe/package_manager.py             |   83 +++++++++++++++++++++++++++-
 meta/recipes-core/meta/meta-environment.bb |    4 ++
 scripts/oe-pkgdata-util                    |    5 +-
 6 files changed, 111 insertions(+), 6 deletions(-)

-- 
1.7.9.5

[PATCH 1/6] toolchain-shar-extract.sh: remove checkbashism

[Robert Yang]

Fixed when sh is bash:
$ sh ./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh
Poky (Yocto Project Reference Distro) SDK installer version 1.8+snapshot
===========================================================
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh: 77: read: Illegal option -e
You are about to install the SDK to "/opt/poky/1.8+snapshot". Proceed[Y/n]? ../SDK2
Installation aborted!

There is ony one bashism "read -e" in toolchain-shar-extract.sh, but
'-e' is useless here, so remove it and use /bin/sh.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/files/toolchain-shar-extract.sh |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/files/toolchain-shar-extract.sh b/meta/files/toolchain-shar-extract.sh
index cd0a547..b56bab0 100644
--- a/meta/files/toolchain-shar-extract.sh
+++ b/meta/files/toolchain-shar-extract.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
 
 INST_ARCH=$(uname -m | sed -e "s/i[3-6]86/ix86/" -e "s/x86[-_]64/x86_64/")
 SDK_ARCH=$(echo @SDK_ARCH@ | sed -e "s/i[3-6]86/ix86/" -e "s/x86[-_]64/x86_64/")
@@ -86,7 +86,7 @@ if [ "$target_sdk_dir" = "" ]; then
 	if [ "$answer" = "Y" ]; then
 		target_sdk_dir="$DEFAULT_INSTALL_DIR"
 	else
-		read -e -p "Enter target directory for SDK (default: $DEFAULT_INSTALL_DIR): " target_sdk_dir
+		read -p "Enter target directory for SDK (default: $DEFAULT_INSTALL_DIR): " target_sdk_dir
 		[ "$target_sdk_dir" = "" ] && target_sdk_dir=$DEFAULT_INSTALL_DIR
 	fi
 fi
-- 
1.7.9.5

[PATCH 2/6] oe-pkgdata-util: avoid returning skipped packages

[Robert Yang]

The skipped packages may be pulled in by another package, for example,
when libc6-dbg is already installed and should be skipped, but it would
be pulled in by libsegfault, this patch fixes the issue.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 scripts/oe-pkgdata-util |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index b075775..116cfae 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -60,6 +60,7 @@ def glob(args):
         skipval += "|" + args.exclude
     skipregex = re.compile(skipval)
 
+    skippedpkgs = set()
     mappedpkgs = set()
     with open(args.pkglistfile, 'r') as f:
         for line in f:
@@ -73,6 +74,7 @@ def glob(args):
             # Skip packages for which there is no point applying globs
             if skipregex.search(pkg):
                 logger.debug("%s -> !!" % pkg)
+                skippedpkgs.add(pkg)
                 continue
 
             # Skip packages that already match the globs, so if e.g. a dev package
@@ -84,6 +86,7 @@ def glob(args):
                     already = True
                     break
             if already:
+                skippedpkgs.add(pkg)
                 logger.debug("%s -> !" % pkg)
                 continue
 
@@ -152,7 +155,7 @@ def glob(args):
 
     logger.debug("------")
 
-    print("\n".join(mappedpkgs))
+    print("\n".join(mappedpkgs - skippedpkgs))
 
 def read_value(args):
     # Handle both multiple arguments and multiple values within an arg (old syntax)
-- 
1.7.9.5

[PATCH 3/6] package_manager.py: make rpm install mutilib pkgs corectly

[Robert Yang]

When configure multilib, "bitbake <image_bb> -c populate_sdk" should
install all arch toolchains (for example, 32 and 64bit), but rpm not
handle the multilib requires correctly, for example:
lib32-packagegroup-core-standalone-sdk-target requires lib32-libc6, rpm
may pull in libc6 rather than lib32-libc6, there are the similar issue
when:

IMAGE_INSTALL_append += "lib32-packagegroup-foo foo"

Use bitbake to expand the RDEPENDS will fix the problem since bitbake
knows mlprefix and handle it well, but rpm doesn't.

This patch only affects when:
IMAGE_INSTALL = "lib32-foo foo"
Doesn't affect:
IMAGE_INSTALL = "lib32-foo1 lib32-foo2"
Or:
IMAGE_INSTALL = "foo1 foo2"

[YOCTO #8089]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/lib/oe/package_manager.py |   83 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 81 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 292ed44..c51e88b 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -605,12 +605,12 @@ class PackageManager(object):
             cmd.extend(['-x', exclude])
         try:
             bb.note("Installing complementary packages ...")
+            bb.note('Running %s' % cmd)
             complementary_pkgs = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
         except subprocess.CalledProcessError as e:
             bb.fatal("Could not compute complementary packages list. Command "
                      "'%s' returned %d:\n%s" %
                      (' '.join(cmd), e.returncode, e.output))
-
         self.install(complementary_pkgs.split(), attempt_only=True)
 
     def deploy_dir_lock(self):
@@ -1050,6 +1050,35 @@ class RpmPM(PackageManager):
     def update(self):
         self._invoke_smart('update rpmsys')
 
+    def get_rdepends_recursively(self, pkgs):
+        # pkgs will be changed during the loop, so use [:] to make a copy.
+        for pkg in pkgs[:]:
+            sub_data = oe.packagedata.read_subpkgdata(pkg, self.d)
+            sub_rdep = sub_data.get("RDEPENDS_" + pkg)
+            if not sub_rdep:
+                continue
+            done = bb.utils.explode_dep_versions2(sub_rdep).keys()
+            next = done
+            # Find all the rdepends on dependency chain
+            while next:
+                new = []
+                for sub_pkg in next:
+                    sub_data = oe.packagedata.read_subpkgdata(sub_pkg, self.d)
+                    sub_pkg_rdep = sub_data.get("RDEPENDS_" + sub_pkg)
+                    if not sub_pkg_rdep:
+                        continue
+                    for p in bb.utils.explode_dep_versions2(sub_pkg_rdep):
+                        # Already handled, skip it.
+                        if p in done or p in pkgs:
+                            continue
+                        # It's a new dep
+                        if oe.packagedata.has_subpkgdata(p, self.d):
+                            done.append(p)
+                            new.append(p)
+                next = new
+            pkgs.extend(done)
+        return pkgs
+
     '''
     Install pkgs with smart, the pkg name is oe format
     '''
@@ -1059,8 +1088,58 @@ class RpmPM(PackageManager):
             bb.note("There are no packages to install")
             return
         bb.note("Installing the following packages: %s" % ' '.join(pkgs))
+        if not attempt_only:
+            # Pull in multilib requires since rpm may not pull in them
+            # correctly, for example,
+            # lib32-packagegroup-core-standalone-sdk-target requires
+            # lib32-libc6, but rpm may pull in libc6 rather than lib32-libc6
+            # since it doesn't know mlprefix (lib32-), bitbake knows it and
+            # can handle it well, find out the RDEPENDS on the chain will
+            # fix the problem. Both do_rootfs and do_populate_sdk have this
+            # issue.
+            # The attempt_only packages don't need this since they are
+            # based on the installed ones.
+            #
+            # Separate pkgs into two lists, one is multilib, the other one
+            # is non-multilib.
+            ml_pkgs = []
+            non_ml_pkgs = pkgs[:]
+            for pkg in pkgs:
+                for mlib in (self.d.getVar("MULTILIB_VARIANTS", True) or "").split():
+                    if pkg.startswith(mlib + '-'):
+                        ml_pkgs.append(pkg)
+                        non_ml_pkgs.remove(pkg)
+
+            if len(ml_pkgs) > 0 and len(non_ml_pkgs) > 0:
+                # Found both foo and lib-foo
+                ml_pkgs = self.get_rdepends_recursively(ml_pkgs)
+                non_ml_pkgs = self.get_rdepends_recursively(non_ml_pkgs)
+                # Longer list makes smart slower, so only keep the pkgs
+                # which have the same BPN, and smart can handle others
+                # correctly.
+                pkgs_new = []
+                for pkg in non_ml_pkgs:
+                    for mlib in (self.d.getVar("MULTILIB_VARIANTS", True) or "").split():
+                        mlib_pkg = mlib + "-" + pkg
+                        if mlib_pkg in ml_pkgs:
+                            pkgs_new.append(pkg)
+                            pkgs_new.append(mlib_pkg)
+                for pkg in pkgs:
+                    if pkg not in pkgs_new:
+                        pkgs_new.append(pkg)
+                pkgs = pkgs_new
+                new_depends = {}
+                deps = bb.utils.explode_dep_versions2(" ".join(pkgs))
+                for depend in deps:
+                    data = oe.packagedata.read_subpkgdata(depend, self.d)
+                    key = "PKG_%s" % depend
+                    if key in data:
+                        new_depend = data[key]
+                    else:
+                        new_depend = depend
+                    new_depends[new_depend] = deps[depend]
+                pkgs = bb.utils.join_deps(new_depends, commasep=True).split(', ')
         pkgs = self._pkg_translate_oe_to_smart(pkgs, attempt_only)
-
         if not attempt_only:
             bb.note('to be installed: %s' % ' '.join(pkgs))
             cmd = "%s %s install -y %s" % \
-- 
1.7.9.5

[PATCH 4/6] multilib.bbclass: install all bits toochains when populate mlprefix SDK

[Robert Yang]

Fixed when:
$ bitbake lib32-core-image-minimal  -cpopulate_sdk

Only 32 bit toolchain was installed but there were both
environment-setup-core2-64-poky-linux and
environment-setup-core2-64-pokymllib32-linux in extracted sdk, this was
becase multilib.bbclass mapped TOOLCHAIN_TARGET_TASK into only lib32-
ones, and dropped 64 bit ones. This patches fixes the problem.

[YOCTO #8089]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/multilib.bbclass |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/meta/classes/multilib.bbclass b/meta/classes/multilib.bbclass
index 8f61d8d..465e208 100644
--- a/meta/classes/multilib.bbclass
+++ b/meta/classes/multilib.bbclass
@@ -94,8 +94,22 @@ python __anonymous () {
         d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
 
     if bb.data.inherits_class('populate_sdk_base', d):
+        orig_task = (d.getVar('TOOLCHAIN_TARGET_TASK', True) or "").split()
+        orig_task_attemptonly = (d.getVar('TOOLCHAIN_TARGET_TASK_ATTEMPTONLY', True) or "").split()
         clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
         clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
+        # Append original tasks since all bits (such as 32 and 64)
+        # toolchains should be installed when mutilib.
+        task = (d.getVar('TOOLCHAIN_TARGET_TASK', True) or "").split()
+        task_attemptonly = (d.getVar('TOOLCHAIN_TARGET_TASK_ATTEMPTONLY', True) or "").split()
+        for t in orig_task:
+            if t not in task:
+                task.append(t)
+        for t in orig_task_attemptonly:
+            if t not in task_attemptonly:
+                task_attemptonly.append(t)
+        d.setVar("TOOLCHAIN_TARGET_TASK", " ".join(task))
+        d.setVar("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY", " ".join(task_attemptonly))
 
     if bb.data.inherits_class('image', d):
         return
-- 
1.7.9.5

[PATCH 5/6] populate_sdk_base.bbclass: fix SDKTARGETSYSROOT when populate mlprefix SDK

[Robert Yang]

Fixed when populate multlib sdk:
$ bitbake lib32-core-image-minimal -cpopulate_sdk

When extract sdk, there was a sysroots/core2-64-pokymllib32-linux, but the
SDKTARGETSYSROOT path in environment-setup-x86-pokymllib32-linux was
sysroots/core2-64-poky-linux, so it was incorrect, install sysroot to
sysroots/core2-64-poky-linux would fix the problem since:
1) We install both 32 and 64 bit toolchains
2) The meta-environment is a nativesdk recipe which doesn't have
   multilib extend, so we can't or hard to fix from meta-environment.

[YOCTO #8089]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/multilib.bbclass          |    1 +
 meta/classes/populate_sdk_base.bbclass |    6 +++++-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/classes/multilib.bbclass b/meta/classes/multilib.bbclass
index 465e208..fd54b69 100644
--- a/meta/classes/multilib.bbclass
+++ b/meta/classes/multilib.bbclass
@@ -28,6 +28,7 @@ python multilib_virtclass_handler () {
         e.data.setVar("PN", variant + "-" + e.data.getVar("PN", False))
         target_vendor = e.data.getVar("TARGET_VENDOR_" + "virtclass-multilib-" + variant, False)
         if target_vendor:
+            e.data.setVar("TARGET_VENDOR_MULTILIB_ORIGINAL", e.data.getVar("TARGET_VENDOR", True))
             e.data.setVar("TARGET_VENDOR", target_vendor)
         return
 
diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index aa7a9a5..9c5a360 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -28,7 +28,11 @@ SDK_DEPLOY = "${DEPLOY_DIR}/sdk"
 
 B_task-populate-sdk = "${SDK_DIR}"
 
-SDKTARGETSYSROOT = "${SDKPATH}/sysroots/${REAL_MULTIMACH_TARGET_SYS}"
+# Install toolchain to the path with TARGET_VENDOR_MULTILIB_ORIGINAL since:
+# 1) Both toolchains (32 and 64 bits) are installed
+# 2) The environment-setup-* doesn't contain mllib
+TARGET_VENDOR_MULTILIB_ORIGINAL ??= "${TARGET_VENDOR}"
+SDKTARGETSYSROOT = "${SDKPATH}/sysroots/${TUNE_PKGARCH}${TARGET_VENDOR_MULTILIB_ORIGINAL}-${TARGET_OS}"
 
 TOOLCHAIN_HOST_TASK ?= "nativesdk-packagegroup-sdk-host packagegroup-cross-canadian-${MACHINE}"
 TOOLCHAIN_HOST_TASK_ATTEMPTONLY ?= ""
-- 
1.7.9.5

[PATCH 6/6] meta-environment.bb: fix environment-setup* when populate mlprefix SDK

[Robert Yang]

Fixed when extract mutilib SDK like
poky-glibc-x86_64-lib32-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh:

Each time you wish to use the SDK in a new shell session, you need to
source the environment setup script e.g.
 $ ./path/to/environment-setup-core2-64-pokymllib32-linux

There was no environment-setup-core2-64-pokymllib32-linux, but
environment-setup-x86-pokymllib32-linux, this was because image recipe
didn't upgrade TUNE_PKGARCH from core2-64 to x86, but meta-environment
did, let meta-environment not upgrade TUNE_PKGARCH will fix the
problem.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-core/meta/meta-environment.bb |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-core/meta/meta-environment.bb b/meta/recipes-core/meta/meta-environment.bb
index 49d45f6..c671c13 100644
--- a/meta/recipes-core/meta/meta-environment.bb
+++ b/meta/recipes-core/meta/meta-environment.bb
@@ -42,6 +42,10 @@ python do_generate_content() {
         localdata.setVar("OVERRIDES", overrides)
         localdata.setVar("MLPREFIX", item + "-")
         bb.data.update_data(localdata)
+        # Don't upgrade TUNE_PKGARCH since image doesn't upgrade it,
+        # otherwise the siteconfig, env_script and version would
+        # mismatch image's REAL_MULTIMACH_TARGET_SYS.
+        localdata.setVar('TUNE_PKGARCH', d.getVar('TUNE_PKGARCH', True))
         bb.build.exec_func("create_sdk_files", localdata)
 }
 addtask generate_content before do_install after do_compile
-- 
1.7.9.5

Re: [PATCH 0/6] Fixes for mutilib SDK

[Robert Yang]

On 09/16/2015 10:04 AM, Robert Yang wrote:
> * This fixed:
>    MACHINE = "qemux86-64"
>    require conf/multilib.conf
>    MULTILIBS = "multilib:lib32"
>    DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
>
>    $ bitbake core-image-minimal -cpopulate_sdk
>    Install poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh,
>    then source environment-setup-core2-64-pokymllib32-linux or
>    environment-setup-core2-64-poky-linux, both of them will compile hello.c
>    well. (The 32bit was broken in the past)
>
>    $ bitbake lib32-core-image-minimal -cpopulate_sdk
>    Install poky-glibc-x86_64-lib32-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh,
>    then source environment-setup-core2-64-poky-linux or
>    environment-setup-core2-64-pokymllib32-linux, both of them will compile
>    hello.c well (neither of them worked in the past)
>
>    This also fixed the problem that not all dependencies mutilib packages
>    are installed, please see:
>    https://bugzilla.yoctoproject.org/show_bug.cgi?id=4408

And also fixed this one:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=7610

Add IMAGE_INSTALL_append = " bash" to local.conf
$ bitbake lib32-core-image-minimal

32bit bash will be installed, but not 64bit.

Now 64 bit will be installed.

// Robert

>
> Tested:
> $ bitbake core-image-minimal core-image-sato -cpopulate_sdk && bitbake core-image-sato-sdk world core-image-minimal core-image-sato
>
> // Robert
>
> The following changes since commit f0189829498e30231d826c9f55aad73e622d076e:
>
>    qemu: Update to upstream patches (2015-09-14 11:22:02 +0100)
>
> are available in the git repository at:
>
>    git://git.openembedded.org/openembedded-core-contrib rbt/sdk
>    http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/sdk
>
> Robert Yang (6):
>    toolchain-shar-extract.sh: remove checkbashism
>    oe-pkgdata-util: avoid returning skipped packages
>    package_manager.py: make rpm install mutilib pkgs corectly
>    multilib.bbclass: install all bits toochains when populate mlprefix
>      SDK
>    populate_sdk_base.bbclass: fix SDKTARGETSYSROOT when populate
>      mlprefix SDK
>    meta-environment.bb: fix environment-setup* when populate mlprefix
>      SDK
>
>   meta/classes/multilib.bbclass              |   15 +++++
>   meta/classes/populate_sdk_base.bbclass     |    6 +-
>   meta/files/toolchain-shar-extract.sh       |    4 +-
>   meta/lib/oe/package_manager.py             |   83 +++++++++++++++++++++++++++-
>   meta/recipes-core/meta/meta-environment.bb |    4 ++
>   scripts/oe-pkgdata-util                    |    5 +-
>   6 files changed, 111 insertions(+), 6 deletions(-)
>

Re: [PATCH 0/6] Fixes for mutilib SDK

[Robert Yang]

Hi RP and Ross,

These patches are required by multilib's do_rootfs and do_populate_sdk,
otherwise they are broken or partly broken. These patches don't affect
do_rootfs or do_populate_sdk without multilib are to be installed.

// Robert

On 09/16/2015 10:04 AM, Robert Yang wrote:
> * This fixed:
>    MACHINE = "qemux86-64"
>    require conf/multilib.conf
>    MULTILIBS = "multilib:lib32"
>    DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
>
>    $ bitbake core-image-minimal -cpopulate_sdk
>    Install poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh,
>    then source environment-setup-core2-64-pokymllib32-linux or
>    environment-setup-core2-64-poky-linux, both of them will compile hello.c
>    well. (The 32bit was broken in the past)
>
>    $ bitbake lib32-core-image-minimal -cpopulate_sdk
>    Install poky-glibc-x86_64-lib32-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh,
>    then source environment-setup-core2-64-poky-linux or
>    environment-setup-core2-64-pokymllib32-linux, both of them will compile
>    hello.c well (neither of them worked in the past)
>
>    This also fixed the problem that not all dependencies mutilib packages
>    are installed, please see:
>    https://bugzilla.yoctoproject.org/show_bug.cgi?id=4408
>
> Tested:
> $ bitbake core-image-minimal core-image-sato -cpopulate_sdk && bitbake core-image-sato-sdk world core-image-minimal core-image-sato
>
> // Robert
>
> The following changes since commit f0189829498e30231d826c9f55aad73e622d076e:
>
>    qemu: Update to upstream patches (2015-09-14 11:22:02 +0100)
>
> are available in the git repository at:
>
>    git://git.openembedded.org/openembedded-core-contrib rbt/sdk
>    http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/sdk
>
> Robert Yang (6):
>    toolchain-shar-extract.sh: remove checkbashism
>    oe-pkgdata-util: avoid returning skipped packages
>    package_manager.py: make rpm install mutilib pkgs corectly
>    multilib.bbclass: install all bits toochains when populate mlprefix
>      SDK
>    populate_sdk_base.bbclass: fix SDKTARGETSYSROOT when populate
>      mlprefix SDK
>    meta-environment.bb: fix environment-setup* when populate mlprefix
>      SDK
>
>   meta/classes/multilib.bbclass              |   15 +++++
>   meta/classes/populate_sdk_base.bbclass     |    6 +-
>   meta/files/toolchain-shar-extract.sh       |    4 +-
>   meta/lib/oe/package_manager.py             |   83 +++++++++++++++++++++++++++-
>   meta/recipes-core/meta/meta-environment.bb |    4 ++
>   scripts/oe-pkgdata-util                    |    5 +-
>   6 files changed, 111 insertions(+), 6 deletions(-)
>

Re: [PATCH 0/6] Fixes for mutilib SDK

[Richard Purdie]

On Tue, 2015-09-22 at 10:45 +0800, Robert Yang wrote:
> Hi RP and Ross,
> 
> These patches are required by multilib's do_rootfs and do_populate_sdk,
> otherwise they are broken or partly broken. These patches don't affect
> do_rootfs or do_populate_sdk without multilib are to be installed.

Thanks for these. Patches 1-4 look ok, we can try them in -next however
I do worry about 5 and 6. Could you confirm if you tested those before
or after the recent data store changes? Its possible the OVERRIDE fixes
there may have fixed the issues 5 and 6 were trying to fix so I'd like
to confirm if they're still needed or not.

If they are, they don't look like the right solution so I'll need to
look further at the exact issue there.

Cheers,

Richard

Re: [PATCH 0/6] Fixes for mutilib SDK

[Robert Yang]

On 09/24/2015 06:00 AM, Richard Purdie wrote:
> On Tue, 2015-09-22 at 10:45 +0800, Robert Yang wrote:
>> Hi RP and Ross,
>>
>> These patches are required by multilib's do_rootfs and do_populate_sdk,
>> otherwise they are broken or partly broken. These patches don't affect
>> do_rootfs or do_populate_sdk without multilib are to be installed.
>
> Thanks for these. Patches 1-4 look ok, we can try them in -next however
> I do worry about 5 and 6. Could you confirm if you tested those before
> or after the recent data store changes? Its possible the OVERRIDE fixes
> there may have fixed the issues 5 and 6 were trying to fix so I'd like
> to confirm if they're still needed or not.
>
> If they are, they don't look like the right solution so I'll need to
> look further at the exact issue there.

Hi RP,

I tested on latest master branch without patch 5 and 6, we still have
the issues.

# local.conf
MACHINE = "qemux86-64"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"

# populate sdk for lib32-core-image-minimal
$ bitbake lib32-core-image-minimal -cpopulate_sdk

# Install sdk
$ 
./tmp/deploy/sdk/poky-glibc-x86_64-lib32-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh

# Check messages on the screen:
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the 
environment setup script e.g.
  $ . /buildarea/lyang1/sdk32/environment-setup-core2-64-pokymllib32-linux

This is incorrect, it should be environment-setup-x86-pokymllib32-linuxm
(core2-64 != x86).

# Check the installed sdk, the sysroots are:
core2-64-pokymllib32-linux  x86_64-pokysdk-linux

But SDKTARGETSYSROOT in environment-setup-core2-64-poky-linux and
environment-setup-x86-pokymllib32-linux is /path/to/sysroots/core2-64-poky-linux
, this is incorrect.

// Robert

>
> Cheers,
>
> Richard
>
>

Re: [PATCH 0/6] Fixes for mutilib SDK

[Richard Purdie]

On Thu, 2015-09-24 at 14:29 +0800, Robert Yang wrote:
> 
> On 09/24/2015 06:00 AM, Richard Purdie wrote:
> > On Tue, 2015-09-22 at 10:45 +0800, Robert Yang wrote:
> >> Hi RP and Ross,
> >>
> >> These patches are required by multilib's do_rootfs and do_populate_sdk,
> >> otherwise they are broken or partly broken. These patches don't affect
> >> do_rootfs or do_populate_sdk without multilib are to be installed.
> >
> > Thanks for these. Patches 1-4 look ok, we can try them in -next however
> > I do worry about 5 and 6. Could you confirm if you tested those before
> > or after the recent data store changes? Its possible the OVERRIDE fixes
> > there may have fixed the issues 5 and 6 were trying to fix so I'd like
> > to confirm if they're still needed or not.
> >
> > If they are, they don't look like the right solution so I'll need to
> > look further at the exact issue there.
> 
> Hi RP,
> 
> I tested on latest master branch without patch 5 and 6, we still have
> the issues.
> 
> # local.conf
> MACHINE = "qemux86-64"
> require conf/multilib.conf
> MULTILIBS = "multilib:lib32"
> DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
> 
> # populate sdk for lib32-core-image-minimal
> $ bitbake lib32-core-image-minimal -cpopulate_sdk
> 
> # Install sdk
> $ 
> ./tmp/deploy/sdk/poky-glibc-x86_64-lib32-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh
> 
> # Check messages on the screen:
> SDK has been successfully set up and is ready to be used.
> Each time you wish to use the SDK in a new shell session, you need to source the 
> environment setup script e.g.
>   $ . /buildarea/lyang1/sdk32/environment-setup-core2-64-pokymllib32-linux
> 
> This is incorrect, it should be environment-setup-x86-pokymllib32-linuxm
> (core2-64 != x86).
> 
> # Check the installed sdk, the sysroots are:
> core2-64-pokymllib32-linux  x86_64-pokysdk-linux
> 
> But SDKTARGETSYSROOT in environment-setup-core2-64-poky-linux and
> environment-setup-x86-pokymllib32-linux is /path/to/sysroots/core2-64-poky-linux
> , this is incorrect.

Thanks, I will look into that further. With your first four patches
applied, the autobuilder showed this issue:

https://autobuilder.yoctoproject.org/main/builders/nightly-multilib/builds/500/steps/Running%20Sanity%20Tests_2/logs/stdio

I've not looked into what is going on there as yet.

Cheers,

Richard

Re: [PATCH 0/6] Fixes for mutilib SDK

[Robert Yang]

On 09/24/2015 05:18 PM, Richard Purdie wrote:
> On Thu, 2015-09-24 at 14:29 +0800, Robert Yang wrote:
>>
>> On 09/24/2015 06:00 AM, Richard Purdie wrote:
>>> On Tue, 2015-09-22 at 10:45 +0800, Robert Yang wrote:
>>>> Hi RP and Ross,
>>>>
>>>> These patches are required by multilib's do_rootfs and do_populate_sdk,
>>>> otherwise they are broken or partly broken. These patches don't affect
>>>> do_rootfs or do_populate_sdk without multilib are to be installed.
>>>
>>> Thanks for these. Patches 1-4 look ok, we can try them in -next however
>>> I do worry about 5 and 6. Could you confirm if you tested those before
>>> or after the recent data store changes? Its possible the OVERRIDE fixes
>>> there may have fixed the issues 5 and 6 were trying to fix so I'd like
>>> to confirm if they're still needed or not.
>>>
>>> If they are, they don't look like the right solution so I'll need to
>>> look further at the exact issue there.
>>
>> Hi RP,
>>
>> I tested on latest master branch without patch 5 and 6, we still have
>> the issues.
>>
>> # local.conf
>> MACHINE = "qemux86-64"
>> require conf/multilib.conf
>> MULTILIBS = "multilib:lib32"
>> DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
>>
>> # populate sdk for lib32-core-image-minimal
>> $ bitbake lib32-core-image-minimal -cpopulate_sdk
>>
>> # Install sdk
>> $
>> ./tmp/deploy/sdk/poky-glibc-x86_64-lib32-core-image-minimal-core2-64-toolchain-1.8+snapshot.sh
>>
>> # Check messages on the screen:
>> SDK has been successfully set up and is ready to be used.
>> Each time you wish to use the SDK in a new shell session, you need to source the
>> environment setup script e.g.
>>    $ . /buildarea/lyang1/sdk32/environment-setup-core2-64-pokymllib32-linux
>>
>> This is incorrect, it should be environment-setup-x86-pokymllib32-linuxm
>> (core2-64 != x86).
>>
>> # Check the installed sdk, the sysroots are:
>> core2-64-pokymllib32-linux  x86_64-pokysdk-linux
>>
>> But SDKTARGETSYSROOT in environment-setup-core2-64-poky-linux and
>> environment-setup-x86-pokymllib32-linux is /path/to/sysroots/core2-64-poky-linux
>> , this is incorrect.
>
> Thanks, I will look into that further. With your first four patches
> applied, the autobuilder showed this issue:
>
> https://autobuilder.yoctoproject.org/main/builders/nightly-multilib/builds/500/steps/Running%20Sanity%20Tests_2/logs/stdio
>
> I've not looked into what is going on there as yet.

Sorry, I looked at it just now, but I think that we need fix the test case,
the test case checks whether /usr/bin/connman-applet is 32bit, and I think
that the config is:

IMAGE_INSTALL_append = " lib32-connman-gnome"

Before these patches, only lib32-connman-gnome were installed which was
incorrect since connman-gnome should be installed, too (RDEPENDS by
packagegroup-core-x11-sato), this was similar to the bugs:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=4408
https://bugzilla.yoctoproject.org/show_bug.cgi?id=7610

Now these patches have fixed the problem, so both connman-gnome
and lib32-connman-gnome have installed, if we want to make sure
/usr/bin/connman-applet is 32bit, we need set:

RPM_PREFER_ELF_ARCH = "1"

If we'd like 64bit wins, we need set:

RPM_PREFER_ELF_ARCH = "2"

Maybe we need update our test cases or autobuilder's multilib
settings, please ?

// Robert

>
> Cheers,
>
> Richard
>
>
>

Re: [PATCH 0/6] Fixes for mutilib SDK

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

On 24 September 2015 at 12:29, Robert Yang <liezhi.yang@windriver.com>
wrote:

> Now these patches have fixed the problem, so both connman-gnome
> and lib32-connman-gnome have installed, if we want to make sure
> /usr/bin/connman-applet is 32bit, we need set:
>
> RPM_PREFER_ELF_ARCH = "1"
>
> If we'd like 64bit wins, we need set:
>
> RPM_PREFER_ELF_ARCH = "2"
>
> Maybe we need update our test cases or autobuilder's multilib
> settings, please ?
>

I've just sent a patch for the autobuilder to do this.

Ross

[-- Attachment #2: Type: text/html, Size: 998 bytes --]