[PATCH 0/3] populate_sdk_base.bbclass: fixes for multilib sdk

[PATCH 0/3] populate_sdk_base.bbclass: fixes for multilib sdk

[Robert Yang]

Fixed for multilib sdk:
MACHINE ?= "qemux86-64"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"

$ bitbake core-image-minimal -cpopulate_sdk 

Extrat the sdk, both environment-setup-core2-64-poky-linux and
environment-setup-x86-pokymllib32-linux can compile hello.c after
sourced.

This has fixed "bitbake core-image-minimal -cpopulate_sdk", but the sdk
of "bitbake lib32-core-image-minimal -cpopulate_sdk" doesn't work, I'm
not sure what is preferred, include both 32 and 64 bit sdk as "bitbake
core-image-minimal -cpopulate_sdk" does or only 32bit ? What's your
opinion, please ?

// Robert

The following changes since commit cb8d309ff0ea0ca11edc2aae75ddd869491cb330:

  perf: fix build breakage on kernels after 4.1 (2015-08-12 11:31:01 -0700)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/sdk
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/sdk

Robert Yang (3):
  toolchain-shar-extract.sh: add a space in the end
  populate_sdk_base.bbclass: help rpm to pull in multilib toolchain
  packagegroup-core-standalone-sdk-target: remove qemuwrapper-cross

 meta/classes/populate_sdk_base.bbclass             |   32 ++++++++++++++++++++
 meta/files/toolchain-shar-extract.sh               |    4 +--
 .../packagegroup-core-standalone-sdk-target.bb     |    1 -
 3 files changed, 34 insertions(+), 3 deletions(-)

-- 
1.7.9.5

[PATCH 1/3] toolchain-shar-extract.sh: add a space in the end

[Robert Yang]

For a clear look when input.

[YOCTO #8089]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/files/toolchain-shar-extract.sh |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/files/toolchain-shar-extract.sh b/meta/files/toolchain-shar-extract.sh
index 0a33ee8..3a50991 100644
--- a/meta/files/toolchain-shar-extract.sh
+++ b/meta/files/toolchain-shar-extract.sh
@@ -92,11 +92,11 @@ fi
 
 if [ -e "$target_sdk_dir/environment-setup-@REAL_MULTIMACH_TARGET_SYS@" ]; then
 	echo "The directory \"$target_sdk_dir\" already contains a SDK for this architecture."
-	printf "If you continue, existing files will be overwritten! Proceed[y/N]?"
+	printf "If you continue, existing files will be overwritten! Proceed[y/N]? "
 
 	default_answer="n"
 else
-	printf "You are about to install the SDK to \"$target_sdk_dir\". Proceed[Y/n]?"
+	printf "You are about to install the SDK to \"$target_sdk_dir\". Proceed[Y/n]? "
 
 	default_answer="y"
 fi
-- 
1.7.9.5

[PATCH 2/3] populate_sdk_base.bbclass: help rpm to pull in multilib toolchain

[Robert Yang]

When configure multilib, "bitbake <image_bb> -c populate_sdk" should
install all arch toolchains (for example, 32 and 64bit), but rpm not
handle the multilib requires correctly, for example:
lib32-packagegroup-core-standalone-sdk-target requires lib32-libc6, rpm
may pull in libc6 rather than lib32-libc6, if we directly use something
like IMAGE_INSTALL_append = " lib32-libc6", then it works well, so
update TOOLCHAIN_TARGET_TASK to include the RDEPENDS packages will fix
the issue.

[YOCTO #8089]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/populate_sdk_base.bbclass |   32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index a9e9bd7..c2491ea 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -83,8 +83,40 @@ POPULATE_SDK_POST_HOST_COMMAND_append = " write_host_sdk_manifest; "
 fakeroot python do_populate_sdk() {
     from oe.sdk import populate_sdk
     from oe.manifest import create_manifest, Manifest
+    import oe.packagedata
 
     pn = d.getVar('PN', True)
+    # Help rpm to pull in multilib requires since it may not pull in
+    # them correctly, for example,
+    # lib32-packagegroup-core-standalone-sdk-target requires
+    # lib32-libc6, rpm may pull in libc6 rather than lib32-libc6, but
+    # directly use
+    # TOOLCHAIN_TARGET_TASK_append = " lib32-libc6" works well.
+    for pkg in (d.getVar('TOOLCHAIN_TARGET_TASK', True) or "").split():
+        sub_data = oe.packagedata.read_subpkgdata(pkg, d)
+        sub_rdep = sub_data.get("RDEPENDS_" + pkg)
+        if not sub_rdep:
+            continue
+        done = sub_rdep.split()
+        next = done
+        # Find all the rdepends on the dependency chain
+        while next:
+            new = []
+            for sub_pkg in next:
+                sub_data = oe.packagedata.read_subpkgdata(sub_pkg, d)
+                sub_pkg_rdep = sub_data.get("RDEPENDS_" + sub_pkg)
+                if not sub_pkg_rdep:
+                    continue
+                for p in sub_pkg_rdep.split():
+                    if p in done:
+                        continue
+                    if not p.startswith('(') and \
+                            oe.packagedata.has_subpkgdata(p, d):
+                        # It's a new rdep
+                        done.append(p)
+                        new.append(p)
+            next = new
+        d.appendVar('TOOLCHAIN_TARGET_TASK', ' ' + " ".join(done))
     runtime_mapping_rename("TOOLCHAIN_TARGET_TASK", pn, d)
     runtime_mapping_rename("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY", pn, d)
 
-- 
1.7.9.5

[PATCH 3/3] packagegroup-core-standalone-sdk-target: remove qemuwrapper-cross

[Robert Yang]

Remove qemuwrapper-cross from RDEPENDS, install a cross pkg in sysroots
isn't useful, if we really need run qemuwrapper in SDK, we should add it
as nativesdk, and it has multilib conflicts when populate_ sdk:

error: file /usr/bin/crossscripts/qemuwrapper from install of
qemuwrapper-cross-1.0-r0.lib32_x86 conflicts with file from package
qemuwrapper-cross-1.0-r0.core2_64

[YOCTO #8089]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 .../packagegroup-core-standalone-sdk-target.bb     |    1 -
 1 file changed, 1 deletion(-)

diff --git a/meta/recipes-core/packagegroups/packagegroup-core-standalone-sdk-target.bb b/meta/recipes-core/packagegroups/packagegroup-core-standalone-sdk-target.bb
index 5d1ce97..37f5e43 100644
--- a/meta/recipes-core/packagegroups/packagegroup-core-standalone-sdk-target.bb
+++ b/meta/recipes-core/packagegroups/packagegroup-core-standalone-sdk-target.bb
@@ -10,5 +10,4 @@ RDEPENDS_${PN} = "\
     libstdc++ \
     libstdc++-dev \
     ${LIBC_DEPENDENCIES} \
-    qemuwrapper-cross \
     "
-- 
1.7.9.5

Re: [PATCH 1/3] toolchain-shar-extract.sh: add a space in the end

[Randy MacLeod]

Nit pick on logs...

In the short log:
s/add a space in the end/
  /Add a space after asking a question/

On 2015-08-13 03:37 AM, Robert Yang wrote:
> For a clear look when input.
This is a sentence fragment. It will read better if you repeat yourself:

Add a space at the end of a line when asking a question.
This makes it easier to read.

../Randy
>
> [YOCTO #8089]
>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>   meta/files/toolchain-shar-extract.sh |    4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/files/toolchain-shar-extract.sh b/meta/files/toolchain-shar-extract.sh
> index 0a33ee8..3a50991 100644
> --- a/meta/files/toolchain-shar-extract.sh
> +++ b/meta/files/toolchain-shar-extract.sh
> @@ -92,11 +92,11 @@ fi
>
>   if [ -e "$target_sdk_dir/environment-setup-@REAL_MULTIMACH_TARGET_SYS@" ]; then
>   	echo "The directory \"$target_sdk_dir\" already contains a SDK for this architecture."
> -	printf "If you continue, existing files will be overwritten! Proceed[y/N]?"
> +	printf "If you continue, existing files will be overwritten! Proceed[y/N]? "
>
>   	default_answer="n"
>   else
> -	printf "You are about to install the SDK to \"$target_sdk_dir\". Proceed[Y/n]?"
> +	printf "You are about to install the SDK to \"$target_sdk_dir\". Proceed[Y/n]? "
>
>   	default_answer="y"
>   fi
>


-- 
# Randy MacLeod. SMTS, Linux, Wind River
Direct: 613.963.1350 | 350 Terry Fox Drive, Suite 200, Ottawa, ON, 
Canada, K2K 2W5

Re: [PATCH 2/3] populate_sdk_base.bbclass: help rpm to pull in multilib toolchain

[Richard Purdie]

On Thu, 2015-08-13 at 00:37 -0700, Robert Yang wrote:
> When configure multilib, "bitbake <image_bb> -c populate_sdk" should
> install all arch toolchains (for example, 32 and 64bit), but rpm not
> handle the multilib requires correctly, for example:
> lib32-packagegroup-core-standalone-sdk-target requires lib32-libc6, rpm
> may pull in libc6 rather than lib32-libc6, if we directly use something
> like IMAGE_INSTALL_append = " lib32-libc6", then it works well, so
> update TOOLCHAIN_TARGET_TASK to include the RDEPENDS packages will fix
> the issue.

Whilst I understand why you're doing this, its rather sad that as the
rpm multilib code is broken, we have to hack the core sdk code like this
to work around it.

Could/should we not put this into the rpm sdk backend instead?

I also can't help this is just a bandaid and that we'll likely find
other ways this breaks in the future, its not a complete solution to the
issue :/.

Cheers,

Richard

> [YOCTO #8089]
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/classes/populate_sdk_base.bbclass |   32 ++++++++++++++++++++++++++++++++
>  1 file changed, 32 insertions(+)
> 
> diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
> index a9e9bd7..c2491ea 100644
> --- a/meta/classes/populate_sdk_base.bbclass
> +++ b/meta/classes/populate_sdk_base.bbclass
> @@ -83,8 +83,40 @@ POPULATE_SDK_POST_HOST_COMMAND_append = " write_host_sdk_manifest; "
>  fakeroot python do_populate_sdk() {
>      from oe.sdk import populate_sdk
>      from oe.manifest import create_manifest, Manifest
> +    import oe.packagedata
>  
>      pn = d.getVar('PN', True)
> +    # Help rpm to pull in multilib requires since it may not pull in
> +    # them correctly, for example,
> +    # lib32-packagegroup-core-standalone-sdk-target requires
> +    # lib32-libc6, rpm may pull in libc6 rather than lib32-libc6, but
> +    # directly use
> +    # TOOLCHAIN_TARGET_TASK_append = " lib32-libc6" works well.
> +    for pkg in (d.getVar('TOOLCHAIN_TARGET_TASK', True) or "").split():
> +        sub_data = oe.packagedata.read_subpkgdata(pkg, d)
> +        sub_rdep = sub_data.get("RDEPENDS_" + pkg)
> +        if not sub_rdep:
> +            continue
> +        done = sub_rdep.split()
> +        next = done
> +        # Find all the rdepends on the dependency chain
> +        while next:
> +            new = []
> +            for sub_pkg in next:
> +                sub_data = oe.packagedata.read_subpkgdata(sub_pkg, d)
> +                sub_pkg_rdep = sub_data.get("RDEPENDS_" + sub_pkg)
> +                if not sub_pkg_rdep:
> +                    continue
> +                for p in sub_pkg_rdep.split():
> +                    if p in done:
> +                        continue
> +                    if not p.startswith('(') and \
> +                            oe.packagedata.has_subpkgdata(p, d):
> +                        # It's a new rdep
> +                        done.append(p)
> +                        new.append(p)
> +            next = new
> +        d.appendVar('TOOLCHAIN_TARGET_TASK', ' ' + " ".join(done))
>      runtime_mapping_rename("TOOLCHAIN_TARGET_TASK", pn, d)
>      runtime_mapping_rename("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY", pn, d)
>  
> -- 
> 1.7.9.5
> 

Re: [PATCH 2/3] populate_sdk_base.bbclass: help rpm to pull in multilib toolchain

[Robert Yang]

On 08/15/2015 04:55 PM, Richard Purdie wrote:
> On Thu, 2015-08-13 at 00:37 -0700, Robert Yang wrote:
>> When configure multilib, "bitbake <image_bb> -c populate_sdk" should
>> install all arch toolchains (for example, 32 and 64bit), but rpm not
>> handle the multilib requires correctly, for example:
>> lib32-packagegroup-core-standalone-sdk-target requires lib32-libc6, rpm
>> may pull in libc6 rather than lib32-libc6, if we directly use something
>> like IMAGE_INSTALL_append = " lib32-libc6", then it works well, so
>> update TOOLCHAIN_TARGET_TASK to include the RDEPENDS packages will fix
>> the issue.
>
> Whilst I understand why you're doing this, its rather sad that as the
> rpm multilib code is broken, we have to hack the core sdk code like this
> to work around it.
>
> Could/should we not put this into the rpm sdk backend instead?

Sorry, I can't understand what did you mean, do you mean put this into rpm
sdk backend such as meta/lib/oe/sdk.py ?

// Robert

>
> I also can't help this is just a bandaid and that we'll likely find
> other ways this breaks in the future, its not a complete solution to the
> issue :/.
>
> Cheers,
>
> Richard
>
>> [YOCTO #8089]
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/classes/populate_sdk_base.bbclass |   32 ++++++++++++++++++++++++++++++++
>>   1 file changed, 32 insertions(+)
>>
>> diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
>> index a9e9bd7..c2491ea 100644
>> --- a/meta/classes/populate_sdk_base.bbclass
>> +++ b/meta/classes/populate_sdk_base.bbclass
>> @@ -83,8 +83,40 @@ POPULATE_SDK_POST_HOST_COMMAND_append = " write_host_sdk_manifest; "
>>   fakeroot python do_populate_sdk() {
>>       from oe.sdk import populate_sdk
>>       from oe.manifest import create_manifest, Manifest
>> +    import oe.packagedata
>>
>>       pn = d.getVar('PN', True)
>> +    # Help rpm to pull in multilib requires since it may not pull in
>> +    # them correctly, for example,
>> +    # lib32-packagegroup-core-standalone-sdk-target requires
>> +    # lib32-libc6, rpm may pull in libc6 rather than lib32-libc6, but
>> +    # directly use
>> +    # TOOLCHAIN_TARGET_TASK_append = " lib32-libc6" works well.
>> +    for pkg in (d.getVar('TOOLCHAIN_TARGET_TASK', True) or "").split():
>> +        sub_data = oe.packagedata.read_subpkgdata(pkg, d)
>> +        sub_rdep = sub_data.get("RDEPENDS_" + pkg)
>> +        if not sub_rdep:
>> +            continue
>> +        done = sub_rdep.split()
>> +        next = done
>> +        # Find all the rdepends on the dependency chain
>> +        while next:
>> +            new = []
>> +            for sub_pkg in next:
>> +                sub_data = oe.packagedata.read_subpkgdata(sub_pkg, d)
>> +                sub_pkg_rdep = sub_data.get("RDEPENDS_" + sub_pkg)
>> +                if not sub_pkg_rdep:
>> +                    continue
>> +                for p in sub_pkg_rdep.split():
>> +                    if p in done:
>> +                        continue
>> +                    if not p.startswith('(') and \
>> +                            oe.packagedata.has_subpkgdata(p, d):
>> +                        # It's a new rdep
>> +                        done.append(p)
>> +                        new.append(p)
>> +            next = new
>> +        d.appendVar('TOOLCHAIN_TARGET_TASK', ' ' + " ".join(done))
>>       runtime_mapping_rename("TOOLCHAIN_TARGET_TASK", pn, d)
>>       runtime_mapping_rename("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY", pn, d)
>>
>> --
>> 1.7.9.5
>>
>
>
>
>

Re: [PATCH 1/3] toolchain-shar-extract.sh: add a space in the end

[Robert Yang]

On 08/15/2015 09:23 AM, Randy MacLeod wrote:
> Nit pick on logs...
>
> In the short log:
> s/add a space in the end/
>   /Add a space after asking a question/
>
> On 2015-08-13 03:37 AM, Robert Yang wrote:
>> For a clear look when input.
> This is a sentence fragment. It will read better if you repeat yourself:
>
> Add a space at the end of a line when asking a question.
> This makes it easier to read.

Thanks, I will update it.

// Robert

>
> ../Randy
>>
>> [YOCTO #8089]
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/files/toolchain-shar-extract.sh |    4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/files/toolchain-shar-extract.sh
>> b/meta/files/toolchain-shar-extract.sh
>> index 0a33ee8..3a50991 100644
>> --- a/meta/files/toolchain-shar-extract.sh
>> +++ b/meta/files/toolchain-shar-extract.sh
>> @@ -92,11 +92,11 @@ fi
>>
>>   if [ -e "$target_sdk_dir/environment-setup-@REAL_MULTIMACH_TARGET_SYS@" ]; then
>>       echo "The directory \"$target_sdk_dir\" already contains a SDK for this
>> architecture."
>> -    printf "If you continue, existing files will be overwritten! Proceed[y/N]?"
>> +    printf "If you continue, existing files will be overwritten! Proceed[y/N]? "
>>
>>       default_answer="n"
>>   else
>> -    printf "You are about to install the SDK to \"$target_sdk_dir\".
>> Proceed[Y/n]?"
>> +    printf "You are about to install the SDK to \"$target_sdk_dir\".
>> Proceed[Y/n]? "
>>
>>       default_answer="y"
>>   fi
>>
>
>