[PATCH 0/3] Image/SDK multilib fixes

[PATCH 0/3] Image/SDK multilib fixes

[Mark Hatle]

The image generation for RPM needs to be synced with the python fragement in
the populate_sdk case.  Otherwise the multilib configuations get out of sync.

Similar changes are NOT needed for deb or ipkg.

We also needed to enable multilib remapping when the populate_sdk_base is
loaded, otherwise we don't get matching package and sdk images.

Finally to avoid conflicts in the SDK generation, we compute the install 
solutions by adding --replacepkgs to the temporary database.  This does
not change the installed image behavior in any way.

The following changes since commit 8f16811a8d51982a8b3d70e6087aef4a41926840:

  bootimg: Use STAGING_KERNEL_DIR (2012-07-06 09:38:17 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib mhatle/sdk
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/sdk

Mark Hatle (3):
  populate_sdk_rpm: Sync multilib configuration with rootfs_rpm
  multilib: Enable multilib remapping for SDK generation
  package_rpm: Avoid duplicate package generation failures

 meta/classes/multilib.bbclass         |    6 ++++++
 meta/classes/package_rpm.bbclass      |   10 +++++-----
 meta/classes/populate_sdk_rpm.bbclass |   15 ++++++++++-----
 meta/classes/rootfs_rpm.bbclass       |    1 +
 4 files changed, 22 insertions(+), 10 deletions(-)

-- 
1.7.3.4

[PATCH 1/3] populate_sdk_rpm: Sync multilib configuration with rootfs_rpm

[Mark Hatle]

An anonymous python chunk configures some multilib settings for both
populate_sdk_rpm and rootfs_rpm.  The two classes should contain identical
versions to ensure that the generated multilib configuration is the same
for the SDK and the rootfs.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/populate_sdk_rpm.bbclass |   15 ++++++++++-----
 meta/classes/rootfs_rpm.bbclass       |    1 +
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/meta/classes/populate_sdk_rpm.bbclass b/meta/classes/populate_sdk_rpm.bbclass
index b115c27..365a337 100644
--- a/meta/classes/populate_sdk_rpm.bbclass
+++ b/meta/classes/populate_sdk_rpm.bbclass
@@ -115,18 +115,23 @@ EOF
 }
 
 python () {
+    # The following code should be kept in sync w/ the rootfs_rpm version.
     ml_package_archs = ""
+    ml_prefix_list = ""
     multilibs = d.getVar('MULTILIBS', True) or ""
     for ext in multilibs.split():
         eext = ext.split(':')
         if len(eext) > 1 and eext[0] == 'multilib':
             localdata = bb.data.createCopy(d)
-            overrides = localdata.getVar("OVERRIDES", False) + ":virtclass-multilib-" + eext[1]
-            localdata.setVar("OVERRIDES", overrides)
-            # TEMP: OVERRIDES isn't working right
-            localdata.setVar("DEFAULTTUNE", localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False) or "")
-            ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
+            default_tune = localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False)
+            if default_tune:
+                localdata.setVar("DEFAULTTUNE", default_tune)
+            package_archs = localdata.getVar("PACKAGE_ARCHS", True) or ""
+            package_archs = " ".join([i in "all noarch any".split() and i or eext[1]+"_"+i for i in package_archs.split()])
+            ml_package_archs += " " + package_archs
+            ml_prefix_list += " " + eext[1]
             #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
     d.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs)
+    d.setVar('MULTILIB_PREFIX_LIST', ml_prefix_list)
 }
 
diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index 202f9ce..4551f7a 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -198,6 +198,7 @@ python () {
         d.setVar('RPM_PREPROCESS_COMMANDS', '')
         d.setVar('RPM_POSTPROCESS_COMMANDS', '')
 
+    # The following code should be kept in sync w/ the populate_sdk_rpm version.
     ml_package_archs = ""
     ml_prefix_list = ""
     multilibs = d.getVar('MULTILIBS', True) or ""
-- 
1.7.3.4

[PATCH 2/3] multilib: Enable multilib remapping for SDK generation

[Mark Hatle]

Enable the remapping for SDK generation, this is required to be able to
create an SDK that targets an alternative multilib.  Note, this work does
not finish SDK/multilib support, but it is one more step toward making
it work properly.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/multilib.bbclass |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/meta/classes/multilib.bbclass b/meta/classes/multilib.bbclass
index f3e7618..b92a3ff 100644
--- a/meta/classes/multilib.bbclass
+++ b/meta/classes/multilib.bbclass
@@ -54,6 +54,12 @@ python __anonymous () {
         d.setVar("LINGUAS_INSTALL", "")
         # FIXME, we need to map this to something, not delete it!
         d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
+
+    if bb.data.inherits_class('populate_sdk_base', d):
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
+
+    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
         return
 
     clsextend.rename_packages()
-- 
1.7.3.4

[PATCH 3/3] package_rpm: Avoid duplicate package generation failures

[Mark Hatle]

When constructing a multilib based image, it's possible to end up with
the same package listed in multiple steps of the install.  During resolution
we use the --replacepkgs option to avoid errors, as the resulting install
solution will be as expected.  (We do not enable the replacepkgs option
for the final install step, only the generation of the install solution.)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/package_rpm.bbclass |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 78279f5..2a29917 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -318,7 +318,7 @@ package_install_internal_rpm () {
 			--root "${target_rootfs}/install" \
 			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}-base_archs.macro`" \
 			-D "__dbi_txn create nofsync" \
-			-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+			-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 			${target_rootfs}/install/install.manifest
 	fi
 
@@ -348,7 +348,7 @@ package_install_internal_rpm () {
 				--root "${target_rootfs}/install" \
 				-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
 				-D "__dbi_txn create nofsync private" \
-				-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 			$pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}" || true
 		done
 	fi
@@ -399,7 +399,7 @@ package_install_internal_rpm () {
 				--root "${target_rootfs}/install" \
 				-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
 				-D "__dbi_txn create nofsync private" \
-				-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 				$pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_recommend.${PID}" 2>&1 || true
 		done
 		cat ${target_rootfs}/install/recommend.list ${target_rootfs}/install/recommend.new | sort -u > ${target_rootfs}/install/recommend.new.list
@@ -428,7 +428,7 @@ package_install_internal_rpm () {
 			--root "${target_rootfs}/install" \
 			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}-ml_archs.macro`" \
 			-D "__dbi_txn create nofsync" \
-			-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+			-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 			${target_rootfs}/install/install_multilib.manifest
 
 		# Now that we have a solution, pull out a list of what to install...
@@ -524,7 +524,7 @@ EOF
 				--root "${target_rootfs}/install" \
 				-D "_dbpath ${target_rootfs}/initial" -D "`cat ${confbase}.macro`" \
 				-D "__dbi_txn create nofsync" \
-				-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 				${target_rootfs}/install/initial_install.manifest
 
 			${RPM} -D "_dbpath ${target_rootfs}/initial" -qa --qf "%{packageorigin}\n" \
-- 
1.7.3.4

Re: [PATCH 2/3] multilib: Enable multilib remapping for SDK generation

[Richard Purdie]

On Fri, 2012-07-06 at 14:14 -0500, Mark Hatle wrote:
> Enable the remapping for SDK generation, this is required to be able to
> create an SDK that targets an alternative multilib.  Note, this work does
> not finish SDK/multilib support, but it is one more step toward making
> it work properly.
> 
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
>  meta/classes/multilib.bbclass |    6 ++++++
>  1 files changed, 6 insertions(+), 0 deletions(-)
> 
> diff --git a/meta/classes/multilib.bbclass b/meta/classes/multilib.bbclass
> index f3e7618..b92a3ff 100644
> --- a/meta/classes/multilib.bbclass
> +++ b/meta/classes/multilib.bbclass
> @@ -54,6 +54,12 @@ python __anonymous () {
>          d.setVar("LINGUAS_INSTALL", "")
>          # FIXME, we need to map this to something, not delete it!
>          d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
> +
> +    if bb.data.inherits_class('populate_sdk_base', d):
> +        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
> +        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
> +
> +    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
>          return
>  

There is something odd going on with this patch. You've removed the
earlier return but this patch doesn't show that...

Cheers,

Richard

Re: [PATCH 2/3] multilib: Enable multilib remapping for SDK generation

[Mark Hatle]

On 7/9/12 5:39 AM, Richard Purdie wrote:
> On Fri, 2012-07-06 at 14:14 -0500, Mark Hatle wrote:
>> Enable the remapping for SDK generation, this is required to be able to
>> create an SDK that targets an alternative multilib.  Note, this work does
>> not finish SDK/multilib support, but it is one more step toward making
>> it work properly.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>> ---
>>   meta/classes/multilib.bbclass |    6 ++++++
>>   1 files changed, 6 insertions(+), 0 deletions(-)
>>
>> diff --git a/meta/classes/multilib.bbclass b/meta/classes/multilib.bbclass
>> index f3e7618..b92a3ff 100644
>> --- a/meta/classes/multilib.bbclass
>> +++ b/meta/classes/multilib.bbclass
>> @@ -54,6 +54,12 @@ python __anonymous () {
>>           d.setVar("LINGUAS_INSTALL", "")
>>           # FIXME, we need to map this to something, not delete it!
>>           d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
>> +
>> +    if bb.data.inherits_class('populate_sdk_base', d):
>> +        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
>> +        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
>> +
>> +    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
>>           return
>>
>
> There is something odd going on with this patch. You've removed the
> earlier return but this patch doesn't show that...

I didn't remove it, I 'moved' it later.  Otherwise the populate_sdk_base case is 
never run, when the image case has already been loaded.

So the flow goes from:

if image class:
   do work
   return

do other stuff

to

if image class:
   do work

if sdk class:
   do other work

if either image or sdk class:
   return

do other stuff

The behavior is the same for the image only class, the only new behavior is 
adding in the remap for the SDK only (or combined w/ the image) case.

--Mark

> Cheers,
>
> Richard
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: [PATCH 2/3] multilib: Enable multilib remapping for SDK generation

[Richard Purdie]

On Mon, 2012-07-09 at 08:08 -0500, Mark Hatle wrote:
> On 7/9/12 5:39 AM, Richard Purdie wrote:
> > On Fri, 2012-07-06 at 14:14 -0500, Mark Hatle wrote:
> >> Enable the remapping for SDK generation, this is required to be able to
> >> create an SDK that targets an alternative multilib.  Note, this work does
> >> not finish SDK/multilib support, but it is one more step toward making
> >> it work properly.
> >>
> >> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> >> ---
> >>   meta/classes/multilib.bbclass |    6 ++++++
> >>   1 files changed, 6 insertions(+), 0 deletions(-)
> >>
> >> diff --git a/meta/classes/multilib.bbclass b/meta/classes/multilib.bbclass
> >> index f3e7618..b92a3ff 100644
> >> --- a/meta/classes/multilib.bbclass
> >> +++ b/meta/classes/multilib.bbclass
> >> @@ -54,6 +54,12 @@ python __anonymous () {
> >>           d.setVar("LINGUAS_INSTALL", "")
> >>           # FIXME, we need to map this to something, not delete it!
> >>           d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
> >> +
> >> +    if bb.data.inherits_class('populate_sdk_base', d):
> >> +        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
> >> +        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
> >> +
> >> +    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
> >>           return
> >>
> >
> > There is something odd going on with this patch. You've removed the
> > earlier return but this patch doesn't show that...
> 
> I didn't remove it, I 'moved' it later.  Otherwise the populate_sdk_base case is 
> never run, when the image case has already been loaded.
> 
> So the flow goes from:

I understand this. Let me put this in a simpler way. Could you send a
patch that cleanly applies against master please? There is something
wrong with it and it doesn't apply.

Cheers,

Richard

Re: [PATCH 0/3] Image/SDK multilib fixes

[Saul Wold]

On 07/06/2012 12:13 PM, Mark Hatle wrote:
> The image generation for RPM needs to be synced with the python fragement in
> the populate_sdk case.  Otherwise the multilib configuations get out of sync.
>
> Similar changes are NOT needed for deb or ipkg.
>
> We also needed to enable multilib remapping when the populate_sdk_base is
> loaded, otherwise we don't get matching package and sdk images.
>
> Finally to avoid conflicts in the SDK generation, we compute the install
> solutions by adding --replacepkgs to the temporary database.  This does
> not change the installed image behavior in any way.
>
> The following changes since commit 8f16811a8d51982a8b3d70e6087aef4a41926840:
>
>    bootimg: Use STAGING_KERNEL_DIR (2012-07-06 09:38:17 +0100)
>
> are available in the git repository at:
>    git://git.pokylinux.org/poky-contrib mhatle/sdk
>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/sdk
>
> Mark Hatle (3):
>    populate_sdk_rpm: Sync multilib configuration with rootfs_rpm
>    multilib: Enable multilib remapping for SDK generation
>    package_rpm: Avoid duplicate package generation failures
>
>   meta/classes/multilib.bbclass         |    6 ++++++
>   meta/classes/package_rpm.bbclass      |   10 +++++-----
>   meta/classes/populate_sdk_rpm.bbclass |   15 ++++++++++-----
>   meta/classes/rootfs_rpm.bbclass       |    1 +
>   4 files changed, 22 insertions(+), 10 deletions(-)
>

Merged into OE-Core with the updated request

Thanks
	Sau!