[PATCH v2][2/2] image.bbclass/rootfs.py: use PREFERRED_RPROVIDER to select package

[PATCH v2][2/2] image.bbclass/rootfs.py: use PREFERRED_RPROVIDER to select package

[Jate Sujjavanich]

[-- Attachment #1: Type: text/plain, Size: 2378 bytes --]

The following changes since commit 55bd8d189f647443b28f988ccfc9eebffaec84d1:

  providers.py: Add PREFERRED_RPROVIDER (2015-05-13 11:29:52 -0400)

are available in the git repository at:

  git://github.com/jatedev/poky.git preferred-provider-runtime-2

for you to fetch changes up to 832a47ff79a06a797ddd39db86c792a43cdf0865:

  image.bbclass/rootfs.py: use PREFERRED_RPROVIDER to select package
(2015-05-13 12:24:02 -0400)

----------------------------------------------------------------
Jate Sujjavanich (1):
      image.bbclass/rootfs.py: use PREFERRED_RPROVIDER to select package

 meta/classes/image.bbclass |   14 ++++++++++++++
 meta/lib/oe/rootfs.py      |    2 +-
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 01f8b3f..80484d6 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -283,6 +283,20 @@ python rootfs_process_ignore() {
 }
 do_rootfs[prefuncs] += "rootfs_process_ignore"

+python rootfs_process_preferred_rproviders() {
+    inst_pkgs = d.getVar("PACKAGE_INSTALL", True).split()
+    pref_pkgs = list()
+    for pkg in inst_pkgs:
+        prefervar = d.getVar("PREFERRED_RPROVIDER_%s" % pkg, True)
+        if prefervar:
+            inst_pkgs.remove(pkg)
+            pref_pkgs.append(prefervar)
+            bb.note("Selecting %s to provide %s due to
PREFERRED_RPROVIDER" % (prefervar, pkg))
+    inst_pkgs.extend(pref_pkgs)
+    d.setVar("PACKAGE_INSTALL", ' '.join(inst_pkgs))
+}
+do_rootfs[prefuncs] += "rootfs_process_preferred_rproviders"
+
 # We have to delay the runtime_mapping_rename until just before rootfs runs
 # otherwise, the multilib renaming could step in and squash any fixups that
 # may have occurred.
diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index 6fb749f..4bad7ef 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -393,7 +393,7 @@ class RpmRootfs(Rootfs):
         pass

     def _log_check_error(self):
-        r = re.compile('(unpacking of archive failed|Cannot find
package|exit 1|ERR|Fail)')
+        r = re.compile('(unpacking of archive failed|Cannot find
package|exit 1|ERR(?!ED_RPROVIDER)|Fail)')
         log_path = self.d.expand("${T}/log.do_rootfs")
         with open(log_path, 'r') as log:
             found_error = 0

[-- Attachment #2: Type: text/html, Size: 2810 bytes --]

Re: [PATCH v2][2/2] image.bbclass/rootfs.py: use PREFERRED_RPROVIDER to select package

[Richard Purdie]

On Wed, 2015-05-13 at 18:47 -0400, Jate Sujjavanich wrote:
> +python rootfs_process_preferred_rproviders() {
> +    inst_pkgs = d.getVar("PACKAGE_INSTALL", True).split()
> +    pref_pkgs = list()
> +    for pkg in inst_pkgs:
> +        prefervar = d.getVar("PREFERRED_RPROVIDER_%s" % pkg, True)
> +        if prefervar:
> +            inst_pkgs.remove(pkg)
> +            pref_pkgs.append(prefervar)
> +            bb.note("Selecting %s to provide %s due to
> PREFERRED_RPROVIDER" % (prefervar, pkg))
> +    inst_pkgs.extend(pref_pkgs)
> +    d.setVar("PACKAGE_INSTALL", ' '.join(inst_pkgs))
> +}
> +do_rootfs[prefuncs] += "rootfs_process_preferred_rproviders"
> +
>  # We have to delay the runtime_mapping_rename until just before
> rootfs runs
>  # otherwise, the multilib renaming could step in and squash any
> fixups that
>  # may have occurred.

This kind of approach to the problem basically doesn't scale. Yes, it
will rewrite things in PACKAGE_INSTALL however if package A depends on
package B using one of these names, the mechanism will simply not work.
This will in turn generate a whole new world of bug reports with no way
to really fix the problems.

This is why we have other mechanisms for handling the runtime provider
names, usually through parametrisation with variables so that when you
rewrite an entry, *all* entries get rewritten and what the package
manager sees is consistent.

I would also note that in the past, bitbake once did have
PREFERRED_RPROVIDER support. We dropped it as it caused problems. I
suspect I would have written up some rationale for that when we did it
which would be in the list archives.

Cheers,

Richard

Re: [PATCH v2][2/2] image.bbclass/rootfs.py: use PREFERRED_RPROVIDER to select package

[Jate Sujjavanich]

[-- Attachment #1: Type: text/plain, Size: 3103 bytes --]

I have taken a look at the code, and there two places that I have found so
far that contain the RPROVIDES_* for each package. One is the taskdata at
the bitbake level, and the other is the pkgdata directory within the
sysroot for the machine's build. The taskdata seems more readily accessible
in memory. However, I've neither figured out a way to access it from the
image recipe namespace, nor do I know if it's intended to be accessed at
the recipe level. On the other hand, the pkgdata directory holds
information about all built packages, and I've written some code derived
from packageinfo.bbclass (used in hob) that loads all the package
information into memory.

For the case provided in bug 6149, the code could access the
PREFERRED_PROVIDER for the missing sshd package, search the pkgdata, and
then replace 'sshd' within PACKAGE_INSTALL with 'dropbear' as specified
within RPROVIDES_dropbear = "ssh sshd".

A problem is the inefficiency of parsing through the pkgdata files. Maybe a
rewrite of IMAGE_INSTALL be done during the calculations taskdata.py
instead, or maybe the name of the package rproviding the item needs to be
saved and referred to during the image generation.


- Jate


On Thu, May 14, 2015 at 5:46 AM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Wed, 2015-05-13 at 18:47 -0400, Jate Sujjavanich wrote:
> > +python rootfs_process_preferred_rproviders() {
> > +    inst_pkgs = d.getVar("PACKAGE_INSTALL", True).split()
> > +    pref_pkgs = list()
> > +    for pkg in inst_pkgs:
> > +        prefervar = d.getVar("PREFERRED_RPROVIDER_%s" % pkg, True)
> > +        if prefervar:
> > +            inst_pkgs.remove(pkg)
> > +            pref_pkgs.append(prefervar)
> > +            bb.note("Selecting %s to provide %s due to
> > PREFERRED_RPROVIDER" % (prefervar, pkg))
> > +    inst_pkgs.extend(pref_pkgs)
> > +    d.setVar("PACKAGE_INSTALL", ' '.join(inst_pkgs))
> > +}
> > +do_rootfs[prefuncs] += "rootfs_process_preferred_rproviders"
> > +
> >  # We have to delay the runtime_mapping_rename until just before
> > rootfs runs
> >  # otherwise, the multilib renaming could step in and squash any
> > fixups that
> >  # may have occurred.
>
> This kind of approach to the problem basically doesn't scale. Yes, it
> will rewrite things in PACKAGE_INSTALL however if package A depends on
> package B using one of these names, the mechanism will simply not work.
> This will in turn generate a whole new world of bug reports with no way
> to really fix the problems.
>
> This is why we have other mechanisms for handling the runtime provider
> names, usually through parametrisation with variables so that when you
> rewrite an entry, *all* entries get rewritten and what the package
> manager sees is consistent.
>
> I would also note that in the past, bitbake once did have
> PREFERRED_RPROVIDER support. We dropped it as it caused problems. I
> suspect I would have written up some rationale for that when we did it
> which would be in the list archives.
>
> Cheers,
>
> Richard
>
>
>

[-- Attachment #2: Type: text/html, Size: 3759 bytes --]