[PATCH 00/13] Fix two regressions with RPM introduced with Pyro

[PATCH 00/13] Fix two regressions with RPM introduced with Pyro

[Peter Kjellerstedt]

Right before the release of Pyro (about a month ago), we noticed a
problem with missing automatically determined runtime file
dependencies after the transition to rpm4 and dnf. Unfortunately it
has taken me quite a while to work out all the details, but here is
the result finally.

During the search for the missing dependencies, I also stumbled on
another regression where the installed-packages.txt file that is
generated by buildhistory turned out to no longer contain the actual
package names that were used to create the image, but rather some
fictitious source RPM names... The first patch in the series addresses
that.

The following patches to the libxml2 recipe, insane.bbclass,
buildhistory.bbclass and toaster.bbclass prepare the stage for the
changes to rpm and package.bbclass so that they will not break once
the automatically generated dependencies are restored.

Then rpm is updated to make it possible to yet again access the
runtime dependencies we need, and finally package.bbclass is updated
to make use of them.

See the patches, especially the last one, for more in depth
information on what went wrong during the transition to rpm4 and dnf,
and how to solve it.

I would also like to thank Mark Hatle and Martin Jansa who set me on
the right track to find the problem.

//Peter

The following changes since commit b1d5267f0e31605b346af40778da0ac1ff298b78:

  README.hardware: update flashing instructions for MPC8315 (2017-06-09 17:13:45 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib pkj/rpm_regressions
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/rpm_regressions

Peter Kjellerstedt (13):
  package_manager.py: Generate correct RPM package names again
  libxml2: Make ptest run the Python tests if Python support is enabled
  insane.bbclass: Report all file-rdeps errors, not just the first
  insane.bbclass: Improve the handling of runtime file dependencies
  buildhistory.bbclass: Improve the generated depends.dot file
  toaster.bbclass: Simplify parsing of depends.dot
  toaster.bbclass: Ignore some dependencies in
    toaster_buildhistory_dump()
  rpm: Simplify the creation of wrappers for the native tools
  rpm: Create a wrapper for the native rpmdeps tool
  rpm: Do not require that ELF binaries are executable to be
    identifiable
  rpm: Use conditional to access %{_docdir} in macros.in
  rpm: Add a new option --alldeps to rpmdeps
  package.bbclass: Restore functionality to detect RPM dependencies

 meta/classes/buildhistory.bbclass                  |  15 +-
 meta/classes/insane.bbclass                        |  18 ++-
 meta/classes/package.bbclass                       |   5 +-
 meta/classes/toaster.bbclass                       |  36 +++--
 meta/lib/oe/package.py                             |  47 ++++---
 meta/lib/oe/package_manager.py                     |   6 +-
 ...run-the-python-tests-if-python-is-enabled.patch |  99 ++++++++++++++
 meta/recipes-core/libxml/libxml2_2.9.4.bb          |  11 +-
 ...ire-that-ELF-binaries-are-executable-to-b.patch |  32 +++++
 ...onditional-to-access-_docdir-in-macros.in.patch |  36 +++++
 .../0013-Add-a-new-option-alldeps-to-rpmdeps.patch | 152 +++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_git.bb               |  77 ++++-------
 12 files changed, 426 insertions(+), 108 deletions(-)
 create mode 100644 meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch
 create mode 100644 meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch
 create mode 100644 meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch
 create mode 100644 meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch

-- 
2.12.0

[PATCH 01/13] package_manager.py: Generate correct RPM package names again

[Peter Kjellerstedt]

During the transition to RPM4, the package names returned by
RpmPM.list_installed() changed from the expected names of the packages
that were installed into the image to some fictitious source RPM
names.

This restores the original functionality so that the
installed-packages.txt files produced by inheriting buildhistory yet
again contains a list of the names of the installed packages.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/lib/oe/package_manager.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index f1b65bdbbc..caccd7baa9 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -647,7 +647,7 @@ class RpmPM(PackageManager):
                             symlinks=True)
 
     def list_installed(self):
-        output = self._invoke_dnf(["repoquery", "--installed", "--queryformat", "Package: %{name} %{arch} %{version} %{sourcerpm}\nDependencies:\n%{requires}\nRecommendations:\n%{recommends}\nDependenciesEndHere:\n"],
+        output = self._invoke_dnf(["repoquery", "--installed", "--queryformat", "Package: %{name} %{arch} %{version} %{name}-%{version}-%{release}.%{arch}.rpm\nDependencies:\n%{requires}\nRecommendations:\n%{recommends}\nDependenciesEndHere:\n"],
                                   print_output = False)
         packages = {}
         current_package = None
@@ -659,8 +659,8 @@ class RpmPM(PackageManager):
                 current_package = package_info[0]
                 package_arch = package_info[1]
                 package_version = package_info[2]
-                package_srpm = package_info[3]
-                packages[current_package] = {"arch":package_arch, "ver":package_version, "filename":package_srpm}
+                package_rpm = package_info[3]
+                packages[current_package] = {"arch":package_arch, "ver":package_version, "filename":package_rpm}
                 current_deps = []
             elif line.startswith("Dependencies:"):
                 current_state = "dependencies"
-- 
2.12.0

[PATCH 02/13] libxml2: Make ptest run the Python tests if Python support is enabled

[Peter Kjellerstedt]

Since we go through the trouble of copying the Python tests, we may as
well actually run them...

This also avoids the following QA issue:

  ERROR: libxml2-2.9.4-r0 do_package_qa: QA Issue:
  /usr/lib/libxml2/ptest/python/tests/push.py contained in package
  libxml2-ptest requires /usr/bin/python, but no providers found in
  RDEPENDS_libxml2-ptest? [file-rdeps]

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 ...run-the-python-tests-if-python-is-enabled.patch | 99 ++++++++++++++++++++++
 meta/recipes-core/libxml/libxml2_2.9.4.bb          | 11 ++-
 2 files changed, 108 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch

diff --git a/meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch b/meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch
new file mode 100644
index 0000000000..d1ad55116a
--- /dev/null
+++ b/meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch
@@ -0,0 +1,99 @@
+From 78dbd4c09d617a9cb730d796f94ee4d93840d3cc Mon Sep 17 00:00:00 2001
+From: Peter Kjellerstedt <pkj@axis.com>
+Date: Fri, 9 Jun 2017 17:50:46 +0200
+Subject: [PATCH] Make ptest run the python tests if python is enabled
+
+One of the tests (tstLastError.py) needed a minor correction. It might
+be due to the fact that the tests are forced to run with Python 3.
+
+Upstream-Status: Inappropriate [OE specific]
+Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
+---
+ Makefile.am                  |  2 +-
+ python/Makefile.am           |  9 +++++++++
+ python/tests/Makefile.am     | 12 ++++++++++--
+ python/tests/tstLastError.py |  2 +-
+ 4 files changed, 21 insertions(+), 4 deletions(-)
+
+diff --git a/Makefile.am b/Makefile.am
+index e181ee0..7960e7d 100644
+--- a/Makefile.am
++++ b/Makefile.am
+@@ -207,9 +207,9 @@ install-ptest:
+ 	install $(noinst_PROGRAMS) $(DESTDIR))
+ 	cp -r $(srcdir)/test $(DESTDIR)
+ 	cp -r $(srcdir)/result $(DESTDIR)
+-	cp -r $(srcdir)/python $(DESTDIR)
+ 	cp Makefile $(DESTDIR)
+ 	sed -i -e 's|^Makefile:|_Makefile:|' $(DESTDIR)/Makefile
++	$(MAKE) -C python install-ptest
+ 
+ runtests:
+ 	[ -d test   ] || $(LN_S) $(srcdir)/test   .
+diff --git a/python/Makefile.am b/python/Makefile.am
+index 34aed96..ba3ec6a 100644
+--- a/python/Makefile.am
++++ b/python/Makefile.am
+@@ -48,7 +48,16 @@ GENERATED = libxml2class.py libxml2class.txt $(BUILT_SOURCES)
+ 
+ $(GENERATED): $(srcdir)/generator.py $(API_DESC)
+ 	$(PYTHON) $(srcdir)/generator.py $(srcdir)
++
++install-ptest:
++	cp -r $(srcdir) $(DESTDIR)
++	sed -e 's|^Makefile:|_Makefile:|' \
++	    -e 's|^\(tests test:\) all|\1|' Makefile >$(DESTDIR)/python/Makefile
++	$(MAKE) -C tests install-ptest
++else
++install-ptest:
+ endif
+ 
++.PHONY: tests test
+ tests test: all
+ 	cd tests && $(MAKE) tests
+diff --git a/python/tests/Makefile.am b/python/tests/Makefile.am
+index 95ebead..418e21e 100644
+--- a/python/tests/Makefile.am
++++ b/python/tests/Makefile.am
+@@ -59,6 +59,11 @@ XMLS=		\
+ CLEANFILES = core tmp.xml *.pyc
+ 
+ if WITH_PYTHON
++install-ptest:
++	cp -r $(srcdir) $(DESTDIR)/python
++	sed -e 's|^Makefile:|_Makefile:|' \
++	    -e 's|^\(srcdir = \).*|\1.|' Makefile >$(DESTDIR)/python/tests/Makefile
++
+ tests: $(PYTESTS)
+ 	@for f in $(XMLS) ; do test -f $$f || $(LN_S) $(srcdir)/$$f . ; done
+ 	@echo "## running Python regression tests"
+@@ -68,8 +73,11 @@ tests: $(PYTESTS)
+ 	   export LD_LIBRARY_PATH; \
+ 	   for test in $(PYTESTS) ; \
+ 	   do log=`$(PYTHON) $(srcdir)/$$test` ; \
+-	   if [ "`echo $$log | grep OK`" = "" ] ; then \
+-	   echo "-- $$test" ; echo "$$log" ; fi ; done)
++	   if [ "`echo $$log | grep OK`" ]; then \
++	   echo "PASS: $$test"; else \
++	   echo "$$log"; echo "FAIL: $$test"; fi; done)
+ else
++install-ptest:
++
+ tests:
+ endif
+diff --git a/python/tests/tstLastError.py b/python/tests/tstLastError.py
+index d5f9be7..3e5bef8 100755
+--- a/python/tests/tstLastError.py
++++ b/python/tests/tstLastError.py
+@@ -25,7 +25,7 @@ class TestCase(unittest.TestCase):
+         when the exception is raised, check the libxml2.lastError for
+         expected values."""
+         # disable the default error handler
+-        libxml2.registerErrorHandler(None,None)
++        libxml2.registerErrorHandler(lambda ctx,str: None,None)
+         try:
+             f(*args)
+         except exc:
+-- 
+2.12.0
+
diff --git a/meta/recipes-core/libxml/libxml2_2.9.4.bb b/meta/recipes-core/libxml/libxml2_2.9.4.bb
index ea0d3b88af..4f60781d22 100644
--- a/meta/recipes-core/libxml/libxml2_2.9.4.bb
+++ b/meta/recipes-core/libxml/libxml2_2.9.4.bb
@@ -24,7 +24,8 @@ SRC_URI = "ftp://xmlsoft.org/libxml2/libxml2-${PV}.tar.gz;name=libtar \
            file://libxml2-CVE-2016-4658.patch \
            file://libxml2-fix_NULL_pointer_derefs.patch \
            file://CVE-2016-9318.patch \
-          "
+           file://0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch \
+           "
 
 SRC_URI[libtar.md5sum] = "ae249165c173b1ff386ee8ad676815f5"
 SRC_URI[libtar.sha256sum] = "ffb911191e509b966deb55de705387f14156e1a56b21824357cdf0053233633c"
@@ -43,7 +44,7 @@ inherit autotools pkgconfig binconfig-disabled ptest
 
 inherit ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3native', '', d)}
 
-RDEPENDS_${PN}-ptest += "make ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-core', '', d)}"
+RDEPENDS_${PN}-ptest += "make ${@bb.utils.contains('PACKAGECONFIG', 'python', 'libgcc python3-core python3-argparse python3-logging python3-shell python3-signal python3-stringold python3-threading python3-unittest ${PN}-python', '', d)}"
 
 RDEPENDS_${PN}-python += "${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-core', '', d)}"
 
@@ -77,6 +78,12 @@ do_configure_prepend () {
 
 do_install_ptest () {
 	cp -r ${WORKDIR}/xmlconf ${D}${PTEST_PATH}
+	if [ "${@bb.utils.filter('PACKAGECONFIG', 'python', d)}" ]; then
+		sed -i -e 's|^\(PYTHON = \).*|\1${USRBINPATH}/${PYTHON_PN}|' \
+		    ${D}${PTEST_PATH}/python/tests/Makefile
+		grep -lrZ '#!/usr/bin/python' ${D}${PTEST_PATH}/python |
+			xargs -0 sed -i -e 's|/usr/bin/python|${USRBINPATH}/${PYTHON_PN}|'
+	fi
 }
 
 do_install_append_class-native () {
-- 
2.12.0

[PATCH 03/13] insane.bbclass: Report all file-rdeps errors, not just the first

[Peter Kjellerstedt]

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/insane.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 5a3d017004..258856266e 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -968,7 +968,7 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
                 for key in filerdepends:
                     error_msg = "%s contained in package %s requires %s, but no providers found in RDEPENDS_%s?" % \
                             (filerdepends[key].replace("_%s" % pkg, "").replace("@underscore@", "_"), pkg, key, pkg)
-                package_qa_handle_error("file-rdeps", error_msg, d)
+                    package_qa_handle_error("file-rdeps", error_msg, d)
 
 def package_qa_check_deps(pkg, pkgdest, skip, d):
 
-- 
2.12.0

[PATCH 04/13] insane.bbclass: Improve the handling of runtime file dependencies

[Peter Kjellerstedt]

This makes the file-rdeps test support:
* versioned dependencies, e.g., "perl (>= 5.000)", and
* package dependencies among the file dependencies, e.g., "perl".

It also ignores all "perl(...)" dependencies since it is expected that
these are generated and handled by rpm itself and there is no reason
to second guess what it is doing.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/insane.bbclass | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 258856266e..fa932ef576 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -912,8 +912,9 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
             rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
             for key in rdep_data:
                 if key.startswith("FILERDEPENDS_"):
-                    for subkey in rdep_data[key].split():
-                        if subkey not in ignored_file_rdeps:
+                    for subkey in bb.utils.explode_deps(rdep_data[key]):
+                        if subkey not in ignored_file_rdeps and \
+                                not subkey.startswith('perl('):
                             # We already know it starts with FILERDEPENDS_
                             filerdepends[subkey] = key[13:]
 
@@ -928,11 +929,10 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
                         sub_rdeps = rdep_data.get("RDEPENDS_" + rdep)
                         if not sub_rdeps:
                             continue
-                        for sub_rdep in sub_rdeps.split():
+                        for sub_rdep in bb.utils.explode_deps(sub_rdeps):
                             if sub_rdep in done:
                                 continue
-                            if not sub_rdep.startswith('(') and \
-                                    oe.packagedata.has_subpkgdata(sub_rdep, d):
+                            if oe.packagedata.has_subpkgdata(sub_rdep, d):
                                 # It's a new rdep
                                 done.append(sub_rdep)
                                 new.append(sub_rdep)
@@ -950,11 +950,15 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
                         filerdepends.pop("/usr/bin/python",None)
                         done.remove(py)
                 for rdep in done:
+                    # The file dependencies may contain package names, e.g.,
+                    # perl
+                    filerdepends.pop(rdep,None)
+
                     # For Saving the FILERPROVIDES, RPROVIDES and FILES_INFO
                     rdep_data = oe.packagedata.read_subpkgdata(rdep, d)
                     for key in rdep_data:
                         if key.startswith("FILERPROVIDES_") or key.startswith("RPROVIDES_"):
-                            for subkey in rdep_data[key].split():
+                            for subkey in bb.utils.explode_deps(rdep_data[key]):
                                 filerdepends.pop(subkey,None)
                         # Add the files list to the rprovides
                         if key == "FILES_INFO":
-- 
2.12.0

[PATCH 05/13] buildhistory.bbclass: Improve the generated depends.dot file

[Peter Kjellerstedt]

* Convert incorrectly formatted dependencies such as:
    "bar -> "foo" ">=" "1.2.3"
  into dependencies with edge labels:
    "bar -> "foo" [label=">= 1.2.3"]
* Remove rpmlib() and config() dependencies such as:
    "foo" -> "rpmlib(CompressedFileNames)" [label="<= 3.0.4-1"]
  and:
    "base-files" -> "config(base-files)" [label="= 3.0.14-r89.49"]
* Remove the trailing semicolon that was added to each line. It fills
  no purpose.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/buildhistory.bbclass | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 3e907fc3f9..4ee7094348 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -434,13 +434,20 @@ buildhistory_get_installed() {
 
 	# Produce dependency graph
 	# First, quote each name to handle characters that cause issues for dot
-	sed 's:\([^| ]*\):"\1":g' ${WORKDIR}/bh_installed_pkgs_deps.txt > $1/depends.tmp && \
+	sed 's:\([^| ]*\):"\1":g' ${WORKDIR}/bh_installed_pkgs_deps.txt > $1/depends.tmp &&
 		rm ${WORKDIR}/bh_installed_pkgs_deps.txt
-	# Change delimiter from pipe to -> and set style for recommend lines
-	sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
+	# Remove lines with rpmlib(...) and config(...) dependencies, change the
+	# delimiter from pipe to "->", set the style for recommend lines and
+	# turn versioned dependencies into edge labels.
+	sed -i -e '/rpmlib(/d' \
+	       -e '/config(/d' \
+	       -e 's:|: -> :' \
+	       -e 's:"\[REC\]":[style=dotted]:' \
+	       -e 's:"\([<>=]\+\)" "\([^"]*\)":[label="\1 \2"]:' \
+		$1/depends.tmp
 	# Add header, sorted and de-duped contents and footer and then delete the temp file
 	printf "digraph depends {\n    node [shape=plaintext]\n" > $1/depends.dot
-	cat $1/depends.tmp | sort | uniq >> $1/depends.dot
+	cat $1/depends.tmp | sort -u >> $1/depends.dot
 	echo "}" >>  $1/depends.dot
 	rm $1/depends.tmp
 
-- 
2.12.0

[PATCH 06/13] toaster.bbclass: Simplify parsing of depends.dot

[Peter Kjellerstedt]

By using a single regular expression, the parsing of the depends.dot
file can be simplified a lot. This should also make it less
susceptible to formatting changes in that file.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/toaster.bbclass | 28 +++++++++++++---------------
 1 file changed, 13 insertions(+), 15 deletions(-)

diff --git a/meta/classes/toaster.bbclass b/meta/classes/toaster.bbclass
index 296e4764f0..fbf463bbb7 100644
--- a/meta/classes/toaster.bbclass
+++ b/meta/classes/toaster.bbclass
@@ -270,22 +270,20 @@ python toaster_buildhistory_dump() {
                     images[target][pname.strip()] = {'size':int(psize)*1024, 'depends' : []}
 
             with open("%s/depends.dot" % installed_img_path, "r") as fin:
-                p = re.compile(r' -> ')
-                dot = re.compile(r'.*style=dotted')
+                p = re.compile(r'\s*"(?P<name>[^"]+)"\s*->\s*"(?P<dep>[^"]+)"(?P<rec>.*?\[style=dotted\])?')
                 for line in fin:
-                    line = line.rstrip(';')
-                    linesplit = p.split(line)
-                    if len(linesplit) == 2:
-                        pname = linesplit[0].rstrip('"').strip('"')
-                        dependsname = linesplit[1].split(" ")[0].strip().strip(";").strip('"').rstrip('"')
-                        deptype = "depends"
-                        if dot.match(line):
-                            deptype = "recommends"
-                        if not pname in images[target]:
-                            images[target][pname] = {'size': 0, 'depends' : []}
-                        if not dependsname in images[target]:
-                            images[target][dependsname] = {'size': 0, 'depends' : []}
-                        images[target][pname]['depends'].append((dependsname, deptype))
+                    m = p.match(line)
+                    if not m:
+                        continue
+                    pname = m.group('name')
+                    dependsname = m.group('dep')
+                    deptype = 'recommends' if m.group('rec') else 'depends'
+
+                    if not pname in images[target]:
+                        images[target][pname] = {'size': 0, 'depends' : []}
+                    if not dependsname in images[target]:
+                        images[target][dependsname] = {'size': 0, 'depends' : []}
+                    images[target][pname]['depends'].append((dependsname, deptype))
 
             # files-in-image.txt is only generated if an image file is created,
             # so the file entries ('syms', 'dirs', 'files') for a target will be
-- 
2.12.0

[PATCH 07/13] toaster.bbclass: Ignore some dependencies in toaster_buildhistory_dump()

[Peter Kjellerstedt]

When using RPM, depends.dot may contain dependencies such as
"/bin/sh", which will confuse _toaster_load_pkgdatafile(). Ignore
them. While at it, also ignore dependencies that contain parentheses,
e.g., "libc.so.6(GLIBC_2.7)".

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/toaster.bbclass | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/meta/classes/toaster.bbclass b/meta/classes/toaster.bbclass
index fbf463bbb7..6cef0b8f6e 100644
--- a/meta/classes/toaster.bbclass
+++ b/meta/classes/toaster.bbclass
@@ -279,6 +279,14 @@ python toaster_buildhistory_dump() {
                     dependsname = m.group('dep')
                     deptype = 'recommends' if m.group('rec') else 'depends'
 
+                    # If RPM is used for packaging, then there may be
+                    # dependencies such as "/bin/sh", which will confuse
+                    # _toaster_load_pkgdatafile() later on. While at it, ignore
+                    # any dependencies that contain parentheses, e.g.,
+                    # "libc.so.6(GLIBC_2.7)".
+                    if dependsname.startswith('/') or '(' in dependsname:
+                        continue
+
                     if not pname in images[target]:
                         images[target][pname] = {'size': 0, 'depends' : []}
                     if not dependsname in images[target]:
-- 
2.12.0

[PATCH 08/13] rpm: Simplify the creation of wrappers for the native tools

[Peter Kjellerstedt]

Use a loop rather than calling create_wrapper for each individual
tool.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/recipes-devtools/rpm/rpm_git.bb | 73 ++++++++++--------------------------
 1 file changed, 19 insertions(+), 54 deletions(-)

diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index 2310ee6b09..531669ae73 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -67,62 +67,27 @@ BBCLASSEXTEND = "native nativesdk"
 # Direct rpm-native to read configuration from our sysroot, not the one it was compiled in
 # libmagic also has sysroot path contamination, so override it
 do_install_append_class-native() {
-        create_wrapper ${D}/${bindir}/rpmbuild \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpmsign \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpmkeys \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpm \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpm2archive \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpm2cpio \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpmdb \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpmgraph \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
-
-        create_wrapper ${D}/${bindir}/rpmspec \
-                RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
-                RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
-                MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
-                RPM_NO_CHROOT_FOR_SCRIPTS=1
+        tools="\
+                ${bindir}/rpm \
+                ${bindir}/rpm2archive \
+                ${bindir}/rpm2cpio \
+                ${bindir}/rpmbuild \
+                ${bindir}/rpmdb \
+                ${bindir}/rpmgraph \
+                ${bindir}/rpmkeys \
+                ${bindir}/rpmsign \
+                ${bindir}/rpmspec \
+        "
+
+        for tool in $tools; do
+                create_wrapper ${D}$tool \
+                        RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \
+                        RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \
+                        MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \
+                        RPM_NO_CHROOT_FOR_SCRIPTS=1
+        done
 }
 
-
 # Rpm's make install creates var/tmp which clashes with base-files packaging
 do_install_append_class-target() {
     rm -rf ${D}/var
-- 
2.12.0

[PATCH 09/13] rpm: Create a wrapper for the native rpmdeps tool

[Peter Kjellerstedt]

Rather than trying to call rpmdeps with the correct arguments to work
with the sysroot as was done in package.bbclass, create a wrapper for
it like all the other native tools already had.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/recipes-devtools/rpm/rpm_git.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index 531669ae73..7665d54cdc 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -77,6 +77,7 @@ do_install_append_class-native() {
                 ${bindir}/rpmkeys \
                 ${bindir}/rpmsign \
                 ${bindir}/rpmspec \
+                ${libdir}/rpm/rpmdeps \
         "
 
         for tool in $tools; do
-- 
2.12.0

[PATCH 10/13] rpm: Do not require that ELF binaries are executable to be identifiable

[Peter Kjellerstedt]

There is nothing that requires, e.g., a DSO to be executable, but it
is still an ELF binary and should be identified as such.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 ...ire-that-ELF-binaries-are-executable-to-b.patch | 32 ++++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_git.bb               |  1 +
 2 files changed, 33 insertions(+)
 create mode 100644 meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch

diff --git a/meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch b/meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch
new file mode 100644
index 0000000000..c910a478ee
--- /dev/null
+++ b/meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch
@@ -0,0 +1,32 @@
+From d65d6e8760afbd7f70b22a1f3297a037bc475fea Mon Sep 17 00:00:00 2001
+From: Peter Kjellerstedt <pkj@axis.com>
+Date: Mon, 15 May 2017 10:21:08 +0200
+Subject: [PATCH 11/13] Do not require that ELF binaries are executable to be
+ identifiable
+
+There is nothing that requires, e.g., a DSO to be executable, but it
+is still an ELF binary and should be identified as such.
+
+Upstream probably expects all ELF binaries to be marked as executable,
+but rather than imposing such a limitation for OE, allow any file to
+be identified as an ELF binary regardless of whether it is executable
+or not.
+
+Upstream-Status: Inappropriate
+Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
+---
+ fileattrs/elf.attr | 1 -
+ 1 file changed, 1 deletion(-)
+
+diff --git a/fileattrs/elf.attr b/fileattrs/elf.attr
+index 595b33e09..bac52649d 100644
+--- a/fileattrs/elf.attr
++++ b/fileattrs/elf.attr
+@@ -1,4 +1,3 @@
+ %__elf_provides		%{_rpmconfigdir}/elfdeps --provides %{?__filter_GLIBC_PRIVATE:--filter-private}
+ %__elf_requires		%{_rpmconfigdir}/elfdeps --requires %{?__filter_GLIBC_PRIVATE:--filter-private}
+ %__elf_magic		^(setuid )?(setgid )?(sticky )?ELF (32|64)-bit.*$
+-%__elf_flags		exeonly
+-- 
+2.12.0
+
diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index 7665d54cdc..15880377b8 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -35,6 +35,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \
            file://0001-Fix-build-with-musl-C-library.patch \
            file://0001-Add-a-color-setting-for-mips64_n32-binaries.patch \
            file://0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch \
+           file://0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch \
            "
 
 PV = "4.13.90+git${SRCPV}"
-- 
2.12.0

[PATCH 11/13] rpm: Use conditional to access %{_docdir} in macros.in

[Peter Kjellerstedt]

This avoids the following warning:

warning: Ignoring invalid regex %{_docdir}

when runing `rpmdeps -R <file>`, since %{_docdir} is only defined when
parsing a spec file.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 ...onditional-to-access-_docdir-in-macros.in.patch | 36 ++++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_git.bb               |  1 +
 2 files changed, 37 insertions(+)
 create mode 100644 meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch

diff --git a/meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch b/meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch
new file mode 100644
index 0000000000..996da90d43
--- /dev/null
+++ b/meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch
@@ -0,0 +1,36 @@
+From 77808db4036dc4a012c47aca36255549ed764a6a Mon Sep 17 00:00:00 2001
+From: Peter Kjellerstedt <pkj@axis.com>
+Date: Tue, 16 May 2017 10:58:18 +0200
+Subject: [PATCH 12/13] Use conditional to access %{_docdir} in macros.in
+
+This avoids the following warning:
+
+warning: Ignoring invalid regex %{_docdir}
+
+when runing `rpmdeps -R <file>`, since %{_docdir} is only defined when
+parsing a spec file (in parseSpec()).
+
+Upstream-Status: Accepted [https://github.com/rpm-software-management/rpm/pull/216]
+Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
+---
+ macros.in | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/macros.in b/macros.in
+index 84ae25275..cca67a500 100644
+--- a/macros.in
++++ b/macros.in
+@@ -200,8 +200,8 @@ package or when debugging this package.\
+ # Their purpouse is to set up global filtering for all packages. If you need
+ # to set up specific filtering for your package use %__requires_exclude_from
+ # and %__provides_exclude_from instead.
+-%__global_requires_exclude_from		%{_docdir}
+-%__global_provides_exclude_from		%{_docdir}
++%__global_requires_exclude_from		%{?_docdir:%{_docdir}}
++%__global_provides_exclude_from		%{?_docdir:%{_docdir}}
+ 
+ #	The path to the gzip executable (legacy, use %{__gzip} instead).
+ %_gzipbin		%{__gzip}
+-- 
+2.12.0
+
diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index 15880377b8..7c5f649a8b 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -36,6 +36,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \
            file://0001-Add-a-color-setting-for-mips64_n32-binaries.patch \
            file://0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch \
            file://0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch \
+           file://0012-Use-conditional-to-access-_docdir-in-macros.in.patch \
            "
 
 PV = "4.13.90+git${SRCPV}"
-- 
2.12.0

[PATCH 12/13] rpm: Add a new option --alldeps to rpmdeps

[Peter Kjellerstedt]

This will send the output from rpmfcPrint() to stdout. This is an
alternative to using the --rpmfcdebug option, which will send the same
output to stderr. The two options have totally different use cases
though. While --alldeps is used when the output from rpmfcPrint() is
what is wanted, --rpmfcdebug can be used together with the other
output options, e.g., --requires, without affecting their output.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 .../0013-Add-a-new-option-alldeps-to-rpmdeps.patch | 152 +++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_git.bb               |   1 +
 2 files changed, 153 insertions(+)
 create mode 100644 meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch

diff --git a/meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch b/meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch
new file mode 100644
index 0000000000..faaf62960c
--- /dev/null
+++ b/meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch
@@ -0,0 +1,152 @@
+From 3bf20a6116ae3e1a5a3a6907bee7e881b17efb2f Mon Sep 17 00:00:00 2001
+From: Peter Kjellerstedt <pkj@axis.com>
+Date: Mon, 15 May 2017 11:23:26 +0200
+Subject: [PATCH 13/13] Add a new option --alldeps to rpmdeps
+
+This will send the output from rpmfcPrint() to stdout. This is an
+alternative to using the --rpmfcdebug option, which will send the same
+output to stderr. The two options have totally different use cases
+though. While --alldeps is used when the output from rpmfcPrint() is
+what is wanted, --rpmfcdebug can be used together with the other
+output options, e.g., --requires, without affecting their output.
+
+Upstream-Status: Submitted [https://github.com/rpm-software-management/rpm/pull/220]
+Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
+---
+ build/rpmfc.c   | 27 ++++++++++++++-------------
+ build/rpmfc.h   |  1 -
+ tools/rpmdeps.c | 44 +++++++++++++++++++++++++-------------------
+ 3 files changed, 39 insertions(+), 33 deletions(-)
+
+diff --git a/build/rpmfc.c b/build/rpmfc.c
+index c8e2f876a..44f1cdc9a 100644
+--- a/build/rpmfc.c
++++ b/build/rpmfc.c
+@@ -732,7 +732,6 @@ static rpm_color_t rpmfcColor(const char * fmstr)
+ 
+ void rpmfcPrint(const char * msg, rpmfc fc, FILE * fp)
+ {
+-    rpm_color_t fcolor;
+     int ndx;
+     int dx;
+     int fx;
+@@ -744,21 +743,23 @@ void rpmfcPrint(const char * msg, rpmfc fc, FILE * fp)
+ 
+     if (fc)
+     for (fx = 0; fx < fc->nfiles; fx++) {
+-	rpmsid cx = fc->fcdictx[fx] + 1; /* id's are one off */
+-	fcolor = fc->fcolor[fx];
+-	ARGV_t fattrs = fc->fattrs[fx];
+-
+ 	fprintf(fp, "%3d %s", fx, fc->fn[fx]);
+-	if (fcolor != RPMFC_BLACK)
++	if (_rpmfc_debug) {
++	    rpmsid cx = fc->fcdictx[fx] + 1; /* id's are one off */
++	    rpm_color_t fcolor = fc->fcolor[fx];
++	    ARGV_t fattrs = fc->fattrs[fx];
++
++	    if (fcolor != RPMFC_BLACK)
+ 		fprintf(fp, "\t0x%x", fc->fcolor[fx]);
+-	else
++	    else
+ 		fprintf(fp, "\t%s", rpmstrPoolStr(fc->cdict, cx));
+-	if (fattrs) {
+-	    char *attrs = argvJoin(fattrs, ",");
+-	    fprintf(fp, " [%s]", attrs);
+-	    free(attrs);
+-	} else {
+-	    fprintf(fp, " [none]");
++	    if (fattrs) {
++		char *attrs = argvJoin(fattrs, ",");
++		fprintf(fp, " [%s]", attrs);
++		free(attrs);
++	    } else {
++		fprintf(fp, " [none]");
++	    }
+ 	}
+ 	fprintf(fp, "\n");
+ 
+diff --git a/build/rpmfc.h b/build/rpmfc.h
+index dae8ea5b1..3d87b31cf 100644
+--- a/build/rpmfc.h
++++ b/build/rpmfc.h
+@@ -45,7 +45,6 @@ typedef const struct rpmfcTokens_s * rpmfcToken;
+ 
+ /** \ingroup rpmfc
+  * Print results of file classification.
+- * @todo Remove debugging routine.
+  * @param msg		message prefix (NULL for none)
+  * @param fc		file classifier
+  * @param fp		output file handle (NULL for stderr)
+diff --git a/tools/rpmdeps.c b/tools/rpmdeps.c
+index a414b6343..f260a38c4 100644
+--- a/tools/rpmdeps.c
++++ b/tools/rpmdeps.c
+@@ -23,6 +23,8 @@ static int print_conflicts;
+ 
+ static int print_obsoletes;
+ 
++static int print_alldeps;
++
+ static void rpmdsPrint(const char * msg, rpmds ds, FILE * fp)
+ {
+     if (fp == NULL) fp = stderr;
+@@ -57,6 +59,8 @@ static struct poptOption optionsTable[] = {
+         NULL, NULL },
+  { "obsoletes", '\0', POPT_ARG_VAL, &print_obsoletes, -1,
+         NULL, NULL },
++ { "alldeps", '\0', POPT_ARG_VAL, &print_alldeps, -1,
++        NULL, NULL },
+ 
+    POPT_AUTOALIAS
+    POPT_AUTOHELP
+@@ -100,25 +104,27 @@ main(int argc, char *argv[])
+     if (rpmfcClassify(fc, av, NULL) || rpmfcApply(fc))
+ 	goto exit;
+ 
+-    if (_rpmfc_debug)
+-	rpmfcPrint(buf, fc, NULL);
+-
+-    if (print_provides)
+-	rpmdsPrint(NULL, rpmfcProvides(fc), stdout);
+-    if (print_requires)
+-	rpmdsPrint(NULL, rpmfcRequires(fc), stdout);
+-    if (print_recommends)
+-	rpmdsPrint(NULL, rpmfcRecommends(fc), stdout);
+-    if (print_suggests)
+-	rpmdsPrint(NULL, rpmfcSuggests(fc), stdout);
+-    if (print_supplements)
+-	rpmdsPrint(NULL, rpmfcSupplements(fc), stdout);
+-    if (print_enhances)
+-	rpmdsPrint(NULL, rpmfcEnhances(fc), stdout);
+-    if (print_conflicts)
+-	rpmdsPrint(NULL, rpmfcConflicts(fc), stdout);
+-    if (print_obsoletes)
+-	rpmdsPrint(NULL, rpmfcObsoletes(fc), stdout);
++    if (print_alldeps || _rpmfc_debug)
++	rpmfcPrint(NULL, fc, print_alldeps ? stdout : NULL);
++
++    if (!print_alldeps) {
++	if (print_provides)
++	    rpmdsPrint(NULL, rpmfcProvides(fc), stdout);
++	if (print_requires)
++	    rpmdsPrint(NULL, rpmfcRequires(fc), stdout);
++	if (print_recommends)
++	    rpmdsPrint(NULL, rpmfcRecommends(fc), stdout);
++	if (print_suggests)
++	    rpmdsPrint(NULL, rpmfcSuggests(fc), stdout);
++	if (print_supplements)
++	    rpmdsPrint(NULL, rpmfcSupplements(fc), stdout);
++	if (print_enhances)
++	    rpmdsPrint(NULL, rpmfcEnhances(fc), stdout);
++	if (print_conflicts)
++	    rpmdsPrint(NULL, rpmfcConflicts(fc), stdout);
++	if (print_obsoletes)
++	    rpmdsPrint(NULL, rpmfcObsoletes(fc), stdout);
++    }
+ 
+     ec = 0;
+ 
+-- 
+2.12.0
+
diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index 7c5f649a8b..5fe52b83ec 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -37,6 +37,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \
            file://0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch \
            file://0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch \
            file://0012-Use-conditional-to-access-_docdir-in-macros.in.patch \
+           file://0013-Add-a-new-option-alldeps-to-rpmdeps.patch \
            "
 
 PV = "4.13.90+git${SRCPV}"
-- 
2.12.0

[PATCH 13/13] package.bbclass: Restore functionality to detect RPM dependencies

[Peter Kjellerstedt]

During the transition to dnf and rpm4, the functionality to
automatically make RPM determine dependencies was lost.

Before the transition, an OE specific tool called rpmdeps-oecore had
been added to the rpm suit. It was based on the rpmdeps tool that is
part of rpm. For each file specified on its command line, it would
output the provides and requires that RPM could determine.

During the transition to rpm4, rpmdeps-oecore was replaced with the
standard rpmdeps. However, what no one noticed was that unless rpmdeps
is given options, e.g., -P or -R, to tell it what it should output, it
will not output anything. Thus, it would do all the work to determine
the requirements, but would keep silent about it. And since no output
from rpmdeps is expected unless there are requirements, there were no
warnings indicating that everything was not working as expected.

Porting the old rpmdeps-oecore to work with rpm4 is not really
possible since it relied on being able to access internals of RPM that
are no longer available. However, it turned out that rpmdeps had a
debug option, --rpmfcdebug, that would output exactly the information
that we need, albeit in a different format and to stderr. To make this
usable, rpmdeps has now received a new option, --alldeps, which sends
the information we need to stdout.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/package.bbclass |  5 ++---
 meta/lib/oe/package.py       | 47 +++++++++++++++++++++++++-------------------
 2 files changed, 29 insertions(+), 23 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index cc466bd1b2..65cf5fac5f 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1434,7 +1434,7 @@ if [ x"$D" = "x" ]; then
 fi
 }
 
-RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --rcfile ${STAGING_LIBDIR_NATIVE}/rpm/rpmrc  --macros ${STAGING_LIBDIR_NATIVE}/rpm/macros --define '_rpmconfigdir ${STAGING_LIBDIR_NATIVE}/rpm/'"
+RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --alldeps"
 
 # Collect perfile run-time dependency metadata
 # Output:
@@ -1451,7 +1451,6 @@ python package_do_filedeps() {
     pkgdest = d.getVar('PKGDEST')
     packages = d.getVar('PACKAGES')
     rpmdeps = d.getVar('RPMDEPS')
-    magic = d.expand("${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc")
 
     def chunks(files, n):
         return [files[i:i+n] for i in range(0, len(files), n)]
@@ -1463,7 +1462,7 @@ python package_do_filedeps() {
         if pkg.endswith('-dbg') or pkg.endswith('-doc') or pkg.find('-locale-') != -1 or pkg.find('-localedata-') != -1 or pkg.find('-gconv-') != -1 or pkg.find('-charmap-') != -1 or pkg.startswith('kernel-module-'):
             continue
         for files in chunks(pkgfiles[pkg], 100):
-            pkglist.append((pkg, files, rpmdeps, pkgdest, magic))
+            pkglist.append((pkg, files, rpmdeps, pkgdest))
 
     processed = oe.utils.multiprocess_exec( pkglist, oe.package.filedeprunner)
 
diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 52c5f16cf8..811d27964b 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -57,44 +57,51 @@ def file_translate(file):
 def filedeprunner(arg):
     import re, subprocess, shlex
 
-    (pkg, pkgfiles, rpmdeps, pkgdest, magic) = arg
+    (pkg, pkgfiles, rpmdeps, pkgdest) = arg
     provides = {}
     requires = {}
 
-    r = re.compile(r'[<>=]+ +[^ ]*')
+    file_re = re.compile(r'\s+\d+\s(.*)')
+    dep_re = re.compile(r'\s+(\S)\s+(.*)')
+    r = re.compile(r'[<>=]+\s+\S*')
 
     def process_deps(pipe, pkg, pkgdest, provides, requires):
+        file = None
         for line in pipe:
-            f = line.decode("utf-8").split(" ", 1)[0].strip()
-            line = line.decode("utf-8").split(" ", 1)[1].strip()
+            line = line.decode("utf-8")
 
-            if line.startswith("Requires:"):
+            m = file_re.match(line)
+            if m:
+                file = m.group(1)
+                file = file.replace(pkgdest + "/" + pkg, "")
+                file = file_translate(file)
+                continue
+
+            m = dep_re.match(line)
+            if not m or not file:
+                continue
+
+            type, dep = m.groups()
+
+            if type == 'R':
                 i = requires
-            elif line.startswith("Provides:"):
+            elif type == 'P':
                 i = provides
             else:
-                continue
-
-            file = f.replace(pkgdest + "/" + pkg, "")
-            file = file_translate(file)
-            value = line.split(":", 1)[1].strip()
-            value = r.sub(r'(\g<0>)', value)
+               continue
 
-            if value.startswith("rpmlib("):
-                continue
-            if value == "python":
+            if dep.startswith("python("):
                 continue
+            dep = r.sub(r'(\g<0>)',dep)
+
             if file not in i:
                 i[file] = []
-            i[file].append(value)
+            i[file].append(dep)
 
         return provides, requires
 
-    env = os.environ.copy()
-    env["MAGIC"] = magic
-
     try:
-        dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE, env=env)
+        dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE)
         provides, requires = process_deps(dep_popen.stdout, pkg, pkgdest, provides, requires)
     except OSError as e:
         bb.error("rpmdeps: '%s' command failed, '%s'" % (shlex.split(rpmdeps) + pkgfiles, e))
-- 
2.12.0

Re: [PATCH 13/13] package.bbclass: Restore functionality to detect RPM dependencies

[Alexander Kanavin]

On 06/09/2017 10:34 PM, Peter Kjellerstedt wrote:
> During the transition to rpm4, rpmdeps-oecore was replaced with the
> standard rpmdeps. However, what no one noticed was that unless rpmdeps
> is given options, e.g., -P or -R, to tell it what it should output, it
> will not output anything. Thus, it would do all the work to determine
> the requirements, but would keep silent about it. And since no output
> from rpmdeps is expected unless there are requirements, there were no
> warnings indicating that everything was not working as expected.

Apologies for that, however, this is not tested in any of our 
autobuilder scenarios or otherwise, and if it's not tested, then it's 
not broken. :-)

I would recommend you add some kind of test to this patchset to make 
sure it doesn't happen again (for example, when I would update rpm to a 
new upstream release).


Alex

Re: [PATCH 11/13] rpm: Use conditional to access %{_docdir} in macros.in

[Alexander Kanavin]

On 06/09/2017 10:34 PM, Peter Kjellerstedt wrote:
> This avoids the following warning:
>
> warning: Ignoring invalid regex %{_docdir}
>
> when runing `rpmdeps -R <file>`, since %{_docdir} is only defined when
> parsing a spec file.

Is this possible to do by adding a -D/--define option to rpmdeps? The 
less 'Inappropriate' patches we have to carry, the better.

Alex

Re: [PATCH 11/13] rpm: Use conditional to access %{_docdir} in macros.in

[Alexander Kanavin]

On 06/09/2017 11:25 PM, Alexander Kanavin wrote:
> On 06/09/2017 10:34 PM, Peter Kjellerstedt wrote:
>> This avoids the following warning:
>>
>> warning: Ignoring invalid regex %{_docdir}
>>
>> when runing `rpmdeps -R <file>`, since %{_docdir} is only defined when
>> parsing a spec file.
>
> Is this possible to do by adding a -D/--define option to rpmdeps? The
> less 'Inappropriate' patches we have to carry, the better.

Nevermind, got this mixed up :)

Alex