[PATCH 0/6] Fix RPM4 regressions based on Pyro

[PATCH 0/6] Fix RPM4 regressions based on Pyro

[Mark Hatle]

Work is available at:

git://git.openembedded.org/openembedded-core-contrib mgh/rpm_dep_regression

This is work based on Peter Kjellerstedt's fixes for the rpm dependency
generation.  (See poky-contrib pkj/rpm_regressions2 for the original work.)

The major difference is the last commit which disabled the perl requirement
generation.  This resolves the only problem I found in Peter's patches.

From the comments to the original patch, the only not addressed was adding
an OE self-test for this behavior.  I tried to do this, but I don't see any
existing tests that build a recipe and inspect the results of the built
package.  I'm not sure of any other way to identify if things are working or
not.

To give you an idea of what this does, the provides for libc6, x86-64, prior
to this change were:
eglibc
glibc = 2.26
libc6 = 2.26-r0

After the change, the following were added (other remain):
ld-linux-x86-64.so.2()(64bit)
ld-linux-x86-64.so.2(GLIBC_2.2.5)(64bit)
ld-linux-x86-64.so.2(GLIBC_2.3)(64bit)
ld-linux-x86-64.so.2(GLIBC_2.4)(64bit)
ld-linux-x86-64.so.2(GLIBC_PRIVATE)(64bit)
libanl.so.1()(64bit)
libanl.so.1(GLIBC_2.2.5)(64bit)
libBrokenLocale.so.1()(64bit)
libBrokenLocale.so.1(GLIBC_2.2.5)(64bit)
libcrypt.so.1()(64bit)
libcrypt.so.1(GLIBC_2.2.5)(64bit)
libc.so.6()(64bit)
libc.so.6(GLIBC_2.10)(64bit)
libc.so.6(GLIBC_2.11)(64bit)
libc.so.6(GLIBC_2.12)(64bit)
libc.so.6(GLIBC_2.13)(64bit)
libc.so.6(GLIBC_2.14)(64bit)
libc.so.6(GLIBC_2.15)(64bit)
libc.so.6(GLIBC_2.16)(64bit)
libc.so.6(GLIBC_2.17)(64bit)
libc.so.6(GLIBC_2.18)(64bit)
libc.so.6(GLIBC_2.22)(64bit)
libc.so.6(GLIBC_2.23)(64bit)
libc.so.6(GLIBC_2.24)(64bit)
libc.so.6(GLIBC_2.2.5)(64bit)
libc.so.6(GLIBC_2.25)(64bit)
libc.so.6(GLIBC_2.2.6)(64bit)
libc.so.6(GLIBC_2.26)(64bit)
libc.so.6(GLIBC_2.3.2)(64bit)
libc.so.6(GLIBC_2.3.3)(64bit)
libc.so.6(GLIBC_2.3.4)(64bit)
libc.so.6(GLIBC_2.3)(64bit)
libc.so.6(GLIBC_2.4)(64bit)
libc.so.6(GLIBC_2.5)(64bit)
libc.so.6(GLIBC_2.6)(64bit)
libc.so.6(GLIBC_2.7)(64bit)
libc.so.6(GLIBC_2.8)(64bit)
libc.so.6(GLIBC_2.9)(64bit)
libc.so.6(GLIBC_PRIVATE)(64bit)
libdl.so.2()(64bit)
libdl.so.2(GLIBC_2.2.5)(64bit)
libdl.so.2(GLIBC_2.3.3)(64bit)
libdl.so.2(GLIBC_2.3.4)(64bit)
libdl.so.2(GLIBC_PRIVATE)(64bit)
libm.so.6()(64bit)
libm.so.6(GLIBC_2.15)(64bit)
libm.so.6(GLIBC_2.18)(64bit)
libm.so.6(GLIBC_2.23)(64bit)
libm.so.6(GLIBC_2.24)(64bit)
libm.so.6(GLIBC_2.2.5)(64bit)
libm.so.6(GLIBC_2.25)(64bit)
libm.so.6(GLIBC_2.26)(64bit)
libm.so.6(GLIBC_2.4)(64bit)
libmvec.so.1()(64bit)
libmvec.so.1(GLIBC_2.22)(64bit)
libnsl.so.1()(64bit)
libnsl.so.1(GLIBC_2.2.5)(64bit)
libnsl.so.1(GLIBC_PRIVATE)(64bit)
libnss_compat.so.2()(64bit)
libnss_compat.so.2(GLIBC_PRIVATE)(64bit)
libnss_dns.so.2()(64bit)
libnss_dns.so.2(GLIBC_PRIVATE)(64bit)
libnss_files.so.2()(64bit)
libnss_files.so.2(GLIBC_PRIVATE)(64bit)
libpthread.so.0()(64bit)
libpthread.so.0(GLIBC_2.11)(64bit)
libpthread.so.0(GLIBC_2.12)(64bit)
libpthread.so.0(GLIBC_2.18)(64bit)
libpthread.so.0(GLIBC_2.2.5)(64bit)
libpthread.so.0(GLIBC_2.2.6)(64bit)
libpthread.so.0(GLIBC_2.3.2)(64bit)
libpthread.so.0(GLIBC_2.3.3)(64bit)
libpthread.so.0(GLIBC_2.3.4)(64bit)
libpthread.so.0(GLIBC_2.4)(64bit)
libpthread.so.0(GLIBC_PRIVATE)(64bit)
libresolv.so.2()(64bit)
libresolv.so.2(GLIBC_2.2.5)(64bit)
libresolv.so.2(GLIBC_2.3.2)(64bit)
libresolv.so.2(GLIBC_2.9)(64bit)
libresolv.so.2(GLIBC_PRIVATE)(64bit)
librt.so.1()(64bit)
librt.so.1(GLIBC_2.2.5)(64bit)
librt.so.1(GLIBC_2.3.3)(64bit)
librt.so.1(GLIBC_2.3.4)(64bit)
librt.so.1(GLIBC_2.4)(64bit)
librt.so.1(GLIBC_2.7)(64bit)
libutil.so.1()(64bit)
libutil.so.1(GLIBC_2.2.5)(64bit)
rtld(GNU_HASH)


I have built and tested this for:

core-image-base core-image-minimal core-image-full-cmdline 
core-image-kernel-dev core-image-lsb core-image-lsb-dev 
core-image-lsb-sdk core-image-testmaster core-image-clutter 
core-image-weston core-image-x11 core-image-sato core-image-sato-dev 
core-image-sato-sdk core-image-sato-sdk-ptest

Also using qemux86, qemux86-64 and qemuarm64.

The following description is from Peter's original pull request:

Right before the release of Pyro (about a month ago), we noticed a
problem with missing automatically determined runtime file
dependencies after the transition to rpm4 and dnf. Unfortunately it
has taken me quite a while to work out all the details, but here is
the result finally.

....

Then rpm is updated to make it possible to yet again access the
runtime dependencies we need, and finally package.bbclass is updated
to make use of them.

See the patches, especially the last one, for more in depth
information on what went wrong during the transition to rpm4 and dnf,
and how to solve it.


Mark Hatle (1):
  rpm: Disable perl dependency generation

Peter Kjellerstedt (5):
  rpm: Add dependencies on bash, perl and python3-core
  git: Do not install git cvsserver and git svn by default
  texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs)
  package_rpm.bbclass: Filter out unwanted file deps for nativesdk
    packages
  package.bbclass: Restore functionality to detect RPM dependencies

 meta/classes/package.bbclass                       |  5 +-
 meta/classes/package_rpm.bbclass                   | 25 ++++++++-
 meta/lib/oe/package.py                             | 60 +++++++++++++++-------
 meta/recipes-devtools/git/git.inc                  | 21 ++++++++
 .../rpm/files/0001-perl-disable-auto-reqs.patch    | 30 +++++++++++
 meta/recipes-devtools/rpm/rpm_git.bb               |  6 +++
 meta/recipes-extended/texinfo/texinfo_6.3.bb       |  6 +++
 7 files changed, 130 insertions(+), 23 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch

-- 
1.8.3.1

[PATCH 1/6] rpm: Add dependencies on bash, perl and python3-core

[Mark Hatle]

From: Peter Kjellerstedt <pkj@axis.com>

Also modify a Python script (pythondistdeps.py) to use Python 3.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/recipes-devtools/rpm/rpm_git.bb | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index d9ff2c7..513b569 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -106,6 +106,9 @@ do_install_append_class-target() {
 
 do_install_append () {
 	sed -i -e 's:${HOSTTOOLS_DIR}/::g' ${D}/${libdir}/rpm/macros
+
+	sed -i -e 's|/usr/bin/python|${USRBINPATH}/env ${PYTHON_PN}|' \
+	    ${D}${libdir}/rpm/pythondistdeps.py
 }
 
 FILES_${PN} += "${libdir}/rpm-plugins/*.so \
@@ -120,3 +123,5 @@ FILES_python3-rpm = "${PYTHON_SITEPACKAGES_DIR}/rpm/*"
 
 # rpm 5.x was packaging the rpm build tools separately
 RPROVIDES_${PN} += "rpm-build"
+
+RDEPENDS_${PN} = "bash perl python3-core"
-- 
1.8.3.1

[PATCH 2/6] git: Do not install git cvsserver and git svn by default

[Mark Hatle]

From: Peter Kjellerstedt <pkj@axis.com>

These git commands require Perl modules that do not exist in OE-Core.
Add PACKAGECONFIGs to enable them. Be aware though that if you enable
them you must also provide the missing dependencies.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/recipes-devtools/git/git.inc | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/meta/recipes-devtools/git/git.inc b/meta/recipes-devtools/git/git.inc
index 4390b8d..9b4c128 100644
--- a/meta/recipes-devtools/git/git.inc
+++ b/meta/recipes-devtools/git/git.inc
@@ -13,6 +13,10 @@ S = "${WORKDIR}/git-${PV}"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
 
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[cvsserver] = ""
+PACKAGECONFIG[svn] = ""
+
 EXTRA_OECONF = "--with-perl=${STAGING_BINDIR_NATIVE}/perl-native/perl \
 		--without-tcltk \
 "
@@ -54,6 +58,23 @@ perl_native_fixup () {
 	mkdir -p ${D}${libdir}
 	mv ${D}${exec_prefix}/lib/perl-native/perl ${D}${libdir}
 	rmdir -p ${D}${exec_prefix}/lib/perl-native || true
+
+	if [ ! "${@bb.utils.filter('PACKAGECONFIG', 'cvsserver', d)}" ]; then
+		# Only install the git cvsserver command if explicitly requested
+		# as it requires the DBI Perl module, which does not exist in
+		# OE-Core.
+		rm ${D}${libexecdir}/git-core/git-cvsserver \
+		   ${D}${bindir}/git-cvsserver
+	fi
+
+	if [ ! "${@bb.utils.filter('PACKAGECONFIG', 'svn', d)}" ]; then
+		# Only install the git svn command and all Git::SVN Perl modules
+		# if explicitly requested as they require the SVN::Core Perl
+		# module, which does not exist in OE-Core.
+		rm -r ${D}${libexecdir}/git-core/git-svn \
+		      ${D}${libdir}/perl/site_perl/*/Git/SVN*
+		sed -i -e '/SVN/d' ${D}${libdir}/perl/site_perl/*/auto/Git/.packlist
+	fi
 }
 
 REL_GIT_EXEC_PATH = "${@os.path.relpath(libexecdir, bindir)}/git-core"
-- 
1.8.3.1

[PATCH 3/6] texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs)

[Mark Hatle]

From: Peter Kjellerstedt <pkj@axis.com>

We do not build the Locale::gettext_xs Perl module and the code will
test for it and happily use Locale::gettext_pp instead if it is not
found. However, this still causes a file dependency on
perl(Locale::gettext_xs) to be generated, which must be satisfied by
adding an explicit provide for it.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/recipes-extended/texinfo/texinfo_6.3.bb | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/recipes-extended/texinfo/texinfo_6.3.bb b/meta/recipes-extended/texinfo/texinfo_6.3.bb
index d82731e..f58df92 100644
--- a/meta/recipes-extended/texinfo/texinfo_6.3.bb
+++ b/meta/recipes-extended/texinfo/texinfo_6.3.bb
@@ -79,4 +79,10 @@ FILES_${PN}-doc = "${infodir}/texinfo* \
                    ${datadir}/${tex_texinfo} \
                    ${mandir}/man1 ${mandir}/man5"
 
+# Lie about providing the Locale::gettext_xs module. It is not actually built,
+# but the code will test for it and if not found use Locale::gettext_pp instead.
+# However, this causes a file dependency on perl(Locale::gettext_xs) to be
+# generated, which must be satisfied.
+RPROVIDES_${PN} += "perl(Locale::gettext_xs)"
+
 BBCLASSEXTEND = "native nativesdk"
-- 
1.8.3.1

[PATCH 4/6] package_rpm.bbclass: Filter out unwanted file deps for nativesdk packages

[Mark Hatle]

From: Peter Kjellerstedt <pkj@axis.com>

Filter out any file dependencies on absolute paths and any
dependencies on Perl modules for nativesdk packages. It is assumed
that they will be provided by the native host if needed, and they mess
up the dependency handling if they are present.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/package_rpm.bbclass | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 5562074..1deaf83 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -7,10 +7,28 @@ RPMBUILD="rpmbuild"
 
 PKGWRITEDIRRPM = "${WORKDIR}/deploy-rpms"
 
-# Maintaining the perfile dependencies has singificant overhead when writing the 
+# Maintaining the perfile dependencies has singificant overhead when writing the
 # packages. When set, this value merges them for efficiency.
 MERGEPERFILEDEPS = "1"
 
+# Filter dependencies based on a provided function.
+def filter_deps(var, f):
+    import collections
+
+    depends_dict = bb.utils.explode_dep_versions2(var)
+    newdeps_dict = collections.OrderedDict()
+    for dep in depends_dict:
+        if f(dep):
+            newdeps_dict[dep] = depends_dict[dep]
+    return bb.utils.join_deps(newdeps_dict, commasep=False)
+
+# Filter out absolute paths (typically /bin/sh and /usr/bin/env) and any perl
+# dependencies for nativesdk packages.
+def filter_nativesdk_deps(srcname, var):
+    if var and srcname.startswith("nativesdk-"):
+        var = filter_deps(var, lambda dep: not dep.startswith('/') and dep != 'perl' and not dep.startswith('perl('))
+    return var
+
 # Construct per file dependencies file
 def write_rpm_perfiledata(srcname, d):
     workdir = d.getVar('WORKDIR')
@@ -26,7 +44,8 @@ def write_rpm_perfiledata(srcname, d):
             dependsflist = (d.getVar(dependsflist_key) or "")
             for dfile in dependsflist.split():
                 key = "FILE" + varname + "_" + dfile + "_" + pkg
-                depends_dict = bb.utils.explode_dep_versions(d.getVar(key) or "")
+                deps = filter_nativesdk_deps(srcname, d.getVar(key) or "")
+                depends_dict = bb.utils.explode_dep_versions(deps)
                 file = dfile.replace("@underscore@", "_")
                 file = file.replace("@closebrace@", "]")
                 file = file.replace("@openbrace@", "[")
@@ -359,6 +378,8 @@ python write_specfile () {
             splitrdepends = splitrdepends + " " + get_perfile('RDEPENDS', pkg, d)
             splitrprovides = splitrprovides + " " + get_perfile('RPROVIDES', pkg, d)
 
+        splitrdepends = filter_nativesdk_deps(srcname, splitrdepends)
+
         # Gather special src/first package data
         if srcname == splitname:
             srcrdepends    = splitrdepends
-- 
1.8.3.1

[PATCH 5/6] package.bbclass: Restore functionality to detect RPM dependencies

[Mark Hatle]

From: Peter Kjellerstedt <pkj@axis.com>

During the transition to dnf and rpm4, the functionality to
automatically make RPM determine dependencies was lost.

Before the transition, an OE specific tool called rpmdeps-oecore had
been added to the rpm suit. It was based on the rpmdeps tool that is
part of rpm. For each file specified on its command line, it would
output the provides and requires that RPM could determine.

During the transition to rpm4, rpmdeps-oecore was replaced with the
standard rpmdeps. However, what no one noticed was that unless rpmdeps
is given options, e.g., -P or -R, to tell it what it should output, it
will not output anything. Thus, it would do all the work to determine
the requirements, but would keep silent about it. And since no output
from rpmdeps is expected unless there are requirements, there were no
warnings indicating that everything was not working as expected.

Porting the old rpmdeps-oecore to work with rpm4 is not really
possible since it relied on being able to access internals of RPM that
are no longer available. However, it turned out that rpmdeps had a
debug option, --rpmfcdebug, that would output exactly the information
that we need, albeit in a different format and to stderr. To make this
usable, rpmdeps has now received a new option, --alldeps, which sends
the information we need to stdout.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/package.bbclass |  5 ++--
 meta/lib/oe/package.py       | 60 +++++++++++++++++++++++++++++++-------------
 2 files changed, 44 insertions(+), 21 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index d2fa617..2fe30da 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1453,7 +1453,7 @@ if [ x"$D" = "x" ]; then
 fi
 }
 
-RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --rcfile ${STAGING_LIBDIR_NATIVE}/rpm/rpmrc  --macros ${STAGING_LIBDIR_NATIVE}/rpm/macros --define '_rpmconfigdir ${STAGING_LIBDIR_NATIVE}/rpm/'"
+RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --alldeps"
 
 # Collect perfile run-time dependency metadata
 # Output:
@@ -1470,7 +1470,6 @@ python package_do_filedeps() {
     pkgdest = d.getVar('PKGDEST')
     packages = d.getVar('PACKAGES')
     rpmdeps = d.getVar('RPMDEPS')
-    magic = d.expand("${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc")
 
     def chunks(files, n):
         return [files[i:i+n] for i in range(0, len(files), n)]
@@ -1482,7 +1481,7 @@ python package_do_filedeps() {
         if pkg.endswith('-dbg') or pkg.endswith('-doc') or pkg.find('-locale-') != -1 or pkg.find('-localedata-') != -1 or pkg.find('-gconv-') != -1 or pkg.find('-charmap-') != -1 or pkg.startswith('kernel-module-') or pkg.endswith('-src'):
             continue
         for files in chunks(pkgfiles[pkg], 100):
-            pkglist.append((pkg, files, rpmdeps, pkgdest, magic))
+            pkglist.append((pkg, files, rpmdeps, pkgdest))
 
     processed = oe.utils.multiprocess_exec( pkglist, oe.package.filedeprunner)
 
diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 43748b2..a79c668 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -162,44 +162,68 @@ def file_translate(file):
 def filedeprunner(arg):
     import re, subprocess, shlex
 
-    (pkg, pkgfiles, rpmdeps, pkgdest, magic) = arg
+    (pkg, pkgfiles, rpmdeps, pkgdest) = arg
     provides = {}
     requires = {}
 
-    r = re.compile(r'[<>=]+ +[^ ]*')
+    file_re = re.compile(r'\s+\d+\s(.*)')
+    dep_re = re.compile(r'\s+(\S)\s+(.*)')
+    r = re.compile(r'[<>=]+\s+\S*')
 
     def process_deps(pipe, pkg, pkgdest, provides, requires):
+        file = None
         for line in pipe:
-            f = line.decode("utf-8").split(" ", 1)[0].strip()
-            line = line.decode("utf-8").split(" ", 1)[1].strip()
+            line = line.decode("utf-8")
 
-            if line.startswith("Requires:"):
+            m = file_re.match(line)
+            if m:
+                file = m.group(1)
+                file = file.replace(pkgdest + "/" + pkg, "")
+                file = file_translate(file)
+                continue
+
+            m = dep_re.match(line)
+            if not m or not file:
+                continue
+
+            type, dep = m.groups()
+
+            if type == 'R':
                 i = requires
-            elif line.startswith("Provides:"):
+            elif type == 'P':
                 i = provides
             else:
-                continue
+               continue
 
-            file = f.replace(pkgdest + "/" + pkg, "")
-            file = file_translate(file)
-            value = line.split(":", 1)[1].strip()
-            value = r.sub(r'(\g<0>)', value)
+            if dep.startswith("python("):
+                continue
 
-            if value.startswith("rpmlib("):
+            # Ignore all perl(VMS::...) and perl(Mac::...) dependencies. These
+            # are typically used conditionally from the Perl code, but are
+            # generated as unconditional dependencies.
+            if dep.startswith('perl(VMS::') or dep.startswith('perl(Mac::'):
                 continue
-            if value == "python":
+
+            # Ignore perl dependencies on .pl files.
+            if dep.startswith('perl(') and dep.endswith('.pl)'):
                 continue
+
+            # Remove perl versions and perl module versions since they typically
+            # do not make sense when used as package versions.
+            if dep.startswith('perl') and r.search(dep):
+                dep = dep.split()[0]
+
+            # Put parentheses around any version specifications.
+            dep = r.sub(r'(\g<0>)',dep)
+
             if file not in i:
                 i[file] = []
-            i[file].append(value)
+            i[file].append(dep)
 
         return provides, requires
 
-    env = os.environ.copy()
-    env["MAGIC"] = magic
-
     try:
-        dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE, env=env)
+        dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE)
         provides, requires = process_deps(dep_popen.stdout, pkg, pkgdest, provides, requires)
     except OSError as e:
         bb.error("rpmdeps: '%s' command failed, '%s'" % (shlex.split(rpmdeps) + pkgfiles, e))
-- 
1.8.3.1

[PATCH 6/6] rpm: Disable perl dependency generation

[Mark Hatle]

When rpmdeps files a perl script, it attempts to determine what it provides
and what it requires.  Often the requires are incorrect, within the context
of Wind River Linux.  This results in an error that DNF is unable to install
a package due to one or more unresolved dependencies.

In RPM5 we had disabled this behavior, the alternative is to require that all
perl scripts be 'complete', in that they only require things they absolutely
need and that OE provides.  If we ever enforce that, this commit can be
reverted.  Until they fall back to prior behavior (which also matches ipkg
and deb style packages.)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 .../rpm/files/0001-perl-disable-auto-reqs.patch    | 30 ++++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_git.bb               |  1 +
 2 files changed, 31 insertions(+)
 create mode 100644 meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch

diff --git a/meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch b/meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch
new file mode 100644
index 0000000..0e1a579
--- /dev/null
+++ b/meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch
@@ -0,0 +1,30 @@
+perl: disable auto requires
+
+When generating automatic requirements, it's possible for perl scripts to
+declare 'optional' dependencies.  These seem to often be incorrect and will
+cause installation failures in OE.  Instead of fixing the perl scripts, it
+was decided it is better to simply disable the automatic dependency
+generation.  This matches the behavior from the previous RPM5 implementation.
+
+Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
+
+Index: git/fileattrs/perl.attr
+===================================================================
+--- git.orig/fileattrs/perl.attr
++++ git/fileattrs/perl.attr
+@@ -1,3 +1,3 @@
+-%__perl_requires	%{_rpmconfigdir}/perl.req
++#__perl_requires	%{_rpmconfigdir}/perl.req
+ %__perl_magic		^.*[Pp]erl .*$
+ %__perl_flags		exeonly
+Index: git/fileattrs/perllib.attr
+===================================================================
+--- git.orig/fileattrs/perllib.attr
++++ git/fileattrs/perllib.attr
+@@ -1,5 +1,5 @@
+ %__perllib_provides	%{_rpmconfigdir}/perl.prov
+-%__perllib_requires	%{_rpmconfigdir}/perl.req
++#__perllib_requires	%{_rpmconfigdir}/perl.req
+ %__perllib_magic	^Perl[[:digit:]] module source.*
+ %__perllib_path		\\.pm$
+ %__perllib_flags	magic_and_path
diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index 513b569..d3d042f 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -42,6 +42,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \
            file://0002-Run-binary-package-creation-via-thread-pools.patch \
            file://0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch \
            file://0004-build-pack.c-remove-static-local-variables-from-buil.patch \
+           file://0001-perl-disable-auto-reqs.patch \
            "
 
 PV = "4.13.90+git${SRCPV}"
-- 
1.8.3.1

✗ patchtest: failure for Fix RPM4 regressions based on Pyro

[Patchwork]

== Series Details ==

Series: Fix RPM4 regressions based on Pyro
Revision: 1
URL   : https://patchwork.openembedded.org/series/8245/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Added patch file is missing Upstream-Status in the header [test_upstream_status_presence] 
  Suggested fix    Add Upstream-Status: <status> to the header of meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch (possible values: Pending, Submitted, Accepted, Backport, Denied, Inappropriate)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: [PATCH 0/6] Fix RPM4 regressions based on Pyro

[Mark Hatle]

On 8/15/17 4:19 PM, Mark Hatle wrote:
> Work is available at:
> 
> git://git.openembedded.org/openembedded-core-contrib mgh/rpm_dep_regression
> 
> This is work based on Peter Kjellerstedt's fixes for the rpm dependency
> generation.  (See poky-contrib pkj/rpm_regressions2 for the original work.)
> 
> The major difference is the last commit which disabled the perl requirement
> generation.  This resolves the only problem I found in Peter's patches.

Sorry, I forgot to mention, Peter's patches 2 and 3 attempt to address this in a
few places.  The last patch serves the same general purpose -- but it's better
in my opinion to fix these perl scripts when we find problems with them then
just ignore them.  (Even though ignoring is the standard behavior!)

--Mark

> From the comments to the original patch, the only not addressed was adding
> an OE self-test for this behavior.  I tried to do this, but I don't see any
> existing tests that build a recipe and inspect the results of the built
> package.  I'm not sure of any other way to identify if things are working or
> not.
> 
> To give you an idea of what this does, the provides for libc6, x86-64, prior
> to this change were:
> eglibc
> glibc = 2.26
> libc6 = 2.26-r0
> 
> After the change, the following were added (other remain):
> ld-linux-x86-64.so.2()(64bit)
> ld-linux-x86-64.so.2(GLIBC_2.2.5)(64bit)
> ld-linux-x86-64.so.2(GLIBC_2.3)(64bit)
> ld-linux-x86-64.so.2(GLIBC_2.4)(64bit)
> ld-linux-x86-64.so.2(GLIBC_PRIVATE)(64bit)
> libanl.so.1()(64bit)
> libanl.so.1(GLIBC_2.2.5)(64bit)
> libBrokenLocale.so.1()(64bit)
> libBrokenLocale.so.1(GLIBC_2.2.5)(64bit)
> libcrypt.so.1()(64bit)
> libcrypt.so.1(GLIBC_2.2.5)(64bit)
> libc.so.6()(64bit)
> libc.so.6(GLIBC_2.10)(64bit)
> libc.so.6(GLIBC_2.11)(64bit)
> libc.so.6(GLIBC_2.12)(64bit)
> libc.so.6(GLIBC_2.13)(64bit)
> libc.so.6(GLIBC_2.14)(64bit)
> libc.so.6(GLIBC_2.15)(64bit)
> libc.so.6(GLIBC_2.16)(64bit)
> libc.so.6(GLIBC_2.17)(64bit)
> libc.so.6(GLIBC_2.18)(64bit)
> libc.so.6(GLIBC_2.22)(64bit)
> libc.so.6(GLIBC_2.23)(64bit)
> libc.so.6(GLIBC_2.24)(64bit)
> libc.so.6(GLIBC_2.2.5)(64bit)
> libc.so.6(GLIBC_2.25)(64bit)
> libc.so.6(GLIBC_2.2.6)(64bit)
> libc.so.6(GLIBC_2.26)(64bit)
> libc.so.6(GLIBC_2.3.2)(64bit)
> libc.so.6(GLIBC_2.3.3)(64bit)
> libc.so.6(GLIBC_2.3.4)(64bit)
> libc.so.6(GLIBC_2.3)(64bit)
> libc.so.6(GLIBC_2.4)(64bit)
> libc.so.6(GLIBC_2.5)(64bit)
> libc.so.6(GLIBC_2.6)(64bit)
> libc.so.6(GLIBC_2.7)(64bit)
> libc.so.6(GLIBC_2.8)(64bit)
> libc.so.6(GLIBC_2.9)(64bit)
> libc.so.6(GLIBC_PRIVATE)(64bit)
> libdl.so.2()(64bit)
> libdl.so.2(GLIBC_2.2.5)(64bit)
> libdl.so.2(GLIBC_2.3.3)(64bit)
> libdl.so.2(GLIBC_2.3.4)(64bit)
> libdl.so.2(GLIBC_PRIVATE)(64bit)
> libm.so.6()(64bit)
> libm.so.6(GLIBC_2.15)(64bit)
> libm.so.6(GLIBC_2.18)(64bit)
> libm.so.6(GLIBC_2.23)(64bit)
> libm.so.6(GLIBC_2.24)(64bit)
> libm.so.6(GLIBC_2.2.5)(64bit)
> libm.so.6(GLIBC_2.25)(64bit)
> libm.so.6(GLIBC_2.26)(64bit)
> libm.so.6(GLIBC_2.4)(64bit)
> libmvec.so.1()(64bit)
> libmvec.so.1(GLIBC_2.22)(64bit)
> libnsl.so.1()(64bit)
> libnsl.so.1(GLIBC_2.2.5)(64bit)
> libnsl.so.1(GLIBC_PRIVATE)(64bit)
> libnss_compat.so.2()(64bit)
> libnss_compat.so.2(GLIBC_PRIVATE)(64bit)
> libnss_dns.so.2()(64bit)
> libnss_dns.so.2(GLIBC_PRIVATE)(64bit)
> libnss_files.so.2()(64bit)
> libnss_files.so.2(GLIBC_PRIVATE)(64bit)
> libpthread.so.0()(64bit)
> libpthread.so.0(GLIBC_2.11)(64bit)
> libpthread.so.0(GLIBC_2.12)(64bit)
> libpthread.so.0(GLIBC_2.18)(64bit)
> libpthread.so.0(GLIBC_2.2.5)(64bit)
> libpthread.so.0(GLIBC_2.2.6)(64bit)
> libpthread.so.0(GLIBC_2.3.2)(64bit)
> libpthread.so.0(GLIBC_2.3.3)(64bit)
> libpthread.so.0(GLIBC_2.3.4)(64bit)
> libpthread.so.0(GLIBC_2.4)(64bit)
> libpthread.so.0(GLIBC_PRIVATE)(64bit)
> libresolv.so.2()(64bit)
> libresolv.so.2(GLIBC_2.2.5)(64bit)
> libresolv.so.2(GLIBC_2.3.2)(64bit)
> libresolv.so.2(GLIBC_2.9)(64bit)
> libresolv.so.2(GLIBC_PRIVATE)(64bit)
> librt.so.1()(64bit)
> librt.so.1(GLIBC_2.2.5)(64bit)
> librt.so.1(GLIBC_2.3.3)(64bit)
> librt.so.1(GLIBC_2.3.4)(64bit)
> librt.so.1(GLIBC_2.4)(64bit)
> librt.so.1(GLIBC_2.7)(64bit)
> libutil.so.1()(64bit)
> libutil.so.1(GLIBC_2.2.5)(64bit)
> rtld(GNU_HASH)
> 
> 
> I have built and tested this for:
> 
> core-image-base core-image-minimal core-image-full-cmdline 
> core-image-kernel-dev core-image-lsb core-image-lsb-dev 
> core-image-lsb-sdk core-image-testmaster core-image-clutter 
> core-image-weston core-image-x11 core-image-sato core-image-sato-dev 
> core-image-sato-sdk core-image-sato-sdk-ptest
> 
> Also using qemux86, qemux86-64 and qemuarm64.
> 
> The following description is from Peter's original pull request:
> 
> Right before the release of Pyro (about a month ago), we noticed a
> problem with missing automatically determined runtime file
> dependencies after the transition to rpm4 and dnf. Unfortunately it
> has taken me quite a while to work out all the details, but here is
> the result finally.
> 
> ....
> 
> Then rpm is updated to make it possible to yet again access the
> runtime dependencies we need, and finally package.bbclass is updated
> to make use of them.
> 
> See the patches, especially the last one, for more in depth
> information on what went wrong during the transition to rpm4 and dnf,
> and how to solve it.
> 
> 
> Mark Hatle (1):
>   rpm: Disable perl dependency generation
> 
> Peter Kjellerstedt (5):
>   rpm: Add dependencies on bash, perl and python3-core
>   git: Do not install git cvsserver and git svn by default
>   texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs)
>   package_rpm.bbclass: Filter out unwanted file deps for nativesdk
>     packages
>   package.bbclass: Restore functionality to detect RPM dependencies
> 
>  meta/classes/package.bbclass                       |  5 +-
>  meta/classes/package_rpm.bbclass                   | 25 ++++++++-
>  meta/lib/oe/package.py                             | 60 +++++++++++++++-------
>  meta/recipes-devtools/git/git.inc                  | 21 ++++++++
>  .../rpm/files/0001-perl-disable-auto-reqs.patch    | 30 +++++++++++
>  meta/recipes-devtools/rpm/rpm_git.bb               |  6 +++
>  meta/recipes-extended/texinfo/texinfo_6.3.bb       |  6 +++
>  7 files changed, 130 insertions(+), 23 deletions(-)
>  create mode 100644 meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch
> 

Re: [PATCH 0/6] Fix RPM4 regressions based on Pyro

[Alexander Kanavin]

On 08/16/2017 12:19 AM, Mark Hatle wrote:
>  From the comments to the original patch, the only not addressed was adding
> an OE self-test for this behavior.  I tried to do this, but I don't see any
> existing tests that build a recipe and inspect the results of the built
> package.  I'm not sure of any other way to identify if things are working or
> not.

You can add a recipe to the meta-selftest layer that builds successfully 
only when this behaviour is working properly, and then build it in the 
selftest.


Alex

✗ patchtest: failure for Fix RPM4 regressions based on Pyro

[Patchwork]

== Series Details ==

Series: Fix RPM4 regressions based on Pyro
Revision: 1
URL   : https://patchwork.openembedded.org/series/8245/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Added patch file is missing Upstream-Status in the header [test_upstream_status_presence] 
  Suggested fix    Add Upstream-Status: <status> to the header of meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch (possible values: Pending, Submitted, Accepted, Backport, Denied, Inappropriate)

* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 4d65421f38)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe