* [PATCH 1/6 v2] rpm: Add dependencies on bash, perl and python3-core
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
@ 2017-08-15 21:41 ` Mark Hatle
2017-08-15 21:41 ` [PATCH 2/6 v2] git: Do not install git cvsserver and git svn by default Mark Hatle
` (7 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-15 21:41 UTC (permalink / raw)
To: openembedded-core; +Cc: peter.kjellerstedt
From: Peter Kjellerstedt <pkj@axis.com>
Also modify a Python script (pythondistdeps.py) to use Python 3.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-devtools/rpm/rpm_git.bb | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index d9ff2c7..513b569 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -106,6 +106,9 @@ do_install_append_class-target() {
do_install_append () {
sed -i -e 's:${HOSTTOOLS_DIR}/::g' ${D}/${libdir}/rpm/macros
+
+ sed -i -e 's|/usr/bin/python|${USRBINPATH}/env ${PYTHON_PN}|' \
+ ${D}${libdir}/rpm/pythondistdeps.py
}
FILES_${PN} += "${libdir}/rpm-plugins/*.so \
@@ -120,3 +123,5 @@ FILES_python3-rpm = "${PYTHON_SITEPACKAGES_DIR}/rpm/*"
# rpm 5.x was packaging the rpm build tools separately
RPROVIDES_${PN} += "rpm-build"
+
+RDEPENDS_${PN} = "bash perl python3-core"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 2/6 v2] git: Do not install git cvsserver and git svn by default
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
2017-08-15 21:41 ` [PATCH 1/6 v2] rpm: Add dependencies on bash, perl and python3-core Mark Hatle
@ 2017-08-15 21:41 ` Mark Hatle
2017-08-15 21:41 ` [PATCH 3/6 v2] texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs) Mark Hatle
` (6 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-15 21:41 UTC (permalink / raw)
To: openembedded-core; +Cc: peter.kjellerstedt
From: Peter Kjellerstedt <pkj@axis.com>
These git commands require Perl modules that do not exist in OE-Core.
Add PACKAGECONFIGs to enable them. Be aware though that if you enable
them you must also provide the missing dependencies.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-devtools/git/git.inc | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/meta/recipes-devtools/git/git.inc b/meta/recipes-devtools/git/git.inc
index 4390b8d..9b4c128 100644
--- a/meta/recipes-devtools/git/git.inc
+++ b/meta/recipes-devtools/git/git.inc
@@ -13,6 +13,10 @@ S = "${WORKDIR}/git-${PV}"
LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[cvsserver] = ""
+PACKAGECONFIG[svn] = ""
+
EXTRA_OECONF = "--with-perl=${STAGING_BINDIR_NATIVE}/perl-native/perl \
--without-tcltk \
"
@@ -54,6 +58,23 @@ perl_native_fixup () {
mkdir -p ${D}${libdir}
mv ${D}${exec_prefix}/lib/perl-native/perl ${D}${libdir}
rmdir -p ${D}${exec_prefix}/lib/perl-native || true
+
+ if [ ! "${@bb.utils.filter('PACKAGECONFIG', 'cvsserver', d)}" ]; then
+ # Only install the git cvsserver command if explicitly requested
+ # as it requires the DBI Perl module, which does not exist in
+ # OE-Core.
+ rm ${D}${libexecdir}/git-core/git-cvsserver \
+ ${D}${bindir}/git-cvsserver
+ fi
+
+ if [ ! "${@bb.utils.filter('PACKAGECONFIG', 'svn', d)}" ]; then
+ # Only install the git svn command and all Git::SVN Perl modules
+ # if explicitly requested as they require the SVN::Core Perl
+ # module, which does not exist in OE-Core.
+ rm -r ${D}${libexecdir}/git-core/git-svn \
+ ${D}${libdir}/perl/site_perl/*/Git/SVN*
+ sed -i -e '/SVN/d' ${D}${libdir}/perl/site_perl/*/auto/Git/.packlist
+ fi
}
REL_GIT_EXEC_PATH = "${@os.path.relpath(libexecdir, bindir)}/git-core"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 3/6 v2] texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs)
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
2017-08-15 21:41 ` [PATCH 1/6 v2] rpm: Add dependencies on bash, perl and python3-core Mark Hatle
2017-08-15 21:41 ` [PATCH 2/6 v2] git: Do not install git cvsserver and git svn by default Mark Hatle
@ 2017-08-15 21:41 ` Mark Hatle
2017-08-15 21:41 ` [PATCH 4/6 v2] package_rpm.bbclass: Filter out unwanted file deps for nativesdk packages Mark Hatle
` (5 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-15 21:41 UTC (permalink / raw)
To: openembedded-core; +Cc: peter.kjellerstedt
From: Peter Kjellerstedt <pkj@axis.com>
We do not build the Locale::gettext_xs Perl module and the code will
test for it and happily use Locale::gettext_pp instead if it is not
found. However, this still causes a file dependency on
perl(Locale::gettext_xs) to be generated, which must be satisfied by
adding an explicit provide for it.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-extended/texinfo/texinfo_6.3.bb | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/meta/recipes-extended/texinfo/texinfo_6.3.bb b/meta/recipes-extended/texinfo/texinfo_6.3.bb
index d82731e..f58df92 100644
--- a/meta/recipes-extended/texinfo/texinfo_6.3.bb
+++ b/meta/recipes-extended/texinfo/texinfo_6.3.bb
@@ -79,4 +79,10 @@ FILES_${PN}-doc = "${infodir}/texinfo* \
${datadir}/${tex_texinfo} \
${mandir}/man1 ${mandir}/man5"
+# Lie about providing the Locale::gettext_xs module. It is not actually built,
+# but the code will test for it and if not found use Locale::gettext_pp instead.
+# However, this causes a file dependency on perl(Locale::gettext_xs) to be
+# generated, which must be satisfied.
+RPROVIDES_${PN} += "perl(Locale::gettext_xs)"
+
BBCLASSEXTEND = "native nativesdk"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 4/6 v2] package_rpm.bbclass: Filter out unwanted file deps for nativesdk packages
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
` (2 preceding siblings ...)
2017-08-15 21:41 ` [PATCH 3/6 v2] texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs) Mark Hatle
@ 2017-08-15 21:41 ` Mark Hatle
2017-08-15 21:41 ` [PATCH 5/6 v2] package.bbclass: Restore functionality to detect RPM dependencies Mark Hatle
` (4 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-15 21:41 UTC (permalink / raw)
To: openembedded-core; +Cc: peter.kjellerstedt
From: Peter Kjellerstedt <pkj@axis.com>
Filter out any file dependencies on absolute paths and any
dependencies on Perl modules for nativesdk packages. It is assumed
that they will be provided by the native host if needed, and they mess
up the dependency handling if they are present.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/classes/package_rpm.bbclass | 25 +++++++++++++++++++++++--
1 file changed, 23 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 5562074..1deaf83 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -7,10 +7,28 @@ RPMBUILD="rpmbuild"
PKGWRITEDIRRPM = "${WORKDIR}/deploy-rpms"
-# Maintaining the perfile dependencies has singificant overhead when writing the
+# Maintaining the perfile dependencies has singificant overhead when writing the
# packages. When set, this value merges them for efficiency.
MERGEPERFILEDEPS = "1"
+# Filter dependencies based on a provided function.
+def filter_deps(var, f):
+ import collections
+
+ depends_dict = bb.utils.explode_dep_versions2(var)
+ newdeps_dict = collections.OrderedDict()
+ for dep in depends_dict:
+ if f(dep):
+ newdeps_dict[dep] = depends_dict[dep]
+ return bb.utils.join_deps(newdeps_dict, commasep=False)
+
+# Filter out absolute paths (typically /bin/sh and /usr/bin/env) and any perl
+# dependencies for nativesdk packages.
+def filter_nativesdk_deps(srcname, var):
+ if var and srcname.startswith("nativesdk-"):
+ var = filter_deps(var, lambda dep: not dep.startswith('/') and dep != 'perl' and not dep.startswith('perl('))
+ return var
+
# Construct per file dependencies file
def write_rpm_perfiledata(srcname, d):
workdir = d.getVar('WORKDIR')
@@ -26,7 +44,8 @@ def write_rpm_perfiledata(srcname, d):
dependsflist = (d.getVar(dependsflist_key) or "")
for dfile in dependsflist.split():
key = "FILE" + varname + "_" + dfile + "_" + pkg
- depends_dict = bb.utils.explode_dep_versions(d.getVar(key) or "")
+ deps = filter_nativesdk_deps(srcname, d.getVar(key) or "")
+ depends_dict = bb.utils.explode_dep_versions(deps)
file = dfile.replace("@underscore@", "_")
file = file.replace("@closebrace@", "]")
file = file.replace("@openbrace@", "[")
@@ -359,6 +378,8 @@ python write_specfile () {
splitrdepends = splitrdepends + " " + get_perfile('RDEPENDS', pkg, d)
splitrprovides = splitrprovides + " " + get_perfile('RPROVIDES', pkg, d)
+ splitrdepends = filter_nativesdk_deps(srcname, splitrdepends)
+
# Gather special src/first package data
if srcname == splitname:
srcrdepends = splitrdepends
--
1.8.3.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 5/6 v2] package.bbclass: Restore functionality to detect RPM dependencies
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
` (3 preceding siblings ...)
2017-08-15 21:41 ` [PATCH 4/6 v2] package_rpm.bbclass: Filter out unwanted file deps for nativesdk packages Mark Hatle
@ 2017-08-15 21:41 ` Mark Hatle
2017-08-15 21:41 ` [PATCH 6/6 v2] rpm: Disable perl dependency generation Mark Hatle
` (3 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-15 21:41 UTC (permalink / raw)
To: openembedded-core; +Cc: peter.kjellerstedt
From: Peter Kjellerstedt <pkj@axis.com>
During the transition to dnf and rpm4, the functionality to
automatically make RPM determine dependencies was lost.
Before the transition, an OE specific tool called rpmdeps-oecore had
been added to the rpm suit. It was based on the rpmdeps tool that is
part of rpm. For each file specified on its command line, it would
output the provides and requires that RPM could determine.
During the transition to rpm4, rpmdeps-oecore was replaced with the
standard rpmdeps. However, what no one noticed was that unless rpmdeps
is given options, e.g., -P or -R, to tell it what it should output, it
will not output anything. Thus, it would do all the work to determine
the requirements, but would keep silent about it. And since no output
from rpmdeps is expected unless there are requirements, there were no
warnings indicating that everything was not working as expected.
Porting the old rpmdeps-oecore to work with rpm4 is not really
possible since it relied on being able to access internals of RPM that
are no longer available. However, it turned out that rpmdeps had a
debug option, --rpmfcdebug, that would output exactly the information
that we need, albeit in a different format and to stderr. To make this
usable, rpmdeps has now received a new option, --alldeps, which sends
the information we need to stdout.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/classes/package.bbclass | 5 ++--
meta/lib/oe/package.py | 60 +++++++++++++++++++++++++++++++-------------
2 files changed, 44 insertions(+), 21 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index d2fa617..2fe30da 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1453,7 +1453,7 @@ if [ x"$D" = "x" ]; then
fi
}
-RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --rcfile ${STAGING_LIBDIR_NATIVE}/rpm/rpmrc --macros ${STAGING_LIBDIR_NATIVE}/rpm/macros --define '_rpmconfigdir ${STAGING_LIBDIR_NATIVE}/rpm/'"
+RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --alldeps"
# Collect perfile run-time dependency metadata
# Output:
@@ -1470,7 +1470,6 @@ python package_do_filedeps() {
pkgdest = d.getVar('PKGDEST')
packages = d.getVar('PACKAGES')
rpmdeps = d.getVar('RPMDEPS')
- magic = d.expand("${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc")
def chunks(files, n):
return [files[i:i+n] for i in range(0, len(files), n)]
@@ -1482,7 +1481,7 @@ python package_do_filedeps() {
if pkg.endswith('-dbg') or pkg.endswith('-doc') or pkg.find('-locale-') != -1 or pkg.find('-localedata-') != -1 or pkg.find('-gconv-') != -1 or pkg.find('-charmap-') != -1 or pkg.startswith('kernel-module-') or pkg.endswith('-src'):
continue
for files in chunks(pkgfiles[pkg], 100):
- pkglist.append((pkg, files, rpmdeps, pkgdest, magic))
+ pkglist.append((pkg, files, rpmdeps, pkgdest))
processed = oe.utils.multiprocess_exec( pkglist, oe.package.filedeprunner)
diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 43748b2..a79c668 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -162,44 +162,68 @@ def file_translate(file):
def filedeprunner(arg):
import re, subprocess, shlex
- (pkg, pkgfiles, rpmdeps, pkgdest, magic) = arg
+ (pkg, pkgfiles, rpmdeps, pkgdest) = arg
provides = {}
requires = {}
- r = re.compile(r'[<>=]+ +[^ ]*')
+ file_re = re.compile(r'\s+\d+\s(.*)')
+ dep_re = re.compile(r'\s+(\S)\s+(.*)')
+ r = re.compile(r'[<>=]+\s+\S*')
def process_deps(pipe, pkg, pkgdest, provides, requires):
+ file = None
for line in pipe:
- f = line.decode("utf-8").split(" ", 1)[0].strip()
- line = line.decode("utf-8").split(" ", 1)[1].strip()
+ line = line.decode("utf-8")
- if line.startswith("Requires:"):
+ m = file_re.match(line)
+ if m:
+ file = m.group(1)
+ file = file.replace(pkgdest + "/" + pkg, "")
+ file = file_translate(file)
+ continue
+
+ m = dep_re.match(line)
+ if not m or not file:
+ continue
+
+ type, dep = m.groups()
+
+ if type == 'R':
i = requires
- elif line.startswith("Provides:"):
+ elif type == 'P':
i = provides
else:
- continue
+ continue
- file = f.replace(pkgdest + "/" + pkg, "")
- file = file_translate(file)
- value = line.split(":", 1)[1].strip()
- value = r.sub(r'(\g<0>)', value)
+ if dep.startswith("python("):
+ continue
- if value.startswith("rpmlib("):
+ # Ignore all perl(VMS::...) and perl(Mac::...) dependencies. These
+ # are typically used conditionally from the Perl code, but are
+ # generated as unconditional dependencies.
+ if dep.startswith('perl(VMS::') or dep.startswith('perl(Mac::'):
continue
- if value == "python":
+
+ # Ignore perl dependencies on .pl files.
+ if dep.startswith('perl(') and dep.endswith('.pl)'):
continue
+
+ # Remove perl versions and perl module versions since they typically
+ # do not make sense when used as package versions.
+ if dep.startswith('perl') and r.search(dep):
+ dep = dep.split()[0]
+
+ # Put parentheses around any version specifications.
+ dep = r.sub(r'(\g<0>)',dep)
+
if file not in i:
i[file] = []
- i[file].append(value)
+ i[file].append(dep)
return provides, requires
- env = os.environ.copy()
- env["MAGIC"] = magic
-
try:
- dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE, env=env)
+ dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE)
provides, requires = process_deps(dep_popen.stdout, pkg, pkgdest, provides, requires)
except OSError as e:
bb.error("rpmdeps: '%s' command failed, '%s'" % (shlex.split(rpmdeps) + pkgfiles, e))
--
1.8.3.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 6/6 v2] rpm: Disable perl dependency generation
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
` (4 preceding siblings ...)
2017-08-15 21:41 ` [PATCH 5/6 v2] package.bbclass: Restore functionality to detect RPM dependencies Mark Hatle
@ 2017-08-15 21:41 ` Mark Hatle
2017-08-16 5:19 ` [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro akuster808
` (2 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-15 21:41 UTC (permalink / raw)
To: openembedded-core; +Cc: peter.kjellerstedt
When rpmdeps files a perl script, it attempts to determine what it provides
and what it requires. Often the requires are incorrect, within the context
of Wind River Linux. This results in an error that DNF is unable to install
a package due to one or more unresolved dependencies.
In RPM5 we had disabled this behavior, the alternative is to require that all
perl scripts be 'complete', in that they only require things they absolutely
need and that OE provides. If we ever enforce that, this commit can be
reverted. Until they fall back to prior behavior (which also matches ipkg
and deb style packages.)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
.../rpm/files/0001-perl-disable-auto-reqs.patch | 32 ++++++++++++++++++++++
meta/recipes-devtools/rpm/rpm_git.bb | 1 +
2 files changed, 33 insertions(+)
create mode 100644 meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch
diff --git a/meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch b/meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch
new file mode 100644
index 0000000..a6c5869
--- /dev/null
+++ b/meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch
@@ -0,0 +1,32 @@
+perl: disable auto requires
+
+When generating automatic requirements, it's possible for perl scripts to
+declare 'optional' dependencies. These seem to often be incorrect and will
+cause installation failures in OE. Instead of fixing the perl scripts, it
+was decided it is better to simply disable the automatic dependency
+generation. This matches the behavior from the previous RPM5 implementation.
+
+Upstream-Status: Inappropriate [OE specific configuration]
+
+Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
+
+Index: git/fileattrs/perl.attr
+===================================================================
+--- git.orig/fileattrs/perl.attr
++++ git/fileattrs/perl.attr
+@@ -1,3 +1,3 @@
+-%__perl_requires %{_rpmconfigdir}/perl.req
++#__perl_requires %{_rpmconfigdir}/perl.req
+ %__perl_magic ^.*[Pp]erl .*$
+ %__perl_flags exeonly
+Index: git/fileattrs/perllib.attr
+===================================================================
+--- git.orig/fileattrs/perllib.attr
++++ git/fileattrs/perllib.attr
+@@ -1,5 +1,5 @@
+ %__perllib_provides %{_rpmconfigdir}/perl.prov
+-%__perllib_requires %{_rpmconfigdir}/perl.req
++#__perllib_requires %{_rpmconfigdir}/perl.req
+ %__perllib_magic ^Perl[[:digit:]] module source.*
+ %__perllib_path \\.pm$
+ %__perllib_flags magic_and_path
diff --git a/meta/recipes-devtools/rpm/rpm_git.bb b/meta/recipes-devtools/rpm/rpm_git.bb
index 513b569..d3d042f 100644
--- a/meta/recipes-devtools/rpm/rpm_git.bb
+++ b/meta/recipes-devtools/rpm/rpm_git.bb
@@ -42,6 +42,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \
file://0002-Run-binary-package-creation-via-thread-pools.patch \
file://0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch \
file://0004-build-pack.c-remove-static-local-variables-from-buil.patch \
+ file://0001-perl-disable-auto-reqs.patch \
"
PV = "4.13.90+git${SRCPV}"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
` (5 preceding siblings ...)
2017-08-15 21:41 ` [PATCH 6/6 v2] rpm: Disable perl dependency generation Mark Hatle
@ 2017-08-16 5:19 ` akuster808
2017-08-16 13:43 ` Richard Purdie
2017-08-16 13:34 ` Richard Purdie
2017-08-16 15:32 ` ✗ patchtest: failure for Fix RPM4 regressions based on Pyro (rev2) Patchwork
8 siblings, 1 reply; 19+ messages in thread
From: akuster808 @ 2017-08-16 5:19 UTC (permalink / raw)
To: Mark Hatle, openembedded-core; +Cc: peter.kjellerstedt
On 08/15/2017 02:41 PM, Mark Hatle wrote:
Are these meant for Pyro?
- armin
> v2:
> Fixed the patch header (6/6) to silence patchwork/patchtest failure. All
> other patches remain the same.
>
> v1:
>
> Work is available at:
>
> git://git.openembedded.org/openembedded-core-contrib mgh/rpm_dep_regression
>
> This is work based on Peter Kjellerstedt's fixes for the rpm dependency
> generation. (See poky-contrib pkj/rpm_regressions2 for the original work.)
>
> The major difference is the last commit which disabled the perl requirement
> generation. This resolves the only problem I found in Peter's patches.
>
> Peter's patches 2 and 3 attempt to address this in a few places. The
> last patch serves the same general purpose -- but it's better in my
> opinion to fix these perl scripts when we find problems with them then
> just ignore them. (Even though ignoring is the standard behavior!)
>
> From the comments to the original patch, the only not addressed was adding
> an OE self-test for this behavior. I tried to do this, but I don't see any
> existing tests that build a recipe and inspect the results of the built
> package. I'm not sure of any other way to identify if things are working or
> not.
>
> To give you an idea of what this does, the provides for libc6, x86-64, prior
> to this change were:
> eglibc
> glibc = 2.26
> libc6 = 2.26-r0
>
> After the change, the following were added (other remain):
> ld-linux-x86-64.so.2()(64bit)
> ld-linux-x86-64.so.2(GLIBC_2.2.5)(64bit)
> ld-linux-x86-64.so.2(GLIBC_2.3)(64bit)
> ld-linux-x86-64.so.2(GLIBC_2.4)(64bit)
> ld-linux-x86-64.so.2(GLIBC_PRIVATE)(64bit)
> libanl.so.1()(64bit)
> libanl.so.1(GLIBC_2.2.5)(64bit)
> libBrokenLocale.so.1()(64bit)
> libBrokenLocale.so.1(GLIBC_2.2.5)(64bit)
> libcrypt.so.1()(64bit)
> libcrypt.so.1(GLIBC_2.2.5)(64bit)
> libc.so.6()(64bit)
> libc.so.6(GLIBC_2.10)(64bit)
> libc.so.6(GLIBC_2.11)(64bit)
> libc.so.6(GLIBC_2.12)(64bit)
> libc.so.6(GLIBC_2.13)(64bit)
> libc.so.6(GLIBC_2.14)(64bit)
> libc.so.6(GLIBC_2.15)(64bit)
> libc.so.6(GLIBC_2.16)(64bit)
> libc.so.6(GLIBC_2.17)(64bit)
> libc.so.6(GLIBC_2.18)(64bit)
> libc.so.6(GLIBC_2.22)(64bit)
> libc.so.6(GLIBC_2.23)(64bit)
> libc.so.6(GLIBC_2.24)(64bit)
> libc.so.6(GLIBC_2.2.5)(64bit)
> libc.so.6(GLIBC_2.25)(64bit)
> libc.so.6(GLIBC_2.2.6)(64bit)
> libc.so.6(GLIBC_2.26)(64bit)
> libc.so.6(GLIBC_2.3.2)(64bit)
> libc.so.6(GLIBC_2.3.3)(64bit)
> libc.so.6(GLIBC_2.3.4)(64bit)
> libc.so.6(GLIBC_2.3)(64bit)
> libc.so.6(GLIBC_2.4)(64bit)
> libc.so.6(GLIBC_2.5)(64bit)
> libc.so.6(GLIBC_2.6)(64bit)
> libc.so.6(GLIBC_2.7)(64bit)
> libc.so.6(GLIBC_2.8)(64bit)
> libc.so.6(GLIBC_2.9)(64bit)
> libc.so.6(GLIBC_PRIVATE)(64bit)
> libdl.so.2()(64bit)
> libdl.so.2(GLIBC_2.2.5)(64bit)
> libdl.so.2(GLIBC_2.3.3)(64bit)
> libdl.so.2(GLIBC_2.3.4)(64bit)
> libdl.so.2(GLIBC_PRIVATE)(64bit)
> libm.so.6()(64bit)
> libm.so.6(GLIBC_2.15)(64bit)
> libm.so.6(GLIBC_2.18)(64bit)
> libm.so.6(GLIBC_2.23)(64bit)
> libm.so.6(GLIBC_2.24)(64bit)
> libm.so.6(GLIBC_2.2.5)(64bit)
> libm.so.6(GLIBC_2.25)(64bit)
> libm.so.6(GLIBC_2.26)(64bit)
> libm.so.6(GLIBC_2.4)(64bit)
> libmvec.so.1()(64bit)
> libmvec.so.1(GLIBC_2.22)(64bit)
> libnsl.so.1()(64bit)
> libnsl.so.1(GLIBC_2.2.5)(64bit)
> libnsl.so.1(GLIBC_PRIVATE)(64bit)
> libnss_compat.so.2()(64bit)
> libnss_compat.so.2(GLIBC_PRIVATE)(64bit)
> libnss_dns.so.2()(64bit)
> libnss_dns.so.2(GLIBC_PRIVATE)(64bit)
> libnss_files.so.2()(64bit)
> libnss_files.so.2(GLIBC_PRIVATE)(64bit)
> libpthread.so.0()(64bit)
> libpthread.so.0(GLIBC_2.11)(64bit)
> libpthread.so.0(GLIBC_2.12)(64bit)
> libpthread.so.0(GLIBC_2.18)(64bit)
> libpthread.so.0(GLIBC_2.2.5)(64bit)
> libpthread.so.0(GLIBC_2.2.6)(64bit)
> libpthread.so.0(GLIBC_2.3.2)(64bit)
> libpthread.so.0(GLIBC_2.3.3)(64bit)
> libpthread.so.0(GLIBC_2.3.4)(64bit)
> libpthread.so.0(GLIBC_2.4)(64bit)
> libpthread.so.0(GLIBC_PRIVATE)(64bit)
> libresolv.so.2()(64bit)
> libresolv.so.2(GLIBC_2.2.5)(64bit)
> libresolv.so.2(GLIBC_2.3.2)(64bit)
> libresolv.so.2(GLIBC_2.9)(64bit)
> libresolv.so.2(GLIBC_PRIVATE)(64bit)
> librt.so.1()(64bit)
> librt.so.1(GLIBC_2.2.5)(64bit)
> librt.so.1(GLIBC_2.3.3)(64bit)
> librt.so.1(GLIBC_2.3.4)(64bit)
> librt.so.1(GLIBC_2.4)(64bit)
> librt.so.1(GLIBC_2.7)(64bit)
> libutil.so.1()(64bit)
> libutil.so.1(GLIBC_2.2.5)(64bit)
> rtld(GNU_HASH)
>
>
> I have built and tested this for:
>
> core-image-base core-image-minimal core-image-full-cmdline
> core-image-kernel-dev core-image-lsb core-image-lsb-dev
> core-image-lsb-sdk core-image-testmaster core-image-clutter
> core-image-weston core-image-x11 core-image-sato core-image-sato-dev
> core-image-sato-sdk core-image-sato-sdk-ptest
>
> Also using qemux86, qemux86-64 and qemuarm64.
>
> The following description is from Peter's original pull request:
>
> Right before the release of Pyro (about a month ago), we noticed a
> problem with missing automatically determined runtime file
> dependencies after the transition to rpm4 and dnf. Unfortunately it
> has taken me quite a while to work out all the details, but here is
> the result finally.
>
> ....
>
> Then rpm is updated to make it possible to yet again access the
> runtime dependencies we need, and finally package.bbclass is updated
> to make use of them.
>
> See the patches, especially the last one, for more in depth
> information on what went wrong during the transition to rpm4 and dnf,
> and how to solve it.
>
>
> Mark Hatle (1):
> rpm: Disable perl dependency generation
>
> Peter Kjellerstedt (5):
> rpm: Add dependencies on bash, perl and python3-core
> git: Do not install git cvsserver and git svn by default
> texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs)
> package_rpm.bbclass: Filter out unwanted file deps for nativesdk
> packages
> package.bbclass: Restore functionality to detect RPM dependencies
>
> meta/classes/package.bbclass | 5 +-
> meta/classes/package_rpm.bbclass | 25 ++++++++-
> meta/lib/oe/package.py | 60 +++++++++++++++-------
> meta/recipes-devtools/git/git.inc | 21 ++++++++
> .../rpm/files/0001-perl-disable-auto-reqs.patch | 30 +++++++++++
> meta/recipes-devtools/rpm/rpm_git.bb | 6 +++
> meta/recipes-extended/texinfo/texinfo_6.3.bb | 6 +++
> 7 files changed, 130 insertions(+), 23 deletions(-)
> create mode 100644 meta/recipes-devtools/rpm/files/0001-perl-disable-auto-reqs.patch
>
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-16 5:19 ` [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro akuster808
@ 2017-08-16 13:43 ` Richard Purdie
2017-08-16 20:43 ` Peter Kjellerstedt
0 siblings, 1 reply; 19+ messages in thread
From: Richard Purdie @ 2017-08-16 13:43 UTC (permalink / raw)
To: akuster808, Mark Hatle, openembedded-core; +Cc: peter.kjellerstedt
On Tue, 2017-08-15 at 22:19 -0700, akuster808 wrote:
>
> On 08/15/2017 02:41 PM, Mark Hatle wrote:
>
> Are these meant for Pyro?
Some of the initial pieces may be ok for pyro, the actual enablement of
the rpm deps piece cannot be taken back though as the world will likely
break.
Cheers,
Richard
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-16 13:43 ` Richard Purdie
@ 2017-08-16 20:43 ` Peter Kjellerstedt
0 siblings, 0 replies; 19+ messages in thread
From: Peter Kjellerstedt @ 2017-08-16 20:43 UTC (permalink / raw)
To: Richard Purdie, akuster808, Mark Hatle,
openembedded-core@lists.openembedded.org
> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org
> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of
> Richard Purdie
> Sent: den 16 augusti 2017 15:44
> To: akuster808 <akuster808@gmail.com>; Mark Hatle
> <mark.hatle@windriver.com>; openembedded-core@lists.openembedded.org
> Cc: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Subject: Re: [OE-core] [PATCH 0/6 v2] Fix RPM4 regressions based on
> Pyro
>
> On Tue, 2017-08-15 at 22:19 -0700, akuster808 wrote:
> >
> > On 08/15/2017 02:41 PM, Mark Hatle wrote:
> >
> > Are these meant for Pyro?
>
> Some of the initial pieces may be ok for pyro, the actual enablement of
> the rpm deps piece cannot be taken back though as the world will likely
> break.
>
> Cheers,
>
> Richard
Getting the fixes into Pyro has been my intention from the beginning,
as I otherwise need to carry a number of backported bbclasses and other
files in our layers, which is always a maintenance burden. The lack of
of these fixes in Pyro is the main reason we have not been able to
upgrade our distribution to Pyro yet.
I believe it will be possible to get all the changes into Pyro. The
only thing that cannot be done by default is to enable it. However,
it should be easy to make that conditional (basically the only thing
needed is to determine whether --alldeps is passed to rpmdeps or not).
Once all the changes have landed in master, I can prepare a version
for Pyro that does that.
//Peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
` (6 preceding siblings ...)
2017-08-16 5:19 ` [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro akuster808
@ 2017-08-16 13:34 ` Richard Purdie
2017-08-16 15:18 ` Mark Hatle
2017-08-16 15:37 ` Mark Hatle
2017-08-16 15:32 ` ✗ patchtest: failure for Fix RPM4 regressions based on Pyro (rev2) Patchwork
8 siblings, 2 replies; 19+ messages in thread
From: Richard Purdie @ 2017-08-16 13:34 UTC (permalink / raw)
To: Mark Hatle, openembedded-core; +Cc: peter.kjellerstedt
On Tue, 2017-08-15 at 16:41 -0500, Mark Hatle wrote:
> From the comments to the original patch, the only not addressed was
> adding an OE self-test for this behavior. I tried to do this, but I
> don't see any existing tests that build a recipe and inspect the
> results of the built package. I'm not sure of any other way to
> identify if things are working or not.
oe-selftest can do this.
> I have built and tested this for:
>
> core-image-base core-image-minimal core-image-full-cmdline
> core-image-kernel-dev core-image-lsb core-image-lsb-dev
> core-image-lsb-sdk core-image-testmaster core-image-clutter
> core-image-weston core-image-x11 core-image-sato core-image-sato-dev
> core-image-sato-sdk core-image-sato-sdk-ptest
As we both suspected there are some issues with enabling this again,
many are showcased here:
https://autobuilder.yoctoproject.org/main/builders/nightly-arm-lsb/builds/1156/steps/BuildImages/logs/stdio
* ltp has ksh/csh requirements in ptest
* package dhcp-client-4.3.5-r0.0.armv5e requires libdns.so.170, but none of the providers can be installed
* nothing provides /usr/bin/python33 needed by bind-9.10.5+P3-r0.0.armv5e
ERROR: openssl-1.1.0f-r0 do_package_qa: QA Issue: /usr/lib/openssl/ptest/util/find-unused-errs contained in package openssl-ptest requires /bin/bash, but no providers found in RDEPENDS_openssl-ptest? [file-rdeps]
ERROR: openssl-1.1.0f-r0 do_package_qa: QA Issue: /usr/lib/openssl/ptest/fuzz/helper.py contained in package openssl-ptest requires /usr/bin/python, but no providers found in RDEPENDS_openssl-ptest? [file-rdeps]
ERROR: bind-9.10.5-P3-r0 do_package_qa: QA Issue: /usr/sbin/dnssec-checkds contained in package bind requires /usr/bin/python33, but no providers found in RDEPENDS_bind? [file-rdeps]
I suspect the key piece of testing you were missing may have been ptest enablement?
I might try and sort a few patches for pieces of this.
Cheers,
Richard
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-16 13:34 ` Richard Purdie
@ 2017-08-16 15:18 ` Mark Hatle
2017-08-16 15:37 ` Mark Hatle
1 sibling, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-16 15:18 UTC (permalink / raw)
To: Richard Purdie, openembedded-core; +Cc: peter.kjellerstedt
On 8/16/17 8:34 AM, Richard Purdie wrote:
> On Tue, 2017-08-15 at 16:41 -0500, Mark Hatle wrote:
>> From the comments to the original patch, the only not addressed was
>> adding an OE self-test for this behavior. I tried to do this, but I
>> don't see any existing tests that build a recipe and inspect the
>> results of the built package. I'm not sure of any other way to
>> identify if things are working or not.
>
> oe-selftest can do this.
>
>> I have built and tested this for:
>>
>> core-image-base core-image-minimal core-image-full-cmdline
>> core-image-kernel-dev core-image-lsb core-image-lsb-dev
>> core-image-lsb-sdk core-image-testmaster core-image-clutter
>> core-image-weston core-image-x11 core-image-sato core-image-sato-dev
>> core-image-sato-sdk core-image-sato-sdk-ptest
>
> As we both suspected there are some issues with enabling this again,
> many are showcased here:
>
> https://autobuilder.yoctoproject.org/main/builders/nightly-arm-lsb/builds/1156/steps/BuildImages/logs/stdio
>
> * ltp has ksh/csh requirements in ptest
> * package dhcp-client-4.3.5-r0.0.armv5e requires libdns.so.170, but none of the providers can be installed
> * nothing provides /usr/bin/python33 needed by bind-9.10.5+P3-r0.0.armv5e
> ERROR: openssl-1.1.0f-r0 do_package_qa: QA Issue: /usr/lib/openssl/ptest/util/find-unused-errs contained in package openssl-ptest requires /bin/bash, but no providers found in RDEPENDS_openssl-ptest? [file-rdeps]
> ERROR: openssl-1.1.0f-r0 do_package_qa: QA Issue: /usr/lib/openssl/ptest/fuzz/helper.py contained in package openssl-ptest requires /usr/bin/python, but no providers found in RDEPENDS_openssl-ptest? [file-rdeps]
> ERROR: bind-9.10.5-P3-r0 do_package_qa: QA Issue: /usr/sbin/dnssec-checkds contained in package bind requires /usr/bin/python33, but no providers found in RDEPENDS_bind? [file-rdeps]
The above are valid packaging bugs. RP may get there first, but I'll try to get
patches setup for them.
--Mark
> I suspect the key piece of testing you were missing may have been ptest enablement?
>
> I might try and sort a few patches for pieces of this.
>
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-16 13:34 ` Richard Purdie
2017-08-16 15:18 ` Mark Hatle
@ 2017-08-16 15:37 ` Mark Hatle
2017-08-17 10:53 ` Alexander Kanavin
1 sibling, 1 reply; 19+ messages in thread
From: Mark Hatle @ 2017-08-16 15:37 UTC (permalink / raw)
To: Richard Purdie, openembedded-core; +Cc: peter.kjellerstedt
On 8/16/17 8:34 AM, Richard Purdie wrote:
> On Tue, 2017-08-15 at 16:41 -0500, Mark Hatle wrote:
>> From the comments to the original patch, the only not addressed was
>> adding an OE self-test for this behavior. I tried to do this, but I
>> don't see any existing tests that build a recipe and inspect the
>> results of the built package. I'm not sure of any other way to
>> identify if things are working or not.
> oe-selftest can do this.
>
Are there any examples? I was looking there and simply didn't see any. I saw
the packaging for instance exercised python interfaces, version comparison
tools, etc.. but nothing that built a recipe/package and then verified the
contents were 'correct'.
Ideally the test would be to fabricate something with a known set of file
dependencies, produce a package from it and then verify that the package
properly included those dependencies.
I had looked at just picking some random library out of the deploy directory,
and doing a pattern search on it's provides.. but I'm not sure that type of test
would be very robust.
--Mark
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-16 15:37 ` Mark Hatle
@ 2017-08-17 10:53 ` Alexander Kanavin
2017-08-17 13:01 ` Peter Kjellerstedt
2017-08-17 14:37 ` Mark Hatle
0 siblings, 2 replies; 19+ messages in thread
From: Alexander Kanavin @ 2017-08-17 10:53 UTC (permalink / raw)
To: Mark Hatle, Richard Purdie, openembedded-core; +Cc: peter.kjellerstedt
On 08/16/2017 06:37 PM, Mark Hatle wrote:
> Are there any examples? I was looking there and simply didn't see any. I saw
> the packaging for instance exercised python interfaces, version comparison
> tools, etc.. but nothing that built a recipe/package and then verified the
> contents were 'correct'.
>
> Ideally the test would be to fabricate something with a known set of file
> dependencies, produce a package from it and then verify that the package
> properly included those dependencies.
>
> I had looked at just picking some random library out of the deploy directory,
> and doing a pattern search on it's provides.. but I'm not sure that type of test
> would be very robust.
How was this regression discovered in the first place? I guess that some
recipe used to build without error with rpm5, and stopped building with
rpm4. This is what you could test for: isolate the problematic bit,
remove everything else, and write a simple recipe(s) that can be placed
into meta-selftest. Then simply build those from the selftest; if they
build without error, you don't need to do more sophisticated checks.
Alex
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-17 10:53 ` Alexander Kanavin
@ 2017-08-17 13:01 ` Peter Kjellerstedt
2017-08-17 13:17 ` Alexander Kanavin
2017-08-17 14:41 ` Mark Hatle
2017-08-17 14:37 ` Mark Hatle
1 sibling, 2 replies; 19+ messages in thread
From: Peter Kjellerstedt @ 2017-08-17 13:01 UTC (permalink / raw)
To: Alexander Kanavin, Mark Hatle, Richard Purdie,
openembedded-core@lists.openembedded.org
> -----Original Message-----
> From: Alexander Kanavin [mailto:alexander.kanavin@linux.intel.com]
> Sent: den 17 augusti 2017 12:53
> To: Mark Hatle <mark.hatle@windriver.com>; Richard Purdie
> <richard.purdie@linuxfoundation.org>; openembedded-
> core@lists.openembedded.org
> Cc: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Subject: Re: [OE-core] [PATCH 0/6 v2] Fix RPM4 regressions based on
> Pyro
>
> On 08/16/2017 06:37 PM, Mark Hatle wrote:
> > Are there any examples? I was looking there and simply didn't see
> > any. I saw the packaging for instance exercised python interfaces,
> > version comparison tools, etc.. but nothing that built a
> > recipe/package and then verified the contents were 'correct'.
> >
> > Ideally the test would be to fabricate something with a known set
> > of file dependencies, produce a package from it and then verify
> > that the package properly included those dependencies.
> >
> > I had looked at just picking some random library out of the deploy
> > directory, and doing a pattern search on it's provides.. but I'm
> > not sure that type of test would be very robust.
>
> How was this regression discovered in the first place? I guess that
> some recipe used to build without error with rpm5, and stopped
> building with rpm4. This is what you could test for: isolate the
> problematic bit, remove everything else, and write a simple recipe(s)
> that can be placed into meta-selftest. Then simply build those from
> the selftest; if they build without error, you don't need to do more
> sophisticated checks.
>
> Alex
The problem we have, which caused me to look into this, is:
We unfortunately have a lot of unversioned libraries, e.g.,
"libfoo.so" instead of "libfoo.so.1.2.3". There is no problem
building these (except we need to work around OE's default of
putting *.so in ${PN}-dev rather than ${PN}). However, when rpm
creates the packages for the applications linked with these
libraries, it fails to automatically determine these runtime
dependencies. However, since there are no errors, what then
happens is that the image is created, lacking most of our
libraries, which of course leads to the image failing to boot
when applications cannot find the libraries they need...
The reason this is not a problem with versioned libraries is
that the code in OE has support for determining these and will
feed that information to rpm. In hindsight, it would probably
have been better if I looked at fixing that code to support
unversioned libraries, but since rpm's automatic discovery of
runtime dependencies worked in Morty, I thought it would be
easiest to get the needed functionality back that was lost in
the transition to dnf by fixing rpm...
//Peter
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-17 13:01 ` Peter Kjellerstedt
@ 2017-08-17 13:17 ` Alexander Kanavin
2017-08-17 14:41 ` Mark Hatle
1 sibling, 0 replies; 19+ messages in thread
From: Alexander Kanavin @ 2017-08-17 13:17 UTC (permalink / raw)
To: Peter Kjellerstedt, Mark Hatle, Richard Purdie,
openembedded-core@lists.openembedded.org
On 08/17/2017 04:01 PM, Peter Kjellerstedt wrote:
> The problem we have, which caused me to look into this, is:
>
> We unfortunately have a lot of unversioned libraries, e.g.,
> "libfoo.so" instead of "libfoo.so.1.2.3". There is no problem
> building these (except we need to work around OE's default of
> putting *.so in ${PN}-dev rather than ${PN}). However, when rpm
> creates the packages for the applications linked with these
> libraries, it fails to automatically determine these runtime
> dependencies. However, since there are no errors, what then
> happens is that the image is created, lacking most of our
> libraries, which of course leads to the image failing to boot
> when applications cannot find the libraries they need...
Thanks. The problem with relying on rpm for this discovery is that this
mechanism is not used anywhere in oe-core. I may update rpm to 4.14 and
again break this stuff without noticing. I'd say it's better if you fix
oe-core's code to discover those in the same way versioned libraries are
discovered.
Alex
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-17 13:01 ` Peter Kjellerstedt
2017-08-17 13:17 ` Alexander Kanavin
@ 2017-08-17 14:41 ` Mark Hatle
1 sibling, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-17 14:41 UTC (permalink / raw)
To: Peter Kjellerstedt, Alexander Kanavin, Richard Purdie,
openembedded-core@lists.openembedded.org
On 8/17/17 8:01 AM, Peter Kjellerstedt wrote:
>> -----Original Message-----
>> From: Alexander Kanavin [mailto:alexander.kanavin@linux.intel.com]
>> Sent: den 17 augusti 2017 12:53
>> To: Mark Hatle <mark.hatle@windriver.com>; Richard Purdie
>> <richard.purdie@linuxfoundation.org>; openembedded-
>> core@lists.openembedded.org
>> Cc: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
>> Subject: Re: [OE-core] [PATCH 0/6 v2] Fix RPM4 regressions based on
>> Pyro
>>
>> On 08/16/2017 06:37 PM, Mark Hatle wrote:
>>> Are there any examples? I was looking there and simply didn't see
>>> any. I saw the packaging for instance exercised python interfaces,
>>> version comparison tools, etc.. but nothing that built a
>>> recipe/package and then verified the contents were 'correct'.
>>>
>>> Ideally the test would be to fabricate something with a known set
>>> of file dependencies, produce a package from it and then verify
>>> that the package properly included those dependencies.
>>>
>>> I had looked at just picking some random library out of the deploy
>>> directory, and doing a pattern search on it's provides.. but I'm
>>> not sure that type of test would be very robust.
>>
>> How was this regression discovered in the first place? I guess that
>> some recipe used to build without error with rpm5, and stopped
>> building with rpm4. This is what you could test for: isolate the
>> problematic bit, remove everything else, and write a simple recipe(s)
>> that can be placed into meta-selftest. Then simply build those from
>> the selftest; if they build without error, you don't need to do more
>> sophisticated checks.
>>
>> Alex
>
> The problem we have, which caused me to look into this, is:
>
> We unfortunately have a lot of unversioned libraries, e.g.,
> "libfoo.so" instead of "libfoo.so.1.2.3". There is no problem
You shouldn't have to version, but you do need to declare an soname. (This can
be done after link as well if I remember correctly.) Once an soname is declared
in your library, OE will know that it's runtime and not -dev.)
> building these (except we need to work around OE's default of
> putting *.so in ${PN}-dev rather than ${PN}). However, when rpm
> creates the packages for the applications linked with these
> libraries, it fails to automatically determine these runtime
> dependencies. However, since there are no errors, what then
> happens is that the image is created, lacking most of our
> libraries, which of course leads to the image failing to boot
> when applications cannot find the libraries they need...
>
> The reason this is not a problem with versioned libraries is
> that the code in OE has support for determining these and will
> feed that information to rpm. In hindsight, it would probably
> have been better if I looked at fixing that code to support
> unversioned libraries, but since rpm's automatic discovery of
> runtime dependencies worked in Morty, I thought it would be
> easiest to get the needed functionality back that was lost in
> the transition to dnf by fixing rpm...
It might be a good idea to have an optional QA warning for unversioned
libraries. Last time I looked (a few years ago) we didn't have any by default.
This made it easy to define the rules that we use the declared soname(s) to
define what goes into the runtime library package and what goes into the -dev
package.
--Mark
> //Peter
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro
2017-08-17 10:53 ` Alexander Kanavin
2017-08-17 13:01 ` Peter Kjellerstedt
@ 2017-08-17 14:37 ` Mark Hatle
1 sibling, 0 replies; 19+ messages in thread
From: Mark Hatle @ 2017-08-17 14:37 UTC (permalink / raw)
To: Alexander Kanavin, Richard Purdie, openembedded-core; +Cc: peter.kjellerstedt
On 8/17/17 5:53 AM, Alexander Kanavin wrote:
> On 08/16/2017 06:37 PM, Mark Hatle wrote:
>> Are there any examples? I was looking there and simply didn't see any. I saw
>> the packaging for instance exercised python interfaces, version comparison
>> tools, etc.. but nothing that built a recipe/package and then verified the
>> contents were 'correct'.
>>
>> Ideally the test would be to fabricate something with a known set of file
>> dependencies, produce a package from it and then verify that the package
>> properly included those dependencies.
>>
>> I had looked at just picking some random library out of the deploy directory,
>> and doing a pattern search on it's provides.. but I'm not sure that type of test
>> would be very robust.
>
> How was this regression discovered in the first place? I guess that some
> recipe used to build without error with rpm5, and stopped building with
> rpm4. This is what you could test for: isolate the problematic bit,
> remove everything else, and write a simple recipe(s) that can be placed
> into meta-selftest. Then simply build those from the selftest; if they
> build without error, you don't need to do more sophisticated checks.
>
In -my- case (I can't speak for Peter), I found the issue trying to build SRPMs
on the target. I either ran into dep problems or could not install what I built.
Further I went in and simply looked at the output (deploy dir): rpm -qp
<package> --provides
I was able to quickly determine that only OE style dependencies were present.
So from a test perspective, we probably need to build something with a known
content and run rpmdeps against it, and then build a package (recipe) and verify
the dependencies are there as well. I just don't know how to implement that in
the existing unit-test framework.
--Mark
> Alex
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* ✗ patchtest: failure for Fix RPM4 regressions based on Pyro (rev2)
2017-08-15 21:41 [PATCH 0/6 v2] Fix RPM4 regressions based on Pyro Mark Hatle
` (7 preceding siblings ...)
2017-08-16 13:34 ` Richard Purdie
@ 2017-08-16 15:32 ` Patchwork
8 siblings, 0 replies; 19+ messages in thread
From: Patchwork @ 2017-08-16 15:32 UTC (permalink / raw)
To: Mark Hatle; +Cc: openembedded-core
== Series Details ==
Series: Fix RPM4 regressions based on Pyro (rev2)
Revision: 2
URL : https://patchwork.openembedded.org/series/8245/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:
* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at 4d65421f38)
If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).
---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
^ permalink raw reply [flat|nested] 19+ messages in thread