* [PATCH v3 2/4] package_deb: Revert versioned providers workaround
2023-10-09 14:39 [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation Pavel Zhukov
@ 2023-10-09 14:39 ` Pavel Zhukov
2023-10-09 14:40 ` [PATCH v3 3/4] dummy-sdk-package.inc: Specify providers version Pavel Zhukov
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Pavel Zhukov @ 2023-10-09 14:39 UTC (permalink / raw)
To: openembedded-core; +Cc: Pavel Zhukov
dpkg and apt seem to handle versioned provides correctly now [1] so this
workaround is not needed anymore.
This fixes [Yocto #14995] for package_deb.
[1] https://www.debian.org/doc/debian-policy/ch-relationships.html#virtual-packages-provides
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
---
meta/classes-global/package_deb.bbclass | 9 +--------
1 file changed, 1 insertion(+), 8 deletions(-)
diff --git a/meta/classes-global/package_deb.bbclass b/meta/classes-global/package_deb.bbclass
index 5aba3837bd..d7bbfae3fb 100644
--- a/meta/classes-global/package_deb.bbclass
+++ b/meta/classes-global/package_deb.bbclass
@@ -184,8 +184,6 @@ def deb_write_pkg(pkg, d):
# '<' = less or equal
# '>' = greater or equal
# adjust these to the '<<' and '>>' equivalents
- # Also, "=" specifiers only work if they have the PR in, so 1.2.3 != 1.2.3-r0
- # so to avoid issues, map this to ">= 1.2.3 << 1.2.3.0"
for dep in list(var.keys()):
if '(' in dep or '/' in dep:
newdep = re.sub(r'[(:)/]', '__', dep)
@@ -200,10 +198,6 @@ def deb_write_pkg(pkg, d):
var[dep][i] = var[dep][i].replace("< ", "<< ")
elif (v or "").startswith("> "):
var[dep][i] = var[dep][i].replace("> ", ">> ")
- elif (v or "").startswith("= ") and "-r" not in v:
- ver = var[dep][i].replace("= ", "")
- var[dep][i] = var[dep][i].replace("= ", ">= ")
- var[dep].append("<< " + ver + ".0")
rdepends = bb.utils.explode_dep_versions2(localdata.getVar("RDEPENDS") or "")
debian_cmp_remap(rdepends)
@@ -220,8 +214,7 @@ def deb_write_pkg(pkg, d):
del rrecommends[dep]
rsuggests = bb.utils.explode_dep_versions2(localdata.getVar("RSUGGESTS") or "")
debian_cmp_remap(rsuggests)
- # Deliberately drop version information here, not wanted/supported by deb
- rprovides = dict.fromkeys(bb.utils.explode_dep_versions2(localdata.getVar("RPROVIDES") or ""), [])
+ rprovides = bb.utils.explode_dep_versions2(localdata.getVar("RPROVIDES") or "")
# Remove file paths if any from rprovides, debian does not support custom providers
for key in list(rprovides.keys()):
if key.startswith('/'):
--
2.41.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v3 3/4] dummy-sdk-package.inc: Specify providers version
2023-10-09 14:39 [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation Pavel Zhukov
2023-10-09 14:39 ` [PATCH v3 2/4] package_deb: Revert versioned providers workaround Pavel Zhukov
@ 2023-10-09 14:40 ` Pavel Zhukov
2023-10-09 14:40 ` [PATCH v3 4/4] selftest: Add tests for populate_sdk task Pavel Zhukov
2023-10-12 7:36 ` [OE-core] [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation Luca Ceresoli
3 siblings, 0 replies; 7+ messages in thread
From: Pavel Zhukov @ 2023-10-09 14:40 UTC (permalink / raw)
To: openembedded-core; +Cc: Pavel Zhukov
Some of the packages require versioned providers
(DEPENDS: perl (>= 5.38) is an example and for such packages
do_populate_sdk fails because dummy packages provided unversioned
packages (PROVIDES: perl) which doesn't meet the version requirement.
Specify 999.9-r9 version for such provides to work this around
Fixes [Yocto #14995]
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
---
meta/recipes-core/meta/dummy-sdk-package.inc | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-core/meta/dummy-sdk-package.inc b/meta/recipes-core/meta/dummy-sdk-package.inc
index 99863b86f1..b1571a2cc5 100644
--- a/meta/recipes-core/meta/dummy-sdk-package.inc
+++ b/meta/recipes-core/meta/dummy-sdk-package.inc
@@ -18,14 +18,17 @@ ALLOW_EMPTY:${PN} = "1"
PR[vardeps] += "DUMMYPROVIDES"
PR[vardeps] += "DUMMYPROVIDES_PACKAGES"
+DUMMYMAXVER ??= "999.9-r9"
DUMMYPROVIDES_PACKAGES ??= ""
DUMMYPROVIDES_PACKAGES_FILTERED = "${@' '.join([pkg for pkg in d.getVar('DUMMYPROVIDES_PACKAGES').split() if d.getVar('IMAGE_INSTALL') is None or pkg not in d.getVar('IMAGE_INSTALL').split()])}"
DUMMYPROVIDES_PACKAGES_MULTILIB = "${@' '.join([multilib_pkg_extend(d, pkg) for pkg in d.getVar('DUMMYPROVIDES_PACKAGES_FILTERED').split()])}"
+DUMMYPROVIDES_MAXVER = "${@' '.join(['{} (= {})'.format(pkg, d.getVar('DUMMYMAXVER')) for pkg in d.getVar('DUMMYPROVIDES_PACKAGES_MULTILIB').split()]) + d.getVar('DUMMYPROVIDES')}"
+
DUMMYPROVIDES += "${DUMMYPROVIDES_PACKAGES_MULTILIB}"
python populate_packages:prepend() {
p = d.getVar("PN")
- d.appendVar("RPROVIDES:%s" % p, "${DUMMYPROVIDES}")
+ d.appendVar("RPROVIDES:%s" % p, "${DUMMYPROVIDES_MAXVER}")
d.appendVar("RCONFLICTS:%s" % p, "${DUMMYPROVIDES_PACKAGES_MULTILIB}")
d.appendVar("RREPLACES:%s" % p, "${DUMMYPROVIDES_PACKAGES_MULTILIB}")
}
--
2.41.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v3 4/4] selftest: Add tests for populate_sdk task
2023-10-09 14:39 [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation Pavel Zhukov
2023-10-09 14:39 ` [PATCH v3 2/4] package_deb: Revert versioned providers workaround Pavel Zhukov
2023-10-09 14:40 ` [PATCH v3 3/4] dummy-sdk-package.inc: Specify providers version Pavel Zhukov
@ 2023-10-09 14:40 ` Pavel Zhukov
2023-10-12 7:36 ` [OE-core] [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation Luca Ceresoli
3 siblings, 0 replies; 7+ messages in thread
From: Pavel Zhukov @ 2023-10-09 14:40 UTC (permalink / raw)
To: openembedded-core; +Cc: Pavel Zhukov
Two tests to cover both installing package with IMAGE_INSTALL as
well as installing versioned dependencies of the package (using perl (>=
5.XX).
Related: [Yocto #13338] [Yocto #14995] [Yocto #14066]
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
---
.../testsdk-perldepends.bb | 16 ++++
meta/lib/oeqa/selftest/cases/sdk.py | 79 +++++++++++++++++++
2 files changed, 95 insertions(+)
create mode 100644 meta-selftest/recipes-test/testsdk-perldepends/testsdk-perldepends.bb
create mode 100644 meta/lib/oeqa/selftest/cases/sdk.py
diff --git a/meta-selftest/recipes-test/testsdk-perldepends/testsdk-perldepends.bb b/meta-selftest/recipes-test/testsdk-perldepends/testsdk-perldepends.bb
new file mode 100644
index 0000000000..428e1f52e2
--- /dev/null
+++ b/meta-selftest/recipes-test/testsdk-perldepends/testsdk-perldepends.bb
@@ -0,0 +1,16 @@
+SUMMARY = "Test recipe for sdk populate test case"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
+
+INHIBIT_DEFAULT_DEPS = "1"
+
+RDEPENDS:${PN}="perl (>= 5.12)"
+
+do_install(){
+ mkdir -p ${D}/var/lib/
+ touch ${D}/var/lib/testname
+ echo "${PN}" > ${D}/var/lib/testname
+}
+
+
+
diff --git a/meta/lib/oeqa/selftest/cases/sdk.py b/meta/lib/oeqa/selftest/cases/sdk.py
new file mode 100644
index 0000000000..35734d8a7a
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/sdk.py
@@ -0,0 +1,79 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+import os
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import get_bb_var, bitbake
+
+class PopulateSDK(OESelftestTestCase):
+ '''Tests for populate SDK task(s)'''
+
+ @classmethod
+ def setUpClass(self):
+
+ super(PopulateSDK, self).setUpClass()
+ self.buildtarget = 'core-image-minimal'
+ self.classname = 'PopulateSDK'
+
+ def sdk_build(self):
+ """
+ Test if package added via IMAGE_INSTALL
+ conflicts with dummy SDK providers
+ """
+ SDKTASK = '-c populate_sdk'
+ bbargs = "{} {}".format(SDKTASK, self.buildtarget)
+ self.logger.info("{}: doing bitbake {} ".format(self.classname, bbargs))
+ return bitbake(bbargs).status
+
+ def set_config(self, image_install, package_classes):
+ config = 'IMAGE_INSTALL:append = " {}"\n'.format(image_install)
+ config += 'PACKAGE_CLASSES = " package_{}"\n'.format(package_classes)
+ return config
+
+ def do_test_build(self, image_install, package_manager):
+ self.write_config(self.set_config(image_install, package_manager))
+ res = self.sdk_build()
+ self.assertEqual(0, res, "Failed to populate SDK with {} in IMAGE_INSTALL and {} package manager"\
+ .format(image_install, package_manager))
+
+ def test_image_install_ipk(self):
+ """
+ Regression test for [Yocto #13338]
+ """
+ self.do_test_build("bash", "ipk")
+
+ def test_image_install_rpm(self):
+ """
+ Regression test for [Yocto #13338]
+ """
+ self.do_test_build("bash", "rpm")
+
+ def test_image_install_deb(self):
+ """
+ Regression test for [Yocto #13338]
+ """
+ self.do_test_build("bash", "deb")
+
+
+ def test_image_install_confictdeps_ipk(self):
+ """
+ Regression test for [Yocto #14995]
+ """
+ self.skipTest("Disabled until [Yocto #14995] is fixed")
+ self.do_test_build("testsdk-perldepends", "ipk")
+
+ def test_image_install_confictdeps_rpm(self):
+ """
+ Regression test for [Yocto #14995]
+ """
+ self.do_test_build("testsdk-perldepends", "rpm")
+
+ def test_image_install_confictdeps_deb(self):
+ """
+ Regression test for [Yocto #14995]
+ """
+ self.do_test_build("testsdk-perldepends", "deb")
--
2.41.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [OE-core] [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation
2023-10-09 14:39 [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation Pavel Zhukov
` (2 preceding siblings ...)
2023-10-09 14:40 ` [PATCH v3 4/4] selftest: Add tests for populate_sdk task Pavel Zhukov
@ 2023-10-12 7:36 ` Luca Ceresoli
2023-12-18 10:28 ` pazhukov
3 siblings, 1 reply; 7+ messages in thread
From: Luca Ceresoli @ 2023-10-12 7:36 UTC (permalink / raw)
To: Pavel Zhukov; +Cc: openembedded-core
Hello Pavel,
On Mon, 9 Oct 2023 16:39:58 +0200
"Pavel Zhukov" <pavel@zhukoff.net> wrote:
> if packages is provided by dummysdk and in the same time marked for
> installation with IMAGE_INSTALL it causes conflict in apt because virtual providers are
> not taken into account if package is asked to be installed explicitly.
> Filter such packages from provides/conflicts to workaround this problem.
> This workaround brakes RPM usecase because of file conlicts with
> DUMMYPROVIDES, use DUMMYPROVIDES_PACKAGES_MULTILIB instead (which
> doesn't include file based conflicts).
> While this is needed for the case of package_deb only adding it for all
> package managers to not complicate the code.
>
> Fixes: [Yocto #13338] [Yocto #14066]
>
> Fixes:
> The following information may help to resolve the situation:
>
> The following packages have unmet dependencies:
> target-sdk-provides-dummy : Conflicts: bash
> E: Unable to correct problems, you have held broken packages.
>
> Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Testing with this series applied results in lots of errors like these:
stdio: ERROR: core-image-minimal-1.0-r0 do_rootfs: No SPDX file found for package base-files, False sstate:base-files:qemux86_64-poky-linux:3.0.14:r0:qemux86_64:11: sstate:base-files::3.0.14:r0::11:
stdio: ERROR: core-image-minimal-mtdutils-1.0-r0 do_rootfs: No SPDX file found for package base-files, False sstate:base-files:qemux86_64-poky-linux:3.0.14:r0:qemux86_64:11: sstate:base-files::3.0.14:r0::11:
See the full log at:
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5894/steps/14/logs/stdio
Luca
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [OE-core] [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation
2023-10-12 7:36 ` [OE-core] [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation Luca Ceresoli
@ 2023-12-18 10:28 ` pazhukov
2024-01-01 12:57 ` Alexandre Belloni
0 siblings, 1 reply; 7+ messages in thread
From: pazhukov @ 2023-12-18 10:28 UTC (permalink / raw)
To: alexandre.belloni
Cc: Pavel Zhukov, openembedded-core, luca.ceresoli, richard.purdie
Hi Alex, Luca,
Can you please retest this patchset one more time please?
Neither myself nor RP could reproduce build failure with recent
master/sstate so it might be fixed in the meantime.
--
Pavel
On 2023-10-12 09:36, Luca Ceresoli via lists.openembedded.org wrote:
> Hello Pavel,
>
> On Mon, 9 Oct 2023 16:39:58 +0200
> "Pavel Zhukov" <pavel@zhukoff.net> wrote:
>
>> if packages is provided by dummysdk and in the same time marked for
>> installation with IMAGE_INSTALL it causes conflict in apt because
>> virtual providers are
>> not taken into account if package is asked to be installed explicitly.
>> Filter such packages from provides/conflicts to workaround this
>> problem.
>> This workaround brakes RPM usecase because of file conlicts with
>> DUMMYPROVIDES, use DUMMYPROVIDES_PACKAGES_MULTILIB instead (which
>> doesn't include file based conflicts).
>> While this is needed for the case of package_deb only adding it for
>> all
>> package managers to not complicate the code.
>>
>> Fixes: [Yocto #13338] [Yocto #14066]
>>
>> Fixes:
>> The following information may help to resolve the situation:
>>
>> The following packages have unmet dependencies:
>> target-sdk-provides-dummy : Conflicts: bash
>> E: Unable to correct problems, you have held broken packages.
>>
>> Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
>
> Testing with this series applied results in lots of errors like these:
>
> stdio: ERROR: core-image-minimal-1.0-r0 do_rootfs: No SPDX file found
> for package base-files, False
> sstate:base-files:qemux86_64-poky-linux:3.0.14:r0:qemux86_64:11:
> sstate:base-files::3.0.14:r0::11:
> stdio: ERROR: core-image-minimal-mtdutils-1.0-r0 do_rootfs: No SPDX
> file found for package base-files, False
> sstate:base-files:qemux86_64-poky-linux:3.0.14:r0:qemux86_64:11:
> sstate:base-files::3.0.14:r0::11:
>
> See the full log at:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5894/steps/14/logs/stdio
>
> Luca
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#188989):
> https://lists.openembedded.org/g/openembedded-core/message/188989
> Mute This Topic: https://lists.openembedded.org/mt/101853406/7591736
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
> [pazhukov@suse.de]
> -=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] [PATCH v3 1/4] dummy-sdk-package.inc: Filter packages which are marked for installation
2023-12-18 10:28 ` pazhukov
@ 2024-01-01 12:57 ` Alexandre Belloni
0 siblings, 0 replies; 7+ messages in thread
From: Alexandre Belloni @ 2024-01-01 12:57 UTC (permalink / raw)
To: Pavel Zhukov
Cc: Pavel Zhukov, openembedded-core, luca.ceresoli, richard.purdie
Hello,
This still causes issues:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6252/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6203/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6256/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6263/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/2682/steps/15/logs/stdio
2023-12-31 22:35:38,306 - oe-selftest - INFO - 5: 63/78 555/579 (65.04s) (0 failed) (spdx.SPDXCheck.test_spdx_base_files)
2023-12-31 22:35:38,306 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/spdx.py", line 54, in test_spdx_base_files
self.check_recipe_spdx("packages", "base-files.spdx.json", "base-files")
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/spdx.py", line 50, in check_recipe_spdx
self.assertExists(full_file_path)
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/case.py", line 251, in assertExists
raise self.failureException(msg)
AssertionError: '/home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-3943718/tmp/deploy/spdx/qemux86_64/packages/base-files.spdx.json' does not exist
On 18/12/2023 11:28:04+0100, Pavel Zhukov wrote:
> Hi Alex, Luca,
>
> Can you please retest this patchset one more time please?
> Neither myself nor RP could reproduce build failure with recent
> master/sstate so it might be fixed in the meantime.
>
> --
> Pavel
>
> On 2023-10-12 09:36, Luca Ceresoli via lists.openembedded.org wrote:
> > Hello Pavel,
> >
> > On Mon, 9 Oct 2023 16:39:58 +0200
> > "Pavel Zhukov" <pavel@zhukoff.net> wrote:
> >
> > > if packages is provided by dummysdk and in the same time marked for
> > > installation with IMAGE_INSTALL it causes conflict in apt because
> > > virtual providers are
> > > not taken into account if package is asked to be installed explicitly.
> > > Filter such packages from provides/conflicts to workaround this
> > > problem.
> > > This workaround brakes RPM usecase because of file conlicts with
> > > DUMMYPROVIDES, use DUMMYPROVIDES_PACKAGES_MULTILIB instead (which
> > > doesn't include file based conflicts).
> > > While this is needed for the case of package_deb only adding it for
> > > all
> > > package managers to not complicate the code.
> > >
> > > Fixes: [Yocto #13338] [Yocto #14066]
> > >
> > > Fixes:
> > > The following information may help to resolve the situation:
> > >
> > > The following packages have unmet dependencies:
> > > target-sdk-provides-dummy : Conflicts: bash
> > > E: Unable to correct problems, you have held broken packages.
> > >
> > > Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
> >
> > Testing with this series applied results in lots of errors like these:
> >
> > stdio: ERROR: core-image-minimal-1.0-r0 do_rootfs: No SPDX file found
> > for package base-files, False
> > sstate:base-files:qemux86_64-poky-linux:3.0.14:r0:qemux86_64:11:
> > sstate:base-files::3.0.14:r0::11:
> > stdio: ERROR: core-image-minimal-mtdutils-1.0-r0 do_rootfs: No SPDX
> > file found for package base-files, False
> > sstate:base-files:qemux86_64-poky-linux:3.0.14:r0:qemux86_64:11:
> > sstate:base-files::3.0.14:r0::11:
> >
> > See the full log at:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5894/steps/14/logs/stdio
> >
> > Luca
> >
> >
> >
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#192628): https://lists.openembedded.org/g/openembedded-core/message/192628
> Mute This Topic: https://lists.openembedded.org/mt/101853406/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 7+ messages in thread