* [PATCH 0/1] python-smartpm-native: prefer same arch when install @ 2015-10-27 14:05 Robert Yang 2015-10-27 14:05 ` [PATCH 1/1] " Robert Yang 0 siblings, 1 reply; 5+ messages in thread From: Robert Yang @ 2015-10-27 14:05 UTC (permalink / raw) To: openembedded-core The following changes since commit f787b688f2884ce3fa888b4041030538c7d2bf55: oeqa/utils/decorators: fix missing keyword arguments on decorators (2015-10-27 07:22:22 +0000) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib rbt/smart http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/smart Robert Yang (1): python-smartpm-native: prefer same arch when install .../python-smartpm/smart-prefer-same-arch.patch | 119 ++++++++++++++++++++ meta/recipes-devtools/python/python-smartpm_git.bb | 2 + 2 files changed, 121 insertions(+) create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch -- 1.7.9.5 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/1] python-smartpm-native: prefer same arch when install 2015-10-27 14:05 [PATCH 0/1] python-smartpm-native: prefer same arch when install Robert Yang @ 2015-10-27 14:05 ` Robert Yang 2015-10-27 18:45 ` Burton, Ross 0 siblings, 1 reply; 5+ messages in thread From: Robert Yang @ 2015-10-27 14:05 UTC (permalink / raw) To: openembedded-core We had made smart install multilib RDEPENDS correctly from package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix the issue from python-smartpm-native, and make it work well. The logic is: when pkg_A rdepends/rrecommands pkg_B, then let pkg_B use pkg_A's arch when possible. This patch fixed: require conf/multilib.conf MULTILIBS = "multilib:lib32" DEFAULTTUNE_virtclass-multilib-lib32 = "x86" RPM_PREFER_ELF_ARCH = "1" IMAGE_INSTALL_append = " lib32-connman-gnome" MACHINE = "qemux86-64" $ bitbake core-image-sato Only 64bit loaders like libgdk-pixbuf-2.0-loader-jpeg and libgdk-pixbuf-2.0-loader-png were installed before this patch since they are in RRECOMMANDS, now both 32bit and 64bit are installed. This patch is for native only to minimize the impact for this release, the main problem is the "if attempt:", otherwise, more packages like lib32-libc6-dev will be installed when populate_sdk, but only libc6-dev were installed before. We need reconstruct the way that we install the complentary packages in future release, the "if attempt:" make they work together atm. [YOCTO #8570] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> --- .../python-smartpm/smart-prefer-same-arch.patch | 119 ++++++++++++++++++++ meta/recipes-devtools/python/python-smartpm_git.bb | 2 + 2 files changed, 121 insertions(+) create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch diff --git a/meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch b/meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch new file mode 100644 index 0000000..b1305b5 --- /dev/null +++ b/meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch @@ -0,0 +1,119 @@ +From 1749592a76425449c1f3119fb14db3f5e14226ed Mon Sep 17 00:00:00 2001 +From: Robert Yang <liezhi.yang@windriver.com> +Date: Tue, 27 Oct 2015 06:20:45 -0700 +Subject: [PATCH] smart/transaction.py: prefer same arch when install + +We had made smart install multilib RDEPENDS correctly from +package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix +the issue from python-smartpm-native, and make it work well. + +The logic is: when pkg_A rdepends/rrecommands pkg_B, then let pkg_B use +pkg_A's arch when possible. + +This patch fixed: +require conf/multilib.conf +MULTILIBS = "multilib:lib32" +DEFAULTTUNE_virtclass-multilib-lib32 = "x86" +RPM_PREFER_ELF_ARCH = "1" +IMAGE_INSTALL_append = " lib32-connman-gnome" +MACHINE = "qemux86-64" + +$ bitbake core-image-sato + +Only 64bit loaders like libgdk-pixbuf-2.0-loader-jpeg and +libgdk-pixbuf-2.0-loader-png were installed before this patch since they +are in RRECOMMANDS, now both 32bit and 64bit are installed. + +This patch is for native only to minimize the impact for this release, +the main problem is the "if attempt:", otherwise, more packages like +lib32-libc6-dev will be installed when populate_sdk, but only libc6-dev +were installed before. We need reconstruct the way that we install the +complentary packages in future release, the "if attempt:" make they work +together atm. + +Upstream-Status: Inappropriate [oe specific] + +Signed-off-by: Robert Yang <liezhi.yang@windriver.com> +--- + smart/transaction.py | 28 +++++++++++++++++++++++++--- + 1 file changed, 25 insertions(+), 3 deletions(-) + +diff --git a/smart/transaction.py b/smart/transaction.py +index f022aae..63a837c 100644 +--- a/smart/transaction.py ++++ b/smart/transaction.py +@@ -22,6 +22,7 @@ + from smart.const import INSTALL, REMOVE, UPGRADE, FIX, REINSTALL, KEEP, LOCKED_EXCLUDE, LOCKED_INSTALL, LOCKED_CONFLICT, LOCKED_CONFLICT_BY, LOCKED_NO_COEXIST, LOCKED_SYSCONF, LOCKED_REMOVE + from smart.cache import PreRequires, Package + from smart import * ++from smart.backends.rpm.rpmver import splitarch + + def lock_reason(pkg, lockvalue): + try: +@@ -618,6 +619,7 @@ class Transaction(object): + self._remove(namepkg, changeset, locked, pending, depth) + + # Install packages required by this one. ++ pkgarch = splitarch(pkg.version)[1] + for req in pkg.requires + pkg.recommends: + + reqrequired = req in pkg.requires +@@ -626,6 +628,8 @@ class Transaction(object): + prvpkgs = {} + lockedpkgs = {} + found = False ++ found_same_arch = False ++ same_arch_in_provides = False + for prv in req.providedby: + for prvpkg in prv.packages: + if not reqrequired: +@@ -633,9 +637,19 @@ class Transaction(object): + continue + elif pkgconf.testFlag("ignore-recommends", prvpkg): + continue ++ prvpkgarch = splitarch(prvpkg.version)[1] ++ if pkgarch == prvpkgarch: ++ # Found a same arch in provides ++ same_arch_in_provides = True + if isinst(prvpkg): + found = True +- break ++ if attempt: ++ break ++ if pkgarch == prvpkgarch: ++ # Found the best match ++ found_same_arch = True ++ break ++ continue + if prvpkg not in locked: + prvpkgs[prvpkg] = True + else: +@@ -643,7 +657,7 @@ class Transaction(object): + else: + continue + break +- if found: ++ if found_same_arch or (found and attempt) or (found and not same_arch_in_provides): + # Someone is already providing it. Good. + continue + +@@ -672,7 +686,15 @@ class Transaction(object): + + # It's only a recommend, skip + continue +- ++ if len(prvpkgs) > 1 and not attempt: ++ prvarchs = set() ++ for prv in prvpkgs: ++ prvarchs.add(splitarch(prv.version)[1]) ++ if pkgarch in prvarchs: ++ # Prefer the one which has the same arch ++ for prv in prvpkgs.copy(): ++ if splitarch(prv.version)[1] != pkgarch: ++ del prvpkgs[prv] + if len(prvpkgs) == 1: + # Don't check locked here. prvpkgs was + # already filtered above. +-- +1.7.9.5 + diff --git a/meta/recipes-devtools/python/python-smartpm_git.bb b/meta/recipes-devtools/python/python-smartpm_git.bb index d6c378b..1ebf590 100644 --- a/meta/recipes-devtools/python/python-smartpm_git.bb +++ b/meta/recipes-devtools/python/python-smartpm_git.bb @@ -26,6 +26,8 @@ SRC_URI = "\ file://smart-cache.py-getPackages-matches-name-version.patch \ " +SRC_URI_append_class-native = "file://smart-prefer-same-arch.patch" + SRCREV = "407a7eca766431257dcd1da15175cc36a1bb22d0" PV = "1.5+git${SRCPV}" -- 1.7.9.5 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 1/1] python-smartpm-native: prefer same arch when install 2015-10-27 14:05 ` [PATCH 1/1] " Robert Yang @ 2015-10-27 18:45 ` Burton, Ross 2015-10-27 19:04 ` Mark Hatle 2015-10-28 2:11 ` Robert Yang 0 siblings, 2 replies; 5+ messages in thread From: Burton, Ross @ 2015-10-27 18:45 UTC (permalink / raw) To: Robert Yang; +Cc: OE-core [-- Attachment #1: Type: text/plain, Size: 1418 bytes --] On 27 October 2015 at 14:05, Robert Yang <liezhi.yang@windriver.com> wrote: > We had made smart install multilib RDEPENDS correctly from > package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix > the issue from python-smartpm-native, and make it work well. > > The logic is: when pkg_A rdepends/rrecommands pkg_B, then let pkg_B use > pkg_A's arch when possible. > I added this patch and rebuilt core-image-sato, then ran buildhistory-diff: Changes to images/intel_corei7_64/glibc/core-image-sato (installed-package-names.txt): lib32-ncurses-terminfo-base was added lib32-libjpeg9 was added lib32-glibc-gconv-iso8859-1 was added lib32-shared-mime-info was added lib32-libxml2 was added lib32-wpa-supplicant-cli was added lib32-libgdk-pixbuf-2.0-loader-png was added lib32-wpa-supplicant-passphrase was added lib32-pango-module-basic-fc was added lib32-bash was added lib32-libgdk-pixbuf-2.0-loader-jpeg was added lib32-libgdk-pixbuf-2.0-loader-gif was added lib32-libgdk-pixbuf-2.0-loader-xpm was added lib32-glibc-gconv was added Some of those are good (gdk-pixbuf and pango), some bad (bash, wpa-supplicant-cli), some probably bugs in packages that should be allarch (termino, shared-mime-info). I'm undecided what we do about this. Fixing the dependency chains is good, but we're definitely pulling in more than expected. Ross [-- Attachment #2: Type: text/html, Size: 2567 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/1] python-smartpm-native: prefer same arch when install 2015-10-27 18:45 ` Burton, Ross @ 2015-10-27 19:04 ` Mark Hatle 2015-10-28 2:11 ` Robert Yang 1 sibling, 0 replies; 5+ messages in thread From: Mark Hatle @ 2015-10-27 19:04 UTC (permalink / raw) To: Burton, Ross, Robert Yang; +Cc: OE-core On 10/27/15 1:45 PM, Burton, Ross wrote: > > On 27 October 2015 at 14:05, Robert Yang <liezhi.yang@windriver.com > <mailto:liezhi.yang@windriver.com>> wrote: > > We had made smart install multilib RDEPENDS correctly from > package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix > the issue from python-smartpm-native, and make it work well. > > The logic is: when pkg_A rdepends/rrecommands pkg_B, then let pkg_B use > pkg_A's arch when possible. > > > I added this patch and rebuilt core-image-sato, then ran buildhistory-diff: > > Changes to images/intel_corei7_64/glibc/core-image-sato > (installed-package-names.txt): > lib32-ncurses-terminfo-base was added > lib32-libjpeg9 was added > lib32-glibc-gconv-iso8859-1 was added > lib32-shared-mime-info was added > lib32-libxml2 was added > lib32-wpa-supplicant-cli was added > lib32-libgdk-pixbuf-2.0-loader-png was added > lib32-wpa-supplicant-passphrase was added > lib32-pango-module-basic-fc was added > lib32-bash was added > lib32-libgdk-pixbuf-2.0-loader-jpeg was added > lib32-libgdk-pixbuf-2.0-loader-gif was added > lib32-libgdk-pixbuf-2.0-loader-xpm was added > lib32-glibc-gconv was added > > Some of those are good (gdk-pixbuf and pango), some bad (bash, > wpa-supplicant-cli), some probably bugs in packages that should be allarch > (termino, shared-mime-info). bash, wpa-supplicant-cli is a good example of why this is an issue. RPM luckily says, well we've got two versions of bash, lets resolve this by following the ELF resolution rule. So while two packages may exist in the manifest, you won't get any conflicts and space is likely not duplicated. But you can't automatically say "give me any bash, but give me arch specific versions of everything else". The default in RPM is if I ask for 'bash', give me any version of bash.. if I ask for 'bash.i586' I was -the- i586 version. This is why the fix for this is either adjust the behavior (as Robert has done) to suck in more dependencies and consider the ELF resolver to be adequate -- or to add some kind of a hint to the recipe mechanisms that select dependencies must be of the same arch type. One we handle in the resolver, and one we could handle when we generate the packages themselves. I'm not sure there is an obvious solution, I prefer the later with a variant of the former myself. (Variant being if all things are equal we weight dependencies to match the arch.. but if something is already in with that provide we use it..) Then the hint would cause the produced packages to be able to say they need explicit versions of the plugins. > I'm undecided what we do about this. Fixing the dependency chains is good, but > we're definitely pulling in more than expected. I agree, I have a feeling what I wrote about is the right answer... but I don't have any code to prove it.. and I'm not sure that we should be making that big of a change right now. Robert's patch SHOULD resolve the problem at the expense of a slightly larger resolution manifest, which may be acceptable. --Mark > Ross > > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/1] python-smartpm-native: prefer same arch when install 2015-10-27 18:45 ` Burton, Ross 2015-10-27 19:04 ` Mark Hatle @ 2015-10-28 2:11 ` Robert Yang 1 sibling, 0 replies; 5+ messages in thread From: Robert Yang @ 2015-10-28 2:11 UTC (permalink / raw) To: Burton, Ross; +Cc: OE-core On 10/28/2015 02:45 AM, Burton, Ross wrote: > > On 27 October 2015 at 14:05, Robert Yang <liezhi.yang@windriver.com > <mailto:liezhi.yang@windriver.com>> wrote: > > We had made smart install multilib RDEPENDS correctly from > package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix > the issue from python-smartpm-native, and make it work well. > > The logic is: when pkg_A rdepends/rrecommands pkg_B, then let pkg_B use > pkg_A's arch when possible. > > > I added this patch and rebuilt core-image-sato, then ran buildhistory-diff: > > Changes to images/intel_corei7_64/glibc/core-image-sato > (installed-package-names.txt): > lib32-ncurses-terminfo-base was added > lib32-libjpeg9 was added > lib32-glibc-gconv-iso8859-1 was added > lib32-shared-mime-info was added > lib32-libxml2 was added > lib32-wpa-supplicant-cli was added > lib32-libgdk-pixbuf-2.0-loader-png was added > lib32-wpa-supplicant-passphrase was added > lib32-pango-module-basic-fc was added > lib32-bash was added > lib32-libgdk-pixbuf-2.0-loader-jpeg was added > lib32-libgdk-pixbuf-2.0-loader-gif was added > lib32-libgdk-pixbuf-2.0-loader-xpm was added > lib32-glibc-gconv was added > > Some of those are good (gdk-pixbuf and pango), some bad (bash, > wpa-supplicant-cli), some probably bugs in packages that should be allarch > (termino, shared-mime-info). I think that lacking something so that it doesn't work is worse than have more things but works ? // Robert > > I'm undecided what we do about this. Fixing the dependency chains is good, but > we're definitely pulling in more than expected. > > Ross ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-10-28 2:11 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-10-27 14:05 [PATCH 0/1] python-smartpm-native: prefer same arch when install Robert Yang 2015-10-27 14:05 ` [PATCH 1/1] " Robert Yang 2015-10-27 18:45 ` Burton, Ross 2015-10-27 19:04 ` Mark Hatle 2015-10-28 2:11 ` Robert Yang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox