* [PATCH 0/3] Fix issue with unpopulated buildhistory depends files
@ 2014-03-05 12:39 Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 1/3] package_manager.py: make list_installed() list pkg dependencies too Laurentiu Palcu
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-05 12:39 UTC (permalink / raw)
To: openembedded-core
The following changes since commit f03955041d0e44d377ca1c4def630982f24f1e8b:
Revert "ncurses: use ln -r to generate relative symlink" (2014-03-03 15:55:27 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib lpalcu/b5904_buildhistory_depends_empty
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5904_buildhistory_depends_empty
Laurentiu Palcu (3):
package_manager.py: make list_installed() list pkg dependencies too
buildhistory.bbclass: Fix dependency files creation
populate_sdk_*.bbclass: remove old rootfs_list_installed_depends()
meta/classes/buildhistory.bbclass | 9 ++++++++-
meta/classes/populate_sdk_deb.bbclass | 5 -----
meta/classes/populate_sdk_ipk.bbclass | 4 ----
meta/classes/populate_sdk_rpm.bbclass | 4 ----
meta/lib/oe/package_manager.py | 31 +++++++++++++++++++++++++++++++
5 files changed, 39 insertions(+), 14 deletions(-)
--
1.7.9.5
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 1/3] package_manager.py: make list_installed() list pkg dependencies too
2014-03-05 12:39 [PATCH 0/3] Fix issue with unpopulated buildhistory depends files Laurentiu Palcu
@ 2014-03-05 12:39 ` Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 3/3] populate_sdk_*.bbclass: remove old rootfs_list_installed_depends() Laurentiu Palcu
2 siblings, 0 replies; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-05 12:39 UTC (permalink / raw)
To: openembedded-core
list_installed("deps") will now return the package dependencies.
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
meta/lib/oe/package_manager.py | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index ff4f1de..f2c88bb 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -765,7 +765,22 @@ class RpmPM(PackageManager):
self.image_rpmlib,
symlinks=True)
+ def _list_pkg_deps(self):
+ cmd = [bb.utils.which(os.getenv('PATH'), "rpmresolve"),
+ "-t", self.image_rpmlib]
+
+ try:
+ output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip()
+ except subprocess.CalledProcessError as e:
+ bb.fatal("Cannot get the package dependencies. Command '%s' "
+ "returned %d:\n%s" % (' '.join(cmd), e.returncode, e.output))
+
+ return output
+
def list_installed(self, format=None):
+ if format == "deps":
+ return self._list_pkg_deps()
+
cmd = self.rpm_cmd + ' --root ' + self.target_rootfs
cmd += ' -D "_dbpath /var/lib/rpm" -qa'
cmd += " --qf '[%{NAME} %{ARCH} %{VERSION} %{PACKAGEORIGIN}\n]'"
@@ -1131,6 +1146,9 @@ class OpkgPM(PackageManager):
elif format == "ver":
cmd = "%s %s status | %s -v" % \
(self.opkg_cmd, self.opkg_args, opkg_query_cmd)
+ elif format == "deps":
+ cmd = "%s %s status | %s" % \
+ (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
else:
cmd = "%s %s list_installed | cut -d' ' -f1" % \
(self.opkg_cmd, self.opkg_args)
@@ -1494,6 +1512,8 @@ class DpkgPM(PackageManager):
cmd.append("-f=${Package} ${Package}_${Version}_${Architecture}.deb ${PackageArch}\n")
elif format == "ver":
cmd.append("-f=${Package} ${PackageArch} ${Version}\n")
+ elif format == "deps":
+ cmd.append("-f=Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n")
else:
cmd.append("-f=${Package}\n")
@@ -1514,6 +1534,17 @@ class DpkgPM(PackageManager):
tmp_output += "%s %s %s\n" % (pkg, pkg_file, pkg_arch)
output = tmp_output
+ elif format == "deps":
+ opkg_query_cmd = bb.utils.which(os.getenv('PATH'), "opkg-query-helper.py")
+
+ try:
+ output = subprocess.check_output("echo -e '%s' | %s" %
+ (output, opkg_query_cmd),
+ stderr=subprocess.STDOUT,
+ shell=True)
+ except subprocess.CalledProcessError as e:
+ bb.fatal("Cannot compute packages dependencies. Command '%s' "
+ "returned %d:\n%s" % (e.cmd, e.returncode, e.output))
return output
--
1.7.9.5
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-05 12:39 [PATCH 0/3] Fix issue with unpopulated buildhistory depends files Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 1/3] package_manager.py: make list_installed() list pkg dependencies too Laurentiu Palcu
@ 2014-03-05 12:39 ` Laurentiu Palcu
2014-03-06 21:55 ` Richard Purdie
2014-03-05 12:39 ` [PATCH 3/3] populate_sdk_*.bbclass: remove old rootfs_list_installed_depends() Laurentiu Palcu
2 siblings, 1 reply; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-05 12:39 UTC (permalink / raw)
To: openembedded-core
Call the new python routines.
[YOCTO #5904]
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
meta/classes/buildhistory.bbclass | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index ef4135b..01b0082 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -319,6 +319,12 @@ python buildhistory_list_installed() {
with open(pkgs_list_file, 'w') as pkgs_list:
pkgs_list.write(list_installed_packages(d, 'file'))
+
+ pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
+ "bh_installed_pkgs_deps.txt")
+
+ with open(pkgs_deps_file, 'w') as pkgs_deps:
+ pkgs_deps.write(list_installed_packages(d, 'deps'))
}
@@ -338,7 +344,8 @@ buildhistory_get_installed() {
# Produce dependency graph
# First, quote each name to handle characters that cause issues for dot
- rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
+ cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
+ rm ${WORKDIR}/bh_installed_pkgs_deps.txt
# Change delimiter from pipe to -> and set style for recommend lines
sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
# Add header, sorted and de-duped contents and footer and then delete the temp file
--
1.7.9.5
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 3/3] populate_sdk_*.bbclass: remove old rootfs_list_installed_depends()
2014-03-05 12:39 [PATCH 0/3] Fix issue with unpopulated buildhistory depends files Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 1/3] package_manager.py: make list_installed() list pkg dependencies too Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation Laurentiu Palcu
@ 2014-03-05 12:39 ` Laurentiu Palcu
2 siblings, 0 replies; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-05 12:39 UTC (permalink / raw)
To: openembedded-core
We're using the python routines now.
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
meta/classes/populate_sdk_deb.bbclass | 5 -----
meta/classes/populate_sdk_ipk.bbclass | 4 ----
meta/classes/populate_sdk_rpm.bbclass | 4 ----
3 files changed, 13 deletions(-)
diff --git a/meta/classes/populate_sdk_deb.bbclass b/meta/classes/populate_sdk_deb.bbclass
index 462525f..833feca 100644
--- a/meta/classes/populate_sdk_deb.bbclass
+++ b/meta/classes/populate_sdk_deb.bbclass
@@ -10,8 +10,3 @@ do_populate_sdk[lockfiles] += "${DEPLOY_DIR_DEB}/deb.lock"
# This will of course only work after rootfs_deb_do_rootfs or populate_sdk_deb has been called
DPKG_QUERY_COMMAND = "${STAGING_BINDIR_NATIVE}/dpkg-query --admindir=$INSTALL_ROOTFS_DEB/var/lib/dpkg"
-
-rootfs_list_installed_depends() {
- # Cheat here a little bit by using the opkg query helper util
- ${DPKG_QUERY_COMMAND} -W -f='Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n' | opkg-query-helper.py
-}
diff --git a/meta/classes/populate_sdk_ipk.bbclass b/meta/classes/populate_sdk_ipk.bbclass
index 4c23f05..e2e3523 100644
--- a/meta/classes/populate_sdk_ipk.bbclass
+++ b/meta/classes/populate_sdk_ipk.bbclass
@@ -2,7 +2,3 @@ do_populate_sdk[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:d
do_populate_sdk[recrdeptask] += "do_package_write_ipk"
do_populate_sdk[lockfiles] += "${WORKDIR}/ipk.lock"
-
-rootfs_list_installed_depends() {
- opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py
-}
diff --git a/meta/classes/populate_sdk_rpm.bbclass b/meta/classes/populate_sdk_rpm.bbclass
index 06605d3..0fdef3b 100644
--- a/meta/classes/populate_sdk_rpm.bbclass
+++ b/meta/classes/populate_sdk_rpm.bbclass
@@ -16,7 +16,3 @@ do_populate_sdk[recrdeptask] += "do_package_write_rpm"
rpmlibdir = "/var/lib/rpm"
do_populate_sdk[lockfiles] += "${DEPLOY_DIR_RPM}/rpm.lock"
-
-rootfs_list_installed_depends() {
- rpmresolve -t $INSTALL_ROOTFS_RPM/${rpmlibdir}
-}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-05 12:39 ` [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation Laurentiu Palcu
@ 2014-03-06 21:55 ` Richard Purdie
2014-03-07 6:35 ` Laurentiu Palcu
0 siblings, 1 reply; 12+ messages in thread
From: Richard Purdie @ 2014-03-06 21:55 UTC (permalink / raw)
To: Laurentiu Palcu; +Cc: openembedded-core
On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
> Call the new python routines.
>
> [YOCTO #5904]
>
> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
> ---
> meta/classes/buildhistory.bbclass | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> index ef4135b..01b0082 100644
> --- a/meta/classes/buildhistory.bbclass
> +++ b/meta/classes/buildhistory.bbclass
> @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
>
> with open(pkgs_list_file, 'w') as pkgs_list:
> pkgs_list.write(list_installed_packages(d, 'file'))
> +
> + pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> + "bh_installed_pkgs_deps.txt")
> +
> + with open(pkgs_deps_file, 'w') as pkgs_deps:
> + pkgs_deps.write(list_installed_packages(d, 'deps'))
> }
>
>
> @@ -338,7 +344,8 @@ buildhistory_get_installed() {
>
> # Produce dependency graph
> # First, quote each name to handle characters that cause issues for dot
> - rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
> + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
> + rm ${WORKDIR}/bh_installed_pkgs_deps.txt
> # Change delimiter from pipe to -> and set style for recommend lines
> sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
> # Add header, sorted and de-duped contents and footer and then delete the temp file
With this patch, a bitbake core-image-minimal -c populate_sdk resulted
in:
ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
The stack trace of python calls that resulted in this exception/failure was:
File: 'buildhistory_list_installed', lineno: 18, function: <module>
0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
0016:
0017:
*** 0018:buildhistory_list_installed(d)
0019:
File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
0006: "bh_installed_pkgs.txt")
0007:
0008: with open(pkgs_list_file, 'w') as pkgs_list:
*** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
0010:
0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
0012: "bh_installed_pkgs_deps.txt")
0013:
File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
0717: if img_type == "rpm":
0718: return RpmPM(d,
0719: rootfs_dir,
0720: d.getVar('TARGET_VENDOR', True)
*** 0721: ).list_installed(format)
0722: elif img_type == "ipk":
0723: return OpkgPM(d,
0724: rootfs_dir,
0725: d.getVar("IPKGCONF_TARGET", True),
File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
0850: pkg = line.split()[0]
0851: arch = line.split()[1]
0852: ver = line.split()[2]
0853: pkgorigin = line.split()[3]
*** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
0855:
0856: if format == "arch":
0857: output.append('%s %s' % (new_pkg, new_arch))
0858: elif format == "file":
File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
0472:
0473: if found == 1 and fixed_arch == fixed_cmp_arch:
0474: break
0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
*** 0476: return new_pkg, new_arch
0477:
0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
0479: for arch in feed_archs:
0480: arch = arch.replace('-', '_')
Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
ERROR: Function failed: buildhistory_list_installed
ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-06 21:55 ` Richard Purdie
@ 2014-03-07 6:35 ` Laurentiu Palcu
2014-03-08 8:23 ` Hongxu Jia
2014-03-26 9:15 ` Sarbu, Florin-Ionut (Florin)
0 siblings, 2 replies; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-07 6:35 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On Thu, Mar 06, 2014 at 09:55:38PM +0000, Richard Purdie wrote:
> On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
> > Call the new python routines.
> >
> > [YOCTO #5904]
> >
> > Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
> > ---
> > meta/classes/buildhistory.bbclass | 9 ++++++++-
> > 1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> > index ef4135b..01b0082 100644
> > --- a/meta/classes/buildhistory.bbclass
> > +++ b/meta/classes/buildhistory.bbclass
> > @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
> >
> > with open(pkgs_list_file, 'w') as pkgs_list:
> > pkgs_list.write(list_installed_packages(d, 'file'))
> > +
> > + pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> > + "bh_installed_pkgs_deps.txt")
> > +
> > + with open(pkgs_deps_file, 'w') as pkgs_deps:
> > + pkgs_deps.write(list_installed_packages(d, 'deps'))
> > }
> >
> >
> > @@ -338,7 +344,8 @@ buildhistory_get_installed() {
> >
> > # Produce dependency graph
> > # First, quote each name to handle characters that cause issues for dot
> > - rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
> > + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
> > + rm ${WORKDIR}/bh_installed_pkgs_deps.txt
> > # Change delimiter from pipe to -> and set style for recommend lines
> > sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
> > # Add header, sorted and de-duped contents and footer and then delete the temp file
>
> With this patch, a bitbake core-image-minimal -c populate_sdk resulted
No, not quite. The bug appears to be in
RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
CCing Hongxu.
laurentiu
> in:
>
> ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'buildhistory_list_installed', lineno: 18, function: <module>
> 0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
> 0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
> 0016:
> 0017:
> *** 0018:buildhistory_list_installed(d)
> 0019:
> File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
> 0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
> 0006: "bh_installed_pkgs.txt")
> 0007:
> 0008: with open(pkgs_list_file, 'w') as pkgs_list:
> *** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
> 0010:
> 0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> 0012: "bh_installed_pkgs_deps.txt")
> 0013:
> File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
> 0717: if img_type == "rpm":
> 0718: return RpmPM(d,
> 0719: rootfs_dir,
> 0720: d.getVar('TARGET_VENDOR', True)
> *** 0721: ).list_installed(format)
> 0722: elif img_type == "ipk":
> 0723: return OpkgPM(d,
> 0724: rootfs_dir,
> 0725: d.getVar("IPKGCONF_TARGET", True),
> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
> 0850: pkg = line.split()[0]
> 0851: arch = line.split()[1]
> 0852: ver = line.split()[2]
> 0853: pkgorigin = line.split()[3]
> *** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
> 0855:
> 0856: if format == "arch":
> 0857: output.append('%s %s' % (new_pkg, new_arch))
> 0858: elif format == "file":
> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
> 0472:
> 0473: if found == 1 and fixed_arch == fixed_cmp_arch:
> 0474: break
> 0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
> *** 0476: return new_pkg, new_arch
> 0477:
> 0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
> 0479: for arch in feed_archs:
> 0480: arch = arch.replace('-', '_')
> Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
>
> ERROR: Function failed: buildhistory_list_installed
> ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-07 6:35 ` Laurentiu Palcu
@ 2014-03-08 8:23 ` Hongxu Jia
2014-03-10 7:46 ` Laurentiu Palcu
2014-03-26 9:15 ` Sarbu, Florin-Ionut (Florin)
1 sibling, 1 reply; 12+ messages in thread
From: Hongxu Jia @ 2014-03-08 8:23 UTC (permalink / raw)
To: Laurentiu Palcu, Richard Purdie; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 5430 bytes --]
On 03/07/2014 02:35 PM, Laurentiu Palcu wrote:
> On Thu, Mar 06, 2014 at 09:55:38PM +0000, Richard Purdie wrote:
>> On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
>>> Call the new python routines.
>>>
>>> [YOCTO #5904]
>>>
>>> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
>>> ---
>>> meta/classes/buildhistory.bbclass | 9 ++++++++-
>>> 1 file changed, 8 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
>>> index ef4135b..01b0082 100644
>>> --- a/meta/classes/buildhistory.bbclass
>>> +++ b/meta/classes/buildhistory.bbclass
>>> @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
>>>
>>> with open(pkgs_list_file, 'w') as pkgs_list:
>>> pkgs_list.write(list_installed_packages(d, 'file'))
>>> +
>>> + pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
>>> + "bh_installed_pkgs_deps.txt")
>>> +
>>> + with open(pkgs_deps_file, 'w') as pkgs_deps:
>>> + pkgs_deps.write(list_installed_packages(d, 'deps'))
>>> }
>>>
>>>
>>> @@ -338,7 +344,8 @@ buildhistory_get_installed() {
>>>
>>> # Produce dependency graph
>>> # First, quote each name to handle characters that cause issues for dot
>>> - rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
>>> + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
>>> + rm ${WORKDIR}/bh_installed_pkgs_deps.txt
>>> # Change delimiter from pipe to -> and set style for recommend lines
>>> sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
>>> # Add header, sorted and de-duped contents and footer and then delete the temp file
>> With this patch, a bitbake core-image-minimal -c populate_sdk resulted
> No, not quite. The bug appears to be in
> RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
>
> CCing Hongxu.
Hi Laurentiu and Richard,
The reason is pkg's arch was not found in the multilib arch list.
I could not reproduce the defect on my local build with these
patches applyed.
The attach a patch that add some trace code to see which
pkg's arch is not found in this situation.
//Hongxu
> laurentiu
>
>> in:
>>
>> ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
>>
>> The stack trace of python calls that resulted in this exception/failure was:
>> File: 'buildhistory_list_installed', lineno: 18, function: <module>
>> 0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
>> 0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
>> 0016:
>> 0017:
>> *** 0018:buildhistory_list_installed(d)
>> 0019:
>> File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
>> 0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
>> 0006: "bh_installed_pkgs.txt")
>> 0007:
>> 0008: with open(pkgs_list_file, 'w') as pkgs_list:
>> *** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
>> 0010:
>> 0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
>> 0012: "bh_installed_pkgs_deps.txt")
>> 0013:
>> File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
>> 0717: if img_type == "rpm":
>> 0718: return RpmPM(d,
>> 0719: rootfs_dir,
>> 0720: d.getVar('TARGET_VENDOR', True)
>> *** 0721: ).list_installed(format)
>> 0722: elif img_type == "ipk":
>> 0723: return OpkgPM(d,
>> 0724: rootfs_dir,
>> 0725: d.getVar("IPKGCONF_TARGET", True),
>> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
>> 0850: pkg = line.split()[0]
>> 0851: arch = line.split()[1]
>> 0852: ver = line.split()[2]
>> 0853: pkgorigin = line.split()[3]
>> *** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
>> 0855:
>> 0856: if format == "arch":
>> 0857: output.append('%s %s' % (new_pkg, new_arch))
>> 0858: elif format == "file":
>> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
>> 0472:
>> 0473: if found == 1 and fixed_arch == fixed_cmp_arch:
>> 0474: break
>> 0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
>> *** 0476: return new_pkg, new_arch
>> 0477:
>> 0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
>> 0479: for arch in feed_archs:
>> 0480: arch = arch.replace('-', '_')
>> Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
>>
>> ERROR: Function failed: buildhistory_list_installed
>> ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
>>
>>
[-- Attachment #2: 0001-package_manager.py-add-trace-code-in-_pkg_translate_.patch --]
[-- Type: text/x-patch, Size: 1615 bytes --]
From 78f580e8f58ac852061b2b5879c4111607d20f45 Mon Sep 17 00:00:00 2001
From: Hongxu Jia <hongxu.jia@windriver.com>
Date: Sat, 8 Mar 2014 16:17:49 +0800
Subject: [PATCH] package_manager.py: add trace code in
_pkg_translate_smart_to_oe
There is a error while invoking RpmPM._pkg_translate_smart_to_oe(),
the reason is pkg's arch was not found in the multilib arch list.
Add some trace code to see which pkg's arch is not found.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/lib/oe/package_manager.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 76d0ce5..4cbf623 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -400,6 +400,7 @@ class RpmPM(PackageManager):
'''
def _pkg_translate_smart_to_oe(self, pkg, arch):
new_pkg = pkg
+ new_arch = ""
fixed_arch = arch.replace('_', '-')
found = 0
for mlib in self.ml_prefix_list:
@@ -426,6 +427,13 @@ class RpmPM(PackageManager):
if found == 1 and fixed_arch == fixed_cmp_arch:
break
#bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
+
+ if new_arch == "":
+ bb.warn('%s, %s is not found' % (pkg, arch))
+ for mlib in self.ml_prefix_list:
+ bb.warn('%s: %s' % (mlib, ' '.join(self.ml_prefix_list[mlib])))
+ new_arch = arch
+
return new_pkg, new_arch
def _search_pkg_name_in_feeds(self, pkg, feed_archs):
--
1.8.1.2
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-08 8:23 ` Hongxu Jia
@ 2014-03-10 7:46 ` Laurentiu Palcu
0 siblings, 0 replies; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-10 7:46 UTC (permalink / raw)
To: Hongxu Jia; +Cc: openembedded-core
I succeeded to replicate this. I'll take a look at it.
laurentiu
On Sat, Mar 08, 2014 at 04:23:22PM +0800, Hongxu Jia wrote:
> On 03/07/2014 02:35 PM, Laurentiu Palcu wrote:
> >On Thu, Mar 06, 2014 at 09:55:38PM +0000, Richard Purdie wrote:
> >>On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
> >>>Call the new python routines.
> >>>
> >>>[YOCTO #5904]
> >>>
> >>>Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
> >>>---
> >>> meta/classes/buildhistory.bbclass | 9 ++++++++-
> >>> 1 file changed, 8 insertions(+), 1 deletion(-)
> >>>
> >>>diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> >>>index ef4135b..01b0082 100644
> >>>--- a/meta/classes/buildhistory.bbclass
> >>>+++ b/meta/classes/buildhistory.bbclass
> >>>@@ -319,6 +319,12 @@ python buildhistory_list_installed() {
> >>> with open(pkgs_list_file, 'w') as pkgs_list:
> >>> pkgs_list.write(list_installed_packages(d, 'file'))
> >>>+
> >>>+ pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> >>>+ "bh_installed_pkgs_deps.txt")
> >>>+
> >>>+ with open(pkgs_deps_file, 'w') as pkgs_deps:
> >>>+ pkgs_deps.write(list_installed_packages(d, 'deps'))
> >>> }
> >>>
> >>>@@ -338,7 +344,8 @@ buildhistory_get_installed() {
> >>> # Produce dependency graph
> >>> # First, quote each name to handle characters that cause issues for dot
> >>>- rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
> >>>+ cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
> >>>+ rm ${WORKDIR}/bh_installed_pkgs_deps.txt
> >>> # Change delimiter from pipe to -> and set style for recommend lines
> >>> sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
> >>> # Add header, sorted and de-duped contents and footer and then delete the temp file
> >>With this patch, a bitbake core-image-minimal -c populate_sdk resulted
> >No, not quite. The bug appears to be in
> >RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
> >
> >CCing Hongxu.
>
> Hi Laurentiu and Richard,
>
> The reason is pkg's arch was not found in the multilib arch list.
>
> I could not reproduce the defect on my local build with these
> patches applyed.
>
> The attach a patch that add some trace code to see which
> pkg's arch is not found in this situation.
>
> //Hongxu
>
> >laurentiu
> >
> >>in:
> >>
> >>ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
> >>
> >>The stack trace of python calls that resulted in this exception/failure was:
> >>File: 'buildhistory_list_installed', lineno: 18, function: <module>
> >> 0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
> >> 0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
> >> 0016:
> >> 0017:
> >> *** 0018:buildhistory_list_installed(d)
> >> 0019:
> >>File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
> >> 0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
> >> 0006: "bh_installed_pkgs.txt")
> >> 0007:
> >> 0008: with open(pkgs_list_file, 'w') as pkgs_list:
> >> *** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
> >> 0010:
> >> 0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> >> 0012: "bh_installed_pkgs_deps.txt")
> >> 0013:
> >>File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
> >> 0717: if img_type == "rpm":
> >> 0718: return RpmPM(d,
> >> 0719: rootfs_dir,
> >> 0720: d.getVar('TARGET_VENDOR', True)
> >> *** 0721: ).list_installed(format)
> >> 0722: elif img_type == "ipk":
> >> 0723: return OpkgPM(d,
> >> 0724: rootfs_dir,
> >> 0725: d.getVar("IPKGCONF_TARGET", True),
> >>File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
> >> 0850: pkg = line.split()[0]
> >> 0851: arch = line.split()[1]
> >> 0852: ver = line.split()[2]
> >> 0853: pkgorigin = line.split()[3]
> >> *** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
> >> 0855:
> >> 0856: if format == "arch":
> >> 0857: output.append('%s %s' % (new_pkg, new_arch))
> >> 0858: elif format == "file":
> >>File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
> >> 0472:
> >> 0473: if found == 1 and fixed_arch == fixed_cmp_arch:
> >> 0474: break
> >> 0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
> >> *** 0476: return new_pkg, new_arch
> >> 0477:
> >> 0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
> >> 0479: for arch in feed_archs:
> >> 0480: arch = arch.replace('-', '_')
> >>Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
> >>
> >>ERROR: Function failed: buildhistory_list_installed
> >>ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
> >>
> >>
>
> From 78f580e8f58ac852061b2b5879c4111607d20f45 Mon Sep 17 00:00:00 2001
> From: Hongxu Jia <hongxu.jia@windriver.com>
> Date: Sat, 8 Mar 2014 16:17:49 +0800
> Subject: [PATCH] package_manager.py: add trace code in
> _pkg_translate_smart_to_oe
>
> There is a error while invoking RpmPM._pkg_translate_smart_to_oe(),
> the reason is pkg's arch was not found in the multilib arch list.
>
> Add some trace code to see which pkg's arch is not found.
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/lib/oe/package_manager.py | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
> index 76d0ce5..4cbf623 100644
> --- a/meta/lib/oe/package_manager.py
> +++ b/meta/lib/oe/package_manager.py
> @@ -400,6 +400,7 @@ class RpmPM(PackageManager):
> '''
> def _pkg_translate_smart_to_oe(self, pkg, arch):
> new_pkg = pkg
> + new_arch = ""
> fixed_arch = arch.replace('_', '-')
> found = 0
> for mlib in self.ml_prefix_list:
> @@ -426,6 +427,13 @@ class RpmPM(PackageManager):
> if found == 1 and fixed_arch == fixed_cmp_arch:
> break
> #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
> +
> + if new_arch == "":
> + bb.warn('%s, %s is not found' % (pkg, arch))
> + for mlib in self.ml_prefix_list:
> + bb.warn('%s: %s' % (mlib, ' '.join(self.ml_prefix_list[mlib])))
> + new_arch = arch
> +
> return new_pkg, new_arch
>
> def _search_pkg_name_in_feeds(self, pkg, feed_archs):
> --
> 1.8.1.2
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-07 6:35 ` Laurentiu Palcu
2014-03-08 8:23 ` Hongxu Jia
@ 2014-03-26 9:15 ` Sarbu, Florin-Ionut (Florin)
2014-03-26 10:09 ` Laurentiu Palcu
1 sibling, 1 reply; 12+ messages in thread
From: Sarbu, Florin-Ionut (Florin) @ 2014-03-26 9:15 UTC (permalink / raw)
To: PALCU, LAURENTIU, Richard Purdie; +Cc: openembedded-core@lists.openembedded.org
I get the same error on do_rootfs for a rpm based image:
Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
Florin
________________________________________
From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of Laurentiu Palcu [laurentiu.palcu@intel.com]
Sent: Friday, March 07, 2014 8:35 AM
To: Richard Purdie
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
On Thu, Mar 06, 2014 at 09:55:38PM +0000, Richard Purdie wrote:
> On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
> > Call the new python routines.
> >
> > [YOCTO #5904]
> >
> > Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
> > ---
> > meta/classes/buildhistory.bbclass | 9 ++++++++-
> > 1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> > index ef4135b..01b0082 100644
> > --- a/meta/classes/buildhistory.bbclass
> > +++ b/meta/classes/buildhistory.bbclass
> > @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
> >
> > with open(pkgs_list_file, 'w') as pkgs_list:
> > pkgs_list.write(list_installed_packages(d, 'file'))
> > +
> > + pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> > + "bh_installed_pkgs_deps.txt")
> > +
> > + with open(pkgs_deps_file, 'w') as pkgs_deps:
> > + pkgs_deps.write(list_installed_packages(d, 'deps'))
> > }
> >
> >
> > @@ -338,7 +344,8 @@ buildhistory_get_installed() {
> >
> > # Produce dependency graph
> > # First, quote each name to handle characters that cause issues for dot
> > - rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
> > + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
> > + rm ${WORKDIR}/bh_installed_pkgs_deps.txt
> > # Change delimiter from pipe to -> and set style for recommend lines
> > sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
> > # Add header, sorted and de-duped contents and footer and then delete the temp file
>
> With this patch, a bitbake core-image-minimal -c populate_sdk resulted
No, not quite. The bug appears to be in
RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
CCing Hongxu.
laurentiu
> in:
>
> ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'buildhistory_list_installed', lineno: 18, function: <module>
> 0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
> 0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
> 0016:
> 0017:
> *** 0018:buildhistory_list_installed(d)
> 0019:
> File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
> 0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
> 0006: "bh_installed_pkgs.txt")
> 0007:
> 0008: with open(pkgs_list_file, 'w') as pkgs_list:
> *** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
> 0010:
> 0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> 0012: "bh_installed_pkgs_deps.txt")
> 0013:
> File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
> 0717: if img_type == "rpm":
> 0718: return RpmPM(d,
> 0719: rootfs_dir,
> 0720: d.getVar('TARGET_VENDOR', True)
> *** 0721: ).list_installed(format)
> 0722: elif img_type == "ipk":
> 0723: return OpkgPM(d,
> 0724: rootfs_dir,
> 0725: d.getVar("IPKGCONF_TARGET", True),
> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
> 0850: pkg = line.split()[0]
> 0851: arch = line.split()[1]
> 0852: ver = line.split()[2]
> 0853: pkgorigin = line.split()[3]
> *** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
> 0855:
> 0856: if format == "arch":
> 0857: output.append('%s %s' % (new_pkg, new_arch))
> 0858: elif format == "file":
> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
> 0472:
> 0473: if found == 1 and fixed_arch == fixed_cmp_arch:
> 0474: break
> 0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
> *** 0476: return new_pkg, new_arch
> 0477:
> 0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
> 0479: for arch in feed_archs:
> 0480: arch = arch.replace('-', '_')
> Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
>
> ERROR: Function failed: buildhistory_list_installed
> ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
>
>
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-26 9:15 ` Sarbu, Florin-Ionut (Florin)
@ 2014-03-26 10:09 ` Laurentiu Palcu
2014-03-26 11:45 ` Florin Sarbu
0 siblings, 1 reply; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-26 10:09 UTC (permalink / raw)
To: Sarbu, Florin-Ionut (Florin); +Cc: openembedded-core@lists.openembedded.org
On Wed, Mar 26, 2014 at 09:15:43AM +0000, Sarbu, Florin-Ionut (Florin) wrote:
> I get the same error on do_rootfs for a rpm based image:
>
> Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
Does this happen on latest master? Do you have the 4 patches, starting with this one
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d91e35640d19471213122d36288315f071c37432
in your tree? Did you activate buildhistory? Can you provide more info
regarding the steps you followed to reproduce this?
laurentiu
>
> Florin
> ________________________________________
> From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of Laurentiu Palcu [laurentiu.palcu@intel.com]
> Sent: Friday, March 07, 2014 8:35 AM
> To: Richard Purdie
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
>
> On Thu, Mar 06, 2014 at 09:55:38PM +0000, Richard Purdie wrote:
> > On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
> > > Call the new python routines.
> > >
> > > [YOCTO #5904]
> > >
> > > Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
> > > ---
> > > meta/classes/buildhistory.bbclass | 9 ++++++++-
> > > 1 file changed, 8 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> > > index ef4135b..01b0082 100644
> > > --- a/meta/classes/buildhistory.bbclass
> > > +++ b/meta/classes/buildhistory.bbclass
> > > @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
> > >
> > > with open(pkgs_list_file, 'w') as pkgs_list:
> > > pkgs_list.write(list_installed_packages(d, 'file'))
> > > +
> > > + pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> > > + "bh_installed_pkgs_deps.txt")
> > > +
> > > + with open(pkgs_deps_file, 'w') as pkgs_deps:
> > > + pkgs_deps.write(list_installed_packages(d, 'deps'))
> > > }
> > >
> > >
> > > @@ -338,7 +344,8 @@ buildhistory_get_installed() {
> > >
> > > # Produce dependency graph
> > > # First, quote each name to handle characters that cause issues for dot
> > > - rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
> > > + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
> > > + rm ${WORKDIR}/bh_installed_pkgs_deps.txt
> > > # Change delimiter from pipe to -> and set style for recommend lines
> > > sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
> > > # Add header, sorted and de-duped contents and footer and then delete the temp file
> >
> > With this patch, a bitbake core-image-minimal -c populate_sdk resulted
> No, not quite. The bug appears to be in
> RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
>
> CCing Hongxu.
>
> laurentiu
>
> > in:
> >
> > ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
> >
> > The stack trace of python calls that resulted in this exception/failure was:
> > File: 'buildhistory_list_installed', lineno: 18, function: <module>
> > 0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
> > 0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
> > 0016:
> > 0017:
> > *** 0018:buildhistory_list_installed(d)
> > 0019:
> > File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
> > 0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
> > 0006: "bh_installed_pkgs.txt")
> > 0007:
> > 0008: with open(pkgs_list_file, 'w') as pkgs_list:
> > *** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
> > 0010:
> > 0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> > 0012: "bh_installed_pkgs_deps.txt")
> > 0013:
> > File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
> > 0717: if img_type == "rpm":
> > 0718: return RpmPM(d,
> > 0719: rootfs_dir,
> > 0720: d.getVar('TARGET_VENDOR', True)
> > *** 0721: ).list_installed(format)
> > 0722: elif img_type == "ipk":
> > 0723: return OpkgPM(d,
> > 0724: rootfs_dir,
> > 0725: d.getVar("IPKGCONF_TARGET", True),
> > File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
> > 0850: pkg = line.split()[0]
> > 0851: arch = line.split()[1]
> > 0852: ver = line.split()[2]
> > 0853: pkgorigin = line.split()[3]
> > *** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
> > 0855:
> > 0856: if format == "arch":
> > 0857: output.append('%s %s' % (new_pkg, new_arch))
> > 0858: elif format == "file":
> > File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
> > 0472:
> > 0473: if found == 1 and fixed_arch == fixed_cmp_arch:
> > 0474: break
> > 0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
> > *** 0476: return new_pkg, new_arch
> > 0477:
> > 0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
> > 0479: for arch in feed_archs:
> > 0480: arch = arch.replace('-', '_')
> > Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
> >
> > ERROR: Function failed: buildhistory_list_installed
> > ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
> >
> >
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-26 10:09 ` Laurentiu Palcu
@ 2014-03-26 11:45 ` Florin Sarbu
2014-03-26 13:30 ` Laurentiu Palcu
0 siblings, 1 reply; 12+ messages in thread
From: Florin Sarbu @ 2014-03-26 11:45 UTC (permalink / raw)
To: Laurentiu Palcu; +Cc: openembedded-core@lists.openembedded.org
On 03/26/2014 12:09 PM, Laurentiu Palcu wrote:
> On Wed, Mar 26, 2014 at 09:15:43AM +0000, Sarbu, Florin-Ionut (Florin) wrote:
>> I get the same error on do_rootfs for a rpm based image:
>>
>> Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
> Does this happen on latest master?
Yes.
> Do you have the 4 patches, starting with this one
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d91e35640d19471213122d36288315f071c37432
>
> in your tree?
Yes, I have that commit in poky.
> Did you activate buildhistory? Can you provide more info
> regarding the steps you followed to reproduce this?
I am trying to build the gemini-image from the meta-ivi layer
git@git.yoctoproject.org:meta-ivi.git The build steps are those from the
README.md file
Florin
>
> laurentiu
>> Florin
>> ________________________________________
>> From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of Laurentiu Palcu [laurentiu.palcu@intel.com]
>> Sent: Friday, March 07, 2014 8:35 AM
>> To: Richard Purdie
>> Cc: openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
>>
>> On Thu, Mar 06, 2014 at 09:55:38PM +0000, Richard Purdie wrote:
>>> On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
>>>> Call the new python routines.
>>>>
>>>> [YOCTO #5904]
>>>>
>>>> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
>>>> ---
>>>> meta/classes/buildhistory.bbclass | 9 ++++++++-
>>>> 1 file changed, 8 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
>>>> index ef4135b..01b0082 100644
>>>> --- a/meta/classes/buildhistory.bbclass
>>>> +++ b/meta/classes/buildhistory.bbclass
>>>> @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
>>>>
>>>> with open(pkgs_list_file, 'w') as pkgs_list:
>>>> pkgs_list.write(list_installed_packages(d, 'file'))
>>>> +
>>>> + pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
>>>> + "bh_installed_pkgs_deps.txt")
>>>> +
>>>> + with open(pkgs_deps_file, 'w') as pkgs_deps:
>>>> + pkgs_deps.write(list_installed_packages(d, 'deps'))
>>>> }
>>>>
>>>>
>>>> @@ -338,7 +344,8 @@ buildhistory_get_installed() {
>>>>
>>>> # Produce dependency graph
>>>> # First, quote each name to handle characters that cause issues for dot
>>>> - rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
>>>> + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
>>>> + rm ${WORKDIR}/bh_installed_pkgs_deps.txt
>>>> # Change delimiter from pipe to -> and set style for recommend lines
>>>> sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
>>>> # Add header, sorted and de-duped contents and footer and then delete the temp file
>>> With this patch, a bitbake core-image-minimal -c populate_sdk resulted
>> No, not quite. The bug appears to be in
>> RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
>>
>> CCing Hongxu.
>>
>> laurentiu
>>
>>> in:
>>>
>>> ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
>>>
>>> The stack trace of python calls that resulted in this exception/failure was:
>>> File: 'buildhistory_list_installed', lineno: 18, function: <module>
>>> 0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
>>> 0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
>>> 0016:
>>> 0017:
>>> *** 0018:buildhistory_list_installed(d)
>>> 0019:
>>> File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
>>> 0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
>>> 0006: "bh_installed_pkgs.txt")
>>> 0007:
>>> 0008: with open(pkgs_list_file, 'w') as pkgs_list:
>>> *** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
>>> 0010:
>>> 0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
>>> 0012: "bh_installed_pkgs_deps.txt")
>>> 0013:
>>> File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
>>> 0717: if img_type == "rpm":
>>> 0718: return RpmPM(d,
>>> 0719: rootfs_dir,
>>> 0720: d.getVar('TARGET_VENDOR', True)
>>> *** 0721: ).list_installed(format)
>>> 0722: elif img_type == "ipk":
>>> 0723: return OpkgPM(d,
>>> 0724: rootfs_dir,
>>> 0725: d.getVar("IPKGCONF_TARGET", True),
>>> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
>>> 0850: pkg = line.split()[0]
>>> 0851: arch = line.split()[1]
>>> 0852: ver = line.split()[2]
>>> 0853: pkgorigin = line.split()[3]
>>> *** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
>>> 0855:
>>> 0856: if format == "arch":
>>> 0857: output.append('%s %s' % (new_pkg, new_arch))
>>> 0858: elif format == "file":
>>> File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
>>> 0472:
>>> 0473: if found == 1 and fixed_arch == fixed_cmp_arch:
>>> 0474: break
>>> 0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
>>> *** 0476: return new_pkg, new_arch
>>> 0477:
>>> 0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
>>> 0479: for arch in feed_archs:
>>> 0480: arch = arch.replace('-', '_')
>>> Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
>>>
>>> ERROR: Function failed: buildhistory_list_installed
>>> ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
>>>
>>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
2014-03-26 11:45 ` Florin Sarbu
@ 2014-03-26 13:30 ` Laurentiu Palcu
0 siblings, 0 replies; 12+ messages in thread
From: Laurentiu Palcu @ 2014-03-26 13:30 UTC (permalink / raw)
To: Florin Sarbu; +Cc: openembedded-core@lists.openembedded.org
Please file a bug on this.
thanks,
laurentiu
On Wed, Mar 26, 2014 at 01:45:40PM +0200, Florin Sarbu wrote:
> On 03/26/2014 12:09 PM, Laurentiu Palcu wrote:
> >On Wed, Mar 26, 2014 at 09:15:43AM +0000, Sarbu, Florin-Ionut (Florin) wrote:
> >>I get the same error on do_rootfs for a rpm based image:
> >>
> >>Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
> >Does this happen on latest master?
> Yes.
> >Do you have the 4 patches, starting with this one
> >
> >http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d91e35640d19471213122d36288315f071c37432
> >
> >in your tree?
> Yes, I have that commit in poky.
> > Did you activate buildhistory? Can you provide more info
> >regarding the steps you followed to reproduce this?
> I am trying to build the gemini-image from the meta-ivi layer
> git@git.yoctoproject.org:meta-ivi.git The build steps are those from
> the README.md file
>
> Florin
> >
> >laurentiu
> >>Florin
> >>________________________________________
> >>From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of Laurentiu Palcu [laurentiu.palcu@intel.com]
> >>Sent: Friday, March 07, 2014 8:35 AM
> >>To: Richard Purdie
> >>Cc: openembedded-core@lists.openembedded.org
> >>Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
> >>
> >>On Thu, Mar 06, 2014 at 09:55:38PM +0000, Richard Purdie wrote:
> >>>On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
> >>>>Call the new python routines.
> >>>>
> >>>>[YOCTO #5904]
> >>>>
> >>>>Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
> >>>>---
> >>>> meta/classes/buildhistory.bbclass | 9 ++++++++-
> >>>> 1 file changed, 8 insertions(+), 1 deletion(-)
> >>>>
> >>>>diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> >>>>index ef4135b..01b0082 100644
> >>>>--- a/meta/classes/buildhistory.bbclass
> >>>>+++ b/meta/classes/buildhistory.bbclass
> >>>>@@ -319,6 +319,12 @@ python buildhistory_list_installed() {
> >>>>
> >>>> with open(pkgs_list_file, 'w') as pkgs_list:
> >>>> pkgs_list.write(list_installed_packages(d, 'file'))
> >>>>+
> >>>>+ pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> >>>>+ "bh_installed_pkgs_deps.txt")
> >>>>+
> >>>>+ with open(pkgs_deps_file, 'w') as pkgs_deps:
> >>>>+ pkgs_deps.write(list_installed_packages(d, 'deps'))
> >>>> }
> >>>>
> >>>>
> >>>>@@ -338,7 +344,8 @@ buildhistory_get_installed() {
> >>>>
> >>>> # Produce dependency graph
> >>>> # First, quote each name to handle characters that cause issues for dot
> >>>>- rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
> >>>>+ cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp && \
> >>>>+ rm ${WORKDIR}/bh_installed_pkgs_deps.txt
> >>>> # Change delimiter from pipe to -> and set style for recommend lines
> >>>> sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
> >>>> # Add header, sorted and de-duped contents and footer and then delete the temp file
> >>>With this patch, a bitbake core-image-minimal -c populate_sdk resulted
> >>No, not quite. The bug appears to be in
> >>RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
> >>
> >>CCing Hongxu.
> >>
> >>laurentiu
> >>
> >>>in:
> >>>
> >>>ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
> >>>
> >>>The stack trace of python calls that resulted in this exception/failure was:
> >>>File: 'buildhistory_list_installed', lineno: 18, function: <module>
> >>> 0014: with open(pkgs_deps_file, 'w') as pkgs_deps:
> >>> 0015: pkgs_deps.write(list_installed_packages(d, 'deps'))
> >>> 0016:
> >>> 0017:
> >>> *** 0018:buildhistory_list_installed(d)
> >>> 0019:
> >>>File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed
> >>> 0005: pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
> >>> 0006: "bh_installed_pkgs.txt")
> >>> 0007:
> >>> 0008: with open(pkgs_list_file, 'w') as pkgs_list:
> >>> *** 0009: pkgs_list.write(list_installed_packages(d, 'file'))
> >>> 0010:
> >>> 0011: pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
> >>> 0012: "bh_installed_pkgs_deps.txt")
> >>> 0013:
> >>>File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages
> >>> 0717: if img_type == "rpm":
> >>> 0718: return RpmPM(d,
> >>> 0719: rootfs_dir,
> >>> 0720: d.getVar('TARGET_VENDOR', True)
> >>> *** 0721: ).list_installed(format)
> >>> 0722: elif img_type == "ipk":
> >>> 0723: return OpkgPM(d,
> >>> 0724: rootfs_dir,
> >>> 0725: d.getVar("IPKGCONF_TARGET", True),
> >>>File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed
> >>> 0850: pkg = line.split()[0]
> >>> 0851: arch = line.split()[1]
> >>> 0852: ver = line.split()[2]
> >>> 0853: pkgorigin = line.split()[3]
> >>> *** 0854: new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
> >>> 0855:
> >>> 0856: if format == "arch":
> >>> 0857: output.append('%s %s' % (new_pkg, new_arch))
> >>> 0858: elif format == "file":
> >>>File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe
> >>> 0472:
> >>> 0473: if found == 1 and fixed_arch == fixed_cmp_arch:
> >>> 0474: break
> >>> 0475: #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
> >>> *** 0476: return new_pkg, new_arch
> >>> 0477:
> >>> 0478: def _search_pkg_name_in_feeds(self, pkg, feed_archs):
> >>> 0479: for arch in feed_archs:
> >>> 0480: arch = arch.replace('-', '_')
> >>>Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment
> >>>
> >>>ERROR: Function failed: buildhistory_list_installed
> >>>ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
> >>>
> >>>
> >>_______________________________________________
> >>Openembedded-core mailing list
> >>Openembedded-core@lists.openembedded.org
> >>http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2014-03-26 13:33 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-03-05 12:39 [PATCH 0/3] Fix issue with unpopulated buildhistory depends files Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 1/3] package_manager.py: make list_installed() list pkg dependencies too Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation Laurentiu Palcu
2014-03-06 21:55 ` Richard Purdie
2014-03-07 6:35 ` Laurentiu Palcu
2014-03-08 8:23 ` Hongxu Jia
2014-03-10 7:46 ` Laurentiu Palcu
2014-03-26 9:15 ` Sarbu, Florin-Ionut (Florin)
2014-03-26 10:09 ` Laurentiu Palcu
2014-03-26 11:45 ` Florin Sarbu
2014-03-26 13:30 ` Laurentiu Palcu
2014-03-05 12:39 ` [PATCH 3/3] populate_sdk_*.bbclass: remove old rootfs_list_installed_depends() Laurentiu Palcu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.