[V2 1/2] buildhistory_analysis.py: ignore lines without '='

[V2 1/2] buildhistory_analysis.py: ignore lines without '='

[Andreas Müller]

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
---
 meta/lib/oe/buildhistory_analysis.py |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/buildhistory_analysis.py b/meta/lib/oe/buildhistory_analysis.py
index 29dc4a9..2b4582e 100644
--- a/meta/lib/oe/buildhistory_analysis.py
+++ b/meta/lib/oe/buildhistory_analysis.py
@@ -167,9 +167,10 @@ def blob_to_dict(blob):
     alines = blob.data_stream.read().splitlines()
     adict = {}
     for line in alines:
-        splitv = [i.strip() for i in line.split('=',1)]
-        if splitv.count > 1:
-            adict[splitv[0]] = splitv[1]
+        if line.find("=") != -1:
+            splitv = [i.strip() for i in line.split('=',1)]
+            if splitv.count > 1:
+                adict[splitv[0]] = splitv[1]
     return adict
 
 
-- 
1.7.6.5

[V2 2/2] buildhistory.bbclass: add pkg_pre/postinst/rm contents

[Andreas Müller]

A use-case would have been [1].

The following tests were performed:
* image from scratch with old buildhistory contents
* image from scratch with buildhistory contents from scratch
* decrement a PR for a test recipe and check if the message
  'ERROR: Package version for xy went backwards' is displayed.

[1] http://patches.openembedded.org/patch/28841/

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
---
 meta/classes/buildhistory.bbclass |   80 +++++++++++++++++++++++--------------
 1 files changed, 50 insertions(+), 30 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index d2d19ff..9c49bab 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -57,6 +57,10 @@ python buildhistory_emit_pkghistory() {
 			self.rrecommends = ""
 			self.files = ""
 			self.filelist = ""
+			self.preinst = ""
+			self.postinst = ""
+			self.prerm = ""
+			self.postrm = ""
 
 	# Should check PACKAGES here to see if anything removed
 
@@ -74,18 +78,19 @@ python buildhistory_emit_pkghistory() {
 		try:
 			for line in f:
 				lns = line.split('=')
-				name = lns[0].strip()
-				value = lns[1].strip(" \t\r\n").strip('"')
-				if name == "PE":
-					rcpinfo.pe = value
-				elif name == "PV":
-					rcpinfo.pv = value
-				elif name == "PR":
-					rcpinfo.pr = value
-				elif name == "DEPENDS":
-					rcpinfo.depends = value
-				elif name == "PACKAGES":
-					rcpinfo.packages = value
+				if len(lns)==2:
+					name = lns[0].strip()
+					value = lns[1].strip(" \t\r\n").strip('"')
+					if name == "PE":
+						rcpinfo.pe = value
+					elif name == "PV":
+						rcpinfo.pv = value
+					elif name == "PR":
+						rcpinfo.pr = value
+					elif name == "DEPENDS":
+						rcpinfo.depends = value
+					elif name == "PACKAGES":
+						rcpinfo.packages = value
 		finally:
 			f.close()
 		return rcpinfo
@@ -96,24 +101,25 @@ python buildhistory_emit_pkghistory() {
 		try:
 			for line in f:
 				lns = line.split('=')
-				name = lns[0].strip()
-				value = lns[1].strip(" \t\r\n").strip('"')
-				if name == "PE":
-					pkginfo.pe = value
-				elif name == "PV":
-					pkginfo.pv = value
-				elif name == "PR":
-					pkginfo.pr = value
-				elif name == "RDEPENDS":
-					pkginfo.rdepends = value
-				elif name == "RRECOMMENDS":
-					pkginfo.rrecommends = value
-				elif name == "PKGSIZE":
-					pkginfo.size = long(value)
-				elif name == "FILES":
-					pkginfo.files = value
-				elif name == "FILELIST":
-					pkginfo.filelist = value
+				if len(lns)==2:
+					name = lns[0].strip()
+					value = lns[1].strip(" \t\r\n").strip('"')
+					if name == "PE":
+						pkginfo.pe = value
+					elif name == "PV":
+						pkginfo.pv = value
+					elif name == "PR":
+						pkginfo.pr = value
+					elif name == "RDEPENDS":
+						pkginfo.rdepends = value
+					elif name == "RRECOMMENDS":
+						pkginfo.rrecommends = value
+					elif name == "PKGSIZE":
+						pkginfo.size = long(value)
+					elif name == "FILES":
+						pkginfo.files = value
+					elif name == "FILELIST":
+						pkginfo.filelist = value
 		finally:
 			f.close()
 		return pkginfo
@@ -198,6 +204,12 @@ python buildhistory_emit_pkghistory() {
 		filelist.sort()
 		pkginfo.filelist = " ".join(filelist)
 
+		# pre/postinst/rm
+		pkginfo.preinst = getpkgvar(pkg, 'pkg_preinst') or ""
+		pkginfo.postinst = getpkgvar(pkg, 'pkg_postinst') or ""
+		pkginfo.prerm = getpkgvar(pkg, 'pkg_prerm') or ""
+		pkginfo.postrm = getpkgvar(pkg, 'pkg_postrm') or ""
+
 		write_pkghistory(pkginfo, d)
 
 		write_latestlink(pkg, pe, pv, pr, d)
@@ -246,6 +258,14 @@ def write_pkghistory(pkginfo, d):
 		f.write("PKGSIZE = %d\n" %  pkginfo.size)
 		f.write("FILES = %s\n" %  pkginfo.files)
 		f.write("FILELIST = %s\n" %  pkginfo.filelist)
+		if pkginfo.preinst != "":
+			f.write("\npkg_preinst=\n%s\n" %  pkginfo.preinst)
+		if pkginfo.postinst != "":
+			f.write("\npkg_postinst=\n%s\n" %  pkginfo.postinst)
+		if pkginfo.prerm != "":
+			f.write("\npkg_prerm=\n%s\n" %  pkginfo.prerm)
+		if pkginfo.postrm != "":
+			f.write("\npkg_postrm=\n%s\n" %  pkginfo.postrm)
 	finally:
 		f.close()
 
-- 
1.7.6.5

Re: [V2 2/2] buildhistory.bbclass: add pkg_pre/postinst/rm contents

[Paul Eggleton]

On Thursday 31 May 2012 13:12:45 Andreas Müller wrote:
> A use-case would have been [1].
> 
> The following tests were performed:
> * image from scratch with old buildhistory contents
> * image from scratch with buildhistory contents from scratch
> * decrement a PR for a test recipe and check if the message
>   'ERROR: Package version for xy went backwards' is displayed.
> 
> [1] http://patches.openembedded.org/patch/28841/
> 
> Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
> ---
>  meta/classes/buildhistory.bbclass |   80
> +++++++++++++++++++++++-------------- 1 files changed, 50 insertions(+), 30
> deletions(-)
> 
> diff --git a/meta/classes/buildhistory.bbclass
> b/meta/classes/buildhistory.bbclass index d2d19ff..9c49bab 100644
> --- a/meta/classes/buildhistory.bbclass
> +++ b/meta/classes/buildhistory.bbclass
> @@ -57,6 +57,10 @@ python buildhistory_emit_pkghistory() {
>  			self.rrecommends = ""
>  			self.files = ""
>  			self.filelist = ""
> +			self.preinst = ""
> +			self.postinst = ""
> +			self.prerm = ""
> +			self.postrm = ""
> 
>  	# Should check PACKAGES here to see if anything removed
> 
> @@ -74,18 +78,19 @@ python buildhistory_emit_pkghistory() {
>  		try:
>  			for line in f:
>  				lns = line.split('=')
> -				name = lns[0].strip()
> -				value = lns[1].strip(" \t\r\n").strip('"')
> -				if name == "PE":
> -					rcpinfo.pe = value
> -				elif name == "PV":
> -					rcpinfo.pv = value
> -				elif name == "PR":
> -					rcpinfo.pr = value
> -				elif name == "DEPENDS":
> -					rcpinfo.depends = value
> -				elif name == "PACKAGES":
> -					rcpinfo.packages = value
> +				if len(lns)==2:
> +					name = lns[0].strip()
> +					value = lns[1].strip(" \t\r\n").strip('"')
> +					if name == "PE":
> +						rcpinfo.pe = value
> +					elif name == "PV":
> +						rcpinfo.pv = value
> +					elif name == "PR":
> +						rcpinfo.pr = value
> +					elif name == "DEPENDS":
> +						rcpinfo.depends = value
> +					elif name == "PACKAGES":
> +						rcpinfo.packages = value
>  		finally:
>  			f.close()
>  		return rcpinfo
> @@ -96,24 +101,25 @@ python buildhistory_emit_pkghistory() {
>  		try:
>  			for line in f:
>  				lns = line.split('=')
> -				name = lns[0].strip()
> -				value = lns[1].strip(" \t\r\n").strip('"')
> -				if name == "PE":
> -					pkginfo.pe = value
> -				elif name == "PV":
> -					pkginfo.pv = value
> -				elif name == "PR":
> -					pkginfo.pr = value
> -				elif name == "RDEPENDS":
> -					pkginfo.rdepends = value
> -				elif name == "RRECOMMENDS":
> -					pkginfo.rrecommends = value
> -				elif name == "PKGSIZE":
> -					pkginfo.size = long(value)
> -				elif name == "FILES":
> -					pkginfo.files = value
> -				elif name == "FILELIST":
> -					pkginfo.filelist = value
> +				if len(lns)==2:
> +					name = lns[0].strip()
> +					value = lns[1].strip(" \t\r\n").strip('"')
> +					if name == "PE":
> +						pkginfo.pe = value
> +					elif name == "PV":
> +						pkginfo.pv = value
> +					elif name == "PR":
> +						pkginfo.pr = value
> +					elif name == "RDEPENDS":
> +						pkginfo.rdepends = value
> +					elif name == "RRECOMMENDS":
> +						pkginfo.rrecommends = value
> +					elif name == "PKGSIZE":
> +						pkginfo.size = long(value)
> +					elif name == "FILES":
> +						pkginfo.files = value
> +					elif name == "FILELIST":
> +						pkginfo.filelist = value
>  		finally:
>  			f.close()
>  		return pkginfo
> @@ -198,6 +204,12 @@ python buildhistory_emit_pkghistory() {
>  		filelist.sort()
>  		pkginfo.filelist = " ".join(filelist)
> 
> +		# pre/postinst/rm
> +		pkginfo.preinst = getpkgvar(pkg, 'pkg_preinst') or ""
> +		pkginfo.postinst = getpkgvar(pkg, 'pkg_postinst') or ""
> +		pkginfo.prerm = getpkgvar(pkg, 'pkg_prerm') or ""
> +		pkginfo.postrm = getpkgvar(pkg, 'pkg_postrm') or ""
> +
>  		write_pkghistory(pkginfo, d)
> 
>  		write_latestlink(pkg, pe, pv, pr, d)
> @@ -246,6 +258,14 @@ def write_pkghistory(pkginfo, d):
>  		f.write("PKGSIZE = %d\n" %  pkginfo.size)
>  		f.write("FILES = %s\n" %  pkginfo.files)
>  		f.write("FILELIST = %s\n" %  pkginfo.filelist)
> +		if pkginfo.preinst != "":
> +			f.write("\npkg_preinst=\n%s\n" %  pkginfo.preinst)
> +		if pkginfo.postinst != "":
> +			f.write("\npkg_postinst=\n%s\n" %  pkginfo.postinst)
> +		if pkginfo.prerm != "":
> +			f.write("\npkg_prerm=\n%s\n" %  pkginfo.prerm)
> +		if pkginfo.postrm != "":
> +			f.write("\npkg_postrm=\n%s\n" %  pkginfo.postrm)
>  	finally:
>  		f.close()

I'd still like to rework this to use separate files. What if a line in the 
postinst script includes an = (which is not unlikely)?

Note that I'm happy do the reworking myself, I just won't be able to do it 
until the week after next.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [V2 2/2] buildhistory.bbclass: add pkg_pre/postinst/rm contents

[Andreas Müller]

On Fri, Jun 1, 2012 at 6:54 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> I'd still like to rework this to use separate files. What if a line in the
> postinst script includes an = (which is not unlikely)?

Yes after first usage (it is not possible on all my machines due to
pythongit version) and taking a look into the parsing scripts I now
understand your concern. It might cause non predictable results. Worst
case if there is a oe varaible name used in scripts. One question:
when outputting into separate files: isn't there a simple way to
attach the raw git diff output for these (maybe as option) in the
results of buildhistory-diff? Then we would have all information
together under one hood.

>
> Note that I'm happy do the reworking myself, I just won't be able to do it
> until the week after next.
>
Thanks. Until then I have my version :)

Andreas

Re: [V2 2/2] buildhistory.bbclass: add pkg_pre/postinst/rm contents

[Paul Eggleton]

On Friday 01 June 2012 19:24:45 Andreas Müller wrote:
> On Fri, Jun 1, 2012 at 6:54 PM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
> > I'd still like to rework this to use separate files. What if a line in the
> > postinst script includes an = (which is not unlikely)?
> 
> Yes after first usage (it is not possible on all my machines due to
> pythongit version) and taking a look into the parsing scripts I now
> understand your concern. It might cause non predictable results. Worst
> case if there is a oe varaible name used in scripts. One question:
> when outputting into separate files: isn't there a simple way to
> attach the raw git diff output for these (maybe as option) in the
> results of buildhistory-diff? Then we would have all information
> together under one hood.

Definitely - if it doesn't already work once we've reworked it we can poke the 
code in lib/oe/buildhistory_analysis.py to just report the diff of the blobs 
(we already do this for other classes of files, some of the image ones IIRC).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre