[PATCH 0/3]distrodata.bbcalss: To get more precise upstream version for pkg report system

[PATCH 0/3]distrodata.bbcalss: To get more precise upstream version for pkg report system

[Mei Lei]

There are some changes to get more recipes upstream version and to generate some necessary data for pkg report system.

Pull URL: git://git.pokylinux.org/poky-contrib.git
  Branch: lmei3/pkg-report-system
  Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=lmei3/pkg-report-system

Thanks,
    Mei Lei <lei.mei@intel.com>
---


Mei Lei (3):
  distrodata.bbclass: Change for package report system
  base.bbclass: Add inherit information for checkpkg task
  bitbake.conf: Change the GNU_MIRROR to get more precise result

 meta/classes/base.bbclass       |    4 +-
 meta/classes/distrodata.bbclass |   44 +++++++++++++++++++++++++++++++-------
 meta/conf/bitbake.conf          |    2 +-
 3 files changed, 39 insertions(+), 11 deletions(-)

[PATCH 1/3] distrodata.bbclass: Change for package report system

[Mei Lei]

Add some code to generate data for package report system. Fix some bugs when checking upstrem version.

Signed-off-by: Mei Lei <lei.mei@intel.com>
---
 meta/classes/distrodata.bbclass |   44 +++++++++++++++++++++++++++++++-------
 1 files changed, 36 insertions(+), 8 deletions(-)

diff --git a/meta/classes/distrodata.bbclass b/meta/classes/distrodata.bbclass
index 0378790..f4eb6fd 100644
--- a/meta/classes/distrodata.bbclass
+++ b/meta/classes/distrodata.bbclass
@@ -1,4 +1,3 @@
-
 require conf/distro/include/distro_tracking_fields.inc
 
 addhandler distro_eventhandler
@@ -215,7 +214,6 @@ addhandler checkpkg_eventhandler
 python checkpkg_eventhandler() {
     from bb.event import Handled, NotHandled
     # if bb.event.getName(e) == "TaskStarted":
-
     if bb.event.getName(e) == "BuildStarted":
 	"""initialize log files."""
 	logpath = bb.data.getVar('LOG_DIR', e.data, 1)
@@ -234,6 +232,9 @@ python checkpkg_eventhandler() {
 	f.write("Package\tOwner\tURI Type\tVersion\tTracking\tUpstream\tTMatch\tRMatch\n")
         f.close()
         bb.utils.unlockfile(lf)
+	"""initialize log files for Package Report System"""
+	logpath2 = bb.data.getVar('LOG_DIR', e.data, 1)
+	bb.utils.mkdirhier(logpath2)
 
     return NotHandled
 }
@@ -244,7 +245,6 @@ python do_checkpkg() {
 	import sys
 	import re
 	import tempfile
-
 	"""
 	sanity check to ensure same name and type. Match as many patterns as possible
 	such as:
@@ -262,8 +262,8 @@ python do_checkpkg() {
 	prefix1 = "[a-zA-Z][a-zA-Z0-9]*([\-_][a-zA-Z]\w+)*[\-_]"	# match most patterns which uses "-" as separator to version digits
 	prefix2 = "[a-zA-Z]+"			# a loose pattern such as for unzip552.tar.gz
 	prefix = "(%s|%s)" % (prefix1, prefix2)
-	suffix = "(tar\.gz|tgz|tar\.bz2|zip)"
-	suffixtuple = ("tar.gz", "tgz", "zip", "tar.bz2")
+	suffix = "(tar\.gz|tgz|tar\.bz2|zip|tar\.xz)"
+	suffixtuple = ("tar.gz", "tgz", "zip", "tar.bz2", "tar.xz")
 
 	sinterstr = "(?P<name>%s?)(?P<ver>.*)" % prefix
 	sdirstr = "(?P<name>%s)(?P<ver>.*)\.(?P<type>%s$)" % (prefix, suffix)
@@ -327,7 +327,7 @@ python do_checkpkg() {
 			which is designed for check purpose but we override check command for our own purpose
 			"""
 			ld = bb.data.createCopy(d)
-			bb.data.setVar('CHECKCOMMAND_wget', "/usr/bin/env wget -t 1 --passive-ftp -O %s '${URI}'" \
+			bb.data.setVar('CHECKCOMMAND_wget', "/usr/bin/env wget -t 1 --passive-ftp -O %s --user-agent=\"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.12) Gecko/20101027 Ubuntu/9.10 (karmic) Firefox/3.6.12\" '${URI}'" \
 					% tmpf.name, d)
 			bb.data.update_data(ld)
 
@@ -371,7 +371,7 @@ python do_checkpkg() {
 			else:
 				s = "(\d+[\.\-_])+\d+/?"
 				
-			searchstr = "[hH][rR][eE][fF]=\"%s\">" % s
+			searchstr = "[hH][rR][eE][fF]=\"%s\".*>" % s
 			reg = re.compile(searchstr)
 
 			valid = 0
@@ -425,7 +425,7 @@ python do_checkpkg() {
 			"""match "{PN}-5.21.1.tar.gz">{PN}-5.21.1.tar.gz """
 			pn1 = re.search("^%s" % prefix, curname).group()
 			s = "[^\"]*%s[^\d\"]*?(\d+[\.\-_])+[^\"]*" % pn1
-			searchstr = "[hH][rR][eE][fF]=\"%s\">" % s
+			searchstr = "[hH][rR][eE][fF]=\"%s\".*>" % s
 			reg = re.compile(searchstr)
 	
 			valid = 0
@@ -465,10 +465,30 @@ python do_checkpkg() {
 	bb.utils.mkdirhier(logpath)
 	logfile = os.path.join(logpath, "checkpkg.csv")
 
+	"""initialize log files for package report system"""
+	logpath2 = bb.data.getVar('LOG_DIR', d, 1)
+	bb.utils.mkdirhier(logpath2)
+	logfile2 = os.path.join(logpath2, "get_pkg_info.log.%s" % bb.data.getVar('PN', d, 1))
+	if not os.path.exists(logfile2):
+		slogfile = os.path.join(logpath2, "get_pkg_info.log")
+		if os.path.exists(slogfile):
+			os.remove(slogfile)
+		os.system("touch %s" % logfile2)
+		os.symlink(logfile2, slogfile)
+
 	"""generate package information from .bb file"""
 	pname = bb.data.getVar('PN', d, 1)
 	pdesc = bb.data.getVar('DESCRIPTION', d, 1)
 	pgrp = bb.data.getVar('SECTION', d, 1)
+	pversion = bb.data.getVar('PV', d, 1)
+	plicense = bb.data.getVar('LICENSE',d,1)
+	psection = bb.data.getVar('SECTION',d,1)
+	phome = bb.data.getVar('HOMEPAGE', d, 1)
+	prelease = bb.data.getVar('PR',d,1)
+	ppriority = bb.data.getVar('PRIORITY',d,1)
+	pdepends = bb.data.getVar('DEPENDS',d,1)
+	pbugtracker = bb.data.getVar('BUGTRACKER',d,1)
+	ppe = bb.data.getVar('PE',d,1)
 
 	found = 0
 	for uri in src_uri.split():
@@ -614,6 +634,14 @@ python do_checkpkg() {
 		  (pname, maintainer, pproto, pcurver, pmver, pupver, pmstatus, pstatus))
 	f.close()
 	bb.utils.unlockfile(lf)
+
+  	"""write into get_pkg_info log file to supply data for package report system"""
+	lf2 = bb.utils.lockfile(logfile2 + ".lock")
+	f2 = open(logfile2, "a")
+	f2.write("%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n" % \
+		  (pname,pversion,pupver,plicense,psection, phome,prelease, ppriority,pdepends,pbugtracker,ppe,pdesc,pstatus))
+	f2.close()
+	bb.utils.unlockfile(lf2)
 }
 
 addtask checkpkgall after do_checkpkg
-- 
1.6.3.3

[PATCH 2/3] base.bbclass: Add inherit information for checkpkg task

[Mei Lei]

Add inherit distrodata for running checkpkg task

Signed-off-by: Mei Lei <lei.mei@intel.com>
---
 meta/classes/base.bbclass |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index c60048b..a8cf4fd 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -2,12 +2,12 @@ BB_DEFAULT_TASK ?= "build"
 
 inherit patch
 inherit staging
-
 inherit mirrors
 inherit utils
 inherit utility-tasks
 inherit metadata_scm
-
+inherit distrodata
+ 
 python sys_path_eh () {
     if isinstance(e, bb.event.ConfigParsed):
         import sys
-- 
1.6.3.3

[PATCH 3/3] bitbake.conf: Change the GNU_MIRROR to get more precise result

[Mei Lei]

Use this mirror, more result can be checked

Signed-off-by: Mei Lei <lei.mei@intel.com>
---
 meta/conf/bitbake.conf |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index c26beea..67fc3fb 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -502,7 +502,7 @@ FREESMARTPHONE_GIT = "git://git.freesmartphone.org"
 GENTOO_MIRROR = "http://distfiles.gentoo.org/distfiles"
 GNOME_GIT = "git://git.gnome.org"
 GNOME_MIRROR = "http://ftp.gnome.org/pub/GNOME/sources"
-GNU_MIRROR = "ftp://ftp.gnu.org/gnu"
+GNU_MIRROR = "ftp://ftp.gnu.org/pub/gnu"
 GPE_MIRROR = "http://gpe.linuxtogo.org/download/source"
 GPE_EXTRA_SVN = "svn://projects.linuxtogo.org/svn/gpe/trunk/extra;module=${PN}"
 GPE_SVN = "svn://projects.linuxtogo.org/svn/gpe/trunk/base;module=${PN}"
-- 
1.6.3.3

Re: [PATCH 0/3]distrodata.bbcalss: To get more precise upstream version for pkg report system

[Saul Wold]

On 12/22/2010 05:22 AM, Mei Lei wrote:
> There are some changes to get more recipes upstream version and to generate some necessary data for pkg report system.
>
> Pull URL: git://git.pokylinux.org/poky-contrib.git
>    Branch: lmei3/pkg-report-system
>    Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=lmei3/pkg-report-system
>
> Thanks,
>      Mei Lei<lei.mei@intel.com>
> ---
>
Lei:

There are various issues with these patches please see my comments below

>
> Mei Lei (3):
>    distrodata.bbclass: Change for package report system
Not sure why you needed to add a logpath2 as you are using the same 
LOG_DIR var as logpath. Are you intending to create a separate directory.

Is there a reason you need to have each package into in separate files? 
Can your tool not read 1 larger flat file then we can just use 1 output 
format for the spreadsheet

>    base.bbclass: Add inherit information for checkpkg task
We don't want to inherit the distrodata  by default, you can add this to 
your local.conf

>    bitbake.conf: Change the GNU_MIRROR to get more precise result
>
Not sure how this gives a more precise result, gnu and pub/gnu are the 
same location, one is linked to the other.

Sau!

>   meta/classes/base.bbclass       |    4 +-
>   meta/classes/distrodata.bbclass |   44 +++++++++++++++++++++++++++++++-------
>   meta/conf/bitbake.conf          |    2 +-
>   3 files changed, 39 insertions(+), 11 deletions(-)
>
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky
>

Re: [PATCH 0/3]distrodata.bbcalss: To get more precise upstream version for pkg report system

[Mei, Lei]

>-----Original Message-----
>From: Wold, Saul
>Sent: Thursday, December 23, 2010 12:46 AM
>To: Mei, Lei
>Cc: poky@yoctoproject.org
>Subject: Re: [poky] [PATCH 0/3]distrodata.bbcalss: To get more precise
>upstream version for pkg report system
>
>On 12/22/2010 05:22 AM, Mei Lei wrote:
>> There are some changes to get more recipes upstream version and to
>generate some necessary data for pkg report system.
>>
>> Pull URL: git://git.pokylinux.org/poky-contrib.git
>>    Branch: lmei3/pkg-report-system
>>    Browse:
>http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=lmei3/pkg-report-system
>>
>> Thanks,
>>      Mei Lei<lei.mei@intel.com>
>> ---
>>
>Lei:
>
>There are various issues with these patches please see my comments below
>
>>
>> Mei Lei (3):
>>    distrodata.bbclass: Change for package report system
>Not sure why you needed to add a logpath2 as you are using the same
>LOG_DIR var as logpath. Are you intending to create a separate directory.

Oh, I didn't realize this problem before, I will change this.

>
>Is there a reason you need to have each package into in separate files?
>Can your tool not read 1 larger flat file then we can just use 1 output
>format for the spreadsheet

Ok, I will change this output as one file.
The reason why I generate every output file for every package is that the default output this is it when I first use checkpkg task long long time ago, so the package report system obey this all the time.
I will change this.

>
>>    base.bbclass: Add inherit information for checkpkg task
>We don't want to inherit the distrodata  by default, you can add this to
>your local.conf

Ok, I understand it.

>
>>    bitbake.conf: Change the GNU_MIRROR to get more precise result
>>
>Not sure how this gives a more precise result, gnu and pub/gnu are the
>same location, one is linked to the other.
>

Ok , I get it. This patch is not necessary.

>Sau!
>
>>   meta/classes/base.bbclass       |    4 +-
>>   meta/classes/distrodata.bbclass |   44
>+++++++++++++++++++++++++++++++-------
>>   meta/conf/bitbake.conf          |    2 +-
>>   3 files changed, 39 insertions(+), 11 deletions(-)
>>
>> _______________________________________________
>> poky mailing list
>> poky@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/poky
>>