[PATCH 00/12] Usage bitbake fetcher latest_versionstring in distrodata_class

[PATCH 00/12] Usage bitbake fetcher latest_versionstring in distrodata_class

[Aníbal Limón]

Since Bitbake fetcher has latest_versionstring method that support search last version
of package in upstream sites, distrodata class should use it instead of do itself,

Summary of changes,

distrodata_class: Re-implement upstream version detection using Bitbake fetcher methods.
package_regex.inc: Update regex because Bitbake fetcher already provides support for search
in HTML href's tags instead of RAW lines.
bitbake: Improvements in fetcher wget latest_versionstring method based on [1] and clean code.

Compatibility testing was made running distrodata class and the result files can be found at [1].

The code has DEPENDENCIES the Bitbake changes should be MERGE first then package_regex.inc
and finally distrodata_class.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=1813

The following changes since commit 0bc03af7ee6112fa0af0608b02f715ec8495e4ff:

  maintainers.inc: update maintainers (2014-11-26 17:06:10 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib alimon/fetcher-distrodata
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/fetcher-distrodata

Aníbal Limón (12):
  distrodata_class: checkpkg make usage of latest_versionstring methods
    in bitbake fetcher
  package_regex.inc: Update REGEX'es in order to find upstream versions
    correctly
  fetch2/wget.py: latest_versionstring create _init_regex method for
    have one place when regex'es are defined
  fetch2/wget.py: _init_regexes rename variables to be more consistent
    and move dirver_regex into it
  fetch/wget.php: latest version string only try to find latest
    directory when REGEX_URI isn't specified to avoid unnecessary
    processing and makes code easier
  fetch2/wget.py: latest_versionstring add package_custom_regex_comp
  fetch2/wget.py: latest_versionstring improvments in searching
  fetch/wget.py: latest_versionstring remove unnecessary usage for name
    in version comparision
  fetch2/wget: latest_versionstring add support for search in RAW html
    lines
  fetch/wget.py: latest_versionstring remove newuri set because is
    previous set
  tests/fetch.py: Update wget latest_versionstring cups case
  fetch2/wget.py: latest_versionstring clean improvments minor

 bitbake/lib/bb/fetch2/wget.py                    | 146 ++++---
 bitbake/lib/bb/tests/fetch.py                    |   2 +-
 meta-yocto/conf/distro/include/package_regex.inc | 400 +++++++++---------
 meta/classes/distrodata.bbclass                  | 492 ++---------------------
 4 files changed, 337 insertions(+), 703 deletions(-)

-- 
1.9.1

[PATCH 03/12] fetch2/wget.py: latest_versionstring create _init_regex method for have one place when regex'es are defined

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index db5f27b..687d494 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -273,12 +273,9 @@ class Wget(FetchMethod):
         if valid and version:
             return re.sub('_', '.', version[1])
 
-    def latest_versionstring(self, ud, d):
+    def _init_regexes(self):
         """
-        Manipulate the URL and try to obtain the latest package version
-
-        sanity check to ensure same name and type. Match as many patterns as possible
-        such as:
+        Match as many patterns as possible such as:
                 gnome-common-2.20.0.tar.gz (most common format)
                 gtk+-2.90.1.tar.gz
                 xf86-input-synaptics-12.6.9.tar.gz
@@ -310,10 +307,18 @@ class Wget(FetchMethod):
         # match name, version and archive type of a package
         self.name_version_type_regex = re.compile("(?P<name>%s?)\.?v?(?P<ver>%s)(\-source)?[\.\-](?P<type>%s$)" % (self.pn_regex, version_regex, suffixlist))
 
+    def latest_versionstring(self, ud, d):
+        """
+        Manipulate the URL and try to obtain the latest package version
+
+        sanity check to ensure same name and type.
+        """
         regex_uri = d.getVar("REGEX_URI", True)
         newpath = ud.path
         pupver = ""
 
+        self._init_regexes()
+
         # search for version matches on folders inside the path, like:
         # "5.7" in http://download.gnome.org/sources/${PN}/5.7/${PN}-${PV}.tar.gz
         m = re.search("(?P<dirver>[^/]*(\d+\.)*\d+([\-_]r\d+)*)/", ud.path)
-- 
1.9.1

[PATCH 04/12] fetch2/wget.py: _init_regexes rename variables to be more consistent and move dirver_regex into it

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 687d494..b6b1339 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -151,7 +151,7 @@ class Wget(FetchMethod):
         Check for a new suffix type that we have never heard of before
         """
         if (newsuffix):
-            m = self.suffixregex.search(newsuffix)
+            m = self.suffix_regex_comp.search(newsuffix)
             if not m:
                 bb.warn("%s has a possible unknown suffix: %s" % (newpn, newsuffix))
                 return False
@@ -233,7 +233,7 @@ class Wget(FetchMethod):
         If error or no version, return ""
         """
         valid = 0
-        version = self._parse_path(self.name_version_type_regex, packagename)
+        version = self._parse_path(self.package_regex_comp, packagename)
 
         bb.debug(3, "VersionURL: %s" % (url))
         soup = BeautifulSoup(self._fetch_index(url, ud, d))
@@ -258,7 +258,7 @@ class Wget(FetchMethod):
                 else:
                     continue
             else:
-                newver = self._parse_path(self.name_version_type_regex, line['href'])
+                newver = self._parse_path(self.package_regex_comp, line['href'])
             valid = 1
             if newver and self._vercmp(version, newver) == True:
                 version = newver
@@ -294,18 +294,26 @@ class Wget(FetchMethod):
         # a loose pattern such as for 80325-quicky-0.4.tar.gz
         pn_prefix3 = "[0-9]+[\-]?[a-zA-Z]+"
         # Save the Package Name (pn) Regex for use later
-        self.pn_regex = "(%s|%s|%s)" % (pn_prefix1, pn_prefix2, pn_prefix3)
+        pn_regex = "(%s|%s|%s)" % (pn_prefix1, pn_prefix2, pn_prefix3)
 
         # match version
-        version_regex = "(([A-Z]*\d+[a-zA-Z]*[\.\-_]*)+)"
+        pver_regex = "(([A-Z]*\d+[a-zA-Z]*[\.\-_]*)+)"
+
+        # match arch
+        parch_regex = "\-source|_all_"
 
         # src.rpm extension was added only for rpm package. Can be removed if the rpm
         # packaged will always be considered as having to be manually upgraded
-        suffixlist = "(tar\.gz|tgz|tar\.bz2|zip|xz|rpm|bz2|orig\.tar\.gz|tar\.xz|src\.tar\.gz|src\.tgz|svnr\d+\.tar\.bz2|stable\.tar\.gz|src\.rpm)"
-        self.suffixregex = re.compile(suffixlist)
+        psuffix_regex = "(tar\.gz|tgz|tar\.bz2|zip|xz|rpm|bz2|orig\.tar\.gz|tar\.xz|src\.tar\.gz|src\.tgz|svnr\d+\.tar\.bz2|stable\.tar\.gz|src\.rpm)"
 
         # match name, version and archive type of a package
-        self.name_version_type_regex = re.compile("(?P<name>%s?)\.?v?(?P<ver>%s)(\-source)?[\.\-](?P<type>%s$)" % (self.pn_regex, version_regex, suffixlist))
+        self.package_regex_comp = re.compile("(?P<name>%s?)\.?v?(?P<ver>%s)(?P<arch>%s)?[\.\-](?P<type>%s$)"
+                                                    % (pn_regex, pver_regex, parch_regex, psuffix_regex))
+        self.suffix_regex_comp = re.compile(psuffix_regex)
+
+        # search for version matches on folders inside the path, like:
+        # "5.7" in http://download.gnome.org/sources/${PN}/5.7/${PN}-${PV}.tar.gz
+        self.dirver_regex_comp = re.compile("(?P<dirver>[^/]*(\d+\.)*\d+([\-_]r\d+)*)/")
 
     def latest_versionstring(self, ud, d):
         """
@@ -319,11 +327,9 @@ class Wget(FetchMethod):
 
         self._init_regexes()
 
-        # search for version matches on folders inside the path, like:
-        # "5.7" in http://download.gnome.org/sources/${PN}/5.7/${PN}-${PV}.tar.gz
-        m = re.search("(?P<dirver>[^/]*(\d+\.)*\d+([\-_]r\d+)*)/", ud.path)
+        m = self.dirver_regex_comp.search(ud.path)
         bb.debug(3, "path = %s" % (ud.path))
-        bb.debug(3, "Regex: %s" % (self.name_version_type_regex.pattern))
+        bb.debug(3, "Regex: %s" % (self.package_regex_comp.pattern))
         if m and not regex_uri:
             dirver = m.group('dirver')
             # generate the new uri after removing version directory name
-- 
1.9.1

[PATCH 05/12] fetch/wget.php: latest version string only try to find latest directory when REGEX_URI isn't specified to avoid unnecessary processing and makes code easier

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 55 +++++++++++++++++++++++--------------------
 1 file changed, 29 insertions(+), 26 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index b6b1339..85485bf 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -197,7 +197,7 @@ class Wget(FetchMethod):
         bb.debug(3, "DirURL: %s, %s" % (url, versionstring))
         soup = BeautifulSoup(self._fetch_index(url, ud, d))
         if not soup:
-            return ""
+            return None
 
         valid = 0
         prefix = ''
@@ -225,21 +225,21 @@ class Wget(FetchMethod):
             return prefix+version[1]
         else:
             bb.debug(3, "Not Valid")
-            return ""
+            return None
 
-    def _check_latest_version(self, url, packagename, ud, d):
+    def _check_latest_version(self, url, package, ud, d):
         """
         Return the latest version of a package inside a given directory path
         If error or no version, return ""
         """
         valid = 0
-        version = self._parse_path(self.package_regex_comp, packagename)
+        version = self._parse_path(self.package_regex_comp, package)
 
         bb.debug(3, "VersionURL: %s" % (url))
         soup = BeautifulSoup(self._fetch_index(url, ud, d))
         if not soup:
-            bb.debug(3, "*** %s NO SOUP" % (packagename))
-            return ""
+            bb.debug(3, "*** %s NO SOUP" % (package))
+            return None
 
         pn_regex = d.getVar('REGEX', True)
         if pn_regex:
@@ -269,10 +269,12 @@ class Wget(FetchMethod):
             version = ('', '', '')
         if not pn_regex:
             testversion = ('', '', '')
-        bb.debug(3, "*** %s -> %s (TestVersion = %s)" % (packagename, version[1], testversion[1]))
+        bb.debug(3, "*** %s -> %s (TestVersion = %s)" % (package, version[1], testversion[1]))
         if valid and version:
             return re.sub('_', '.', version[1])
 
+        return None
+
     def _init_regexes(self):
         """
         Match as many patterns as possible such as:
@@ -321,36 +323,37 @@ class Wget(FetchMethod):
 
         sanity check to ensure same name and type.
         """
+        package = ud.path.split("/")[-1]
         regex_uri = d.getVar("REGEX_URI", True)
-        newpath = ud.path
+        newpath = regex_uri or ud.path
         pupver = ""
 
         self._init_regexes()
 
-        m = self.dirver_regex_comp.search(ud.path)
-        bb.debug(3, "path = %s" % (ud.path))
-        bb.debug(3, "Regex: %s" % (self.package_regex_comp.pattern))
-        if m and not regex_uri:
-            dirver = m.group('dirver')
-            # generate the new uri after removing version directory name
-            newuri = bb.fetch.encodeurl([ud.type, ud.host, ud.path.split(dirver)[0], ud.user, ud.pswd, {}])
-            newversion = self._check_latest_dir(newuri, dirver, ud, d)
-            if newversion and dirver != newversion:
-                newpath = ud.path.replace(dirver, newversion, True)
-
-        # try to acquire all remote files in current directory
-        packagename = newpath.split("/")[-1]            # current package name
-        newpath = newpath.split(packagename)[0] or "/"  # path to directory
+        if not regex_uri:
+            # generate the new uri with the appropriate latest directory
+            m = self.dirver_regex_comp.search(ud.path)
+            if m:
+                dirver = m.group('dirver')
+                newuri = bb.fetch.encodeurl([ud.type, ud.host,
+                            ud.path.split(dirver)[0], ud.user, ud.pswd, {}])
+                new_dirver = self._check_latest_dir(newuri, dirver, ud, d)
+                if new_dirver and dirver != new_dirver:
+                    newpath = ud.path.replace(dirver, new_dirver, True)
+
+            newpath = newpath.split(package)[0] or "/"  # path to directory
+            newuri = bb.fetch.encodeurl([ud.type, ud.host, newpath, ud.user, ud.pswd, {}])
+        else:
+            newuri = newpath
 
         # generate the new uri with the appropriate latest directory
         newuri = regex_uri or bb.fetch.encodeurl([ud.type, ud.host, newpath, ud.user, ud.pswd, {}])
-        newversion = self._check_latest_version(newuri, packagename, ud, d)
+        newversion = self._check_latest_version(newuri, package, ud, d)
         while not newversion:
             # maybe it's hiding in a download directory so try there
             newuri = "/".join(newuri.split("/")[0:-2]) + "/download"
             if newuri == "/download" or newuri == "http://download":
                 break
-            newversion = self._check_latest_version(newuri, packagename, ud, d)
-
-        return newversion
+            newversion = self._check_latest_version(newuri, package, ud, d)
 
+        return newversion or ""
-- 
1.9.1

[PATCH 06/12] fetch2/wget.py: latest_versionstring add package_custom_regex_comp

[Aníbal Limón]

package_custom_regex_comp is built with the current package name and
then used to search upstream version this reduces custom regex'es in
sites that have different packages in the same directory.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 85485bf..1a585a5 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -233,7 +233,7 @@ class Wget(FetchMethod):
         If error or no version, return ""
         """
         valid = 0
-        version = self._parse_path(self.package_regex_comp, package)
+        version = self._parse_path(self.package_custom_regex_comp, package)
 
         bb.debug(3, "VersionURL: %s" % (url))
         soup = BeautifulSoup(self._fetch_index(url, ud, d))
@@ -258,7 +258,7 @@ class Wget(FetchMethod):
                 else:
                     continue
             else:
-                newver = self._parse_path(self.package_regex_comp, line['href'])
+                newver = self._parse_path(self.package_custom_regex_comp, line['href'])
             valid = 1
             if newver and self._vercmp(version, newver) == True:
                 version = newver
@@ -275,7 +275,7 @@ class Wget(FetchMethod):
 
         return None
 
-    def _init_regexes(self):
+    def _init_regexes(self, package):
         """
         Match as many patterns as possible such as:
                 gnome-common-2.20.0.tar.gz (most common format)
@@ -317,6 +317,13 @@ class Wget(FetchMethod):
         # "5.7" in http://download.gnome.org/sources/${PN}/5.7/${PN}-${PV}.tar.gz
         self.dirver_regex_comp = re.compile("(?P<dirver>[^/]*(\d+\.)*\d+([\-_]r\d+)*)/")
 
+        # get current version and make custom regex for search in uri's
+        version = self._parse_path(self.package_regex_comp, package)
+        if version:
+            self.package_custom_regex_comp = re.compile(
+                "(?P<name>%s)(?P<ver>%s)(?P<arch>%s)?[\.\-](?P<type>%s)$" %
+                (version[0], pver_regex, parch_regex, psuffix_regex))
+
     def latest_versionstring(self, ud, d):
         """
         Manipulate the URL and try to obtain the latest package version
@@ -328,7 +335,7 @@ class Wget(FetchMethod):
         newpath = regex_uri or ud.path
         pupver = ""
 
-        self._init_regexes()
+        self._init_regexes(package)
 
         if not regex_uri:
             # generate the new uri with the appropriate latest directory
-- 
1.9.1

[PATCH 07/12] fetch2/wget.py: latest_versionstring improvments in searching

[Aníbal Limón]

Validate if package contain version string if not return the current
version cases for spectrum-fw and corpus recipes.

_check_latest_version return the latest version available don't
take into account the current version previous this only return
the upstream version if it greater than the current version.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 37 +++++++++++++++++++++++--------------
 1 file changed, 23 insertions(+), 14 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 1a585a5..7e4f432 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -227,18 +227,18 @@ class Wget(FetchMethod):
             bb.debug(3, "Not Valid")
             return None
 
-    def _check_latest_version(self, url, package, ud, d):
+    def _check_latest_version(self, url, package, current_version, ud, d):
         """
         Return the latest version of a package inside a given directory path
         If error or no version, return ""
         """
         valid = 0
-        version = self._parse_path(self.package_custom_regex_comp, package)
+        version = ('', '', '')
 
         bb.debug(3, "VersionURL: %s" % (url))
         soup = BeautifulSoup(self._fetch_index(url, ud, d))
         if not soup:
-            bb.debug(3, "*** %s NO SOUP" % (package))
+            bb.debug(3, "*** %s NO SOUP" % (url))
             return None
 
         pn_regex = d.getVar('REGEX', True)
@@ -248,7 +248,7 @@ class Wget(FetchMethod):
             bb.debug(3, "pn_regex = '%s'" % (pn_regex.pattern))
             
         for line in soup.find_all('a', href=True):
-            newver = ('', '', '')
+            newver = None
             bb.debug(3, "line = '%s'" % (line['href']))
             if pn_regex:
                 m = pn_regex.search(line['href'])
@@ -259,17 +259,19 @@ class Wget(FetchMethod):
                     continue
             else:
                 newver = self._parse_path(self.package_custom_regex_comp, line['href'])
-            valid = 1
-            if newver and self._vercmp(version, newver) == True:
-                version = newver
+
+            if newver:
+                bb.debug(3, "Upstream version found: %s" % newver[1])
+                if valid == 0:
+                    version = newver
+                    valid = 1
+                elif self._vercmp(version, newver) == True:
+                    version = newver
                 
         # check whether a valid package and version were found
+        bb.debug(3, "*** %s -> UpstreamVersion = %s (CurrentVersion = %s)" %
+                (package, version[1] or "N/A", current_version[1]))
 
-        if not valid:
-            version = ('', '', '')
-        if not pn_regex:
-            testversion = ('', '', '')
-        bb.debug(3, "*** %s -> %s (TestVersion = %s)" % (package, version[1], testversion[1]))
         if valid and version:
             return re.sub('_', '.', version[1])
 
@@ -336,6 +338,11 @@ class Wget(FetchMethod):
         pupver = ""
 
         self._init_regexes(package)
+        current_version = ('', d.getVar('PV', True), '')
+
+        """possible to have no version in pkg name, such as spectrum-fw"""
+        if not re.search("\d+", package):
+            return re.sub('_', '.', current_version[1])
 
         if not regex_uri:
             # generate the new uri with the appropriate latest directory
@@ -355,12 +362,14 @@ class Wget(FetchMethod):
 
         # generate the new uri with the appropriate latest directory
         newuri = regex_uri or bb.fetch.encodeurl([ud.type, ud.host, newpath, ud.user, ud.pswd, {}])
-        newversion = self._check_latest_version(newuri, package, ud, d)
+        newversion = self._check_latest_version(newuri, package,
+                        current_version, ud, d)
         while not newversion:
             # maybe it's hiding in a download directory so try there
             newuri = "/".join(newuri.split("/")[0:-2]) + "/download"
             if newuri == "/download" or newuri == "http://download":
                 break
-            newversion = self._check_latest_version(newuri, package, ud, d)
+            newversion = self._check_latest_version(newuri, package,
+                            current_version, ud, d)
 
         return newversion or ""
-- 
1.9.1

[PATCH 08/12] fetch/wget.py: latest_versionstring remove unnecessary usage for name in version comparision

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 7e4f432..20e8df1 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -253,8 +253,8 @@ class Wget(FetchMethod):
             if pn_regex:
                 m = pn_regex.search(line['href'])
                 if m:
-                    bb.debug(3, "Name = '%s', Pver = '%s'" % (m.group('name'), m.group('pver')))
-                    newver = (m.group('name'), m.group('pver'), '')
+                    bb.debug(3, "Pver = '%s'" % (m.group('pver')))
+                    newver = ('', m.group('pver'), '')
                 else:
                     continue
             else:
-- 
1.9.1

[PATCH 09/12] fetch2/wget: latest_versionstring add support for search in RAW html lines

[Aníbal Limón]

Some upstream sites put the name of the package in the body of href tags,
i.e. <a href="#43">somepackage-v1.4.10.tar.gz </a>.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 20e8df1..f5bdfc7 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -256,9 +256,14 @@ class Wget(FetchMethod):
                     bb.debug(3, "Pver = '%s'" % (m.group('pver')))
                     newver = ('', m.group('pver'), '')
                 else:
-                    continue
+                    m = pn_regex.search(str(line))
+                    if m:
+                        bb.debug(3, "Pver = '%s'" % (m.group('pver')))
+                        newver = ('', m.group('pver'), '')
             else:
                 newver = self._parse_path(self.package_custom_regex_comp, line['href'])
+                if not newver:
+                    newver = self._parse_path(self.package_custom_regex_comp, str(line))
 
             if newver:
                 bb.debug(3, "Upstream version found: %s" % newver[1])
-- 
1.9.1

[PATCH 10/12] fetch/wget.py: latest_versionstring remove newuri set because is previous set

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index f5bdfc7..7841553 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -365,8 +365,6 @@ class Wget(FetchMethod):
         else:
             newuri = newpath
 
-        # generate the new uri with the appropriate latest directory
-        newuri = regex_uri or bb.fetch.encodeurl([ud.type, ud.host, newpath, ud.user, ud.pswd, {}])
         newversion = self._check_latest_version(newuri, package,
                         current_version, ud, d)
         while not newversion:
-- 
1.9.1

[PATCH 11/12] tests/fetch.py: Update wget latest_versionstring cups case

[Aníbal Limón]

Update test case for cups is needed because match only 2.0.0
versions see VERSION=2\.0\.0 in the previous string.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/tests/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py
index 9d45743..3f80c4a 100644
--- a/bitbake/lib/bb/tests/fetch.py
+++ b/bitbake/lib/bb/tests/fetch.py
@@ -619,7 +619,7 @@ class FetchMethodTest(FetcherTest):
         ("xserver-xorg", "http://xorg.freedesktop.org/releases/individual/xserver/xorg-server-1.15.1.tar.bz2", "", "")
             : "1.15.1",
         # packages with valid REGEX_URI and REGEX
-        ("cups", "http://www.cups.org/software/1.7.2/cups-1.7.2-source.tar.bz2", "http://www.cups.org/software.php", "/software\.php\?VERSION=2\.0\.0&FILE=2\.0\.0/(?P<name>cups\-)(?P<pver>((\d+[\.\-_]*)+))\-source\.tar\.gz")
+        ("cups", "http://www.cups.org/software/1.7.2/cups-1.7.2-source.tar.bz2", "http://www.cups.org/software.php", "(?P<name>cups\-)(?P<pver>((\d+[\.\-_]*)+))\-source\.tar\.gz")
             : "2.0.0",
         ("db", "http://download.oracle.com/berkeley-db/db-5.3.21.tar.gz", "http://www.oracle.com/technetwork/products/berkeleydb/downloads/index-082944.html", "http://download.oracle.com/otn/berkeley-db/(?P<name>db-)(?P<pver>((\d+[\.\-_]*)+))\.tar\.gz")
             : "6.1.19",
-- 
1.9.1

[PATCH 12/12] fetch2/wget.py: latest_versionstring clean improvments minor

[Aníbal Limón]

Update documentation strings in _check_latest_dir and _check_latest_version
methods with  the correct return types.

_check_latest_version method remove unused testversion variable.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 bitbake/lib/bb/fetch2/wget.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 7841553..d82e78c 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -192,7 +192,7 @@ class Wget(FetchMethod):
     def _check_latest_dir(self, url, versionstring, ud, d):
         """
         Return the name of the directory with the greatest package version
-        If error or no version, return ""
+        If error or no version, return None
         """
         bb.debug(3, "DirURL: %s, %s" % (url, versionstring))
         soup = BeautifulSoup(self._fetch_index(url, ud, d))
@@ -230,7 +230,7 @@ class Wget(FetchMethod):
     def _check_latest_version(self, url, package, current_version, ud, d):
         """
         Return the latest version of a package inside a given directory path
-        If error or no version, return ""
+        If error or no version, return None
         """
         valid = 0
         version = ('', '', '')
@@ -243,7 +243,6 @@ class Wget(FetchMethod):
 
         pn_regex = d.getVar('REGEX', True)
         if pn_regex:
-            testversion = version
             pn_regex = re.compile(pn_regex)
             bb.debug(3, "pn_regex = '%s'" % (pn_regex.pattern))
             
-- 
1.9.1