[PATCH 0/3] A few fetch bugfixes

[PATCH 0/3] A few fetch bugfixes

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

These were discovered and fixed as a part of the git shallow mirror tarball
support implementation.

Christopher Larson (3):
  bb.fetch.git: remove leading '.' from gitsrcname
  bb.fetch: handle checksums consistently for mirrors
  bb.fetch: don't remove the clone when an update fails

 lib/bb/fetch2/__init__.py | 13 ++++++++-----
 lib/bb/fetch2/git.py      |  5 ++++-
 2 files changed, 12 insertions(+), 6 deletions(-)

-- 
2.2.1

[PATCH 1/3] bb.fetch.git: remove leading '.' from gitsrcname

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

When using an absolute file URI, there's no host, and the path starts with
'/', the dir under ${DL_DIR}/git2/ ends up starting with '.', so is hidden.
Remove any leading '.' to fix this.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 lib/bb/fetch2/git.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 374d846..40658ff 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -137,7 +137,10 @@ class Git(FetchMethod):
                     ud.unresolvedrev[name] = ud.revisions[name]
                 ud.revisions[name] = self.latest_revision(ud, d, name)
 
-        gitsrcname = '%s%s' % (ud.host.replace(':','.'), ud.path.replace('/', '.').replace('*', '.'))
+        gitsrcname = '%s%s' % (ud.host.replace(':', '.'), ud.path.replace('/', '.').replace('*', '.'))
+        if gitsrcname.startswith('.'):
+            gitsrcname = gitsrcname[1:]
+
         # for rebaseable git repo, it is necessary to keep mirror tar ball
         # per revision, so that even the revision disappears from the
         # upstream repo in the future, the mirror will remain intact and still
-- 
2.2.1

[PATCH 2/3] bb.fetch: handle checksums consistently for mirrors

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

If the main fetch method doesn't support checksums, the user will not be
defining them in the recipe, so we don't want to check them for
premirrors/mirrors either. This ensures that we never error due to missing
checksums on a git mirror tarball.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 lib/bb/fetch2/__init__.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index ec0c31a..e229c30 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -628,7 +628,7 @@ def verify_checksum(ud, d, precomputed={}):
     }
 
 
-def verify_donestamp(ud, d):
+def verify_donestamp(ud, d, origud=None):
     """
     Check whether the done stamp file has the right checksums (if the fetch
     method supports them). If it doesn't, delete the done stamp and force
@@ -640,7 +640,8 @@ def verify_donestamp(ud, d):
     if not os.path.exists(ud.donestamp):
         return False
 
-    if not ud.method.supports_checksum(ud):
+    if (not ud.method.supports_checksum(ud) or
+        (origud and not origud.method.supports_checksum(origud))):
         # done stamp exists, checksums not supported; assume the local file is
         # current
         return True
@@ -922,7 +923,7 @@ def try_mirror_url(fetch, origud, ud, ld, check = False):
 
         os.chdir(ld.getVar("DL_DIR", True))
 
-        if not verify_donestamp(ud, ld) or ud.method.need_update(ud, ld):
+        if not verify_donestamp(ud, ld, origud) or ud.method.need_update(ud, ld):
             ud.method.download(ud, ld)
             if hasattr(ud.method,"build_mirror_data"):
                 ud.method.build_mirror_data(ud, ld)
-- 
2.2.1

[PATCH 3/3] bb.fetch: don't remove the clone when an update fails

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

When our clone exists, but is out of date, and the attempt to update it fails,
we don't necessarily want to remove the entire clone, particularly if it's
a large repository.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 lib/bb/fetch2/__init__.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index e229c30..3d53b63 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -1590,7 +1590,8 @@ class Fetch(object):
                 os.chdir(self.d.getVar("DL_DIR", True))
 
                 firsterr = None
-                if not localpath and ((not verify_donestamp(ud, self.d)) or m.need_update(ud, self.d)):
+                verified_stamp = verify_donestamp(ud, self.d)
+                if not localpath and (not verified_stamp or m.need_update(ud, self.d)):
                     try:
                         if not trusted_network(self.d, ud.url):
                             raise UntrustedUrl(ud.url)
@@ -1618,7 +1619,8 @@ class Fetch(object):
                             logger.debug(1, str(e))
                         firsterr = e
                         # Remove any incomplete fetch
-                        m.clean(ud, self.d)
+                        if not verified_stamp:
+                            m.clean(ud, self.d)
                         logger.debug(1, "Trying MIRRORS")
                         mirrors = mirror_from_string(self.d.getVar('MIRRORS', True))
                         localpath = try_mirrors(self, self.d, ud, mirrors)
-- 
2.2.1