[PATCH 0/1] Allow resuming interrupted fetch

[PATCH 0/1] Allow resuming interrupted fetch

[Paul Eggleton]

This patch fixes an annoyance of mine that I've observed others running
into, where (either due to network issues or deliberate Ctrl+C) a fetch
is interrupted and won't properly resume the next time.

The patch (against Poky, but applies cleanly with -p2 against bitbake
master) is available in the git repository at:
  git://git.yoctoproject.org/poky-contrib paule/fetchresume
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/fetchresume

Paul Eggleton (1):
  bitbake/fetch2: allow resuming interrupted fetch

 bitbake/lib/bb/fetch2/__init__.py |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

-- 
1.7.5.4

[PATCH 1/1] bitbake/fetch2: allow resuming interrupted fetch

[Paul Eggleton]

A lot of our checks in the fetcher code assume that if the downloaded
file exists, the download finished successfully; this meant that if
BitBake was interrupted in the middle of a fetch then it would not
resume the fetch the next time, but instead attempt to use the
half-fetched file and usually fail as a result.

Since we're already writing a ".done" stamp file when a fetch completes,
just check for its existence and assume the download didn't complete
successfully if it isn't present.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/lib/bb/fetch2/__init__.py |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index fe4f4b1..07aac4c 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -479,7 +479,7 @@ def try_mirrors(d, origud, mirrors, check = False):
                     return found
                 continue
 
-            if ud.method.need_update(newuri, ud, ld):
+            if not os.path.exists(ud.donestamp) or ud.method.need_update(newuri, ud, ld):
                 ud.method.download(newuri, ud, ld)
                 if hasattr(ud.method,"build_mirror_data"):
                     ud.method.build_mirror_data(newuri, ud, ld)
@@ -955,7 +955,7 @@ class Fetch(object):
             try:
                 self.d.setVar("BB_NO_NETWORK", network)
  
-                if not m.need_update(u, ud, self.d):
+                if os.path.exists(ud.donestamp) and not m.need_update(u, ud, self.d):
                     localpath = ud.localpath
                 elif m.try_premirror(u, ud, self.d):
                     logger.debug(1, "Trying PREMIRRORS")
@@ -966,7 +966,7 @@ class Fetch(object):
                     self.d.setVar("BB_NO_NETWORK", "1")
 
                 firsterr = None
-                if not localpath and m.need_update(u, ud, self.d):
+                if not localpath and ((not os.path.exists(ud.donestamp)) or m.need_update(u, ud, self.d)):
                     try:
                         logger.debug(1, "Trying Upstream")
                         m.download(u, ud, self.d)
-- 
1.7.5.4

Re: [PATCH 1/1] bitbake/fetch2: allow resuming interrupted fetch

[Richard Purdie]

On Mon, 2012-01-16 at 15:01 +0000, Paul Eggleton wrote:
> A lot of our checks in the fetcher code assume that if the downloaded
> file exists, the download finished successfully; this meant that if
> BitBake was interrupted in the middle of a fetch then it would not
> resume the fetch the next time, but instead attempt to use the
> half-fetched file and usually fail as a result.
> 
> Since we're already writing a ".done" stamp file when a fetch completes,
> just check for its existence and assume the download didn't complete
> successfully if it isn't present.
> 
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> ---
>  bitbake/lib/bb/fetch2/__init__.py |    6 +++---
>  1 files changed, 3 insertions(+), 3 deletions(-)

Merged to master, thanks.

Richard