[Buildroot] [PATCH] pkg-download: handle interrupted wget downloads

[Buildroot] [PATCH] pkg-download: handle interrupted wget downloads

[Arnout Vandecappelle]

When a wget download is interrupted, the downloaded file is still created.
It will therefore not be re-downloaded in the next build, and the
extraction will fail.

To avoid this, download to a temporary file first and rename when the
download is successful.

The existing mechanism doesn't work for interrupted downloads because the
whole sub-shell is interrupted, so the rm-part never gets executed.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
Note that using wget's -c option (continue an interrupted download) is
not a good idea, because it refuses to do anything on servers that don't
support the Range header.

 package/pkg-download.mk |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index 7d1e543..7983706 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -158,11 +158,14 @@ endef
 # Download a file using wget. Only download the file if it doesn't
 # already exist in the download directory. If the download fails,
 # remove the file (because wget -O creates a 0-byte file even if the
-# download fails).
+# download fails).  To handle an interrupted download as well, download
+# to a temporary file first.  The temporary file will be overwritten
+# the next time the download is tried.
 define DOWNLOAD_WGET
 	test -e $(DL_DIR)/$(2) || \
-	$(WGET) -O $(DL_DIR)/$(2) '$(call qstrip,$(1))' || \
-	(rm -f $(DL_DIR)/$(2) ; exit 1)
+	($(WGET) -O $(DL_DIR)/$(2).tmp '$(call qstrip,$(1))' && \
+	 mv $(DL_DIR)/$(2).tmp $(DL_DIR)/$(2)) || \
+	(rm -f $(DL_DIR)/$(2).tmp ; exit 1)
 endef
 
 define SOURCE_CHECK_WGET
-- 
tg: (dc1d20a..) t/wget-fail (depends on: master)

[Buildroot] [PATCH] pkg-download: handle interrupted wget downloads

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> writes:

 Arnout> When a wget download is interrupted, the downloaded file is
 Arnout> still created.  It will therefore not be re-downloaded in the
 Arnout> next build, and the extraction will fail.

 Arnout> To avoid this, download to a temporary file first and rename
 Arnout> when the download is successful.

 Arnout> The existing mechanism doesn't work for interrupted downloads
 Arnout> because the whole sub-shell is interrupted, so the rm-part
 Arnout> never gets executed.

Committed, thanks.

-- 
Bye, Peter Korsgaard