[Buildroot] [PATCH 0/2 v3] Fix git download helper (branch yem/download-fixes)

[Buildroot] [PATCH 0/2 v3] Fix git download helper (branch yem/download-fixes)

[Yann E. MORIN]

Hello All!

This series fixes the git helper after some fall-overs when the wrapper
was introduced.

Changes v2 -> v3:
  - include initial patch from Peter  (Peter, Rohit)
  - fix the output file name  (Graham)


Regards,
Yann E. MORIN.


The following changes since commit 10b6d10009c70262cfb492b2abaa3a091a8fb4aa:

  espeak: new package (2014-08-04 22:40:45 +0200)

are available in the git repository at:

  git://gitorious.org/buildroot/buildroot.git yem/download-fixes

for you to fetch changes up to 2205cccc9b1580208e604067224448da94599151:

  support/download: fix the git helper output file format (2014-08-06 18:33:33 +0200)

----------------------------------------------------------------
Peter Seiderer (1):
      support/download: fix the git helper

Yann E. MORIN (1):
      support/download: fix the git helper output file format

 support/download/git | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/2 v3] support/download: fix the git helper

[Yann E. MORIN]

From: Peter Seiderer <ps.report@gmx.net>

Re-add the git_done variable (lost in commit [1]).

Fixes download problem reported by Rohit Kumar [2].

[1] http://git.buildroot.net/buildroot/commit/?id=7e40a1103a919a8177f00ddca2b46b4439953511
[2] http://lists.busybox.net/pipermail/buildroot/2014-August/103733.html

Signed-off-by: Peter Seiderer <ps.report@gmx.net>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 support/download/git | 1 +
 1 file changed, 1 insertion(+)

diff --git a/support/download/git b/support/download/git
index 6edaa90..d451530 100755
--- a/support/download/git
+++ b/support/download/git
@@ -19,6 +19,7 @@ basename="${4}"
 
 # Try to see if we can do a shallow clone, since it is faster
 # than a full clone.
+git_done=0
 if [ -n "$(${GIT} ls-remote "${repo}" "${cset}" 2>&1)" ]; then
     printf "Doing shallow clone\n"
     if ${GIT} clone --depth 1 -b "${cset}" --bare "${repo}" "${basename}"; then
-- 
1.9.1

[Buildroot] [PATCH 2/2 v3] support/download: fix the git helper output file format

[Yann E. MORIN]

The git helper uses gzip to compress the intermediate tarball. But gzip
removes the source file, and create a new file named by appending .gz to
the original file name.

Thus, we end up with output.gz, while the download wrapper expects jsut
output, and thus believes the downlaod failed.

Fix that by storing the tar from git to a temporary file, then pipe this
file to gzip's stdin, and redirect gzip's stdout to the output file.

Reported-by: Graham Newton <gnewton@peavey-eu.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Seiderer <ps.report@gmx.net>
---
 support/download/git | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/support/download/git b/support/download/git
index d451530..ff6b5c3 100755
--- a/support/download/git
+++ b/support/download/git
@@ -34,6 +34,6 @@ if [ ${git_done} -eq 0 ]; then
 fi
 
 GIT_DIR="${basename}" \
-${GIT} archive --prefix="${basename}/" -o "${output}" --format=tar "${cset}"
+${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
 
-gzip "${output}"
+gzip <"${output}.tmp" >"${output}"
-- 
1.9.1

[Buildroot] [PATCH 2/2 v3] support/download: fix the git helper output file format

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

 > The git helper uses gzip to compress the intermediate tarball. But gzip
 > removes the source file, and create a new file named by appending .gz to
 > the original file name.

 > Thus, we end up with output.gz, while the download wrapper expects jsut
 > output, and thus believes the downlaod failed.

 > Fix that by storing the tar from git to a temporary file, then pipe this
 > file to gzip's stdin, and redirect gzip's stdout to the output file.

 > Reported-by: Graham Newton <gnewton@peavey-eu.com>
 > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
 > Cc: Peter Seiderer <ps.report@gmx.net>
 > ---
 >  support/download/git | 4 ++--
 >  1 file changed, 2 insertions(+), 2 deletions(-)

 > diff --git a/support/download/git b/support/download/git
 > index d451530..ff6b5c3 100755
 > --- a/support/download/git
 > +++ b/support/download/git
 > @@ -34,6 +34,6 @@ if [ ${git_done} -eq 0 ]; then
 >  fi
 
 >  GIT_DIR="${basename}" \
 > -${GIT} archive --prefix="${basename}/" -o "${output}" --format=tar "${cset}"
 > +${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
 
 > -gzip "${output}"
 > +gzip <"${output}.tmp" >"${output}"

Don't we then end up with a bunch of foo.tmp files?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/2 v3] support/download: fix the git helper output file format

[Yann E. MORIN]

Peter, All,

On 2014-08-15 19:22 +0200, Peter Korsgaard spake thusly:
> >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:
> 
>  > The git helper uses gzip to compress the intermediate tarball. But gzip
>  > removes the source file, and create a new file named by appending .gz to
>  > the original file name.
> 
>  > Thus, we end up with output.gz, while the download wrapper expects jsut
>  > output, and thus believes the downlaod failed.
> 
>  > Fix that by storing the tar from git to a temporary file, then pipe this
>  > file to gzip's stdin, and redirect gzip's stdout to the output file.
> 
>  > Reported-by: Graham Newton <gnewton@peavey-eu.com>
>  > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
>  > Cc: Peter Seiderer <ps.report@gmx.net>
>  > ---
>  >  support/download/git | 4 ++--
>  >  1 file changed, 2 insertions(+), 2 deletions(-)
> 
>  > diff --git a/support/download/git b/support/download/git
>  > index d451530..ff6b5c3 100755
>  > --- a/support/download/git
>  > +++ b/support/download/git
>  > @@ -34,6 +34,6 @@ if [ ${git_done} -eq 0 ]; then
>  >  fi
>  
>  >  GIT_DIR="${basename}" \
>  > -${GIT} archive --prefix="${basename}/" -o "${output}" --format=tar "${cset}"
>  > +${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
>  
>  > -gzip "${output}"
>  > +gzip <"${output}.tmp" >"${output}"
> 
> Don't we then end up with a bunch of foo.tmp files?

Nope, because the wrapper scripts first 'cd' into a temp dir, that ets
removed at the end of the wrapper.

See:
    http://git.buildroot.org/buildroot/tree/support/download/wrapper#n42
and:
    http://git.buildroot.org/buildroot/tree/support/download/wrapper#n91

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/2 v3] support/download: fix the git helper output file format

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

Hi,

 >> > -gzip "${output}"
 >> > +gzip <"${output}.tmp" >"${output}"
 >> 
 >> Don't we then end up with a bunch of foo.tmp files?

 > Nope, because the wrapper scripts first 'cd' into a temp dir, that ets
 > removed at the end of the wrapper.

 > See:
 >     http://git.buildroot.org/buildroot/tree/support/download/wrapper#n42
 > and:
 >     http://git.buildroot.org/buildroot/tree/support/download/wrapper#n91

Ahh yes, good - Thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 0/2 v3] Fix git download helper (branch yem/download-fixes)

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Wed,  6 Aug 2014 18:47:48 +0200, Yann E. MORIN wrote:

> Peter Seiderer (1):
>       support/download: fix the git helper
> 
> Yann E. MORIN (1):
>       support/download: fix the git helper output file format

Thanks, both patches have been applied.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com