[Buildroot] [PATCH 0/8 v4] toolchain: better handle hashes (branch yem/dl-hash-toolchains)

[Buildroot] [PATCH 0/8 v4] toolchain: better handle hashes (branch yem/dl-hash-toolchains)

[Yann E. MORIN]

Hello All!

This series makes it so we can download custom external toolchains, and
still have mandatory checks of hashes.

It also adds the long-awaited-for hashes for gcc and binutils.


Changes v3 -> v4:
  - add missing fake-hash for uClibc eXtensa git snapshot; add uClibc
    daily snapshots to exclusion list  (Arnout)
  - drop using a magic value; use a list of excluded files  (Arnout)

Changes v2 -> v3:
  - add hashes for gcc and binutils
  - fix a few more typoes

Changes v1 -> v2:
  - fix a few typoes


Regards,
Yann E. MORIN.


The following changes since commit 2552729232b4d9c5cb63ad247d6d94ab83e48e49:

  package/mplayer: add optional libenca support (2015-04-07 23:27:23 +0200)

are available in the git repository at:

  git://git.busybox.net/~ymorin/git/buildroot yem/dl-hash-toolchains

for you to fetch changes up to ee02447feb64c265dcf5e7a60dddb630c7dc9402:

  support/download: do not accept exlcuded files with ashes (2015-04-08 18:54:13 +0200)

----------------------------------------------------------------
Yann E. MORIN (8):
      package/gcc: add hashes
      package/binutils: add hashes
      package/uclibc: add missing hash for eXtensa
      support/download: add possibility to not fail on missing hash
      toolchain/external: ignore missing hash for custom downloaded toolchain
      package/uclibc: ignore missing hash for snapshots
      support/download: restore mandatory check of hashes
      support/download: do not accept exlcuded files with ashes

 package/binutils/binutils.hash                     |  7 +++++++
 package/gcc/gcc-final/gcc-final.hash               |  1 +
 package/gcc/gcc-initial/gcc-initial.hash           |  1 +
 package/gcc/gcc.hash                               |  7 +++++++
 package/pkg-download.mk                            |  3 +++
 package/uclibc/uclibc.hash                         |  3 +++
 package/uclibc/uclibc.mk                           |  1 +
 support/download/check-hash                        | 17 ++++++++++++++++-
 toolchain/toolchain-external/toolchain-external.mk |  2 ++
 9 files changed, 41 insertions(+), 1 deletion(-)
 create mode 100644 package/binutils/binutils.hash
 create mode 120000 package/gcc/gcc-final/gcc-final.hash
 create mode 120000 package/gcc/gcc-initial/gcc-initial.hash
 create mode 100644 package/gcc/gcc.hash

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/8 v4] package/gcc: add hashes

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 package/gcc/gcc-final/gcc-final.hash     | 1 +
 package/gcc/gcc-initial/gcc-initial.hash | 1 +
 package/gcc/gcc.hash                     | 7 +++++++
 3 files changed, 9 insertions(+)
 create mode 120000 package/gcc/gcc-final/gcc-final.hash
 create mode 120000 package/gcc/gcc-initial/gcc-initial.hash
 create mode 100644 package/gcc/gcc.hash

diff --git a/package/gcc/gcc-final/gcc-final.hash b/package/gcc/gcc-final/gcc-final.hash
new file mode 120000
index 0000000..7ac9361
--- /dev/null
+++ b/package/gcc/gcc-final/gcc-final.hash
@@ -0,0 +1 @@
+../gcc.hash
\ No newline at end of file
diff --git a/package/gcc/gcc-initial/gcc-initial.hash b/package/gcc/gcc-initial/gcc-initial.hash
new file mode 120000
index 0000000..7ac9361
--- /dev/null
+++ b/package/gcc/gcc-initial/gcc-initial.hash
@@ -0,0 +1 @@
+../gcc.hash
\ No newline at end of file
diff --git a/package/gcc/gcc.hash b/package/gcc/gcc.hash
new file mode 100644
index 0000000..d9a04fc
--- /dev/null
+++ b/package/gcc/gcc.hash
@@ -0,0 +1,7 @@
+# Locally computed (upstream only has sigs):
+sha256  eef3f0456db8c3d992cbb51d5d32558190bc14f3bc19383dd93acc27acc6befc  gcc-4.5.4.tar.bz2
+sha256  92e61c6dc3a0a449e62d72a38185fda550168a86702dea07125ebd3ec3996282  gcc-4.7.4.tar.bz2
+sha256  4a80aa23798b8e9b5793494b8c976b39b8d9aa2e53cd5ed5534aff662a7f8695  gcc-4.8.4.tar.bz2
+sha256  2020c98295856aa13fda0f2f3a4794490757fc24bcca918d52cc8b4917b972dd  gcc-4.9.2.tar.bz2
+# No hash for the ARC variant, comes from the github-helper:
+none    xxx                                                               gcc-arc-2014.12.tar.gz
-- 
1.9.1

[Buildroot] [PATCH 1/8 v4] package/gcc: add hashes

[Arnout Vandecappelle]

On 08/04/15 18:57, Yann E. MORIN wrote:
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  package/gcc/gcc-final/gcc-final.hash     | 1 +
>  package/gcc/gcc-initial/gcc-initial.hash | 1 +
>  package/gcc/gcc.hash                     | 7 +++++++
>  3 files changed, 9 insertions(+)
>  create mode 120000 package/gcc/gcc-final/gcc-final.hash
>  create mode 120000 package/gcc/gcc-initial/gcc-initial.hash
>  create mode 100644 package/gcc/gcc.hash
> 
> diff --git a/package/gcc/gcc-final/gcc-final.hash b/package/gcc/gcc-final/gcc-final.hash
> new file mode 120000
> index 0000000..7ac9361
> --- /dev/null
> +++ b/package/gcc/gcc-final/gcc-final.hash
> @@ -0,0 +1 @@
> +../gcc.hash
> \ No newline at end of file
> diff --git a/package/gcc/gcc-initial/gcc-initial.hash b/package/gcc/gcc-initial/gcc-initial.hash
> new file mode 120000
> index 0000000..7ac9361
> --- /dev/null
> +++ b/package/gcc/gcc-initial/gcc-initial.hash
> @@ -0,0 +1 @@
> +../gcc.hash
> \ No newline at end of file
> diff --git a/package/gcc/gcc.hash b/package/gcc/gcc.hash
> new file mode 100644
> index 0000000..d9a04fc
> --- /dev/null
> +++ b/package/gcc/gcc.hash
> @@ -0,0 +1,7 @@
> +# Locally computed (upstream only has sigs):

 Which upstream is that?

# From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.5.4/sha512.sum
sha512	78696b287d46aacd6f150920da376ea32f58ad9f0dafd2d3b7fa6dbdd8dd7afe659108d1dfaa5807617fc5248719f50b39c37ade0173ea5b9fec2091f4656115	gcc-4.5.4.tar.bz2
# From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.7.4/sha512.sum
sha512	dfcb737073191e628231031a3571ec77ee760a59377630f4a6e4fdfa66f9ddad39fde47e3f0f227eb43cdf90e0d34cde5abdc9ac892c1e111a911062a66c9189	gcc-4.7.4.tar.bz2
# From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.8.4/sha512.sum
sha512	b4c84ebd754c026029d706ef52c4242df4dcadc6a22cc5669b3aa37ac9dc25bc9cd9651fda83fb24dd5d17b1710e6865a35bb6c5cd9b95d0971717453935ae8e	gcc-4.8.4.tar.bz2
# From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.9.2/sha512.sum
sha512	e96cba06d572dbe7f382a2b00fd9297dcf9f9219d46a9ad0bd904dc36d5e7db3335e422640b79a27fed6fde91b0731732b94a0e2fbcd47344d6779f8ba1f4f9c	gcc-4.9.2.tar.bz2


 Regards,
 Arnout

> +sha256  eef3f0456db8c3d992cbb51d5d32558190bc14f3bc19383dd93acc27acc6befc  gcc-4.5.4.tar.bz2
> +sha256  92e61c6dc3a0a449e62d72a38185fda550168a86702dea07125ebd3ec3996282  gcc-4.7.4.tar.bz2
> +sha256  4a80aa23798b8e9b5793494b8c976b39b8d9aa2e53cd5ed5534aff662a7f8695  gcc-4.8.4.tar.bz2
> +sha256  2020c98295856aa13fda0f2f3a4794490757fc24bcca918d52cc8b4917b972dd  gcc-4.9.2.tar.bz2
> +# No hash for the ARC variant, comes from the github-helper:
> +none    xxx                                                               gcc-arc-2014.12.tar.gz
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 1/8 v4] package/gcc: add hashes

[Yann E. MORIN]

Arnout, All,

On 2015-04-14 01:32 +0200, Arnout Vandecappelle spake thusly:
> On 08/04/15 18:57, Yann E. MORIN wrote:
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> > Cc: Arnout Vandecappelle <arnout@mind.be>
> > ---
> >  package/gcc/gcc-final/gcc-final.hash     | 1 +
> >  package/gcc/gcc-initial/gcc-initial.hash | 1 +
> >  package/gcc/gcc.hash                     | 7 +++++++
> >  3 files changed, 9 insertions(+)
> >  create mode 120000 package/gcc/gcc-final/gcc-final.hash
> >  create mode 120000 package/gcc/gcc-initial/gcc-initial.hash
> >  create mode 100644 package/gcc/gcc.hash
> > 
> > diff --git a/package/gcc/gcc-final/gcc-final.hash b/package/gcc/gcc-final/gcc-final.hash
> > new file mode 120000
> > index 0000000..7ac9361
> > --- /dev/null
> > +++ b/package/gcc/gcc-final/gcc-final.hash
> > @@ -0,0 +1 @@
> > +../gcc.hash
> > \ No newline at end of file
> > diff --git a/package/gcc/gcc-initial/gcc-initial.hash b/package/gcc/gcc-initial/gcc-initial.hash
> > new file mode 120000
> > index 0000000..7ac9361
> > --- /dev/null
> > +++ b/package/gcc/gcc-initial/gcc-initial.hash
> > @@ -0,0 +1 @@
> > +../gcc.hash
> > \ No newline at end of file
> > diff --git a/package/gcc/gcc.hash b/package/gcc/gcc.hash
> > new file mode 100644
> > index 0000000..d9a04fc
> > --- /dev/null
> > +++ b/package/gcc/gcc.hash
> > @@ -0,0 +1,7 @@
> > +# Locally computed (upstream only has sigs):
> 
>  Which upstream is that?

For gcc-4.9.2, I went there: https://ftp.gnu.org/pub/gnu/gcc/gcc-4.9.2/

Similarly for other versions.

> # From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.5.4/sha512.sum
> sha512	78696b287d46aacd6f150920da376ea32f58ad9f0dafd2d3b7fa6dbdd8dd7afe659108d1dfaa5807617fc5248719f50b39c37ade0173ea5b9fec2091f4656115	gcc-4.5.4.tar.bz2
> # From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.7.4/sha512.sum
> sha512	dfcb737073191e628231031a3571ec77ee760a59377630f4a6e4fdfa66f9ddad39fde47e3f0f227eb43cdf90e0d34cde5abdc9ac892c1e111a911062a66c9189	gcc-4.7.4.tar.bz2
> # From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.8.4/sha512.sum
> sha512	b4c84ebd754c026029d706ef52c4242df4dcadc6a22cc5669b3aa37ac9dc25bc9cd9651fda83fb24dd5d17b1710e6865a35bb6c5cd9b95d0971717453935ae8e	gcc-4.8.4.tar.bz2
> # From ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.9.2/sha512.sum
> sha512	e96cba06d572dbe7f382a2b00fd9297dcf9f9219d46a9ad0bd904dc36d5e7db3335e422640b79a27fed6fde91b0731732b94a0e2fbcd47344d6779f8ba1f4f9c	gcc-4.9.2.tar.bz2

OK, I'll use the gcc.gnu.org as the source for hashes.

Ditto for binutils (your comment on my next patch).

Thanks! :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/8 v4] package/binutils: add hashes

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 package/binutils/binutils.hash | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 package/binutils/binutils.hash

diff --git a/package/binutils/binutils.hash b/package/binutils/binutils.hash
new file mode 100644
index 0000000..4f99514
--- /dev/null
+++ b/package/binutils/binutils.hash
@@ -0,0 +1,7 @@
+# Locally computed:
+sha256 6c7af8ed1c8cf9b4b9d6e6fe09a3e1d3d479fe63984ba8b9b26bf356b6313ca9  binutils-2.22.tar.bz2
+sha256 fe914e56fed7a9ec2eb45274b1f2e14b0d8b4f41906a5194eac6883cfe5c1097  binutils-2.23.2.tar.bz2
+sha256 e5e8c5be9664e7f7f96e0d09919110ab5ad597794f5b1809871177a0f0f14137  binutils-2.24.tar.bz2
+sha256 22defc65cfa3ef2a3395faaea75d6331c6e62ea5dfacfed3e2ec17b08c882923  binutils-2.25.tar.bz2
+# No hash for the ARC variant, comes from the github-helper:
+none   xxx                                                               binutils-arc-2014.12.tar.gz
-- 
1.9.1

[Buildroot] [PATCH 2/8 v4] package/binutils: add hashes

[Arnout Vandecappelle]

On 08/04/15 18:57, Yann E. MORIN wrote:
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  package/binutils/binutils.hash | 7 +++++++
>  1 file changed, 7 insertions(+)
>  create mode 100644 package/binutils/binutils.hash
> 
> diff --git a/package/binutils/binutils.hash b/package/binutils/binutils.hash
> new file mode 100644
> index 0000000..4f99514
> --- /dev/null
> +++ b/package/binutils/binutils.hash
> @@ -0,0 +1,7 @@
> +# Locally computed:
> +sha256 6c7af8ed1c8cf9b4b9d6e6fe09a3e1d3d479fe63984ba8b9b26bf356b6313ca9  binutils-2.22.tar.bz2
> +sha256 fe914e56fed7a9ec2eb45274b1f2e14b0d8b4f41906a5194eac6883cfe5c1097  binutils-2.23.2.tar.bz2
> +sha256 e5e8c5be9664e7f7f96e0d09919110ab5ad597794f5b1809871177a0f0f14137  binutils-2.24.tar.bz2
> +sha256 22defc65cfa3ef2a3395faaea75d6331c6e62ea5dfacfed3e2ec17b08c882923  binutils-2.25.tar.bz2
> +# No hash for the ARC variant, comes from the github-helper:
> +none   xxx                                                               binutils-arc-2014.12.tar.gz
> 

# From ftp://gcc.gnu.org/pub/binutils/releases/sha512.sum
sha512	ffe8ef263ef99183e8cc823fe8487ff7d0f7bf9a8efd2853b5f4636aca0023850d13de4eac7d77a5f69413d8a50e6f95bb14569be53df86c0bce38034525ab74	binutils-2.22.tar.bz2
sha512	dec753bbba008f1526b89cf1bd85feba78f362f5333ffdf93953fd131eb755976dec82a0a4ba38c43d2434da007137780cfe674de5414be5cf7ce7fbc6af6d16	binutils-2.23.2.tar.bz2
sha512	5ec95ad47d49b12c4558a8db0ca2109d3ee1955e3776057f3330c4506f8f4d1cf5e505fbf8a16b98403a0fcdeaaf986fe0a22be6456247dbdace63ce1f776b12	binutils-2.24.tar.bz2
sha512	49438970642f379978aba988e3d2e7b63dab6fd0b46d91f4100f083e87a84d242089daca58572245fe620457a77bbbe2d78588ff3fb79314e500d1a39bf96f30	binutils-2.25.tar.bz2

(Just cut&paste from the ftp site, not tested.)

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 3/8 v4] package/uclibc: add missing hash for eXtensa

[Yann E. MORIN]

This is a git snapshot, we can't have a hash for it.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 package/uclibc/uclibc.hash | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/package/uclibc/uclibc.hash b/package/uclibc/uclibc.hash
index 69cc103..971ab26 100644
--- a/package/uclibc/uclibc.hash
+++ b/package/uclibc/uclibc.hash
@@ -6,3 +6,6 @@ sha256	0a9085a11615296cfe7783f16ff29073393eb330749947adece0eeddb31705a4	uClibc-a
 
 # From http://www.uclibc-ng.org/trac/
 sha256  e561040293ba4f21dcf3285ca1f683b5a8493eb0ffd3b3e495122cfde5a64a0e        uClibc-ng-1.0.1.tar.xz
+
+# No hash for this git snapshot
+none  xxx  uClibc-7bf35c8b7d4a1f97174eb49f47f33946b282114c.tar.gz
-- 
1.9.1

[Buildroot] [PATCH 3/8 v4] package/uclibc: add missing hash for eXtensa

[Arnout Vandecappelle]

On 08/04/15 18:57, Yann E. MORIN wrote:
> This is a git snapshot, we can't have a hash for it.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Regards,
 Arnout

> ---
>  package/uclibc/uclibc.hash | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/package/uclibc/uclibc.hash b/package/uclibc/uclibc.hash
> index 69cc103..971ab26 100644
> --- a/package/uclibc/uclibc.hash
> +++ b/package/uclibc/uclibc.hash
> @@ -6,3 +6,6 @@ sha256	0a9085a11615296cfe7783f16ff29073393eb330749947adece0eeddb31705a4	uClibc-a
>  
>  # From http://www.uclibc-ng.org/trac/
>  sha256  e561040293ba4f21dcf3285ca1f683b5a8493eb0ffd3b3e495122cfde5a64a0e        uClibc-ng-1.0.1.tar.xz
> +
> +# No hash for this git snapshot
> +none  xxx  uClibc-7bf35c8b7d4a1f97174eb49f47f33946b282114c.tar.gz
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 4/8 v4] support/download: add possibility to not fail on missing hash

[Yann E. MORIN]

In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we oviously can not have hashes for all existing cutom toolchains
(he, "custom"!).

So, add a way to avoid failing in that case.

Form the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>

---
Changes v3 -> v4:
  - drop the magic value, use a list of excluded files  (Arnout)

Changes v1 -> v2:
  - fix typoes in commit log
---
 package/pkg-download.mk     | 3 +++
 support/download/check-hash | 5 +++++
 2 files changed, 8 insertions(+)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index e274712..5b7c861 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -60,6 +60,9 @@ domainseparator = $(if $(1),$(1),/)
 # github(user,package,version): returns site of GitHub repository
 github = https://github.com/$(1)/$(2)/archive/$(3)
 
+# Expressely do not check hashes for those files
+export BR_NO_CHECK_HASH_FOR
+
 ################################################################################
 # The DOWNLOAD_* helpers are in charge of getting a working copy
 # of the source repository for their corresponding SCM,
diff --git a/support/download/check-hash b/support/download/check-hash
index d37f1cd..ca81f00 100755
--- a/support/download/check-hash
+++ b/support/download/check-hash
@@ -99,6 +99,11 @@ while read t h f; do
 done <"${h_file}"
 
 if [ ${nb_checks} -eq 0 ]; then
+    for f in ${BR_NO_CHECK_HASH_FOR}; do
+        if [ "${f}" = "${base}" ]; then
+            exit 0
+        fi
+    done
     printf "ERROR: No hash found for %s\n" "${base}" >&2
     exit 0
 fi
-- 
1.9.1

[Buildroot] [PATCH 4/8 v4] support/download: add possibility to not fail on missing hash

[Arnout Vandecappelle]

On 08/04/15 18:57, Yann E. MORIN wrote:
> In very constrained cases, it might be needed to not fail if a hash is
> missing. This is notably the case for custom external toolchains to be
> downloaded, because we do have a .hash file for external toolchains,
> but we oviously can not have hashes for all existing cutom toolchains
         obviously                                     custom

> (he, "custom"!).

 So what is it going to be cutom or custom? :-)

> 
> So, add a way to avoid failing in that case.
> 
> Form the Makefile, we export the list of files for which not to check
  From

> the hash. Then, from the check-hash script, if no check was done, and
> the file we were trying to match in in this exclusion list, we just exit
                                   is

> without error.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

but more fixes below...

> 
> ---
> Changes v3 -> v4:
>   - drop the magic value, use a list of excluded files  (Arnout)
> 
> Changes v1 -> v2:
>   - fix typoes in commit log
> ---
>  package/pkg-download.mk     | 3 +++
>  support/download/check-hash | 5 +++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/package/pkg-download.mk b/package/pkg-download.mk
> index e274712..5b7c861 100644
> --- a/package/pkg-download.mk
> +++ b/package/pkg-download.mk
> @@ -60,6 +60,9 @@ domainseparator = $(if $(1),$(1),/)
>  # github(user,package,version): returns site of GitHub repository
>  github = https://github.com/$(1)/$(2)/archive/$(3)
>  
> +# Expressely do not check hashes for those files
     Expressly

> +export BR_NO_CHECK_HASH_FOR
> +
>  ################################################################################
>  # The DOWNLOAD_* helpers are in charge of getting a working copy
>  # of the source repository for their corresponding SCM,
> diff --git a/support/download/check-hash b/support/download/check-hash
> index d37f1cd..ca81f00 100755
> --- a/support/download/check-hash
> +++ b/support/download/check-hash
> @@ -99,6 +99,11 @@ while read t h f; do
>  done <"${h_file}"
>  
>  if [ ${nb_checks} -eq 0 ]; then
> +    for f in ${BR_NO_CHECK_HASH_FOR}; do
> +        if [ "${f}" = "${base}" ]; then
> +            exit 0
> +        fi
> +    done

 I can't stop myself from micro-optimising :-)

if [ ${nb_checks} -eq 0 ]; then
	case " ${BR_NO_CHECK_HASH_FOR} " in
	"* ${base} *")
		exit 0
		;;
	esac
	printf "ERROR: No hash found for %s\n" "${base}" >&2
	exit 0
fi

But maybe that's not really more readable...

 Regards,
 Arnout

>      printf "ERROR: No hash found for %s\n" "${base}" >&2
>      exit 0
>  fi
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 4/8 v4] support/download: add possibility to not fail on missing hash

[Yann E. MORIN]

Arnout, All,

On 2015-04-14 01:43 +0200, Arnout Vandecappelle spake thusly:
> On 08/04/15 18:57, Yann E. MORIN wrote:
> > In very constrained cases, it might be needed to not fail if a hash is
> > missing. This is notably the case for custom external toolchains to be
> > downloaded, because we do have a .hash file for external toolchains,
> > but we oviously can not have hashes for all existing cutom toolchains
>          obviously                                     custom
> 
> > (he, "custom"!).
> 
>  So what is it going to be cutom or custom? :-)

I pondered using 'coton', but that had no meaning. So I've decided for
'custom' instead. ;-)

All typoes fixed. Well, at least the ones you pointed out! ;-)

[--SNIP--]
> > diff --git a/support/download/check-hash b/support/download/check-hash
> > index d37f1cd..ca81f00 100755
> > --- a/support/download/check-hash
> > +++ b/support/download/check-hash
> > @@ -99,6 +99,11 @@ while read t h f; do
> >  done <"${h_file}"
> >  
> >  if [ ${nb_checks} -eq 0 ]; then
> > +    for f in ${BR_NO_CHECK_HASH_FOR}; do
> > +        if [ "${f}" = "${base}" ]; then
> > +            exit 0
> > +        fi
> > +    done
> 
>  I can't stop myself from micro-optimising :-)
> 
> if [ ${nb_checks} -eq 0 ]; then
> 	case " ${BR_NO_CHECK_HASH_FOR} " in
> 	"* ${base} *")
> 		exit 0
> 		;;
> 	esac
> 	printf "ERROR: No hash found for %s\n" "${base}" >&2
> 	exit 0
> fi
> 
> But maybe that's not really more readable...

Yeah, I can read that no problem. I don't care which to use, so I'll
switch over to using case.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 5/8 v4] toolchain/external: ignore missing hash for custom downloaded toolchain

[Yann E. MORIN]

We will *always* be missing a hash file for custom external toolchains
that are downloaded.

So, just ignore that failure.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>

---
Chamges v3 -> v4:
  - drop magic value, append to the exclusion list  (Arnout)

Changes v1 -> v2:
  - fix typoes in title
---
 toolchain/toolchain-external/toolchain-external.mk | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index c0429bb..09b16d1 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -400,6 +400,8 @@ else
 # Custom toolchain
 TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
 TOOLCHAIN_EXTERNAL_SOURCE = $(notdir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
+# We can't check hashes for custom downloaded toolchains
+BR_NO_CHECK_HASH_FOR += $(TOOLCHAIN_EXTERNAL_SOURCE)
 endif
 
 # In fact, we don't need to download the toolchain, since it is already
-- 
1.9.1

[Buildroot] [PATCH 5/8 v4] toolchain/external: ignore missing hash for custom downloaded toolchain

[Arnout Vandecappelle]

On 08/04/15 18:57, Yann E. MORIN wrote:
> We will *always* be missing a hash file for custom external toolchains
> that are downloaded.
> 
> So, just ignore that failure.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Regards,
 Arnout

> 
> ---
> Chamges v3 -> v4:
>   - drop magic value, append to the exclusion list  (Arnout)
> 
> Changes v1 -> v2:
>   - fix typoes in title
> ---
>  toolchain/toolchain-external/toolchain-external.mk | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
> index c0429bb..09b16d1 100644
> --- a/toolchain/toolchain-external/toolchain-external.mk
> +++ b/toolchain/toolchain-external/toolchain-external.mk
> @@ -400,6 +400,8 @@ else
>  # Custom toolchain
>  TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
>  TOOLCHAIN_EXTERNAL_SOURCE = $(notdir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
> +# We can't check hashes for custom downloaded toolchains
> +BR_NO_CHECK_HASH_FOR += $(TOOLCHAIN_EXTERNAL_SOURCE)
>  endif
>  
>  # In fact, we don't need to download the toolchain, since it is already
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 6/8 v4] package/uclibc: ignore missing hash for snapshots

[Yann E. MORIN]

We can't have a hash for snapshots, so add them to the exclusion list.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 package/uclibc/uclibc.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/uclibc/uclibc.mk b/package/uclibc/uclibc.mk
index 72a46d9..1ec8b10 100644
--- a/package/uclibc/uclibc.mk
+++ b/package/uclibc/uclibc.mk
@@ -11,6 +11,7 @@ UCLIBC_LICENSE_FILES = COPYING.LIB
 
 ifeq ($(BR2_UCLIBC_VERSION_SNAPSHOT),y)
 UCLIBC_SITE = http://www.uclibc.org/downloads/snapshots
+BR_NO_CHECK_HASH_FOR += uClibc-$(UCLIBC_VERSION).tar.xz
 else ifeq ($(BR2_UCLIBC_VERSION_NG),y)
 UCLIBC_SITE = http://downloads.uclibc-ng.org/releases/$(UCLIBC_VERSION)
 UCLIBC_SOURCE = uClibc-ng-$(UCLIBC_VERSION).tar.xz
-- 
1.9.1

[Buildroot] [PATCH 6/8 v4] package/uclibc: ignore missing hash for snapshots

[Arnout Vandecappelle]

On 08/04/15 18:57, Yann E. MORIN wrote:
> We can't have a hash for snapshots, so add them to the exclusion list.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Regards,
 Arnout

> ---
>  package/uclibc/uclibc.mk | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/package/uclibc/uclibc.mk b/package/uclibc/uclibc.mk
> index 72a46d9..1ec8b10 100644
> --- a/package/uclibc/uclibc.mk
> +++ b/package/uclibc/uclibc.mk
> @@ -11,6 +11,7 @@ UCLIBC_LICENSE_FILES = COPYING.LIB
>  
>  ifeq ($(BR2_UCLIBC_VERSION_SNAPSHOT),y)
>  UCLIBC_SITE = http://www.uclibc.org/downloads/snapshots
> +BR_NO_CHECK_HASH_FOR += uClibc-$(UCLIBC_VERSION).tar.xz
>  else ifeq ($(BR2_UCLIBC_VERSION_NG),y)
>  UCLIBC_SITE = http://downloads.uclibc-ng.org/releases/$(UCLIBC_VERSION)
>  UCLIBC_SOURCE = uClibc-ng-$(UCLIBC_VERSION).tar.xz
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 7/8 v4] support/download: restore mandatory check of hashes

[Yann E. MORIN]

Now that custom external toolchains to be downloaded proprely instruct
to not fail on a missing hash, restore the mandatory hash check for
everything else.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 support/download/check-hash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/support/download/check-hash b/support/download/check-hash
index ca81f00..96f5965 100755
--- a/support/download/check-hash
+++ b/support/download/check-hash
@@ -105,5 +105,5 @@ if [ ${nb_checks} -eq 0 ]; then
         fi
     done
     printf "ERROR: No hash found for %s\n" "${base}" >&2
-    exit 0
+    exit 3
 fi
-- 
1.9.1

[Buildroot] [PATCH 7/8 v4] support/download: restore mandatory check of hashes

[Arnout Vandecappelle]

On 08/04/15 18:57, Yann E. MORIN wrote:
> Now that custom external toolchains to be downloaded proprely instruct
                                                       properly

> to not fail on a missing hash, restore the mandatory hash check for
> everything else.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>


 Regards,
 Arnout

> ---
>  support/download/check-hash | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/support/download/check-hash b/support/download/check-hash
> index ca81f00..96f5965 100755
> --- a/support/download/check-hash
> +++ b/support/download/check-hash
> @@ -105,5 +105,5 @@ if [ ${nb_checks} -eq 0 ]; then
>          fi
>      done
>      printf "ERROR: No hash found for %s\n" "${base}" >&2
> -    exit 0
> +    exit 3
>  fi
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 8/8 v4] support/download: do not accept exlcuded files with ashes

[Yann E. MORIN]

If a file is excluded from checking its hashes, it must effectively have
no hash for it.

So, if an excluded file indeed has at least one hash, there is an issue
somewhere; that file should probably not be excluded.

When this happens, bail out in error.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 support/download/check-hash | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/support/download/check-hash b/support/download/check-hash
index 96f5965..849637e 100755
--- a/support/download/check-hash
+++ b/support/download/check-hash
@@ -18,6 +18,9 @@ set -e
 #       one of its hashes
 #   3:  the hash file exists and there was no hash to check the file against
 #   4:  the hash file exists and at least one hash type is unknown
+#   5:  the hash file exists and there was at least one hash for the file
+#       to check and all those hashes did match, but the file is excluded
+#       from being checked
 
 while getopts :q OPT; do
     case "${OPT}" in
@@ -106,4 +109,11 @@ if [ ${nb_checks} -eq 0 ]; then
     done
     printf "ERROR: No hash found for %s\n" "${base}" >&2
     exit 3
+else
+    for f in ${BR_NO_CHECK_HASH_FOR}; do
+        if [ "${f}" = "${base}" ]; then
+            printf "ERROR: Hash found for excluded %s\n" "${base}" >&2
+            exit 5
+        fi
+    done
 fi
-- 
1.9.1

[Buildroot] [PATCH 8/8 v4] support/download: do not accept exlcuded files with ashes

[Arnout Vandecappelle]

 In subject: excluded hashes

 However, I'm not so sure of this one. Let's say that I configure a custom
toolchain that comes from Linaro, and later on upstream buildroot adds official
support for this toolchain including a hash, then suddenly I'll get an error for
my custom toolchain...

 Also:

On 08/04/15 18:57, Yann E. MORIN wrote:
> If a file is excluded from checking its hashes, it must effectively have
> no hash for it.
> 
> So, if an excluded file indeed has at least one hash, there is an issue
> somewhere; that file should probably not be excluded.
> 
> When this happens, bail out in error.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  support/download/check-hash | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/support/download/check-hash b/support/download/check-hash
> index 96f5965..849637e 100755
> --- a/support/download/check-hash
> +++ b/support/download/check-hash
> @@ -18,6 +18,9 @@ set -e
>  #       one of its hashes
>  #   3:  the hash file exists and there was no hash to check the file against
>  #   4:  the hash file exists and at least one hash type is unknown
> +#   5:  the hash file exists and there was at least one hash for the file
> +#       to check and all those hashes did match, but the file is excluded
> +#       from being checked
>  
>  while getopts :q OPT; do
>      case "${OPT}" in
> @@ -106,4 +109,11 @@ if [ ${nb_checks} -eq 0 ]; then
>      done
>      printf "ERROR: No hash found for %s\n" "${base}" >&2
>      exit 3
> +else
> +    for f in ${BR_NO_CHECK_HASH_FOR}; do
> +        if [ "${f}" = "${base}" ]; then
> +            printf "ERROR: Hash found for excluded %s\n" "${base}" >&2
> +            exit 5
> +        fi
> +    done

 I'd move this check before actually checking the hash; I have the feeling it
will be pretty likely that the hashes will mismatch when this error occurs, and
then this error is probably the root issue.

 Regards,
 Arnout

>  fi
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F