[Buildroot] [PATCH] package/at91bootstrap3: fix hash

[Buildroot] [PATCH] package/at91bootstrap3: fix hash

[Pierre-Jean Texier]

When at91bootstrap3 was bumped to 3.9.0 in commit 513899e471890f4eb677066876890d43ea91b25e,
an incorrect hash was set.

However, after some checks on the archive with the wrong hash, it appears that it is identical to
the one with the good hash.

Fixes:

ERROR: at91bootstrap3-v3.9.0.tar.gz has wrong sha256 hash:
ERROR: expected: 9960b0d18fe42feee566d4c52efa0d7c8251685bf9acfdf343f30a27951ada1e
ERROR: got     : e23e6df23b79ca81e412cb73a1f48bd95df8d46c7d52a1d073c2ed9d4f3a1a71
ERROR: Incomplete download, or man-in-the-middle (MITM) attack

Reported-by: Ludovic Desroches <ludovic.desroches@microchip.com>
Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
---
 boot/at91bootstrap3/at91bootstrap3.hash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/boot/at91bootstrap3/at91bootstrap3.hash b/boot/at91bootstrap3/at91bootstrap3.hash
index 93f495f..b2ad624 100644
--- a/boot/at91bootstrap3/at91bootstrap3.hash
+++ b/boot/at91bootstrap3/at91bootstrap3.hash
@@ -1,2 +1,2 @@
 # Locally calculated
-sha256	9960b0d18fe42feee566d4c52efa0d7c8251685bf9acfdf343f30a27951ada1e  at91bootstrap3-v3.9.0.tar.gz
+sha256	e23e6df23b79ca81e412cb73a1f48bd95df8d46c7d52a1d073c2ed9d4f3a1a71  at91bootstrap3-v3.9.0.tar.gz
-- 
2.7.4

[Buildroot] [PATCH] package/at91bootstrap3: fix hash

[Thomas Petazzoni]

On Thu, 26 Dec 2019 21:49:43 +0100
Pierre-Jean Texier <pjtexier@koncepto.io> wrote:

> When at91bootstrap3 was bumped to 3.9.0 in commit 513899e471890f4eb677066876890d43ea91b25e,
> an incorrect hash was set.
> 
> However, after some checks on the archive with the wrong hash, it appears that it is identical to
> the one with the good hash.
> 
> Fixes:
> 
> ERROR: at91bootstrap3-v3.9.0.tar.gz has wrong sha256 hash:
> ERROR: expected: 9960b0d18fe42feee566d4c52efa0d7c8251685bf9acfdf343f30a27951ada1e
> ERROR: got     : e23e6df23b79ca81e412cb73a1f48bd95df8d46c7d52a1d073c2ed9d4f3a1a71
> ERROR: Incomplete download, or man-in-the-middle (MITM) attack
> 
> Reported-by: Ludovic Desroches <ludovic.desroches@microchip.com>
> Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>

Did you figure out how you got this bogus archive ? There are really no
differences ? Could you put the bogus archive online somewhere ?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH] package/at91bootstrap3: fix hash

[Pierre-Jean Texier]

Hello Thomas,

Le 26/12/2019 ? 21:59, Thomas Petazzoni a ?crit?:
> Did you figure out how you got this bogus archive ? There are really no
> differences ? Could you put the bogus archive online somewhere ?

No, I don't understand :/, this seems strange.
I can see a difference on the size (441K vs 444K) for sure, but not 
really differences on the files in the archive.

You can download the bogus archive here:
  - https://jirafeau.net/f.php?h=1j8hLl-Y

Thanks !

Pierre-Jean

> 
> Thanks!
> 
> Thomas

-- 
Pierre-Jean Texier
Embedded Linux Engineer
https://koncepto.io

[Buildroot] [PATCH] package/at91bootstrap3: fix hash

[Thomas Petazzoni]

Hello,

+Yann.

On Thu, 26 Dec 2019 22:16:50 +0100
Pierre-Jean Texier <pjtexier@koncepto.io> wrote:

> No, I don't understand :/, this seems strange.
> I can see a difference on the size (441K vs 444K) for sure, but not 
> really differences on the files in the archive.
> 
> You can download the bogus archive here:
>   - https://jirafeau.net/f.php?h=1j8hLl-Y

Thanks. Diffoscope reports some useful information:

??? filetype from file(1)
? @@ -1 +1 @@
? -POSIX tar archive (GNU)
? +POSIX tar archive

And indeed, after uncompressing the archives:

$ file at91bootstrap3-v3.9.0.tar.bad at91bootstrap3-v3.9.0.tar.good
at91bootstrap3-v3.9.0.tar.bad:  POSIX tar archive (GNU)
at91bootstrap3-v3.9.0.tar.good: POSIX tar archive

So, what I think happened is that your at91bootstrap3-v3.9.0.tar.gz is
the result from using the Buildroot feature to fetch from a Git
repository and then creating a .tar.gz archive (we create them in GNU
format). Which is different from the .tar.gz created by Github.

And indeed, if I configure the at91bootstrap3 package like this:

BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_GIT=y
BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_REPO_URL="https://github.com/linux4sam/at91bootstrap.git"
BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_REPO_VERSION="v3.9.0"

The tarball I get has the original hash you provided:

at91bootstrap3-v3.9.0.tar.gz: OK (sha256: 9960b0d18fe42feee566d4c52efa0d7c8251685bf9acfdf343f30a27951ada1e)

So basically, there is a naming conflict between the name of the
tarball that we retrieve directly from Github, and the name of the
tarball we produce locally by cloning the Git repository + creating the
tarball.

I'll apply your patch to fix the immediate issue, but I guess we have a
larger issue here.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH] package/at91bootstrap3: fix hash

[Yann E. MORIN]

Thomas, Pierre-Jean, All,

On 2019-12-26 22:34 +0100, Thomas Petazzoni spake thusly:
> On Thu, 26 Dec 2019 22:16:50 +0100
> Pierre-Jean Texier <pjtexier@koncepto.io> wrote:
> > No, I don't understand :/, this seems strange.
> > I can see a difference on the size (441K vs 444K) for sure, but not 
> > really differences on the files in the archive.
> > 
> > You can download the bogus archive here:
> >   - https://jirafeau.net/f.php?h=1j8hLl-Y
> 
> Thanks. Diffoscope reports some useful information:
> 
> ??? filetype from file(1)
> ? @@ -1 +1 @@
> ? -POSIX tar archive (GNU)
> ? +POSIX tar archive
> 
> And indeed, after uncompressing the archives:
> 
> $ file at91bootstrap3-v3.9.0.tar.bad at91bootstrap3-v3.9.0.tar.good
> at91bootstrap3-v3.9.0.tar.bad:  POSIX tar archive (GNU)
> at91bootstrap3-v3.9.0.tar.good: POSIX tar archive
> 
> So, what I think happened is that your at91bootstrap3-v3.9.0.tar.gz is
> the result from using the Buildroot feature to fetch from a Git
> repository and then creating a .tar.gz archive (we create them in GNU
> format). Which is different from the .tar.gz created by Github.
> 
> And indeed, if I configure the at91bootstrap3 package like this:
> 
> BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_GIT=y
> BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_REPO_URL="https://github.com/linux4sam/at91bootstrap.git"
> BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_REPO_VERSION="v3.9.0"
> 
> The tarball I get has the original hash you provided:
> 
> at91bootstrap3-v3.9.0.tar.gz: OK (sha256: 9960b0d18fe42feee566d4c52efa0d7c8251685bf9acfdf343f30a27951ada1e)
> 
> So basically, there is a naming conflict between the name of the
> tarball that we retrieve directly from Github, and the name of the
> tarball we produce locally by cloning the Git repository + creating the
> tarball.

So, my position on git repositories: we stop asking the forges to
generate a tarball on their side. Instead, we use git the way it has
been designed to be used: we clone the repositories and we generate
the archives locally.

Yes, this is a bold suggestion, and I am aware that this may hurt some
feelings. But let's consider the following:

  - we are doing a git cache now, so only the first downloads are
    penalised; subsequent downloads will be much, much faster [0]

  - the problem is that the dl/ directory is removed by default, which
    is not nice

  - we already have a cache of sorts that is not removed: the ccache is
    located (by default) in the user's home, in ~/.buildroot-ccache/

  - we could change the default for the download directory to be also
    stored in a non-trashed location

  - I would suggest that:

    2. we remove the download and ccache locations options from the
       configuration (they are site-local settings, thye have no place
       in the build configuration)
    1. we change the defaults to: ~/.buildroot/ccache/ and ~/.buildroot/dl/
    3. users can still override each independently with the already
       existing BR2_CCACHE_DIR and BR2_DL_DIR, or a new one to set both,
       BR2_BASE_CACHE_DIR (or whatever) if either is not already set.


Thoughts?

Regards,
Yann E. MORIN.

[0] some corner-cases might be slower, especially when doing a biiig
update-leap to a biiig repository, but even in the case of the Linux
tree it would still be faster to git-fetch new objects than download
a whole new tarball: a github-generated tar.gz for 5.5-rc3 is ~166MiB
(give or take); 166MiB would represent a shitload of new objects in
the repository...

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH] package/at91bootstrap3: fix hash

[Thomas Petazzoni]

On Thu, 26 Dec 2019 21:49:43 +0100
Pierre-Jean Texier <pjtexier@koncepto.io> wrote:

> When at91bootstrap3 was bumped to 3.9.0 in commit 513899e471890f4eb677066876890d43ea91b25e,
> an incorrect hash was set.
> 
> However, after some checks on the archive with the wrong hash, it appears that it is identical to
> the one with the good hash.
> 
> Fixes:
> 
> ERROR: at91bootstrap3-v3.9.0.tar.gz has wrong sha256 hash:
> ERROR: expected: 9960b0d18fe42feee566d4c52efa0d7c8251685bf9acfdf343f30a27951ada1e
> ERROR: got     : e23e6df23b79ca81e412cb73a1f48bd95df8d46c7d52a1d073c2ed9d4f3a1a71
> ERROR: Incomplete download, or man-in-the-middle (MITM) attack
> 
> Reported-by: Ludovic Desroches <ludovic.desroches@microchip.com>
> Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
> ---
>  boot/at91bootstrap3/at91bootstrap3.hash | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Applied to master with an improved commit log. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com