[Buildroot] [PATCH 0/4] support/download: make the git backend more robust

[Thomas Petazzoni <thomas.petazzoni@bootlin.com>]

Hello,

On Wed, 18 Apr 2018 10:28:24 -0300, Ricardo Martincoski wrote:

> Could be the case your build server has a blacklisted tar version and you run
> the commands in a clean output (actually without host-tar built)?

It is indeed the case (I have an old tar), and indeed building host-tar
first fixes the problem.

When I don't build host-tar, what happens is:

test at build:~/buildroot$ make host-squashfs-extract
>>> host-squashfs e38956b92f738518c29734399629e7cdb33072d3 Downloading  
Initialized empty Git repository in /home/test/dl/squashfs/git/.git/
Fetching all references 
remote: Counting objects: 8972, done.
remote: Total 8972 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (8972/8972), 1.56 MiB | 2.51 MiB/s, done.
Resolving deltas: 100% (6544/6544), done.
From https://git.kernel.org/pub/scm/fs/squashfs/squashfs-tools
 * [new branch]      lz4        -> origin/lz4
 * [new branch]      master     -> origin/master
 * [new branch]      stable     -> origin/stable
Could not fetch special ref 'e38956b92f738518c29734399629e7cdb33072d3'; assuming it is not special.
ERROR: squashfs-e38956b92f738518c29734399629e7cdb33072d3.tar.gz has wrong sha256 hash:
ERROR: expected: bd0aa3011320b8ebee68aa406060de277bef16daf81bad5b9f70cbea6db1a779
ERROR: got     : c7a61e3bcabb716b268f5a341055ac5ecda8b9f2b42025f82926f201ff5c8881
ERROR: Incomplete download, or man-in-the-middle (MITM) attack

So I assume it has used the system tar, which generates tar archives
whose hash doesn't match the one generated by "good" tar versions. Is
that the problem I was having ?

So, we indeed have a serious problem here. host-tar is not an extract
dependency, but a download dependency. Meh. Crap. This breaks several
things:

 - make <foo>-source on Git packages from a clean build

 - A regular build, if the first package downloaded is fetched from Git
   and no other package has been extracted before. Indeed, in such a
   case, host-tar would not yet be built/installed.

Gaaaah.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com