[Buildroot] linux-firmware hash mismatch (tar-1.30)

[Buildroot] linux-firmware hash mismatch (tar-1.30)

[John Keeping]

Hi,

I'm getting the following error cloning linux-firmware:

ERROR: linux-firmware-17e6288135d4500f9fe60224dce2b46d850c346b.tar.gz has wrong sha256 hash:
ERROR: expected: 28d359523a36c1cdc3e85a8e148bb2d68b036d28b10f0e80a192f3dc29f02c16
ERROR: got     : bf6fe8d7620949a3e771954cb6d9d18dcf000d37ecc910a7cf69723c1798e246
ERROR: Incomplete download, or man-in-the-middle (MITM) attack


After a bit of digging, it looks like this is caused by tar-1.30 which
includes the following fix:

* --numeric-owner now affects private headers too.

Comparing the tarball created on my system with one from
sources.buildroot.net gives the output like this, which shows that the
uname/gname fields are set for a GNUTYPE_LONGNAME record in the version
from sources.buildroot.net and not set in the version created with
tar-1.30:

-- >8 --
@@ -669379,24 +669379,24 @@
 00a36c20: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36c30: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36c40: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36c50: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36c60: 0000 0000 3030 3030 3634 3400 3030 3030  ....0000644.0000
 00a36c70: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
 00a36c80: 3030 3030 3135 3600 3030 3030 3030 3030  0000156.00000000
-00a36c90: 3030 3000 3031 3136 3035 0020 4c00 0000  000.011605. L...
+00a36c90: 3030 3000 3030 3737 3735 0020 4c00 0000  000.007775. L...
 00a36ca0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36cb0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36cc0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36cd0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36ce0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36cf0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-00a36d00: 0075 7374 6172 2020 0072 6f6f 7400 0000  .ustar  .root...
+00a36d00: 0075 7374 6172 2020 0000 0000 0000 0000  .ustar  ........
 00a36d10: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-00a36d20: 0000 0000 0000 0000 0072 6f6f 7400 0000  .........root...
+00a36d20: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36d30: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36d40: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36d50: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36d60: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36d70: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36d80: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00a36d90: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-- 8< --

I'm not sure whether anything can be done to avoid this problem, but
hopefully reporting it will save the next person some debugging time.


Regards,
John

[Buildroot] linux-firmware hash mismatch (tar-1.30)

[Peter Korsgaard]

>>>>> "John" == John Keeping <john@metanate.com> writes:

 > Hi,
 > I'm getting the following error cloning linux-firmware:

 > ERROR: linux-firmware-17e6288135d4500f9fe60224dce2b46d850c346b.tar.gz has wrong sha256 hash:
 > ERROR: expected: 28d359523a36c1cdc3e85a8e148bb2d68b036d28b10f0e80a192f3dc29f02c16
 > ERROR: got     : bf6fe8d7620949a3e771954cb6d9d18dcf000d37ecc910a7cf69723c1798e246
 > ERROR: Incomplete download, or man-in-the-middle (MITM) attack


 > After a bit of digging, it looks like this is caused by tar-1.30 which
 > includes the following fix:

 > * --numeric-owner now affects private headers too.

 > Comparing the tarball created on my system with one from
 > sources.buildroot.net gives the output like this, which shows that the
 > uname/gname fields are set for a GNUTYPE_LONGNAME record in the version
 > from sources.buildroot.net and not set in the version created with
 > tar-1.30:

Gaah, what a mess :/

 > I'm not sure whether anything can be done to avoid this problem, but
 > hopefully reporting it will save the next person some debugging time.

I also don't quite see any good solutions either :/

Did you try bringing it up with tar upstream? Perhaps there is a way to
disable this? Alternatively we can build tar-1.29 for the host and use
that instead of whichever tar version is available on the build machine,
but this will slow down the build.

-- 
Bye, Peter Korsgaard

[Buildroot] linux-firmware hash mismatch (tar-1.30)

[John Keeping]

On Wed, 10 Jan 2018 20:15:38 +0100, Peter Korsgaard wrote:

> >>>>> "John" == John Keeping <john@metanate.com> writes:  
> 
>  > Hi,
>  > I'm getting the following error cloning linux-firmware:  
> 
>  > ERROR: linux-firmware-17e6288135d4500f9fe60224dce2b46d850c346b.tar.gz has wrong sha256 hash:
>  > ERROR: expected: 28d359523a36c1cdc3e85a8e148bb2d68b036d28b10f0e80a192f3dc29f02c16
>  > ERROR: got     : bf6fe8d7620949a3e771954cb6d9d18dcf000d37ecc910a7cf69723c1798e246
>  > ERROR: Incomplete download, or man-in-the-middle (MITM) attack  
> 
> 
>  > After a bit of digging, it looks like this is caused by tar-1.30 which
>  > includes the following fix:  
> 
>  > * --numeric-owner now affects private headers too.  
> 
>  > Comparing the tarball created on my system with one from
>  > sources.buildroot.net gives the output like this, which shows that the
>  > uname/gname fields are set for a GNUTYPE_LONGNAME record in the version
>  > from sources.buildroot.net and not set in the version created with
>  > tar-1.30:  
> 
> Gaah, what a mess :/
> 
>  > I'm not sure whether anything can be done to avoid this problem, but
>  > hopefully reporting it will save the next person some debugging time.  
> 
> I also don't quite see any good solutions either :/
> 
> Did you try bringing it up with tar upstream? Perhaps there is a way to
> disable this? Alternatively we can build tar-1.29 for the host and use
> that instead of whichever tar version is available on the build machine,
> but this will slow down the build.

No, I didn't bring it up with tar upstream; I'm not sure there's much
point since it is a clear bug fix to the --numeric-owner option.

I don't think it's possible to reproducibly create bit-identical
archives without using the same software version to produce the archive.

[Buildroot] linux-firmware hash mismatch (tar-1.30)

[Yann E. MORIN]

John, Peter, All,

(adding Thomas and Arnout)

On 2018-01-11 10:55 +0000, John Keeping spake thusly:
> On Wed, 10 Jan 2018 20:15:38 +0100, Peter Korsgaard wrote:
> > >>>>> "John" == John Keeping <john@metanate.com> writes:  
> >  > ERROR: linux-firmware-17e6288135d4500f9fe60224dce2b46d850c346b.tar.gz has wrong sha256 hash:
> >  > ERROR: expected: 28d359523a36c1cdc3e85a8e148bb2d68b036d28b10f0e80a192f3dc29f02c16
> >  > ERROR: got     : bf6fe8d7620949a3e771954cb6d9d18dcf000d37ecc910a7cf69723c1798e246
> >  > ERROR: Incomplete download, or man-in-the-middle (MITM) attack  
> >  > After a bit of digging, it looks like this is caused by tar-1.30 which
> >  > includes the following fix:  
> >  > * --numeric-owner now affects private headers too.  
> > Gaah, what a mess :/

And that's not all.

Between 1.26 and 1.28 [*] the header for "long links" or "long names"
has changed. Previously, the "mode" field was al set to zero, but now it
is set to some non-zero value (not sure which, though...)

> >  > I'm not sure whether anything can be done to avoid this problem, but
> >  > hopefully reporting it will save the next person some debugging time.  
> > 
> > I also don't quite see any good solutions either :/
> > 
> > Did you try bringing it up with tar upstream? Perhaps there is a way to
> > disable this? Alternatively we can build tar-1.29 for the host and use
> > that instead of whichever tar version is available on the build machine,
> > but this will slow down the build.
> 
> No, I didn't bring it up with tar upstream; I'm not sure there's much
> point since it is a clear bug fix to the --numeric-owner option.
> 
> I don't think it's possible to reproducibly create bit-identical
> archives without using the same software version to produce the archive.

Alas, this means that we can only depend on building our own tar...

Even when we eventually support using a local git-clone cache, this we
not solve the issue has the hashes we store are on the generated
tarball...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] linux-firmware hash mismatch (tar-1.30)

[Arnout Vandecappelle]

On 14-01-18 14:30, Yann E. MORIN wrote:
> John, Peter, All,
> 
> (adding Thomas and Arnout)
> 
> On 2018-01-11 10:55 +0000, John Keeping spake thusly:
>> On Wed, 10 Jan 2018 20:15:38 +0100, Peter Korsgaard wrote:
>>>>>>>> "John" == John Keeping <john@metanate.com> writes:  
>>>  > ERROR: linux-firmware-17e6288135d4500f9fe60224dce2b46d850c346b.tar.gz has wrong sha256 hash:
>>>  > ERROR: expected: 28d359523a36c1cdc3e85a8e148bb2d68b036d28b10f0e80a192f3dc29f02c16
>>>  > ERROR: got     : bf6fe8d7620949a3e771954cb6d9d18dcf000d37ecc910a7cf69723c1798e246
>>>  > ERROR: Incomplete download, or man-in-the-middle (MITM) attack  
>>>  > After a bit of digging, it looks like this is caused by tar-1.30 which
>>>  > includes the following fix:  
>>>  > * --numeric-owner now affects private headers too.  
>>> Gaah, what a mess :/

 Note that this also means our github hashes will break again...

[snip]
>> I don't think it's possible to reproducibly create bit-identical
>> archives without using the same software version to produce the archive.
> 
> Alas, this means that we can only depend on building our own tar...

 Nah, let's not go there.

> Even when we eventually support using a local git-clone cache, this we
> not solve the issue has the hashes we store are on the generated
> tarball...

 However, the way we create a git tarball can serve as a source of inspiration
of how to solve it. Instead of hashing the tarball, we can hash the contents of
the tarball instead, using --to-command and a support script (--to-command
exists since 1.15.90 and our minimal tar version is 1.17). The support script
would print the metadata and a hash of the contents. And the output of all that
is piped into sort (to make sure the order of files in the tarball isn't
relevant) and shasum.

 That way we don't rely on the way the user's tar behaves at all. We really
check the contents of the tarball. It would also mean we can drop the complexity
in the git download helper and directly use git archive.

 It does create additional overhead, since the tarball will be uncompressed
twice: once for calculating the hash, and again when extracting. But normally
calculating a shasum should take significantly longer than decompressing, so I
don't expect this to have a big impact.


 Of course, introducing such a feature would break all existing hashes in one
fell swoop... So to avoid that, I would introduce a new hash type, sha256-tar,
that does this trickery.


 One small additional thing: we'll probably have to pass the result of
suitable-extractor to the download helper and down check-hash; alternatively, we
have to duplicate that logic, or move it to a script that is called from check-hash.


 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] linux-firmware hash mismatch (tar-1.30)

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >>>> Gaah, what a mess :/

 >  Note that this also means our github hashes will break again...

Indeed, I also mentioned that earlier.

 >> Alas, this means that we can only depend on building our own tar...

 >  Nah, let's not go there.

 >> Even when we eventually support using a local git-clone cache, this we
 >> not solve the issue has the hashes we store are on the generated
 >> tarball...

 >  However, the way we create a git tarball can serve as a source of inspiration
 > of how to solve it. Instead of hashing the tarball, we can hash the contents of
 > the tarball instead, using --to-command and a support script (--to-command
 > exists since 1.15.90 and our minimal tar version is 1.17). The support script
 > would print the metadata and a hash of the contents. And the output of all that
 > is piped into sort (to make sure the order of files in the tarball isn't
 > relevant) and shasum.

Hmm, that is perhaps an option. It does make it somewhat more cumbersome
to calculate the expected hash by hand, but maybe that is ok.

Alternatively we don't do anything and instead ensure that tarballs on
s.b.o matches the hashes we commit, and people using incompatible tar
versions will fall back to the tarballs on s.b.o.


 >  That way we don't rely on the way the user's tar behaves at all. We really
 > check the contents of the tarball. It would also mean we can drop the complexity
 > in the git download helper and directly use git archive.

I thought the main reason for our custom tarball handling was support
for git submodules? I take it that git-archive still doesn't handle those?

-- 
Bye, Peter Korsgaard

[Buildroot] linux-firmware hash mismatch (tar-1.30)

[Arnout Vandecappelle]

On 18-01-18 22:40, Peter Korsgaard wrote:
>>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:
> 
> Hi,
> 
>  >>>> Gaah, what a mess :/
> 
>  >  Note that this also means our github hashes will break again...
> 
> Indeed, I also mentioned that earlier.
> 
>  >> Alas, this means that we can only depend on building our own tar...
> 
>  >  Nah, let's not go there.
> 
>  >> Even when we eventually support using a local git-clone cache, this we
>  >> not solve the issue has the hashes we store are on the generated
>  >> tarball...
> 
>  >  However, the way we create a git tarball can serve as a source of inspiration
>  > of how to solve it. Instead of hashing the tarball, we can hash the contents of
>  > the tarball instead, using --to-command and a support script (--to-command
>  > exists since 1.15.90 and our minimal tar version is 1.17). The support script
>  > would print the metadata and a hash of the contents. And the output of all that
>  > is piped into sort (to make sure the order of files in the tarball isn't
>  > relevant) and shasum.
> 
> Hmm, that is perhaps an option. It does make it somewhat more cumbersome
> to calculate the expected hash by hand, but maybe that is ok.

 There's still a patch floating around that doesn't remove the downloaded file
[1], and the calculated hash is already reported. So not much of an issue I think.


> Alternatively we don't do anything and instead ensure that tarballs on
> s.b.o matches the hashes we commit, and people using incompatible tar
> versions will fall back to the tarballs on s.b.o.

 But then they'll still be downloaded twice. Not very nice.

 Let's discuss it at the meeting in two weeks.


>  >  That way we don't rely on the way the user's tar behaves at all. We really
>  > check the contents of the tarball. It would also mean we can drop the complexity
>  > in the git download helper and directly use git archive.
> 
> I thought the main reason for our custom tarball handling was support
> for git submodules? I take it that git-archive still doesn't handle those?

 Right, I forgot about that.

 Regards,
 Arnout


[1] http://patchwork.ozlabs.org/patch/834425/

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF