Re: [Buildroot] [PATCH 1/1] support/dependencies: bump minimal tar version to 1.35

["Yann E. MORIN" <yann.morin.1998@free.fr>]

Vincent, Arnout, All,

On 2023-10-18 19:43 +0200, Arnout Vandecappelle via buildroot spake thusly:
> On 18/10/2023 16:11, Vincent Fazio wrote:
> >GNU tar 1.35 includes a breaking change [0] that changes the tar header
> >created for each regular file even for "stable" formats like pax (which
> >we use in support/download/helpers::mk_tar_gz).
>  The funny thing is that the tar docs have a section about making the
> archive reproducible [1]. They call it "more reproducible", though, so
> perhaps they realize that they fail...

Yes, I think this is what served as a base for Vincent to come up with
the options we have in the tar helper.

[--SNIP--]
> >diff --git a/package/pkg-download.mk b/package/pkg-download.mk
> >index e5cd83d859..d33eb2d811 100644
> >--- a/package/pkg-download.mk
> >+++ b/package/pkg-download.mk
> >@@ -20,8 +20,8 @@ export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
> >  # Version of the format of the archives we generate in the corresponding
> >  # download backend:
> >-BR_FMT_VERSION_git = -br1
> >-BR_FMT_VERSION_svn = -br3
> >+BR_FMT_VERSION_git = -br2
> >+BR_FMT_VERSION_svn = -br4
>  Vendored cargo and go packages will have the same issue, no? Those will
> need to be updated as well...

Good catch, indeed. I'll try and resurect my patch to add a -br suffix
to those...

>  Perhaps we should look for an alternative archive format that is actually
> reproducible. If we anyway have to do a mass update, then it's better if now
> is the last time... Though I don't know if an actually reproducible format
> exists.

There is a lost there:
    https://en.wikipedia.org/wiki/List_of_archive_formats

but it does not provide any information about stability...

We could of course come up with our own, but I would very much like that
we do avoid that at all cost. The odds of us failing if we go that route
are higher than those of us succeeding, I'd say...

> GNU cpio has the --reproducible option, but I don't know if it
> guarantees reproducibility across cpio versions.

[0] states that --reproducible "is equivalent to ‘--ignore-devno
--ignore-dirnlink --renumber-inodes’", which to me means that some
metadata is not recorded (or is faked), not that there is any
guarantee on the stability of the stream.

[0] https://www.gnu.org/software/cpio/manual/cpio.html#Copy_002dpass-mode

> Also, we still have gzip to
> contend with.

I did test that a while ago, and gzip *has* been ultra-super stable over
the years, to the point that all the versions that I could build (we was
almost back to the initial release 30 yerrs ago) did produce the same
output as the latest version. So, if I had to worry about
reproducibility, I would not look at gzip.

Regards,
Yann E. MORIN.

>  Regards,
>  Arnout
> 
> [1] https://www.gnu.org/software/tar/manual/html_chapter/Formats.html#Reproducibility
> 
> 
> >  DL_WRAPPER = support/download/dl-wrapper
> >diff --git a/support/dependencies/check-host-tar.sh b/support/dependencies/check-host-tar.sh
> >index b7d607a47a..d56d0242a9 100755
> >--- a/support/dependencies/check-host-tar.sh
> >+++ b/support/dependencies/check-host-tar.sh
> >@@ -27,11 +27,13 @@ if [ -n "${version_bsd}" ] ; then
> >      exit 1
> >  fi
> >-# Minimal version = 1.27 (previous versions do not correctly unpack archives
> >-# containing hard-links if the --strip-components option is used or create
> >-# different gnu long link headers for path elements > 100 characters).
> >+# Minimal version = 1.35
> >+# GNU tar upstream commit 738de9ecd introduced a "breaking" change that
> >+# affects tarballs regardless of mode (pax/ustar/v7) such that tarballs
> >+# generated with 1.35 will not hash to the same value as those generated
> >+# by previous versions of tar.
> >  major_min=1
> >-minor_min=27
> >+minor_min=35
> >  if [ $major -lt $major_min ]; then
> >  	# echo nothing: no suitable tar found
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot