Re: [Buildroot] [PATCH 1/1] support/dependencies: bump minimal tar version to 1.35

[Arnout Vandecappelle via buildroot <buildroot@buildroot.org>]

On 18/10/2023 16:11, Vincent Fazio wrote:
> GNU tar 1.35 includes a breaking change [0] that changes the tar header
> created for each regular file even for "stable" formats like pax (which
> we use in support/download/helpers::mk_tar_gz).

  The funny thing is that the tar docs have a section about making the archive 
reproducible [1]. They call it "more reproducible", though, so perhaps they 
realize that they fail...


> Previously, normal files had the devmajor/devminor fields of the header
> filled with ASCII zero (0x30). These fields are now null which also
> affects the checksum value in the header as well.
> 
>    $ diff <(head -n 100 xxd.1.34.output) <(head -n 100 xxd.1.35.output)
>    10c10
>    < 00000090: 3037 3500 3031 3531 3637 0020 3000 0000  075.015167. 0...
>    ---
>    > 00000090: 3037 3500 3031 3337 3237 0020 3000 0000  075.013727. 0...
>    21,22c21,22
>    < 00000140: 0000 0000 0000 0000 0030 3030 3030 3030  .........0000000
>    < 00000150: 0030 3030 3030 3030 0000 0000 0000 0000  .0000000........
>    ---
>    > 00000140: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>    > 00000150: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> 
> This has the consequence of causing hash mismatches for any tarball
> created with a version prior to 1.35.
> 
> Instead of reverting back to a host specific version (see 37a909cacf)
> and adding a cap to the maximal tar version (see ec50e407be), just
> establish that 1.35+ should be used going forward.
> 
> This change requires an updated BR_FMT_VERSION for both git and svn.
> 
> [0] https://git.savannah.gnu.org/cgit/tar.git/commit/?id=738de9ecdec45ebfeb999628742373b5f8253bd0
> 
> Signed-off-by: Vincent Fazio <vfazio@gmail.com>
> ---
> This patch should be coordinated with other patches [1] that update the
> BR_FMT_VERSION revision for git or svn as all of the tarball hashes will
> need to be regenerated tree-wide.
> 
> [1] https://patchwork.ozlabs.org/project/buildroot/list/?series=373742
> ---
>   package/pkg-download.mk                |  4 ++--
>   support/dependencies/check-host-tar.sh | 10 ++++++----
>   2 files changed, 8 insertions(+), 6 deletions(-)
> 
> diff --git a/package/pkg-download.mk b/package/pkg-download.mk
> index e5cd83d859..d33eb2d811 100644
> --- a/package/pkg-download.mk
> +++ b/package/pkg-download.mk
> @@ -20,8 +20,8 @@ export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
>   
>   # Version of the format of the archives we generate in the corresponding
>   # download backend:
> -BR_FMT_VERSION_git = -br1
> -BR_FMT_VERSION_svn = -br3
> +BR_FMT_VERSION_git = -br2
> +BR_FMT_VERSION_svn = -br4

  Vendored cargo and go packages will have the same issue, no? Those will need 
to be updated as well...

  Perhaps we should look for an alternative archive format that is actually 
reproducible. If we anyway have to do a mass update, then it's better if now is 
the last time... Though I don't know if an actually reproducible format exists. 
GNU cpio has the --reproducible option, but I don't know if it guarantees 
reproducibility across cpio versions. Also, we still have gzip to contend with.

  Regards,
  Arnout

[1] 
https://www.gnu.org/software/tar/manual/html_chapter/Formats.html#Reproducibility


>   
>   DL_WRAPPER = support/download/dl-wrapper
>   
> diff --git a/support/dependencies/check-host-tar.sh b/support/dependencies/check-host-tar.sh
> index b7d607a47a..d56d0242a9 100755
> --- a/support/dependencies/check-host-tar.sh
> +++ b/support/dependencies/check-host-tar.sh
> @@ -27,11 +27,13 @@ if [ -n "${version_bsd}" ] ; then
>       exit 1
>   fi
>   
> -# Minimal version = 1.27 (previous versions do not correctly unpack archives
> -# containing hard-links if the --strip-components option is used or create
> -# different gnu long link headers for path elements > 100 characters).
> +# Minimal version = 1.35
> +# GNU tar upstream commit 738de9ecd introduced a "breaking" change that
> +# affects tarballs regardless of mode (pax/ustar/v7) such that tarballs
> +# generated with 1.35 will not hash to the same value as those generated
> +# by previous versions of tar.
>   major_min=1
> -minor_min=27
> +minor_min=35
>   
>   if [ $major -lt $major_min ]; then
>   	# echo nothing: no suitable tar found
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot