[Buildroot] [PATCH v2 1/2] dependencies.mk: check for valid host-tar before other host dependencies

[Buildroot] [PATCH v2 1/2] dependencies.mk: check for valid host-tar before other host dependencies

[Peter Korsgaard]

host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
before these in case they are both needed, otherwise the tools will fail to
extract.

With the upcoming change to blacklist modern tar versions this situation is
likely to trigger more often.

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
but that series is a bit too intrusive to add this close to 2018.02, so
therefore this hack.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
---
Changes since v1:
 - Issue not specific to lzip, so put tar in front of everything else

 support/dependencies/dependencies.mk | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
index 4f606f849d..3fc235863c 100644
--- a/support/dependencies/dependencies.mk
+++ b/support/dependencies/dependencies.mk
@@ -12,7 +12,10 @@
 define suitable-host-package
 $(shell support/dependencies/check-host-$(1).sh $(2))
 endef
--include $(sort $(wildcard support/dependencies/check-host-*.mk))
+# host utilities needs host-tar to extract the source code tarballs, so
+# ensure check-host-tar.mk is included before the rest
+include support/dependencies/check-host-tar.mk
+-include $(sort $(filter-out %-tar.mk,$(wildcard support/dependencies/check-host-*.mk)))
 
 ifeq ($(BR2_CCACHE),y)
 DEPENDENCIES_HOST_PREREQ += host-ccache
-- 
2.11.0

[Buildroot] [PATCH v2 2/2] check-host-tar.sh: blacklist tar 1.30+

[Peter Korsgaard]

Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
leading to hash mismatches for the tar archives we create ourselves from
git.  This is really a fix for a bug in earlier tar versions regarding
deterministic output, so it is unlikely to be reverted in later versions.

To work around this issue, blacklist tar 1.30+ similar to how we do it for
pre-1.17 versions so Buildroot falls back to building host-tar.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
---
 Changes since v1:
  - None

 support/dependencies/check-host-tar.sh | 33 ++++++++++++++++++++++++---------
 1 file changed, 24 insertions(+), 9 deletions(-)

diff --git a/support/dependencies/check-host-tar.sh b/support/dependencies/check-host-tar.sh
index 932d3c4fb2..2143877524 100755
--- a/support/dependencies/check-host-tar.sh
+++ b/support/dependencies/check-host-tar.sh
@@ -30,13 +30,28 @@ fi
 # containing hard-links if the --strip-components option is used).
 major_min=1
 minor_min=17
-if [ $major -gt $major_min ]; then
-	echo $tar
-else
-	if [ $major -eq $major_min -a $minor -ge $minor_min ]; then
-		echo $tar
-	else
-		# echo nothing: no suitable tar found
-		exit 1
-	fi
+
+# Maximal version = 1.29 (1.30 changed --numeric-owner output for
+# filenames > 100 characters). This is really a fix for a bug in
+# earlier tar versions regarding deterministic output so it is
+# unlikely to be reverted in later versions.
+major_max=1
+minor_max=29
+
+if [ $major -lt $major_min -o $major -gt $major_max ]; then
+	# echo nothing: no suitable tar found
+	exit 1
 fi
+
+if [ $major -eq $major_min -a $minor -lt $minor_min ]; then
+	# echo nothing: no suitable tar found
+	exit 1
+fi
+
+if [ $major -eq $major_max -a $minor -gt $minor_max ]; then
+	# echo nothing: no suitable tar found
+	exit 1
+fi
+
+# valid
+echo $tar
-- 
2.11.0

[Buildroot] [PATCH v2 2/2] check-host-tar.sh: blacklist tar 1.30+

[Peter Korsgaard]

>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

 > Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
 > leading to hash mismatches for the tar archives we create ourselves from
 > git.  This is really a fix for a bug in earlier tar versions regarding
 > deterministic output, so it is unlikely to be reverted in later versions.

 > To work around this issue, blacklist tar 1.30+ similar to how we do it for
 > pre-1.17 versions so Buildroot falls back to building host-tar.

 > Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
 > ---
 >  Changes since v1:
 >   - None

Committed after adding a link to
http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html
which explains the issue in more details, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 2/2] check-host-tar.sh: blacklist tar 1.30+

[Peter Korsgaard]

>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

 > Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
 > leading to hash mismatches for the tar archives we create ourselves from
 > git.  This is really a fix for a bug in earlier tar versions regarding
 > deterministic output, so it is unlikely to be reverted in later versions.

 > To work around this issue, blacklist tar 1.30+ similar to how we do it for
 > pre-1.17 versions so Buildroot falls back to building host-tar.

 > Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
 > ---
 >  Changes since v1:
 >   - None

 >  support/dependencies/check-host-tar.sh | 33 ++++++++++++++++++++++++---------
 >  1 file changed, 24 insertions(+), 9 deletions(-)

Committed to 2017.02.x, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 1/2] dependencies.mk: check for valid host-tar before other host dependencies

[Matthew Weber]

All,

On Tue, Feb 27, 2018 at 2:55 AM, Peter Korsgaard <peter@korsgaard.com> wrote:
> host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
> we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
> before these in case they are both needed, otherwise the tools will fail to
> extract.
>
> With the upcoming change to blacklist modern tar versions this situation is
> likely to trigger more often.
>
> The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
> but that series is a bit too intrusive to add this close to 2018.02, so
> therefore this hack.
>
> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>

[Buildroot] [PATCH v2 1/2] dependencies.mk: check for valid host-tar before other host dependencies

[Peter Korsgaard]

>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

 > host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
 > we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
 > before these in case they are both needed, otherwise the tools will fail to
 > extract.

 > With the upcoming change to blacklist modern tar versions this situation is
 > likely to trigger more often.

 > The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
 > but that series is a bit too intrusive to add this close to 2018.02, so
 > therefore this hack.

 > Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
 > ---
 > Changes since v1:
 >  - Issue not specific to lzip, so put tar in front of everything else

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 1/2] dependencies.mk: check for valid host-tar before other host dependencies

[Peter Korsgaard]

>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

 > host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
 > we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
 > before these in case they are both needed, otherwise the tools will fail to
 > extract.

 > With the upcoming change to blacklist modern tar versions this situation is
 > likely to trigger more often.

 > The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
 > but that series is a bit too intrusive to add this close to 2018.02, so
 > therefore this hack.

 > Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
 > ---
 > Changes since v1:
 >  - Issue not specific to lzip, so put tar in front of everything else

Committed to 2017.02.x, thanks.

-- 
Bye, Peter Korsgaard