* [Buildroot] [PATCH 1 of 5] toolchain-external: allow downloading a custom toolchain
2011-08-02 18:33 [Buildroot] [PATCH 0 of 5] toolchain-external: support downloading custom toolchains + related misc. fixes Thomas De Schampheleire
@ 2011-08-02 18:33 ` Thomas De Schampheleire
2011-08-02 18:33 ` [Buildroot] [PATCH 2 of 5] Support URI schemes in DOWNLOAD function, when not using gentargets Thomas De Schampheleire
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Thomas De Schampheleire @ 2011-08-02 18:33 UTC (permalink / raw)
To: buildroot
Custom external toolchains currently already have to be extracted by the user. This patch adds support for downloading a custom toolchain in the form of a tarball, which will be extracted by buildroot.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -189,12 +189,23 @@
config BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD
bool "Download toolchain automatically"
default y
- depends on !BR2_TOOLCHAIN_EXTERNAL_CUSTOM
help
When enabled, Buildroot will automatically download and
install the selected external toolchain. When disabled,
Buildroot will use a pre-installed toolchain.
+config BR2_TOOLCHAIN_EXTERNAL_SITE
+ string "Toolchain download site"
+ depends on BR2_TOOLCHAIN_EXTERNAL_CUSTOM && BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD
+ help
+ Base URL at which the custom toolchain can be downloaded.
+
+config BR2_TOOLCHAIN_EXTERNAL_SOURCE
+ string "Toolchain filename"
+ depends on BR2_TOOLCHAIN_EXTERNAL_CUSTOM && BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD
+ help
+ Filename of the toolchain package to download.
+
config BR2_TOOLCHAIN_EXTERNAL_PATH
string "Toolchain path"
default "/path/to/toolchain/usr"
diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/ext-tool.mk
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/ext-tool.mk
@@ -160,6 +160,10 @@
TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(STAMP_DIR)/ext-toolchain-checked
endif
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CUSTOM),y)
+TOOLCHAIN_EXTERNAL_DEPENDENCIES += $(STAMP_DIR)/ext-toolchain-checked
+endif
+
ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM2009Q1),y)
TOOLCHAIN_EXTERNAL_SITE=http://www.codesourcery.com/sgpp/lite/arm/portal/package4571/public/arm-none-linux-gnueabi/
TOOLCHAIN_EXTERNAL_SOURCE=arm-2009q1-203-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
@@ -197,11 +201,16 @@
TOOLCHAIN_EXTERNAL_SOURCE_2 = blackfin-toolchain-uclibc-full-2010R1-RC4.i386.tar.bz2
TOOLCHAIN_EXTERNAL_SOURCE = $(TOOLCHAIN_EXTERNAL_SOURCE_1) $(TOOLCHAIN_EXTERNAL_SOURCE_2)
else
+# Custom toolchain
+TOOLCHAIN_EXTERNAL_SITE=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_SITE))
+TOOLCHAIN_EXTERNAL_SOURCE=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_SOURCE))
# A value must be set (even if unused), otherwise the
# $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE) rule would override the main
# $(DL_DIR) rule
+ifeq (,$(TOOLCHAIN_EXTERNAL_SOURCE))
TOOLCHAIN_EXTERNAL_SOURCE=none
endif
+endif
# Special handling for Blackfin toolchain, because of the split in two
# tarballs, and the organization of tarball contents. The tarballs
^ permalink raw reply [flat|nested] 9+ messages in thread* [Buildroot] [PATCH 2 of 5] Support URI schemes in DOWNLOAD function, when not using gentargets
2011-08-02 18:33 [Buildroot] [PATCH 0 of 5] toolchain-external: support downloading custom toolchains + related misc. fixes Thomas De Schampheleire
2011-08-02 18:33 ` [Buildroot] [PATCH 1 of 5] toolchain-external: allow downloading a custom toolchain Thomas De Schampheleire
@ 2011-08-02 18:33 ` Thomas De Schampheleire
2011-08-02 18:33 ` [Buildroot] [PATCH 3 of 5] toolchain-external: fix some typos CodeSoucery -> CodeSourcery Thomas De Schampheleire
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Thomas De Schampheleire @ 2011-08-02 18:33 UTC (permalink / raw)
To: buildroot
When using the gentargets/autotools/cmake infrastructure, the DOWNLOAD function uses an appropriate download method based on $(PKG)_SITE_METHOD, which is autodetected based on the URI if none was explicitly set.
When the DOWNLOAD function is called directly, when not using gentargets, the SITE_METHOD is not autodetected, and thus the download defaults to wget.
This patch adds URI scheme detection directly to the DOWNLOAD method, in case no SITE_METHOD was detected yet.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
diff --git a/package/Makefile.package.in b/package/Makefile.package.in
--- a/package/Makefile.package.in
+++ b/package/Makefile.package.in
@@ -184,12 +184,19 @@
# $(call DOWNLOAD,$(FOO_SITE),$(FOO_SOURCE))
################################################################################
+geturischeme=$(firstword $(subst ://, ,$(call qstrip,$(1))))
+
define DOWNLOAD
$(Q)if test -n "$(call qstrip,$(BR2_PRIMARY_SITE))" ; then \
$(call $(DL_MODE)_WGET,$(BR2_PRIMARY_SITE),$(2)) && exit ; \
fi ; \
if test -n "$(1)" ; then \
- case "$($(PKG)_SITE_METHOD)" in \
+ if test -z "$($(PKG)_SITE_METHOD)" ; then \
+ scheme="$(call geturischeme,$(1))" ; \
+ else \
+ scheme="$($(PKG)_SITE_METHOD)" ; \
+ fi ; \
+ case "$$scheme" in \
git) $($(DL_MODE)_GIT) && exit ;; \
svn) $($(DL_MODE)_SVN) && exit ;; \
bzr) $($(DL_MODE)_BZR) && exit ;; \
^ permalink raw reply [flat|nested] 9+ messages in thread* [Buildroot] [PATCH 3 of 5] toolchain-external: fix some typos CodeSoucery -> CodeSourcery
2011-08-02 18:33 [Buildroot] [PATCH 0 of 5] toolchain-external: support downloading custom toolchains + related misc. fixes Thomas De Schampheleire
2011-08-02 18:33 ` [Buildroot] [PATCH 1 of 5] toolchain-external: allow downloading a custom toolchain Thomas De Schampheleire
2011-08-02 18:33 ` [Buildroot] [PATCH 2 of 5] Support URI schemes in DOWNLOAD function, when not using gentargets Thomas De Schampheleire
@ 2011-08-02 18:33 ` Thomas De Schampheleire
2011-08-04 19:47 ` Thomas Petazzoni
2011-08-04 19:57 ` Peter Korsgaard
2011-08-02 18:33 ` [Buildroot] [PATCH 4 of 5] tar: create a host version Thomas De Schampheleire
2011-08-02 18:33 ` [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains Thomas De Schampheleire
4 siblings, 2 replies; 9+ messages in thread
From: Thomas De Schampheleire @ 2011-08-02 18:33 UTC (permalink / raw)
To: buildroot
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -120,7 +120,7 @@
- MIPS32 little endian soft float uclibc
config BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_POWERPC201009
- bool "CodeSoucery PowerPC 2010.09"
+ bool "CodeSourcery PowerPC 2010.09"
depends on BR2_powerpc
select BR2_TOOLCHAIN_EXTERNAL_GLIBC
select BR2_INSTALL_LIBSTDCPP
@@ -138,7 +138,7 @@
- 970 glibc hard-float, 64 bits
config BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_SH201009
- bool "CodeSoucery SH 2010.09"
+ bool "CodeSourcery SH 2010.09"
depends on BR2_sh4a || BR2_sh4aeb
select BR2_TOOLCHAIN_EXTERNAL_GLIBC
select BR2_INSTALL_LIBSTDCPP
@@ -153,7 +153,7 @@
- SH4A, uClibc, big endian
config BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_SH2A_201009
- bool "CodeSoucery SH 2010.09"
+ bool "CodeSourcery SH 2010.09"
depends on BR2_sh2a
select BR2_TOOLCHAIN_EXTERNAL_UCLIBC
select BR2_INSTALL_LIBSTDCPP
^ permalink raw reply [flat|nested] 9+ messages in thread* [Buildroot] [PATCH 3 of 5] toolchain-external: fix some typos CodeSoucery -> CodeSourcery
2011-08-02 18:33 ` [Buildroot] [PATCH 3 of 5] toolchain-external: fix some typos CodeSoucery -> CodeSourcery Thomas De Schampheleire
@ 2011-08-04 19:47 ` Thomas Petazzoni
2011-08-04 19:57 ` Peter Korsgaard
1 sibling, 0 replies; 9+ messages in thread
From: Thomas Petazzoni @ 2011-08-04 19:47 UTC (permalink / raw)
To: buildroot
Le Tue, 02 Aug 2011 20:33:17 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Peter: this one should go in 2011.08, it's just fixes. The other
patches need discussion, so I'd prefer to handle them post 2011.08.
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Buildroot] [PATCH 3 of 5] toolchain-external: fix some typos CodeSoucery -> CodeSourcery
2011-08-02 18:33 ` [Buildroot] [PATCH 3 of 5] toolchain-external: fix some typos CodeSoucery -> CodeSourcery Thomas De Schampheleire
2011-08-04 19:47 ` Thomas Petazzoni
@ 2011-08-04 19:57 ` Peter Korsgaard
1 sibling, 0 replies; 9+ messages in thread
From: Peter Korsgaard @ 2011-08-04 19:57 UTC (permalink / raw)
To: buildroot
>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> writes:
Thomas> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Committed, thanks.
--
Bye, Peter Korsgaard
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Buildroot] [PATCH 4 of 5] tar: create a host version
2011-08-02 18:33 [Buildroot] [PATCH 0 of 5] toolchain-external: support downloading custom toolchains + related misc. fixes Thomas De Schampheleire
` (2 preceding siblings ...)
2011-08-02 18:33 ` [Buildroot] [PATCH 3 of 5] toolchain-external: fix some typos CodeSoucery -> CodeSourcery Thomas De Schampheleire
@ 2011-08-02 18:33 ` Thomas De Schampheleire
2011-08-02 18:33 ` [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains Thomas De Schampheleire
4 siblings, 0 replies; 9+ messages in thread
From: Thomas De Schampheleire @ 2011-08-02 18:33 UTC (permalink / raw)
To: buildroot
Some systems, like RedHat/CentOS, are even today shipped with ancient versions of GNU tar, like 1.15. This can cause trouble, for example when unpacking archives containing hardlinks with the --strip-components option.
To avoid such problems, let buildroot create a host-tar that can be used when unpacking.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
diff --git a/package/tar/tar.mk b/package/tar/tar.mk
--- a/package/tar/tar.mk
+++ b/package/tar/tar.mk
@@ -8,3 +8,6 @@
TAR_SITE = $(BR2_GNU_MIRROR)/tar
$(eval $(call AUTOTARGETS,package,tar))
+$(eval $(call AUTOTARGETS,package,tar,host))
+
+HOST_TAR = $(HOST_DIR)/usr/bin/tar
^ permalink raw reply [flat|nested] 9+ messages in thread* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-08-02 18:33 [Buildroot] [PATCH 0 of 5] toolchain-external: support downloading custom toolchains + related misc. fixes Thomas De Schampheleire
` (3 preceding siblings ...)
2011-08-02 18:33 ` [Buildroot] [PATCH 4 of 5] tar: create a host version Thomas De Schampheleire
@ 2011-08-02 18:33 ` Thomas De Schampheleire
2011-08-19 9:11 ` Thomas De Schampheleire
4 siblings, 1 reply; 9+ messages in thread
From: Thomas De Schampheleire @ 2011-08-02 18:33 UTC (permalink / raw)
To: buildroot
Using the system tar with --strip-components causes problems when its version is too old (<1.17). Even recent releases of RedHat/CentOS still ship with tar 1.15. To avoid such problems, always use host-tar instead of tar to unpack toolchains.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
To be honest, I'm not sure what the best way is to handle this. The same problems with tar could occur in other places than just external toolchains. So maybe we should use host-tar in more places.
On the other hand, most users have a recent, working, tar version on there system. It may thus make more sense to detect the system tar version and only use host-tar if it's too old.
diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/ext-tool.mk
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/ext-tool.mk
@@ -155,7 +155,7 @@
endif
ifeq ($(BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD),y)
-TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(TOOLCHAIN_EXTERNAL_DIR)/.extracted
+TOOLCHAIN_EXTERNAL_DEPENDENCIES = host-tar $(TOOLCHAIN_EXTERNAL_DIR)/.extracted
else
TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(STAMP_DIR)/ext-toolchain-checked
endif
@@ -227,9 +227,9 @@
$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2)
mkdir -p $(@D)
$(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE_1))) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) | \
- $(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
+ $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
$(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE_2))) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2) | \
- $(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
+ $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
ifeq ($(TOOLCHAIN_EXTERNAL_PREFIX),bfin-uclinux)
rm -rf $(TOOLCHAIN_EXTERNAL_DIR)/bfin-linux-uclibc
mv $(TOOLCHAIN_EXTERNAL_DIR)/bfin-uclinux $(TOOLCHAIN_EXTERNAL_DIR)/tmp
@@ -249,7 +249,7 @@
$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE)
mkdir -p $(@D)
- $(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE))) $^ | $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) $(TAR_OPTIONS) -
+ $(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE))) $^ | $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) $(TAR_OPTIONS) -
$(Q)touch $@
endif
^ permalink raw reply [flat|nested] 9+ messages in thread* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-08-02 18:33 ` [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains Thomas De Schampheleire
@ 2011-08-19 9:11 ` Thomas De Schampheleire
0 siblings, 0 replies; 9+ messages in thread
From: Thomas De Schampheleire @ 2011-08-19 9:11 UTC (permalink / raw)
To: buildroot
On Tue, Aug 2, 2011 at 8:33 PM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> Using the system tar with --strip-components causes problems when its version is too old (<1.17). Even recent releases of RedHat/CentOS still ship with tar 1.15. To avoid such problems, always use host-tar instead of tar to unpack toolchains.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>
> ---
> To be honest, I'm not sure what the best way is to handle this. The same problems with tar could occur in other places than just external toolchains. So maybe we should use host-tar in more places.
> On the other hand, most users have a recent, working, tar version on there system. It may thus make more sense to detect the system tar version and only use host-tar if it's too old.
>
> diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/ext-tool.mk
> --- a/toolchain/toolchain-external/ext-tool.mk
> +++ b/toolchain/toolchain-external/ext-tool.mk
> @@ -155,7 +155,7 @@
> ?endif
>
> ?ifeq ($(BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD),y)
> -TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(TOOLCHAIN_EXTERNAL_DIR)/.extracted
> +TOOLCHAIN_EXTERNAL_DEPENDENCIES = host-tar $(TOOLCHAIN_EXTERNAL_DIR)/.extracted
It turns out that this has the annoying side-effect of re-running the
'ext-toolchain-installed' step each time, because host-tar is not an
existing file. One can workaround this as follows:
include package/tar/tar.mk
$(STAMP_DIR)/tar-installed:
$(BUILD_DIR)/host-tar-$(TAR_VERSION)/.stamp_host_installed
$(Q)touch $@
TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(STAMP_DIR)tar-installed
$(TOOLCHAIN_EXTERNAL_DIR)/.extracted
This is not very beautiful because you need to include
package/tar/tar.mk to get to $(TAR_VERSION). Is there a better way?
Thanks,
Thomas
> ?else
> ?TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(STAMP_DIR)/ext-toolchain-checked
> ?endif
> @@ -227,9 +227,9 @@
> ?$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2)
> ? ? ? ?mkdir -p $(@D)
> ? ? ? ?$(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE_1))) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) | \
> - ? ? ? ? ? ? ? $(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
> + ? ? ? ? ? ? ? $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
> ? ? ? ?$(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE_2))) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2) | \
> - ? ? ? ? ? ? ? $(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
> + ? ? ? ? ? ? ? $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
> ?ifeq ($(TOOLCHAIN_EXTERNAL_PREFIX),bfin-uclinux)
> ? ? ? ?rm -rf $(TOOLCHAIN_EXTERNAL_DIR)/bfin-linux-uclibc
> ? ? ? ?mv $(TOOLCHAIN_EXTERNAL_DIR)/bfin-uclinux $(TOOLCHAIN_EXTERNAL_DIR)/tmp
> @@ -249,7 +249,7 @@
>
> ?$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE)
> ? ? ? ?mkdir -p $(@D)
> - ? ? ? $(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE))) $^ | $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) $(TAR_OPTIONS) -
> + ? ? ? $(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE))) $^ | $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) $(TAR_OPTIONS) -
> ? ? ? ?$(Q)touch $@
> ?endif
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread