[Buildroot] [PATCH] linux: support "local" as download method

[Buildroot] [PATCH] linux: support "local" as download method

[Tzu-Jung Lee]

Signed-off-by: Tzu-Jung Lee <tjlee@ambarella.com>
---
?linux/Config.in | ? 10 ++++++++++
?linux/linux.mk ?| ? ?3 +++
?2 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/linux/Config.in b/linux/Config.in
index dc6aa73..4053c41 100644
--- a/linux/Config.in
+++ b/linux/Config.in
@@ -53,6 +53,12 @@ config BR2_LINUX_KERNEL_CUSTOM_GIT
? ? ? ? ?This option allows Buildroot to get the Linux kernel source
? ? ? ? ?code from a Git repository.

+config BR2_LINUX_KERNEL_CUSTOM_LOCAL
+ ? ? ? bool "Custom local source tree"
+ ? ? ? help
+ ? ? ? ? This option allows Buildroot to get the Linux kernel source
+ ? ? ? ? code from a local repository.
+
?endchoice

?config BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE
@@ -72,6 +78,10 @@ config BR2_LINUX_KERNEL_CUSTOM_GIT_VERSION
? ? ? ?string "Custom Git version"
? ? ? ?depends on BR2_LINUX_KERNEL_CUSTOM_GIT

+config BR2_LINUX_KERNEL_CUSTOM_LOCAL_PATH
+ ? ? ? string "Custom Local path"
+ ? ? ? depends on BR2_LINUX_KERNEL_CUSTOM_LOCAL
+
?config BR2_LINUX_KERNEL_VERSION
? ? ? ?string
? ? ? ?default "3.1.4" if BR2_LINUX_KERNEL_3_1
diff --git a/linux/linux.mk b/linux/linux.mk
index 5c5a1d2..d5617a2 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -13,6 +13,9 @@ LINUX_SOURCE = $(notdir $(LINUX_TARBALL))
?else ifeq ($(BR2_LINUX_KERNEL_CUSTOM_GIT),y)
?LINUX_SITE = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_GIT_REPO_URL))
?LINUX_SITE_METHOD = git
+else ifeq ($(BR2_LINUX_KERNEL_CUSTOM_LOCAL),y)
+LINUX_SITE = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_LOCAL_PATH))
+LINUX_SITE_METHOD = local
?else
?LINUX_SOURCE = linux-$(LINUX_VERSION).tar.bz2
?# In X.Y.Z, get X and Y. We replace dots and dashes by spaces in order
--
1.7.8

[Buildroot] [PATCH] linux: support "local" as download method

[Thomas Petazzoni]

Hello,

What this patch does is already possible through the "override source
directory" mechanism, which is available for every package.

It's true that the Linux package is special in that it is very likely
that many people will want to use a local source directory, but I'm
fearing that we will end up adding many of those
"<something>_CUSTOM_LOCAL" option that just duplicate what is possible
thanks to the override source directory mechanism.

See http://free-electrons.com/blog/buildroot-2011-11/ for more details
on this source directory override mechanism.

Best regards,

Thomas

Le Wed, 7 Dec 2011 20:09:20 +0800,
Tzu-Jung Lee <roylee17@gmail.com> a ?crit :

> Signed-off-by: Tzu-Jung Lee <tjlee@ambarella.com>
> ---
> ?linux/Config.in | ? 10 ++++++++++
> ?linux/linux.mk ?| ? ?3 +++
> ?2 files changed, 13 insertions(+), 0 deletions(-)
> 
> diff --git a/linux/Config.in b/linux/Config.in
> index dc6aa73..4053c41 100644
> --- a/linux/Config.in
> +++ b/linux/Config.in
> @@ -53,6 +53,12 @@ config BR2_LINUX_KERNEL_CUSTOM_GIT
> ? ? ? ? ?This option allows Buildroot to get the Linux kernel source
> ? ? ? ? ?code from a Git repository.
> 
> +config BR2_LINUX_KERNEL_CUSTOM_LOCAL
> + ? ? ? bool "Custom local source tree"
> + ? ? ? help
> + ? ? ? ? This option allows Buildroot to get the Linux kernel source
> + ? ? ? ? code from a local repository.
> +
> ?endchoice
> 
> ?config BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE
> @@ -72,6 +78,10 @@ config BR2_LINUX_KERNEL_CUSTOM_GIT_VERSION
> ? ? ? ?string "Custom Git version"
> ? ? ? ?depends on BR2_LINUX_KERNEL_CUSTOM_GIT
> 
> +config BR2_LINUX_KERNEL_CUSTOM_LOCAL_PATH
> + ? ? ? string "Custom Local path"
> + ? ? ? depends on BR2_LINUX_KERNEL_CUSTOM_LOCAL
> +
> ?config BR2_LINUX_KERNEL_VERSION
> ? ? ? ?string
> ? ? ? ?default "3.1.4" if BR2_LINUX_KERNEL_3_1
> diff --git a/linux/linux.mk b/linux/linux.mk
> index 5c5a1d2..d5617a2 100644
> --- a/linux/linux.mk
> +++ b/linux/linux.mk
> @@ -13,6 +13,9 @@ LINUX_SOURCE = $(notdir $(LINUX_TARBALL))
> ?else ifeq ($(BR2_LINUX_KERNEL_CUSTOM_GIT),y)
> ?LINUX_SITE = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_GIT_REPO_URL))
> ?LINUX_SITE_METHOD = git
> +else ifeq ($(BR2_LINUX_KERNEL_CUSTOM_LOCAL),y)
> +LINUX_SITE = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_LOCAL_PATH))
> +LINUX_SITE_METHOD = local
> ?else
> ?LINUX_SOURCE = linux-$(LINUX_VERSION).tar.bz2
> ?# In X.Y.Z, get X and Y. We replace dots and dashes by spaces in order
> --
> 1.7.8
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH] linux: support "local" as download method

[Tzu-Jung Lee]

Hi Thomas,

On Fri, Dec 9, 2011 at 5:30 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> What this patch does is already possible through the "override source
> directory" mechanism, which is available for every package.
>
> It's true that the Linux package is special in that it is very likely
> that many people will want to use a local source directory, but I'm
> fearing that we will end up adding many of those
> "<something>_CUSTOM_LOCAL" option that just duplicate what is possible
> thanks to the override source directory mechanism.

I tried both LINUX_OVERRIDE_SRCDIR and LINUX_CUSTOM_LOCAL, and sent the
patch for the latter in case someone else might end up doing the same thing.

But soon I realized that our team may need another METHOD to support
out use case.
Have we considered IN-PLACE build alternative or option?


For most of the packages, it is good enough for users/developers to
leverage the power of buildroot to

    fetch, ..., configure, build, mkimage

However for some packages, ex: linux kernel, that a team or company
actually works on,

   modify, configure, build, mkimage

has much higher frequency than the former case.

I tried to find a "OVERRIDE" ways to do this haven't worked out yet.
Any comments or suggestion will be appreciated.

Do I need to extend the following code for support IN-PLACE case?

ifeq ($$($(2)_OVERRIDE_SRCDIR),)
# In the normal case (no package override), the sequence of steps is
#  source, by downloading
#  depends
#  extract
#  patch
#  configure
#  build
else if <OVERRIDE CASE>
# In the package override case, the sequence of steps
#  source, by rsyncing
#  depends
#  configure
# build

else if <IN-PLACE CASE>
#  depends
#  configure
#  build

> See http://free-electrons.com/blog/buildroot-2011-11/ for more details
> on this source directory override mechanism.

Actually, it was this post that brought me to buildroot again :)

Thanks,
Roy

[Buildroot] [PATCH] linux: support "local" as download method

[Thomas Petazzoni]

Le Fri, 9 Dec 2011 10:44:24 +0800,
Tzu-Jung Lee <roylee17@gmail.com> a ?crit :

> But soon I realized that our team may need another METHOD to support
> out use case.
> Have we considered IN-PLACE build alternative or option?

> 
> 
> For most of the packages, it is good enough for users/developers to
> leverage the power of buildroot to
> 
>     fetch, ..., configure, build, mkimage
> 
> However for some packages, ex: linux kernel, that a team or company
> actually works on,
> 
>    modify, configure, build, mkimage
> 
> has much higher frequency than the former case.

The "override source directory" mechanism allows this, as explained in
the blog post at http://free-electrons.com/blog/buildroot-2011-11/.

Basically, the first time you will do the build, Buildroot will do a
full copy of the Linux kernel sources to the Buildroot build directory.
But then, subsequent calls to "make linux-rebuild" will only :

 * Do a "rsync" of the build tree, so it will be very fast

 * Restart the build process (will only rebuild what you have changed)

 * Re-run the installation process

So this is exactly what you want.

Supporting "in-place" building in the infrastructure is simply not
possible. Buildring directory in the source directory cannot work,
because some packages needs to be built *twice* : once for the host,
once for the target. So we definitely need two build directories for
those packages. Of course, this is not the case for the Linux kernel,
which is always built for the target in Buildroot, but the package
infrastructure is common to all packages, so we have to take into
account this constraint.

One possibility is to support out-of-tree building, which we discussed
at the latest Buildroot meeting in Prague, see
http://lists.busybox.net/pipermail/buildroot/2011-November/047229.html.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH] linux: support "local" as download method

[Arnout Vandecappelle]

On Friday 09 December 2011 03:44:24 Tzu-Jung Lee wrote:
> > It's true that the Linux package is special in that it is very likely
> > that many people will want to use a local source directory, but I'm
> > fearing that we will end up adding many of those
> > "<something>_CUSTOM_LOCAL" option that just duplicate what is possible
> > thanks to the override source directory mechanism.
> 
> I tried both LINUX_OVERRIDE_SRCDIR and LINUX_CUSTOM_LOCAL, and sent the
> patch for the latter in case someone else might end up doing the same thing.

 You can actually achieve the same from the command line:

make LINUX_SITE=/... LINUX_SITE_METHOD=file linux

 Regards,
 Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43