[Buildroot] New kernel source mode [Proposal]

[Buildroot] New kernel source mode [Proposal]

[Przemyslaw Wrzos]

Hi,

 

I'd like to get some feedback on a feature I'd like to contribute to the
buildroot project. I'm not quite sure what the process is so I thought
I'd start on the mailing list.

 

 

PROPOSAL:  Allow a directory (containing the source tree) to be used as
the source for the kernel build. The source isn't copied to the output
directory but instead the kernel's outputmakfile wrapper facility is
used.

 

RATIONALE:  In my project, changes to the kernel are frequent enough
that it is ideal to be able to update the kernel source tree and then
build directly from it.

 

IMPLEMENTATION:  In addition to providing a new source for the linux
build process itself, some package makefiles will need to be updated as
they currently don't differentiate between the kernel source tree and
build tree.

 

 

I have this working already though I still need to update some of the
packages as mentioned above. I wanted to ask if there would be any
interest in rolling this into the buildroot project upstream.

 

Thanks for your feedback.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130115/2bcaf14f/attachment.html>

[Buildroot] New kernel source mode [Proposal]

[Stephan Hoffmann]

Am 15.01.2013 02:12, schrieb Przemyslaw Wrzos:
>
> Hi,
>
>  
>
> I'd like to get some feedback on a feature I'd like to contribute to
> the buildroot project. I'm not quite sure what the process is so I
> thought I'd start on the mailing list.
>
>  
>
>  
>
> PROPOSAL:  Allow a directory (containing the source tree) to be used
> as the source for the kernel build. The source isn't copied to the
> output directory but instead the kernel's outputmakfile wrapper
> facility is used.
>
>  
>
Hello,

do you mean the kernel's out of tree build feature?

> RATIONALE:  In my project, changes to the kernel are frequent enough
> that it is ideal to be able to update the kernel source tree and then
> build directly from it.
>

Isn't this almost the same as can be achieved using the "Source
Override" feature?

But the advantages of your proposal would be saving some time on each
build and some hard disk space. Especially when using huge kernel git
archives.
>
>  
>
> IMPLEMENTATION:  In addition to providing a new source for the linux
> build process itself, some package makefiles will need to be updated
> as they currently don't differentiate between the kernel source tree
> and build tree.
>
>  
>
>  
>
> I have this working already though I still need to update some of the
> packages as mentioned above. I wanted to ask if there would be any
> interest in rolling this into the buildroot project upstream.
>
If you send your patch set I'll give it a try, but I cannot judge the
impact of buildroot's overall structure.

Kind regards

Stephan
>
>  
>
> Thanks for your feedback.
>
>
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130115/2fd7397d/attachment.html>

[Buildroot] New kernel source mode [Proposal]

[Przemyslaw Wrzos]

Here's the patch I mentioned that will allow Linux to be built out of
tree, from an existing source directory.

 

This patch doesn't correct for the packages that may require direct
access to the source tree (and which assume it's found under LINUX_DIR)
but I can do that later if it's decided there's any merit to the patch
I'm proposing here.

 

I'm going to submit a separate suggestion for the generic out of tree
build system discussed by Thomas Petazzoni, but that is going to be
fairly compatible with what I have here so there's no reason both
suggestions can't be examined.

 

I appreciate your feedback.

 

Cheers,

Przemyslaw

 

From: Stephan Hoffmann [mailto:sho at relinux.de] 
Sent: Tuesday, 15 January 2013 18:51
To: Przemyslaw Wrzos
Cc: buildroot at busybox.net
Subject: Re: [Buildroot] New kernel source mode [Proposal]

 

Am 15.01.2013 02:12, schrieb Przemyslaw Wrzos:

	Hi,

	 

	I'd like to get some feedback on a feature I'd like to
contribute to the buildroot project. I'm not quite sure what the process
is so I thought I'd start on the mailing list.

	 

	 

	PROPOSAL:  Allow a directory (containing the source tree) to be
used as the source for the kernel build. The source isn't copied to the
output directory but instead the kernel's outputmakfile wrapper facility
is used.

	 

Hello,

do you mean the kernel's out of tree build feature?




RATIONALE:  In my project, changes to the kernel are frequent enough
that it is ideal to be able to update the kernel source tree and then
build directly from it.


Isn't this almost the same as can be achieved using the "Source
Override" feature?

But the advantages of your proposal would be saving some time on each
build and some hard disk space. Especially when using huge kernel git
archives.



 

IMPLEMENTATION:  In addition to providing a new source for the linux
build process itself, some package makefiles will need to be updated as
they currently don't differentiate between the kernel source tree and
build tree.

 

 

I have this working already though I still need to update some of the
packages as mentioned above. I wanted to ask if there would be any
interest in rolling this into the buildroot project upstream.

If you send your patch set I'll give it a try, but I cannot judge the
impact of buildroot's overall structure.

Kind regards

Stephan



 

Thanks for your feedback.






_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130117/0d384e28/attachment.html>

[Buildroot] New kernel source mode [Proposal]

[Przemyslaw Wrzos]

And of course I forgot to include the patch. Here it is.

 

Cheers,

Przemyslaw

 

From: Przemyslaw Wrzos 
Sent: Thursday, 17 January 2013 17:26
To: 'Stephan Hoffmann'
Cc: buildroot at busybox.net
Subject: RE: [Buildroot] New kernel source mode [Proposal]

 

Here's the patch I mentioned that will allow Linux to be built out of
tree, from an existing source directory.

 

This patch doesn't correct for the packages that may require direct
access to the source tree (and which assume it's found under LINUX_DIR)
but I can do that later if it's decided there's any merit to the patch
I'm proposing here.

 

I'm going to submit a separate suggestion for the generic out of tree
build system discussed by Thomas Petazzoni, but that is going to be
fairly compatible with what I have here so there's no reason both
suggestions can't be examined.

 

I appreciate your feedback.

 

Cheers,

Przemyslaw

 

From: Stephan Hoffmann [mailto:sho at relinux.de] 
Sent: Tuesday, 15 January 2013 18:51
To: Przemyslaw Wrzos
Cc: buildroot at busybox.net
Subject: Re: [Buildroot] New kernel source mode [Proposal]

 

Am 15.01.2013 02:12, schrieb Przemyslaw Wrzos:

	Hi,

	 

	I'd like to get some feedback on a feature I'd like to
contribute to the buildroot project. I'm not quite sure what the process
is so I thought I'd start on the mailing list.

	 

	 

	PROPOSAL:  Allow a directory (containing the source tree) to be
used as the source for the kernel build. The source isn't copied to the
output directory but instead the kernel's outputmakfile wrapper facility
is used.

	 

Hello,

do you mean the kernel's out of tree build feature?



RATIONALE:  In my project, changes to the kernel are frequent enough
that it is ideal to be able to update the kernel source tree and then
build directly from it.


Isn't this almost the same as can be achieved using the "Source
Override" feature?

But the advantages of your proposal would be saving some time on each
build and some hard disk space. Especially when using huge kernel git
archives.

 

IMPLEMENTATION:  In addition to providing a new source for the linux
build process itself, some package makefiles will need to be updated as
they currently don't differentiate between the kernel source tree and
build tree.

 

 

I have this working already though I still need to update some of the
packages as mentioned above. I wanted to ask if there would be any
interest in rolling this into the buildroot project upstream.

If you send your patch set I'll give it a try, but I cannot judge the
impact of buildroot's overall structure.

Kind regards

Stephan

 

Thanks for your feedback.





_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130117/8c7ea3e8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: linux-dir-source.patch
Type: application/octet-stream
Size: 3914 bytes
Desc: linux-dir-source.patch
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130117/8c7ea3e8/attachment-0001.obj>

[Buildroot] New kernel source mode [Proposal]

[Przemyslaw Wrzos]

Hi Stephen,

Just an update to the patch I sent in yesterday. Between trying to get
both the generic and non-generic versions working I seem to have
introduced a bug at some point into my proposed patch for the linux
kernel. I'm including an updated version for you to look at (just a
missing line that caused the download step to try to download a
nonsensical non-existent file).

One other thing I should note about the patch which I didn't elaborate
on yesterday. After digging around in the linux source code there
doesn't seem to be a way to perform the buildroot_defconfig make step in
an out of tree build so I had to settle for simply copying the
configuration file across.

Cheers,
Przemyslaw Wrzos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: linux-dir-source.patch
Type: application/octet-stream
Size: 3931 bytes
Desc: linux-dir-source.patch
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130118/85d7026f/attachment.obj>

[Buildroot] New kernel source mode [Proposal]

[Stephan Hoffmann]

Am 18.01.2013 02:10, schrieb Przemyslaw Wrzos:
> Hi Stephen,
>
> Just an update to the patch I sent in yesterday. Between trying to get
> both the generic and non-generic versions working I seem to have
> introduced a bug at some point into my proposed patch for the linux
> kernel. I'm including an updated version for you to look at (just a
> missing line that caused the download step to try to download a
> nonsensical non-existent file).
Hello Przemyslaw,

I think I don't have the time to try it out before next week.

Regards

Stephan
>
> One other thing I should note about the patch which I didn't elaborate
> on yesterday. After digging around in the linux source code there
> doesn't seem to be a way to perform the buildroot_defconfig make step in
> an out of tree build so I had to settle for simply copying the
> configuration file across.
>
> Cheers,
> Przemyslaw Wrzos

[Buildroot] New kernel source mode [Proposal]

[Przemyslaw Wrzos]

> -----Original Message-----
> From: Stephan Hoffmann [mailto:sho at relinux.de]
> Sent: Friday, 18 January 2013 19:14
> To: Przemyslaw Wrzos
> Cc: buildroot at busybox.net
> Subject: Re: [Buildroot] New kernel source mode [Proposal]
> 
> Am 18.01.2013 02:10, schrieb Przemyslaw Wrzos:
> > Hi Stephen,
> >
> > Just an update to the patch I sent in yesterday. Between trying to
get
> > both the generic and non-generic versions working I seem to have
> > introduced a bug at some point into my proposed patch for the linux
> > kernel. I'm including an updated version for you to look at (just a
> > missing line that caused the download step to try to download a
> > nonsensical non-existent file).
> Hello Przemyslaw,
> 
> I think I don't have the time to try it out before next week.
> 
> Regards
> 
> Stephan
> >
> > One other thing I should note about the patch which I didn't
> elaborate
> > on yesterday. After digging around in the linux source code there
> > doesn't seem to be a way to perform the buildroot_defconfig make
> step
> > in an out of tree build so I had to settle for simply copying the
> > configuration file across.
> >
> > Cheers,
> > Przemyslaw Wrzos

No worries, let me know if you have any dramas once you look at it.

Cheers,
Przemyslaw Wrzos

[Buildroot] New kernel source mode [Proposal]

[Thomas Petazzoni]

Dear Przemyslaw Wrzos,

On Tue, 15 Jan 2013 12:12:54 +1100, Przemyslaw Wrzos wrote:

> I'd like to get some feedback on a feature I'd like to contribute to the
> buildroot project. I'm not quite sure what the process is so I thought
> I'd start on the mailing list.
> 
> PROPOSAL:  Allow a directory (containing the source tree) to be used as
> the source for the kernel build. The source isn't copied to the output
> directory but instead the kernel's outputmakfile wrapper facility is
> used.
> 
> RATIONALE:  In my project, changes to the kernel are frequent enough
> that it is ideal to be able to update the kernel source tree and then
> build directly from it.
> 
> IMPLEMENTATION:  In addition to providing a new source for the linux
> build process itself, some package makefiles will need to be updated as
> they currently don't differentiate between the kernel source tree and
> build tree.
> 
> I have this working already though I still need to update some of the
> packages as mentioned above. I wanted to ask if there would be any
> interest in rolling this into the buildroot project upstream.

Thanks a lot for your detailed proposal! This is indeed a request that
has shown up a number of times, and that we need to address.

However, I've always believed that solving this by specifically
modifying the kernel .mk file in Buildroot (linux/linux.mk) was not the
right solution. Instead, we want a solution that potentially works for
all packages in Buildroot.

In, fact the <pkg>_OVERRIDE_SRCDIR mechanism was the first step in this
direction. For now, this mechanism is a little bit annoying because it
does rsync the entire source tree, which for the kernel tree is long
and useless. So, I've started experimenting this direction:

 (1) Build all packages in Buildroot out-of-tree. The packages
     for which the source code is downloaded, extracted and patched by
     Buildroot (so things you are not actively developing on) are built
     out-of-tree. The source directory is shared between the host and
     target builds of a same package. That's the first step.

     So, for each package, the source code is extracted in
     output/sources/<pkg>-<version>/, and the build takes places in
     output/build/<pkg>-<version>/ for target packages, and
     output/build/host-<pkg>-<version>/ for host packages.

     Of course since some packages do not support out-of-tree, we have
     for those packages to copy the source tree to the build directory.
     But usually, only very small projects don't support out-of-tree
     build, and also using out-of-tree build in Buildroot may encourage
     us to push upstream patches to make out-of-tree build work.

 (2) Logically, packages whose source directory is overriden using
    <pkg>_OVERRIDE_SRCDIR will work easily. The only difference with
    normal packages would be that the source directory is no longer
    output/sources/<pkg>-<version>/, but the value passed in
    <pkg>_OVERRIDE_SRCDIR.

I already have a beginning of prototype for step (1), with:

 * Busybox being built out-of-tree
 * All autotools packages being built out-of-tree
 * All generic packages that do not explicitly say they can built
   out-of-tree see their source code copied from the source directory
   to the build directory.

But there were still more cases to cover, so this work is not finished
for now.

I now I've been pushing back kernel-specific solutions for a while, and
maybe this is discouraging contributors and developers. I do believe
that a kernel-specific solution is not the right way of solving this
problem, and I'm working on a solution, but the generic solution is
more complex to develop than the kernel specific one. That said, if
other Buildroot developers feel that the need for this feature at the
kernel level is important enough, we can merge a kernel-specific
solution and later replace it with a more generic solution, but I fear
that this might break the API as both solutions would not work the same
way.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] New kernel source mode [Proposal]

[Jeremy Rosen]

> 
>      So, for each package, the source code is extracted in
>      output/sources/<pkg>-<version>/, and the build takes places in
>      output/build/<pkg>-<version>/ for target packages, and
>      output/build/host-<pkg>-<version>/ for host packages.
> 

just a small detail but...

could you consider building in output/build/target instead of directly int
output/build/ for the target ?

1) the host/ part won't be hidden in the middle of all sorts of directories that are package specific
2) it would avoid confusion in the case we somday include a utility named "host"

    Regards

    J?r?my Rosen

fight key loggers : write some perl using vim

[Buildroot] New kernel source mode [Proposal]

[Thomas Petazzoni]

Dear Jeremy Rosen,

On Tue, 15 Jan 2013 09:18:14 +0100 (CET), Jeremy Rosen wrote:

> >      So, for each package, the source code is extracted in
> >      output/sources/<pkg>-<version>/, and the build takes places in
> >      output/build/<pkg>-<version>/ for target packages, and
> >      output/build/host-<pkg>-<version>/ for host packages.
> > 
> 
> just a small detail but...
> 
> could you consider building in output/build/target instead of directly int
> output/build/ for the target ?

Hum, why? Building in output/build/<pkgname>-<pkgversion>/ is already
what we do today.

> 1) the host/ part won't be hidden in the middle of all sorts of directories that are package specific

Hum? I didn't suggest a host/ part. My proposal, in terms of build
directories, do not change anything compared to what we have today:

	output/build/host-foo-1.0/ for the host version of package foo
	output/build/foo-1.0/ for the target version of package foo

That's what we do today.

> 2) it would avoid confusion in the case we somday include a utility named "host"

It would be:

	output/build/host-host-4.2/
	output/build/host-4.2/

But that's the same thing today. I am not sure there is an upstream
project called simply "host", so at this point, I don't see the need to
change the naming of those directories.

Thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] New kernel source mode [Proposal]

[Jeremy Rosen]

> 
> Hum? I didn't suggest a host/ part. My proposal, in terms of build
> directories, do not change anything compared to what we have today:
> 
> 	output/build/host-foo-1.0/ for the host version of package foo
> 	output/build/foo-1.0/ for the target version of package foo
> 

oh, my bad then I misread it as:


 	output/build/host/foo-1.0/ for the host version of package foo
 	output/build/foo-1.0/ for the target version of package foo


nevermind my comment then....

[Buildroot] New kernel source mode [Proposal]

[Thomas Petazzoni]

Dear Jeremy Rosen,

On Tue, 15 Jan 2013 10:13:08 +0100 (CET), Jeremy Rosen wrote:

> > Hum? I didn't suggest a host/ part. My proposal, in terms of build
> > directories, do not change anything compared to what we have today:
> > 
> > 	output/build/host-foo-1.0/ for the host version of package foo
> > 	output/build/foo-1.0/ for the target version of package foo
> 
> oh, my bad then I misread it as:
> 
> 
>  	output/build/host/foo-1.0/ for the host version of package foo
>  	output/build/foo-1.0/ for the target version of package foo
> 
> nevermind my comment then....

Ok. That said, your comment still stands in that if we have a package
named "host", things will be confusing. But it will already be
confusing as things are today, and I'm not sure we want to fix this
potential corner case anyway.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] New kernel source mode [Proposal]

[Jeremy Rosen]

well, I just checked on my main debian system and apparently host is a standard DNS lookup utility...

oh well, as you said it's enough of a border case not to worry too much....

    Cordialement

    J?r?my Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> Dear Jeremy Rosen,
> 
> On Tue, 15 Jan 2013 10:13:08 +0100 (CET), Jeremy Rosen wrote:
> 
> > > Hum? I didn't suggest a host/ part. My proposal, in terms of
> > > build
> > > directories, do not change anything compared to what we have
> > > today:
> > > 
> > > 	output/build/host-foo-1.0/ for the host version of package foo
> > > 	output/build/foo-1.0/ for the target version of package foo
> > 
> > oh, my bad then I misread it as:
> > 
> > 
> >  	output/build/host/foo-1.0/ for the host version of package foo
> >  	output/build/foo-1.0/ for the target version of package foo
> > 
> > nevermind my comment then....
> 
> Ok. That said, your comment still stands in that if we have a package
> named "host", things will be confusing. But it will already be
> confusing as things are today, and I'm not sure we want to fix this
> potential corner case anyway.
> 
> Thomas
> --
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> 

[Buildroot] New kernel source mode [Proposal]

[Arnout Vandecappelle]

On 15/01/13 10:27, Jeremy Rosen wrote:
> well, I just checked on my main debian system and apparently host is a standard DNS lookup utility...

  Yes, buildroot has it too: it's part of the 'bind' package.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] New kernel source mode [Proposal]

[Przemyslaw Wrzos]

Hi Thomas,

> -----Original Message-----
> From: Thomas Petazzoni [mailto:thomas.petazzoni at free-electrons.com]
> Sent: Tuesday, 15 January 2013 18:55
> 
> Dear Przemyslaw Wrzos,
> 
> Thanks a lot for your detailed proposal! This is indeed a request that
has
> shown up a number of times, and that we need to address.
> 
> However, I've always believed that solving this by specifically
modifying the
> kernel .mk file in Buildroot (linux/linux.mk) was not the right
solution.
> Instead, we want a solution that potentially works for all packages in
> Buildroot.
> 
> In, fact the <pkg>_OVERRIDE_SRCDIR mechanism was the first step in
this
> direction. For now, this mechanism is a little bit annoying because it
does
> rsync the entire source tree, which for the kernel tree is long and
useless.
> So, I've started experimenting this direction:
> 
>  (1) Build all packages in Buildroot out-of-tree. The packages
>      for which the source code is downloaded, extracted and patched by
>      Buildroot (so things you are not actively developing on) are
built
>      out-of-tree. The source directory is shared between the host and
>      target builds of a same package. That's the first step.
> 
>      So, for each package, the source code is extracted in
>      output/sources/<pkg>-<version>/, and the build takes places in
>      output/build/<pkg>-<version>/ for target packages, and
>      output/build/host-<pkg>-<version>/ for host packages.
> 
>      Of course since some packages do not support out-of-tree, we have
>      for those packages to copy the source tree to the build
directory.
>      But usually, only very small projects don't support out-of-tree
>      build, and also using out-of-tree build in Buildroot may
encourage
>      us to push upstream patches to make out-of-tree build work.
> 
>  (2) Logically, packages whose source directory is overriden using
>     <pkg>_OVERRIDE_SRCDIR will work easily. The only difference with
>     normal packages would be that the source directory is no longer
>     output/sources/<pkg>-<version>/, but the value passed in
>     <pkg>_OVERRIDE_SRCDIR.
> 
> I already have a beginning of prototype for step (1), with:
> 
>  * Busybox being built out-of-tree
>  * All autotools packages being built out-of-tree
>  * All generic packages that do not explicitly say they can built
>    out-of-tree see their source code copied from the source directory
>    to the build directory.
> 
> But there were still more cases to cover, so this work is not finished
for now.
> 
> I now I've been pushing back kernel-specific solutions for a while,
and
> maybe this is discouraging contributors and developers. I do believe
that a
> kernel-specific solution is not the right way of solving this problem,
and I'm
> working on a solution, but the generic solution is more complex to
develop
> than the kernel specific one. That said, if other Buildroot developers
feel
> that the need for this feature at the kernel level is important
enough, we
> can merge a kernel-specific solution and later replace it with a more
generic
> solution, but I fear that this might break the API as both solutions
would not
> work the same way.
> 
> Best regards,
> 
> Thomas

That's an interesting idea. I personally would suggest we shouldn't
force this behaviour on packages but instead provide an infrastructure
that allows them to use it if they so choose.

If you think that is reasonable then I have put together a patch that
will achieve something along those lines. Obviously some of the naming
might need to be fixed up and other issues resolved but it's a good
place to get the conversation started. I have included two patches, one
with the proposed generic out-of-tree building mechanism. The other
shows how linux/linux.mk would need to be altered to support this
scheme.

Some general things worth mentioning about the patch:
    - In addition to <PKG>_DIR I have also added the <PKG>_SRC variable
to point to the location of the source directory in the build output.
    - Two new variables are added to provide information to the generic
build infrastructure:
        - <PKG>_OUTOFTREE_BUILD can be set to YES or NO to indicate an
out of tree build
        - <PKG>_OUTOFTREE_SETUP_CMDS setting this automatically turns on
<PKG>_OUTOFTREE_BUILD, it includes the commands needed to prep the build
directory from the source directory
    - The <PKG>_OVERRIDE_SRCDIR works together with the out of tree
build, meaning we can specify an external directory and everything still
works as expected
    - By default if the out of tree build is not enabled then the build
continues to work as before which I think is ideal because then packages
can be ported gradually to the new scheme, or not ported at all since it
might not make any sense for some
    - I have also added a new variable <PKG>_REINSTALL_TARGETS so that
package maintainers can specify targets used to reinstall package
results into the target tree, this is mostly separate to the rest of
this patch and perhaps should be considered separately

The linux specific part of the patch is missing fixes for other packages
that might be unaware it now has a separate source and build directory,
I could fix that if you decide this approach is the way to go.

Anyway, I'd appreciate your feedback on whether you think this approach
is viable and if it fits with what you had in mind.

Cheers,
Przemyslaw Wrzos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 001-Generic-out-of-tree-build.patch
Type: application/octet-stream
Size: 9454 bytes
Desc: 001-Generic-out-of-tree-build.patch
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130117/4f1eabe0/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 002-Linux-out-of-tree-build.patch
Type: application/octet-stream
Size: 4056 bytes
Desc: 002-Linux-out-of-tree-build.patch
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130117/4f1eabe0/attachment-0001.obj>

[Buildroot] New kernel source mode [Proposal]

[Steve Calfee]

On Mon, Jan 14, 2013 at 5:12 PM, Przemyslaw Wrzos
<przemyslaw.wrzos@calyptech.com> wrote:
> Hi,
>
> I?d like to get some feedback on a feature I?d like to contribute to the
> buildroot project. I?m not quite sure what the process is so I thought I?d
> start on the mailing list.
>
> PROPOSAL:  Allow a directory (containing the source tree) to be used as the
> source for the kernel build. The source isn?t copied to the output directory
> but instead the kernel?s outputmakfile wrapper facility is used.
>
> RATIONALE:  In my project, changes to the kernel are frequent enough that it
> is ideal to be able to update the kernel source tree and then build directly
> from it.
>
> IMPLEMENTATION:  In addition to providing a new source for the linux build
> process itself, some package makefiles will need to be updated as they
> currently don?t differentiate between the kernel source tree and build tree.
>
> I have this working already though I still need to update some of the
> packages as mentioned above. I wanted to ask if there would be any interest
> in rolling this into the buildroot project upstream.
>

Hi, I am not opposed to a simple integration of kernel development
into buildroot.

However, the last time I did kernel hacking a few years ago with
buildroot, we just did not build the kernel with BR. The greatly
speeds up a BR build after a clean (If you also use an external
toolchain).

We had the kernel tree in an external git tree where we did standard
cross builds using the same compilers used by the BR. A simple script
to copy uImage and the modules to the nfs filesystem where BR was
extracted made a testable system.

So until Thomas gets his generic external package development working,
this is a handy way to split kernel development from rootfs
development.

Regards, Steve

[Buildroot] New kernel source mode [Proposal]

[Peter Korsgaard]

>>>>> "Steve" == Steve Calfee <stevecalfee@gmail.com> writes:

Hi,

 Steve> Hi, I am not opposed to a simple integration of kernel development
 Steve> into buildroot.

 Steve> However, the last time I did kernel hacking a few years ago with
 Steve> buildroot, we just did not build the kernel with BR. The greatly
 Steve> speeds up a BR build after a clean (If you also use an external
 Steve> toolchain).

 Steve> We had the kernel tree in an external git tree where we did standard
 Steve> cross builds using the same compilers used by the BR. A simple script
 Steve> to copy uImage and the modules to the nfs filesystem where BR was
 Steve> extracted made a testable system.

I work somewhat similar. I build the kernel by hand while I'm developing
on it (working on a driver or whatever), and the same for any custom
applications, and only use buildroot as an integration tool once the
individual components are more-or-less stable to ensure everything is
under version control and can be reproduced.

-- 
Bye, Peter Korsgaard

[Buildroot] New kernel source mode [Proposal]

[Arnout Vandecappelle]

On 15/01/13 02:12, Przemyslaw Wrzos wrote:
> Hi,
>
> I?d like to get some feedback on a feature I?d like to contribute to the
> buildroot project. I?m not quite sure what the process is so I thought
> I?d start on the mailing list.
>
> PROPOSAL:  Allow a directory (containing the source tree) to be used as
> the source for the kernel build. The source isn?t copied to the output
> directory but instead the kernel?s outputmakfile wrapper facility is used.
>

  A patch with similar functionality was posted in August by Sonic Zhang:

http://patchwork.ozlabs.org/patch/177618/

  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F