[Buildroot] oddity with project_ vs. kernel dir

[Buildroot] oddity with project_ vs. kernel dir

[Bernhard Fischer]

Ulf,

I'm trying to build a kernel with openswan (for i386, for example).
Everything works as expected when generating and installing the
kernel-headers ??) but the, when unpacking/patching the kernel for the
target, soomebody seems to try to apply a duplicate patch:

rm -rf
/scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc/linux-2.6.22.6
*** Unpacking kernel source
bzcat /scratch/obj.i686/buildroot_trunk/down/linux-2.6.22.6.tar.bz2 |
tar -C /scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc   -xf -
touch /scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc/linux-2.6.22.6/.unpacked
toolchain/patch-kernel.sh /scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc/linux-2.6.22.6
toolchain/kernel-headers \
                linux-2.6.22.6-\*.patch{,.gz,.bz2}
toolchain/patch-kernel.sh /scratch/obj.i686/buildroot_trunk/toolchain_build_i386/linux-2.6.22.6
package/openswan \
                linux-2.6.22.6-\*.patch{,.gz,.bz2}

Applying linux-2.6.22.6-openswan-2.4.9.kernel-2.6-klips.patch using
plaintext: 
The next patch would create the file README.openswan-2,
which already exists!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
(a bit line-wrapped, sorry).

Does that sound familiar to you?

??): 
touch
/scratch/obj.i686/buildroot_trunk/toolchain_build_i386/linux-2.6.22.6/.unp
acked
toolchain/patch-kernel.sh
/scratch/obj.i686/buildroot_trunk/toolchain_build_i386
/linux-2.6.22.6 toolchain/kernel-headers \
                linux-2.6.22.6-\*.patch{,.gz,.bz2}
toolchain/patch-kernel.sh
/scratch/obj.i686/buildroot_trunk/toolchain_build_i386
/linux-2.6.22.6 package/openswan \
                linux-2.6.22.6-\*.patch{,.gz,.bz2}

Applying linux-2.6.22.6-openswan-2.4.9.kernel-2.6-klips.patch using
plaintext: 
patching file README.openswan-2
patching file crypto/ciphers/aes/test_main.c
patching file crypto/ciphers/aes/test_main_mac.c
patching file include/crypto/aes.h
[snip rest of stuff working fine for kernel-headers]
patching file net/ipsec/Makefile.ver

Applying linux-2.6.22.6-openswan-2.4.9.kernel-2.6-natt.patch using
plaintext: 
patching file include/net/xfrmudp.h
patching file net/ipv4/Kconfig
patching file net/ipv4/udp.c
touch /scratch/obj.i686/buildroot_trunk/toolchain_build_i386/linux-2.6.22.6/.patched

[Buildroot] oddity with project_ vs. kernel dir

[Bernhard Fischer]

On Tue, Sep 18, 2007 at 06:30:08PM +0200, Bernhard Fischer wrote:
>
>Ulf,
>
>I'm trying to build a kernel with openswan (for i386, for example).
>Everything works as expected when generating and installing the
>kernel-headers ??) but the, when unpacking/patching the kernel for the
>target, soomebody seems to try to apply a duplicate patch:
>
>rm -rf
>/scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc/linux-2.6.22.6
>*** Unpacking kernel source
>bzcat /scratch/obj.i686/buildroot_trunk/down/linux-2.6.22.6.tar.bz2 |
>tar -C /scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc   -xf -
>touch /scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc/linux-2.6.22.6/.unpacked
>toolchain/patch-kernel.sh /scratch/obj.i686/buildroot_trunk/project_build_i386/uclibc/linux-2.6.22.6
>toolchain/kernel-headers \
>                linux-2.6.22.6-\*.patch{,.gz,.bz2}
>toolchain/patch-kernel.sh /scratch/obj.i686/buildroot_trunk/toolchain_build_i386/linux-2.6.22.6
>package/openswan \
>                linux-2.6.22.6-\*.patch{,.gz,.bz2}
>
>Applying linux-2.6.22.6-openswan-2.4.9.kernel-2.6-klips.patch using
>plaintext: 
>The next patch would create the file README.openswan-2,
>which already exists!  Assume -R? [n] 
>Apply anyway? [n] 
>Skipping patch.
>(a bit line-wrapped, sorry).
>
>Does that sound familiar to you?

Ah, yes. Has to be KERNEL26_DIR there.
 19671        ulf ifeq ($(BR2_PACKAGE_OPENSWAN),y)
 19671        ulf       toolchain/patch-kernel.sh $(LINUX_HEADERS_UNPACK_DIR) package/openswan \
 19671        ulf 	linux-$(LINUX26_VERSION)-\*.patch{,.gz,.bz2}
 19671        ulf endif

fixing this up..

[Buildroot] using PROJECTs with buildroot

[Bernhard Fischer]

Hi,

I'd like to ask how that project thing is supposed to be used and how it
could be fixed (assuming that i'm not doing something wrong, which could
easily be).

Suppose i want to create two images (or BSP's like some call those).

To keep things simple, let's assume that i'm using the same version of
binutils and gcc for all.


BR2_PROJECT=imageA: kernel 2.6.22.6 with LFS support on
BR2_PROJECT=imageB: kernel 2.6.22.1 without LFS

0) I 'rm -rf *_i386 binar*'
1) I configure and build imageA. So far so good.
2) I configure and build imageB. The resulting image looks like this:
$ tar -tvf binaries/imageB/rootfs.i386-20070919.tar | grep "2.6.22../$"
drwxr-xr-x root/root         0 2007-09-19 19:05 ./lib/modules/2.6.22.6/
$ egrep "(2_6_22|PROJECT)" .config
BR2_PROJECT="imageB"
BR2_KERNEL_HEADERS_2_6_22_1=y
# BR2_KERNEL_HEADERS_2_6_22 is not set

Not exactly what i'd have expected.
Perhaps i made a mistake? If so please explain how that would work
properly?



A more general question that i already raised is what the
project_build_arch dir is supposed to be good for. I didn't get an
answer on this question, i think.

I'm questioning it's usefulness because if done properly, nearly all
packages would have to be built there, leaving the "good old" build_arch
directory completely empty.

Your reasoning seems to be along the lines of:
"A package that is customized on a per project basis has to be built in
project_build_arch".

While this may be true, customization that qualifies for building in
project_build_arch dir includes these toggles:

1) LFS on/off
2a) locale on/off
2b) wchar on/off
3) IPv6 on/off

These alone would lead to the build_arch dir to be nearly empty, but
let's assume that it would still contain 25% of the packages since those
would be locale+wchar+IPv6+LFS agnostic, fine.

If you now take into account that there is a potential
4) TARGET_CFLAGS
then suddenly project_build_arch dir would be the exact same thing
build_arch was before, leaving build_arch empty (the exact same thing we
had before that project thing, just back then the project_dir was the
empty -- non-existing one -- and build_arch was populated).

It would be very kind if you would share your thoughts on these issues.

regards,
Bernhard

[Buildroot] using PROJECTs with buildroot

[Ulf Samuelsson]

m?n 2007-09-24 klockan 14:31 +0200 skrev Bernhard Fischer:
> Ulf?
> 
> Did you receive this mail?
> 
> On Fri, Sep 21, 2007 at 05:33:40PM +0200, Bernhard Fischer wrote:
> >Hi,
> >
> >I'd like to ask how that project thing is supposed to be used and how it
> >could be fixed (assuming that i'm not doing something wrong, which could
> >easily be).
> >
> >Suppose i want to create two images (or BSP's like some call those).
> >
> >To keep things simple, let's assume that i'm using the same version of
> >binutils and gcc for all.
> >
> >
> >BR2_PROJECT=imageA: kernel 2.6.22.6 with LFS support on
> >BR2_PROJECT=imageB: kernel 2.6.22.1 without LFS
> >
> >0) I 'rm -rf *_i386 binar*'
> >1) I configure and build imageA. So far so good.
> >2) I configure and build imageB. The resulting image looks like this:
> >$ tar -tvf binaries/imageB/rootfs.i386-20070919.tar | grep "2.6.22../$"
> >drwxr-xr-x root/root         0 2007-09-19 19:05 ./lib/modules/2.6.22.6/
> >$ egrep "(2_6_22|PROJECT)" .config
> >BR2_PROJECT="imageB"
> >BR2_KERNEL_HEADERS_2_6_22_1=y
> ># BR2_KERNEL_HEADERS_2_6_22 is not set
> >
> >Not exactly what i'd have expected.
> >Perhaps i made a mistake? If so please explain how that would work
> >properly?
> >

The project thingy will allow you to share a toolchain
with one set of kernel headers.
You can't have two set of kernel headers.
Then you need to build a separate toolchain.

It looks to me that the user has changed the kernel headers version
between imageA and imageB.
That is illegal, then he needs to rebuild the toolset.

Instead, he should change LINUX26_VERSION
which needs to be done outside menuconfig

> >
> >
> >A more general question that i already raised is what the
> >project_build_arch dir is supposed to be good for. I didn't get an
> >answer on this question, i think.
> >
> >I'm questioning it's usefulness because if done properly, nearly all
> >packages would have to be built there, leaving the "good old" build_arch
> >directory completely empty.
> >
> >Your reasoning seems to be along the lines of:
> >"A package that is customized on a per project basis has to be built in
> >project_build_arch".
> >
> >While this may be true, customization that qualifies for building in
> >project_build_arch dir includes these toggles:
> >
> >1) LFS on/off
> >2a) locale on/off
> >2b) wchar on/off
> >3) IPv6 on/off

No, these are global stuff which affects the toolchain.
If you have several versions of these, then you need
to have several toolchains.

I keep these constant, and have only a couple of packages in
project_build_arch, like linux, busybox, u-boot and at91-bootstrap.
I build up to 7 boards from the same build_arch and it is much much
faster.

The "old" way would require you to have 7 different trees.


> >These alone would lead to the build_arch dir to be nearly empty, but
> >let's assume that it would still contain 25% of the packages since those
> >would be locale+wchar+IPv6+LFS agnostic, fine.
> >

It is more like 95% in practice.
I have a feeling you don't use the feature, so why argue.


> >If you now take into account that there is a potential
> >4) TARGET_CFLAGS
> >then suddenly project_build_arch dir would be the exact same thing
> >build_arch was before, leaving build_arch empty (the exact same thing we
> >had before that project thing, just back then the project_dir was the
> >empty -- non-existing one -- and build_arch was populated).
> >
> >It would be very kind if you would share your thoughts on these issues.
> >
BR
Ulf Samuelsson


> >regards,
> >Bernhard
> >_______________________________________________
> >buildroot mailing list
> >buildroot at uclibc.org
> >http://busybox.net/mailman/listinfo/buildroot
> >

[Buildroot] using PROJECTs with buildroot

[Bernhard Fischer]

On Mon, Sep 24, 2007 at 04:43:29PM +0200, Ulf Samuelsson wrote:
>m?n 2007-09-24 klockan 14:31 +0200 skrev Bernhard Fischer:

>> On Fri, Sep 21, 2007 at 05:33:40PM +0200, Bernhard Fischer wrote:
>> >Hi,
>> >
>> >I'd like to ask how that project thing is supposed to be used and how it
>> >could be fixed (assuming that i'm not doing something wrong, which could
>> >easily be).
>> >
>> >Suppose i want to create two images (or BSP's like some call those).
>> >
>> >To keep things simple, let's assume that i'm using the same version of
>> >binutils and gcc for all.
>> >
>> >
>> >BR2_PROJECT=imageA: kernel 2.6.22.6 with LFS support on
>> >BR2_PROJECT=imageB: kernel 2.6.22.1 without LFS
>> >
>> >0) I 'rm -rf *_i386 binar*'
>> >1) I configure and build imageA. So far so good.
>> >2) I configure and build imageB. The resulting image looks like this:
>> >$ tar -tvf binaries/imageB/rootfs.i386-20070919.tar | grep "2.6.22../$"
>> >drwxr-xr-x root/root         0 2007-09-19 19:05 ./lib/modules/2.6.22.6/
>> >$ egrep "(2_6_22|PROJECT)" .config
>> >BR2_PROJECT="imageB"
>> >BR2_KERNEL_HEADERS_2_6_22_1=y
>> ># BR2_KERNEL_HEADERS_2_6_22 is not set
>> >
>> >Not exactly what i'd have expected.
>> >Perhaps i made a mistake? If so please explain how that would work
>> >properly?
>> >
>
>The project thingy will allow you to share a toolchain
>with one set of kernel headers.
>You can't have two set of kernel headers.

Ah. Can you think of a non-intrusive way to prevent this (or is that in
the docs already? I admit that i didn't look, but this should be
mentioned in a very prominent place).

>Then you need to build a separate toolchain.
>
>It looks to me that the user has changed the kernel headers version
>between imageA and imageB.
>That is illegal, then he needs to rebuild the toolset.

Ok. I was under the impression that the project was ment to build
distinct projects (as in very distinct, not just differing in the
package-selection). Thanks for the explanation.

[Buildroot] using PROJECTs with buildroot

[Ulf Samuelsson]

m?n 2007-09-24 klockan 20:48 +0200 skrev Bernhard Fischer:
> On Mon, Sep 24, 2007 at 04:43:29PM +0200, Ulf Samuelsson wrote:
> >m?n 2007-09-24 klockan 14:31 +0200 skrev Bernhard Fischer:
> 
> >> On Fri, Sep 21, 2007 at 05:33:40PM +0200, Bernhard Fischer wrote:
> >> >Hi,
> >> >
> >> >I'd like to ask how that project thing is supposed to be used and how it
> >> >could be fixed (assuming that i'm not doing something wrong, which could
> >> >easily be).
> >> >
> >> >Suppose i want to create two images (or BSP's like some call those).
> >> >
> >> >To keep things simple, let's assume that i'm using the same version of
> >> >binutils and gcc for all.
> >> >
> >> >
> >> >BR2_PROJECT=imageA: kernel 2.6.22.6 with LFS support on
> >> >BR2_PROJECT=imageB: kernel 2.6.22.1 without LFS
> >> >
> >> >0) I 'rm -rf *_i386 binar*'
> >> >1) I configure and build imageA. So far so good.
> >> >2) I configure and build imageB. The resulting image looks like this:
> >> >$ tar -tvf binaries/imageB/rootfs.i386-20070919.tar | grep "2.6.22../$"
> >> >drwxr-xr-x root/root         0 2007-09-19 19:05 ./lib/modules/2.6.22.6/
> >> >$ egrep "(2_6_22|PROJECT)" .config
> >> >BR2_PROJECT="imageB"
> >> >BR2_KERNEL_HEADERS_2_6_22_1=y
> >> ># BR2_KERNEL_HEADERS_2_6_22 is not set
> >> >
> >> >Not exactly what i'd have expected.
> >> >Perhaps i made a mistake? If so please explain how that would work
> >> >properly?
> >> >
> >
> >The project thingy will allow you to share a toolchain
> >with one set of kernel headers.
> >You can't have two set of kernel headers.
> 
> Ah. Can you think of a non-intrusive way to prevent this (or is that in
> the docs already? I admit that i didn't look, but this should be
> mentioned in a very prominent place).
> 

Once you have generated your compiler, then you should not change
the values. This is not new.

If you compile some packages with an OABI toolchain, and then 
you select an external toolchain, and compile some packages
with EABI, then there could be some issues during run time.
There is nothing that prevents you from doing that (I think)...

One way of doing that, is to save the original tool configuration
and compare every time against that.

You could also have a separate config file for the toolchain, 
(like you have for busybox and uclibc)
and make all the packages depend on that toolchain .config file
and rebuild all packages if that .config changes.
That is probably the cleanest but you need to make toolchain-menuconfig.

> >Then you need to build a separate toolchain.
> >
> >It looks to me that the user has changed the kernel headers version
> >between imageA and imageB.
> >That is illegal, then he needs to rebuild the toolset.
> 
> Ok. I was under the impression that the project was ment to build
> distinct projects (as in very distinct, not just differing in the
> package-selection). Thanks for the explanation.
> 

You should be able to build two different projects where the kernel
version differs, but then you have to accept that your kernel
headers are somewhat off.
If you buy a commercially supported gcc you get a certain kernel 
header version, and build against that, so again, nothing new.

BR
Ulf Samuelsson.