[Buildroot] linux: why is vmlinux only available for mips?

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

Hi,

linux/Config.in contains the following configuration option:

config BR2_LINUX_KERNEL_VMLINUX
        bool "vmlinux"
        depends on BR2_mips || BR2_mipsel

The depend statement causes the vmlinux target only to be available
for mips processors.
For powerpc for example, only uImage and zImage are selectable.

However, I'd like to have vmlinux in the output/images directory.
First of all because I am directly loading this on target, rather than
uImage or zImage.
Secondly, right now my wrapper scripts go and fetch the vmlinux
executable in output/build/linux-<version>/vmlinux, which has the
disadvantage of needing to know what version of Linux was configured
in buildroot. I'd like to have a way to get the vmlinux file from
whichever version was configured in buildroot. Making sure it ends up
in output/images seems the simplest way.

I patched the Config.in file to read:

config BR2_LINUX_KERNEL_VMLINUX
        bool "vmlinux"
        depends on BR2_mips || BR2_mipsel || BR2_powerpc

which works fine.
I would even assume that every target can benefit from this option.

Moreover, why don't we allow multiple targets? For example, in some
projects you have different ways of loading your target, e.g. u-boot
(uImage) and directly copying to memory with a probe (in which case
vmlinux would be better suited).

Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

On Fri, Jul 15, 2011 at 8:20 AM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> Hi,
>
> linux/Config.in contains the following configuration option:
>
> config BR2_LINUX_KERNEL_VMLINUX
> ? ? ? ?bool "vmlinux"
> ? ? ? ?depends on BR2_mips || BR2_mipsel
>
> The depend statement causes the vmlinux target only to be available
> for mips processors.
> For powerpc for example, only uImage and zImage are selectable.
>
> However, I'd like to have vmlinux in the output/images directory.
> First of all because I am directly loading this on target, rather than
> uImage or zImage.
> Secondly, right now my wrapper scripts go and fetch the vmlinux
> executable in output/build/linux-<version>/vmlinux, which has the
> disadvantage of needing to know what version of Linux was configured
> in buildroot. I'd like to have a way to get the vmlinux file from
> whichever version was configured in buildroot. Making sure it ends up
> in output/images seems the simplest way.

In fact I have more problems with this version encoding in the output
build/linux-X directory:
* I need to get to the vmlinux file (mentioned above)
* I need to get to the dtc (device tree compiler) which resides in
<linux>/scripts/dtc/dtc
* I need to be able to compile external kernel modules, so in that
Makefile I need to point to the right kernel directory.

If you stick with one kernel version, then you can hardcode it in the
relevant scripts and Makefiles. However, when switching to another
version, you have to update all of these files.
One approach to solve all of this would be to create a symbolic link
output/build/linux, pointing to the output/build/linux-X directory
corresponding to the actually configured version.

What do you think of that?

>
> I patched the Config.in file to read:
>
> config BR2_LINUX_KERNEL_VMLINUX
> ? ? ? ?bool "vmlinux"
> ? ? ? ?depends on BR2_mips || BR2_mipsel || BR2_powerpc
>
> which works fine.
> I would even assume that every target can benefit from this option.
>
> Moreover, why don't we allow multiple targets? For example, in some
> projects you have different ways of loading your target, e.g. u-boot
> (uImage) and directly copying to memory with a probe (in which case
> vmlinux would be better suited).
>

[Buildroot] linux: why is vmlinux only available for mips?

[Yann E. MORIN]

Thomas, All,

On Friday 15 July 2011 08:34:34 Thomas De Schampheleire wrote:
> If you stick with one kernel version, then you can hardcode it in the
> relevant scripts and Makefiles. However, when switching to another
> version, you have to update all of these files.
> One approach to solve all of this would be to create a symbolic link
> output/build/linux, pointing to the output/build/linux-X directory
> corresponding to the actually configured version.
> 
> What do you think of that?

Or you can use the BR2_LINUX_KERNEL_VERSION variable:
  output/build/linux-${BR2_LINUX_KERNEL_VERSION}/

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

On Fri, Jul 15, 2011 at 11:22 AM, Yann E. MORIN
<yann.morin.1998@anciens.enib.fr> wrote:
> Thomas, All,
>
> On Friday 15 July 2011 08:34:34 Thomas De Schampheleire wrote:
>> If you stick with one kernel version, then you can hardcode it in the
>> relevant scripts and Makefiles. However, when switching to another
>> version, you have to update all of these files.
>> One approach to solve all of this would be to create a symbolic link
>> output/build/linux, pointing to the output/build/linux-X directory
>> corresponding to the actually configured version.
>>
>> What do you think of that?
>
> Or you can use the BR2_LINUX_KERNEL_VERSION variable:
> ?output/build/linux-${BR2_LINUX_KERNEL_VERSION}/

This works fine in the context of buildroot. However, I need to access
these files from a non-buildroot context (external makefiles/scripts)
and so the variable BR2_LINUX_KERNEL_VERSION is not known.

Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Peter Korsgaard]

>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> writes:

 >> Or you can use the BR2_LINUX_KERNEL_VERSION variable:
 >> ?output/build/linux-${BR2_LINUX_KERNEL_VERSION}/

 Thomas> This works fine in the context of buildroot. However, I need to access
 Thomas> these files from a non-buildroot context (external makefiles/scripts)
 Thomas> and so the variable BR2_LINUX_KERNEL_VERSION is not known.

You can always use wildcards - E.G. output/build/linux-*/scripts/..

-- 
Bye, Peter Korsgaard

[Buildroot] linux: why is vmlinux only available for mips?

[Daniele Salvatore Albano]

Il 15/07/2011 11:34, Thomas De Schampheleire ha scritto:
> This works fine in the context of buildroot. However, I need to access
> these files from a non-buildroot context (external makefiles/scripts)
> and so the variable BR2_LINUX_KERNEL_VERSION is not known.
>
> Thomas

If you're using *sh scripts, you can just do something like
. /PATH/TO/BUILDROOT/.config

to access buildroot config variables so you can use that variable

If you're using make file i don't know, but there is a way to do it too, 
sure :)

In the end, if you're using anithing else you can try to parse using 
regular expressions (needs delimiters and/or backslashes around quotes)
BR2_LINUX_KERNEL_VERSION="?([a-zA-Z0-9\.\-]+)"?

or cat, grep and cut
cat .config | grep BR2_LINUX_KERNEL_VERSION | cut -d"\"" -f2


Best Regards,
Daniele

-- 
____________________________________________________________
                                         |
Daniele Salvatore Albano                | web site:
  IT Consultant                          | http://www.itechcon.it
  Webapps Design and Development         |
  Software Engineer and Developer        | e-mail:
  Linux Servers SetUp And Administration | info at itechcon.it
  Embedded Network Solutions             | d.albano at itechcon.it

Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in 
questo messaggio sono riservate ed a uso esclusivo del destinatario. 
Qualora il messaggio in parola Le fosse pervenuto per errore, La 
invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, 
dandocene gentilmente comunicazione. Grazie.
Pursuant to Legislative Decree No. 196/2003, you are hereby informed 
that this message contains confidential information intended only for 
the use of the addressee. If you are not the addressee, and have 
received this message by mistake, please delete it and immediately 
notify us. You may not copy or disseminate this message to anyone. Thank 
you.

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

On Fri, Jul 15, 2011 at 11:38 AM, Peter Korsgaard <jacmet@uclibc.org> wrote:
>>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> writes:
>
> ?>> Or you can use the BR2_LINUX_KERNEL_VERSION variable:
> ?>> ?output/build/linux-${BR2_LINUX_KERNEL_VERSION}/
>
> ?Thomas> This works fine in the context of buildroot. However, I need to access
> ?Thomas> these files from a non-buildroot context (external makefiles/scripts)
> ?Thomas> and so the variable BR2_LINUX_KERNEL_VERSION is not known.
>
> You can always use wildcards - E.G. output/build/linux-*/scripts/..

That doesn't work if you started out with one version, say 2.6.34,
then step up to another version 2.6.35 in the same buildroot
environment. The wildcard would match both, while you want only the
active one.
Forcing the developer to remove the old linux directory is not
user-friendly IMO.

The patch I currently applied to fix this is:
diff --git a/linux/linux.mk b/linux/linux.mk
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -175,7 +175,11 @@ endif
        fi
        $(Q)touch $@

-linux linux26: host-module-init-tools $(LINUX26_DEPENDENCIES)
$(LINUX26_DIR)/.stamp_installed
+# Generic 'linux' symlink
+$(BUILD_DIR)/linux: $(LINUX26_DIR)/.stamp_installed
+       ln -snf $(LINUX26_DIR) $@
+
+linux linux26: host-module-init-tools $(LINUX26_DEPENDENCIES)
$(LINUX26_DIR)/.stamp_installed $(BUILD_DIR)/linux

 linux-menuconfig linux-xconfig linux-gconfig linux-nconfig
linux26-menuconfig linux26-xconfig linux26-gconfig linux26-nconfig:
dirs $(LINUX26_DIR)/.stamp_configured
        $(MAKE) $(LINUX26_MAKE_FLAGS) -C $(LINUX26_DIR) \

Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

On Fri, Jul 15, 2011 at 11:46 AM, Daniele Salvatore Albano
<info@daccii.it> wrote:
> Il 15/07/2011 11:34, Thomas De Schampheleire ha scritto:
>>
>> This works fine in the context of buildroot. However, I need to access
>> these files from a non-buildroot context (external makefiles/scripts)
>> and so the variable BR2_LINUX_KERNEL_VERSION is not known.
>>
>> Thomas
>
> If you're using *sh scripts, you can just do something like
> . /PATH/TO/BUILDROOT/.config
>
> to access buildroot config variables so you can use that variable
>
> If you're using make file i don't know, but there is a way to do it too,
> sure :)
>
> In the end, if you're using anithing else you can try to parse using regular
> expressions (needs delimiters and/or backslashes around quotes)
> BR2_LINUX_KERNEL_VERSION="?([a-zA-Z0-9\.\-]+)"?
>
> or cat, grep and cut
> cat .config | grep BR2_LINUX_KERNEL_VERSION | cut -d"\"" -f2

This is indeed a good suggestion that I didn't think of before.
For Makefiles, you can also include the .config file as-is; in fact,
this is how the buildroot Makefile does it.
I will try and see if this solves all my use-cases.

Thanks!
Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas Petazzoni]

Le Fri, 15 Jul 2011 08:34:34 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :

> > The depend statement causes the vmlinux target only to be available
> > for mips processors.
> > For powerpc for example, only uImage and zImage are selectable.
> >
> > However, I'd like to have vmlinux in the output/images directory.
> > First of all because I am directly loading this on target, rather than
> > uImage or zImage.

If vmlinux is useful on powerpc platforms, then it should be added to
the list of architectures for which vmlinux is proposed.

> In fact I have more problems with this version encoding in the output
> build/linux-X directory:
> * I need to get to the vmlinux file (mentioned above)

Just need to add "|| BR2_powerpc" as you suggested.

> * I need to get to the dtc (device tree compiler) which resides in
> <linux>/scripts/dtc/dtc

This tool looks like a useful host tool built by the kernel. Maybe we
should install it in $(HOST_DIR)/usr/bin ?

> * I need to be able to compile external kernel modules, so in that
> Makefile I need to point to the right kernel directory.

Ok. Those could be integrated as Buildroot packages, in which case
getting access to the Linux source directory is easy.

> If you stick with one kernel version, then you can hardcode it in the
> relevant scripts and Makefiles. However, when switching to another
> version, you have to update all of these files.
> One approach to solve all of this would be to create a symbolic link
> output/build/linux, pointing to the output/build/linux-X directory
> corresponding to the actually configured version.

I think I'd prefer something like :

$ make <pkg>-show-srcdir

which would be implemented at the package infrastructure, and which
would return the source directory of a particular package. This way,
you can easily ask where a given package source code has been extracted
by Buildroot.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] linux: why is vmlinux only available for mips?

[Yann E. MORIN]

Thomas and Thomas, All,

On Friday 15 July 2011 14:30:53 Thomas Petazzoni wrote:
> Le Fri, 15 Jul 2011 08:34:34 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> > * I need to get to the dtc (device tree compiler) which resides in
> > <linux>/scripts/dtc/dtc
> 
> This tool looks like a useful host tool built by the kernel. Maybe we
> should install it in $(HOST_DIR)/usr/bin ?
> 
> > * I need to be able to compile external kernel modules, so in that
> > Makefile I need to point to the right kernel directory.
> 
> Ok. Those could be integrated as Buildroot packages, in which case
> getting access to the Linux source directory is easy.

Or use the 'build' and 'source' symlinks in the (rootfs)/lib/modules/(version)
Of course, it still requires the version string, which Daniele provided
a very simple way to get it.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

Hi,

On Fri, Jul 15, 2011 at 2:30 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Fri, 15 Jul 2011 08:34:34 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> > The depend statement causes the vmlinux target only to be available
>> > for mips processors.
>> > For powerpc for example, only uImage and zImage are selectable.
>> >
>> > However, I'd like to have vmlinux in the output/images directory.
>> > First of all because I am directly loading this on target, rather than
>> > uImage or zImage.
>
> If vmlinux is useful on powerpc platforms, then it should be added to
> the list of architectures for which vmlinux is proposed.
>
>> In fact I have more problems with this version encoding in the output
>> build/linux-X directory:
>> * I need to get to the vmlinux file (mentioned above)
>
> Just need to add "|| BR2_powerpc" as you suggested.

I'm not sure why you want to restrict this to specific architectures
instead of allowing it for all.
What is wrong with the following?

diff --git a/linux/Config.in b/linux/Config.in
--- a/linux/Config.in
+++ b/linux/Config.in
@@ -131,7 +131,6 @@ config BR2_LINUX_KERNEL_VMLINUX_BIN

 config BR2_LINUX_KERNEL_VMLINUX
 	bool "vmlinux"
-	depends on BR2_mips || BR2_mipsel

 config BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM
 	bool "custom target"


It will be up to the user/developer to decide what kind of output he
wants. The linux kernel Makefile will always generate vmlinux. Whether
it is of direct use for a certain architecture is another question.
Most developers will select e.g. uImage for use with U-boot, but some
(like us) may want to do our own transformations on vmlinux to package
it in another way (e.g. with a proprietary bootloader).

>
>> * I need to get to the dtc (device tree compiler) which resides in
>> <linux>/scripts/dtc/dtc
>
> This tool looks like a useful host tool built by the kernel. Maybe we
> should install it in $(HOST_DIR)/usr/bin ?

Yes, I agree. How do you do that?

>
>> * I need to be able to compile external kernel modules, so in that
>> Makefile I need to point to the right kernel directory.
>
> Ok. Those could be integrated as Buildroot packages, in which case
> getting access to the Linux source directory is easy.

Good point. I'll have to add Mercurial support in buildroot to get there though.

>
>> If you stick with one kernel version, then you can hardcode it in the
>> relevant scripts and Makefiles. However, when switching to another
>> version, you have to update all of these files.
>> One approach to solve all of this would be to create a symbolic link
>> output/build/linux, pointing to the output/build/linux-X directory
>> corresponding to the actually configured version.
>
> I think I'd prefer something like :
>
> $ make <pkg>-show-srcdir
>
> which would be implemented at the package infrastructure, and which
> would return the source directory of a particular package. This way,
> you can easily ask where a given package source code has been extracted
> by Buildroot.

I like this as well. It's simple and generic.
I'm not sure whether it still is needed after I include the .config though.

Thanks,
Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Quotient Remainder]

Thomas,

On Wed, 2011-07-20 at 15:38 +0200, Thomas De Schampheleire wrote:
> >
> >> * I need to be able to compile external kernel modules, so in that
> >> Makefile I need to point to the right kernel directory.
> >
> > Ok. Those could be integrated as Buildroot packages, in which case
> > getting access to the Linux source directory is easy.
> 
> Good point. I'll have to add Mercurial support in buildroot to get there though.

I may be misinterpreting, but does the above mean that you have a decent
generic external kernel module build structure going?  I have a terribly
hacky setup working here but it's completely incompatible with the
recent linux GENTARGETS approach.
Care to share your external module build stuff?

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas Petazzoni]

Le Wed, 20 Jul 2011 15:38:14 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :

> > Just need to add "|| BR2_powerpc" as you suggested.
> 
> I'm not sure why you want to restrict this to specific architectures
> instead of allowing it for all.
> What is wrong with the following?

At least on x86 and ARM, "vmlinux" does not exist in arch/<ARCH>/boot/,
so allowing "vmlinux" would simply trigger a failure at build time.

> It will be up to the user/developer to decide what kind of output he
> wants. The linux kernel Makefile will always generate vmlinux.

No. The "vmlinux" that the Linux kernel generates in its top directory
is not a valid bootable image on architectures such as x86 and ARM. And
for those architectures, there is no "vmlinux" file in
arch/<ARCH>/boot/.

> Whether it is of direct use for a certain architecture is another question.
> Most developers will select e.g. uImage for use with U-boot, but some
> (like us) may want to do our own transformations on vmlinux to package
> it in another way (e.g. with a proprietary bootloader).

Which vmlinux are you using ? The one at the top directory of the
kernel source tree, or the one in arch/<ARCH>/boot ?

> >> * I need to get to the dtc (device tree compiler) which resides in
> >> <linux>/scripts/dtc/dtc
> >
> > This tool looks like a useful host tool built by the kernel. Maybe we
> > should install it in $(HOST_DIR)/usr/bin ?
> 
> Yes, I agree. How do you do that?

You can hijack any of the installation steps of the kernel to install
this tool to $(HOST_DIR)/usr/bin. But technically speaking, this
wouldn't respect the convention of the package infrastructure: the
"linux" package is a target package, and it is therefore not supposed
to install anything in $(HOST_DIR).

> >> * I need to be able to compile external kernel modules, so in that
> >> Makefile I need to point to the right kernel directory.
> >
> > Ok. Those could be integrated as Buildroot packages, in which case
> > getting access to the Linux source directory is easy.
> 
> Good point. I'll have to add Mercurial support in buildroot to get there though.

At some point, I had patches to add Mercurial support, but I guess they
got lost together with my previous laptop :-(

> > I think I'd prefer something like :
> >
> > $ make <pkg>-show-srcdir
> >
> > which would be implemented at the package infrastructure, and which
> > would return the source directory of a particular package. This way,
> > you can easily ask where a given package source code has been extracted
> > by Buildroot.
> 
> I like this as well. It's simple and generic.
> I'm not sure whether it still is needed after I include the .config though.

Well, if you don't need it anymore, then let's keep the idea for
another time, when someone will actually need it for something. It
should be just 3 lines of code to add to the package infrastructure.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas Petazzoni]

Le Wed, 20 Jul 2011 17:10:12 +0100,
Quotient Remainder <quotientvremainder@gmail.com> a ?crit :

> I may be misinterpreting, but does the above mean that you have a decent
> generic external kernel module build structure going?  I have a terribly
> hacky setup working here but it's completely incompatible with the
> recent linux GENTARGETS approach.
> Care to share your external module build stuff?

Could you detail what problems you have, and give an example of an
external kernel module for which the GENTARGETS approach is a problem ?
The linux-fusion package in Buildroot is an external kernel module and
seems to build fine.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

On Wed, Jul 20, 2011 at 6:10 PM, Quotient Remainder
<quotientvremainder@gmail.com> wrote:
> Thomas,
>
> On Wed, 2011-07-20 at 15:38 +0200, Thomas De Schampheleire wrote:
>> >
>> >> * I need to be able to compile external kernel modules, so in that
>> >> Makefile I need to point to the right kernel directory.
>> >
>> > Ok. Those could be integrated as Buildroot packages, in which case
>> > getting access to the Linux source directory is easy.
>>
>> Good point. I'll have to add Mercurial support in buildroot to get there though.
>
> I may be misinterpreting, but does the above mean that you have a decent
> generic external kernel module build structure going? ?I have a terribly
> hacky setup working here but it's completely incompatible with the
> recent linux GENTARGETS approach.
> Care to share your external module build stuff?

I wouldn't call it a 'generic build structure'. So far I have been
using a simple Makefile, taking one command-line variable pointing to
the buildroot 'output' directory. Based on this variable, the location
of the kernel is inferred (currently based on a 'linux' symlink I
added, as mentioned earlier), together with the compiler.

TARGETS=module1 module2
OPTIONS= -C $(BUILDROOT_OUTPUT)/build/linux M=$(PWD) ARCH=powerpc
CROSS_COMPILE=$(BUILDROOT_OUTPUT)/host/usr/bin/powerpc-linux-
EXTRA_CFLAGS += -DDEBUG

OBJECTS=$(addsuffix .o, $(TARGETS))
obj-m += $(OBJECTS)

all:
        make $(OPTIONS) modules

clean:
        make $(OPTIONS) clean

install:
        make $(OPTIONS) INSTALL_MOD_PATH=$(BUILDROOT_OUTPUT)/target
INSTALL_MOD_STRIP=1 modules_install



After the suggestions in this thread, plus the intention to have as
much of the project builds being coordinated from buildroot, I'm going
to step away from this approach though. The linux-fusion Makefile that
Thomas referred to seems like a good starting point for me.

Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

On Wed, Jul 20, 2011 at 6:17 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Wed, 20 Jul 2011 15:38:14 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> > Just need to add "|| BR2_powerpc" as you suggested.
>>
>> I'm not sure why you want to restrict this to specific architectures
>> instead of allowing it for all.
>> What is wrong with the following?
>
> At least on x86 and ARM, "vmlinux" does not exist in arch/<ARCH>/boot/,
> so allowing "vmlinux" would simply trigger a failure at build time.
>
>> It will be up to the user/developer to decide what kind of output he
>> wants. The linux kernel Makefile will always generate vmlinux.
>
> No. The "vmlinux" that the Linux kernel generates in its top directory
> is not a valid bootable image on architectures such as x86 and ARM. And
> for those architectures, there is no "vmlinux" file in
> arch/<ARCH>/boot/.
>
>> Whether it is of direct use for a certain architecture is another question.
>> Most developers will select e.g. uImage for use with U-boot, but some
>> (like us) may want to do our own transformations on vmlinux to package
>> it in another way (e.g. with a proprietary bootloader).
>
> Which vmlinux are you using ? The one at the top directory of the
> kernel source tree, or the one in arch/<ARCH>/boot ?

I am using vmlinux from the kernel top directory. In fact, this is
also where the linux.mk rules take it from:

ifeq ($(BR2_LINUX_KERNEL_VMLINUX),y)
LINUX26_IMAGE_PATH=$(LINUX26_DIR)/$(LINUX26_IMAGE_NAME)
else
ifeq ($(KERNEL_ARCH),avr32)
LINUX26_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/images/$(LINUX26_IMAGE_NAME)
else
LINUX26_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/$(LINUX26_IMAGE_NAME)
endif
endif # BR2_LINUX_KERNEL_VMLINUX

If one would want to use vmlinux from arch/<arch>/boot, then he or she
could use BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM and
BR2_LINUX_KERNEL_IMAGE_TARGET_NAME without a problem.

As you mentioned, arch/<arch>/boot doesn't contain vmlinux for all
architectures, while the top kernel directory does. Since buildroot
already has a rule to get to this package, I simply wanted to remove
the architecture dependency from it.

>
>> >> * I need to get to the dtc (device tree compiler) which resides in
>> >> <linux>/scripts/dtc/dtc
>> >
>> > This tool looks like a useful host tool built by the kernel. Maybe we
>> > should install it in $(HOST_DIR)/usr/bin ?
>>
>> Yes, I agree. How do you do that?
>
> You can hijack any of the installation steps of the kernel to install
> this tool to $(HOST_DIR)/usr/bin. But technically speaking, this
> wouldn't respect the convention of the package infrastructure: the
> "linux" package is a target package, and it is therefore not supposed
> to install anything in $(HOST_DIR).

What about making a separate dtc package? The official sources are
here, I think:
http://jdl.com/software/

>
>> >> * I need to be able to compile external kernel modules, so in that
>> >> Makefile I need to point to the right kernel directory.
>> >
>> > Ok. Those could be integrated as Buildroot packages, in which case
>> > getting access to the Linux source directory is easy.
>>
>> Good point. I'll have to add Mercurial support in buildroot to get there though.
>
> At some point, I had patches to add Mercurial support, but I guess they
> got lost together with my previous laptop :-(
>
>> > I think I'd prefer something like :
>> >
>> > $ make <pkg>-show-srcdir
>> >
>> > which would be implemented at the package infrastructure, and which
>> > would return the source directory of a particular package. This way,
>> > you can easily ask where a given package source code has been extracted
>> > by Buildroot.
>>
>> I like this as well. It's simple and generic.
>> I'm not sure whether it still is needed after I include the .config though.
>
> Well, if you don't need it anymore, then let's keep the idea for
> another time, when someone will actually need it for something. It
> should be just 3 lines of code to add to the package infrastructure.
>

Ok, that's fine.


Best regards,
Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas Petazzoni]

Hello,

Le Sun, 24 Jul 2011 13:46:40 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :

> > Which vmlinux are you using ? The one at the top directory of the
> > kernel source tree, or the one in arch/<ARCH>/boot ?
> 
> I am using vmlinux from the kernel top directory. In fact, this is
> also where the linux.mk rules take it from:
> 
> ifeq ($(BR2_LINUX_KERNEL_VMLINUX),y)
> LINUX26_IMAGE_PATH=$(LINUX26_DIR)/$(LINUX26_IMAGE_NAME)
> else
> ifeq ($(KERNEL_ARCH),avr32)
> LINUX26_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/images/$(LINUX26_IMAGE_NAME)
> else
> LINUX26_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/$(LINUX26_IMAGE_NAME)
> endif
> endif # BR2_LINUX_KERNEL_VMLINUX
> 
> If one would want to use vmlinux from arch/<arch>/boot, then he or she
> could use BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM and
> BR2_LINUX_KERNEL_IMAGE_TARGET_NAME without a problem.
> 
> As you mentioned, arch/<arch>/boot doesn't contain vmlinux for all
> architectures, while the top kernel directory does. Since buildroot
> already has a rule to get to this package, I simply wanted to remove
> the architecture dependency from it.

Ok, but typically, the vmlinux in the top directory isn't a bootable
image, as it is a pure ELF binary image. Are there
architectures/bootloaders combinations for which this format makes
sense ?

> >> >> * I need to get to the dtc (device tree compiler) which resides in
> >> >> <linux>/scripts/dtc/dtc
> >> >
> >> > This tool looks like a useful host tool built by the kernel. Maybe we
> >> > should install it in $(HOST_DIR)/usr/bin ?
> >>
> >> Yes, I agree. How do you do that?
> >
> > You can hijack any of the installation steps of the kernel to install
> > this tool to $(HOST_DIR)/usr/bin. But technically speaking, this
> > wouldn't respect the convention of the package infrastructure: the
> > "linux" package is a target package, and it is therefore not supposed
> > to install anything in $(HOST_DIR).
> 
> What about making a separate dtc package? The official sources are
> here, I think:
> http://jdl.com/software/

Ok, would indeed be nice to have a separate tarball for this. Is this
kept in sync with the version inside the kernel ?

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

On Mon, Jul 25, 2011 at 9:05 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> Le Sun, 24 Jul 2011 13:46:40 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> > Which vmlinux are you using ? The one at the top directory of the
>> > kernel source tree, or the one in arch/<ARCH>/boot ?
>>
>> I am using vmlinux from the kernel top directory. In fact, this is
>> also where the linux.mk rules take it from:
>>
>> ifeq ($(BR2_LINUX_KERNEL_VMLINUX),y)
>> LINUX26_IMAGE_PATH=$(LINUX26_DIR)/$(LINUX26_IMAGE_NAME)
>> else
>> ifeq ($(KERNEL_ARCH),avr32)
>> LINUX26_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/images/$(LINUX26_IMAGE_NAME)
>> else
>> LINUX26_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/$(LINUX26_IMAGE_NAME)
>> endif
>> endif # BR2_LINUX_KERNEL_VMLINUX
>>
>> If one would want to use vmlinux from arch/<arch>/boot, then he or she
>> could use BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM and
>> BR2_LINUX_KERNEL_IMAGE_TARGET_NAME without a problem.
>>
>> As you mentioned, arch/<arch>/boot doesn't contain vmlinux for all
>> architectures, while the top kernel directory does. Since buildroot
>> already has a rule to get to this package, I simply wanted to remove
>> the architecture dependency from it.
>
> Ok, but typically, the vmlinux in the top directory isn't a bootable
> image, as it is a pure ELF binary image. Are there
> architectures/bootloaders combinations for which this format makes
> sense ?

Well, at least in my case it does: having a proprietary boot strategy,
we take this ELF image and package it according to our bootloader's
needs.
Also: suppose you have a custom U-boot setup, and you do not want the
standard uImage that is being made by the kernel. In that case, you'd
run mkimage yourself with the custom options you need, but this
requires access to the vmlinux file as well. Having it in a fixed
place like output/images makes sense then.

>
>> >> >> * I need to get to the dtc (device tree compiler) which resides in
>> >> >> <linux>/scripts/dtc/dtc
>> >> >
>> >> > This tool looks like a useful host tool built by the kernel. Maybe we
>> >> > should install it in $(HOST_DIR)/usr/bin ?
>> >>
>> >> Yes, I agree. How do you do that?
>> >
>> > You can hijack any of the installation steps of the kernel to install
>> > this tool to $(HOST_DIR)/usr/bin. But technically speaking, this
>> > wouldn't respect the convention of the package infrastructure: the
>> > "linux" package is a target package, and it is therefore not supposed
>> > to install anything in $(HOST_DIR).
>>
>> What about making a separate dtc package? The official sources are
>> here, I think:
>> http://jdl.com/software/
>
> Ok, would indeed be nice to have a separate tarball for this. Is this
> kept in sync with the version inside the kernel ?

Honestly, I have no idea. Grant Likely (added to this thread) probably
knows the answer.

*Grant*: what is the difference between the dtc sources in the kernel
tree, and those at http://jdl.com/software/ ? Are they kept in sync?

Thanks,
Thomas

[Buildroot] linux: why is vmlinux only available for mips?

[Thomas De Schampheleire]

Hi,

On Sun, Jul 24, 2011 at 1:36 PM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> On Wed, Jul 20, 2011 at 6:10 PM, Quotient Remainder
> <quotientvremainder@gmail.com> wrote:
>> Thomas,
>>
>> On Wed, 2011-07-20 at 15:38 +0200, Thomas De Schampheleire wrote:
>>> >
>>> >> * I need to be able to compile external kernel modules, so in that
>>> >> Makefile I need to point to the right kernel directory.
>>> >
>>> > Ok. Those could be integrated as Buildroot packages, in which case
>>> > getting access to the Linux source directory is easy.
>>>
>>> Good point. I'll have to add Mercurial support in buildroot to get there though.
>>
>> I may be misinterpreting, but does the above mean that you have a decent
>> generic external kernel module build structure going? ?I have a terribly
>> hacky setup working here but it's completely incompatible with the
>> recent linux GENTARGETS approach.
>> Care to share your external module build stuff?
>
> I wouldn't call it a 'generic build structure'. So far I have been
> using a simple Makefile, taking one command-line variable pointing to
> the buildroot 'output' directory. Based on this variable, the location
> of the kernel is inferred (currently based on a 'linux' symlink I
> added, as mentioned earlier), together with the compiler.
>
> TARGETS=module1 module2
> OPTIONS= -C $(BUILDROOT_OUTPUT)/build/linux M=$(PWD) ARCH=powerpc
> CROSS_COMPILE=$(BUILDROOT_OUTPUT)/host/usr/bin/powerpc-linux-
> EXTRA_CFLAGS += -DDEBUG
>
> OBJECTS=$(addsuffix .o, $(TARGETS))
> obj-m += $(OBJECTS)
>
> all:
> ? ? ? ?make $(OPTIONS) modules
>
> clean:
> ? ? ? ?make $(OPTIONS) clean
>
> install:
> ? ? ? ?make $(OPTIONS) INSTALL_MOD_PATH=$(BUILDROOT_OUTPUT)/target
> INSTALL_MOD_STRIP=1 modules_install
>
>
>
> After the suggestions in this thread, plus the intention to have as
> much of the project builds being coordinated from buildroot, I'm going
> to step away from this approach though. The linux-fusion Makefile that
> Thomas referred to seems like a good starting point for me.

Just to follow this up: I now integrated my external kernel modules in
buildroot. I used linux-fusion as a starting point, but it seemed
overly complex to me. For example, there are variables KERNEL_VERSION,
KERNEL_BUILD, KERNEL_SOURCE and SYSROOT which may be needed
specifically for the linux-fusion package but is not generically
needed IMO.

Here is the .mk file I use:

MY_LINUX_DRIVERS_VERSION = 17e422f57c8b
MY_LINUX_DRIVERS_SOURCE = my-linux-drivers-$(MY_LINUX_DRIVERS_VERSION).tar.gz
MY_LINUX_DRIVERS_SITE = /repo/tdescham/reborn/my-linux-drivers
MY_LINUX_DRIVERS_SITE_METHOD = hg

MY_LINUX_DRIVERS_DEPENDENCIES = linux26

MY_LINUX_DRIVERS_MAKE_OPTS += ARCH=$(KERNEL_ARCH)
MY_LINUX_DRIVERS_MAKE_OPTS += CROSS_COMPILE=$(TARGET_CROSS)
MY_LINUX_DRIVERS_MAKE_OPTS += -C $(LINUX26_DIR)
MY_LINUX_DRIVERS_MAKE_OPTS += M=$(@D)

MY_LINUX_DRIVERS_INSTALL_MOD_DIR = mydrivers

define MY_LINUX_DRIVERS_BUILD_CMDS
        $(MAKE) $(TARGET_CONFIGURE_OPTS) $(MY_LINUX_DRIVERS_MAKE_OPTS)
endef

define MY_LINUX_DRIVERS_INSTALL_TARGET_CMDS
        $(MAKE) $(TARGET_CONFIGURE_OPTS) $(MY_LINUX_DRIVERS_MAKE_OPTS) \
                INSTALL_MOD_PATH=$(TARGET_DIR) INSTALL_MOD_STRIP=1 \
                INSTALL_MOD_DIR=$(MY_LINUX_DRIVERS_INSTALL_MOD_DIR) \
                modules_install
endef

define MY_LINUX_DRIVERS_UNINSTALL_TARGET_CMDS
        rm -rf $(TARGET_DIR)/lib/modules/$(LINUX26_VERSION_PROBED)/$(MY_LINUX_DRIVERS_INSTALL_MOD_DIR)
endef

$(eval $(call GENTARGETS,package,my_linux_drivers))


Then, in the driver sources, all you need is a Makefile or Kbuild file
containing something like:
obj-m += module1.o module2.o

plus any other Kbuild directives you may need, like ccflags-y.

I'm planning to add a way to select which drivers from the
my_linux_drivers repo you want, so that e.g. the Makefile looks like:
obj-$(CONFIG_MODULE1) += module1.o
obj-$(CONFIG_MODULE2) += module2.o

and CONFIG_MODULE1 and 2 are then passed in by the .mk file, based on
the buildroot configuration file.

Comments are welcome,
Thomas