Cleaning up the "linux" directory

Cleaning up the "linux" directory

[Ulf Samuelsson]

As everyone knows, the kernel directory is a mess:
I have been thinking about what to do about this,
and with the ability to include files in subdirectories,
I think I have found a nice solution.

New variables

CPU_FAMILY
KERNEL_MAJOR
KERNEL_MINOR

For future kernels,

Did not try this, yet but this looks promising...

-----------------------------------------------------------------------------
# linux-2.6.30.bb (replacing the old one)
# New style recipe for kernel builds
# Hopefully no need to have custom recipies in the top directory
# Mark archs/machines that this kernel supports
DEFAULT_PREFERENCE = "-1"
require linux-v2.inc
PR = "r0"
# Yes, that's all!

-----------------------------------------------------------------------------
# "linux-v2.inc" Contains
require linux.inc

CPU_FAMILY ?= "dummy-arch"
KERNEL_MAJOR  = "${PV}"
KERNEL_DIR = "${FILE_DIRNAME}/linux-${KERNEL_MAJOR}"
require ${KERNEL_DIR}/version.inc

KERNEL_MINOR_DIR = ${KERNEL_DIR}/${KERNEL_VERSION}
KERNEL_CPU_FAMILY_DIR=${KERNEL_DIR}/${CPU_FAMILY}

S = "${WORKDIR}/linux-${KERNEL_MAJOR}"

KERNEL_SOURCE_DIR="${KERNELORG_MIRROR}/pub/linux/kernel/v2.6"
SRC_URI = "${KERNEL_SOURCE_DIR}/linux-${KERNEL_MAJOR}.tar.bz2 "
MINOR_PATCH=${KERNEL_SOURCE_DIR}/patch-${KERNEL_MAJOR}.${KERNEL_MINOR}.bz2

# Include ${MINOR_PATCH}
# Include any other common patches for this minor version
require	${KERNEL_MINOR_DIR}/patches/SRC_URI_append.inc
# Any customizations for this minor version
require	${KERNEL_MINOR_DIR}/linux-${KERNEL_VERSION}.inc

# Customize for the CPU_FAMILY
# Note that this can contain further includes for minor versions
require	${KERNEL_CPU_FAMILY_DIR}/patches/SRC_URI_append.inc
require	${KERNEL_CPU_FAMILY_DIR}/common/linux-${KERNEL_VERSION}.inc
include
${KERNEL_CPU_FAMILY_DIR}/boards/${MACHINE}/linux-${KERNEL_VERSION}.inc
#
-----------------------------------------------------------------------------
# "${KERNEL_DIR}/version.inc" contains
KERNEL_LATEST_MINOR = "5"
KERNEL_MINOR ?= "${KERNEL_LATEST_MINOR}"
KERNEL_VERSION = "${KERNEL_MAJOR}.${KERNEL_MINOR}"



-----------------------------------------------------------------------------
# Possible contents of ${KERNEL_CPU_FAMILY_DIR}/patches/SRC_URI_append.inc
require {KERNEL_CPU_FAMILY_DIR}/patches/${KERNEL_VERSION}/SRC_URI_append.inc

#
-- 
Best Regards
Ulf Samuelsson

Re: Cleaning up the "linux" directory

[Ulf Samuelsson]

Ulf Samuelsson skrev:
> As everyone knows, the kernel directory is a mess:
> I have been thinking about what to do about this,
> and with the ability to include files in subdirectories,
> I think I have found a nice solution.
> 
> New variables
> 
> CPU_FAMILY
> KERNEL_MAJOR
> KERNEL_MINOR
> 
> For future kernels,
> 
> Did not try this, yet but this looks promising...
> 
I have tested a recipe which configures itself based on CPU_FAMILY.
The recipe is totally CPU/Architecure independent!


----------------------------------------------------------------------------
linux-2.6.30.2.bb contains:

SECTION = "kernel"
DESCRIPTION = "Linux kernel 2.6.30.2"
LICENSE = "GPL"

KERNEL_MAJOR = "2.6.30"
KERNEL_MINOR = "2"
PR = "r0"

DEFAULT_PREFERENCE = "-1"

require all-linux-v2.inc

----------------------------------------------------------------------------
As you can see, there is only ONE require/include,
and that is to a totally generic file,
which configures the build based on

CPU_FAMILY
KERNEL_MAJOR
KERNEL_MINOR

The directory structure used to support the at91 for 2.6.30.2:

2.6.30
	2.6.30.2		# common , currently does nothing...
		linux.inc
		patches
			network_files
			SRC_URI_append.inc
	at91			# at91 specific
		at91.inc
		logo_linux_clut224.ppm
		2.6.30.2
			patch-sets	# Not called "patches" by design
				network_files
				SRC_URI_append.inc
				<patches>
			boards
				at91sam9g45ek
					defconfig
					logo_linux_clut224.ppm
	The "2.6.30/2.6.30.2" directory can normally be copied to
	"2.6.30/2.6.30.3" and renamed if the next kernel version is
	needed. No need to edit, if this is pristine.


It supports adding yet another level of directory in the "patch-sets"
directory, so that you can have one directory per patch-set.




----------------------------------------------------------------------------
all-linux-v2.inc contains:

require	linux.inc
CPU_FAMILY ?= "dummy-arch"
KERNEL_MAJOR  ?= "${PV}"
#${FILE_DIRNAME}/
KERNEL_MAJOR_DIR = "linux-${KERNEL_MAJOR}"
require ${KERNEL_MAJOR_DIR}/version.inc

KERNEL_VERSION_DIR = ${KERNEL_MAJOR_DIR}/${KERNEL_VERSION}
KERNEL_CPU_FAMILY_DIR=${KERNEL_MAJOR_DIR}/${CPU_FAMILY}

S = "${WORKDIR}/linux-${KERNEL_MAJOR}"

KERNEL_SOURCE_DIR="${KERNELORG_MIRROR}/pub/linux/kernel/v2.6"
SRC_URI = "${KERNEL_SOURCE_DIR}/linux-${KERNEL_MAJOR}.tar.bz2 "
MINOR_PATCH=${KERNEL_SOURCE_DIR}/patch-${KERNEL_MAJOR}.${KERNEL_MINOR}.bz2

# Include ${MINOR_PATCH} and any other common patches for this minor version
require	${KERNEL_VERSION_DIR}/patches/SRC_URI_append.inc
# Any customizations for this minor version
require	${KERNEL_VERSION_DIR}/linux.inc

# Customize for the CPU_FAMILY
# Note that this can contain further includes for minor versions
require	${KERNEL_CPU_FAMILY_DIR}/${CPU_FAMILY}.inc
require	${KERNEL_CPU_FAMILY_DIR}/${KERNEL_VERSION}/${KERNEL_VERSION}.inc
require
${KERNEL_CPU_FAMILY_DIR}/${KERNEL_VERSION}/patch-sets/SRC_URI_append.inc


----------------------------------------------------------------------------
I ran into one problem:
SRC_URI_append_at91sam9g45ek =         "
file://linux-2.6.30/at91/2.6.30.2/boards/at91sam9g45ek/defconfig "

moves the file to ${S} and not to ${WORKDIR}


I took the easy way out with a workaround:

	mv ${S}/defconfig ${WORKDIR}/defconfig

What do I do wrong?

-- 
Best Regards
Ulf Samuelsson

Re: Cleaning up the "linux" directory - Available in origin/ulf/linux-2.6.30.2

[Ulf Samuelsson]

Ulf Samuelsson skrev:

Just pushed this stuff to my private branch
Also added at91bootstrap-2.14-rc4 (which enables user reset)
a tftpboot recipe, which copies the results to /tftpboot/Angstrom/${MACHINE}
Added the gnome scripts to the scripts/nautilus-scripts directory.

BR
Ulf Samuelsson


> Ulf Samuelsson skrev:
>> As everyone knows, the kernel directory is a mess:
>> I have been thinking about what to do about this,
>> and with the ability to include files in subdirectories,
>> I think I have found a nice solution.
>>
>> New variables
>>
>> CPU_FAMILY
>> KERNEL_MAJOR
>> KERNEL_MINOR
>>
>> For future kernels,
>>
>> Did not try this, yet but this looks promising...
>>
> I have tested a recipe which configures itself based on CPU_FAMILY.
> The recipe is totally CPU/Architecure independent!
> 
> 
> ----------------------------------------------------------------------------
> linux-2.6.30.2.bb contains:
> 
> SECTION = "kernel"
> DESCRIPTION = "Linux kernel 2.6.30.2"
> LICENSE = "GPL"
> 
> KERNEL_MAJOR = "2.6.30"
> KERNEL_MINOR = "2"
> PR = "r0"
> 
> DEFAULT_PREFERENCE = "-1"
> 
> require all-linux-v2.inc
> 
> ----------------------------------------------------------------------------
> As you can see, there is only ONE require/include,
> and that is to a totally generic file,
> which configures the build based on
> 
> CPU_FAMILY
> KERNEL_MAJOR
> KERNEL_MINOR
> 
> The directory structure used to support the at91 for 2.6.30.2:
> 
> 2.6.30
> 	2.6.30.2		# common , currently does nothing...
> 		linux.inc
> 		patches
> 			network_files
> 			SRC_URI_append.inc
> 	at91			# at91 specific
> 		at91.inc
> 		logo_linux_clut224.ppm
> 		2.6.30.2
> 			patch-sets	# Not called "patches" by design
> 				network_files
> 				SRC_URI_append.inc
> 				<patches>
> 			boards
> 				at91sam9g45ek
> 					defconfig
> 					logo_linux_clut224.ppm
> 	The "2.6.30/2.6.30.2" directory can normally be copied to
> 	"2.6.30/2.6.30.3" and renamed if the next kernel version is
> 	needed. No need to edit, if this is pristine.
> 
> 
> It supports adding yet another level of directory in the "patch-sets"
> directory, so that you can have one directory per patch-set.
> 
> 
> 
> 
> ----------------------------------------------------------------------------
> all-linux-v2.inc contains:
> 
> require	linux.inc
> CPU_FAMILY ?= "dummy-arch"
> KERNEL_MAJOR  ?= "${PV}"
> #${FILE_DIRNAME}/
> KERNEL_MAJOR_DIR = "linux-${KERNEL_MAJOR}"
> require ${KERNEL_MAJOR_DIR}/version.inc
> 
> KERNEL_VERSION_DIR = ${KERNEL_MAJOR_DIR}/${KERNEL_VERSION}
> KERNEL_CPU_FAMILY_DIR=${KERNEL_MAJOR_DIR}/${CPU_FAMILY}
> 
> S = "${WORKDIR}/linux-${KERNEL_MAJOR}"
> 
> KERNEL_SOURCE_DIR="${KERNELORG_MIRROR}/pub/linux/kernel/v2.6"
> SRC_URI = "${KERNEL_SOURCE_DIR}/linux-${KERNEL_MAJOR}.tar.bz2 "
> MINOR_PATCH=${KERNEL_SOURCE_DIR}/patch-${KERNEL_MAJOR}.${KERNEL_MINOR}.bz2
> 
> # Include ${MINOR_PATCH} and any other common patches for this minor version
> require	${KERNEL_VERSION_DIR}/patches/SRC_URI_append.inc
> # Any customizations for this minor version
> require	${KERNEL_VERSION_DIR}/linux.inc
> 
> # Customize for the CPU_FAMILY
> # Note that this can contain further includes for minor versions
> require	${KERNEL_CPU_FAMILY_DIR}/${CPU_FAMILY}.inc
> require	${KERNEL_CPU_FAMILY_DIR}/${KERNEL_VERSION}/${KERNEL_VERSION}.inc
> require
> ${KERNEL_CPU_FAMILY_DIR}/${KERNEL_VERSION}/patch-sets/SRC_URI_append.inc
> 
> 
> ----------------------------------------------------------------------------
> I ran into one problem:
> SRC_URI_append_at91sam9g45ek =         "
> file://linux-2.6.30/at91/2.6.30.2/boards/at91sam9g45ek/defconfig "
> 
> moves the file to ${S} and not to ${WORKDIR}
> 
> 
> I took the easy way out with a workaround:
> 
> 	mv ${S}/defconfig ${WORKDIR}/defconfig
> 
> What do I do wrong?
> 


-- 
Best Regards
Ulf Samuelsson

Re: Cleaning up the "linux" directory

[Marcin Juszkiewicz]

Dnia sobota, 22 sierpnia 2009 o 11:38:41 Ulf Samuelsson napisał(a):
> As everyone knows, the kernel directory is a mess:
> I have been thinking about what to do about this,
> and with the ability to include files in subdirectories,
> I think I have found a nice solution.

I had a look at this and for me it adds lot of things to care when kernel 
needs to be updated/changed.

With current linux_X.Y.Z(.A).bb recipes it is easy to add new device - just 
one file needs to be edited and thats all. If new patch has to be added then 
still one file to edit. New board to be added - same amount of work. If 
patches common for few targets needs to be added then you can create new 
variable for them:

AT91_PATCHES = "\
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-001-
andrew-victor.patch;patch=1 \
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-002-
experimental.patch;patch=1 \
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-003-
ac97-0001-sam9g45-r1.patch;patch=1 \
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-003-
ac97-0002-ac97-sam9rlek-r1.patch;patch=1 \
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-004-
NAND-001-configurable-partition-size.patch;patch=1 \
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-005-
touchscreen-001-sam9g45ek.patch;patch=1 \
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-006-
EHCI-001.sam9g45ek.patch;patch=1 \
        file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-007-
CLK-001-debug-clocks.patch;patch=1 \
        "

And use it in SRC_URI for all boards. If patches do not touch files used by 
other devices then adding them directly to default SRC_URI should also be 
possible (thats how at91 patch was added in linux_2.6.30.bb recipe).

With your solution I have to check about 10 files in many directories or 
regenerate all of them by some script (what about caring of manual changes in 
those files?).

Maybe Buildroot uses such way (I do not know because last time I used it 
before OE) but for me as long time OE developer it just adds lot of possible 
errors to do.

Regards, 
-- 
JID:      hrw@jabber.org
Website:  http://marcin.juszkiewicz.com.pl/
LinkedIn: http://www.linkedin.com/in/marcinjuszkiewicz

Re: Cleaning up the "linux" directory

[Ulf Samuelsson]

Marcin Juszkiewicz skrev:
> Dnia sobota, 22 sierpnia 2009 o 11:38:41 Ulf Samuelsson napisał(a):
>> As everyone knows, the kernel directory is a mess:
>> I have been thinking about what to do about this,
>> and with the ability to include files in subdirectories,
>> I think I have found a nice solution.
> 
> I had a look at this and for me it adds lot of things to care when kernel 
> needs to be updated/changed.
> 
> With current linux_X.Y.Z(.A).bb recipes it is easy to add new device - just 
> one file needs to be edited and thats all. If new patch has to be added then 
> still one file to edit. New board to be added - same amount of work. If 
> patches common for few targets needs to be added then you can create new 
> variable for them:
> 

The SRC_URI_appends should only contain automatically
generated things and there is warning against editing inserted.

When you say you only  have to edit one file to add a patch,
I think that is wrong. You have to enter the directory
and place the patch there.
It is defintely faster to select all the patches, and
run the script than to edit the toplevel file.

You also do not have to know the variable names.

If you want to add a board, you create the directory,
and possibly add a defconfig.
Rerun the script and again no editing of any files.

If you want to create a new CPU family, you probably are going
to create a file in conf/machine, and  you have to add the cpu  family
directory.
Easiest way to do this, is to copy another directory
and clean out the files.
Then you fill them with new board patches and defconfigs.

Personally, I see very little reasons for doing any editing
but if you want to do something for the at91, it is basically
the linux-2.6.3/at91/at91.inc that will the single file to edit.

It is different, but I think you should give it a try.



> AT91_PATCHES = "\
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-001-
> andrew-victor.patch;patch=1 \
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-002-
> experimental.patch;patch=1 \
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-003-
> ac97-0001-sam9g45-r1.patch;patch=1 \
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-003-
> ac97-0002-ac97-sam9rlek-r1.patch;patch=1 \
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-004-
> NAND-001-configurable-partition-size.patch;patch=1 \
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-005-
> touchscreen-001-sam9g45ek.patch;patch=1 \
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-006-
> EHCI-001.sam9g45ek.patch;patch=1 \
>         file://linux-2.6.30/at91/2.6.30.2/patch-sets/linux-2.6.30.2-at91-007-
> CLK-001-debug-clocks.patch;patch=1 \
>         "
> 
> And use it in SRC_URI for all boards. If patches do not touch files used by 
> other devices then adding them directly to default SRC_URI should also be 
> possible (thats how at91 patch was added in linux_2.6.30.bb recipe).
> 
> With your solution I have to check about 10 files in many directories or 
> regenerate all of them by some script (what about caring of manual changes in 
> those files?).


Running the scripts manually will be bad, you need to run them
from gnome (as of today).
What would be good to have is a bitbake command.

append	linux-${KERNEL_MAJOR}/${CPU_FAMILY}/${KERNEL_VERSION}/patch-sets/*
which would add all the *.patch files in the selected directories
in order.

Then there would be no need to run a script and no files to edit.
If you add a patch to the right directory,
then it will be added to the build.


> Maybe Buildroot uses such way (I do not know because last time I used it 
> before OE) but for me as long time OE developer it just adds lot of possible 
> errors to do.

No, this is brand new and it also >reduces< the number of errors.
Spelling errors in file names in particular.
I believe the whole systen will have less errors,
and the top level can be cleaned out.

The only real error is if you forget to run the script,
but once you are in the patch directory, it takes less than
5 seconds to select the patches, and run the script.
Much faster and less error prone than editing.


> Regards, 


-- 
Best Regards
Ulf Samuelsson