[PATCH] bitbake.conf: rework FILESPATH generation.

[PATCH] bitbake.conf: rework FILESPATH generation.

[Chris Larson]

From: Chris Larson <clarson@mvista.com>

Rework FILESPATH generation to be done in bitbake.conf, avoiding the
confusion about it being in multiple places.  Adds FILESPATHBASE and
FILESPATHPKG which can be manipulated rather than manipulating FILESPATH
directly.

One usage possibility:

FILESPATHBASE =. "${TOPDIR}/files:"

Which would let me provide a custom busybox config for this build by
copying the defconfig from the openembedded metadata into my
build/files/busybox-1.0/ directory, for example.

Signed-off-by: Chris Larson <clarson@mvista.com>
---
 classes/base.bbclass |   12 ------------
 conf/bitbake.conf    |    6 ++++--
 2 files changed, 4 insertions(+), 14 deletions(-)

diff --git a/classes/base.bbclass b/classes/base.bbclass
index 411adc9..2cfd398 100644
--- a/classes/base.bbclass
+++ b/classes/base.bbclass
@@ -180,18 +180,6 @@ def base_package_name(d):
 
   return pn
 
-def base_set_filespath(path, d):
-	import os, bb
-	filespath = []
-	# The ":" ensures we have an 'empty' override
-	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
-	for p in path:
-		for o in overrides.split(":"):
-			filespath.append(os.path.join(p, o))
-	return ":".join(filespath)
-
-FILESPATH = "${@base_set_filespath([ "${FILE_DIRNAME}/${PF}", "${FILE_DIRNAME}/${P}", "${FILE_DIRNAME}/${PN}", "${FILE_DIRNAME}/${BP}", "${FILE_DIRNAME}/${BPN}", "${FILE_DIRNAME}/files", "${FILE_DIRNAME}" ], d)}"
-
 def oe_filter(f, str, d):
 	from re import match
 	return " ".join(filter(lambda x: match(f, x, 0), str.split()))
diff --git a/conf/bitbake.conf b/conf/bitbake.conf
index b00d125..06efaa0 100644
--- a/conf/bitbake.conf
+++ b/conf/bitbake.conf
@@ -231,9 +231,11 @@ FILES_${PN}-locale = "${datadir}/locale"
 
 export MANIFEST = "${FILESDIR}/manifest"
 
+# file:// search paths
 FILE_DIRNAME = "${@os.path.dirname(bb.data.getVar('FILE', d))}"
-FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/${BP}:${FILE_DIRNAME}/${BPN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
-FILESDIR = "${@bb.which(bb.data.getVar('FILESPATH', d, 1), '.')}"
+FILESPATHBASE = "${FILE_DIRNAME}"
+FILESPATHPKG = "${PF}:${P}:${PN}:${BP}:${BPN}:files:."
+FILESPATH = "${@':'.join(['%s/%s/%s' % (fp, p, o) for fp in d.getVar('FILESPATHBASE', 0).split(':') for p in d.getVar('FILESPATHPKG', 0).split(':') for o in (d.getVar('OVERRIDES', 0) + ':.').split(':')])}:${FILESDIR}"
 
 ##################################################################
 # General work and output directories for the build system.
-- 
1.5.5.1.30.g58c1f

Re: [PATCH] bitbake.conf: rework FILESPATH generation.

[Denys Dmytriyenko]

On Thu, Feb 19, 2009 at 04:28:52PM -0700, Chris Larson wrote:
> From: Chris Larson <clarson@mvista.com>
> 
> Rework FILESPATH generation to be done in bitbake.conf, avoiding the
> confusion about it being in multiple places.  Adds FILESPATHBASE and
> FILESPATHPKG which can be manipulated rather than manipulating FILESPATH
> directly.
> 
> One usage possibility:
> 
> FILESPATHBASE =. "${TOPDIR}/files:"
> 
> Which would let me provide a custom busybox config for this build by
> copying the defconfig from the openembedded metadata into my
> build/files/busybox-1.0/ directory, for example.

Can you please provide more details on this example? Thanks.

-- 
Denys

Re: [PATCH] bitbake.conf: rework FILESPATH generation.

[Denys Dmytriyenko]

On Thu, Feb 19, 2009 at 04:58:25PM -0700, Christopher Larson wrote:
>
> On Feb 19, 2009, at 4:41 PM, Denys Dmytriyenko wrote:
>
>> On Thu, Feb 19, 2009 at 04:28:52PM -0700, Chris Larson wrote:
>>> From: Chris Larson <clarson@mvista.com>
>>>
>>> Rework FILESPATH generation to be done in bitbake.conf, avoiding the
>>> confusion about it being in multiple places.  Adds FILESPATHBASE and
>>> FILESPATHPKG which can be manipulated rather than manipulating FILESPATH
>>> directly.
>>>
>>> One usage possibility:
>>>
>>> FILESPATHBASE =. "${TOPDIR}/files:"
>>>
>>> Which would let me provide a custom busybox config for this build by
>>> copying the defconfig from the openembedded metadata into my
>>> build/files/busybox-1.0/ directory, for example.
>>
>> Can you please provide more details on this example? Thanks.
>
> Normally, all paths in FILESPATH are relative to FILE_DIRNAME, the location 
> of the recipe being operated against.  With this change, you can place 
> files which are in SRC_URI as file:// urls in paths relative to other 
> locations.
>
> You can do package customizations via changes to file:// files without 
> mucking up the local clone of the openembedded repository.
>
> My intent is to create a class to facilitate running make menuconfig on 
> your kernel and busybox, and retain those changes.  Today, people either go 
> into workdir and customize it there, but those changes are lost when you 
> clean the package, or they overwrite files in the metadata repository, 
> which can lead to conflicts, and is just all around more annoying to work 
> with.  It's better to be able to have per build or site wide overrides of 
> those files.
>
> Does that answer your question?

That would be a very welcome addition! Thanks.

-- 
Denys

Re: [PATCH] bitbake.conf: rework FILESPATH generation.

[Chris Larson]

Heh, already sent this reply out, but from the wrong email address.
This time it should actually hit the list...

On Thu, Feb 19, 2009 at 4:41 PM, Denys Dmytriyenko <denis@denix.org> wrote:
> On Thu, Feb 19, 2009 at 04:28:52PM -0700, Chris Larson wrote:
>> From: Chris Larson <clarson@mvista.com>
>>
>> Rework FILESPATH generation to be done in bitbake.conf, avoiding the
>> confusion about it being in multiple places.  Adds FILESPATHBASE and
>> FILESPATHPKG which can be manipulated rather than manipulating FILESPATH
>> directly.
>>
>> One usage possibility:
>>
>> FILESPATHBASE =. "${TOPDIR}/files:"
>>
>> Which would let me provide a custom busybox config for this build by
>> copying the defconfig from the openembedded metadata into my
>> build/files/busybox-1.0/ directory, for example.
>
> Can you please provide more details on this example? Thanks.

Normally, all paths in FILESPATH are relative to FILE_DIRNAME, the
location of the recipe being operated against.  With this change, you
can place files which are in SRC_URI as file:// urls in paths relative
to other locations.

You can do package customizations via changes to file:// files without
mucking up the local clone of the openembedded repository.

My intent is to create a class to facilitate running make menuconfig
on your kernel and busybox, and retain those changes.  Today, people
either go into workdir and customize it there, but those changes are
lost when you clean the package, or they overwrite files in the
metadata repository, which can lead to conflicts, and is just all
around more annoying to work with.  It's better to be able to have per
build or site wide overrides of those files.
-- 
Chris Larson
clarson at kergoth dot com
clarson at mvista dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Software Engineer
MontaVista Software, Inc.

Re: [PATCH] bitbake.conf: rework FILESPATH generation.

[Tom Rini]

On Thu, Feb 19, 2009 at 05:27:34PM -0700, Chris Larson wrote:
[snip]
> My intent is to create a class to facilitate running make menuconfig
> on your kernel and busybox, and retain those changes.  Today, people
> either go into workdir and customize it there, but those changes are
> lost when you clean the package, or they overwrite files in the
> metadata repository, which can lead to conflicts, and is just all
> around more annoying to work with.  It's better to be able to have per
> build or site wide overrides of those files.

Playing the devils advocate here, conflicts make you go "Oh, I need to
see what happened", or at least should.  But so long as wherever this
gets documented emphasizes that people need to be aware what they're
doing (it's like setting up your own overlay, but not quite as
intensive) and need to keep an eye out when re-syncing with upstream, it
should certainly fix more headaches than it creates.

-- 
Tom Rini

Re: [PATCH] bitbake.conf: rework FILESPATH generation.

[Otavio Salvador]

Chris Larson <clarson@kergoth.com> writes:

> Heh, already sent this reply out, but from the wrong email address.
> This time it should actually hit the list...
>
> On Thu, Feb 19, 2009 at 4:41 PM, Denys Dmytriyenko <denis@denix.org> wrote:
>> On Thu, Feb 19, 2009 at 04:28:52PM -0700, Chris Larson wrote:
>>> From: Chris Larson <clarson@mvista.com>
>>>
>>> Rework FILESPATH generation to be done in bitbake.conf, avoiding the
>>> confusion about it being in multiple places.  Adds FILESPATHBASE and
>>> FILESPATHPKG which can be manipulated rather than manipulating FILESPATH
>>> directly.
>>>
>>> One usage possibility:
>>>
>>> FILESPATHBASE =. "${TOPDIR}/files:"
>>>
>>> Which would let me provide a custom busybox config for this build by
>>> copying the defconfig from the openembedded metadata into my
>>> build/files/busybox-1.0/ directory, for example.
>>
>> Can you please provide more details on this example? Thanks.
>
> Normally, all paths in FILESPATH are relative to FILE_DIRNAME, the
> location of the recipe being operated against.  With this change, you
> can place files which are in SRC_URI as file:// urls in paths relative
> to other locations.
>
> You can do package customizations via changes to file:// files without
> mucking up the local clone of the openembedded repository.
>
> My intent is to create a class to facilitate running make menuconfig
> on your kernel and busybox, and retain those changes.  Today, people
> either go into workdir and customize it there, but those changes are
> lost when you clean the package, or they overwrite files in the
> metadata repository, which can lead to conflicts, and is just all
> around more annoying to work with.  It's better to be able to have per
> build or site wide overrides of those files.

I second it; specially the "make menuconfig" feature :-)

I lose a kernel config yestarday by mistake due this :P

-- 
Otavio Salvador                  O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854         http://projetos.ossystems.com.br