deterministic OE builds - multiple providers

deterministic OE builds - multiple providers

[Leon Woestenberg]

Hello,

in trying to build my OE snapshot with a small extra bitbake
collection to keep mildly customized .bb files across different Linux
hosts.

In tracking down a host dependency I found that in cases where
multiple providers are present, the behaviour is undeterministic, i.e.
a provider is picked more or less randomly.

Can someone confirm this?

If this is the case, can we make it obiligatory to demand that in this
case one single provider must be explicitly and unambiguously defined,
and otherwise fail the build?

Regards,
-- 
Leon

Re: deterministic OE builds - multiple providers

[Richard Purdie]

On Wed, 2007-04-11 at 10:32 +0200, Leon Woestenberg wrote:
> in trying to build my OE snapshot with a small extra bitbake
> collection to keep mildly customized .bb files across different Linux
> hosts.
> 
> In tracking down a host dependency I found that in cases where
> multiple providers are present, the behaviour is undeterministic, i.e.
> a provider is picked more or less randomly.
> 
> Can someone confirm this?
> 
> If this is the case, can we make it obiligatory to demand that in this
> case one single provider must be explicitly and unambiguously defined,
> and otherwise fail the build?

Which bitbake version is this with? Later versions should have more
deterministic handling that older versions.

Regards,

Richard

Re: deterministic OE builds - multiple providers

[Leon Woestenberg]

Richard,

On 4/11/07, Richard Purdie <rpurdie@rpsys.net> wrote:
> On Wed, 2007-04-11 at 10:32 +0200, Leon Woestenberg wrote:
> > ...
> > In tracking down a host dependency I found that in cases where
> > multiple providers are present, the behaviour is undeterministic, i.e.
> > a provider is picked more or less randomly.
> > ...
>
> Which bitbake version is this with? Later versions should have more
> deterministic handling that older versions.
>
URL: svn://svn.berlios.de/bitbake/branches/bitbake-1.8
Revision: 812

I tried multiple, also 1.6, same issues remain.

Some issues:

My private bitbake collection has customized copies of upstream .bb.
Although my private bitbake collection has a higher priority, I
noticed bitbake "randomly" selecting either the upstream or local
collection for packages which have multiple providers left.

In one case (ixp4xx-kernel) if I run "bitbake ixp4xx-kernel -c patch",
then "bitbake ixp4xx-kernel" it selects my private .bb for the first
bitbake, and then selects the upstream .bb for the second bitbake
build.

One machine notes about multiple providers for kernel-image-, whereas
the other does not.

They run the exact same copies of the upstream sources, local
collection and the  bitbake tool.

Regards,
-- 
Leon

Re: deterministic OE builds - multiple providers

[Leon Woestenberg]

To illustrate the undeterministic points, this is what happens on one machine:

leon@axonnl-092 ~/sandbox/ixp4xx/openembedded $ bitbake -vvv
ixp4xx-kernel -c patch | grep Running
NOTE: Running task 1 of 3 (ID: 2,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_fetch)
NOTE: Running task 1 of 3 (ID: 2,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_fetch)
NOTE: Running task 2 of 3 (ID: 0,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_unpack)
NOTE: Running task 1 of 3 (ID: 2,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_fetch)
NOTE: Running task 2 of 3 (ID: 0,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_unpack)
NOTE: Running task 3 of 3 (ID: 1,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_patch)
NOTE: Running task 1 of 3 (ID: 2,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_fetch)
NOTE: Running task 2 of 3 (ID: 0,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_unpack)
NOTE: Running task 3 of 3 (ID: 1,
/home/leon/sandbox/openembedded/tv.axon.openembedded/packages/linux/ixp4xx-kernel_2.6.19.bb,
do_patch)

leon@axonnl-092 ~/sandbox/ixp4xx/openembedded $ bitbake -vvv
ixp4xx-kernel | grep Running
cp: cannot stat
`/home/leon/sandbox/openembedded/org.openembedded.dev/packages/linux/linux/defconfig':
No such file or directory
NOTE: Running task 218 of 234 (ID: 96,
/home/leon/sandbox/openembedded/org.openembedded.dev/packages/linux/linux_2.6.19.bb,
do_unpack)
NOTE: Running task 218 of 234 (ID: 96,
/home/leon/sandbox/openembedded/org.openembedded.dev/packages/linux/linux_2.6.19.bb,
do_unpack)