Bitbake task dependency stamp handling

Bitbake task dependency stamp handling

[Richard Purdie]

I had some interesting thoughts regarding tasks and their timestamps.
Bitbake's current method of dealing with timestamps walks the task
digraph structure which only knows about a single .bb file. The result
is if you change an earlier task, say do_patch, later tasks like
do_compile would rerun. If you rebuild gtk, the rest of your system
linking to gtk doesn't rebuild though.

If we move away from the digraph which isn't really needed anymore,
taskData has the complete dependency tree available to it and the option
of rebuilding the whole system if you recompile gtk becomes possible.

At face value this sounds really useful. I'd guess that almost every OE
developer would find it insanely annoying in practise though?

Does anyone have any thoughts?

Richard

Re: Bitbake task dependency stamp handling

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Richard Purdie schreef:
> I had some interesting thoughts regarding tasks and their timestamps.
> Bitbake's current method of dealing with timestamps walks the task
> digraph structure which only knows about a single .bb file. The result
> is if you change an earlier task, say do_patch, later tasks like
> do_compile would rerun. If you rebuild gtk, the rest of your system
> linking to gtk doesn't rebuild though.
> 
> If we move away from the digraph which isn't really needed anymore,
> taskData has the complete dependency tree available to it and the option
> of rebuilding the whole system if you recompile gtk becomes possible.
> 
> At face value this sounds really useful. I'd guess that almost every OE
> developer would find it insanely annoying in practise though?

Annoying, maybe, correct behaviour, yes. Option for local.conf?

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFoUHfMkyGM64RGpERApU6AKCHQ+UumH7Gw7rEbWbTb7fcW+wkzgCfRSbl
n/rabOe9dEZgkB4x617wYqo=
=jfo2
-----END PGP SIGNATURE-----

Re: Bitbake task dependency stamp handling

[Marcin Juszkiewicz]

Dnia niedziela, 7 stycznia 2007 19:54, Koen Kooi napisał:

> > If we move away from the digraph which isn't really needed anymore,
> > taskData has the complete dependency tree available to it and the
> > option of rebuilding the whole system if you recompile gtk becomes
> > possible.
> >
> > At face value this sounds really useful. I'd guess that almost every
> > OE developer would find it insanely annoying in practise though?
>
> Annoying, maybe, correct behaviour, yes. Option for local.conf?

It will be good thing. But I think that there must be kind of option to 
tell which exactly package force other to be rebuilt. I would like to be 
able to ignore situation where gcc packaging change introduce rebuild of 
everything. But if gcc change and gtk+ will be also changed then I want 
to do build which will rebuild all gtk+ dependend recipes.

Having option in local.conf probably will be easier for bitbake parser 
then providing list of recipes which force 'own childs' to be rebuilt.

I think that this is a place where we need some kind of UI which will be 
able to request action from us (but rather on start of build then during 
build - many of our builds goes without any interactive usage (nightly 
runs etc).

-- 
JID: hrw-jabber.org
OpenEmbedded developer/consultant

    42? 7 and a half million years and all you can come up with is 42?!

Re: [Bitbake-dev] Bitbake task dependency stamp handling

[Richard Purdie]

On Sun, 2007-01-07 at 20:23 +0100, Marcin Juszkiewicz wrote:
> Dnia niedziela, 7 stycznia 2007 19:54, Koen Kooi napisał:
> 
> > > If we move away from the digraph which isn't really needed anymore,
> > > taskData has the complete dependency tree available to it and the
> > > option of rebuilding the whole system if you recompile gtk becomes
> > > possible.
> > >
> > > At face value this sounds really useful. I'd guess that almost every
> > > OE developer would find it insanely annoying in practise though?
> >
> > Annoying, maybe, correct behaviour, yes. Option for local.conf?
> 
> It will be good thing. But I think that there must be kind of option to 
> tell which exactly package force other to be rebuilt. I would like to be 
> able to ignore situation where gcc packaging change introduce rebuild of 
> everything. But if gcc change and gtk+ will be also changed then I want 
> to do build which will rebuild all gtk+ dependend recipes.

As an illustration, changing gcc packaging can affect gtk+ due to shlibs
and package renaming. Its likely at the very least gtk+ would repackage
as would almost everything. 

> Having option in local.conf probably will be easier for bitbake parser 
> then providing list of recipes which force 'own childs' to be rebuilt.

Bitbake would work out the list of recipes itself, in fact it already
knows this (have a look at the output from bitbake trunk's -g option). 

I guess you mean defining the criteria for when to rebuild or not to
rebuild and this is hard :-/.

> I think that this is a place where we need some kind of UI which will be 
> able to request action from us (but rather on start of build then during 
> build - many of our builds goes without any interactive usage (nightly 
> runs etc).

Agreed and I'm trying to keep this in mind with the new UIs...

Cheers,

Richard

Re: Bitbake task dependency stamp handling

[Matt Reimer]

On 1/7/07, Koen Kooi <koen@dominion.kabel.utwente.nl> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Richard Purdie schreef:
> > I had some interesting thoughts regarding tasks and their timestamps.
> > Bitbake's current method of dealing with timestamps walks the task
> > digraph structure which only knows about a single .bb file. The result
> > is if you change an earlier task, say do_patch, later tasks like
> > do_compile would rerun. If you rebuild gtk, the rest of your system
> > linking to gtk doesn't rebuild though.
> >
> > If we move away from the digraph which isn't really needed anymore,
> > taskData has the complete dependency tree available to it and the option
> > of rebuilding the whole system if you recompile gtk becomes possible.
> >
> > At face value this sounds really useful. I'd guess that almost every OE
> > developer would find it insanely annoying in practise though?
>
> Annoying, maybe, correct behaviour, yes. Option for local.conf?

Yes please! FreeBSD's portupgrade has a similar feature where you can
upgrade dependencies or packages that depend on the specified one by
passing -r or -R; also, you can have it prompt you to upgrade a
dependency (rather than just doing it) if you run it in interactive
mode. Perhaps if running "bitbake -i" it could prompt? Of course also
check a local.conf knob as Koen suggests.

Matt

Re: Bitbake task dependency stamp handling

[Michael 'Mickey' Lauer]

Richard Purdie wrote:
> I had some interesting thoughts regarding tasks and their timestamps.
> Bitbake's current method of dealing with timestamps walks the task
> digraph structure which only knows about a single .bb file. The result
> is if you change an earlier task, say do_patch, later tasks like
> do_compile would rerun. If you rebuild gtk, the rest of your system
> linking to gtk doesn't rebuild though.

> If we move away from the digraph which isn't really needed anymore,
> taskData has the complete dependency tree available to it and the option
> of rebuilding the whole system if you recompile gtk becomes possible.

Excellent!

> At face value this sounds really useful. I'd guess that almost every OE
> developer would find it insanely annoying in practise though?

Not at all. I prefer waiting a bit more than having link or structure
mismatched due to changes in headers and libraries.

If it doesn't involve leaving too much code in, then I'd love to see a
way to turn it off to the old behaviour though.

Regards,

:M:
-- 
Michael 'Mickey' Lauer | IT-Freelancer | http://www.vanille-media.de