Re: Conditional DAG recalculation at build time

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Mon, 2018-01-29 at 23:06 -0800, Daniel Levin wrote:
> In short: does bitbake support dependency graph recalculation at
> build time?
> 
> I am looking for a dependency resolution similar to BYPRODUCTS in
> CMake and restat in Ninja.
> 
> In other words when task _after_ execution might tell that output did
> not actually change. So all dependencies of that target should not be
> updated. Even when initially they were treated as (potentially) to be
> updated.
> 
> Usually in CMake that results in remaining task count decreased
> instantly during build time, because in fact they must not be
> running.
> 
> I found documentation in bitbake about task _input_ checksum, which
> is used to evaluate whether task need to be executed.
> So I am looking for the similar checksum for the task _output_. And
> when task does not change that output checksum means that
> dependencies of that output does not need to be triggered.
> 
> If this topic is well known and documented then could you please
> point me to the appropriate section in the documentation?

There are mailing list posts from me a long time ago when we added
checksum support and sstate explaining why we use input values rather
than outputs and the pros/cons of either.

Also, more recently, have a read of the thread "[yocto] RFC: Backwards
compatibility checking sstate-cache" on the yocto list. Its a similar
request.

My own thoughts are that the best way forward would be some kind of
equivalence server so that we could remap sstate values back to a
master value.

The first step would be to allow such a server to exist and to allow
resolution of a hash to a master base value.

Allowing bitbake to recalculate its taskgraph and dynamically adapt to
this would then be a secondary step in the interests of efficiency.

The first piece would be minimally invasive. The second piece would
mean huge changes to the way bitbake operates.

I'm not sure if anyone is pursuing this but its an interesting topic
I've given a bit of thought to.

Cheers,

Richard