bitbake -c rebuild not running all tasks

bitbake -c rebuild not running all tasks

[Cliff Brake]

I'm trying to sort out the xorg fonts stuff, and am running into an
issue where "bitbake -c rebuild" does not re-run all tasks when
rebuilding font-util.  It seems to me that bitbake should be running
all tasks including configure, compile, etc.

I'm using bitbake 1.8 HEAD, and a week old or so version of OE.  I'll
try the OE HEAD as soon as I get a build to finish.

Thanks,
Cliff


cbrake@happy:/build/bhthermal/bht_build/oe$ bitbake font-util -c rebuild
NOTE: Handling BitBake files: / (5365/5365) [100 %]
NOTE: Parsing finished. 5137 cached, 0 parsed, 228 skipped, 0 masked.
NOTE: Cache is clean, not saving.
NOTE: build 200804241623: started

OE Build Configuration:
BB_VERSION     = "1.8.11"
OE_REVISION    = "<unknown>"
TARGET_ARCH    = "i586"
TARGET_OS      = "linux"
MACHINE        = "pcm-9375"
DISTRO         = "angstrom"
DISTRO_VERSION = "2008.1-test-20080424"
TARGET_FPU     = ""

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
ERROR: Multiple .bb files are due to be built which each provide xdmcp
(/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-lib/libxdmcp_1.0.2.bb
/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-lib/libxdmcp-native_1.0.2.bb).
 This usually means one provides something the other doesn't and should.
ERROR: Multiple .bb files are due to be built which each provide xau
(/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-lib/libxau_1.0.3.bb
/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-lib/libxau-native_1.0.3.bb).
 This usually means one provides something the other doesn't and should.
NOTE: Executing runqueue
NOTE: Removing stamps:
/build/bhthermal/bht_build/oe/build/angstrom-2008.1/tmp/stamps/geode-angstrom-linux/font-util-1_1.0.1-r2.*
NOTE: Running task 527 of 656 (ID: 2,
/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-font/font-util_1.0.1.bb,
do_prepackaged_stage)
NOTE: package font-util-1.0.1: started
NOTE: package font-util-1_1.0.1-r2: task do_prepackaged_stage: started
NOTE: Uninstalling any existing package from staging...
Removing package staging-font-util-geode-angstrom-linux from root...
NOTE: Following speedup

Installing staging-font-util-geode-angstrom-linux (1.0.1-r2) to root...
Configuring staging-font-util-geode-angstrom-linux
NOTE: package font-util-1_1.0.1-r2: task do_prepackaged_stage: completed
NOTE: package font-util-1.0.1: completed
NOTE: Running task 654 of 656 (ID: 11,
/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-font/font-util_1.0.1.bb,
do_qa_staging)
NOTE: package font-util-1.0.1: started
NOTE: package font-util-1_1.0.1-r2: task do_qa_staging: started
NOTE: QA checking staging
NOTE: package font-util-1_1.0.1-r2: task do_qa_staging: completed
NOTE: package font-util-1.0.1: completed
NOTE: Running task 655 of 656 (ID: 0,
/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-font/font-util_1.0.1.bb,
do_build)
NOTE: package font-util-1.0.1: started
NOTE: package font-util-1_1.0.1-r2: task do_build: started
NOTE: package font-util-1_1.0.1-r2: task do_build: completed
NOTE: package font-util-1.0.1: completed
NOTE: Running task 656 of 656 (ID: 1,
/build/bhthermal/bht_build/oe/oe/org.openembedded.dev/packages/xorg-font/font-util_1.0.1.bb,
do_rebuild)
NOTE: package font-util-1.0.1: started
NOTE: package font-util-1_1.0.1-r2: task do_rebuild: started
NOTE: package font-util-1_1.0.1-r2: task do_rebuild: completed
NOTE: package font-util-1.0.1: completed
NOTE: Tasks Summary: Attempted 656 tasks of which 652 didn't need to
be rerun and 0 failed.
NOTE: build 200804241623: completed


-- 
=======================
Cliff Brake
http://bec-systems.com

Re: bitbake -c rebuild not running all tasks

[Richard Purdie]

On Thu, 2008-04-24 at 16:25 -0400, Cliff Brake wrote:
> I'm trying to sort out the xorg fonts stuff, and am running into an
> issue where "bitbake -c rebuild" does not re-run all tasks when
> rebuilding font-util.  It seems to me that bitbake should be running
> all tasks including configure, compile, etc.
> 
> I'm using bitbake 1.8 HEAD, and a week old or so version of OE.  I'll
> try the OE HEAD as soon as I get a build to finish.

There is a clue here:

> NOTE: package font-util-1_1.0.1-r2: task do_prepackaged_stage: started
> NOTE: Uninstalling any existing package from staging...
> Removing package staging-font-util-geode-angstrom-linux from root...
> NOTE: Following speedup
> 
> Installing staging-font-util-geode-angstrom-linux (1.0.1-r2) to root...
> Configuring staging-font-util-geode-angstrom-linux
> NOTE: package font-util-1_1.0.1-r2: task do_prepackaged_stage: completed

You have packaged staging enabled, it found the staging package and used
it, "accelerating" the build. 

When we run clean, the do_clean_append() in packaged-staging.bbclass
takes care of removing the staging package but it didn't. I can tell you
why too, basically rebuild is totally flawed with bitbake 1.8 HEAD and
some of the recent core changes :(.

Please don't use the rebuild command for now, use -c clean and then
rerun it. I will think about this problem but its a nasty tricky to
solve one. The best idea I have so far is we always have a clean task
that runs before any other, it just only does something in certain cases
governed by stamps.

Cheers,

Richard

Re: bitbake -c rebuild not running all tasks

[Khem Raj]

On Thu, Apr 24, 2008 at 2:51 PM, Richard Purdie <rpurdie@rpsys.net> wrote:
> On Thu, 2008-04-24 at 16:25 -0400, Cliff Brake wrote:
>  > I'm trying to sort out the xorg fonts stuff, and am running into an
>  > issue where "bitbake -c rebuild" does not re-run all tasks when
>  > rebuilding font-util.  It seems to me that bitbake should be running
>  > all tasks including configure, compile, etc.
>  >
>  > I'm using bitbake 1.8 HEAD, and a week old or so version of OE.  I'll
>  > try the OE HEAD as soon as I get a build to finish.
>
>  There is a clue here:
>
>
>  > NOTE: package font-util-1_1.0.1-r2: task do_prepackaged_stage: started
>  > NOTE: Uninstalling any existing package from staging...
>  > Removing package staging-font-util-geode-angstrom-linux from root...
>  > NOTE: Following speedup
>  >
>  > Installing staging-font-util-geode-angstrom-linux (1.0.1-r2) to root...
>  > Configuring staging-font-util-geode-angstrom-linux
>  > NOTE: package font-util-1_1.0.1-r2: task do_prepackaged_stage: completed
>
>  You have packaged staging enabled, it found the staging package and used
>  it, "accelerating" the build.
>
>  When we run clean, the do_clean_append() in packaged-staging.bbclass
>  takes care of removing the staging package but it didn't. I can tell you
>  why too, basically rebuild is totally flawed with bitbake 1.8 HEAD and
>  some of the recent core changes :(.
>
>  Please don't use the rebuild command for now, use -c clean and then
>  rerun it.

I was also being bitten by this problem on many packages. Thanks for clue.
I was trying this inadvertently so far.

I will think about this problem but its a nasty tricky to
>  solve one. The best idea I have so far is we always have a clean task
>  that runs before any other, it just only does something in certain cases
>  governed by stamps.
>
>  Cheers,
>
>  Richard
>
>
>
>
>
>  _______________________________________________
>  Openembedded-devel mailing list
>  Openembedded-devel@lists.openembedded.org
>  http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>

Re: bitbake -c rebuild not running all tasks

[Rolf Leggewie]

Richard Purdie wrote:
> Please don't use the rebuild command for now, use -c clean and then
> rerun it. I will think about this problem but its a nasty tricky to
> solve one.

how about having "bitbake -c rebuild $target" actually be "bitbake -c
clean $target;bitbake $target" for the time being as a stop-gap measure?
 I know you are not a fan of recursive bitbake, but would it be a
problem in this case?

Re: bitbake -c rebuild not running all tasks

[Richard Purdie]

On Sat, 2008-04-26 at 00:30 +0200, Rolf Leggewie wrote:
> Richard Purdie wrote:
> > Please don't use the rebuild command for now, use -c clean and then
> > rerun it. I will think about this problem but its a nasty tricky to
> > solve one.
> 
> how about having "bitbake -c rebuild $target" actually be "bitbake -c
> clean $target;bitbake $target" for the time being as a stop-gap measure?
>  I know you are not a fan of recursive bitbake, but would it be a
> problem in this case?

That breaks our layer separation. At present bitbake knows literally
nothing about what a given task means/does and to teach it that
behaviour it has to special case "rebuild". 

bitbake did used to do this at the task level, running one task after
the other but the way the core changed for many other reasons means it
can't do that. I'm not too happy that rebuild has broken but it is the
only collateral and we did gain a number of other features. I will find
a way to fix it and don't think its a terminal problem, just a hard one.

Yes, you could do something like this with recursive bitbake but that is
a large can of worms which cause infinitely more pain rather than being
stop-gap.

Richard