rm_work causes image tasks to re-run

rm_work causes image tasks to re-run

[Mike Crowe]

When we have rm_work enabled all image tasks are built every time. This has
been happening since at least warrior and is still happening as of master
today (8fc19639f47b959a141dae231395bbababa644e1).

Steps to reproduce:

 bitbake core-image-minimal
 bitbake core-image-minimal
 echo 'INHERIT += "rm_work"' >> conf/local.conf
 bitbake core-image-minimal
 bitbake core-image-minimal

The second invocation of bitbake doesn't build anything at all (as would be
expected.)

The fourth invocation of bitbake always builds:

 Sstate summary: Wanted 43 Found 43 Missed 0 Current 436 (100% match, 100% complete)
 NOTE: Executing Tasks
 NOTE: Setscene tasks completed
 NOTE: Running noexec task 1750 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work_all)
 NOTE: Running task 1998 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_populate_lic_deploy)
 NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Started
 NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Succeeded
 NOTE: Running task 1999 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work)
 NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Started
 NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Succeeded
 NOTE: Running noexec task 2000 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_build)
 NOTE: Tasks Summary: Attempted 2000 tasks of which 1996 didn't need to be rerun and all succeeded.

If I teach rm_work.bbclass to also skip files matching
"*do_populate_lic_deploy*" then this problem goes away. However, I have
form for breaking things in rm_work so I suspect that this isn't the
correct fix. Is there a better one?

Thanks.

Mike.

Re: [OE-core] rm_work causes image tasks to re-run

[Jacob Kroon]

Hi Mike,

On 5/22/20 10:13 AM, Mike Crowe via lists.openembedded.org wrote:
> When we have rm_work enabled all image tasks are built every time. This has
> been happening since at least warrior and is still happening as of master
> today (8fc19639f47b959a141dae231395bbababa644e1).
> 
> Steps to reproduce:
> 
>   bitbake core-image-minimal
>   bitbake core-image-minimal
>   echo 'INHERIT += "rm_work"' >> conf/local.conf
>   bitbake core-image-minimal
>   bitbake core-image-minimal
> 
> The second invocation of bitbake doesn't build anything at all (as would be
> expected.)
> 
> The fourth invocation of bitbake always builds:
> 
>   Sstate summary: Wanted 43 Found 43 Missed 0 Current 436 (100% match, 100% complete)
>   NOTE: Executing Tasks
>   NOTE: Setscene tasks completed
>   NOTE: Running noexec task 1750 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work_all)
>   NOTE: Running task 1998 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_populate_lic_deploy)
>   NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Started
>   NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Succeeded
>   NOTE: Running task 1999 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work)
>   NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Started
>   NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Succeeded
>   NOTE: Running noexec task 2000 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_build)
>   NOTE: Tasks Summary: Attempted 2000 tasks of which 1996 didn't need to be rerun and all succeeded.
> 
> If I teach rm_work.bbclass to also skip files matching
> "*do_populate_lic_deploy*" then this problem goes away. However, I have
> form for breaking things in rm_work so I suspect that this isn't the
> correct fix. Is there a better one?
> 
> Thanks.
> 
> Mike.
> 

Since I'm the last guy touching rm_work.bbclass ...

I don't see the rebuilds in my images. Although I do see

core-image-minimal.bb:do_populate_lic_deploy

being run, there is no regeneration of the rootfs images.

Do you have any more information about the setup so that I can reproduce ?

/Jacob

Re: [OE-core] rm_work causes image tasks to re-run

[Mike Crowe]

On Friday 22 May 2020 at 19:02:47 +0200, Jacob Kroon wrote:
> Hi Mike,
> 
> On 5/22/20 10:13 AM, Mike Crowe via lists.openembedded.org wrote:
> > When we have rm_work enabled all image tasks are built every time. This has
> > been happening since at least warrior and is still happening as of master
> > today (8fc19639f47b959a141dae231395bbababa644e1).
> > 
> > Steps to reproduce:
> > 
> >   bitbake core-image-minimal
> >   bitbake core-image-minimal
> >   echo 'INHERIT += "rm_work"' >> conf/local.conf
> >   bitbake core-image-minimal
> >   bitbake core-image-minimal
> > 
> > The second invocation of bitbake doesn't build anything at all (as would be
> > expected.)
> > 
> > The fourth invocation of bitbake always builds:
> > 
> >   Sstate summary: Wanted 43 Found 43 Missed 0 Current 436 (100% match, 100% complete)
> >   NOTE: Executing Tasks
> >   NOTE: Setscene tasks completed
> >   NOTE: Running noexec task 1750 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work_all)
> >   NOTE: Running task 1998 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_populate_lic_deploy)
> >   NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Started
> >   NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Succeeded
> >   NOTE: Running task 1999 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work)
> >   NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Started
> >   NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Succeeded
> >   NOTE: Running noexec task 2000 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_build)
> >   NOTE: Tasks Summary: Attempted 2000 tasks of which 1996 didn't need to be rerun and all succeeded.
> > 
> > If I teach rm_work.bbclass to also skip files matching
> > "*do_populate_lic_deploy*" then this problem goes away. However, I have
> > form for breaking things in rm_work so I suspect that this isn't the
> > correct fix. Is there a better one?
> > 
> > Thanks.
> > 
> > Mike.
> > 
> 
> Since I'm the last guy touching rm_work.bbclass ...
> 
> I don't see the rebuilds in my images. Although I do see
> 
> core-image-minimal.bb:do_populate_lic_deploy
> 
> being run, there is no regeneration of the rootfs images.

I was originally investigating on warrior where the do_rootfs task does run
unnecessarily. It turns out that if I cherry-pick the three subsequent
rm_work changes from master back to warrior then it behaves the same as
master. I shall do that in our tree.

Is it possible to avoid the do_populate_lic_deploy task running
unnecessarily on master too?

Thanks.

Mike.