Re: [PATCH] Allow for simultaneous do_rootfs tasks with rpm

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2016-07-14 at 14:20 -0700, Stephano Cetola wrote:
> Give each rootfs its own RPM channel to use.  This puts the RPM
> metadata
> in a private subdirectory of $WORKDIR, rather than living in
> DEPLOY_DIR
> where other tasks may race with it.
> 
> This allows us to reduce the time that the rpm.lock is held to only
> the
> time needed to hardlink the RPMs, allowing the majority of the rootfs
> operation to run in parallel.
> 
> [ YOCTO #9255 ]
> 
> Signed-Off-By: Steven Walter <stevenrwalter@gmail.com>
> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
> ---
>  meta/classes/rootfs_rpm.bbclass |  5 -----
>  meta/lib/oe/package_manager.py  | 17 ++++++++++++++---
>  2 files changed, 14 insertions(+), 8 deletions(-)

Sadly, much as I'd love to merge this, testing shows we have some
issues.

Firstly, it means we no longer generate indexes in tmp/deploy/rpm and
this breaks -c testimage since some of the tests connect and look for
packages there, e.g. https://autobuilder.yoctoproject.org/main/builders
/nightly-qa-logrotate/builds/851 but many others would have also
failed.

I've also started wondering what happens if we have some scenario where
we have:

bitbake core-image-sato epiphany
<rootfs for core-image-sato starts>
<rebuild for epiphany or one of its dependencies happens>
<old package from epiphany or one of its dependencies is removed from
tmp/deploy/rpm>
<core-image-sato index references this now removed package>

I'd hope the system should handle this since it should only be
referencing things that are direct task dependencies but with
rpm/smart, who knows :/. We should at least see how this fails (if it
fails).

It could get particularly ugly if we use globbing for the rootfs
construction (e.g. *-locale) and the list of packages changed depending
on what else was built. I'd guess we'd already have this problem but
this patch could magnify any problem.

We are going to have to at least solve the testimage package index
problem before we can merge it.

Cheers,

Richard