Re: [PATCH] image: Attempt to work around do_image_complete race

[Mike Crowe <mac@mcrowe.com>]

On Monday 13 February 2017 at 09:38:04 -0800, Khem Raj wrote:
> On Mon, Feb 13, 2017 at 4:41 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > If two do_image_complete tasks run in parallel they risk both trying to put
> > their image into ${DEPLOY_DIR_IMAGE} at the same time. Both will contain a
> > README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt file. In theory this
> > should be safe because "cp -alf" will just cause one to overwrite the
> > other. Unfortunately, coreutils cp also has a race which means that if one
> > copy creates the file at just the wrong point the other will fail with:
> >
> >  cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to ‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists
> >
> > The coreutils bug has been reported and fixed upstream as
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680 , but it will take
> > a while for host distributions to catch up.
> >
> > In the meantime, let's try and avoid the race altogether by causing
> > sstate.bbclass:sstate_install to take a lock file in
> > ${DEPLOY_DIR_IMAGE} whilst writing. This lock is probably a bit broad,
> > but it's better than the build failing.
> >
> 
> will this lock serialize writing other files too ? then this will slow
> down parallel builds. Perhaps, writing common files can be spun into a
> task of its own which is then common 1 task across all images building
> in parallel.

That sounds like a great idea.

Or, we could just lose the common README file altogether. :)

Mike.