Re: [Patch] locking in do_package_tar

[Jens Erdmann <j.erdmann@road.de>]

On 04/24/2010 06:56 PM, Denys Dmytriyenko wrote:
> On Fri, Apr 23, 2010 at 11:55:15AM -0400, Denys Dmytriyenko wrote:
>> On Fri, Apr 23, 2010 at 12:04:44PM +0200, Jens Erdmann wrote:
>>> Hey folks,
>>>
>>> I would like to introduce locking in do_package_tar. I had a case here were
>>> do_package_tar and do_package_ipk where running parallel. Because of this
>>> "tar: .: file changed as we read it" occurs. I fixed it by introduce
>>> locking in do_package_tar. What do you think about it?
>>
>> Ah, I've been suffering from this problem for quite some time now, when making
>> parallel builds with BB_NUMBER_THREADS. I was trying to debug it and noticed
>> it would fail when kernel recipe would try to do_deploy at the same time
>> another package would try to do_package_stage, so I was digging in the
>> packaged staging direction. Tom Rini even made a patch to add an extra lock in
>> there, which didn't help with my issue...
>> So, let me try your patch and see if this problem is gone for good! Thanks.
>
> It may solve some other similar issue, but unfortunately, it didn't seem to
> work for me, as it failed again the same way with the patch applied:
>
> NOTE: Running task 2011 of 2173 (ID: 953, /OE/arago-oe-dev/recipes/freetype/freetype_2.3.9.bb, do_package_stage)
> /OE/arago-tmp/work/armv7a-none-linux-gnueabi/freetype-2.3.9-r1/staging-pkg/deploy/ipk/armv7a/
> /OE/arago-tmp/work/armv7a-none-linux-gnueabi/freetype-2.3.9-r1/staging-pkg/deploy/ipk/armv7a/
> /OE/arago-tmp/work/armv7a-none-linux-gnueabi/freetype-2.3.9-r1/staging-pkg/deploy/ipk/armv7a/
> /OE/arago-tmp/work/armv7a-none-linux-gnueabi/freetype-2.3.9-r1/staging-pkg/deploy/ipk/armv7a/
> ERROR: function staging_packager failed
> ERROR: log data follows (/OE/arago-tmp/work/dm3730-am3715-evm-none-linux-gnueabi/linux-omap-psp-2.6.32-r60+gitr7b8926aa626991fa087b00f6bbc1fb6b0e8269b0/temp/log.staging_packager.19650)
> | tar: .: file changed as we read it
> NOTE: Task failed: /OE/arago-tmp/work/dm3730-am3715-evm-none-linux-gnueabi/linux-omap-psp-2.6.32-r60+gitr7b8926aa626991fa087b00f6bbc1fb6b0e8269b0/temp/log.staging_packager.19650
> NOTE: oestats: task failed, see http://tinderbox.openembedded.net/packages/556734/
> ERROR: TaskFailed event exception, aborting
> ERROR: Build of /OE/arago-oe-dev/recipes/linux/linux-omap-psp_2.6.32.bb do_package_stage failed
> ERROR: Task 1009 (/OE/arago-oe-dev/recipes/linux/linux-omap-psp_2.6.32.bb, do_package_stage) failed
> NOTE: Waiting for 3 active tasks to finish
> NOTE: 1: /OE/arago-oe-dev/recipes/freetype/freetype_2.3.9.bb, do_package_stage (21747)
>
> So, it looks like my problem is actually still packaged staging related, as is
> fails when 2 recipes run do_package_stage task at the same time (in the
> example above, it's freetype and linux-omap-psp)...
>

This looks more packaged-staging.bbclass related. Maybe you could try to 
make the lock in line 444 two lines earlier so that staging_packager and 
staging_helper will be locked. too.

To my patch: Is locking in tar a good idea?


-- 
------- ROAD ...the handyPC Company - - -  ) ) )

Jens Erdmann
Software Development

ROAD GmbH
Bennigsenstr. 14 | 12159 Berlin | Germany
fon: +49 (30) 230069 - 64 | fax: +49 (30) 230069 - 69
url: www.road.de

Amtsgericht Charlottenburg: HRB 96688 B
Managing director: Hans-Peter Constien