Re: Shared DL_DIR with other users

[Mark Hatle <mark.hatle@windriver.com>]

On 3/26/13 9:25 AM, Paul Eggleton wrote:
> Hi Simon,
>
> On Tuesday 26 March 2013 14:37:36 Vanveerdeghem, Simon wrote:
>> We are using Yocto with a couple of users, to prevent that our build
>> server has duplicates of downloads, we share the DL_DIR.
>>
>> But we have problems with permissions. When a user run yocto, it will
>> checkout sources, or download tar files with permissions 644 with the
>> username of the user who is running yocto.
>>
>> When another user will do a cleanall of a bitbake recipe, yocto will
>> remove the tarball and the svn workingcopy, but because the permissions
>> aren't 777 It cannot remove all the files (yocto doesn't throw an error
>> here).
>>
>> When that other user will do a fetch of thqt recipe, yocto will checkout
>> the svn directory, it will leave as good as clean because it says
>> internally "Skipped ." (yocto doesn't throw an error here) then it wil
>> take a tarball of it, again the tarball has only a directory named of
>> the checked out module, but have no sources. (yocto doesn't throw an
>> error here)
>>
>> When yocto will building the sources it will fail because the tar was
>> clean, and there is nothing to build.
>>
>> The rootcause of this problem is that the shared download directory only
>> use 644. (We can workaround this with the .profile (we can set in the
>> .profile a standard permission mask, but then you always use that mask,
>> something we don't want)
>
> It's likely that sharing DL_DIR directly between multiple users is not a
> heavily tested scenario. One way around it which is fairly widely used
> though is to set up a local mirror shared by all of the build machines
> and keep DL_DIR separate on each machine. There is some information on
> how to do that here:
>
> https://wiki.yoctoproject.org/wiki/How_do_I#Q:_How_do_I_create_my_own_source_download_mirror_.3F

This is how we do it.  We have one user who routinely does a fetchall against 
the trees and populates the mirror.  Then the users have a layer which adds:

PREMIRRORS_append = "\
      git://.*/.* file://path/ \n \
      svn://.*/.* file://path/ \n \
      ftp://.*/.* file://path/ \n \
      http://.*/.* file://path/ \n \
      https://.*/.* file://path/ \n \
"

--Mark

> Cheers,
> Paul
>