Re: Multiple conncetion to svn (svn+ssh)

["mar.krzeminski" <mar.krzeminski@gmail.com>]

Hi Brian, Paul,

Thanks for response.

W dniu 06.07.2015 o 17:11, Paul Eggleton pisze:
> On Monday 06 July 2015 15:04:57 Bryan Evenson wrote:
>> Paul,
>>
>>> -----Original Message-----
>>> From: Paul Eggleton [mailto:paul.eggleton@linux.intel.com]
>>> Sent: Monday, July 06, 2015 10:08 AM
>>> To: Marcin Krzemiński
>>> Cc: Bryan Evenson; yocto@yoctoproject.org
>>> Subject: Re: [yocto] Multiple conncetion to svn (svn+ssh)
>>>
>>> On Monday 06 July 2015 12:57:39 Bryan Evenson wrote:
>>>> Marcin,
>>>>
>>>>> From: yocto-bounces@yoctoproject.org
>>>>> [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Marcin
>>>>> Krzeminski
>>>>> Sent: Friday, July 03, 2015 7:55 AM
>>>>> To: yocto@yoctoproject.org
>>>>> Subject: [yocto] Multiple conncetion to svn (svn+ssh)
>>>>>
>>>>> Hello again,
>>>>>
>>>>> I have 12 recipes that download code from same svn repository (using
>>>>> svn+ssh)protocol. Recipes are grouped in packagegroup.
>>>>> When I want to bitbake packagegroup fetcher fail, but when I run
>>>>> recipe alone all is ok. I think it is because I want to open 8
>>>>> connection to one svn repository.How can I fix this, for example by
>>>>> allowing only eg. 2 recipes from packagegroup to be executed in
>>>>> paralell.
>>>> There is no fetcher-specific variable that I know of, but I think you
>>>> can set PARALLEL_MAKE="2" in your recipes to reduce the number of
>>>> fetches on your repository at a time.  The downside is your build will
>>>> go slower when it is building your recipes as it won't be doing as
>>>> many things in parallel.
>>> That's not going to help. The parallelism we are talking about here is
>>> across recipes - PARALLEL_MAKE is just passed through to make within one
>>> task in one recipe, and make isn't even involved at the fetch stage.
>> Sorry, I grabbed the wrong variable.  I meant BB_NUMBER_THREADS, not
>> PARALLEL_MAKE.
> Sure, but that's not going to work either from within a recipe. It would work at the configuration level but that'll quite severely impact build performance.
This was my first idea.
>
>>> Something that might work would be to set a lockfile on the do_fetch task
>>> such that only one of the recipes could fetch at once. That could not
>>> allow
>>> two executing at once, but at least it would solve the problem. e.g. you
>>> could add this to all of the recipes:
>>>
>>> do_fetch[lockfiles] += "${TMPDIR}/mysvnlock.lock"
>>>
>>> (The file name isn't critical, it just needs to be the same for all of the
>>> recipes you wish to participate in the exclusive fetching.)
This works as you wrote, it is not a perfect solution for my problem,
but at least builds doesn't fail. Thanks!
>> I had no idea that we could do this.  I don't see any documentation on
>> lockfiles anywhere. If you use a lockfile, do the all recipes with the same
>> lockfile wait until the lockfile is available before continuing on with
>> that step?  Is there a timeout for waiting for the lockfile or does the
>> recipe wait indefinitely?
> Yes, it's just a lock on the specified file - whoever gets there first can
> continue, everyone else blocks, and it's an indefinite wait as far as
> I'm aware.
>
> It's a little obscure perhaps, but it is in the BitBake manual:
>
> http://www.yoctoproject.org/docs/current/bitbake-user-manual/bitbake-user-manual.html#variable-flags
>
> Cheers,
> Paul
>
Regards,
Marcin