git and concurrent access to local repository

git and concurrent access to local repository

[Boettger, Heiko]

Hi,

I always thought git allows concurrent access to a repository and expected that this also is true when working with local repositories. The only problem I was aware of is that the use of NFS and windows shares might cause problems. However I sometime see the following error messages in our concurrent build process.

message 1:

Fetching origin
error: cannot lock ref 'refs/remotes/origin/branchname  Unable to create '/home/workingcopy/.git/refs/remotes/origin/ branchname.lock': File exists.

If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.
From servername:gitrepo
! ccea072...349809e branchname  -> origin/branchname  (unable to update local ref)
error: Could not fetch origin

message 2:

error: cannot lock ref 'refs/remotes/origin/master': ref refs/remotes/origin/master is at 07cd4a461229f9352d16063ff209e828cba592ea but expected 7c4ddcf998bad94f4f7e8e806baaa475f2069e60

message 3 (on git pull --rebase):

Cannot rebase onto multiple branches

Does that mean there is an issue inside git's locking or is concurrent access to the same workingcopy not supported  at all? I expected the commands would blocking until they can acquire the lock, but It seems like there are other command which are either aborting or not locking at all (git pull -reabase). I think git pull --rebase is just a chain of other git commands where the lock is done on the individual steps.

Am I doing something wrong or is it concurrent access just not support this way? Does that mean I have to use my own locks such as putting flock around each call?

Best Regards
Heiko Böttger

Re: git and concurrent access to local repository

[Torsten Bögershausen]

On 2016-03-22 09.00, Boettger, Heiko wrote:
> Hi,
> 
> I always thought git allows concurrent access to a repository and expected that this also is true when working with local repositories. The only problem I was aware of is that the use of NFS and windows shares might cause problems. However I sometime see the following error messages in our concurrent build process.
> 
That depends.
concurrent read only access should work, AFAIK.
concurrent read/write access (in the way I understand what you describe below)
is not supported. (Well, that is my knowledge).

> message 1:
> 
> Fetching origin
> error: cannot lock ref 'refs/remotes/origin/branchname  Unable to create '/home/workingcopy/.git/refs/remotes/origin/ branchname.lock': File exists.
> 
> If no other git process is currently running, this probably means a
> git process crashed in this repository earlier. Make sure no other git
> process is running and remove the file manually to continue.
> From servername:gitrepo
> ! ccea072...349809e branchname  -> origin/branchname  (unable to update local ref)
> error: Could not fetch origin
> 
> message 2:
> 
> error: cannot lock ref 'refs/remotes/origin/master': ref refs/remotes/origin/master is at 07cd4a461229f9352d16063ff209e828cba592ea but expected 7c4ddcf998bad94f4f7e8e806baaa475f2069e60
> 
> message 3 (on git pull --rebase):
> 
> Cannot rebase onto multiple branches
> 
> Does that mean there is an issue inside git's locking or is concurrent access to the same workingcopy not supported  at all? I expected the commands would blocking until they can acquire the lock, but It seems like there are other command which are either aborting or not locking at all (git pull -reabase). I think git pull --rebase is just a chain of other git commands where the lock is done on the individual steps.
> 
The whole scenario does make little sense to me.
One user/process/shell script should update one work tree,
like fetch/pull/rebase. After that, the build is started.
This has nothing to do with NFS, it's more that it is clear,
what is done in which order, and to have things reproducible.

like
- fetch
- checkout
- make
- test.

How does the scenario work, that you want to achieve ?

And why and how are multiple users/processes work in the same directory ?


> Am I doing something wrong or is it concurrent access just not support this way? 
> Does that mean I have to use my own locks such as putting flock around each call?
I'm not sure, if I understand the scenario completely.
It looks, as if different processes share one work tree,
as if 2 persons share one keyboard, one doing lets say C-programming,
and the other is doing some shell script development. At the same time.
Using the same keyboard.

I don't know, if flock will help you, but I may miss something.
> 
> Best Regards
> Heiko Böttger

AW: AW: git and concurrent access to local repository

[Boettger, Heiko]

Hi Torsten,

yes, indeed I accidently pressed the wrong reply button.

The solution I'm using is the following:

pushd workingcopy > /dev/null
	(
		flock -x 200
		git reset --hard
		git clean -xfd
		git pull --rebase
	) 200>.git/gitrebase.lock
popd > /dev/null

Best Regards
Heiko

-----Ursprüngliche Nachricht-----
Von: Torsten Bögershausen [mailto:tboegi@web.de] 
Gesendet: Dienstag, 22. März 2016 14:54
An: Boettger, Heiko
Betreff: Re: AW: git and concurrent access to local repository

Thanks for the reply,
you may want to send it to all, including the git mailing list.
These kindof problems arise from time to time, and it may be nice to share your knowledge as well, including work-arounds


On 2016-03-22 10.54, Boettger, Heiko wrote:
> Thanks Torsten,
>
> for sharing your knowledge. That exactly what I thought. As a solution, I'm guarding the calls with "flock -x", which perfectly works since I know exactly where the repos are used. 
>
> Regards
> Heiko