Re: "lock file exists" when fetching in bare clone of repository

[Patrick Steinhardt <ps@pks.im>]

On Wed, Aug 20, 2025 at 05:33:23PM -0400, Jeff King wrote:
> On Wed, Aug 20, 2025 at 08:54:37PM +0000, Joe Drew wrote:
> 
> > The fetch fails, returning status 255, with the below error message:
> > 
> >         remote: Enumerating objects: 18733, done.
> >         remote: Counting objects: 100% (562/562), done.
> >         remote: Compressing objects: 100% (448/448), done.
> >         remote: Total 18733 (delta 399), reused 144 (delta 112), pack-reused 18171
> >         Receiving objects: 100% (18733/18733), 25.32 MiB | 7.93 MiB/s, done.
> >         Resolving deltas: 100% (14608/14608), done.
> >         From <VALID-URL>
> > 
> >         [.... quite a number of branches and tags .....]
> > 
> >         error: cannot lock ref 'refs/heads/ptv-2164': Unable to create '/Users/joe.drew/tmp/./refs/heads/ptv-2164.lock': File exists.
> 
> > 
> >         Another git process seems to be running in this repository, e.g.
> >         an editor opened by 'git commit'. Please make sure all processes
> >         are terminated then try again. If it still fails, a git process
> >         may have crashed in this repository earlier:
> >         remove the file manually to continue.
> > 
> > This file doesn't exist; in fact, no such files exist:
> > 
> >         $ ls refs/heads
> >         $
> > 
> > However, the branch _does_ exist on the remote.
> 
> This is a wild guess, but: are there any case collisions with that
> branch name (e.g., PTV-2164 or something) in the upstream repo?
> 
> If so, and assuming you're on a case-insensitive filesystem, then the
> lock files would collide. You wouldn't see anything after the fact
> because git-fetch itself would create the colliding lockfile, and then
> clean it up after hitting the fatal error.

That's a likely failure mode indeed.

> If that is the case, you can try using the reftables backend in v2.51.0.
> It doesn't use the filesystem for its ref storage or locking. Something
> like:
> 
>   git init --bare --ref-format=reftable

Note that you can also `git clone --ref-format=reftable` directly, so
there's no need to go via git-init(1) first.

> > This error _does not_ happen in the Apple-supplied version of git:
> > `git version 2.39.5 (Apple Git-154)`, but does in 2.51.0, which I
> > installed with homebrew. (If this is a packaging error, I'll happily
> > report to homebrew.)
> 
> That is definitely weird, and not something I'd expect if it's just a
> case collision. Is it possible for you to build Git from source? If so,
> and the problem happens with your build of 2.51.0 but not v2.39.5, it
> would be very enlightening to see the results of "git bisect". We can
> provide more guidance if you need with that process.

Curious indeed. Another relevant part could be the new batched mode that
we have recently introduced in git-fetch(1). Previously we would've
updated refs one by one, and potentially that led to us writing the same
ref multiple times due to case-insensitivity? The result would be quite
broken in that case, but at least the fetch would've finished. Does make
me wonder though what subsequent fetches did, and whether we always tend
to flip-flop the value of such a ref.

But with batched transactions that's not possible anymore, as we would
indeed try to lock the same ref twice. Assuming that this really is the
case, I wonder whether we should detect this case, tell the user that
it's impossible to store all refs on their system, and then continue
regardless while ejecting that specific ref from the transaction.

Patrick