git .lock file error

git .lock file error

[Nate Parsons]

Hi everyone,

I'm trying to switch to git, so I apologize in advance for my
ignorance. For this, I'm using cygwin git, version 1.7.1 to talk to a
svn server running 1.4.2.

When I 'git svn clone' or 'git svn clone; git svn fetch', I sometimes
run into the following error.
> Couldn't open .git/svn/refs/remotes/0.0.0/.rev_map.b8cad480-e46b-48b4-8317-a683ee46c2bd.lock: Device or resource busy
>  at /usr/lib/git-core/git-svn line 5210

And when I 'git svn rebase'
> Couldn't open .git/svn/refs/remotes/git-svn/.rev_map.cc05479a-e8ea-436f-8d71-e07493b7796c.lock: Device or resource busy
> at /usr/lib/git-core/git-svn line 1528

Sometimes the line number is different (578, I think?), but when I
retry the fetch command, it rebuilds the file corresponding to the
lockfile, and then continues a little farther before dying again.

When I watch the directory in question, I see the lock file appear,
and then disappearing a short time after the script gives up.

I don't have a series of steps to reproduce the problem, sometimes it
goes for an hour, and sometimes for just a few minutes/seconds.

I do have TortoiseGit running my machine, but this issue happens even
when TGitCache.exe is not running. If some other instance of git is
accessing the repository, I don't know which or how.

Thanks,
-Nate

Re: git .lock file error

[Nate Parsons]

Hello again,

I realized I have git-cheetah running as well, so I killed
explorer.exe (the only process using git_shell_ext.exe), and tried
again. That's when I noticed that running 'git svn fetch' spawned two
git processes and a perl process (which later spawned another perl).
Is this something that should be happening?

Thanks,
-Nate

On Mon, Sep 13, 2010 at 11:19 AM, Nate Parsons <parsons.nate@gmail.com> wrote:
>
> Hi everyone,
>
> I'm trying to switch to git, so I apologize in advance for my
> ignorance. For this, I'm using cygwin git, version 1.7.1 to talk to a
> svn server running 1.4.2.
>
> When I 'git svn clone' or 'git svn clone; git svn fetch', I sometimes
> run into the following error.
> > Couldn't open .git/svn/refs/remotes/0.0.0/.rev_map.b8cad480-e46b-48b4-8317-a683ee46c2bd.lock: Device or resource busy
> >  at /usr/lib/git-core/git-svn line 5210
>
> And when I 'git svn rebase'
> > Couldn't open .git/svn/refs/remotes/git-svn/.rev_map.cc05479a-e8ea-436f-8d71-e07493b7796c.lock: Device or resource busy
> > at /usr/lib/git-core/git-svn line 1528
>
> Sometimes the line number is different (578, I think?), but when I
> retry the fetch command, it rebuilds the file corresponding to the
> lockfile, and then continues a little farther before dying again.
>
> When I watch the directory in question, I see the lock file appear,
> and then disappearing a short time after the script gives up.
>
> I don't have a series of steps to reproduce the problem, sometimes it
> goes for an hour, and sometimes for just a few minutes/seconds.
>
> I do have TortoiseGit running my machine, but this issue happens even
> when TGitCache.exe is not running. If some other instance of git is
> accessing the repository, I don't know which or how.
>
> Thanks,
> -Nate

Re: git .lock file error

[Nate Parsons]

OK, so this is definitely a win32 issue. I believe that the perl
script is simply creating .lock files too fast for Windows to keep up.
Simply trying again fixes the problem for me.

--- a/git-svn.perl
+++ b/git-svn.perl
@@ -3676,8 +3676,10 @@ sub rev_map_set {
 					    "$db => $db_lock ($!)\n";
 	}

-	sysopen(my $fh, $db_lock, O_RDWR | O_CREAT)
-	     or croak "Couldn't open $db_lock: $!\n";
+   my $fh;
+   my $tries=10;
+   for(; !sysopen($fh, $db_lock, O_RDWR | O_CREAT) && $tries>=0; $tries--) { }
+   if($tries <= 0) { croak "Couldn't open $db_lock: $!\n"; }
 	$update_ref eq 'reset' ? _rev_map_reset($fh, $rev, $commit) :
 				 _rev_map_set($fh, $rev, $commit);
 	if ($sync) {

If this is a reasonable solution, I can go figure out how to properly
format a patch to the list.

-Nate

On Sun, Sep 19, 2010 at 10:42 PM, Nate Parsons <parsons.nate@gmail.com> wrote:
>
> Hello again,
>
> I realized I have git-cheetah running as well, so I killed
> explorer.exe (the only process using git_shell_ext.exe), and tried
> again. That's when I noticed that running 'git svn fetch' spawned two
> git processes and a perl process (which later spawned another perl).
> Is this something that should be happening?
>
> Thanks,
> -Nate
>
> On Mon, Sep 13, 2010 at 11:19 AM, Nate Parsons <parsons.nate@gmail.com> wrote:
> >
> > Hi everyone,
> >
> > I'm trying to switch to git, so I apologize in advance for my
> > ignorance. For this, I'm using cygwin git, version 1.7.1 to talk to a
> > svn server running 1.4.2.
> >
> > When I 'git svn clone' or 'git svn clone; git svn fetch', I sometimes
> > run into the following error.
> > > Couldn't open .git/svn/refs/remotes/0.0.0/.rev_map.b8cad480-e46b-48b4-8317-a683ee46c2bd.lock: Device or resource busy
> > >  at /usr/lib/git-core/git-svn line 5210
> >
> > And when I 'git svn rebase'
> > > Couldn't open .git/svn/refs/remotes/git-svn/.rev_map.cc05479a-e8ea-436f-8d71-e07493b7796c.lock: Device or resource busy
> > > at /usr/lib/git-core/git-svn line 1528
> >
> > Sometimes the line number is different (578, I think?), but when I
> > retry the fetch command, it rebuilds the file corresponding to the
> > lockfile, and then continues a little farther before dying again.
> >
> > When I watch the directory in question, I see the lock file appear,
> > and then disappearing a short time after the script gives up.
> >
> > I don't have a series of steps to reproduce the problem, sometimes it
> > goes for an hour, and sometimes for just a few minutes/seconds.
> >
> > I do have TortoiseGit running my machine, but this issue happens even
> > when TGitCache.exe is not running. If some other instance of git is
> > accessing the repository, I don't know which or how.
> >
> > Thanks,
> > -Nate

Re: git .lock file error

[Ævar Arnfjörð Bjarmason]

On Thu, Sep 30, 2010 at 18:00, Nate Parsons <parsons.nate@gmail.com> wrote:
> OK, so this is definitely a win32 issue. I believe that the perl
> script is simply creating .lock files too fast for Windows to keep up.
> Simply trying again fixes the problem for me.

Sounds like something that might do with a non-hacky solution, but I
don't know what that would be.

> -       sysopen(my $fh, $db_lock, O_RDWR | O_CREAT)
> -            or croak "Couldn't open $db_lock: $!\n";
> +   my $fh;
> +   my $tries=10;
> +   for(; !sysopen($fh, $db_lock, O_RDWR | O_CREAT) && $tries>=0; $tries--) { }
> +   if($tries <= 0) { croak "Couldn't open $db_lock: $!\n"; }
>        $update_ref eq 'reset' ? _rev_map_reset($fh, $rev, $commit) :
>                                 _rev_map_set($fh, $rev, $commit);
>        if ($sync) {

FWIW I think this is more readable, but maybe we want to retry on all
platforms:

    my $fh;
    if ($^O eq 'MSWin32' or $^O eq 'cygwin') {
        # Try 10 times to open our lock file, in case Windows is lagging
        for my $try (1..10) {
            sysopen($fh, $db_lock, O_RDWR | O_CREAT);
            last if $fh;
        }
    } else {
        sysopen($fh, $db_lock, O_RDWR | O_CREAT);
    }

    warn "Couldnt open $db_lock: $!\n" unless $fh;'

Re: git .lock file error

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Thu, Sep 30, 2010 at 18:00, Nate Parsons <parsons.nate@gmail.com> wrote:
>> OK, so this is definitely a win32 issue. I believe that the perl
>> script is simply creating .lock files too fast for Windows to keep up.
>> Simply trying again fixes the problem for me.
>
> Sounds like something that might do with a non-hacky solution, but I
> don't know what that would be.
>
> ...
> FWIW I think this is more readable, but maybe we want to retry on all
> platforms:
>
>     my $fh;
>     if ($^O eq 'MSWin32' or $^O eq 'cygwin') {
>         # Try 10 times to open our lock file, in case Windows is lagging
>         for my $try (1..10) {
>             sysopen($fh, $db_lock, O_RDWR | O_CREAT);
>             last if $fh;
>         }
>     } else {
>         sysopen($fh, $db_lock, O_RDWR | O_CREAT);
>     }

What exactly does "Windows is lagging" mean in the above?

Why does sysopen() randomly fail and why does it succeed (sometimes) when
it immediately gets retried with the same argument?  Is this a shared lock
and is used by some other processes that drive git?  Why does the issue
manifest only on Windows?  If there are competing processes, wouldn't it
exacerbate the situation to run a tight loop to try grabbing the lock
without waiting, like the above patch does?

Re: git .lock file error

[Nate Parsons]

On Thu, Sep 30, 2010 at 4:15 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> > FWIW I think this is more readable, but maybe we want to retry on all
> > platforms:
> >
> >     my $fh;
> >     if ($^O eq 'MSWin32' or $^O eq 'cygwin') {
> >         # Try 10 times to open our lock file, in case Windows is lagging
> >         for my $try (1..10) {
> >             sysopen($fh, $db_lock, O_RDWR | O_CREAT);
> >             last if $fh;
> >         }
> >     } else {
> >         sysopen($fh, $db_lock, O_RDWR | O_CREAT);
> >     }
> >
> >     warn "Couldnt open $db_lock: $!\n" unless $fh;'

I agree that this is more readable, although I don't think there needs
to be separate code sections for each OS. If it's a good OS, it will
only loop once. You would want to die or croak instead of warn,
though. It needs the lock to continue, right?

> What exactly does "Windows is lagging" mean in the above?
>
> Why does sysopen() randomly fail and why does it succeed (sometimes) when
> it immediately gets retried with the same argument?  Is this a shared lock
> and is used by some other processes that drive git?  Why does the issue
> manifest only on Windows?  If there are competing processes, wouldn't it
> exacerbate the situation to run a tight loop to try grabbing the lock
> without waiting, like the above patch does?

All very good questions, and I don't know most of the answers. I know
that sysopen() is just a thin wrapper around the 'open' system call,
so the problem has to be with Windows somehow. It probably doesn't
happen with other OSes because they're better at files :P

I have verified (to the best of my ability) that this problem happens
even when TortoiseGit is not running, and I don't have cheetah
installed, and there are no other 'git.exe's running. I considered
putting some sort of sleep() in there, but since it worked so well
without the sleep (it only retried once at most). Fetching multiple
revisions is slow anyways, so I don't think a 2-second wait would
hurt, though.

-Nate

Re: git .lock file error

[Ævar Arnfjörð Bjarmason]

On Thu, Sep 30, 2010 at 21:10, Nate Parsons <parsons.nate@gmail.com> wrote:
> On Thu, Sep 30, 2010 at 4:15 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>> > FWIW I think this is more readable, but maybe we want to retry on all
>> > platforms:
>> >
>> >     my $fh;
>> >     if ($^O eq 'MSWin32' or $^O eq 'cygwin') {
>> >         # Try 10 times to open our lock file, in case Windows is lagging
>> >         for my $try (1..10) {
>> >             sysopen($fh, $db_lock, O_RDWR | O_CREAT);
>> >             last if $fh;
>> >         }
>> >     } else {
>> >         sysopen($fh, $db_lock, O_RDWR | O_CREAT);
>> >     }
>> >
>> >     warn "Couldnt open $db_lock: $!\n" unless $fh;'
>
> I agree that this is more readable, although I don't think there needs
> to be separate code sections for each OS. If it's a good OS, it will
> only loop once. You would want to die or croak instead of warn,
> though. It needs the lock to continue, right?

Yes, right. Meant to write croak() not warn() there.

>> What exactly does "Windows is lagging" mean in the above?
>>
>> Why does sysopen() randomly fail and why does it succeed (sometimes) when
>> it immediately gets retried with the same argument?  Is this a shared lock
>> and is used by some other processes that drive git?  Why does the issue
>> manifest only on Windows?  If there are competing processes, wouldn't it
>> exacerbate the situation to run a tight loop to try grabbing the lock
>> without waiting, like the above patch does?
>
> All very good questions, and I don't know most of the answers. I know
> that sysopen() is just a thin wrapper around the 'open' system call,
> so the problem has to be with Windows somehow. It probably doesn't
> happen with other OSes because they're better at files :P
>
> I have verified (to the best of my ability) that this problem happens
> even when TortoiseGit is not running, and I don't have cheetah
> installed, and there are no other 'git.exe's running. I considered
> putting some sort of sleep() in there, but since it worked so well
> without the sleep (it only retried once at most). Fetching multiple
> revisions is slow anyways, so I don't think a 2-second wait would
> hurt, though.

Maybe this has to do with our assumptions about being on a POSIX
filesystem, doesn't Windows have different semantics if two processes
open the same file, or something?

Blah, I don't know. I'm about as clueless when it comes to Windows as
VMS :)

Re: git .lock file error

[Erik Faye-Lund]

On Thu, Sep 30, 2010 at 11:10 PM, Nate Parsons <parsons.nate@gmail.com> wrote:
> On Thu, Sep 30, 2010 at 4:15 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>> > FWIW I think this is more readable, but maybe we want to retry on all
>> > platforms:
>> >
>> >     my $fh;
>> >     if ($^O eq 'MSWin32' or $^O eq 'cygwin') {
>> >         # Try 10 times to open our lock file, in case Windows is lagging
>> >         for my $try (1..10) {
>> >             sysopen($fh, $db_lock, O_RDWR | O_CREAT);
>> >             last if $fh;
>> >         }
>> >     } else {
>> >         sysopen($fh, $db_lock, O_RDWR | O_CREAT);
>> >     }
>> >
>> >     warn "Couldnt open $db_lock: $!\n" unless $fh;'
>
> I agree that this is more readable, although I don't think there needs
> to be separate code sections for each OS. If it's a good OS, it will
> only loop once. You would want to die or croak instead of warn,
> though. It needs the lock to continue, right?
>
>> What exactly does "Windows is lagging" mean in the above?
>>
>> Why does sysopen() randomly fail and why does it succeed (sometimes) when
>> it immediately gets retried with the same argument?  Is this a shared lock
>> and is used by some other processes that drive git?  Why does the issue
>> manifest only on Windows?  If there are competing processes, wouldn't it
>> exacerbate the situation to run a tight loop to try grabbing the lock
>> without waiting, like the above patch does?
>
> All very good questions, and I don't know most of the answers. I know
> that sysopen() is just a thin wrapper around the 'open' system call,
> so the problem has to be with Windows somehow. It probably doesn't
> happen with other OSes because they're better at files :P
>
> I have verified (to the best of my ability) that this problem happens
> even when TortoiseGit is not running, and I don't have cheetah
> installed, and there are no other 'git.exe's running.

Are you running some anti-virus, or from a Dropbox-folder (or
something similar)? If so, these applications tend to open files for a
short while and then release the file handle. This can some times
cause race conditions with other software, like Git.

I haven't looked into this particular code-path for what kind of
hazards might be, but this sounds very much like one such case to me.

Re: git .lock file error

[Nate Parsons]

On Thu, Sep 30, 2010 at 7:45 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> Are you running some anti-virus, or from a Dropbox-folder (or
> something similar)? If so, these applications tend to open files for a
> short while and then release the file handle. This can some times
> cause race conditions with other software, like Git.
>
> I haven't looked into this particular code-path for what kind of
> hazards might be, but this sounds very much like one such case to me.
>

Any of these things are possible. As much as I'd like to find the root
cause of this problem, isn't making Git more robust a good thing in
general? Finding every program that could conflict with Git and
disabling/uninstalling it isn't really a good long term solution, I
think.

Re: git .lock file error

[Erik Faye-Lund]

On Fri, Oct 1, 2010 at 2:54 AM, Nate Parsons <parsons.nate@gmail.com> wrote:
> On Thu, Sep 30, 2010 at 7:45 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>>
>> Are you running some anti-virus, or from a Dropbox-folder (or
>> something similar)? If so, these applications tend to open files for a
>> short while and then release the file handle. This can some times
>> cause race conditions with other software, like Git.
>>
>> I haven't looked into this particular code-path for what kind of
>> hazards might be, but this sounds very much like one such case to me.
>
> Any of these things are possible. As much as I'd like to find the root cause
> of this problem, isn't making Git more robust a good thing in general?
> Finding every program that could conflict with Git and
> disabling/uninstalling it isn't really a good long term solution, I think.

While making Git more robust is certainly a worthwhile goal, adding
retry-loops around any timing-sensitive code without finding out
exactly what went wrong is just cargo-cult programming.

I don't think I've ever seen such a breakage with open(..., O_CREAT)
before, so I'm sceptical.