git clone fails intermittently if source repo is updated

git clone fails intermittently if source repo is updated

[Per Cederqvist]

Running "git clone master.git copy" sometimes fails if changes are being
pushed to master.git.  Likewise, running "git clone mirror.git copy"
sometimes fails if the mirror is currently fetching changes from
master.git.

In git 1.7.9 I sometimes get one of two fatal error messages.  In one of
the cases, git still exits with exit code 0, but the working directory
of "copy" is not updated with the proper files.  (See below for the
exact error messages.)

In git 1.7.10 the mirror sometimes ends up in "detached HEAD" state.
Sometimes it prints a warning message.  (See below for the messages.)

The following shell scripts can reproduce the problem.  I've been using
a 4-core CPU on Ubuntu 11.04.  I've compiled Git from source.

The first script, test-git-setup.sh, creates the directory "test-git"
and three git repositories in it: master.git, mirror.git, and committer:

------ cut here for test-git-setup.sh ------
#!/bin/sh
mkdir test-git || exit 1
cd test-git
mkdir master.git
(cd master.git && git init --bare)
git clone master.git committer
(cd committer &&
    touch foo &&
    git add foo &&
    git commit -m"Initial commit" &&
    git push origin master)
git clone --mirror master.git mirror.git
------ cut here ------

The second script continuously modifies the file "foo" in the committer
repository, commits it, and pushes it to master.git:

------ cut here for test-git-commit.sh ------
#!/bin/sh
set -e
cd test-git/committer
i=0
while :
do
    echo $i > foo
    git add foo
    git commit -m"Updated foo"
    git push
    i=`expr $i + 1`
done
------ cut here ------

The third script repeatedly clones a repository, and ensures that the
"git clone" command does exit 0, prints no unexpected output, and
creates a sane clone (the file "foo" must exist).  You must give it the
argument "master" or "mirror" to tell it which repository to clone.

------ cut here for test-git-reclone.sh ------
#!/bin/sh

what=$1

case "$what" in
    master);;
    mirror);;
    *) echo need to specify master or mirror >&2
       exit 1;;
esac

good=0
cd test-git || exit 1
rm -rf $what-clone
while :
do
    git clone $what.git $what-clone >$what.out 2>$what.err || exit 1
    if [ -s $what.err ]
    then
	echo unexpected stderr from git clone:
	cat $what.err
	exit 1
    fi
    if [ "`cat $what.out`" != "Cloning into '$what-clone'...
done." ]
    then
	echo unexpected stdout from git clone:
	cat $what.out
	exit 1
    fi
    if ! [ -f $what-clone/foo ]
    then
	echo failed to create $what-clone/foo
	exit 1
    fi
    rm -rf $what-clone
    good=`expr $good + 1`
    echo "$good good clones"
done
------ cut here ------

The final script repeatedly runs "git fetch" in mirror.git, to update it
from master.git.

------ cut here for test-git-mirror.sh ------
#!/bin/sh
cd test-git/mirror.git
while :
do
    echo running fetch...
    git fetch || exit 1
    echo done.
done
------ cut here ------

To reproduce, run this command:

    ./test-git-setup.sh

When it has completed, run the following commands at once (preferably
in four separate windows so you can see what is going on).  Start them
in the indicated order.

    ./test-git-reclone.sh master
    ./test-git-reclone.sh mirror
    ./test-git-mirror.sh
    ./test-git-commit.sh

The reclone commands will print lines such as these:

    1 good clones
    2 good clones
    3 good clones
    4 good clones

Let them run to at least 200 good clones before you decide that you
cannot reproduce the problem.

Using Git 1.7.10 I get the following error after a while:

> unexpected stderr from git clone:
> Note: checking out 'c210776284dc8bc1c3f6eecc316b539c9aea85c9'.
>
> You are in 'detached HEAD' state. You can look around, make experimental
> changes and commit them, and you can discard any commits you make in this
> state without impacting any branches by performing another checkout.
>
> If you want to create a new branch to retain commits you create, you may
> do so (now or later) by using -b with the checkout command again. Example:
>
>   git checkout -b new_branch_name
>

When this happens, I can recover by running "git checkout master" in the
clone, but before I do so "git status" reports:

> # Not currently on any branch.
> nothing to commit (working directory clean)

and "git branch -a" reports:

> * (no branch)
>   remotes/origin/master

Sometimes I get a different error:

> unexpected stderr from git clone:
> warning: failed to stat /home/cederp/test-git/mirror.git/objects/f6/tmp_obj_TZjRRd
>

or

> unexpected stderr from git clone:
> warning: failed to stat /home/cederp/test-git/master.git/objects/59/tmp_obj_SZtkWc
>

Using Git 1.7.9 I sometimes get this error when running
"./test-git-reclone.sh mirror":

> Cloning into 'mirror-clone'...
> done.
> error: refs/remotes/origin/master does not point to a valid object!
> error: Trying to write ref refs/heads/master with nonexistent object 5d049be4cd93c230a88503d8f9b4057587c76b2d
> fatal: Cannot update the ref 'HEAD'.

Sometimes I get this message:

> Cloning into 'mirror-clone'...
> done.
> error: unable to find 1338993ab85d24f451be1cd6565c4685c24fcc15
> error: unable to read sha1 file of foo (1338993ab85d24f451be1cd6565c4685c24fcc15)

In this case, "git clone" does exit(0), but the file "foo" does not
exist in the created mirror.

    /ceder

Re: git clone fails intermittently if source repo is updated

[Per Cederqvist]

On 04/17/2012 02:54 PM, Per Cederqvist wrote:

> Running "git clone master.git copy" sometimes fails if changes are being
> pushed to master.git.  Likewise, running "git clone mirror.git copy"
> sometimes fails if the mirror is currently fetching changes from
> master.git.
>
> In git 1.7.9 I sometimes get one of two fatal error messages.  In one of
> the cases, git still exits with exit code 0, but the working directory
> of "copy" is not updated with the proper files.  (See below for the
> exact error messages.)
>
> In git 1.7.10 the mirror sometimes ends up in "detached HEAD" state.
> Sometimes it prints a warning message.  (See below for the messages.)
>
> The following shell scripts can reproduce the problem.  I've been using
> a 4-core CPU on Ubuntu 11.04.  I've compiled Git from source.
>
> The first script, test-git-setup.sh, creates the directory "test-git"
> and three git repositories in it: master.git, mirror.git, and committer:
>
 > [...four shellscripts omitted...]

For your convenience, I have now set up a Git repository where you can
get slightly updated versions of the above scripts.  The Gitorious web
page is here:

http://git.lysator.liu.se/ceder/git-clone-fail-demo

You can clone it like this:

git clone git://git.lysator.liu.se/ceder/git-clone-fail-demo.git

I have been able to reproduce the problem on Git 1.6.0 as well, so
this is apparently not a recent regression.

The problem can also be reproduces when you clone, push and fetch via
ssh, but it happens a lot less frequently then.  See the README file in
the git-clone-fail-demo repository for info on how to reproduce it.

> [...]
>
>      /ceder

     /ceder