Race condition with git-status and git-rebase

Race condition with git-status and git-rebase

[Yiannis Marangos]

Hi,

Since I don't know how to fix it, this is my bug report:

git-status reads the index file then holds the `index.lock', it writes
the updated index in `index.lock' and renames the `index.lock' to
`index', but if it used with git-rebase then there is a (rare) race
condition.

I reproduce this at my work under the following conditions:
1) The repository is very big
2) I can reproduce it only in Windows. I can not reproduce it in Linux
   (my Linux machine has faster CPU and SSD drive.)

I saw this race condition at my coworkers when they use two instances
(in the same repository) of Git-Extensions GUI, and when they use one
instance of Git-Extensions and at the same time they rebase from
terminal. This race condition happens because Git-Extensions monitors
the directory of the repository and when a file is accessed it calls
git-status.

Since I can not share the repository, I decided to trace down the
problem. I came up with the following:

process A calls git-rebase
process A applies the 1st commit
process B calls git-status
process B calls read_cache() (status_init_config() -> gitmodules_config() -> read_cache())
process A applies the 2nd commit
process B holds the index.lock
process B writes back the old index (the one that was read from read_cache())
process A applies the 3rd commit (now the 3rd commit contains also a revert of the 2nd)

The content of the files of the working directory is correct (i.e. as
it should be if the race condition didn't happen). But git-status
shows that the files of the 2nd commit are modified, the git-diff show
the exact patch of the 2nd commit.

So, in this case process B must avoid calling write_index(). Maybe
something is missing from update_index_if_able()? Maybe git should
check the state before proceeding to write_index() (for example that
we are in rebase)?

Something else that I noticed is that when the race condition happens
the istate->cache_changed is 0 and has_racy_timestamp(istate) is 1, so
the if statement in update_index_if_able() continues to write_index().

I didn't check if there are other similar race conditions.

Regards,
Yiannis.

PS: I tried to move hold_locked_index() above status_init_config() but
this cause other problems.

Re: Race condition with git-status and git-rebase

[Yiannis Marangos]

Maybe you will find it useful: this is one way that I use to manually
debug the race condition (and it works in Linux):

diff --git a/builtin/commit.c b/builtin/commit.c
index d9550c5..c0511f6 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1283,6 +1283,8 @@ int cmd_status(int argc, const char **argv, const char *prefix)
 		usage_with_options(builtin_status_usage, builtin_status_options);
 
 	status_init_config(&s, git_status_config);
+	printf("read_cache() finished, press enter to continue.\n");
+	getchar();
 	argc = parse_options(argc, argv, prefix,
 			     builtin_status_options,
 			     builtin_status_usage, 0);
diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index df46f4c..6ffd986 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -61,8 +61,8 @@ else
 		return $?
 	fi
 
-	git am $git_am_opt --rebasing --resolvemsg="$resolvemsg" \
-		${gpg_sign_opt:+"$gpg_sign_opt"} <"$GIT_DIR/rebased-patches"
+	git am $git_am_opt -i --rebasing --resolvemsg="$resolvemsg" \
+		${gpg_sign_opt:+"$gpg_sign_opt"} "$GIT_DIR/rebased-patches"
 	ret=$?
 
 	rm -f "$GIT_DIR/rebased-patches"
diff --git a/read-cache.c b/read-cache.c
index ba13353..a6f73a7 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1778,6 +1778,8 @@ static int has_racy_timestamp(struct index_state *istate)
  */
 void update_index_if_able(struct index_state *istate, struct lock_file *lockfile)
 {
+	printf("cache_changed: %d  has_racy_timestamp: %d\n",
+	       istate->cache_changed, has_racy_timestamp(istate));
 	if ((istate->cache_changed || has_racy_timestamp(istate)) &&
 	    !write_index(istate, lockfile->fd))
 		commit_locked_index(lockfile);

Re: Race condition with git-status and git-rebase

[Junio C Hamano]

Yiannis Marangos <yiannis.marangos@gmail.com> writes:

> process A calls git-rebase
> process A applies the 1st commit
> process B calls git-status
> process B calls read_cache()
> process A applies the 2nd commit
> process B holds the index.lock
> process B writes back the old index (the one that was read from read_cache())
> process A applies the 3rd commit (now the 3rd commit contains also a revert of the 2nd)

The above sequence shows we are clearly doing it wrong.  B's taking
of the lock is not protecting anything from anybody.

We try minimizing the lock contention by delaying the taking of the
lock by B, but even if we leave it at the current location in the
above sequence, at least B must verify that the contents of the
index it just took the lock on still matches what it read with its
earlier call to read_cache(), which would make it pretend as if it
took the lock before it did read_cache(), before writing the
refreshed results back.

I think we have a checksum for the entire index file at the end, so
one fix might be to teach read_index_from() to read that and store
it in a new field in "struct index_state", and then make the
"opportunistic update" codepath to verify that the index on the
filesystem still has the same checksum.