[PATCH] merge: avoid write merge state when unable to write index

[PATCH] merge: avoid write merge state when unable to write index

[Kyle Zhao via GitGitGadget]

From: Kyle Zhao <kylezhao@tencent.com>

Currently, when index.lock exists, if a merge occurs, the merge state
will be written and the index will be unchanged.

If the user exec "git commit" instead of "git merge --abort" after this,
the commit will be successful and all modifications from the source
branch will be lost.

Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
---
    merge: avoid write merge state when unable to write index
    
    In some of our monorepos, code is sometimes lost after merging.
    
    After investigation, we discovered the problem.
    
    This happens if we perform "git pull" or "git merge" when another git
    process is writing to the index, especially in a monorepo (because its
    index will be larger).
    
    How to reproduce:
    
    git init demo
    cd demo
    touch 1.txt && git add . && git commit -m "1"
    git checkout -b source-branch
    touch 2.txt && git add . && git commit -m "2"
    echo "1" >> 1.txt && git add . && git commit -m "3"
    touch .git/index.lock
    git merge source-branch
    rm .git/index.lock
    git commit -m "4"
    
    
    Then the modifications from the source branch are lost.
    
    Regards, Kyle
    
    cc: Elijah Newren newren@gmail.com cc: Ævar Arnfjörð Bjarmason
    avarab@gmail.com cc: Junio C Hamano gitster@pobox.com

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1731%2Fkeyu98%2Fkz%2Ffix-merge-when-index-lock-exists-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1731/keyu98/kz/fix-merge-when-index-lock-exists-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1731

 builtin/merge.c  |  6 ++++--
 t/t7600-merge.sh | 10 ++++++++++
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 6a6d3798858..80e3438be25 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -698,8 +698,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 
 	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
 					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
-					 NULL) < 0)
-		return error(_("Unable to write index."));
+					 NULL) < 0) {
+		error(_("Unable to write index."));
+		return 2;
+	}
 
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
 	    !strcmp(strategy, "ort")) {
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index e5ff073099a..f03709ea4be 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
 	verify_parents $c1 $c2
 '
 
+test_expect_success 'merge c1 with c2 when index.lock exists' '
+	test_when_finished rm .git/index.lock &&
+	git reset --hard c1 &&
+	touch .git/index.lock &&
+	test_must_fail git merge c2 &&
+	test_path_is_missing .git/MERGE_HEAD &&
+	test_path_is_missing .git/MERGE_MODE &&
+	test_path_is_missing .git/MERGE_MSG
+'
+
 test_expect_success 'merge --squash c3 with c7' '
 	git reset --hard c3 &&
 	test_must_fail git merge --squash c7 &&

base-commit: 19fe900cfce8096b7645ec9611a0b981f6bbd154
-- 
gitgitgadget

[PATCH v2] merge: avoid write merge state when unable to write index

[Kyle Zhao via GitGitGadget]

From: Kyle Zhao <kylezhao@tencent.com>

Currently, when index.lock exists, if a merge occurs, the merge state
will be written and the index will be unchanged.

If the user exec "git commit" instead of "git merge --abort" after this,
a merge commit will be generated and all modifications from the source
branch will be lost.

Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
---
    merge: avoid write merge state when unable to write index
    
    In some of our monorepos, code is sometimes lost after merging.
    
    After investigation, we discovered the problem.
    
    This happens if we perform "git pull" or "git merge" when another git
    process is writing to the index, especially in a monorepo (because its
    index will be larger).
    
    How to reproduce:
    
    git init demo
    cd demo
    touch 1.txt && git add . && git commit -m "1"
    git checkout -b source-branch
    touch 2.txt && git add . && git commit -m "2"
    git checkout master
    echo "1" >> 1.txt && git add . && git commit -m "3"
    touch .git/index.lock
    git merge source-branch
    rm .git/index.lock
    git commit -m "4"
    
    
    Then the modifications from the source branch are lost.
    
    Regards, Kyle

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1731%2Fkeyu98%2Fkz%2Ffix-merge-when-index-lock-exists-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1731/keyu98/kz/fix-merge-when-index-lock-exists-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1731

Range-diff vs v1:

 1:  aab0be55989 ! 1:  86101e5b778 merge: avoid write merge state when unable to write index
     @@ Commit message
          will be written and the index will be unchanged.
      
          If the user exec "git commit" instead of "git merge --abort" after this,
     -    the commit will be successful and all modifications from the source
     +    a merge commit will be generated and all modifications from the source
          branch will be lost.
      
          Signed-off-by: Kyle Zhao <kylezhao@tencent.com>


 builtin/merge.c  |  6 ++++--
 t/t7600-merge.sh | 10 ++++++++++
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 6a6d3798858..80e3438be25 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -698,8 +698,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 
 	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
 					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
-					 NULL) < 0)
-		return error(_("Unable to write index."));
+					 NULL) < 0) {
+		error(_("Unable to write index."));
+		return 2;
+	}
 
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
 	    !strcmp(strategy, "ort")) {
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index e5ff073099a..f03709ea4be 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
 	verify_parents $c1 $c2
 '
 
+test_expect_success 'merge c1 with c2 when index.lock exists' '
+	test_when_finished rm .git/index.lock &&
+	git reset --hard c1 &&
+	touch .git/index.lock &&
+	test_must_fail git merge c2 &&
+	test_path_is_missing .git/MERGE_HEAD &&
+	test_path_is_missing .git/MERGE_MODE &&
+	test_path_is_missing .git/MERGE_MSG
+'
+
 test_expect_success 'merge --squash c3 with c7' '
 	git reset --hard c3 &&
 	test_must_fail git merge --squash c7 &&

base-commit: 19fe900cfce8096b7645ec9611a0b981f6bbd154
-- 
gitgitgadget

Re: [PATCH v2] merge: avoid write merge state when unable to write index

[Junio C Hamano]

"Kyle Zhao via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Kyle Zhao <kylezhao@tencent.com>
>
> Currently, when index.lock exists, if a merge occurs, the merge state
> will be written and the index will be unchanged.

"Currently, " is superfluous in this project, as by convention our
proposed log message begins with an observation of the current
status (and "what's wrong in it" follows) in the present tense.

My guess, from reading the tests, of the situation this patch
attempts to handle is something like this?

    When running a merge while the index is locked (presumably by
    another process), the merge state is written in SUCH and SUCH
    files, the index is not updated, and then the merge fails.  This
    leaves the resulting state inconsistent.

> If the user exec "git commit" instead of "git merge --abort" after this,
> a merge commit will be generated and all modifications from the source
> branch will be lost.

I do not think this is accurate description of the "an user action
can make it worse".  In reality, if the user runs "git commit", a
safety kicks in and they get:

    fatal: Unable to create '.../.git/index.lock': file exists.

In fact, your "How to reproduce" below the three-dash line removes
the stale index.lock file before running "git commit".

    From this state, if the index.lock file gets removed and the
    user runs "git commit", a merge commit is created, recording the
    tree from the inconsistent state the merge left.

may be a better description of the breakage.

But stepping back a bit, I do not think this extra paragraph is
needed at all.  If there were a competing process holding the
index.lock, it were in the process of updating the index, possibly
even to create a new commit.  If that process were indeed running
the "git commit" command, MERGE_HEAD and other state files we write
on our side will be taken into account by them and cause them to
record a merge, even though they may have been trying to record
something entirely different.  So regardless of what _this_ user,
whose merge failed due to a competing process that held the
index.lock file, does after the merge failure, the recorded history
can be hosed by the other process.

In any case, to prevent the other "git commit" process from using
"our" MERGE_HEAD and other state files to record a wrong contents,
the right fix is to make sure everybody who takes the lock on the
index file does *not* create any other state files to be read by
others before it successfully takes the lock.  That will also
prevent "git commit" we run after a failed merge (and removing the
lockfile) from recording an incorrect merge.

I do not offhand know if returning 2 (aka "I am not equipped to
handle this merg e at all") is a good way to do so, but if it is,
then the patch given here is absolutely the right thing to do.

>...
>     How to reproduce:
>     ...
>     git merge source-branch
>     rm .git/index.lock
>     git commit -m "4"

> diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
> index e5ff073099a..f03709ea4be 100755
> --- a/t/t7600-merge.sh
> +++ b/t/t7600-merge.sh
> @@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
>  	verify_parents $c1 $c2
>  '
>  
> +test_expect_success 'merge c1 with c2 when index.lock exists' '
> +	test_when_finished rm .git/index.lock &&
> +	git reset --hard c1 &&
> +	touch .git/index.lock &&

Do not use "touch" when the timestamp is not the thing we care
about.  In this case, you want that file to _exist_, and the right
way to do so is

	>.git/index.lock &&

which already appears in t7400 (it uses a no-op ":" command, but
that is not needed).

Other than that, the patch looks good to me.

Thanks.

[PATCH v3] merge: avoid write merge state when unable to write index

[Kyle Zhao via GitGitGadget]

From: Kyle Zhao <kylezhao@tencent.com>

When running a merge while the index is locked (presumably by another
process), the merge state is written, the index is not updated, and then
the merge fails. This can cause unexpected results.

i.g. if another running process is "git commit", MERGE_HEAD and other state
files we write on our side will be taken into account by them and cause them
to record a merge, even though they may have been trying to record something
entirely different.

Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
---
    merge: avoid write merge state when unable to write index
    
    In some of our monorepos, code is sometimes lost after merging.
    
    After investigation, we discovered the problem.
    
    This happens if we perform "git pull" or "git merge" when another git
    process is writing to the index, especially in a monorepo (because its
    index will be larger).
    
    How to reproduce:
    
    git init demo
    cd demo
    touch 1.txt && git add . && git commit -m "1"
    git checkout -b source-branch
    touch 2.txt && git add . && git commit -m "2"
    git checkout master
    echo "1" >> 1.txt && git add . && git commit -m "3"
    # another git process runnning
    touch .git/index.lock
    git merge source-branch
    # another git process finished
    rm .git/index.lock
    git commit -m "4"
    
    
    Then the modifications from the source branch are lost.
    
    Regards, Kyle

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1731%2Fkeyu98%2Fkz%2Ffix-merge-when-index-lock-exists-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1731/keyu98/kz/fix-merge-when-index-lock-exists-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1731

Range-diff vs v2:

 1:  86101e5b778 ! 1:  138fe0a054d merge: avoid write merge state when unable to write index
     @@ Metadata
       ## Commit message ##
          merge: avoid write merge state when unable to write index
      
     -    Currently, when index.lock exists, if a merge occurs, the merge state
     -    will be written and the index will be unchanged.
     +    When running a merge while the index is locked (presumably by another
     +    process), the merge state is written, the index is not updated, and then
     +    the merge fails. This can cause unexpected results.
      
     -    If the user exec "git commit" instead of "git merge --abort" after this,
     -    a merge commit will be generated and all modifications from the source
     -    branch will be lost.
     +    i.g. if another running process is "git commit", MERGE_HEAD and other state
     +    files we write on our side will be taken into account by them and cause them
     +    to record a merge, even though they may have been trying to record something
     +    entirely different.
      
          Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
      
       ## builtin/merge.c ##
      @@ builtin/merge.c: static int try_merge_strategy(const char *strategy, struct commit_list *common,
     - 
       	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
       					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
     --					 NULL) < 0)
     + 					 NULL) < 0)
      -		return error(_("Unable to write index."));
     -+					 NULL) < 0) {
     -+		error(_("Unable to write index."));
     -+		return 2;
     -+	}
     ++		die(_("Unable to write index."));
       
       	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
       	    !strcmp(strategy, "ort")) {
     @@ t/t7600-merge.sh: test_expect_success 'merge c1 with c2' '
      +test_expect_success 'merge c1 with c2 when index.lock exists' '
      +	test_when_finished rm .git/index.lock &&
      +	git reset --hard c1 &&
     -+	touch .git/index.lock &&
     ++	: >.git/index.lock &&
      +	test_must_fail git merge c2 &&
      +	test_path_is_missing .git/MERGE_HEAD &&
      +	test_path_is_missing .git/MERGE_MODE &&


 builtin/merge.c  |  2 +-
 t/t7600-merge.sh | 10 ++++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 6a6d3798858..12c1b048fe1 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -699,7 +699,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
 					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
 					 NULL) < 0)
-		return error(_("Unable to write index."));
+		die(_("Unable to write index."));
 
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
 	    !strcmp(strategy, "ort")) {
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index e5ff073099a..c1958095dcf 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
 	verify_parents $c1 $c2
 '
 
+test_expect_success 'merge c1 with c2 when index.lock exists' '
+	test_when_finished rm .git/index.lock &&
+	git reset --hard c1 &&
+	: >.git/index.lock &&
+	test_must_fail git merge c2 &&
+	test_path_is_missing .git/MERGE_HEAD &&
+	test_path_is_missing .git/MERGE_MODE &&
+	test_path_is_missing .git/MERGE_MSG
+'
+
 test_expect_success 'merge --squash c3 with c7' '
 	git reset --hard c3 &&
 	test_must_fail git merge --squash c7 &&

base-commit: 19fe900cfce8096b7645ec9611a0b981f6bbd154
-- 
gitgitgadget

[PATCH v4] merge: avoid write merge state when unable to write index

[Kyle Zhao via GitGitGadget]

From: Kyle Zhao <kylezhao@tencent.com>

When running a merge while the index is locked (presumably by another
process), the merge state is written, the index is not updated, and then
the merge fails. This might cause unexpected results.

i.g. if another running process is "git commit", MERGE_HEAD and other state
files we write on our side will be taken into account by them and cause them
to record a merge, even though they may have been trying to record something
entirely different.

Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
---
    merge: avoid write merge state when unable to write index
    
    In some of our monorepos, code is sometimes lost after merging.
    
    After investigation, we discovered the problem.
    
    This happens if we perform "git pull" or "git merge" when another git
    process is writing to the index, especially in a monorepo (because its
    index will be larger).
    
    How to reproduce:
    
    git init demo
    cd demo
    touch 1.txt && git add . && git commit -m "1"
    git checkout -b source-branch
    touch 2.txt && git add . && git commit -m "2"
    git checkout master
    echo "1" >> 1.txt && git add . && git commit -m "3"
    # another git process runnning
    touch .git/index.lock
    git merge source-branch
    # another git process finished
    rm .git/index.lock
    git commit -m "4"
    
    
    Then the modifications from the source branch are lost.
    
    Regards, Kyle

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1731%2Fkeyu98%2Fkz%2Ffix-merge-when-index-lock-exists-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1731/keyu98/kz/fix-merge-when-index-lock-exists-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1731

Range-diff vs v3:

 1:  138fe0a054d ! 1:  7055dfb82c7 merge: avoid write merge state when unable to write index
     @@ Commit message
      
          When running a merge while the index is locked (presumably by another
          process), the merge state is written, the index is not updated, and then
     -    the merge fails. This can cause unexpected results.
     +    the merge fails. This might cause unexpected results.
      
          i.g. if another running process is "git commit", MERGE_HEAD and other state
          files we write on our side will be taken into account by them and cause them
     @@ t/t7600-merge.sh: test_expect_success 'merge c1 with c2' '
      +test_expect_success 'merge c1 with c2 when index.lock exists' '
      +	test_when_finished rm .git/index.lock &&
      +	git reset --hard c1 &&
     -+	: >.git/index.lock &&
     ++	>.git/index.lock &&
      +	test_must_fail git merge c2 &&
      +	test_path_is_missing .git/MERGE_HEAD &&
      +	test_path_is_missing .git/MERGE_MODE &&


 builtin/merge.c  |  2 +-
 t/t7600-merge.sh | 10 ++++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 6a6d3798858..12c1b048fe1 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -699,7 +699,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
 					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
 					 NULL) < 0)
-		return error(_("Unable to write index."));
+		die(_("Unable to write index."));
 
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
 	    !strcmp(strategy, "ort")) {
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index e5ff073099a..ef54cff4faa 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
 	verify_parents $c1 $c2
 '
 
+test_expect_success 'merge c1 with c2 when index.lock exists' '
+	test_when_finished rm .git/index.lock &&
+	git reset --hard c1 &&
+	>.git/index.lock &&
+	test_must_fail git merge c2 &&
+	test_path_is_missing .git/MERGE_HEAD &&
+	test_path_is_missing .git/MERGE_MODE &&
+	test_path_is_missing .git/MERGE_MSG
+'
+
 test_expect_success 'merge --squash c3 with c7' '
 	git reset --hard c3 &&
 	test_must_fail git merge --squash c7 &&

base-commit: 19fe900cfce8096b7645ec9611a0b981f6bbd154
-- 
gitgitgadget

Re: [PATCH v4] merge: avoid write merge state when unable to write index

[goldofzky]

ping - please let me know if I need to do anything different. Thanks!

[PATCH v5] merge: avoid write merge state when unable to write index

[Kyle Zhao via GitGitGadget]

From: Kyle Zhao <kylezhao@tencent.com>

When running a merge while the index is locked (presumably by another
process), the merge state is written, the index is not updated, and then
the merge fails. This might cause unexpected results.

E.g., if another running process is "git commit", MERGE_HEAD and other
state files we write on our side will be taken into account by them and
cause them to record a merge, even though they may have been trying to
record something entirely different.

Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
---
    merge: avoid write merge state when unable to write index
    
    In some of our monorepos, code is sometimes lost after merging.
    
    After investigation, we discovered the problem.
    
    This happens if we perform "git pull" or "git merge" when another git
    process is writing to the index, especially in a monorepo (because its
    index will be larger).
    
    How to reproduce:
    
    git init demo
    cd demo
    touch 1.txt && git add . && git commit -m "1"
    git checkout -b source-branch
    touch 2.txt && git add . && git commit -m "2"
    git checkout master
    echo "1" >> 1.txt && git add . && git commit -m "3"
    # another git process runnning
    touch .git/index.lock
    git merge source-branch
    # another git process finished
    rm .git/index.lock
    git commit -m "4"
    
    
    Then the modifications from the source branch are lost.
    
    Regards, Kyle

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1731%2Fkeyu98%2Fkz%2Ffix-merge-when-index-lock-exists-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1731/keyu98/kz/fix-merge-when-index-lock-exists-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1731

Range-diff vs v4:

 1:  7055dfb82c7 ! 1:  a5156088514 merge: avoid write merge state when unable to write index
     @@ Commit message
          process), the merge state is written, the index is not updated, and then
          the merge fails. This might cause unexpected results.
      
     -    i.g. if another running process is "git commit", MERGE_HEAD and other state
     -    files we write on our side will be taken into account by them and cause them
     -    to record a merge, even though they may have been trying to record something
     -    entirely different.
     +    E.g., if another running process is "git commit", MERGE_HEAD and other
     +    state files we write on our side will be taken into account by them and
     +    cause them to record a merge, even though they may have been trying to
     +    record something entirely different.
      
          Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
      


 builtin/merge.c  |  2 +-
 t/t7600-merge.sh | 10 ++++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 6a6d3798858..12c1b048fe1 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -699,7 +699,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
 					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
 					 NULL) < 0)
-		return error(_("Unable to write index."));
+		die(_("Unable to write index."));
 
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
 	    !strcmp(strategy, "ort")) {
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index e5ff073099a..ef54cff4faa 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
 	verify_parents $c1 $c2
 '
 
+test_expect_success 'merge c1 with c2 when index.lock exists' '
+	test_when_finished rm .git/index.lock &&
+	git reset --hard c1 &&
+	>.git/index.lock &&
+	test_must_fail git merge c2 &&
+	test_path_is_missing .git/MERGE_HEAD &&
+	test_path_is_missing .git/MERGE_MODE &&
+	test_path_is_missing .git/MERGE_MSG
+'
+
 test_expect_success 'merge --squash c3 with c7' '
 	git reset --hard c3 &&
 	test_must_fail git merge --squash c7 &&

base-commit: 19fe900cfce8096b7645ec9611a0b981f6bbd154
-- 
gitgitgadget

Re: [PATCH v5] merge: avoid write merge state when unable to write index

[Junio C Hamano]

"Kyle Zhao via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Kyle Zhao <kylezhao@tencent.com>
>
> When running a merge while the index is locked (presumably by another
> process), the merge state is written, the index is not updated, and then
> the merge fails. This might cause unexpected results.

Failing the merge is good thing.

> E.g., if another running process is "git commit", MERGE_HEAD and other
> state files we write on our side will be taken into account by them and
> cause them to record a merge, even though they may have been trying to
> record something entirely different.

If I recall the previous discussion correctly, I think the primary
thing this change achieves is to get us closer to a state where
competing commands (a "git commit" run while we are doing something
else like "git merge") take the index.lock as the first thing (so
others are blocked), before making auxiliary files like MERGE_HEAD
that would affect the behaviour of whoever has index.lock (and thus
making a new commit).  And that is what we need to stress in the
proposed log message, I would think.

But this is probably only half-a-solution.

> Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
> ---
> diff --git a/builtin/merge.c b/builtin/merge.c
> index 6a6d3798858..12c1b048fe1 100644
> --- a/builtin/merge.c
> +++ b/builtin/merge.c
> @@ -699,7 +699,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
>  	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
>  					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
>  					 NULL) < 0)
> -		return error(_("Unable to write index."));
> +		die(_("Unable to write index."));
>  
>  	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
>  	    !strcmp(strategy, "ort")) {

If we fail to write the index here, even if we have other strategies
to try after the current one fails, it probably is a good idea to die
and stop the other ones from being tried, not because their attempt
to write the index might fail the same way, but because it is likely
that we are really in a weird situation and the user would want to
inspect the situation before this process makes too much damage to
the working tree and the index.

But this is probably only half-a-solution.  Because we release the
index.lock when the refresh-and-write call returns, and the
index.lock is free for the other process to grab, do whatever they
want to do to the index and the working tree (including making a new
commit out of it and update the HEAD), before or after we write out
the merge state files. So in that sense, this patch is *not* solving
the "E.g., if another running process is ..."  problem at all.

So ...

> diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
> index e5ff073099a..ef54cff4faa 100755
> --- a/t/t7600-merge.sh
> +++ b/t/t7600-merge.sh
> @@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
>  	verify_parents $c1 $c2
>  '
>  
> +test_expect_success 'merge c1 with c2 when index.lock exists' '
> +	test_when_finished rm .git/index.lock &&
> +	git reset --hard c1 &&
> +	>.git/index.lock &&
> +	test_must_fail git merge c2 &&
> +	test_path_is_missing .git/MERGE_HEAD &&
> +	test_path_is_missing .git/MERGE_MODE &&
> +	test_path_is_missing .git/MERGE_MSG

... I do not quite see the point of this exercise.  It is good to
make sure that "git merge c2" fails while it is clear that somebody
else is mucking with the same repository touching the index.  But it
does not help the other process all that much if we stop only when
they happen to be holding lock at the point we try to refresh the
index.  It is making the race window smaller by a tiny bit.

So, I am not sure if this is worth doing.

Re: [Internet]Re: [PATCH v5] merge: avoid write merge state when unable to write index

[goldofzky]

On Fri, Jun 14, 2024 at 1:27 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Kyle Zhao via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Kyle Zhao <kylezhao@tencent.com>
> >
> > When running a merge while the index is locked (presumably by another
> > process), the merge state is written, the index is not updated, and then
> > the merge fails. This might cause unexpected results.
>
> Failing the merge is good thing.
>
> > E.g., if another running process is "git commit", MERGE_HEAD and other
> > state files we write on our side will be taken into account by them and
> > cause them to record a merge, even though they may have been trying to
> > record something entirely different.
>
> If I recall the previous discussion correctly, I think the primary
> thing this change achieves is to get us closer to a state where
> competing commands (a "git commit" run while we are doing something
> else like "git merge") take the index.lock as the first thing (so
> others are blocked), before making auxiliary files like MERGE_HEAD
> that would affect the behaviour of whoever has index.lock (and thus
> making a new commit).  And that is what we need to stress in the
> proposed log message, I would think.

I agree.

>
> But this is probably only half-a-solution.


>
> > Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
> > ---
> > diff --git a/builtin/merge.c b/builtin/merge.c
> > index 6a6d3798858..12c1b048fe1 100644
> > --- a/builtin/merge.c
> > +++ b/builtin/merge.c
> > @@ -699,7 +699,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
> >       if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
> >                                        SKIP_IF_UNCHANGED, 0, NULL, NULL,
> >                                        NULL) < 0)
> > -             return error(_("Unable to write index."));
> > +             die(_("Unable to write index."));
> >
> >       if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
> >           !strcmp(strategy, "ort")) {
>
> If we fail to write the index here, even if we have other strategies
> to try after the current one fails, it probably is a good idea to die
> and stop the other ones from being tried, not because their attempt
> to write the index might fail the same way, but because it is likely
> that we are really in a weird situation and the user would want to
> inspect the situation before this process makes too much damage to
> the working tree and the index.
>
> But this is probably only half-a-solution.  Because we release the
> index.lock when the refresh-and-write call returns, and the
> index.lock is free for the other process to grab, do whatever they
> want to do to the index and the working tree (including making a new
> commit out of it and update the HEAD), before or after we write out
> the merge state files. So in that sense, this patch is *not* solving
> the "E.g., if another running process is ..."  problem at all.
>
> So ...
>

Oops!
Thank you for the reminder, I indeed did not consider this point. 
Even if the index is written successfully, another running "git commit" may 
record a merge (if it generates the commit before merge state is removed).

> > diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
> > index e5ff073099a..ef54cff4faa 100755
> > --- a/t/t7600-merge.sh
> > +++ b/t/t7600-merge.sh
> > @@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
> >       verify_parents $c1 $c2
> >  '
> >
> > +test_expect_success 'merge c1 with c2 when index.lock exists' '
> > +     test_when_finished rm .git/index.lock &&
> > +     git reset --hard c1 &&
> > +     >.git/index.lock &&
> > +     test_must_fail git merge c2 &&
> > +     test_path_is_missing .git/MERGE_HEAD &&
> > +     test_path_is_missing .git/MERGE_MODE &&
> > +     test_path_is_missing .git/MERGE_MSG
>
> ... I do not quite see the point of this exercise.  It is good to
> make sure that "git merge c2" fails while it is clear that somebody
> else is mucking with the same repository touching the index.  But it
> does not help the other process all that much if we stop only when
> they happen to be holding lock at the point we try to refresh the
> index.  It is making the race window smaller by a tiny bit.

This test is only used to verify whether the merge state is generated after a merge fails 
due to the index writing. 
To my knowledge, for a merge, writing the merge state and the index is not an atomic operation. 
Simply modifying the logic of merge is useless and we need to think about other solutions.

>
> So, I am not sure if this is worth doing.
>

[1] As explained in the commit message of v1, we are particularly concerned about the
issue of source code loss. Once this problem occurs, the consequences will be immeasurable 
and detecting these abnormal merge points will be very difficult.

[2] From a usability perspective, the merge state should not be written when the index is being
 written (merge conflicts are not considered failures). To avoid losing changes in the source branch, 
users can only execute 'git merge --abort' and try 'git merge' again. However, if the merge state is
not written in the first place, the user only needs to retry 'git merge'.

In other words, writing the merge state after the index write fails is meaningless and could 
potentially cause Git to lose changes.

Re: Re: [Internet]Re: [PATCH v5] merge: avoid write merge state when unable to write index

[goldofzky]

> [2] From a usability perspective, the merge state should not be written when the index is being
> written (merge conflicts are not considered failures). To avoid losing changes in the source branch,

"the merge state should not be written when the index is being written" should be:
the merge state should not be written if the index write fails.



> users can only execute 'git merge --abort' and try 'git merge' again. However, if the merge state is
> not written in the first place, the user only needs to retry 'git merge'.


>In other words, writing the merge state after the index write fails is meaningless and could
>potentially cause Git to lose changes.

[PATCH v6] merge: avoid write merge state when unable to write index

[Kyle Zhao via GitGitGadget]

From: Kyle Zhao <kylezhao@tencent.com>

Writing the merge state after the index write fails is meaningless and
could potentially cause Git to lose changes.

Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
---
    merge: avoid write merge state when unable to write index
    
    In some of our monorepos, code is sometimes lost after merging.
    
    After investigation, we discovered the problem.
    
    This happens if we perform "git pull" or "git merge" when another git
    process is writing to the index, especially in a monorepo (because its
    index will be larger).
    
    How to reproduce:
    
    git init demo
    cd demo
    touch 1.txt && git add . && git commit -m "1"
    git checkout -b source-branch
    touch 2.txt && git add . && git commit -m "2"
    git checkout master
    echo "1" >> 1.txt && git add . && git commit -m "3"
    # another git process runnning
    touch .git/index.lock
    git merge source-branch
    # another git process finished
    rm .git/index.lock
    git commit -m "4"
    
    
    Then the modifications from the source branch are lost.
    
    Regards, Kyle

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1731%2Fkeyu98%2Fkz%2Ffix-merge-when-index-lock-exists-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1731/keyu98/kz/fix-merge-when-index-lock-exists-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/1731

Range-diff vs v5:

 1:  a5156088514 ! 1:  24150cd814a merge: avoid write merge state when unable to write index
     @@ Metadata
       ## Commit message ##
          merge: avoid write merge state when unable to write index
      
     -    When running a merge while the index is locked (presumably by another
     -    process), the merge state is written, the index is not updated, and then
     -    the merge fails. This might cause unexpected results.
     -
     -    E.g., if another running process is "git commit", MERGE_HEAD and other
     -    state files we write on our side will be taken into account by them and
     -    cause them to record a merge, even though they may have been trying to
     -    record something entirely different.
     +    Writing the merge state after the index write fails is meaningless and
     +    could potentially cause Git to lose changes.
      
          Signed-off-by: Kyle Zhao <kylezhao@tencent.com>
      


 builtin/merge.c  |  2 +-
 t/t7600-merge.sh | 10 ++++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index daed2d4e1e2..03c405fa5df 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -701,7 +701,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 	if (repo_refresh_and_write_index(the_repository, REFRESH_QUIET,
 					 SKIP_IF_UNCHANGED, 0, NULL, NULL,
 					 NULL) < 0)
-		return error(_("Unable to write index."));
+		die(_("Unable to write index."));
 
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree") ||
 	    !strcmp(strategy, "ort")) {
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index e5ff073099a..ef54cff4faa 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -236,6 +236,16 @@ test_expect_success 'merge c1 with c2' '
 	verify_parents $c1 $c2
 '
 
+test_expect_success 'merge c1 with c2 when index.lock exists' '
+	test_when_finished rm .git/index.lock &&
+	git reset --hard c1 &&
+	>.git/index.lock &&
+	test_must_fail git merge c2 &&
+	test_path_is_missing .git/MERGE_HEAD &&
+	test_path_is_missing .git/MERGE_MODE &&
+	test_path_is_missing .git/MERGE_MSG
+'
+
 test_expect_success 'merge --squash c3 with c7' '
 	git reset --hard c3 &&
 	test_must_fail git merge --squash c7 &&

base-commit: d63586cb314731c851f28e14fc8012988467e2da
-- 
gitgitgadget