v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jonathan del Strother]

Heya,

git-stash in v1.7.0-rc0 gives me several hundred lines of :

error: unable to find dd4a345a770f5c82a668932b7289f42b1ae38103
error: unable to find dd4a345a770f5c82a668932b7289f42b1ae38103
error: unable to find dd4a345a770f5c82a668932b7289f42b1ae38103
....
error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0
error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0
error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0
error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0
....
error: unable to find 980f084e71097df3d4086aaab574972321df80f2
error: unable to find 980f084e71097df3d4086aaab574972321df80f2
error: unable to find 980f084e71097df3d4086aaab574972321df80f2
error: unable to find 980f084e71097df3d4086aaab574972321df80f2


Other than that, it appears to work fine.  'git show <sha1>' works too
- they're all a bunch of file contents, with no particular pattern.

This only happens in my main (private) repo.  I can't seem to come up
with a reproducible example for anyone else...

The previous build I was using -
5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show
this problem.  Want me to try and bisect further?


-Jonathan

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jonathan del Strother]

On 29 January 2010 17:44, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan del Strother <maillist@steelskies.com> writes:
>
>> Heya,
>>
>> git-stash in v1.7.0-rc0 gives me several hundred lines of :
>> The previous build I was using -
>> 5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show
>> this problem.  Want me to try and bisect further?
>
> Yes, please.
>

ee6fc514f2df821c2719cc49499a56ef2fb136b0 (Show submodules as modified
when they contain a dirty work tree) seems to be the first bad commit.

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jens Lehmann]

Am 30.01.2010 01:46, schrieb Jonathan del Strother:
> On 29 January 2010 17:44, Junio C Hamano <gitster@pobox.com> wrote:
>> Jonathan del Strother <maillist@steelskies.com> writes:
>>
>>> Heya,
>>>
>>> git-stash in v1.7.0-rc0 gives me several hundred lines of :
>>> The previous build I was using -
>>> 5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show
>>> this problem.  Want me to try and bisect further?
>>
>> Yes, please.
>>
> 
> ee6fc514f2df821c2719cc49499a56ef2fb136b0 (Show submodules as modified
> when they contain a dirty work tree) seems to be the first bad commit.

Thanks for pinning that down.

I assume you have one or more submodules, maybe even with untracked
or yet uncommitted modified files in your tree? If so, what does git
status say in the superproject and in the submodule(s)?

(If you want to keep some information off the list, you can mail me
privately)

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jonathan del Strother]

On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> Am 30.01.2010 01:46, schrieb Jonathan del Strother:
>> On 29 January 2010 17:44, Junio C Hamano <gitster@pobox.com> wrote:
>>> Jonathan del Strother <maillist@steelskies.com> writes:
>>>
>>>> Heya,
>>>>
>>>> git-stash in v1.7.0-rc0 gives me several hundred lines of :
>>>> The previous build I was using -
>>>> 5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show
>>>> this problem.  Want me to try and bisect further?
>>>
>>> Yes, please.
>>>
>>
>> ee6fc514f2df821c2719cc49499a56ef2fb136b0 (Show submodules as modified
>> when they contain a dirty work tree) seems to be the first bad commit.
>
> Thanks for pinning that down.
>
> I assume you have one or more submodules, maybe even with untracked
> or yet uncommitted modified files in your tree? If so, what does git
> status say in the superproject and in the submodule(s)?
>

Yep, I have 10 submodules.  However, they're all completely clean with
no tracked or untracked changes shown in git status.  Anything else I
can investigate?

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jens Lehmann]

Am 30.01.2010 14:23, schrieb Jonathan del Strother:
> On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>> I assume you have one or more submodules, maybe even with untracked
>> or yet uncommitted modified files in your tree? If so, what does git
>> status say in the superproject and in the submodule(s)?
> 
> Yep, I have 10 submodules.  However, they're all completely clean with
> no tracked or untracked changes shown in git status.  Anything else I
> can investigate?

The change in behavior my patch introduced is that "git status" is
called inside each submodule. So i would expect getting the same
errors when using this command:
   git submodule foreach git status -s

It should just show
   Entering '<submodule 1>'
   Entering '<submodule 2>'
   Entering '<submodule 3>'
   Entering '<submodule 4>'
   Entering '<submodule 5>'
   Entering '<submodule 6>'
   Entering '<submodule 7>'
   Entering '<submodule 8>'
   Entering '<submodule 9>'
   Entering '<submodule 10>'
when the submodules are not dirty. What do you get?


The error message "error: unable to find <sha1>" is only generated in
sha1_loose_object_info() when map_sha1_file() fails. Its single caller
being sha1_object_info(), which is called by many git commands. Does
that ring a bell for somebody?

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jonathan del Strother]

On 30 January 2010 14:10, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> Am 30.01.2010 14:23, schrieb Jonathan del Strother:
>> On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>>> I assume you have one or more submodules, maybe even with untracked
>>> or yet uncommitted modified files in your tree? If so, what does git
>>> status say in the superproject and in the submodule(s)?
>>
>> Yep, I have 10 submodules.  However, they're all completely clean with
>> no tracked or untracked changes shown in git status.  Anything else I
>> can investigate?
>
> The change in behavior my patch introduced is that "git status" is
> called inside each submodule. So i would expect getting the same
> errors when using this command:
>   git submodule foreach git status -s
>
> It should just show
>   Entering '<submodule 1>'
>   Entering '<submodule 2>'
>   Entering '<submodule 3>'
>   Entering '<submodule 4>'
>   Entering '<submodule 5>'
>   Entering '<submodule 6>'
>   Entering '<submodule 7>'
>   Entering '<submodule 8>'
>   Entering '<submodule 9>'
>   Entering '<submodule 10>'
> when the submodules are not dirty. What do you get?
>

Correct - I just get that output.

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Junio C Hamano]

Please try this.

The thing is that "add -u" internally runs "diff-files" equivalent, but
the code to check the submodule freshness doesn't reset the GIT_INDEX_FILE
exported from the caller and tries to use that even when checking the
submodule repository.

diff --git a/submodule.c b/submodule.c
index ca0527f..8bd0a30 100644
--- a/submodule.c
+++ b/submodule.c
@@ -126,7 +126,7 @@ int is_submodule_modified(const char *path)
 		"--porcelain",
 		NULL,
 	};
-	char *env[3];
+	char *env[4];
 	struct strbuf buf = STRBUF_INIT;
 
 	strbuf_addf(&buf, "%s/.git/", path);
@@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
 	env[0] = strbuf_detach(&buf, NULL);
 	strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
 	env[1] = strbuf_detach(&buf, NULL);
-	env[2] = NULL;
+	strbuf_addf(&buf, "GIT_INDEX_FILE=");
+	env[2] = strbuf_detach(&buf, NULL);
+	env[3] = NULL;
 
 	memset(&cp, 0, sizeof(cp));
 	cp.argv = argv;
@@ -161,6 +163,7 @@ int is_submodule_modified(const char *path)
 
 	free(env[0]);
 	free(env[1]);
+	free(env[2]);
 	strbuf_release(&buf);
 	return len != 0;
 }

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Please try this.

oops, but with this, too.

> diff --git a/submodule.c b/submodule.c
> index ca0527f..8bd0a30 100644
> --- a/submodule.c
> +++ b/submodule.c
> ...
> @@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
>  	env[0] = strbuf_detach(&buf, NULL);
>  	strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
>  	env[1] = strbuf_detach(&buf, NULL);
> -	env[2] = NULL;
> +	strbuf_addf(&buf, "GIT_INDEX_FILE=");

This should be:

	strbuf_addf(&buf, "GIT_INDEX_FILE");

> +	env[2] = strbuf_detach(&buf, NULL);
> +	env[3] = NULL;
>  
>  	memset(&cp, 0, sizeof(cp));
>  	cp.argv = argv;

[PATCH] is_submodule_modified(): fix breakage with external GIT_INDEX_FILE

[Junio C Hamano]

Even when the environment was given for the top-level process, checking
in the submodule work tree should use the index file associated with the
work tree of the submodule.  Do not export it to the environment.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * This seems to pass all the tests, so I'm inclined to push it out as
   a part of 1.7.0-rc1.  There might be some other enviroment variables
   we shouldn't be leaking out to the submodule environment, but that can
   be spotted and fixed in later patches before the final.

 submodule.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/submodule.c b/submodule.c
index ca0527f..6f7c210 100644
--- a/submodule.c
+++ b/submodule.c
@@ -126,7 +126,7 @@ int is_submodule_modified(const char *path)
 		"--porcelain",
 		NULL,
 	};
-	char *env[3];
+	char *env[4];
 	struct strbuf buf = STRBUF_INIT;
 
 	strbuf_addf(&buf, "%s/.git/", path);
@@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
 	env[0] = strbuf_detach(&buf, NULL);
 	strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
 	env[1] = strbuf_detach(&buf, NULL);
-	env[2] = NULL;
+	strbuf_addf(&buf, "GIT_INDEX_FILE");
+	env[2] = strbuf_detach(&buf, NULL);
+	env[3] = NULL;
 
 	memset(&cp, 0, sizeof(cp));
 	cp.argv = argv;
@@ -161,6 +163,7 @@ int is_submodule_modified(const char *path)
 
 	free(env[0]);
 	free(env[1]);
+	free(env[2]);
 	strbuf_release(&buf);
 	return len != 0;
 }
-- 
1.7.0.rc1.141.gbc8ce

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jonathan del Strother]

On 30 January 2010 20:25, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Please try this.
>
> oops, but with this, too.
>
>> diff --git a/submodule.c b/submodule.c
>> index ca0527f..8bd0a30 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>> ...
>> @@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
>>       env[0] = strbuf_detach(&buf, NULL);
>>       strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
>>       env[1] = strbuf_detach(&buf, NULL);
>> -     env[2] = NULL;
>> +     strbuf_addf(&buf, "GIT_INDEX_FILE=");
>
> This should be:
>
>        strbuf_addf(&buf, "GIT_INDEX_FILE");
>
>> +     env[2] = strbuf_detach(&buf, NULL);
>> +     env[3] = NULL;
>>
>>       memset(&cp, 0, sizeof(cp));
>>       cp.argv = argv;
>

Sorry, I'm not sure I follow your amendment - it looks exactly like
the patch you originally supplied?

With that original patch applied, I no longer get the 'unable to find'
errors on stashing.  However, git-status shows all my submodules as
being modified, but there appear to be no local changes :

[jon@gir:Developer/AudioBooWeb]$ git status
# On branch giterror
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   shared/vendor/plugins/acts_as_list
#	modified:   shared/vendor/plugins/cucumber
#	modified:   shared/vendor/plugins/delayed_job
#	modified:   shared/vendor/plugins/haml
#	modified:   shared/vendor/plugins/hoptoad_notifier
#	modified:   shared/vendor/plugins/machinist
#	modified:   shared/vendor/plugins/newrelic_rpm
#	modified:   shared/vendor/plugins/rspec
#	modified:   shared/vendor/plugins/rspec-rails
#	modified:   shared/vendor/rails
#
no changes added to commit (use "git add" and/or "git commit -a")

[jon@gir:Developer/AudioBooWeb]$ git diff
diff --git a/shared/vendor/plugins/acts_as_list
b/shared/vendor/plugins/acts_as_list
diff --git a/shared/vendor/plugins/cucumber b/shared/vendor/plugins/cucumber
diff --git a/shared/vendor/plugins/delayed_job
b/shared/vendor/plugins/delayed_job
diff --git a/shared/vendor/plugins/haml b/shared/vendor/plugins/haml
diff --git a/shared/vendor/plugins/hoptoad_notifier
b/shared/vendor/plugins/hoptoad_notifier
diff --git a/shared/vendor/plugins/machinist b/shared/vendor/plugins/machinist
diff --git a/shared/vendor/plugins/newrelic_rpm
b/shared/vendor/plugins/newrelic_rpm
diff --git a/shared/vendor/plugins/rspec b/shared/vendor/plugins/rspec
diff --git a/shared/vendor/plugins/rspec-rails
b/shared/vendor/plugins/rspec-rails
diff --git a/shared/vendor/rails b/shared/vendor/rails

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jonathan del Strother]

On 31 January 2010 19:16, Jonathan del Strother <maillist@steelskies.com> wrote:
> On 30 January 2010 20:25, Junio C Hamano <gitster@pobox.com> wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> Please try this.
>>
>> oops, but with this, too.
>>
>>> diff --git a/submodule.c b/submodule.c
>>> index ca0527f..8bd0a30 100644
>>> --- a/submodule.c
>>> +++ b/submodule.c
>>> ...
>>> @@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
>>>       env[0] = strbuf_detach(&buf, NULL);
>>>       strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
>>>       env[1] = strbuf_detach(&buf, NULL);
>>> -     env[2] = NULL;
>>> +     strbuf_addf(&buf, "GIT_INDEX_FILE=");
>>
>> This should be:
>>
>>        strbuf_addf(&buf, "GIT_INDEX_FILE");
>>
>>> +     env[2] = strbuf_detach(&buf, NULL);
>>> +     env[3] = NULL;
>>>
>>>       memset(&cp, 0, sizeof(cp));
>>>       cp.argv = argv;
>>
>
> Sorry, I'm not sure I follow your amendment - it looks exactly like
> the patch you originally supplied?
>
> With that original patch applied, I no longer get the 'unable to find'
> errors on stashing.  However, git-status shows all my submodules as
> being modified, but there appear to be no local changes :
>
> [jon@gir:Developer/AudioBooWeb]$ git status
> # On branch giterror
> # Changed but not updated:
> #   (use "git add <file>..." to update what will be committed)
> #   (use "git checkout -- <file>..." to discard changes in working directory)
> #
> #       modified:   shared/vendor/plugins/acts_as_list
> #       modified:   shared/vendor/plugins/cucumber
> #       modified:   shared/vendor/plugins/delayed_job
> #       modified:   shared/vendor/plugins/haml
> #       modified:   shared/vendor/plugins/hoptoad_notifier
> #       modified:   shared/vendor/plugins/machinist
> #       modified:   shared/vendor/plugins/newrelic_rpm
> #       modified:   shared/vendor/plugins/rspec
> #       modified:   shared/vendor/plugins/rspec-rails
> #       modified:   shared/vendor/rails
> #
> no changes added to commit (use "git add" and/or "git commit -a")
>
> [jon@gir:Developer/AudioBooWeb]$ git diff
> diff --git a/shared/vendor/plugins/acts_as_list
> b/shared/vendor/plugins/acts_as_list
> diff --git a/shared/vendor/plugins/cucumber b/shared/vendor/plugins/cucumber
> diff --git a/shared/vendor/plugins/delayed_job
> b/shared/vendor/plugins/delayed_job
> diff --git a/shared/vendor/plugins/haml b/shared/vendor/plugins/haml
> diff --git a/shared/vendor/plugins/hoptoad_notifier
> b/shared/vendor/plugins/hoptoad_notifier
> diff --git a/shared/vendor/plugins/machinist b/shared/vendor/plugins/machinist
> diff --git a/shared/vendor/plugins/newrelic_rpm
> b/shared/vendor/plugins/newrelic_rpm
> diff --git a/shared/vendor/plugins/rspec b/shared/vendor/plugins/rspec
> diff --git a/shared/vendor/plugins/rspec-rails
> b/shared/vendor/plugins/rspec-rails
> diff --git a/shared/vendor/rails b/shared/vendor/rails
>

...  however, the latest version of master
(b0883aa6c77111e88496bd0afe073caf68ab9f99) gives me no errors, and
(correctly) doesn't show my submodules as being modified.  I still
don't quite follow exactly what was wrong, but thanks :)

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Junio C Hamano]

Jonathan del Strother <maillist@steelskies.com> writes:

>>> +     strbuf_addf(&buf, "GIT_INDEX_FILE=");
>>
>> This should be:
>>
>>        strbuf_addf(&buf, "GIT_INDEX_FILE");
>
> Sorry, I'm not sure I follow your amendment - it looks exactly like
> the patch you originally supplied?

Lack of the last "=" tells: "Instead of exporting GIT_INDEX_FILE with an
empty string as its value, unexport it."

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

[Jens Lehmann]

Am 30.01.2010 17:36, schrieb Jonathan del Strother:
> On 30 January 2010 14:10, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>> Am 30.01.2010 14:23, schrieb Jonathan del Strother:
>>> On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>>>> I assume you have one or more submodules, maybe even with untracked
>>>> or yet uncommitted modified files in your tree? If so, what does git
>>>> status say in the superproject and in the submodule(s)?
>>>
>>> Yep, I have 10 submodules.  However, they're all completely clean with
>>> no tracked or untracked changes shown in git status.  Anything else I
>>> can investigate?
>>
>> The change in behavior my patch introduced is that "git status" is
>> called inside each submodule. So i would expect getting the same
>> errors when using this command:
>>   git submodule foreach git status -s
>>
>> It should just show
>>   Entering '<submodule 1>'
>>   Entering '<submodule 2>'
>>   Entering '<submodule 3>'
>>   Entering '<submodule 4>'
>>   Entering '<submodule 5>'
>>   Entering '<submodule 6>'
>>   Entering '<submodule 7>'
>>   Entering '<submodule 8>'
>>   Entering '<submodule 9>'
>>   Entering '<submodule 10>'
>> when the submodules are not dirty. What do you get?
>>
> 
> Correct - I just get that output.

Hm, so nothing unusual there. I really wonder what is the problem here,
as calling "git status" inside the submodules works fine when issued via
"git submodule foreach", but not when done via run_command()!?

So i would like to ask some more questions:

- Under what operating system and on what filesystem is this happening?

- Is there anything unusual about your repo (e.g. using GIT_WORK_TREE
  or having the object database somewhere else that in .git in the
  superproject or any of the submodules)?

- You are just issuing a "git stash" to stash some changes in the
  superproject when that happens, right?

- The hashes that show up as "unable to find" are reachable via "git
  show" in the superproject, not in the submodules, right?

- Do these hashes have any relation to the contents you are stashing?

- The following patch should suppress (but not solve) this problem when
  applied to a version of git that contains
  4d34477f4c5dbebc55aa1362fd705440590a85f1 (git diff: Don't test
  submodule dirtiness with --ignore-submodules), e.g. current next.
  Could you please verify that?

---8<---
diff --git a/git-stash.sh b/git-stash.sh
index 3a0685f..e9b47b4 100755
--- a/git-stash.sh
+++ b/git-stash.sh
@@ -105,7 +105,7 @@ create_stash () {
                w_tree=$(GIT_INDEX_FILE="$TMP-index" git write-tree) ||
                die "Cannot save the current worktree state"

-               git diff-tree -p HEAD $w_tree > "$TMP-patch" &&
+               git diff-tree --ignore-submodules -p HEAD $w_tree > "$TMP-patch"
                test -s "$TMP-patch" ||
                die "No changes selected"