Re: Recover broken git index?

[Linus Torvalds <torvalds@linux-foundation.org>]

On Thu, 28 May 2009, Misha Koshelev wrote:
>
> I tried you checkout master && git reset --hard and it worked.

It probably "worked" for some limited meaning of "worked". 

It sounds like you have major corruption in your object database, and it 
would be interesting to know how it happened. 

I would suggest:

 - back up your whole source directory - both .git _and_ your checked out 
   tree. That's partly because I'd want you to do some destructive 
   testing, but partly because it might be interesting to see the 
   actual breakage. 

   Is this a public project where you could expose the whole thing for us 
   to look at?

 - AFTER copying the whole tree, blow away your checked-out tree entirely 
   (or just copy your ".git" directory somewhere else), and then do

	rm .git/index
	git reset --hard

Now, I realize you already did a "git reset --hard", but since you had a 
working tree _and_ you had a pre-existing index, that reset will have done 
various shortcuts, rather than actually use the git database fully.

For example, if the 'stat' information in the index matches the 
checked-out copy, then git reset --hard will happily just ignore it. Also, 
even if the stat info doesn't match, rather than writing a new file, it 
will look at the old file and see if the contents match the SHA1, and then 
just update the index. 

So there's just a ton of those kinds of optimizations where git tries to 
avoid causing unnecessary IO, and with a pre-existing index file and a 
checked-out tree, it's entirely possible that git will not have even 
bothered to check the objects themselves.

Now, it's entirely possible that you can re-create a lot of the objects by 
re-committing the last state (if the missing objects are all "recent" 
ones). The simplest way is likely to just do something like

	git add .
	git write-tree

in a version of the tree that has all files checked out. If the objects 
are literally just missing (rather than corrupted), then this will 
re-populate the git object database with the all the objects in the 
current tree (but not in any previous trees!).

Again, if you can make the whole tree available and there is no secret 
data there, I'd be interested in taking a look. No guarantees that I can 
fix anything, but the corruption patterns are always interesting, and so 
I'd like to see what I can do, if anything.

> I guess the problem is when I try to make a commit:
>  git commit -a -n
> error: invalid object 5e35b75e062c7688636a19334bd85e9e8a408b76
> fatal: Error building trees

This very much implies that there are missing or corrupt objects in your 
database. The fact that you had tons of fsck errors supports that too. 
It's probably a pack-file that is corrupt.

> I first noticed this after my computer was acting sluggish (I was
> running processor intensive tasks) and I restarted it.

Restarted without a clean shutdown? What OS, and what kind of filesystem? 
It very much sounds like some files in .git/objects were not fully written 
to disk.



			Linus