Repo corrupted somehow?

Repo corrupted somehow?

[Andrew Arnott]

I was just git commit'ing, and then I was doing a git rebase to squash
several commits into one when the rebase failed.  I then did a
git checkout -f master
git reset --hard
but no matter what I do, git thinks that several files have changed.
The diff shows all the lines in these several files removed and then
added, yet without any changes made to them.  git reset --hard doesn't
revert the change. When I jump around history with git checkout these
files remain in their "changed" state.  I even tried "git clone" to
create a whole new repo, but one of these several files STILL
registered as changed before I made any changes.

Any idea what's wrong and how to recover?

Observe the below command buffer: (I can upload my repo so you can
clone it and perhaps repro it if you want).

Andrew@LACKY /c/git/dotnetoauth
$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#       modified:   tools/Documentation.targets
#       modified:   tools/DotNetOpenAuth.Common.Settings.targets
#       modified:   tools/DotNetOpenAuth.Versioning.targets
#       modified:
tools/Sandcastle/Presentation/vs2005/Content/reference_content.xml
#       modified:   tools/libcheck.ps1
#       modified:   tools/sandcastle.targets
#
no changes added to commit (use "git add" and/or "git commit -a")

Andrew@LACKY /c/git/dotnetoauth
$ git reset --hard
HEAD is now at 13d37b8 Patching up the bad merges in the phases.

Andrew@LACKY /c/git/dotnetoauth
$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#       modified:   tools/Documentation.targets
#       modified:   tools/DotNetOpenAuth.Common.Settings.targets
#       modified:   tools/DotNetOpenAuth.Versioning.targets
#       modified:
tools/Sandcastle/Presentation/vs2005/Content/reference_content.xml
#       modified:   tools/libcheck.ps1
#       modified:   tools/sandcastle.targets
#
no changes added to commit (use "git add" and/or "git commit -a")

Re: Repo corrupted somehow?

[Andrew Arnott]

I guess I was still in the middle of a rebase.  git rebase --abort
sort of ultimately fixed it.

On Mon, Nov 3, 2008 at 11:09 PM, Andrew Arnott <andrewarnott@gmail.com> wrote:
> I was just git commit'ing, and then I was doing a git rebase to squash
> several commits into one when the rebase failed.  I then did a
> git checkout -f master
> git reset --hard
> but no matter what I do, git thinks that several files have changed.
> The diff shows all the lines in these several files removed and then
> added, yet without any changes made to them.  git reset --hard doesn't
> revert the change. When I jump around history with git checkout these
> files remain in their "changed" state.  I even tried "git clone" to
> create a whole new repo, but one of these several files STILL
> registered as changed before I made any changes.
>
> Any idea what's wrong and how to recover?
>
> Observe the below command buffer: (I can upload my repo so you can
> clone it and perhaps repro it if you want).
>
> Andrew@LACKY /c/git/dotnetoauth
> $ git status
> # On branch master
> # Changed but not updated:
> #   (use "git add <file>..." to update what will be committed)
> #
> #       modified:   tools/Documentation.targets
> #       modified:   tools/DotNetOpenAuth.Common.Settings.targets
> #       modified:   tools/DotNetOpenAuth.Versioning.targets
> #       modified:
> tools/Sandcastle/Presentation/vs2005/Content/reference_content.xml
> #       modified:   tools/libcheck.ps1
> #       modified:   tools/sandcastle.targets
> #
> no changes added to commit (use "git add" and/or "git commit -a")
>
> Andrew@LACKY /c/git/dotnetoauth
> $ git reset --hard
> HEAD is now at 13d37b8 Patching up the bad merges in the phases.
>
> Andrew@LACKY /c/git/dotnetoauth
> $ git status
> # On branch master
> # Changed but not updated:
> #   (use "git add <file>..." to update what will be committed)
> #
> #       modified:   tools/Documentation.targets
> #       modified:   tools/DotNetOpenAuth.Common.Settings.targets
> #       modified:   tools/DotNetOpenAuth.Versioning.targets
> #       modified:
> tools/Sandcastle/Presentation/vs2005/Content/reference_content.xml
> #       modified:   tools/libcheck.ps1
> #       modified:   tools/sandcastle.targets
> #
> no changes added to commit (use "git add" and/or "git commit -a")
>

Re: Repo corrupted somehow?

[Andrew Arnott]

Nah, that wasn't a false alarm after all.  It's happening again, only
this time for dozens of files, and
git rebase --abort
git reset --hard

is not helping.

On Mon, Nov 3, 2008 at 11:34 PM, Andrew Arnott <andrewarnott@gmail.com> wrote:
> I guess I was still in the middle of a rebase.  git rebase --abort
> sort of ultimately fixed it.
>
> On Mon, Nov 3, 2008 at 11:09 PM, Andrew Arnott <andrewarnott@gmail.com> wrote:
>> I was just git commit'ing, and then I was doing a git rebase to squash
>> several commits into one when the rebase failed.  I then did a
>> git checkout -f master
>> git reset --hard
>> but no matter what I do, git thinks that several files have changed.
>> The diff shows all the lines in these several files removed and then
>> added, yet without any changes made to them.  git reset --hard doesn't
>> revert the change. When I jump around history with git checkout these
>> files remain in their "changed" state.  I even tried "git clone" to
>> create a whole new repo, but one of these several files STILL
>> registered as changed before I made any changes.
>>
>> Any idea what's wrong and how to recover?
>>
>> Observe the below command buffer: (I can upload my repo so you can
>> clone it and perhaps repro it if you want).
>>
>> Andrew@LACKY /c/git/dotnetoauth
>> $ git status
>> # On branch master
>> # Changed but not updated:
>> #   (use "git add <file>..." to update what will be committed)
>> #
>> #       modified:   tools/Documentation.targets
>> #       modified:   tools/DotNetOpenAuth.Common.Settings.targets
>> #       modified:   tools/DotNetOpenAuth.Versioning.targets
>> #       modified:
>> tools/Sandcastle/Presentation/vs2005/Content/reference_content.xml
>> #       modified:   tools/libcheck.ps1
>> #       modified:   tools/sandcastle.targets
>> #
>> no changes added to commit (use "git add" and/or "git commit -a")
>>
>> Andrew@LACKY /c/git/dotnetoauth
>> $ git reset --hard
>> HEAD is now at 13d37b8 Patching up the bad merges in the phases.
>>
>> Andrew@LACKY /c/git/dotnetoauth
>> $ git status
>> # On branch master
>> # Changed but not updated:
>> #   (use "git add <file>..." to update what will be committed)
>> #
>> #       modified:   tools/Documentation.targets
>> #       modified:   tools/DotNetOpenAuth.Common.Settings.targets
>> #       modified:   tools/DotNetOpenAuth.Versioning.targets
>> #       modified:
>> tools/Sandcastle/Presentation/vs2005/Content/reference_content.xml
>> #       modified:   tools/libcheck.ps1
>> #       modified:   tools/sandcastle.targets
>> #
>> no changes added to commit (use "git add" and/or "git commit -a")
>>
>

Re: Repo corrupted somehow?

[Jeff King]

On Tue, Nov 04, 2008 at 07:12:40AM -0800, Andrew Arnott wrote:

> Nah, that wasn't a false alarm after all.  It's happening again, only
> this time for dozens of files, and
> git rebase --abort
> git reset --hard
> 
> is not helping.

This is certainly unusual. Can you provide a tarball of your repo
(including .git and working tree)? I don't know that a clone will be
sufficient if there is something funny going on in your local git index.

-Peff

Re: Repo corrupted somehow?

[Daniel Barkalow]

On Mon, 3 Nov 2008, Andrew Arnott wrote:

> I was just git commit'ing, and then I was doing a git rebase to squash
> several commits into one when the rebase failed.  I then did a
> git checkout -f master
> git reset --hard
> but no matter what I do, git thinks that several files have changed.
> The diff shows all the lines in these several files removed and then
> added, yet without any changes made to them.

That sounds like some failure of CRLF conversion, like it's converting all 
of the line endings somehow when writing to the working tree and then not 
expecting them to be different. Do you have some sort of interesting 
configuration for those? I wonder if you've got a .gitattributes that 
matches the names that git uses for the files, but are on a 
case-insensitive filesystem which lists those files in a way where their 
names don't match (or vice versa).

	-Daniel
*This .sig left intentionally blank*

Re: Repo corrupted somehow?

[Andrew Arnott]

It was the CRLF conversion.  When I played around with
git config --global core.autocrlf true/false
I got the problem to eventually go away.

Thanks for all your responses.

>
> On Tue, Nov 4, 2008 at 8:21 PM, Daniel Barkalow <barkalow@iabervon.org> wrote:
>>
>> On Mon, 3 Nov 2008, Andrew Arnott wrote:
>>
>> > I was just git commit'ing, and then I was doing a git rebase to squash
>> > several commits into one when the rebase failed.  I then did a
>> > git checkout -f master
>> > git reset --hard
>> > but no matter what I do, git thinks that several files have changed.
>> > The diff shows all the lines in these several files removed and then
>> > added, yet without any changes made to them.
>>
>> That sounds like some failure of CRLF conversion, like it's converting all
>> of the line endings somehow when writing to the working tree and then not
>> expecting them to be different. Do you have some sort of interesting
>> configuration for those? I wonder if you've got a .gitattributes that
>> matches the names that git uses for the files, but are on a
>> case-insensitive filesystem which lists those files in a way where their
>> names don't match (or vice versa).
>>
>>        -Daniel
>> *This .sig left intentionally blank*
>

Re: Repo corrupted somehow?

[Daniel Barkalow]

On Tue, 4 Nov 2008, Andrew Arnott wrote:

> It was the CRLF conversion.  When I played around with
> git config --global core.autocrlf true/false
> I got the problem to eventually go away.
> 
> Thanks for all your responses.

It's still worth debugging further, because git should know that it wrote 
the files differently and not see that as changes. It's not too helpful to 
have autocrlf if it causes this problem.

Are you using something (like .gitattributes) to mark files as text or as 
non-text? Is this under Cygwin? It might be fixed in a newer version, 
also.

	-Daniel
*This .sig left intentionally blank*

Re: Repo corrupted somehow?

[Eyvind Bernhardsen]

On 5. nov.. 2008, at 06.56, Daniel Barkalow wrote:

> On Tue, 4 Nov 2008, Andrew Arnott wrote:
>
>> It was the CRLF conversion.  When I played around with
>> git config --global core.autocrlf true/false
>> I got the problem to eventually go away.
>>
>> Thanks for all your responses.
>
> It's still worth debugging further, because git should know that it  
> wrote
> the files differently and not see that as changes. It's not too  
> helpful to
> have autocrlf if it causes this problem.

I think I know what this is.  If a repository contains files with  
CRLFs, those files will show as modified when core.autcorlf is true  
(if you commit them, the CRLFs will be converted to CRs in the  
repository, so in a sense they _are_ modified).  Try turning autocrlf  
back on, cloning the repository, then touching all the files (to make  
git check them for changes) and see if you get the same problem.

I proposed an alternative autocrlf implementation on the list a while  
back: making it an attribute instead of a configuration setting and  
adding a configuration setting to tell git which line ending is  
preferred when the autocrlf attribute is set.

That would allow you to turn on autocrlf and let git convert all CRLFs  
to CRs in a single commit, thus converting a repository with CRLFs to  
one that can be used with autocrlf in a versioned way.  In theory that  
lets you check out new commits with EOL conversion while old commits  
will be left alone (avoiding the problem you saw), but  
since .gitattributes is read from the working directory and not the  
tree to be checked out, it doesn't work perfectly.

I implemented the easy bit (reading autocrlf from .gitattributes), but  
for various reasons the patch has just been gathering dust in my  
private git.git repo.  Maybe I should dust it off :)
-- 
Eyvind

Re: Repo corrupted somehow?

[Andrew Arnott]

I thought that autocrlf always 'added' CRLF instead of adding merely
CR.  Dang.  I'd rather have CRLF text files.

On Wed, Nov 5, 2008 at 4:26 AM, Eyvind Bernhardsen
<eyvind-git@orakel.ntnu.no> wrote:
> On 5. nov.. 2008, at 06.56, Daniel Barkalow wrote:
>
>> On Tue, 4 Nov 2008, Andrew Arnott wrote:
>>
>>> It was the CRLF conversion.  When I played around with
>>> git config --global core.autocrlf true/false
>>> I got the problem to eventually go away.
>>>
>>> Thanks for all your responses.
>>
>> It's still worth debugging further, because git should know that it wrote
>> the files differently and not see that as changes. It's not too helpful to
>> have autocrlf if it causes this problem.
>
> I think I know what this is.  If a repository contains files with CRLFs,
> those files will show as modified when core.autcorlf is true (if you commit
> them, the CRLFs will be converted to CRs in the repository, so in a sense
> they _are_ modified).  Try turning autocrlf back on, cloning the repository,
> then touching all the files (to make git check them for changes) and see if
> you get the same problem.
>
> I proposed an alternative autocrlf implementation on the list a while back:
> making it an attribute instead of a configuration setting and adding a
> configuration setting to tell git which line ending is preferred when the
> autocrlf attribute is set.
>
> That would allow you to turn on autocrlf and let git convert all CRLFs to
> CRs in a single commit, thus converting a repository with CRLFs to one that
> can be used with autocrlf in a versioned way.  In theory that lets you check
> out new commits with EOL conversion while old commits will be left alone
> (avoiding the problem you saw), but since .gitattributes is read from the
> working directory and not the tree to be checked out, it doesn't work
> perfectly.
>
> I implemented the easy bit (reading autocrlf from .gitattributes), but for
> various reasons the patch has just been gathering dust in my private git.git
> repo.  Maybe I should dust it off :)
> --
> Eyvind
>
>