git filter-branch can "forget" directories on case insensitive filesystems

git filter-branch can "forget" directories on case insensitive filesystems

[Simeon Maxein]

When running git filter-branch on a case insensitive filesystem, the
working tree checked out for filtering is missing a directory in my
project. This is probably related to the problem discussed here:
http://article.gmane.org/gmane.comp.version-control.git/154662

In contrast to that report, this one affects repositories on
case-insensitive filesystems, regardless of the setting of
core.ignorecase.

To reproduce the problem, follow the steps below. I tested them with
git 1.7.0.4 on a laptop running Ubuntu, with the repository on a Fat32
USB drive.

mkdir gittest
cd gittest
git init
mkdir testdir
echo 'abc' >testdir/testfile
git add testdir
git commit -m foo
git rm -r testdir
mkdir testDir
echo 'abc' >testDir/testfile
git add testDir
git commit -m bar

Now, check out master^1 and master to ensure the commits look as
expected. Then, run this:

git filter-branch --tree-filter 'ls' master

You will notice that the directory vanishes when the second commit is filtered.

The problem occurs whether core.ignorecase is set to true or false. It
doesn't occur on case sensitive filesystems.

Simeon

Re: git filter-branch can "forget" directories on case insensitive filesystems

[Simeon Maxein]

In my opinion this is a quite serious issue, because files are lost
without any indication to the user. As of git 1.7.3.1 (tested on
Windows/NTFS with msysGit this time), the problem still exists. Please
give it a look. Fullquote of the problem description / steps to
reproduce follows.

Simeon
> When running git filter-branch on a case insensitive filesystem, the
> working tree checked out for filtering is missing a directory in my
> project. This is probably related to the problem discussed here:
> http://article.gmane.org/gmane.comp.version-control.git/154662
>
> In contrast to that report, this one affects repositories on
> case-insensitive filesystems, regardless of the setting of
> core.ignorecase.
>
> To reproduce the problem, follow the steps below. I tested them with
> git 1.7.0.4 on a laptop running Ubuntu, with the repository on a Fat32
> USB drive.
>
> mkdir gittest
> cd gittest
> git init
> mkdir testdir
> echo 'abc' >testdir/testfile
> git add testdir
> git commit -m foo
> git rm -r testdir
> mkdir testDir
> echo 'abc' >testDir/testfile
> git add testDir
> git commit -m bar
>
> Now, check out master^1 and master to ensure the commits look as
> expected. Then, run this:
>
> git filter-branch --tree-filter 'ls' master
>
> You will notice that the directory vanishes when the second commit is filtered.
>
> The problem occurs whether core.ignorecase is set to true or false. It
> doesn't occur on case sensitive filesystems.
>
> Simeon

Re: git filter-branch can "forget" directories on case insensitive filesystems

[Johannes Sixt]

On Dienstag, 25. Januar 2011, Simeon Maxein wrote:
> In my opinion this is a quite serious issue, because files are lost
> without any indication to the user. As of git 1.7.3.1 (tested on
> Windows/NTFS with msysGit this time), the problem still exists. Please
> give it a look. Fullquote of the problem description / steps to
> reproduce follows.

> > mkdir testdir
> > echo 'abc' >testdir/testfile
> > git add testdir
> > git commit -m foo
> > git rm -r testdir
> > mkdir testDir
> > echo 'abc' >testDir/testfile
> > git add testDir
> > git commit -m bar

Please retry with current release condidate of 1.7.4; it has some 
core.ignorecase improvements w.r.t. directories. It could well be that your 
problem is fixed.

-- Hannes

Re: git filter-branch can "forget" directories on case insensitive filesystems

[Simeon Maxein]

Am 25.01.2011 22:56, schrieb Johannes Sixt:
> On Dienstag, 25. Januar 2011, Simeon Maxein wrote:
>> In my opinion this is a quite serious issue, because files are lost
>> without any indication to the user. As of git 1.7.3.1 (tested on
>> Windows/NTFS with msysGit this time), the problem still exists. Please
>> give it a look. Fullquote of the problem description / steps to
>> reproduce follows.
>>> mkdir testdir
>>> echo 'abc' >testdir/testfile
>>> git add testdir
>>> git commit -m foo
>>> git rm -r testdir
>>> mkdir testDir
>>> echo 'abc' >testDir/testfile
>>> git add testDir
>>> git commit -m bar
> Please retry with current release condidate of 1.7.4; it has some 
> core.ignorecase improvements w.r.t. directories. It could well be that your 
> problem is fixed.
>
> -- Hannes
Thanks for the suggestion. The directory doesn't vanish anymore with
1.7.4, so a big Thank You to the developers for improving this. When
rewriting the second commit ls still prints testdir as lowercase though.
More of a nitpick, but it would still be neat to have it right.

The issue of extra files appearing during filter-branch (
http://article.gmane.org/gmane.comp.version-control.git/154662 ) is
still present.

Simeon

Re: git filter-branch can "forget" directories on case insensitive filesystems

[Erik Faye-Lund]

On Wed, Jan 26, 2011 at 12:26 AM, Simeon Maxein <smaxein@googlemail.com> wrote:
> Am 25.01.2011 22:56, schrieb Johannes Sixt:
>> On Dienstag, 25. Januar 2011, Simeon Maxein wrote:
>>> In my opinion this is a quite serious issue, because files are lost
>>> without any indication to the user. As of git 1.7.3.1 (tested on
>>> Windows/NTFS with msysGit this time), the problem still exists. Please
>>> give it a look. Fullquote of the problem description / steps to
>>> reproduce follows.
>>>> mkdir testdir
>>>> echo 'abc' >testdir/testfile
>>>> git add testdir
>>>> git commit -m foo
>>>> git rm -r testdir
>>>> mkdir testDir
>>>> echo 'abc' >testDir/testfile
>>>> git add testDir
>>>> git commit -m bar
>> Please retry with current release condidate of 1.7.4; it has some
>> core.ignorecase improvements w.r.t. directories. It could well be that your
>> problem is fixed.
>>
>> -- Hannes
> Thanks for the suggestion. The directory doesn't vanish anymore with
> 1.7.4, so a big Thank You to the developers for improving this. When
> rewriting the second commit ls still prints testdir as lowercase though.
> More of a nitpick, but it would still be neat to have it right.
>

This part is correct behavior  - git's internal representation is case
sensitive. So git's record of the file is still 'testdir', even i
you've deleted it and created a new called 'testDir'.

Re: git filter-branch can "forget" directories on case insensitive filesystems

[Simeon Maxein]

Am 26.01.2011 00:31, schrieb Erik Faye-Lund:
> On Wed, Jan 26, 2011 at 12:26 AM, Simeon Maxein <smaxein@googlemail.com> wrote:
>> Am 25.01.2011 22:56, schrieb Johannes Sixt:
>>> On Dienstag, 25. Januar 2011, Simeon Maxein wrote:
>>>> In my opinion this is a quite serious issue, because files are lost
>>>> without any indication to the user. As of git 1.7.3.1 (tested on
>>>> Windows/NTFS with msysGit this time), the problem still exists. Please
>>>> give it a look. Fullquote of the problem description / steps to
>>>> reproduce follows.
>>>>> mkdir testdir
>>>>> echo 'abc' >testdir/testfile
>>>>> git add testdir
>>>>> git commit -m foo
>>>>> git rm -r testdir
>>>>> mkdir testDir
>>>>> echo 'abc' >testDir/testfile
>>>>> git add testDir
>>>>> git commit -m bar
>>> Please retry with current release condidate of 1.7.4; it has some
>>> core.ignorecase improvements w.r.t. directories. It could well be that your
>>> problem is fixed.
>>>
>>> -- Hannes
>> Thanks for the suggestion. The directory doesn't vanish anymore with
>> 1.7.4, so a big Thank You to the developers for improving this. When
>> rewriting the second commit ls still prints testdir as lowercase though.
>> More of a nitpick, but it would still be neat to have it right.
>>
> This part is correct behavior  - git's internal representation is case
> sensitive. So git's record of the file is still 'testdir', even i
> you've deleted it and created a new called 'testDir'.
A normal checkout for those commits results in "testdir" for the first
one and "testDir" for the second one, so git does store the name
difference. I would intuitively expect the trees prepared during
filter-branch to be consistent with the result of a checkout.

Simeon

Re: git filter-branch can "forget" directories on case insensitive filesystems

[Erik Faye-Lund]

On Wed, Jan 26, 2011 at 1:24 AM, Simeon Maxein <smaxein@googlemail.com> wrote:
> Am 26.01.2011 00:31, schrieb Erik Faye-Lund:
>> On Wed, Jan 26, 2011 at 12:26 AM, Simeon Maxein <smaxein@googlemail.com> wrote:
>>> Am 25.01.2011 22:56, schrieb Johannes Sixt:
>>>> On Dienstag, 25. Januar 2011, Simeon Maxein wrote:
>>>>> In my opinion this is a quite serious issue, because files are lost
>>>>> without any indication to the user. As of git 1.7.3.1 (tested on
>>>>> Windows/NTFS with msysGit this time), the problem still exists. Please
>>>>> give it a look. Fullquote of the problem description / steps to
>>>>> reproduce follows.
>>>>>> mkdir testdir
>>>>>> echo 'abc' >testdir/testfile
>>>>>> git add testdir
>>>>>> git commit -m foo
>>>>>> git rm -r testdir
>>>>>> mkdir testDir
>>>>>> echo 'abc' >testDir/testfile
>>>>>> git add testDir
>>>>>> git commit -m bar
>>>> Please retry with current release condidate of 1.7.4; it has some
>>>> core.ignorecase improvements w.r.t. directories. It could well be that your
>>>> problem is fixed.
>>>>
>>>> -- Hannes
>>> Thanks for the suggestion. The directory doesn't vanish anymore with
>>> 1.7.4, so a big Thank You to the developers for improving this. When
>>> rewriting the second commit ls still prints testdir as lowercase though.
>>> More of a nitpick, but it would still be neat to have it right.
>>>
>> This part is correct behavior  - git's internal representation is case
>> sensitive. So git's record of the file is still 'testdir', even i
>> you've deleted it and created a new called 'testDir'.
> A normal checkout for those commits results in "testdir" for the first
> one and "testDir" for the second one, so git does store the name
> difference. I would intuitively expect the trees prepared during
> filter-branch to be consistent with the result of a checkout.
>

I'm sorry, I misread what you wrote (I read the 'git rm testDir' as
'rm testDir').

The reason the old folder name is output appears to be that "git
checkout-index -f -u -a" (called from git-filter-branch) doesn't
change the directory name when the directory is already there. But to
be honest, I'm not really sure if it should or not. I'm too tired to
think it through right now ;)