Cygwin: problem with renaming and case

Cygwin: problem with renaming and case

[Frank]

Hi,
Don't know exactly if this is a bug or a feature or something in the 
middle, but I have a lot of problems while changing just the casing of 
file names and using git mv und cygwin. Here's a test case:

mkdir testrename
cd testrename
git init
echo "AAA" >aaa.txt
echo "BBB" >bbb.txt
git add aaa.txt
git add bbb.txt
git commit -m "First commit"
git checkout -b new_branch
git mv aaa.txt ccc.txt
git commit -a -m "Moved file"
echo "NEW AAA" >Aaa.txt
git add Aaa.txt
git commit -m "Added Aaa"
#aaa.txt exists in master, Aaa.txt in new_branch
git checkout master

Last command gives: "fatal: Untracked working tree file 'aaa.txt' would 
be overwritten by merge".
I know I can use git checkout -f but the problem returns while others do 
merging/pulling from my repo, etc.
Thanks,
Frank

Re: Cygwin: problem with renaming and case

[Linus Torvalds]

On Fri, 21 Mar 2008, Frank wrote:
>
> Don't know exactly if this is a bug or a feature or something in the middle,
> but I have a lot of problems while changing just the casing of file names and
> using git mv und cygwin.

It's not exactly a bug, it's a "feature" of that crap we call Windows and 
OS X that makes them claim that soem files exist even though they don't.

> Here's a test case:
> [ ... ]
> echo "NEW AAA" >Aaa.txt
> git add Aaa.txt
> git commit -m "Added Aaa"
> #aaa.txt exists in master, Aaa.txt in new_branch
> git checkout master
> 
> Last command gives: "fatal: Untracked working tree file 'aaa.txt' would be
> overwritten by merge".

So what happens here is that git is trying to switch back to master, which 
has the file "aaa.txt" in it, and before it does that switch is wants to 
make sure that the new files it creates won't be overwriting some 
untracked file data that you may already have.

Now, you don't *really* have a file called "aaa.txt" any more, but what 
git is doing is that it knows it will create that file, so before it 
starts writing it, it will do a "lstat()" on the file to see that there is 
nothing there.

So git will lstat() that pathname "aaa.txt", and your absolute crap 
filesystem will say "sure, I have that file". Because it will match the 
"Aaa.txt" that you do have from before the merge.

Now, you're tracking "Aaa.txt" in the branch you're leaving, and git knows 
that, but git also knows that the branch you're leaving is *not* tracking 
"aaa.txt", so obviously the copy of "aaa.txt" that the filesystem reports 
is not saved anywhere, and git says "I refuse to overwrite it, because 
that would destroy your untracked content".

> I know I can use git checkout -f but the problem returns while others do
> merging/pulling from my repo, etc.

Case-insensitive filesystems are utter crap. Git doesn't really support 
them, but you can use git on them pretty well as long as you don't 
introduce these kinds of issues by hand. For now, -f is the only 
reasonable thing to do.

Will we fix it? I suspect we will teach git about these kinds of name 
aliases some day, but most of the git developers avoid broken filesystems, 
and the ones that are on broken filesystems tend to avoid having the same 
name in different cases, so it's not exactly a high priority.

It's actually nontrivial to get right and test (on sane filesystems).

I could probably whip up something that gets US-ASCII right for this 
particular case (ie just switching between branches) reasonably easily (ie 
do a pretty stupid "works for the common case" thing without even trying 
to be anythign else). I just really haven't had a lot of motivation to do 
it. Let's see if I can get the energy..

		Linus

Re: Cygwin: problem with renaming and case

[Avery Pennarun]

On Fri, Mar 21, 2008 at 1:46 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>  Now, you're tracking "Aaa.txt" in the branch you're leaving, and git knows
>  that, but git also knows that the branch you're leaving is *not* tracking
>  "aaa.txt", so obviously the copy of "aaa.txt" that the filesystem reports
>  is not saved anywhere, and git says "I refuse to overwrite it, because
>  that would destroy your untracked content".

I don't know if this helps, but if git would delete the files it's
planning to forget before checking the existence of files it's
planning to create, case sensitivity problems like these would
automatically disappear and you wouldn't have to worry about case (and
accent, and and...) folding by hand.

Unfortunately, that would mean git is changing things around before it
can safely make a decision about whether that's a good idea.  That
said, it would be possible to put things back, since it knows the
files it deleted are stored safely in the original branch anyway.

Have fun,

Avery

Re: Cygwin: problem with renaming and case

[Linus Torvalds]

On Fri, 21 Mar 2008, Avery Pennarun wrote:
> 
> I don't know if this helps, but if git would delete the files it's 
> planning to forget before checking the existence of files it's planning 
> to create, case sensitivity problems like these would automatically 
> disappear and you wouldn't have to worry about case (and accent, and 
> and...) folding by hand.

Yeah, but the whole logic of git-read-tree (which is what does this all) 
is to verify everything is up-to-date and a-ok before doing anything to 
your filesystem. Which is just a good idea in general.

Basically, we don't want to do something part-way, and then notice later 
that "oh, but.." and have to try to undo the thing we did partially.

So I agree, in this case the "remove files that go away first" would work 
around the problem, and we could look into whether that is reasonable in 
some cases, but in general it's not trivial either.

My personal guess is that it's probably better to start teaching git about 
case-broken filesystem, even if we start it with some common special case 
rather than getting every case right from the beginning.

		Linus

Re: Cygwin: problem with renaming and case

[Dmitry Potapov]

On Fri, Mar 21, 2008 at 7:07 PM, Frank <streamlake@tiscali.it> wrote:
> Hi,
>  Don't know exactly if this is a bug or a feature or something in the
>  middle, but I have a lot of problems while changing just the casing of
>  file names and using git mv und cygwin. Here's a test case:
>
>  mkdir testrename
>  cd testrename
>  git init
>  echo "AAA" >aaa.txt
>  echo "BBB" >bbb.txt
>  git add aaa.txt
>  git add bbb.txt
>  git commit -m "First commit"
>  git checkout -b new_branch
>  git mv aaa.txt ccc.txt
>  git commit -a -m "Moved file"
>  echo "NEW AAA" >Aaa.txt
>  git add Aaa.txt
>  git commit -m "Added Aaa"
>  #aaa.txt exists in master, Aaa.txt in new_branch
>  git checkout master

I wonder do you really need to have two files on different branches whose
name only differ by case, especially when you work on case insensitive
filesystem? I suspect the answer is no. In this case, you can choose one
policy for file naming and stick to it. For instance, that all names should
be in low case except Makefile, or something like that. This policy can be
enforced using pre-commit hook.

Dmitry

Re: Cygwin: problem with renaming and case

[streamlake]

Dmitry Potapov ha scritto:
>
>
> I wonder do you really need to have two files on different branches whose
> name only differ by case, especially when you work on case insensitive
> filesystem? I suspect the answer is no. In this case, you can choose one
> policy for file naming and stick to it. For instance, that all names should
> be in low case except Makefile, or something like that. This policy can be
> enforced using pre-commit hook.
>
> Dmitry
>
>   

You're right, in fact it usually happens as the result of a mistake in 
naming a file between two branches or deleting a file and creating 
another one months later with the same name, not really a question of 
policies... :-)

@Linus
As always, I'm absolutely not a windz fan (and this is demonstrated by 
the fact that I've been using cygwin for long time instead of the crappy 
win command prompt, and use linux every day for a few non-strictly-windz 
projects), but I 'must' use it if I want to work, there's no choice 
where I come from, and I can't change the market by myself, even if I 
strongly support linux as a substitute...
So, given the fact that git is almost 'officially' supported at least 
under cygwin, I think it would be a good idea, if technically possible, 
to have a look at this kind of features. Not to mention the fact that 
having a broader audience for a project like git can be positive...

Thanks for your help,
Frank

Re: Cygwin: problem with renaming and case

[Brian Dessent]

streamlake@tiscali.it wrote:

> So, given the fact that git is almost 'officially' supported at least
> under cygwin, I think it would be a good idea, if technically possible,
> to have a look at this kind of features. Not to mention the fact that

You can use a Cygwin managed mount if you really need case sensitivity.

Brian

Re: Cygwin: problem with renaming and case

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> So I agree, in this case the "remove files that go away first" would work 
> around the problem, and we could look into whether that is reasonable in 
> some cases, but in general it's not trivial either.

I think we have two phase "remove then create" in git-apply for an
entirely different reason (we do check before doing any removal or
creation).

> My personal guess is that it's probably better to start teaching git about 
> case-broken filesystem, even if we start it with some common special case 
> rather than getting every case right from the beginning.

Hmm.  I have to say I am not very enthused by the prospect, as I agree
with your reasoning in your earlier message why this has been lower
priority ("sane people when forced to use case corrupting systems avoid
problematic paths to make this a non-issue anyway").  My feeling is that
this falls into the "when we are bored to death and have absolutely
nothing better to do" category.

Re: Cygwin: problem with renaming and case

[Steffen Prohaska]

On Fri, 21 Mar 2008, Junio C Hamano wrote:

> Linus Torvalds <torvalds@linux-foundation.org> writes:
> 
> > My personal guess is that it's probably better to start teaching git about 
> > case-broken filesystem, even if we start it with some common special case 
> > rather than getting every case right from the beginning.
> 
> Hmm.  I have to say I am not very enthused by the prospect, as I agree
> with your reasoning in your earlier message why this has been lower
> priority ("sane people when forced to use case corrupting systems avoid
> problematic paths to make this a non-issue anyway").  My feeling is that
> this falls into the "when we are bored to death and have absolutely
> nothing better to do" category.

Sane people might be forced to modify paths in a problematic way more
often than you think.  A common error I see in practice is that
a developer introduces a typo when adding a new header file on
a "case-broken" filesystem.  A typo that only introduces a different
case in the filename and the matching #include statement is
unrecognizable on a "case-broken" filesystem.  The compiler will find
the file regardless of the different case in the include statement.
Only when the source is checked out on a case-sensitive filesystem the
error is recognized and needs to be fixed.  The lucky case is if the
typo is in the #include statement.  The problematic case is if the case
of the filename wrong (violates the coding style of the project).  In
the latter case the file needs to be renamed to a path that only differs
in case, which triggers the problem.

            Steffen

Re: Cygwin: problem with renaming and case

[Nagy Balázs]

Dmitry Potapov wrote:
> I wonder do you really need to have two files on different branches whose
> name only differ by case, especially when you work on case insensitive
> filesystem? I suspect the answer is no. In this case, you can choose one
> policy for file naming and stick to it. For instance, that all names should
> be in low case except Makefile, or something like that. This policy can be
> enforced using pre-commit hook.
>   

qmail-1.03 is one of the rare species which has two files which differ 
only in their case, namely INSTALL and install.  The first one contains 
the documentation, the latter one is compiled from source.  Apart from 
that I don't know any other affected projects.

On the other hand, most of the software developers are morons, 
especially in a corporate environment.  The problem is you cannot refuse 
their work all the way, and they like to create evil twins (file names 
which were removed and added again), and case collisions.  I could even 
see a lot of symlinks in a clearcase vob which poined to 
`..\..\..\../a/b/c'.  Developer stupidity is unlimited.

Regards,
-- 
-jul-

Re: Cygwin: problem with renaming and case

[Linus Torvalds]

On Sat, 22 Mar 2008, Nagy Bal?zs wrote:
> 
> qmail-1.03 is one of the rare species which has two files which differ only in
> their case, namely INSTALL and install.  The first one contains the
> documentation, the latter one is compiled from source.  Apart from that I
> don't know any other affected projects.

The kernel has lots of them. Well, not "lots" (considering that it has 
23000+ filenames), but something like 20+ names that exists in two forms 
with just different case.

Do

	git ls-files | tr '[A-Z]' '[a-z]' | sort | uniq -d

and you'll get

	include/linux/netfilter_ipv4/ipt_connmark.h
	include/linux/netfilter_ipv4/ipt_dscp.h
	include/linux/netfilter_ipv4/ipt_ecn.h
	include/linux/netfilter_ipv4/ipt_mark.h
	include/linux/netfilter_ipv4/ipt_tcpmss.h
	include/linux/netfilter_ipv4/ipt_tos.h
	include/linux/netfilter_ipv4/ipt_ttl.h
	include/linux/netfilter_ipv6/ip6t_hl.h
	include/linux/netfilter_ipv6/ip6t_mark.h
	include/linux/netfilter/xt_connmark.h
	include/linux/netfilter/xt_dscp.h
	include/linux/netfilter/xt_mark.h
	include/linux/netfilter/xt_rateest.h
	include/linux/netfilter/xt_tcpmss.h
	net/ipv4/netfilter/ipt_ecn.c
	net/ipv4/netfilter/ipt_ttl.c
	net/ipv6/netfilter/ip6t_hl.c
	net/netfilter/xt_connmark.c
	net/netfilter/xt_dscp.c
	net/netfilter/xt_mark.c
	net/netfilter/xt_rateest.c
	net/netfilter/xt_tcpmss.c

where you basically have the netfilter people using lower-case version for 
netfilter matching rules and the uppercase vesion for rewriting rules. Or 
something.

Admittedly the netfilter people _are_ strange, and we need to make sure 
that they take all their medication regularly or they do stupid things, 
but it does happen. The kernel people obviously expect people to use sane 
filesystems for kernel development..

			Linus