after first git clone of linux kernel repository there are changed files in working dir

after first git clone of linux kernel repository there are changed files in working dir

[rdkrsr]

I just fetched the sources without changing anything, but git diff
shows, that there are changes that are not yet updated (changed but not
updated: use git add to ...). Why is it like that?

I use msysgit on windows, maybe that is one reason?

Re: after first git clone of linux kernel repository there are changed files in working dir

[Brett Simmers]

On Wed, Dec 10, 2008 at 1:22 PM, rdkrsr <rdkrsr@googlemail.com> wrote:
> I just fetched the sources without changing anything, but git diff
> shows, that there are changes that are not yet updated (changed but not
> updated: use git add to ...). Why is it like that?
>
> I use msysgit on windows, maybe that is one reason?

What are the filenames? I've seen git on Windows get confused if a
repository has two files that are the same except for the case of some
of the letters (since both can't exist by default on NTFS).

-Brett

Fwd: after first git clone of linux kernel repository there are changed files in working dir

[rdkrsr]

I'm sorry that I didn't answer to git mailing list address. So here
comes the email again.


Here is what I did:

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/li
t linux-2.6
Initialize linux-2.6/.git
Initialized empty Git repository in c:/Dokumente und Einstellungen/Ge
Eigene Dateien/vbox/linux-2.6/.git/
remote: Counting objects: 980697, done.←[K
remote: Compressing objects: 100% (161545/161545), done.←[K
remote: Total 980697 (delta 818552), reused 978923 (delta 816954)←[K
Receiving objects: 100% (980697/980697), 236.14 MiB | 90 KiB/s, done.
Resolving deltas: 100% (818552/818552), done.
Checking out files: 100% (25254/25254), done.

$ cd linux-2.6

$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#       modified:   Documentation/IO-mapping.txt
#       modified:   include/linux/netfilter/xt_CONNMARK.h
#       modified:   include/linux/netfilter/xt_DSCP.h
#       modified:   include/linux/netfilter/xt_MARK.h
#       modified:   include/linux/netfilter/xt_RATEEST.h
#       modified:   include/linux/netfilter/xt_TCPMSS.h
#       modified:   include/linux/netfilter_ipv4/ipt_CONNMARK.h
#       modified:   include/linux/netfilter_ipv4/ipt_DSCP.h
#       modified:   include/linux/netfilter_ipv4/ipt_ECN.h
#       modified:   include/linux/netfilter_ipv4/ipt_MARK.h
#       modified:   include/linux/netfilter_ipv4/ipt_TCPMSS.h
#       modified:   include/linux/netfilter_ipv4/ipt_TOS.h
#       modified:   include/linux/netfilter_ipv4/ipt_TTL.h
#       modified:   include/linux/netfilter_ipv6/ip6t_HL.h
#       modified:   include/linux/netfilter_ipv6/ip6t_MARK.h
#       modified:   net/ipv4/netfilter/ipt_ECN.c
#       modified:   net/ipv4/netfilter/ipt_TTL.c
#       modified:   net/ipv6/netfilter/ip6t_HL.c
#       modified:   net/netfilter/xt_CONNMARK.c
#       modified:   net/netfilter/xt_DSCP.c
#       modified:   net/netfilter/xt_MARK.c
#       modified:   net/netfilter/xt_RATEEST.c
#       modified:   net/netfilter/xt_TCPMSS.c
#
no changes added to commit (use "git add" and/or "git commit -a")

2008/12/10 Brett Simmers <swtaarrs@gmail.com>:
> On Wed, Dec 10, 2008 at 1:22 PM, rdkrsr <rdkrsr@googlemail.com> wrote:
>> I just fetched the sources without changing anything, but git diff
>> shows, that there are changes that are not yet updated (changed but not
>> updated: use git add to ...). Why is it like that?
>>
>> I use msysgit on windows, maybe that is one reason?
>
> What are the filenames? I've seen git on Windows get confused if a
> repository has two files that are the same except for the case of some
> of the letters (since both can't exist by default on NTFS).
>
> -Brett
>

Re: Fwd: after first git clone of linux kernel repository there are changed files in working dir

[Linus Torvalds]

On Thu, 11 Dec 2008, rdkrsr wrote:
>
> I'm sorry that I didn't answer to git mailing list address. So here
> comes the email again.

You have a broken filesystem.

> $ git status
> # On branch master
> # Changed but not updated:
> #   (use "git add <file>..." to update what will be committed)
> #
> #       modified:   Documentation/IO-mapping.txt
> #       modified:   include/linux/netfilter/xt_CONNMARK.h
> #       modified:   include/linux/netfilter/xt_DSCP.h
> #       modified:   include/linux/netfilter/xt_MARK.h
> #       modified:   include/linux/netfilter/xt_RATEEST.h
...

This is _exactly_ what happens if you try to develop the Linux kernel on a 
case-insensitive filesystem. The kernel source tree has several files that 
differ only in case, eg

	Documentation/IO-mapping.txt
	Documentation/io-mapping.txt
	include/linux/netfilter/xt_tcpmss.h
	include/linux/netfilter/xt_TCPMSS.h
	..

and if you try to check it out on a broken filesystem, then the second 
file will overwrite the first one, and git will think that you have 
modified it. 

OS X? Afaik, you can fix it by using NFS or UFS. And I think ZFS has a 
case-sensitive mode too (and it may even be the default). In fact, I think 
newer versions of OS X even allow that piece-of-sh*t HFS+ to be case 
sensitive (and thus make it much less sh*tty).

Of course, there are reports of some Mac software breaking when they use a 
real filesystem, but hey, what else is new?

			Linus

Re: Fwd: after first git clone of linux kernel repository there are changed files in working dir

[rdkrsr]

Thank you, Linus and Brett, for your answers.

I'm not developing linux kernel, I just wanted to experiment with git.
And then I didn't know if this is a normal behaviour of git. I'm using
windows xp and msysgit for this. And the file system is NTFS. I'm
using dual boot to sporadicly use linux and tried also linux in
virtual box. But both isn't really good. Maybe one day I dare to use
linux as my primary OS.

Red

2008/12/11 Linus Torvalds <torvalds@linux-foundation.org>:
>
>
> On Thu, 11 Dec 2008, rdkrsr wrote:
>>
>> I'm sorry that I didn't answer to git mailing list address. So here
>> comes the email again.
>
> You have a broken filesystem.
>
>> $ git status
>> # On branch master
>> # Changed but not updated:
>> #   (use "git add <file>..." to update what will be committed)
>> #
>> #       modified:   Documentation/IO-mapping.txt
>> #       modified:   include/linux/netfilter/xt_CONNMARK.h
>> #       modified:   include/linux/netfilter/xt_DSCP.h
>> #       modified:   include/linux/netfilter/xt_MARK.h
>> #       modified:   include/linux/netfilter/xt_RATEEST.h
> ...
>
> This is _exactly_ what happens if you try to develop the Linux kernel on a
> case-insensitive filesystem. The kernel source tree has several files that
> differ only in case, eg
>
>        Documentation/IO-mapping.txt
>        Documentation/io-mapping.txt
>        include/linux/netfilter/xt_tcpmss.h
>        include/linux/netfilter/xt_TCPMSS.h
>        ..
>
> and if you try to check it out on a broken filesystem, then the second
> file will overwrite the first one, and git will think that you have
> modified it.
>
> OS X? Afaik, you can fix it by using NFS or UFS. And I think ZFS has a
> case-sensitive mode too (and it may even be the default). In fact, I think
> newer versions of OS X even allow that piece-of-sh*t HFS+ to be case
> sensitive (and thus make it much less sh*tty).
>
> Of course, there are reports of some Mac software breaking when they use a
> real filesystem, but hey, what else is new?
>
>                        Linus
>

Re: Fwd: after first git clone of linux kernel repository there are changed files in working dir

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 792 bytes --]

On Thursday 2008 December 11 11:58:01 rdkrsr wrote:
>I'm not developing linux kernel, I just wanted to experiment with git.
>And then I didn't know if this is a normal behaviour of git. I'm using
>windows xp and msysgit for this. And the file system is NTFS.

You might want to choose a more MS Windows-friendly codebase to test with.  
For codebases without filenames that differ only in case, git should work 
fine on MS Windows.  For codebases with filenames that differ only in case, I 
don't know a VCS that will handle them well on MS Windows.
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss03@volumehost.net                      ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.org/                      \_/     

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Fwd: after first git clone of linux kernel repository there are changed files in working dir

[Giuseppe Bilotta]

On Thursday 11 December 2008 18:58, rdkrsr wrote:

> Thank you, Linus and Brett, for your answers.
> 
> I'm not developing linux kernel, I just wanted to experiment with git.
> And then I didn't know if this is a normal behaviour of git. I'm using
> windows xp and msysgit for this. And the file system is NTFS. I'm
> using dual boot to sporadicly use linux and tried also linux in
> virtual box. But both isn't really good. Maybe one day I dare to use
> linux as my primary OS.
> 
> Red
> 
> 2008/12/11 Linus Torvalds <torvalds@linux-foundation.org>:
>>
>>
>> On Thu, 11 Dec 2008, rdkrsr wrote:
>>>
>>> I'm sorry that I didn't answer to git mailing list address. So here
>>> comes the email again.
>>
>> You have a broken filesystem.

Actually, the funny (in a grotesque kind of way) thing about NTFS
is that it's a case-*sensitive* filesystem (in the sense that i
can legally hold files with names that differ only for the case),
but the Windows subsystems use it in case-preserving (but
insensitive) mode. It is possible to use NTFS case insensitively
under Windows, but it requires something such as the Interix
subsystem, and in Windows XP (and possibly later versions) it also
needs toggling a security policy that defaults to enforcing case
insensitivity for all subsystems.

Maybe the Windows ports to git (at least the cygwin one, maybe?)
may be able to exploit this.

-- 
Giuseppe "Oblomov" Bilotta

Re: Fwd: after first git clone of linux kernel repository there are changed files in working dir

[Nick Andrew]

On Thu, Dec 11, 2008 at 06:58:01PM +0100, rdkrsr wrote:
> windows xp and msysgit for this. And the file system is NTFS. I'm
> using dual boot to sporadicly use linux and tried also linux in
> virtual box.

You could use git inside your VirtualBox linux install.

> But both isn't really good. Maybe one day I dare to use
> linux as my primary OS.

It's not scary, really.

Nick.

Re: after first git clone of linux kernel repository there are changed files in working dir

[Hannu Koivisto]

rdkrsr <rdkrsr@googlemail.com> writes:

> I just fetched the sources without changing anything, but git diff
> shows, that there are changes that are not yet updated (changed but not
> updated: use git add to ...). Why is it like that?
>
> I use msysgit on windows, maybe that is one reason?

Kernel source contains pairs of files whose names differ only by
case.  Windows cannot store such pairs (at least by default) and
apparently there is no support for such a situation in git so
you'll only get one file from each pair to your workspace and the
other file is shown as modified.

-- 
Hannu

An idea: maybe Git should use a lock/unlock file mode for problematic files? [Was: Re: after first git clone of linux kernel repository there are changed files in working dir]

[thestar]

Quoting Hannu Koivisto <azure@iki.fi>:
<snip>
> Kernel source contains pairs of files whose names differ only by
> case.  Windows cannot store such pairs (at least by default) and
> apparently there is no support for such a situation in git so
> you'll only get one file from each pair to your workspace and the
> other file is shown as modified.

Could git be modified to allow such repositories to be used on windows  
by locking files that are problematic, for example, a given repository  
could have files 'AAA' and 'aAa'.

The one that correctly represents the on-disk file would be 'open for  
edit', while the other file would be locked.  To edit the other file,  
the existing file would need to be locked, and then the other file  
would then need to be open for edit.

This could even be extended to allow one to "open file AAA for edit as  
aAa.v2', giving the file an alternate name.

Such a workflow would only need to be used for such files, and could  
also be used when there are incompatible file names for that given  
partition type.

Re: An idea: maybe Git should use a lock/unlock file mode for problematic files? [Was: Re: after first git clone of linux kernel repository there are changed files in working dir]

[Daniel Barkalow]

On Tue, 20 Jan 2009, thestar@fussycoder.id.au wrote:

> Quoting Hannu Koivisto <azure@iki.fi>:
> <snip>
> > Kernel source contains pairs of files whose names differ only by
> > case.  Windows cannot store such pairs (at least by default) and
> > apparently there is no support for such a situation in git so
> > you'll only get one file from each pair to your workspace and the
> > other file is shown as modified.
> 
> Could git be modified to allow such repositories to be used on windows  
> by locking files that are problematic, for example, a given repository  
> could have files 'AAA' and 'aAa'.
> 
> The one that correctly represents the on-disk file would be 'open for  
> edit', while the other file would be locked.  To edit the other file,  
> the existing file would need to be locked, and then the other file  
> would then need to be open for edit.
> 
> This could even be extended to allow one to "open file AAA for edit as  
> aAa.v2', giving the file an alternate name.
> 
> Such a workflow would only need to be used for such files, and could  
> also be used when there are incompatible file names for that given  
> partition type.

The hard part is actually identifying what the user's filesystem has done. 
There's pretty good internal support for git knowing that, for a 
particular entry, the filesystem should not be consulted for information. 
I don't think anyone's come up with a suitably cross-platform and 
automatic way to figure out what's happened when git tries to write to a 
particular filename and the system decides it is the same as some other 
filename or it decides to use a different filename instead.

Of course, it is reasonably likely that a project whose files can't all be 
checked out can't be dealt with anyway on that platform (IIRC, the Linux 
kernel build system assumes that it can create both .S and .s files, so it 
won't build on FAT). So nobody's been sufficiently motivated to try to 
implement a fix.

	-Daniel
*This .sig left intentionally blank*

Re: An idea: maybe Git should use a lock/unlock file mode for problematic files? [Was: Re: after first git clone of linux kernel repository there are changed files in working dir]

[John Chapman]

On Tue, 2009-01-20 at 15:11 -0500, Daniel Barkalow wrote:
<snip>
> 
> The hard part is actually identifying what the user's filesystem has done. 
> There's pretty good internal support for git knowing that, for a 
> particular entry, the filesystem should not be consulted for information. 
> I don't think anyone's come up with a suitably cross-platform and 
> automatic way to figure out what's happened when git tries to write to a 
> particular filename and the system decides it is the same as some other 
> filename or it decides to use a different filename instead.

This would only need to interact with the git status command, wouldn't
it?

> 
> Of course, it is reasonably likely that a project whose files can't all be 
> checked out can't be dealt with anyway on that platform (IIRC, the Linux 
> kernel build system assumes that it can create both .S and .s files, so it 
> won't build on FAT). So nobody's been sufficiently motivated to try to 
> implement a fix.

I doubt the kernel builds on windows, but this would allow a windows
user to modify such files, perhaps in preparation for a patch that does
allow the kernel to be built on windows?
(Of course, we're using the kernel here as an example, right?  Nobody
would be insane as to want to use windows for that!)

See, a very annoying thing about windows is that it is quite simple for
a team to commit two files that differ by case alone to a git
repository.

Was just an idea, really.

Re: An idea: maybe Git should use a lock/unlock file mode for problematic files? [Was: Re: after first git clone of linux kernel repository there are changed files in working dir]

[Daniel Barkalow]

On Wed, 21 Jan 2009, John Chapman wrote:

> On Tue, 2009-01-20 at 15:11 -0500, Daniel Barkalow wrote:
> <snip>
> > 
> > The hard part is actually identifying what the user's filesystem has done. 
> > There's pretty good internal support for git knowing that, for a 
> > particular entry, the filesystem should not be consulted for information. 
> > I don't think anyone's come up with a suitably cross-platform and 
> > automatic way to figure out what's happened when git tries to write to a 
> > particular filename and the system decides it is the same as some other 
> > filename or it decides to use a different filename instead.
> 
> This would only need to interact with the git status command, wouldn't
> it?

The information is needed in a bunch of commands (diff and add, for 
example), but I believe that's already taken care on. The problem is 
getting it set automatically instead of having git not notice that the 
filesystem isn't doing what it expects.

> > Of course, it is reasonably likely that a project whose files can't all be 
> > checked out can't be dealt with anyway on that platform (IIRC, the Linux 
> > kernel build system assumes that it can create both .S and .s files, so it 
> > won't build on FAT). So nobody's been sufficiently motivated to try to 
> > implement a fix.
> 
> I doubt the kernel builds on windows, but this would allow a windows
> user to modify such files, perhaps in preparation for a patch that does
> allow the kernel to be built on windows?
> (Of course, we're using the kernel here as an example, right?  Nobody
> would be insane as to want to use windows for that!)
> 
> See, a very annoying thing about windows is that it is quite simple for
> a team to commit two files that differ by case alone to a git
> repository.

My impression was that this didn't happen in practice, because teams
would tend to not have two people create the same file at the same time, 
but with different cases, and people interacting with the same file at 
different times would use whatever case it was introduced with.

I think I'd only heard about problems for people who were using 
filesystems with different properties than what the rest of the developers 
on the project were using.

But I've only ever worked on projects that expect case-sensitivity, and 
mostly on projects that have a standard style that prevents duplication.

	-Daniel
*This .sig left intentionally blank*

Re: An idea: maybe Git should use a lock/unlock file mode for problematic files? [Was: Re: after first git clone of linux kernel repository there are changed files in working dir]

[Alex Riesen]

2009/1/20 Daniel Barkalow <barkalow@iabervon.org>:
> My impression was that this didn't happen in practice, because teams
> would tend to not have two people create the same file at the same time,
> but with different cases, and people interacting with the same file at
> different times would use whatever case it was introduced with.

It will and does happen in practice (annoingly too often even). Not with Git
yet (with Perforce), where people do "branching" by simply copying things
in another directory (perforce world does not know real branches),
renaming files randomly, and putting the new directory back in the
system (or maybe it is the strange tools here which do that - often
it is the first character of a directory or file which gets down- or up-cased).
As Perforce itself is case sensitive (like Git), using of such branches
is a nightmare: the files get overwritten in checkout order which is
not always sorted in predictable order. Combined with case-stupidity
of the file system the working directories sometimes cause "interesting
time" for unlucky users.
Luckily (sadly) it is all-opening-in-a-wall shop, so the problem with "fanthom"
files is rare (it is hard to notice) for most. Which actually makes it more
frustrating when the real shit happens.

And it will happen to Git as well, especially if development go crossplatform.
It is not that hard to accidentally rename a file on case-sensitive file system,
"git add *" it and commit without thinking (that's how most of software
development happens, come to think of it).

Re: An idea: maybe Git should use a lock/unlock file mode for problematic files? [Was: Re: after first git clone of linux kernel repository there are changed files in working dir]

[Daniel Barkalow]

On Wed, 21 Jan 2009, Alex Riesen wrote:

> 2009/1/20 Daniel Barkalow <barkalow@iabervon.org>:
> > My impression was that this didn't happen in practice, because teams
> > would tend to not have two people create the same file at the same time,
> > but with different cases, and people interacting with the same file at
> > different times would use whatever case it was introduced with.
> 
> It will and does happen in practice (annoingly too often even). Not with Git
> yet (with Perforce), where people do "branching" by simply copying things
> in another directory (perforce world does not know real branches),
> renaming files randomly, and putting the new directory back in the
> system (or maybe it is the strange tools here which do that - often
> it is the first character of a directory or file which gets down- or up-cased).

How does the resulting code work at all? With a case-sensitive filesystem, 
most of the files you're using don't have the expected names any more, and 
most systems will therefore not actually build or run.

I have to assume it's your strange tools, because we never have this 
problem at my work, where we also use Perforce. Perhaps it's that we 
always use "p4 integrate //some/project/version/... 
//some/other/project/version/..." which inherently preserves the case of 
all of the filenames within the project.

> As Perforce itself is case sensitive (like Git), using of such branches
> is a nightmare: the files get overwritten in checkout order which is
> not always sorted in predictable order. Combined with case-stupidity
> of the file system the working directories sometimes cause "interesting
> time" for unlucky users.
> Luckily (sadly) it is all-opening-in-a-wall shop, so the problem with "fanthom"
> files is rare (it is hard to notice) for most. Which actually makes it more
> frustrating when the real shit happens.
> 
> And it will happen to Git as well, especially if development go crossplatform.
> It is not that hard to accidentally rename a file on case-sensitive file system,
> "git add *" it and commit without thinking (that's how most of software
> development happens, come to think of it).

People can accidentally rename files? And still have things work when they 
do it on a case-sensitive filesystem?

	-Daniel
*This .sig left intentionally blank*

Re: An idea: maybe Git should use a lock/unlock file mode for problematic files? [Was: Re: after first git clone of linux kernel repository there are changed files in working dir]

[Alex Riesen]

2009/1/21 Daniel Barkalow <barkalow@iabervon.org>:
> On Wed, 21 Jan 2009, Alex Riesen wrote:
>
>> 2009/1/20 Daniel Barkalow <barkalow@iabervon.org>:
>> > My impression was that this didn't happen in practice, because teams
>> > would tend to not have two people create the same file at the same time,
>> > but with different cases, and people interacting with the same file at
>> > different times would use whatever case it was introduced with.
>>
>> It will and does happen in practice (annoingly too often even). Not with Git
>> yet (with Perforce), where people do "branching" by simply copying things
>> in another directory (perforce world does not know real branches),
>> renaming files randomly, and putting the new directory back in the
>> system (or maybe it is the strange tools here which do that - often
>> it is the first character of a directory or file which gets down- or up-cased).
>
> How does the resulting code work at all? ...

Sometimes it does not. Sometimes it does. Depends on that particular
checkout order perforce (or user?) selected to use this time.

> ... With a case-sensitive filesystem,
> most of the files you're using don't have the expected names any more, and
> most systems will therefore not actually build or run.

Except that there is no case-sensitive file systems on development machines.
So a botched case wont be noticed by a standard build procedure unless
the content of the files causes an error.

>> As Perforce itself is case sensitive (like Git), using of such branches
>> is a nightmare: the files get overwritten in checkout order which is
>> not always sorted in predictable order. Combined with case-stupidity
>> of the file system the working directories sometimes cause "interesting
>> time" for unlucky users.
>> Luckily (sadly) it is all-opening-in-a-wall shop, so the problem with "fanthom"
>> files is rare (it is hard to notice) for most. Which actually makes it more
>> frustrating when the real shit happens.
>>
>> And it will happen to Git as well, especially if development go crossplatform.
>> It is not that hard to accidentally rename a file on case-sensitive file system,
>> "git add *" it and commit without thinking (that's how most of software
>> development happens, come to think of it).
>
> People can accidentally rename files?

Aside from tools (and in my own experience - I did) - they can and do.

> And still have things work when they do it on a case-sensitive filesystem?

Shameless luck, I'd say. That and "no file systems permitted, but the one
from finance dept".