Re: Switching from CVS to GIT

Re: Switching from CVS to GIT

[Benoit SIGOURE]

[-- Attachment #1.1: Type: text/plain, Size: 1254 bytes --]

Context: GNU make seems to be willing to switch from CVS to ...  
something else.

On Oct 14, 2007, at 6:57 PM, Paul Smith wrote:

> [...] the big thing no one else seems to have addressed much in
> other discussions I've seen is portability.  It LOOKS like there are
> native ports of GIT to MINGW, but I have no idea how complete and  
> usable
> they are.  If someone who has a Windows system could look into that it
> would be a big help.

I think the best thing to do is to ask directly on the Git ML.

Someone already pointed out that he'd like to use Git on Windows but  
doesn't want to install either Cygwin or MSYS.  Is this possible, or  
will it be possible in the near future?  Is it possible to use one of  
the various GUIs (git-gui, gitk, qgit) on Windows without requiring a  
POSIXish shell etc.?

When will the librarification of Git be finished?  (if Git is  
available as a library, and if this library works on Windows, it will  
greatly help truly native Windows ports).

Not that I like Windows in any way, right, but it's legitimate for  
people working on Windows ports of various software to be willing to  
have a truly native port of Git for Windows.

-- 
Benoit Sigoure aka Tsuna
EPITA Research and Development Laboratory



[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

[-- Attachment #2: Type: text/plain, Size: 134 bytes --]

_______________________________________________
Make-w32 mailing list
Make-w32@gnu.org
http://lists.gnu.org/mailman/listinfo/make-w32

Re: Switching from CVS to GIT

[Marco Costalba]

On 10/14/07, Benoit SIGOURE <tsuna@lrde.epita.fr> wrote:
>
> Is it possible to use one of
> the various GUIs (git-gui, gitk, qgit) on Windows without requiring a
> POSIXish shell etc.?
>

qgit-2.0 works natively under Windows

http://sourceforge.net/project/showfiles.php?group_id=139897

Check the README for how to install.


Marco

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Sun, 14 Oct 2007, Benoit SIGOURE wrote:

> Context: GNU make seems to be willing to switch from CVS to ... 
> something else.
> 
> On Oct 14, 2007, at 6:57 PM, Paul Smith wrote:
> 
> > [...] the big thing no one else seems to have addressed much in other 
> > discussions I've seen is portability.  It LOOKS like there are native 
> > ports of GIT to MINGW, but I have no idea how complete and usable they 
> > are.  If someone who has a Windows system could look into that it 
> > would be a big help.

There is msysGit.  This project is nearing to its first beta, being 
self-hosted since mid-August IIRC.

It is a port of Git to MinGW, using parts of MSys as long as we have 
dependencies on bash and perl.

I have no doubt that we'll manage to finish version 0.3 of the installer 
this week, still not decided if it is still alpha or already beta.

There are some issues with using msysGit, none of them really serious, but 
you better be ready to ask questions on this list or #git in case 
something crops up.  msysGit is young.

Having said that, IMHO msysGit is already quite usable, and should be 
pretty stable within a few weeks (if it is not already).

Ciao,
Dscho

Re: Switching from CVS to GIT

[Andreas Ericsson]

Benoit SIGOURE wrote:
> Context: GNU make seems to be willing to switch from CVS to ... 
> something else.
> 
> On Oct 14, 2007, at 6:57 PM, Paul Smith wrote:
> 
>> [...] the big thing no one else seems to have addressed much in
>> other discussions I've seen is portability.  It LOOKS like there are
>> native ports of GIT to MINGW, but I have no idea how complete and usable
>> they are.  If someone who has a Windows system could look into that it
>> would be a big help.
> 
> I think the best thing to do is to ask directly on the Git ML.
> 
> Someone already pointed out that he'd like to use Git on Windows but 
> doesn't want to install either Cygwin or MSYS.  Is this possible, or 
> will it be possible in the near future?

It is sort of possible. Without cygwin he'll be in the black for the few
features that are still implemented as shell-scripts, but perhaps he/she
will then be inclined to help us migrate those scripts to C builtins.

>  Is it possible to use one of 
> the various GUIs (git-gui, gitk, qgit) on Windows without requiring a 
> POSIXish shell etc.?
> 

qgit is possible to use natively, if one installs the qgit4 libraries for
windows, but it's more of a viewer than an action gui. git-gui and gitk
are usable if you have the windows TCL port. I haven't tried it, but
there are installers available, so testing it out (with all dependencies)
shouldn't take too long.

> When will the librarification of Git be finished?

When someone gets around to doing it ;-)

For a real answer, I'll have to defer to others. Everything works to my
satisfaction where I'm using it, so I'm not very inclined to fiddle with
it and risk breaking things.

>  (if Git is available 
> as a library, and if this library works on Windows, it will greatly help 
> truly native Windows ports).
> 

Yup. I believe the primary reason for libification is to easier support
both porting and fully-fledged gui's.

> Not that I like Windows in any way, right, but it's legitimate for 
> people working on Windows ports of various software to be willing to 
> have a truly native port of Git for Windows.
> 

Naturally. Amazingly few of those stuck with windows have so far
volunteered for helping out though, and since many of us on this list
don't even have a windows system for testing, it's kinda slow going :-/

I'd imagine getting in touch with Dscho to get a list of what's needed,
or reading the biweekly msys.git herald on this list, is the best way
of finding out the porting project's current priorities.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Sun, 14 Oct 2007, Andreas Ericsson wrote:

> Benoit SIGOURE wrote:
> > Context: GNU make seems to be willing to switch from CVS to ... something
> > else.
> > 
> > On Oct 14, 2007, at 6:57 PM, Paul Smith wrote:
> > 
> > > [...] the big thing no one else seems to have addressed much in
> > > other discussions I've seen is portability.  It LOOKS like there are
> > > native ports of GIT to MINGW, but I have no idea how complete and usable
> > > they are.  If someone who has a Windows system could look into that it
> > > would be a big help.
> > 
> > I think the best thing to do is to ask directly on the Git ML.
> > 
> > Someone already pointed out that he'd like to use Git on Windows but 
> > doesn't want to install either Cygwin or MSYS.  Is this possible, or 
> > will it be possible in the near future?
> 
> It is sort of possible. Without cygwin he'll be in the black for the few 
> features that are still implemented as shell-scripts, but perhaps he/she 
> will then be inclined to help us migrate those scripts to C builtins.

Umm.  There are quite a few shell scripts still _necessary_ to run git: 
git-commit, git-fetch and git-merge being the most prominent ones.  The 
first two are in the process of being rewritten _right_ _now_, but no 
official git release has them yet.

And I have to disagree strongly with the "black": In msysGit (which brings 
its own minimal version of MSys), it is very smooth.

> >  Is it possible to use one of the various GUIs (git-gui, gitk, qgit) 
> > on Windows without requiring a POSIXish shell etc.?
> > 
> 
> qgit is possible to use natively, if one installs the qgit4 libraries 
> for windows, but it's more of a viewer than an action gui. git-gui and 
> gitk are usable if you have the windows TCL port. I haven't tried it, 
> but there are installers available, so testing it out (with all 
> dependencies) shouldn't take too long.

FWIW msysGit comes with Tcl.  You can run git gui and gitk without any 
hassles.

> > When will the librarification of Git be finished?
> 
> When someone gets around to doing it ;-)

There has been a GSoC project, and it has a nice small API which can be 
called from Python, for example.

Funnily enough, the first user is qgit as far as I know, which is written 
in C++...

> >  (if Git is available as a library, and if this library works on 
> > Windows, it will greatly help truly native Windows ports).
> 
> Yup. I believe the primary reason for libification is to easier support 
> both porting and fully-fledged gui's.

Why?

I do not see any reason why libification helps the user experience on 
Windows.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Andreas Ericsson]

Johannes Schindelin wrote:
> Hi,
> 
> On Sun, 14 Oct 2007, Andreas Ericsson wrote:
> 
>> Benoit SIGOURE wrote:
>>> Context: GNU make seems to be willing to switch from CVS to ... something
>>> else.
>>>
>>> On Oct 14, 2007, at 6:57 PM, Paul Smith wrote:
>>>
>>>> [...] the big thing no one else seems to have addressed much in
>>>> other discussions I've seen is portability.  It LOOKS like there are
>>>> native ports of GIT to MINGW, but I have no idea how complete and usable
>>>> they are.  If someone who has a Windows system could look into that it
>>>> would be a big help.
>>> I think the best thing to do is to ask directly on the Git ML.
>>>
>>> Someone already pointed out that he'd like to use Git on Windows but 
>>> doesn't want to install either Cygwin or MSYS.  Is this possible, or 
>>> will it be possible in the near future?
>> It is sort of possible. Without cygwin he'll be in the black for the few 
>> features that are still implemented as shell-scripts, but perhaps he/she 
>> will then be inclined to help us migrate those scripts to C builtins.
> 
> Umm.  There are quite a few shell scripts still _necessary_ to run git: 
> git-commit, git-fetch and git-merge being the most prominent ones.  The 
> first two are in the process of being rewritten _right_ _now_, but no 
> official git release has them yet.
> 

Ah, right. I think of "accepted into git.git" as being released.

> And I have to disagree strongly with the "black": In msysGit (which brings 
> its own minimal version of MSys), it is very smooth.
> 

Oh? I didn't know that. Windows and its unixifying toolboxes is unknown
territory to me, as I happily spend all my time on various unices.

>>>  Is it possible to use one of the various GUIs (git-gui, gitk, qgit) 
>>> on Windows without requiring a POSIXish shell etc.?
>>>
>> qgit is possible to use natively, if one installs the qgit4 libraries 
>> for windows, but it's more of a viewer than an action gui. git-gui and 
>> gitk are usable if you have the windows TCL port. I haven't tried it, 
>> but there are installers available, so testing it out (with all 
>> dependencies) shouldn't take too long.
> 
> FWIW msysGit comes with Tcl.  You can run git gui and gitk without any 
> hassles.
> 

Yes, my phrasing there was a bit obscure. I meant that all dependencies
are installed by the installer package.

>>>  (if Git is available as a library, and if this library works on 
>>> Windows, it will greatly help truly native Windows ports).
>> Yup. I believe the primary reason for libification is to easier support 
>> both porting and fully-fledged gui's.
> 
> Why?
> 
> I do not see any reason why libification helps the user experience on 
> Windows.
> 

I was under the impression that the windows port suffers from Windows'
lack of a proper fork() and friends and that a proper library would
help solving those problems. Perhaps I was misinformed.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Sun, 14 Oct 2007, Andreas Ericsson wrote:

> Johannes Schindelin wrote:
>
> > I do not see any reason why libification helps the user experience on 
> > Windows.
> 
> I was under the impression that the windows port suffers from Windows' 
> lack of a proper fork() and friends and that a proper library would help 
> solving those problems. Perhaps I was misinformed.

It suffered.  Until Hannes Sixt did a very fine job which cumulated in the 
patch series he posted yesterday.  Of course, this work is the reason 
msysGit is functional.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Alex Riesen]

Johannes Schindelin, Sun, Oct 14, 2007 22:14:25 +0200:
> On Sun, 14 Oct 2007, Andreas Ericsson wrote:
> > Johannes Schindelin wrote:
> >
> > > I do not see any reason why libification helps the user experience on 
> > > Windows.
> > 
> > I was under the impression that the windows port suffers from Windows' 
> > lack of a proper fork() and friends and that a proper library would help 
> > solving those problems. Perhaps I was misinformed.
> 
> It suffered.  Until Hannes Sixt did a very fine job which cumulated in the 
> patch series he posted yesterday.  Of course, this work is the reason 
> msysGit is functional.
> 

Re "functional". Have to remind something (besides the fork):

Filesystem:

- no proper VFS (can't do anything with files opened elsewhere, and we
  have not enough error handling and diagnostic output to detect the
  problems)

- no proper filename semantics (case-insensitivity and stupid rules for
  allowed characters in filenames, like ":" in filenames in
  cross-platform projects)

- no acceptable level of performance in filesystem and VFS (readdir,
  stat, open and read/write are annoyingly slow)

- it is the only OS in the world with multi-root (/a/b/c and /a/b/c
  can be not the same, depending on what current "drive" is) and
  multi-cwd, which hasn't had formed itself into a problem yet, but
  surely will

- no real "mmap" (which kills perfomance and complicates code)

Interprocess communication:

- no reliable text environment (I'm programming in the damn thing for
  10 years and I still don't know how to pass an environment variable
  _for_sure_)

- it has only one argument (limited in size) passed to started
  programs, which means that there is no possible way to safely pass
  file and text arguments on command line (more than one, that is)

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 00:14:46 +0200
> From: Alex Riesen <raa.lkml@gmail.com>
> Cc: Andreas Ericsson <ae@op5.se>, Benoit SIGOURE <tsuna@lrde.epita.fr>,
> 	git list <git@vger.kernel.org>, Eli Zaretskii <eliz@gnu.org>,
> 	Make Windows <make-w32@gnu.org>
> 
> Re "functional". Have to remind something (besides the fork):

That's a 20-20 hindsight: if you deliberately write a program to rely
heavily on Posix-isms, don't be surprised when you discover that it
cannot be easily ported to other platforms.

> - no proper VFS

I'm not sure what you are talking about.  What VFS do you use on
GNU/Linux that cannot work on Windows, and why do you use it?

> - no proper filename semantics (case-insensitivity and stupid rules for
>   allowed characters in filenames, like ":" in filenames in
>   cross-platform projects)

There's a flag on Windows to open files case-sensitively, if you need
that.  In any case, I don't see how this can be of any real relevance
to porting GIT.  As for ":" in file names, simply don't use it, like
you don't use white space or characters below 32 decimal: it's
inconvenient, even if it's allowed.

> - no acceptable level of performance in filesystem and VFS (readdir,
>   stat, open and read/write are annoyingly slow)

With what libraries?  Native `stat' and `readdir' are quite fast.
Perhaps you mean the ported glibc (libgw32c), where `readdir' is
indeed painfully slow, but then you don't need to use it.

> - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
>   can be not the same, depending on what current "drive" is)

So what? on Unix "a/b/c" can be not the same.  Both cases are simply
not complete file names, that's all.  No one said there must be a
single root for all volumes, it's the Posix jingoism creeping in
again.

>   and multi-cwd

No longer a problem on Windows versions since 2000.

> - no real "mmap" (which kills perfomance and complicates code)

You only need mmap because you are accustomed to use it on GNU/Linux.

> Interprocess communication:
> 
> - no reliable text environment (I'm programming in the damn thing for
>   10 years and I still don't know how to pass an environment variable
>   _for_sure_)
> 
> - it has only one argument (limited in size) passed to started
>   programs, which means that there is no possible way to safely pass
>   file and text arguments on command line (more than one, that is)

Not enough context, so I cannot talk intelligently about this.  Why do
you need interprocess communication in the first place? why not simply
give birth to a subsidiary process and pass it a command line (which
can be up to 32KB)?

RE: Switching from CVS to GIT

[Dave Korn]

On 14 October 2007 23:15, Alex Riesen wrote:

> Interprocess communication:
> 
> - no reliable text environment (I'm programming in the damn thing for
>   10 years and I still don't know how to pass an environment variable
>   _for_sure_)
> 
> - it has only one argument (limited in size) passed to started
>   programs, which means that there is no possible way to safely pass
>   file and text arguments on command line (more than one, that is)

  Whuh?

http://msdn2.microsoft.com/en-us/library/y5zz48s1(VS.80).aspx


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Eli Zaretskii wrote:

> Alex Riesen said:
> 
> > - no proper VFS
> 
> I'm not sure what you are talking about.  What VFS do you use on 
> GNU/Linux that cannot work on Windows, and why do you use it?

The problem is that on Windows, you cannot keep a file open and delete it 
at the same time.  This is an issue in Windows' equivalent of VFS.

A neat trick to work with temporary files without permission issues is to 
open the file and delete it right after that.  This does not work on 
Windows.

> > - no proper filename semantics (case-insensitivity and stupid rules for
> >   allowed characters in filenames, like ":" in filenames in
> >   cross-platform projects)
> 
> There's a flag on Windows to open files case-sensitively, if you need
> that.

The problem is not so much opening, but determining if an existing file 
and a file in the index have the same name.

For example, "README" in the index, but "readme" in the working directory, 
will be handled as "deleted/untracked" by the current machinery.  IOW git 
will not know that what it gets from readdir() as "readme" really is the 
same file as "README" in the index.

> > - no acceptable level of performance in filesystem and VFS (readdir,
> >   stat, open and read/write are annoyingly slow)
> 
> With what libraries?  Native `stat' and `readdir' are quite fast. 
> Perhaps you mean the ported glibc (libgw32c), where `readdir' is indeed 
> painfully slow, but then you don't need to use it.

No, native.

Once you experienced the performance of git on Linux, then rebooted into 
Windows on the same box, you will grow a beard while waiting for trivial 
operations.

Sure, git kicks ass on Windows, but only as compared to other programs _on 
Windows_.

> > - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
> >   can be not the same, depending on what current "drive" is)
> 
> So what? on Unix "a/b/c" can be not the same.  Both cases are simply not 
> complete file names, that's all.  No one said there must be a single 
> root for all volumes, it's the Posix jingoism creeping in again.

I think Alex means this: you can have C:\a\b\c and D:\a\b\c.  So depending 
on which drive you are, you mean one or the other.  Just comparing the 
paths is not enough.

> > - no real "mmap" (which kills perfomance and complicates code)
> 
> You only need mmap because you are accustomed to use it on GNU/Linux.

Yes.  And we rely on the performance very much.

Hth,
Dscho

Re: Switching from CVS to GIT

[Andreas Ericsson]

Eli Zaretskii wrote:
>> Date: Mon, 15 Oct 2007 00:14:46 +0200
>> From: Alex Riesen <raa.lkml@gmail.com>
>> Cc: Andreas Ericsson <ae@op5.se>, Benoit SIGOURE <tsuna@lrde.epita.fr>,
>> 	git list <git@vger.kernel.org>, Eli Zaretskii <eliz@gnu.org>,
>> 	Make Windows <make-w32@gnu.org>
>>
>> Re "functional". Have to remind something (besides the fork):
> 
> That's a 20-20 hindsight: if you deliberately write a program to rely
> heavily on Posix-isms, don't be surprised when you discover that it
> cannot be easily ported to other platforms.
> 

True. It was originally developed because Linux kernel development came
to a stand-still and needed an scm quickly. Since the original design
worked out nicely, nobody bothered (then) about possible future porting
issues. Windows is still a second class citizen, but that's true for
pretty much every unix-born application out there, so I'm not all that
stressed out about it.

> 
>> - no proper filename semantics (case-insensitivity and stupid rules for
>>   allowed characters in filenames, like ":" in filenames in
>>   cross-platform projects)
> 
> There's a flag on Windows to open files case-sensitively, if you need
> that.  In any case, I don't see how this can be of any real relevance
> to porting GIT.


Because having

	Path/foo
	path/Foo
	PATH
	path/foo

is possible in git's native playground, but not on windows, so it can
quite seriously hamper cross-platform cooperation. When that happens,
users usually start blaming the tools in use. Browse the list archives
for HFS and you'll see what I mean, although come to think of it, the
HFS problems might actually be worse, since HFS reports case-changes
while not actually being case-sensitive.


>  As for ":" in file names, simply don't use it, like
> you don't use white space or characters below 32 decimal: it's
> inconvenient, even if it's allowed.
> 

It's still a real problem because sooner or later someone will use that,
and it needs to be handled with a bit more grace than just bombing out.

> 
>> - no real "mmap" (which kills perfomance and complicates code)
> 
> You only need mmap because you are accustomed to use it on GNU/Linux.
> 

Not really. mmap() provides a real performance boost when reading large
repos, due to the sliding window code that handles pack-files. mmap
was invented for occasions like that, and was allowed to endure because
it was a much better solution than simply read(fd, buf, st.st_size) and
moving pointers around.


>> Interprocess communication:
>>
>> - no reliable text environment (I'm programming in the damn thing for
>>   10 years and I still don't know how to pass an environment variable
>>   _for_sure_)
>>
>> - it has only one argument (limited in size) passed to started
>>   programs, which means that there is no possible way to safely pass
>>   file and text arguments on command line (more than one, that is)
> 
> Not enough context, so I cannot talk intelligently about this.  Why do
> you need interprocess communication in the first place?


Because some of the commands operate on large data-sets that are best
passed as a stream. It's ridiculously easy to set that up on unix, but
(afaiu) quite troublesome under windows.


> why not simply
> give birth to a subsidiary process and pass it a command line (which
> can be up to 32KB)?

I believe work is in progress that will run things as threads rather
than using fork()+execve(). 32KiB of data is nowhere near enough to
sustain many of the more data-hungry commands. Or rather, it won't be
once the repository has grown passed 50-odd revisions.


All that being said, welcome to the git mailing list. Hopefully you
can help iron out the wrinkles on windows. You seem to have a fairly
good grasp of what's available there, and I'm sure the msys team would
be pretty happy to get a few patches to speed them on their way.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

RE: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Sun, 14 Oct 2007, Dave Korn wrote:

> On 14 October 2007 23:15, Alex Riesen wrote:
> 
> > Interprocess communication:
> > 
> > - no reliable text environment (I'm programming in the damn thing for
> >   10 years and I still don't know how to pass an environment variable
> >   _for_sure_)
> > 
> > - it has only one argument (limited in size) passed to started
> >   programs, which means that there is no possible way to safely pass
> >   file and text arguments on command line (more than one, that is)
> 
>   Whuh?
> 
> http://msdn2.microsoft.com/en-us/library/y5zz48s1(VS.80).aspx

It does have an exec() call, yes, since that is required by the C 
standard.  But internally, it converts that into one single command line.

In corner cases, you find problems with that.

Hth,
Dscho

Re: Switching from CVS to GIT

[David Brown]

On Sun, Oct 14, 2007 at 11:59:35PM +0100, Dave Korn wrote:
>On 14 October 2007 23:15, Alex Riesen wrote:
>
>> Interprocess communication:
>> 
>> - it has only one argument (limited in size) passed to started
>>   programs, which means that there is no possible way to safely pass
>>   file and text arguments on command line (more than one, that is)
>
>  Whuh?
>
>http://msdn2.microsoft.com/en-us/library/y5zz48s1(VS.80).aspx

The MS exec* calls just concatenate all of the argv arguments, separating
them with a space into a single buffer.

Look at the general _exec* page:

   http://msdn2.microsoft.com/en-us/library/431x4c1w(VS.80).aspx

and read the first "Note" section.

If you know what the library on the other end is doing to re-parse the
arguments back into separate strings, it might be possible to quote things
enough to handle names with spaces, but it is hard.

David

Re: Switching from CVS to GIT

[Brian Dessent]

Johannes Schindelin wrote:

> The problem is that on Windows, you cannot keep a file open and delete it
> at the same time.  This is an issue in Windows' equivalent of VFS.
> 
> A neat trick to work with temporary files without permission issues is to
> open the file and delete it right after that.  This does not work on
> Windows.

You can achieve the same thing on Windows with CreateFile() by setting
the dwShareMode parameter to zero and setting the
FILE_FLAG_DELETE_ON_CLOSE attribute on dwFlagsAndAttributes.  This
results in a file that cannot be opened or read by any other process and
that will be automatically deleted when all open handles are closed.

> I think Alex means this: you can have C:\a\b\c and D:\a\b\c.  So depending
> on which drive you are, you mean one or the other.  Just comparing the
> paths is not enough.

This just means that you have to consider the drive letter as part of
the filename.

> > > - no real "mmap" (which kills perfomance and complicates code)
> >
> > You only need mmap because you are accustomed to use it on GNU/Linux.
> 
> Yes.  And we rely on the performance very much.

Windows may not call it mmap() but it most certainly has memory-mapped
file IO:
<http://msdn2.microsoft.com/en-us/library/aa366781.aspx#file_mapping_functions>.

Brian

Re: Switching from CVS to GIT

[Michael Gebetsroither]

["Followup-To:" header set to gmane.comp.version-control.git.]

> - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
>   can be not the same, depending on what current "drive" is) and
>   multi-cwd, which hasn't had formed itself into a problem yet, but
>   surely will

Thats true for linux too.
/a/b/c and /a/b/c can be 2 totally different files depending on the vfs
namespace you are one.

cu,
michael
-- 
It's already too late!

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Sun, 14 Oct 2007, Brian Dessent wrote:

> Johannes Schindelin wrote:
> 
> > The problem is that on Windows, you cannot keep a file open and delete 
> > it at the same time.  This is an issue in Windows' equivalent of VFS.
> > 
> > A neat trick to work with temporary files without permission issues is 
> > to open the file and delete it right after that.  This does not work 
> > on Windows.
> 
> You can achieve the same thing on Windows with CreateFile() by setting 
> the dwShareMode parameter to zero and setting the 
> FILE_FLAG_DELETE_ON_CLOSE attribute on dwFlagsAndAttributes.  This 
> results in a file that cannot be opened or read by any other process and 
> that will be automatically deleted when all open handles are closed.

Aha.  So to support Windows, we have to wrap all sites that use that 
trick, and special case that #ifdef __MINGW32__. 

> > I think Alex means this: you can have C:\a\b\c and D:\a\b\c.  So 
> > depending on which drive you are, you mean one or the other.  Just 
> > comparing the paths is not enough.
> 
> This just means that you have to consider the drive letter as part of 
> the filename.

So to support Windows, we have to special case having a ":" as second 
character in the filename.

> > > > - no real "mmap" (which kills perfomance and complicates code)
> > >
> > > You only need mmap because you are accustomed to use it on GNU/Linux.
> > 
> > Yes.  And we rely on the performance very much.
> 
> Windows may not call it mmap() but it most certainly has memory-mapped
> file IO:
> <http://msdn2.microsoft.com/en-us/library/aa366781.aspx#file_mapping_functions>.

Yes, but there are still incompatibilities with POSIX.  Again, when you 
did not close the file, you cannot delete (or rename) it.  So, to support 
Windows, ...

All this supporting Windows business is certainly possible, if tedious.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Johannes Schindelin wrote:

> All this supporting Windows business is certainly possible, if tedious.

To clarify: git works on Windows.  Most of the time, that is.  But all 
those changes that were necessary to go there have not yet found their way 
into the official git.git repository.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 00:45:47 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Alex Riesen <raa.lkml@gmail.com>, ae@op5.se, tsuna@lrde.epita.fr, 
>     git@vger.kernel.org, make-w32@gnu.org
> 
> The problem is that on Windows, you cannot keep a file open and delete it 
> at the same time.

That is no longer true, for quite some time.  NT4 and later versions
support that almost exactly like Posix filesystems.

> > > - no acceptable level of performance in filesystem and VFS (readdir,
> > >   stat, open and read/write are annoyingly slow)
> > 
> > With what libraries?  Native `stat' and `readdir' are quite fast. 
> > Perhaps you mean the ported glibc (libgw32c), where `readdir' is indeed 
> > painfully slow, but then you don't need to use it.
> 
> No, native.
> 
> Once you experienced the performance of git on Linux, then rebooted into 
> Windows on the same box, you will grow a beard while waiting for trivial 
> operations.

Maybe GIT assumes too much about `readdir' and `stat', and should
refactor its code into better abstractions.

> > > - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
> > >   can be not the same, depending on what current "drive" is)
> > 
> > So what? on Unix "a/b/c" can be not the same.  Both cases are simply not 
> > complete file names, that's all.  No one said there must be a single 
> > root for all volumes, it's the Posix jingoism creeping in again.
> 
> I think Alex means this: you can have C:\a\b\c and D:\a\b\c.  So depending 
> on which drive you are, you mean one or the other.  Just comparing the 
> paths is not enough.

What _I_ meant is that the C: part is part of the full file name,
exactly like the leading / is on Unix.

> > > - No real "mmap" (which kills perfomance and complicates code)
> > 
> > You only need mmap because you are accustomed to use it on GNU/Linux.
> 
> Yes.  And we rely on the performance very much.

There's no need for mmap to get memory performance, except if sbrk and
friends are too slow.

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 02:22:53 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Eli Zaretskii <eliz@gnu.org>, Alex Riesen <raa.lkml@gmail.com>, ae@op5.se, 
>     tsuna@lrde.epita.fr, make-w32@gnu.org
> 
> > You can achieve the same thing on Windows with CreateFile() by setting 
> > the dwShareMode parameter to zero and setting the 
> > FILE_FLAG_DELETE_ON_CLOSE attribute on dwFlagsAndAttributes.  This 
> > results in a file that cannot be opened or read by any other process and 
> > that will be automatically deleted when all open handles are closed.
> 
> Aha.  So to support Windows, we have to wrap all sites that use that 
> trick, and special case that #ifdef __MINGW32__. 

No, you need to think in abstractions rather than POSIX-isms, and then
let each platform implement those abstractions as appropriate.

> > > I think Alex means this: you can have C:\a\b\c and D:\a\b\c.  So 
> > > depending on which drive you are, you mean one or the other.  Just 
> > > comparing the paths is not enough.
> > 
> > This just means that you have to consider the drive letter as part of 
> > the filename.
> 
> So to support Windows, we have to special case having a ":" as second 
> character in the filename.

No, you need to think abstractions like `absolute_file_name' and
`dir_separator'.

> > > > > - no real "mmap" (which kills perfomance and complicates code)
> > > >
> > > > You only need mmap because you are accustomed to use it on GNU/Linux.
> > > 
> > > Yes.  And we rely on the performance very much.
> > 
> > Windows may not call it mmap() but it most certainly has memory-mapped
> > file IO:
> > <http://msdn2.microsoft.com/en-us/library/aa366781.aspx#file_mapping_functions>.
> 
> Yes, but there are still incompatibilities with POSIX.

Stop thinking POSIX.  Think abstractions that are common to POSIX and
non-POSIX systems.  If you think POSIX, don't be surprised that it
won't port.

> Again, when you did not close the file, you cannot delete (or
> rename) it.

Yes, you can, nowadays.  But that doesn't mean it was TRT to use such
dirty tricks to implement temporary files or security.  One needs to
think in abstractions again, and leave the implementation to each
platform.

> All this supporting Windows business is certainly possible, if tedious.

Only if the program was written with disregard to anything but POSIX.

Re: Switching from CVS to GIT

[Martin Langhoff]

On 10/15/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> There is msysGit.  This project is nearing to its first beta, being
> self-hosted since mid-August IIRC.

I've been using it recently, I have to say it's pretty impressive -
you can use it from cmd.com or from a bash window (courtesy of the
msys environment included). The GUIs that ship with git are there
(git-gui, gitk).

I use gitk extensively, and it works *great*. My work-style is of a
shell window for status/diff/commit actions and one or more gitk
windows for browsing of proj history. You can use git-gui for a visual
status/git/commit workflow, or qgit. qgit is more integrated, and
might feel more "at home" for users that expect something more
MDI-ish.

cheers.


martin

Re: Switching from CVS to GIT

[Martin Langhoff]

On 10/15/07, Andreas Ericsson <ae@op5.se> wrote:
> > And I have to disagree strongly with the "black": In msysGit (which brings
> > its own minimal version of MSys), it is very smooth.
> >
>
> Oh? I didn't know that. Windows and its unixifying toolboxes is unknown
> territory to me, as I happily spend all my time on various unices.

I'm a unix-head too. Last couple of weeks had to work on a windows
server, and installed msysGit. Very impressed - all the needed
dependencies are there, from an end-user POV it "just works".

> I was under the impression that the windows port suffers from Windows'
> lack of a proper fork() and friends and that a proper library would
> help solving those problems. Perhaps I was misinformed.

I think msys' DLLs might be doing what cygwin does, an emulated fork.

A quite surprising thing is that msysgit manages to be very fast. Not
as fast as the same git, same hw running on a recent Linux, but pretty
usable fast for a tree with a few thousand files. Earlier/other git
ports to win32 are pretty slow (still faster than svn and friends, but
slooow).

cheers,


m

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 00:45:47 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Alex Riesen <raa.lkml@gmail.com>, ae@op5.se, tsuna@lrde.epita.fr, 
>     git@vger.kernel.org, make-w32@gnu.org
> 
> The problem is not so much opening, but determining if an existing file 
> and a file in the index have the same name.
> 
> For example, "README" in the index, but "readme" in the working directory, 
> will be handled as "deleted/untracked" by the current machinery.  IOW git 
> will not know that what it gets from readdir() as "readme" really is the 
> same file as "README" in the index.

That's because you think file names are simple strings and can be
compared by simple string comparison.  This naìve view is not true
even on POSIX systems: "foo/bar" and "/a/b/foo/bar" can be the same
file, as well as "/a/b/c/d" and "/x/y/z", given the right symlinks.
But for some reason that eludes me, people who are accustomed to POSIX
stop right there and in effect say "file names are strings, if we only
make them absolute and resolve links".  Instead, recognize that file
names are not strings (although they inherit some of the strings'
traits), and think in terms of "file-name comparison" abstraction;
then everything will fall in place just fine.

> > > - no acceptable level of performance in filesystem and VFS (readdir,
> > >   stat, open and read/write are annoyingly slow)
> > 
> > With what libraries?  Native `stat' and `readdir' are quite fast. 
> > Perhaps you mean the ported glibc (libgw32c), where `readdir' is indeed 
> > painfully slow, but then you don't need to use it.
> 
> No, native.

Can you show a test case where this penalty is clearly visible?  I'm
curious to see the numbers.  TIA

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 02:24:53 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Eli Zaretskii <eliz@gnu.org>, Alex Riesen <raa.lkml@gmail.com>, ae@op5.se, 
>     tsuna@lrde.epita.fr, make-w32@gnu.org
> 
> To clarify: git works on Windows.  Most of the time, that is.  But all 
> those changes that were necessary to go there have not yet found their way 
> into the official git.git repository.

I, for one, appreciate all the hard work invested in that.

While we are at that: can you (or someone else) point me to
instructions on how to build the MinGW port of GIT?  I found a tarball
of the MinGW-ported GIT (v1.5.3, I think), but what I don't seem to be
able to find is some kind of HOWTO: what tools I need to have
installed, how to configure them (if there are any special issues
there), what command(s) to type, etc.  Is there anything like that out
there, or can someone post such instructions?

TIA

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Sun, 14 Oct 2007 17:03:47 -0700
> From: David Brown <git@davidb.org>
> Cc: 'Andreas Ericsson' <ae@op5.se>, 'Alex Riesen' <raa.lkml@gmail.com>,
> 	'git list' <git@vger.kernel.org>,
> 	'Johannes Schindelin' <Johannes.Schindelin@gmx.de>,
> 	'Make Windows' <make-w32@gnu.org>
> 
> The MS exec* calls just concatenate all of the argv arguments, separating
> them with a space into a single buffer.

True.

> If you know what the library on the other end is doing to re-parse the
> arguments back into separate strings, it might be possible to quote things
> enough to handle names with spaces, but it is hard.

It's not hard, it's just a bit of work.  And it needs to be done
exactly once.

Re: Switching from CVS to GIT

[Johannes Sixt]

Benoit SIGOURE schrieb:
> Context: GNU make seems to be willing to switch from CVS to ... 
> something else.
> 
> On Oct 14, 2007, at 6:57 PM, Paul Smith wrote:
> 
>> [...] the big thing no one else seems to have addressed much in
>> other discussions I've seen is portability.  It LOOKS like there are
>> native ports of GIT to MINGW, but I have no idea how complete and usable
>> they are.  If someone who has a Windows system could look into that it
>> would be a big help.
> 
> I think the best thing to do is to ask directly on the Git ML.
> 
> Someone already pointed out that he'd like to use Git on Windows but 
> doesn't want to install either Cygwin or MSYS.  Is this possible, or 
> will it be possible in the near future?  Is it possible to use one of 
> the various GUIs (git-gui, gitk, qgit) on Windows without requiring a 
> POSIXish shell etc.?

FWIW, I'm using the MinGW port from cmd.exe, i.e. not from a posix shell, on 
a *production* repository. gitk and git-gui work. Not all operations that I 
regularly use are available[*] via the GUIs, like git-rebase or 
non-fast-forwarding push, so the use of the command line is needed from time 
to time.

Unfortunately, "Fetch" does not yet work[*] from within git-gui, so you have 
to fall back to git-fetch on the command line.

Of course, the Posix toolset, including a shell, must still be installed 
(and in my setup they are in the PATH), but you don't have to use it.

[*] Note the distinction between "not available" and "does not work".

-- Hannes

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 15, 2007, at 8:04 AM, Eli Zaretskii wrote:

>> Date: Mon, 15 Oct 2007 02:24:53 +0100 (BST)
>> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
>> cc: Eli Zaretskii <eliz@gnu.org>, Alex Riesen  
>> <raa.lkml@gmail.com>, ae@op5.se,
>>     tsuna@lrde.epita.fr, make-w32@gnu.org
>>
>> To clarify: git works on Windows.  Most of the time, that is.  But  
>> all
>> those changes that were necessary to go there have not yet found  
>> their way
>> into the official git.git repository.
>
> I, for one, appreciate all the hard work invested in that.
>
> While we are at that: can you (or someone else) point me to
> instructions on how to build the MinGW port of GIT?  I found a tarball
> of the MinGW-ported GIT (v1.5.3, I think), but what I don't seem to be
> able to find is some kind of HOWTO: what tools I need to have
> installed, how to configure them (if there are any special issues
> there), what command(s) to type, etc.  Is there anything like that out
> there, or can someone post such instructions?

If you want to have a full working development environment, such that
you can start contributing to msysgit right away, and have no firewall
issues, go to

	http://code.google.com/p/msysgit/

and install GitMe, currently

	http://msysgit.googlecode.com/files/GitMe-0.4.2.exe

If you only care about an end-user setup, which contains only the git
binaries on your system, but no tools to compile them, stay tuned for
one or two days. We'll release an updated installer soon.

	Steffen

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>, git@vger.kernel.org,
>         raa.lkml@gmail.com, ae@op5.se, tsuna@lrde.epita.fr, make-w32@gnu.org
> From: Steffen Prohaska <prohaska@zib.de>
> Date: Mon, 15 Oct 2007 09:56:40 +0200
> 
> > While we are at that: can you (or someone else) point me to
> > instructions on how to build the MinGW port of GIT?  I found a tarball
> > of the MinGW-ported GIT (v1.5.3, I think), but what I don't seem to be
> > able to find is some kind of HOWTO: what tools I need to have
> > installed, how to configure them (if there are any special issues
> > there), what command(s) to type, etc.  Is there anything like that out
> > there, or can someone post such instructions?
> 
> If you want to have a full working development environment, such that
> you can start contributing to msysgit right away, and have no firewall
> issues, go to
> 
> 	http://code.google.com/p/msysgit/
> 
> and install GitMe, currently
> 
> 	http://msysgit.googlecode.com/files/GitMe-0.4.2.exe
> 
> If you only care about an end-user setup, which contains only the git
> binaries on your system, but no tools to compile them, stay tuned for
> one or two days. We'll release an updated installer soon.

Sorry I wasn't clear: I want neither.  I don't think I will have
enough free time to become an active contributor to GIT any time
soon.  OTOH, since binaries are not available (and I'd prefer a
tarball as opposed to an installer, to be more in control of what's
being installed and where), I asked about the development tools
(compiler and Binutils, obviously, but what else?) required to build
the source tarball with MinGW tools.

Do I understand correctly that building GIT currently requires MSYS?
That'd be unfortunate, at least for me.

Anyway, thanks for replying.

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Eli Zaretskii wrote:

> No, you need to think in abstractions rather than POSIX-isms, and then 
> let each platform implement those abstractions as appropriate.

Last time I checked, POSIX was already an abstraction, thankyouverymuch.

Anyway, this discussion gets out of hand.  The question was: does Git work 
on Windows natively, and the answer as far as you are concerned is: yes.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Eli Zaretskii wrote:

> > Date: Mon, 15 Oct 2007 00:45:47 +0100 (BST)
> > From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > cc: Alex Riesen <raa.lkml@gmail.com>, ae@op5.se, tsuna@lrde.epita.fr, 
> >     git@vger.kernel.org, make-w32@gnu.org
> > 
> > The problem is not so much opening, but determining if an existing file 
> > and a file in the index have the same name.
> > 
> > For example, "README" in the index, but "readme" in the working directory, 
> > will be handled as "deleted/untracked" by the current machinery.  IOW git 
> > will not know that what it gets from readdir() as "readme" really is the 
> > same file as "README" in the index.
> 
> That's because you think file names are simple strings and can be
> compared by simple string comparison.

Almost...

> This na?ve view is not true even on POSIX systems: "foo/bar" and 
> "/a/b/foo/bar" can be the same file, as well as "/a/b/c/d" and "/x/y/z", 
> given the right symlinks.

... not quite, ah ...

> But for some reason that eludes me, people who are accustomed to POSIX
> stop right there and in effect say "file names are strings, if we only
> make them absolute and resolve links".

... yes!  There you have it.  Absolute filenames, resolved by readlink() 
are assumed to be the unique (!) identifiers for the contents.

_Note:_ absolute paths _without_ readlink() resolving are _still_ unique 
identifiers; this time for files/symlinks.

Things like this utter rubbish that two different file names (which are 
the keys in the keystore that a filesystem really is) make Windows' 
filesystem operations so slow.

I wonder when Windows heads will realise that this "convenience" is just 
another reason why Windows is easily outperformed by other OSes (yes, the 
last one is a plural).

> > > > - no acceptable level of performance in filesystem and VFS 
> > > >   (readdir, stat, open and read/write are annoyingly slow)
> > > 
> > > With what libraries?  Native `stat' and `readdir' are quite fast. 
> > > Perhaps you mean the ported glibc (libgw32c), where `readdir' is 
> > > indeed painfully slow, but then you don't need to use it.
> > 
> > No, native.
> 
> Can you show a test case where this penalty is clearly visible?  I'm 
> curious to see the numbers.  TIA

No, I cannot.  I will not go and buy a copy of Windows just to show you 
the numbers.

Since quite some time I only run Linux on my machine(s), and the reason 
was a very unscientific experiment: I kept with the OS that did not freeze 
and let me do nothing for more than one second.

Now, that is my _personal_ decision.  If _you_ have no problem with 
Windows, just stick with it.  (I always thought this goes without saying, 
but Windows users tend to be very religious about this issue, thinking 
just because I hate Windows that I want to make them switch.  Hahaha, no.)

Ciao,
Dscho

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Eli Zaretskii wrote:

> Do I understand correctly that building GIT currently requires MSYS? 
> That'd be unfortunate, at least for me.

If you could make Git compile with Visual C++, that would be fabulous.

TIA,
Dscho

Re: Switching from CVS to GIT

[David Kastrup]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi,
>
> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
>
>> > Date: Mon, 15 Oct 2007 00:45:47 +0100 (BST)
>> > From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
>> > cc: Alex Riesen <raa.lkml@gmail.com>, ae@op5.se, tsuna@lrde.epita.fr, 
>> >     git@vger.kernel.org, make-w32@gnu.org
>> > 
>> > The problem is not so much opening, but determining if an existing file 
>> > and a file in the index have the same name.
>> > 
>> > For example, "README" in the index, but "readme" in the working directory, 
>> > will be handled as "deleted/untracked" by the current machinery.  IOW git 
>> > will not know that what it gets from readdir() as "readme" really is the 
>> > same file as "README" in the index.
>> 
>> That's because you think file names are simple strings and can be
>> compared by simple string comparison.
>
> Almost...
>
>> This na?ve view is not true even on POSIX systems: "foo/bar" and 
>> "/a/b/foo/bar" can be the same file, as well as "/a/b/c/d" and "/x/y/z", 
>> given the right symlinks.
>
> ... not quite, ah ...
>
>> But for some reason that eludes me, people who are accustomed to POSIX
>> stop right there and in effect say "file names are strings, if we only
>> make them absolute and resolve links".
>
> ... yes!  There you have it.  Absolute filenames, resolved by
> readlink() are assumed to be the unique (!) identifiers for the
> contents.

They aren't.  One can mount the same file system several times in
different places.  In Linux, one can even mount directories and files
to several places at once.  Most Unices also support some
case-insensitive file systems, and readlink does not canonicalize the
casing.

> _Note:_ absolute paths _without_ readlink() resolving are _still_
> unique identifiers; this time for files/symlinks.

Not even that.  A unique identifier for files would imply that
touching the file does not affect, say, the access times of files with
other unique identifiers.

-- 
David Kastrup

Re: Switching from CVS to GIT

[David Kastrup]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi,
>
> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
>
>> > Date: Mon, 15 Oct 2007 00:45:47 +0100 (BST)
>> > From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
>> > cc: Alex Riesen <raa.lkml@gmail.com>, ae@op5.se, tsuna@lrde.epita.fr, 
>> >     git@vger.kernel.org, make-w32@gnu.org
>> > 
>> > The problem is not so much opening, but determining if an existing file 
>> > and a file in the index have the same name.
>> > 
>> > For example, "README" in the index, but "readme" in the working directory, 
>> > will be handled as "deleted/untracked" by the current machinery.  IOW git 
>> > will not know that what it gets from readdir() as "readme" really is the 
>> > same file as "README" in the index.
>> 
>> That's because you think file names are simple strings and can be
>> compared by simple string comparison.
>
> Almost...
>
>> This na?ve view is not true even on POSIX systems: "foo/bar" and 
>> "/a/b/foo/bar" can be the same file, as well as "/a/b/c/d" and "/x/y/z", 
>> given the right symlinks.
>
> ... not quite, ah ...
>
>> But for some reason that eludes me, people who are accustomed to POSIX
>> stop right there and in effect say "file names are strings, if we only
>> make them absolute and resolve links".
>
> ... yes!  There you have it.  Absolute filenames, resolved by
> readlink() are assumed to be the unique (!) identifiers for the
> contents.

They aren't.  One can mount the same file system several times in
different places.  In Linux, one can even mount directories and files
to several places at once.  Most Unices also support some
case-insensitive file systems, and readlink does not canonicalize the
casing.

> _Note:_ absolute paths _without_ readlink() resolving are _still_
> unique identifiers; this time for files/symlinks.

Not even that.  A unique identifier for files would imply that
touching the file does not affect, say, the access times of files with
other unique identifiers.

-- 
David Kastrup

Re: Switching from CVS to GIT

[Benoit SIGOURE]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

On Oct 15, 2007, at 10:34 AM, Johannes Schindelin wrote:

> Hi,
>
> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
>
>> No, you need to think in abstractions rather than POSIX-isms, and  
>> then
>> let each platform implement those abstractions as appropriate.
>
> Last time I checked, POSIX was already an abstraction,  
> thankyouverymuch.

But as Eli pointed out, it's not universal, so you need higher  
abstractions on top of them.

> Anyway, this discussion gets out of hand.

Not at all, actually I think some interesting points were made,  
including on the technical side of the thing.

-- 
Benoit Sigoure aka Tsuna
EPITA Research and Development Laboratory



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 15, 2007, at 10:20 AM, Eli Zaretskii wrote:

>> Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>,  
>> git@vger.kernel.org,
>>         raa.lkml@gmail.com, ae@op5.se, tsuna@lrde.epita.fr, make- 
>> w32@gnu.org
>> From: Steffen Prohaska <prohaska@zib.de>
>> Date: Mon, 15 Oct 2007 09:56:40 +0200
>>
>>> While we are at that: can you (or someone else) point me to
>>> instructions on how to build the MinGW port of GIT?  I found a  
>>> tarball
>>> of the MinGW-ported GIT (v1.5.3, I think), but what I don't seem  
>>> to be
>>> able to find is some kind of HOWTO: what tools I need to have
>>> installed, how to configure them (if there are any special issues
>>> there), what command(s) to type, etc.  Is there anything like  
>>> that out
>>> there, or can someone post such instructions?
>>
>> If you want to have a full working development environment, such that
>> you can start contributing to msysgit right away, and have no  
>> firewall
>> issues, go to
>>
>> 	http://code.google.com/p/msysgit/
>>
>> and install GitMe, currently
>>
>> 	http://msysgit.googlecode.com/files/GitMe-0.4.2.exe
>>
>> If you only care about an end-user setup, which contains only the git
>> binaries on your system, but no tools to compile them, stay tuned for
>> one or two days. We'll release an updated installer soon.
>
> Sorry I wasn't clear: I want neither.  I don't think I will have
> enough free time to become an active contributor to GIT any time
> soon.  OTOH, since binaries are not available (and I'd prefer a
> tarball as opposed to an installer, to be more in control of what's
> being installed and where),

Ok, so I uploaded the most recent preview of the installer to

http://msysgit.googlecode.com/files/WinGit-1.5.3-preview20071010.exe

Note, we're about to release an updated version soon. Personally,
I don't plan to put work in providing tar balls. A working installer
has higher priority for me.


> I asked about the development tools
> (compiler and Binutils, obviously, but what else?) required to build
> the source tarball with MinGW tools.
>
> Do I understand correctly that building GIT currently requires MSYS?
> That'd be unfortunate, at least for me.

msysgit's GitMe contains all tools from MSYS required to build git.
It also clones the git source and compiles it. It doesn't install
anything outside the folder that you chose upon installation.

I strongly believe it is the easiest way to compile git from source.

	Steffen

Re: Switching from CVS to GIT

[Andreas Ericsson]

Eli Zaretskii wrote:
> 
>> If you know what the library on the other end is doing to re-parse the
>> arguments back into separate strings, it might be possible to quote things
>> enough to handle names with spaces, but it is hard.
> 
> It's not hard, it's just a bit of work.  And it needs to be done
> exactly once.

Before someone beats me to it: "Patches welcome" ;-)

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Switching from CVS to GIT

[Johannes Sixt]

Andreas Ericsson schrieb:
> Eli Zaretskii wrote:
>>
>>> If you know what the library on the other end is doing to re-parse the
>>> arguments back into separate strings, it might be possible to quote 
>>> things
>>> enough to handle names with spaces, but it is hard.
>>
>> It's not hard, it's just a bit of work.  And it needs to be done
>> exactly once.
> 
> Before someone beats me to it: "Patches welcome" ;-)

Let others do the research for you, hm?

http://repo.or.cz/w/git/mingw.git?a=blob;f=compat/mingw.c;h=2554f19765da5709b787e873da225c59f9d22bb7;hb=HEAD#l306

-- Hannes

Re: Switching from CVS to GIT

[Andreas Ericsson]

Johannes Sixt wrote:
> Andreas Ericsson schrieb:
>> Eli Zaretskii wrote:
>>>
>>>> If you know what the library on the other end is doing to re-parse the
>>>> arguments back into separate strings, it might be possible to quote 
>>>> things
>>>> enough to handle names with spaces, but it is hard.
>>>
>>> It's not hard, it's just a bit of work.  And it needs to be done
>>> exactly once.
>>
>> Before someone beats me to it: "Patches welcome" ;-)
> 
> Let others do the research for you, hm?
> 
> http://repo.or.cz/w/git/mingw.git?a=blob;f=compat/mingw.c;h=2554f19765da5709b787e873da225c59f9d22bb7;hb=HEAD#l306 
> 

Yup. Although it was more in the nature of "whoever wrote it surely knows
he/she did it and where to find the patch", so I expect this wasn't much
of a timesink for you. My apologies if I was incorrect.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Cc: Johannes.Schindelin@gmx.de, git@vger.kernel.org, raa.lkml@gmail.com,
>         ae@op5.se, tsuna@lrde.epita.fr, make-w32@gnu.org
> From: Steffen Prohaska <prohaska@zib.de>
> Date: Mon, 15 Oct 2007 11:23:37 +0200
> 
> Ok, so I uploaded the most recent preview of the installer to
> 
> http://msysgit.googlecode.com/files/WinGit-1.5.3-preview20071010.exe

Thanks!

> Note, we're about to release an updated version soon. Personally,
> I don't plan to put work in providing tar balls. A working installer
> has higher priority for me.

Fair enough.

> msysgit's GitMe contains all tools from MSYS required to build git.
> It also clones the git source and compiles it. It doesn't install
> anything outside the folder that you chose upon installation.
> 
> I strongly believe it is the easiest way to compile git from source.

Okay, thanks a lot for the info.

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 09:47:25 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Steffen Prohaska <prohaska@zib.de>, git@vger.kernel.org, 
>     raa.lkml@gmail.com, ae@op5.se, tsuna@lrde.epita.fr, make-w32@gnu.org
> 
> If you could make Git compile with Visual C++, that would be fabulous.

I prefer GCC (the MinGW port), but without the MSYS ports of
additional tools.  I use the GnuWin32 ports augmented by some of my
own (where GnuWin32 ports are buggy or terribly slow).

RE: Switching from CVS to GIT

[Dave Korn]

On 15 October 2007 11:38, Johannes Sixt wrote:

> http://repo.or.cz/w/git/mingw.git?a=blob;f=compat/mingw.c;h=2554f19765da5709b787e873da225c59f9d22bb7;hb=HEAD#l306
> 
> -- Hannes


 291                         /* Thanks, Bill. You'll burn in hell for that. */


  ;-)


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: Switching from CVS to GIT

[Johannes Sixt]

Eli Zaretskii schrieb:
>> Date: Mon, 15 Oct 2007 09:47:25 +0100 (BST)
>> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
>> cc: Steffen Prohaska <prohaska@zib.de>, git@vger.kernel.org, 
>>     raa.lkml@gmail.com, ae@op5.se, tsuna@lrde.epita.fr, make-w32@gnu.org
>>
>> If you could make Git compile with Visual C++, that would be fabulous.
> 
> I prefer GCC (the MinGW port), but without the MSYS ports of
> additional tools.  I use the GnuWin32 ports augmented by some of my
> own (where GnuWin32 ports are buggy or terribly slow).

They should work, too. If a tool is missing, ought to notice it soon enough.

These are important to note, though:

- The tools must not do their own LF->CRLF conversion when they are used in 
a pipeline, "just because they know it better".

- GNU tar is needed (in the cpio emulator).

- ln must be able to create hard links on NTFS or do the equivalent of
cp -p

- GNU cp -al will be needed and should create hard links on NTFS. (I plan to 
use it for local clones in place of cpio -pl.)

Any feedback on how git works for you with these tools is appreciated.

-- Hannes

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 14:31:01 +0200
> From: Johannes Sixt <j.sixt@viscovery.net>
> Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>,
> 	prohaska@zib.de, make-w32@gnu.org, raa.lkml@gmail.com, ae@op5.se,
> 	git@vger.kernel.org
> 
> > I prefer GCC (the MinGW port), but without the MSYS ports of
> > additional tools.  I use the GnuWin32 ports augmented by some of my
> > own (where GnuWin32 ports are buggy or terribly slow).
> 
> They should work, too. If a tool is missing, ought to notice it soon enough.
> 
> These are important to note, though:
> 
> - The tools must not do their own LF->CRLF conversion when they are used in 
> a pipeline, "just because they know it better".
> 
> - GNU tar is needed (in the cpio emulator).
> 
> - ln must be able to create hard links on NTFS or do the equivalent of
> cp -p
> 
> - GNU cp -al will be needed and should create hard links on NTFS. (I plan to 
> use it for local clones in place of cpio -pl.)
> 
> Any feedback on how git works for you with these tools is appreciated.

Thanks, I will try.

Re: Switching from CVS to GIT

[Alex Riesen]

Johannes Schindelin, Mon, Oct 15, 2007 02:01:00 +0200:
> On Sun, 14 Oct 2007, Dave Korn wrote:
> > On 14 October 2007 23:15, Alex Riesen wrote:
> > > - it has only one argument (limited in size) passed to started
> > >   programs, which means that there is no possible way to safely pass
> > >   file and text arguments on command line (more than one, that is)
> > 
> >   Whuh?
> > 
> > http://msdn2.microsoft.com/en-us/library/y5zz48s1(VS.80).aspx
> 
> It does have an exec() call, yes, since that is required by the C 
> standard.  But internally, it converts that into one single command line.
> 
> In corner cases, you find problems with that.
> 

Like: "damn, it is just IMPOSSIBLE to implement without them corner
cases."

Re: Switching from CVS to GIT

[Alex Riesen]

Michael Gebetsroither, Mon, Oct 15, 2007 02:46:11 +0200:
> > - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
> >   can be not the same, depending on what current "drive" is) and
> >   multi-cwd, which hasn't had formed itself into a problem yet, but
> >   surely will
> 
> Thats true for linux too.
> /a/b/c and /a/b/c can be 2 totally different files depending on the vfs
> namespace you are one.

No it is not. A process will always see the same filesystem object
under the same path at the any given time (IOW, you can't have many
namespaces active at the same time).

Re: Switching from CVS to GIT

[Alex Riesen]

Johannes Schindelin, Mon, Oct 15, 2007 10:44:12 +0200:
> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
> > Can you show a test case where this penalty is clearly visible?  I'm 
> > curious to see the numbers.  TIA
...
> Now, that is my _personal_ decision.  If _you_ have no problem with 
> Windows, just stick with it.  (I always thought this goes without saying, 
> but Windows users tend to be very religious about this issue, thinking 
> just because I hate Windows that I want to make them switch.  Hahaha, no.)

They tend to be so exactly because they know how pathetic they are.
They just want to have something where they don't suck and do
everything to find it. And fail. Then they resort to graphics and
user-friendly interface.

Re: Switching from CVS to GIT

[Alex Riesen]

Johannes Schindelin, Mon, Oct 15, 2007 01:45:47 +0200:
> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
> > Alex Riesen said:
> > > - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
> > >   can be not the same, depending on what current "drive" is)
> > 
> > So what? on Unix "a/b/c" can be not the same.  Both cases are simply not 
> > complete file names, that's all.  No one said there must be a single 
> > root for all volumes, it's the Posix jingoism creeping in again.
> 
> I think Alex means this: you can have C:\a\b\c and D:\a\b\c.  So depending 
> on which drive you are, you mean one or the other.  Just comparing the 
> paths is not enough.

Not really. I meant that "/a/b/c" and "/a/b/c". Note the leading
slash. On windoze it is _NOT_ absolute path. It is relative to the
root of the current drive.

Re: Switching from CVS to GIT

[Alex Riesen]

Johannes Schindelin, Mon, Oct 15, 2007 03:22:53 +0200:
> On Sun, 14 Oct 2007, Brian Dessent wrote:
> > Johannes Schindelin wrote:
> > > The problem is that on Windows, you cannot keep a file open and delete 
> > > it at the same time.  This is an issue in Windows' equivalent of VFS.
> > > 
> > > A neat trick to work with temporary files without permission issues is 
> > > to open the file and delete it right after that.  This does not work 
> > > on Windows.
> > 
> > You can achieve the same thing on Windows with CreateFile() by setting 
> > the dwShareMode parameter to zero and setting the 
> > FILE_FLAG_DELETE_ON_CLOSE attribute on dwFlagsAndAttributes.  This 
> > results in a file that cannot be opened or read by any other process and 
> > that will be automatically deleted when all open handles are closed.
> 
> Aha.  So to support Windows, we have to wrap all sites that use that 
> trick, and special case that #ifdef __MINGW32__. 

He misunderstood. It is not what you meant. You cannot remove the open
file. What he talks about is removing the file after it is _closed_.
Junk.

RE: Switching from CVS to GIT

[Dave Korn]

On 15 October 2007 18:49, Alex Riesen wrote:

> Johannes Schindelin, Mon, Oct 15, 2007 10:44:12 +0200:
>> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
>>> Can you show a test case where this penalty is clearly visible?  I'm
>>> curious to see the numbers.  TIA
> ...
>> Now, that is my _personal_ decision.  If _you_ have no problem with
>> Windows, just stick with it.  (I always thought this goes without saying,
>> but Windows users tend to be very religious about this issue, thinking
>> just because I hate Windows that I want to make them switch.  Hahaha, no.)
> 
> They tend to be so exactly because they know how pathetic they are.
> They just want to have something where they don't suck and do
> everything to find it. And fail. Then they resort to graphics and
> user-friendly interface.

  Translation:  "I feel that I am superior to other people.  This post has no
content apart from me shooting my mouth off in an attempt to prove how much
cleverer I am than anyone else.  However apart from my self-love I have no
contribution to make to the discussion."

  This isn't slashdot.  A computer is just a tool, and it's really *you* who
are being pathetic, because you confuse a choice of mass-manufactured consumer
product with a statement about personal identity.  Loyalty to your favourite
brand is a game of one-upmanship suitable only for kids.  You need to grow up.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: Switching from CVS to GIT

[Paul Smith]

OK, enough.  I am extremely grateful for the porting and maintenance
efforts of the GNU make porting team (since I have no Windows--or Amiga,
or OS/2, or OpenVMS--systems to maintain these ports myself) and I'm not
going to choose a tool unless it supports their environment and helps
them to work more efficiently (or at least no less efficiently).  I'm
not interested in getting into a pissing match over which operating
system is better or worse, and I'm certainly not interested in unfounded
inferences as to the character and quality of my porting team based on
the operating system they are using.

For those who have provided details and pointers regarding the state of
GIT on Windows, thank you very much for your help: it's been very useful
and Eli and others sound like they have enough information to be getting
on with for now.  If you'd like to discuss some Windows porting issues
further there are a number of extremely knowledgeable Windows / FLOSS
programmers on the make-w32@gnu.org list--although they are generally
very busy.

If what you're interested in is self-congratulatory back-slapping over
the superiority of Linux/POSIX, please keep that on the GIT mailing
list, or else an advocacy forum somewhere.


I'm setting followups to the GIT list.

Cheers all!

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@gnu.org>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.mad-scientist.us
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist

RE: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Dave Korn wrote:

> On 15 October 2007 18:49, Alex Riesen wrote:
> 
> > Johannes Schindelin, Mon, Oct 15, 2007 10:44:12 +0200:
> >> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
> >>> Can you show a test case where this penalty is clearly visible?  I'm
> >>> curious to see the numbers.  TIA
> > ...
> >> Now, that is my _personal_ decision.  If _you_ have no problem with
> >> Windows, just stick with it.  (I always thought this goes without saying,
> >> but Windows users tend to be very religious about this issue, thinking
> >> just because I hate Windows that I want to make them switch.  Hahaha, no.)
> > 
> > They tend to be so exactly because they know how pathetic they are. 
> > They just want to have something where they don't suck and do 
> > everything to find it. And fail. Then they resort to graphics and 
> > user-friendly interface.
> 
>   Translation:  "I feel that I am superior to other people.  This post 
> has no content apart from me shooting my mouth off in an attempt to 
> prove how much cleverer I am than anyone else.  However apart from my 
> self-love I have no contribution to make to the discussion."
> 
>   This isn't slashdot.  A computer is just a tool, and it's really *you* 
> who are being pathetic, because you confuse a choice of 
> mass-manufactured consumer product with a statement about personal 
> identity.  Loyalty to your favourite brand is a game of one-upmanship 
> suitable only for kids.  You need to grow up.

I sense a classical Stockholm Syndrome here ;-)

Ciao,
Dscho

Re: Switching from CVS to GIT

[Brian Dessent]

Alex Riesen wrote:

> He misunderstood. It is not what you meant. You cannot remove the open
> file. What he talks about is removing the file after it is _closed_.
> Junk.

I did not misunderstand.  The semantics are equivalent to the POSIX
case: you end up with a handle to an open file that is exclusive to that
process (it cannot be opened by any other process, even root) and that
is automatically reclaimed by the filesystem when all open handles are
closed, without any explicit action by the user.  It's not "unlinking an
open file", no, but it's the same result.

Brian

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

[excluding make-w32 list as per explicit request]

On Mon, 15 Oct 2007, Brian Dessent wrote:

> Alex Riesen wrote:
> 
> > He misunderstood. It is not what you meant. You cannot remove the open
> > file. What he talks about is removing the file after it is _closed_.
> > Junk.
> 
> I did not misunderstand.  The semantics are equivalent to the POSIX 
> case: you end up with a handle to an open file that is exclusive to that 
> process (it cannot be opened by any other process, even root) and that 
> is automatically reclaimed by the filesystem when all open handles are 
> closed, without any explicit action by the user.  It's not "unlinking an 
> open file", no, but it's the same result.

No, it is not equivalent.  For example, you can still see the file.  For 
example, you cannot reuse the filename for another file.  And -- the 
killer -- you cannot remove the directory which contains the file.

But really, we have workarounds in place to make this a non-issue.

My bigger concerns are the performance and stability.  For example, I had 
a very annoying problem on one of the machines I am testing msysGit on.  
The problem was _only_ fixable by deactivating component of Logitech's 
WebCam driver!  Now, if a user-installable 3rd party program can make my 
regular git crash, I am scared what more it can do.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Brian Dessent]

Johannes Schindelin wrote:

> No, it is not equivalent.  For example, you can still see the file.  For
> example, you cannot reuse the filename for another file.  And -- the
> killer -- you cannot remove the directory which contains the file.

Fair enough.  You can move it to another directory in order to delete
the containing directory -- this is what Cygwin does to placate posix
apps that expect this to work.

> But really, we have workarounds in place to make this a non-issue.

Ok.

> My bigger concerns are the performance and stability.  For example, I had
> a very annoying problem on one of the machines I am testing msysGit on.
> The problem was _only_ fixable by deactivating component of Logitech's
> WebCam driver!  Now, if a user-installable 3rd party program can make my
> regular git crash, I am scared what more it can do.

That is because the MSYS runtime is based on an old version of Cygwin,
and it uses the same dirty tricks to emulate fork.  These tricks rely on
having a repeatably consistent memory layout for a process each time it
is started, and when third party tools add hooks that affect the load
order or otherwise screw with the layout, the fork emulation fails. 
This is also why it is sometimes necessary to assign unique base
addresses to all libraries (rebaseall) in order to get fork emulation
working again.

So yes, it is unfortunate that some system tools can drastically affect
the ability of Cygwin and MSYS to function, but it's what we live with
to have fork/exec emulation.  I see that there is work afoot to abstract
process creation so that hopefully this won't be as much a concern in
the near future.

Brian

Re: Switching from CVS to GIT

[David Kastrup]

Alex Riesen <raa.lkml@gmail.com> writes:

> Michael Gebetsroither, Mon, Oct 15, 2007 02:46:11 +0200:
>> > - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
>> >   can be not the same, depending on what current "drive" is) and
>> >   multi-cwd, which hasn't had formed itself into a problem yet, but
>> >   surely will
>> 
>> Thats true for linux too.
>> /a/b/c and /a/b/c can be 2 totally different files depending on the vfs
>> namespace you are one.
>
> No it is not. A process will always see the same filesystem object
> under the same path at the any given time (IOW, you can't have many
> namespaces active at the same time).

dak@lola:/home/tmp/emacs$ mkdir -p /tmp/a/b
dak@lola:/home/tmp/emacs$ cd /tmp/a/b
dak@lola:/tmp/a/b$ sudo mount --bind /usr /tmp/a
Password:
dak@lola:/tmp/a/b$ command pwd
/tmp/a/b
dak@lola:/tmp/a/b$ ls -l
total 0
dak@lola:/tmp/a/b$ ls -l /tmp/a/b
ls: /tmp/a/b: No such file or directory
dak@lola:/tmp/a/b$ 

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Brian Dessent wrote:

> Johannes Schindelin wrote:
> 
> > My bigger concerns are the performance and stability.  For example, I 
> > had a very annoying problem on one of the machines I am testing 
> > msysGit on. The problem was _only_ fixable by deactivating component 
> > of Logitech's WebCam driver!  Now, if a user-installable 3rd party 
> > program can make my regular git crash, I am scared what more it can 
> > do.
> 
> That is because the MSYS runtime is based on an old version of Cygwin, 
> and it uses the same dirty tricks to emulate fork.  These tricks rely on 
> having a repeatably consistent memory layout for a process each time it 
> is started, and when third party tools add hooks that affect the load 
> order or otherwise screw with the layout, the fork emulation fails. This 
> is also why it is sometimes necessary to assign unique base addresses to 
> all libraries (rebaseall) in order to get fork emulation working again.

Ah, thanks for the explanation!  (I knew that this thread still had 
something useful in it ;-)

> So yes, it is unfortunate that some system tools can drastically affect 
> the ability of Cygwin and MSYS to function, but it's what we live with 
> to have fork/exec emulation.  I see that there is work afoot to abstract 
> process creation so that hopefully this won't be as much a concern in 
> the near future.

We never had the problem in git itself, since we never used fork() on 
Windows.  The problem lies in our usage of bash and perl.

Bash we can fix in the long run (this goes under the keyword 
"builtinification" on the git list), but I do not see our reliance on Perl 
going away, not for git {send-email,cvsimport,cvsexportcommit,svn}.  
These are not too common operations, so common users will be able to do 
without them.

However, if you rely on the CVS/SVN connectors, or send-email, and in any 
case in the short run, you better run Git on Windows only when that funny 
Logitech driver is disabled ;-)

Ciao,
Dscho

Re: Switching from CVS to GIT

[Alex Riesen]

David Kastrup, Mon, Oct 15, 2007 21:26:44 +0200:
> Alex Riesen <raa.lkml@gmail.com> writes:
> 
> > Michael Gebetsroither, Mon, Oct 15, 2007 02:46:11 +0200:
> >> > - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
> >> >   can be not the same, depending on what current "drive" is) and
> >> >   multi-cwd, which hasn't had formed itself into a problem yet, but
> >> >   surely will
> >> 
> >> Thats true for linux too.
> >> /a/b/c and /a/b/c can be 2 totally different files depending on the vfs
> >> namespace you are one.
> >
> > No it is not. A process will always see the same filesystem object
> > under the same path at the any given time (IOW, you can't have many
> > namespaces active at the same time).
> 
> dak@lola:/home/tmp/emacs$ mkdir -p /tmp/a/b
> dak@lola:/home/tmp/emacs$ cd /tmp/a/b
> dak@lola:/tmp/a/b$ sudo mount --bind /usr /tmp/a

Well don't do that in your repos (unless need that for something).

It is not like someone creates a distribution which does it
automagically all the time and you're forced to use that distribution.

Re: Switching from CVS to GIT

[Alex Riesen]

Dave Korn, Mon, Oct 15, 2007 20:25:55 +0200:
> On 15 October 2007 18:49, Alex Riesen wrote:
> 
> > Johannes Schindelin, Mon, Oct 15, 2007 10:44:12 +0200:
> >> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
> >>> Can you show a test case where this penalty is clearly visible?  I'm
> >>> curious to see the numbers.  TIA
> > ...
> >> Now, that is my _personal_ decision.  If _you_ have no problem with
> >> Windows, just stick with it.  (I always thought this goes without saying,
> >> but Windows users tend to be very religious about this issue, thinking
> >> just because I hate Windows that I want to make them switch.  Hahaha, no.)
> > 
> > They tend to be so exactly because they know how pathetic they are.
> > They just want to have something where they don't suck and do
> > everything to find it. And fail. Then they resort to graphics and
> > user-friendly interface.
> 
>   Translation:  "I feel that I am superior to other people.  This post has no
> content apart from me shooting my mouth off in an attempt to prove how much
> cleverer I am than anyone else.  However apart from my self-love I have no
> contribution to make to the discussion."

It is interpretation, not translation. Wrong, too.

>   This isn't slashdot.  A computer is just a tool, and it's really *you* who
> are being pathetic, because you confuse a choice of mass-manufactured consumer

it is not a "choice". It is an accident. Like in "caused by careless driving".

> product with a statement about personal identity.  Loyalty to your favourite
> brand is a game of one-upmanship suitable only for kids.  You need to grow up.

Yep. Will do.

Re: Switching from CVS to GIT

[Alex Riesen]

Brian Dessent, Mon, Oct 15, 2007 21:07:53 +0200:
> Johannes Schindelin wrote:
> > My bigger concerns are the performance and stability.  For example, I had
> > a very annoying problem on one of the machines I am testing msysGit on.
> > The problem was _only_ fixable by deactivating component of Logitech's
> > WebCam driver!  Now, if a user-installable 3rd party program can make my
> > regular git crash, I am scared what more it can do.
> 
> That is because the MSYS runtime is based on an old version of Cygwin,
> and it uses the same dirty tricks to emulate fork.  These tricks rely on
> having a repeatably consistent memory layout for a process each time it
> is started, and when third party tools add hooks that affect the load
> order or otherwise screw with the layout, the fork emulation fails. 
> This is also why it is sometimes necessary to assign unique base
> addresses to all libraries (rebaseall) in order to get fork emulation
> working again.

Hmm... Could the allocation of large contiguous blocks also lock the
system hard? For instance, I avoid starting the test suite on my XP
workstation at work: it locks up hard every time. W2k works.
The system has nothing unusual in it. Well, it has an antivirus
program (which hopefully stopped working after a series of crashes,
which is just as well), an NVidia card with native driver (which is
broken in its own usual ways). Maybe that's enough

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 21:42:14 +0200
> From: Alex Riesen <raa.lkml@gmail.com>
> Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>, git@vger.kernel.org,
> 	Eli Zaretskii <eliz@gnu.org>, ae@op5.se, tsuna@lrde.epita.fr
> 
> Hmm... Could the allocation of large contiguous blocks also lock the
> system hard?

No, not on XP.

> For instance, I avoid starting the test suite on my XP
> workstation at work: it locks up hard every time.

Sounds like a bug to me.

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Eli Zaretskii wrote:

> > From: Alex Riesen <raa.lkml@gmail.com>
> 
> > For instance, I avoid starting the test suite on my XP workstation at 
> > work: it locks up hard every time.
> 
> Sounds like a bug to me.

To me, too.  Alas, it works on W2k, so where is the bug?

Ciao,
Dscho

Re: Switching from CVS to GIT

[Mark Watts]

Hello Johannes,

> My bigger concerns are the performance and stability.  For example, I
> had a very annoying problem on one of the machines I am testing
> msysGit on.  The problem was _only_ fixable by deactivating component
> of Logitech's WebCam driver!  Now, if a user-installable 3rd party
> program can make my regular git crash, I am scared what more it can
> do.

Not just git.  This driver has known issues with a number of pieces of software. 
 The company I work with uses Delphi quite a bit and they also have Logitech 
WebCams for the devs and for some reason this driver makes debugging impossible 
with Delphi.  I have personally experienced this.  I have also heard of this 
Logitech software having problems with other software too.  I have not however 
tracked down exactly WHY this piece of software causes so much grief, only 
that it does.

-mark

Re: Switching from CVS to GIT

[Brian Dessent]

Alex Riesen wrote:

> Hmm... Could the allocation of large contiguous blocks also lock the
> system hard? For instance, I avoid starting the test suite on my XP
> workstation at work: it locks up hard every time. W2k works.
> The system has nothing unusual in it. Well, it has an antivirus
> program (which hopefully stopped working after a series of crashes,
> which is just as well), an NVidia card with native driver (which is
> broken in its own usual ways). Maybe that's enough

In terms of the MSYS/Cygwin style of fork emulation, large memory
allocations shouldn't pose any real problem, but they will be slow as
the fork emulation has to manually replicate the state of the parent in
the child and this means copying memory extents.  (Yes, horribly ugly,
no doubt about it, but it allows for porting.)

This emulation code is sensitive enough that the Cygwin list has begun
to maintain a list of software whose hooks/interference can cause Cygwin
apps to fail: <http://cygwin.com/ml/cygwin-talk/2007-q3/msg00174.html>. 
Since MSYS is derived from the same code I see no reason why the list
wouldn't also implicate potential problems with binaries linked to the
MSYS runtime.

Johannes Schindelin wrote:

> We never had the problem in git itself, since we never used fork() on
> Windows.  The problem lies in our usage of bash and perl.
> 
> Bash we can fix in the long run (this goes under the keyword
> "builtinification" on the git list), but I do not see our reliance on Perl
> going away, not for git {send-email,cvsimport,cvsexportcommit,svn}.
> These are not too common operations, so common users will be able to do
> without them.
> 
> However, if you rely on the CVS/SVN connectors, or send-email, and in any
> case in the short run, you better run Git on Windows only when that funny
> Logitech driver is disabled ;-)

Well, instead of using an MSYS build of Perl there's always ActiveState
Perl.  I think you may be stuck on the shell though -- I don't know of
any ports of bash that aren't MSYS or Cygwin based.  However I do think
there's a native port of zsh out there by the GnuWin32 project, which
when renamed as just "/bin/sh" might be suitable, but only if these
scripts don't use bash-isms.  I have not tried this zsh myself and
speed/compatibility wise I'm not sure it's up to snuff.

Brian

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Brian Dessent wrote:

> Well, instead of using an MSYS build of Perl there's always ActiveState 
> Perl.

No, but thanks no.  You haven't been around long enough on this list to 
count the issues, but it is known that ActiveState Perl is close to 
Malbolge.

>  I think you may be stuck on the shell though -- I don't know of any 
> ports of bash that aren't MSYS or Cygwin based.  However I do think 
> there's a native port of zsh out there by the GnuWin32 project, which 
> when renamed as just "/bin/sh" might be suitable, but only if these 
> scripts don't use bash-isms.  I have not tried this zsh myself and 
> speed/compatibility wise I'm not sure it's up to snuff.

There is a port of BusyBox' dash, which is nearing completion.  Once 
Nguyen says it is ready enough, we will try to integrate it into msysGit.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Linus Torvalds]

On Mon, 15 Oct 2007, Johannes Schindelin wrote:
> 
> Bash we can fix in the long run (this goes under the keyword 
> "builtinification" on the git list)

I thought busybox was being used for the core commands? Is ash not 
complete/usable enough (with all the fixes git has had for broken shells) 
to be used? 

I do agree that perl looks unavoidable, but I thought the windows port 
already avoided at least bash. Not true?

(or is it just that even with ash, you end up hitting all the same issues 
with cygwin/msys?)

		Linus

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Linus Torvalds wrote:

> On Mon, 15 Oct 2007, Johannes Schindelin wrote:
> > 
> > Bash we can fix in the long run (this goes under the keyword 
> > "builtinification" on the git list)
> 
> I thought busybox was being used for the core commands? Is ash not 
> complete/usable enough (with all the fixes git has had for broken 
> shells) to be used?

No, not yet.  The problem is not so much ash, as Nguyen, who said that 
gitbox is not there yet.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 15, 2007, at 10:19 PM, Johannes Schindelin wrote:

>>  I think you may be stuck on the shell though -- I don't know of any
>> ports of bash that aren't MSYS or Cygwin based.  However I do think
>> there's a native port of zsh out there by the GnuWin32 project, which
>> when renamed as just "/bin/sh" might be suitable, but only if these
>> scripts don't use bash-isms.  I have not tried this zsh myself and
>> speed/compatibility wise I'm not sure it's up to snuff.

I can't find zsh on the GnuWin32 project page.


> There is a port of BusyBox' dash, which is nearing completion.  Once
> Nguyen says it is ready enough, we will try to integrate it into  
> msysGit.

Gnuarch [1] recommends zsh from the unxutils project [2].

	Steffen

[1] http://www.gnuarch.org/gnuarchwiki/Native_WIN32_Support
[2] http://unxutils.sourceforge.net/

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Mon, 15 Oct 2007, Steffen Prohaska wrote:

> On Oct 15, 2007, at 10:19 PM, Johannes Schindelin wrote:
> 
> > There is a port of BusyBox' dash, which is nearing completion.  Once 
> > Nguyen says it is ready enough, we will try to integrate it into 
> > msysGit.
> 
> Gnuarch [1] recommends zsh from the unxutils project [2].

I have the slight suspicion that we will somehow have problems with 
/etc/gitconfig, /share/git-gui/ and friends, should we try to use zsh.  At 
least with gitbox, we can hack a "/" translation for scripts.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 20:58:26 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Alex Riesen <raa.lkml@gmail.com>, brian@dessent.net, git@vger.kernel.org, 
>     ae@op5.se, tsuna@lrde.epita.fr
> 
> On Mon, 15 Oct 2007, Eli Zaretskii wrote:
> 
> > > From: Alex Riesen <raa.lkml@gmail.com>
> > 
> > > For instance, I avoid starting the test suite on my XP workstation at 
> > > work: it locks up hard every time.
> > 
> > Sounds like a bug to me.
> 
> To me, too.  Alas, it works on W2k, so where is the bug?

I'm not smart enough to know without debugging it.  W2K and WXP have
different system libraries and somewhat different memory layouts.  A
bug can get away on one, but not on the other.

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 13:05:51 -0700
> From: Brian Dessent <brian@dessent.net>
> CC: Johannes Schindelin <Johannes.Schindelin@gmx.de>,
>  	git@vger.kernel.org, Eli Zaretskii <eliz@gnu.org>, ae@op5.se,
>  	tsuna@lrde.epita.fr
> 
> I don't know of
> any ports of bash that aren't MSYS or Cygwin based.  However I do think
> there's a native port of zsh out there by the GnuWin32 project, which
> when renamed as just "/bin/sh" might be suitable, but only if these
> scripts don't use bash-isms.  I have not tried this zsh myself and
> speed/compatibility wise I'm not sure it's up to snuff.

I think you mean Amol's zsh (there's no GnuWin32 port of zsh AFAIK).
Amol's zsh is what I use, but it has a few annoying bugs, even after I
fixed some, that prevent it from running a typical configure script,
for example.

Re: Switching from CVS to GIT

[Shawn O. Pearce]

Johannes Sixt <j.sixt@viscovery.net> wrote:
> FWIW, I'm using the MinGW port from cmd.exe, i.e. not from a posix shell, 
> on a *production* repository. gitk and git-gui work. Not all operations 
> that I regularly use are available[*] via the GUIs, like git-rebase or 
> non-fast-forwarding push, so the use of the command line is needed from 
> time to time.

Rebase in git-gui is starting to be developed.  But its still not even
close to something I can use, let alone that I would be willing to ship
to another person for testing.

Force push (non-fast-forwarding push) is in git-gui.git's master
branch now as part of the 0.9.x series.  There's a new checkbox
option in the push dialog to trigger adding --force to git-push
command line.

> Unfortunately, "Fetch" does not yet work[*] from within git-gui, so you 
> have to fall back to git-fetch on the command line.
> 
> [*] Note the distinction between "not available" and "does not work".

What's broken?  Is this that Git protocol dump showing up in
git-gui's console window thing?

Are you using the C based fetch that is in git.git's next branch,
or the shell script based one that is in master?  Which Tcl/Tk
version are you using to run git-gui?

-- 
Shawn.

Re: Switching from CVS to GIT

[Daniel Barkalow]

Responding only to those portions where I think Windows experience and a 
Windows perspective would be helpful...

On Mon, 15 Oct 2007, Eli Zaretskii wrote:

> > - no proper filename semantics (case-insensitivity and stupid rules for
> >   allowed characters in filenames, like ":" in filenames in
> >   cross-platform projects)
> 
> There's a flag on Windows to open files case-sensitively, if you need
> that.  In any case, I don't see how this can be of any real relevance
> to porting GIT.  As for ":" in file names, simply don't use it, like
> you don't use white space or characters below 32 decimal: it's
> inconvenient, even if it's allowed.

I believe the hassle is that readdir doesn't necessarily report a README in 
a directory which is supposed to have a README, when it has a readme 
instead. I think we want O(n) comparison of sorted lists, which doesn't 
work if equivalent names don't sort the same.

> > - no acceptable level of performance in filesystem and VFS (readdir,
> >   stat, open and read/write are annoyingly slow)
> 
> With what libraries?  Native `stat' and `readdir' are quite fast.
> Perhaps you mean the ported glibc (libgw32c), where `readdir' is
> indeed painfully slow, but then you don't need to use it.

We want getting stat info, using readdir to figure out what files exist, 
for 106083 files in 1603 directories with a hot cache to take under 1s; 
otherwise "git status" takes a noticeable amount of time with a medium-big 
project, and we want people to be able to get info on what's changed 
effectively instantly. My impression is that Windows' native stat and 
readdir are plenty fast for what normal Windows programs want, but we 
actually expect reasonable performance on an unreasonably-big 
metadata-heavy input. AFAICT, nothing but Linux is optimized for this, but 
we're used to being able to find out if there's any change to a large 
directory structure in practically no time. On the other hand, we really 
just want to beat users' expectations for this operation, not our own 
expectations, so this may only be a problem for people benchmarking 
Windows git against Linux git.

> > - no real "mmap" (which kills perfomance and complicates code)
> 
> You only need mmap because you are accustomed to use it on GNU/Linux.

I believe the need here is quick setup and fast access to sparse portions 
of several 100M files. It's hard to beat a page fault for read speed.

We also expect to be able to make a sequence of file system operations 
such that programs starting at any time see the same database as the files 
containing the database get restructured. My impression is that this is 
very hard or impossible with Windows, and also that it doesn't matter for 
Windows users, because they'll only have one program at a time accessing 
the repository. A lot of our filesystem demands are about making a wide 
variety of race conditions give the same result regardless of how the race 
goes, and we're just being overly careful for a Windows environment 
(although not necessarily for users with a UNIX background using Windows 
only because they have to).

> > - it has only one argument (limited in size) passed to started
> >   programs, which means that there is no possible way to safely pass
> >   file and text arguments on command line (more than one, that is)
> 
> Not enough context, so I cannot talk intelligently about this.  Why do
> you need interprocess communication in the first place? why not simply
> give birth to a subsidiary process and pass it a command line (which
> can be up to 32KB)?

A unixy pipeline was convenient, given what else we had already written. 
It's getting converted to single tasks, but it's not a top priority for 
most developers, since streaming 100M from one program to the next under 
most of our environments is trivial.

	-Daniel
*This .sig left intentionally blank*

Re: Switching from CVS to GIT

[Nguyen Thai Ngoc Duy]

On 10/16/07, Steffen Prohaska <prohaska@zib.de> wrote:
>
> On Oct 15, 2007, at 10:19 PM, Johannes Schindelin wrote:
> > There is a port of BusyBox' dash, which is nearing completion.  Once
> > Nguyen says it is ready enough, we will try to integrate it into
> > msysGit.
>
> Gnuarch [1] recommends zsh from the unxutils project [2].

All zsh links in [2] are dead. I did try hard to find the legendary
zsh for Windows before giving up and porting busybox's ash instead. If
you have zsh source of the port, please send me. Thank you.

>         Steffen
>
> [1] http://www.gnuarch.org/gnuarchwiki/Native_WIN32_Support
> [2] http://unxutils.sourceforge.net/
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Duy

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 09:24:50 +0700
> From: "Nguyen Thai Ngoc Duy" <pclouds@gmail.com>
> Cc: "Git Mailing List" <git@vger.kernel.org>, "Alex Riesen" <raa.lkml@gmail.com>, 
> 	"Eli Zaretskii" <eliz@gnu.org>, "Andreas Ericsson" <ae@op5.se>, 
> 	tsuna@lrde.epita.fr, 
> 	"Johannes Schindelin" <Johannes.Schindelin@gmx.de>
> 
> I did try hard to find the legendary
> zsh for Windows before giving up and porting busybox's ash instead.

Where can one find this port of busybox's ash?

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Mon, 15 Oct 2007 20:45:02 -0400 (EDT)
> From: Daniel Barkalow <barkalow@iabervon.org>
> cc: Alex Riesen <raa.lkml@gmail.com>, Johannes.Schindelin@gmx.de, ae@op5.se, 
>     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
> 
> I believe the hassle is that readdir doesn't necessarily report a README in 
> a directory which is supposed to have a README, when it has a readme 
> instead.

Sorry I'm asking potentially stupid questions out of ignorance: why
would you want readdir to return `README' when you have `readme'?

> I think we want O(n) comparison of sorted lists, which doesn't 
> work if equivalent names don't sort the same.

You comparison function should be case-insensitive on Windows, or am I
missing something?

> > > - no acceptable level of performance in filesystem and VFS (readdir,
> > >   stat, open and read/write are annoyingly slow)
> > 
> > With what libraries?  Native `stat' and `readdir' are quite fast.
> > Perhaps you mean the ported glibc (libgw32c), where `readdir' is
> > indeed painfully slow, but then you don't need to use it.
> 
> We want getting stat info, using readdir to figure out what files exist, 
> for 106083 files in 1603 directories with a hot cache to take under 1s; 
> otherwise "git status" takes a noticeable amount of time with a medium-big 
> project, and we want people to be able to get info on what's changed 
> effectively instantly. My impression is that Windows' native stat and 
> readdir are plenty fast for what normal Windows programs want, but we 
> actually expect reasonable performance on an unreasonably-big 
> metadata-heavy input.

If that's the issue, then it's not a good idea to call `stat' and
`readdir' on Windows at all.  `stat' is a single system call on Posix
systems, while on Windows it usually needs to go out of its way
calling half a dozen system services to gather the `struct stat' info.
You need to call something like FindFirstFile, which can do the job of
`stat' and `readdir' together (and of `fnmatch', if you need to filter
only some files) in one go.  I don't know whether this will scan 100K
files under one second (maybe I will try it one of these days), but it
will definitely be faster than `readdir'+`stat' by maybe as much as an
order of magnitude.

> > > - no real "mmap" (which kills perfomance and complicates code)
> > 
> > You only need mmap because you are accustomed to use it on GNU/Linux.
> 
> I believe the need here is quick setup and fast access to sparse portions 
> of several 100M files. It's hard to beat a page fault for read speed.

If you need memory-mapped files, they are available on Windows.  I
thought the original comment about `mmap' was because it was used to
allocate memory, not read files into memory.

> We also expect to be able to make a sequence of file system operations 
> such that programs starting at any time see the same database as the files 
> containing the database get restructured.

Sorry, I don't understand this; please tell more about the operations,
``the same database'' issue (what database?) and what do you mean by
``the files containing the database get restructured''.

> A unixy pipeline was convenient

Windows supports pipelines with almost 100% the same functionality as
Posix.  Again, perhaps I'm missing something.

Re: Switching from CVS to GIT

[Andreas Ericsson]

Eli Zaretskii wrote:
>> Date: Mon, 15 Oct 2007 20:45:02 -0400 (EDT)
>> From: Daniel Barkalow <barkalow@iabervon.org>
>> cc: Alex Riesen <raa.lkml@gmail.com>, Johannes.Schindelin@gmx.de, ae@op5.se, 
>>     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
>>
>> I believe the hassle is that readdir doesn't necessarily report a README in 
>> a directory which is supposed to have a README, when it has a readme 
>> instead.
> 
> Sorry I'm asking potentially stupid questions out of ignorance: why
> would you want readdir to return `README' when you have `readme'?
> 

Because it might have been checked in as README, and since git is case
sensitive that is what it'll think should be there when it reads the
directories. If it's not, users get to see

	removed: README
	untracked: readme

and there's really no easy way out of this one, since users on a case-
sensitive filesystem might be involved in this project too, so it
could be an intentional rename, but we don't know for sure. Just
clobbering the in-git file is wrong, but overwriting a file on disk
is wrong too. git tries hard to not ever lose any data for the user.

> 
>>>> - no acceptable level of performance in filesystem and VFS (readdir,
>>>>   stat, open and read/write are annoyingly slow)
>>> With what libraries?  Native `stat' and `readdir' are quite fast.
>>> Perhaps you mean the ported glibc (libgw32c), where `readdir' is
>>> indeed painfully slow, but then you don't need to use it.
>> We want getting stat info, using readdir to figure out what files exist, 
>> for 106083 files in 1603 directories with a hot cache to take under 1s; 
>> otherwise "git status" takes a noticeable amount of time with a medium-big 
>> project, and we want people to be able to get info on what's changed 
>> effectively instantly. My impression is that Windows' native stat and 
>> readdir are plenty fast for what normal Windows programs want, but we 
>> actually expect reasonable performance on an unreasonably-big 
>> metadata-heavy input.
> 
> If that's the issue, then it's not a good idea to call `stat' and
> `readdir' on Windows at all.  `stat' is a single system call on Posix
> systems, while on Windows it usually needs to go out of its way
> calling half a dozen system services to gather the `struct stat' info.
> You need to call something like FindFirstFile, which can do the job of
> `stat' and `readdir' together (and of `fnmatch', if you need to filter
> only some files) in one go.  I don't know whether this will scan 100K
> files under one second (maybe I will try it one of these days), but it
> will definitely be faster than `readdir'+`stat' by maybe as much as an
> order of magnitude.
> 

To be honest though, there are so many places which do the readdir+stat
that I don't think it'd be worth factoring it out, especially since it
*works* on windows. It's just slow, and only slow compared to various
unices. I *think* (correct me if I'm wrong) that git is still faster
than a whole bunch of other scm's on windows, but to one who's used to
its performance on Linux that waiting several seconds to scan 10k files
just feels wrong.


>> We also expect to be able to make a sequence of file system operations 
>> such that programs starting at any time see the same database as the files 
>> containing the database get restructured.
> 
> Sorry, I don't understand this; please tell more about the operations,
> ``the same database'' issue (what database?)

The object database, located under .git/objects.

> and what do you mean by
> ``the files containing the database get restructured''.
> 

/* I'm on a limb here. Nicolas Pitre knows the git packfile format, so
 * perhaps he'll be kind enough to correct me if I'm wrong */

The mmap() stuff is primarily convenient when reading huge packfiles. As
far as I understand it, they're ordered by some sort of delta similarity
score, so mmap()'ing 100MiB or so of a certain packfile will most likely
mean we have a couple of thousand "connected" revisions in memory. That
database gets sort of restructured as the memory-chunk that's mmap()'ed
get moved to read in the next couple of thousand revisions.

In all honesty, this doesn't matter much for already fully packed projects
unless they're significantly larger than the Linux kernel, since git is so
amazingly good at compressing large repos to a small size. Linux is ~180
MiB fully packed, and most developer's systems could just read() that
entire packfile into memory without much problem. But then again, no-one's
ever had problems supporting the "normal" cases.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Switching from CVS to GIT

[Daniel Barkalow]

On Tue, 16 Oct 2007, Eli Zaretskii wrote:

> > Date: Mon, 15 Oct 2007 20:45:02 -0400 (EDT)
> > From: Daniel Barkalow <barkalow@iabervon.org>
> > cc: Alex Riesen <raa.lkml@gmail.com>, Johannes.Schindelin@gmx.de, ae@op5.se, 
> >     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
> > 
> > I believe the hassle is that readdir doesn't necessarily report a README in 
> > a directory which is supposed to have a README, when it has a readme 
> > instead.
> 
> Sorry I'm asking potentially stupid questions out of ignorance: why
> would you want readdir to return `README' when you have `readme'?

Say the project upstream has the file being "README", but, for some 
reason, it has ended up checked out as "readme" in your directory. Since 
your filesystem is case insensitive, it's supposed to be the same file, 
but when git goes through the list of files in the directory, it sees 
"readme", and there's nothing between reachable.h and read-cache.c in the 
list of tracked files. We've got a sorted list of filenames we're tracking 
along with their most-recently-seen content, and we want to merge the 
results of readdir with them, and this is obviously more straightforward 
if the filename that's the match for "README" is provided byte-for-byte 
the same, and therefore sorts the same.

> > I think we want O(n) comparison of sorted lists, which doesn't 
> > work if equivalent names don't sort the same.
> 
> You comparison function should be case-insensitive on Windows, or am I
> missing something?

We want both lists sorted, so that we can step through the pair together 
and always reach matches together. This requires that the equivalent names 
sort together, as well as comparing equal.

> > > > - no acceptable level of performance in filesystem and VFS (readdir,
> > > >   stat, open and read/write are annoyingly slow)
> > > 
> > > With what libraries?  Native `stat' and `readdir' are quite fast.
> > > Perhaps you mean the ported glibc (libgw32c), where `readdir' is
> > > indeed painfully slow, but then you don't need to use it.
> > 
> > We want getting stat info, using readdir to figure out what files exist, 
> > for 106083 files in 1603 directories with a hot cache to take under 1s; 
> > otherwise "git status" takes a noticeable amount of time with a medium-big 
> > project, and we want people to be able to get info on what's changed 
> > effectively instantly. My impression is that Windows' native stat and 
> > readdir are plenty fast for what normal Windows programs want, but we 
> > actually expect reasonable performance on an unreasonably-big 
> > metadata-heavy input.
> 
> If that's the issue, then it's not a good idea to call `stat' and
> `readdir' on Windows at all.  `stat' is a single system call on Posix
> systems, while on Windows it usually needs to go out of its way
> calling half a dozen system services to gather the `struct stat' info.
> You need to call something like FindFirstFile, which can do the job of
> `stat' and `readdir' together (and of `fnmatch', if you need to filter
> only some files) in one go.  I don't know whether this will scan 100K
> files under one second (maybe I will try it one of these days), but it
> will definitely be faster than `readdir'+`stat' by maybe as much as an
> order of magnitude.

Ah, that's helpful. We don't actually care too much about the particular 
info in stat; we just want to know quickly if the file has changed, so we 
can hash only the ones that have been touched and get the actual content 
changes.

> > > > - no real "mmap" (which kills perfomance and complicates code)
> > > 
> > > You only need mmap because you are accustomed to use it on GNU/Linux.
> > 
> > I believe the need here is quick setup and fast access to sparse portions 
> > of several 100M files. It's hard to beat a page fault for read speed.
> 
> If you need memory-mapped files, they are available on Windows.  I
> thought the original comment about `mmap' was because it was used to
> allocate memory, not read files into memory.

No, we get our memory with malloc like normal people. The mmap is because 
we want to feed files and parts of files to zlib, and mmap makes that 
easy.

> > We also expect to be able to make a sequence of file system operations 
> > such that programs starting at any time see the same database as the files 
> > containing the database get restructured.
> 
> Sorry, I don't understand this; please tell more about the operations,
> ``the same database'' issue (what database?) and what do you mean by
> ``the files containing the database get restructured''.

Git is built around a database of objects, which includes "blobs" (file 
content), "trees" (directory structure), "commits" (history linkage), and 
"tags" (additional annotations). Each of these objects gets hashed, and is 
referenced by hash. So we need to be able to get the object with a given 
hash quickly, and write an object and take its hash (ideally, stream the 
write and find out the hash at the end, with the database key set at that 
point). Also, this database should be compressed effectively, because it 
ought to compress really well, since a lot of the blobs and trees are only 
slightly different from other blobs or trees (by whatever changes were 
made between that revision and other revisions).

The current implementation of the persistant storage of this database is a 
bit complicated, with the goal being that creating objects is really fast, 
and looking up objects doesn't degrade too quickly, and there are 
optimization operations available that take some time and speed up future 
lookups and reduce the storage overhead (especially so that data can be 
transferred efficiently). The tricky thing is that, while the optimization 
process is running, other programs may be reading the database, so (1) the 
files that are no longer needed, because better-optimized versions are in 
place, may be open in another task, and (2) complete and correct new 
files have to appear and be such that pre-existing tasks will find them 
before old files can be removed. The optimization creates "pack files" and 
"pack indices", where the pack file has a lot of objects with delta 
compression between them and zlib compression of them, and the index files 
tell where everything in the pack file is. So we mmap the index files to 
search through, and mmap portions of the pack files to get the data out 
of, and we may be using them as they're replaced with more comprehensive 
pack files by another task.

Now, it's entirely possible that a completely different database 
implementation would be better on Windows, but our current one does a lot 
of creating files under different names, moving them to names where 
they'll be seen (since this is atomic under POSIX, and partial files are 
never seen by other tasks). Also, once we have new files in place, we 
unlink the files that they replace, so that new tasks will use the new 
ones and tasks that already have old ones open can still get the data out 
of them. Also, the files generally get mmaped, 

> > A unixy pipeline was convenient
> 
> Windows supports pipelines with almost 100% the same functionality as
> Posix.  Again, perhaps I'm missing something.

I'm probably the one missing something here; I don't really know anything 
about Windows, and I only know what code other people have had problems 
porting. Mostly what we use for IPC is pipelines, so, if they work well, I 
don't know what the problem is.

	-Daniel
*This .sig left intentionally blank*

Re: Switching from CVS to GIT

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 15 Oct 2007 20:45:02 -0400 (EDT)
>> From: Daniel Barkalow <barkalow@iabervon.org>
>> cc: Alex Riesen <raa.lkml@gmail.com>, Johannes.Schindelin@gmx.de, ae@op5.se, 
>>     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
>> 
>> I believe the hassle is that readdir doesn't necessarily report a README in 
>> a directory which is supposed to have a README, when it has a readme 
>> instead.
>
> Sorry I'm asking potentially stupid questions out of ignorance: why
> would you want readdir to return `README' when you have `readme'?
>
>> I think we want O(n) comparison of sorted lists, which doesn't 
>> work if equivalent names don't sort the same.
>
> You comparison function should be case-insensitive on Windows, or am
> I missing something?

Well, are "I" and "i" the same letters?  What about "İ" and "i"?  Or
"I" and "ı"?  What about Greek where uppercasing loses accents
(actually not unusual in literate French, either).  And what about
German ß and SS/SZ?

"case-insensitive" is a simple word, but the devil is in the details,
and that means basically requiring a system-provided sorting function.
And actually the _killer_ detail here is that git _must_ have the same
sorting order on every platform, since the order of files in a
directory tree affects its SHA-1 sum.  So a system-dependent sorting
order breaks git interoperability.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Switching from CVS to GIT

[Johannes Sixt]

Shawn O. Pearce schrieb:
> Johannes Sixt <j.sixt@viscovery.net> wrote:
>> Unfortunately, "Fetch" does not yet work[*] from within git-gui, so you 
>> have to fall back to git-fetch on the command line.
>>
>> [*] Note the distinction between "not available" and "does not work".
> 
> What's broken?  Is this that Git protocol dump showing up in
> git-gui's console window thing?
> 
> Are you using the C based fetch that is in git.git's next branch,
> or the shell script based one that is in master?  Which Tcl/Tk
> version are you using to run git-gui?

It's the scripted fetch that does not work. The symptom is that the output 
of at least one of the commands (upload-pack, I think, because what I see is 
wire protocol) goes to a newly spawned console instead of wherever it was 
redirected to.

I didn't bother reporting since builtin-fetch is on the way (which will 
hopefully make this a moot point) and our team here is comfortable with 
calling git fetch on the command line.

-- Hannes

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 16, 2007, at 4:24 AM, Nguyen Thai Ngoc Duy wrote:

> On 10/16/07, Steffen Prohaska <prohaska@zib.de> wrote:
>>
>> On Oct 15, 2007, at 10:19 PM, Johannes Schindelin wrote:
>>> There is a port of BusyBox' dash, which is nearing completion.  Once
>>> Nguyen says it is ready enough, we will try to integrate it into
>>> msysGit.
>>
>> Gnuarch [1] recommends zsh from the unxutils project [2].
>
> All zsh links in [2] are dead. I did try hard to find the legendary
> zsh for Windows before giving up and porting busybox's ash instead. If
> you have zsh source of the port, please send me. Thank you.

It is included in UnxUtilsSrc.zip, updated on 2007-03-01 06:22, from

http://sourceforge.net/projects/unxutils
http://sourceforge.net/project/showfiles.php?group_id=9328

	Steffen

Re: Switching from CVS to GIT

[Shawn O. Pearce]

Johannes Sixt <j.sixt@viscovery.net> wrote:
> >Johannes Sixt <j.sixt@viscovery.net> wrote:
> >>Unfortunately, "Fetch" does not yet work[*] from within git-gui, so you 
> >>have to fall back to git-fetch on the command line.
> 
> It's the scripted fetch that does not work. The symptom is that the output 
> of at least one of the commands (upload-pack, I think, because what I see 
> is wire protocol) goes to a newly spawned console instead of wherever it 
> was redirected to.
> 
> I didn't bother reporting since builtin-fetch is on the way (which will 
> hopefully make this a moot point) and our team here is comfortable with 
> calling git fetch on the command line.

Hmm.  The way the builtin-fetch works this shouldn't happen, but
I'd appreciate it if you could test and report back before that
topic merges into master.

-- 
Shawn.

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 07:14:56 +0200
> From: Andreas Ericsson <ae@op5.se>
> CC: Daniel Barkalow <barkalow@iabervon.org>,  raa.lkml@gmail.com, 
>  Johannes.Schindelin@gmx.de,  tsuna@lrde.epita.fr,  git@vger.kernel.org, 
>  make-w32@gnu.org
> 
> > Sorry I'm asking potentially stupid questions out of ignorance: why
> > would you want readdir to return `README' when you have `readme'?
> > 
> 
> Because it might have been checked in as README, and since git is case
> sensitive that is what it'll think should be there when it reads the
> directories. If it's not, users get to see
> 
> 	removed: README
> 	untracked: readme

This is a non-issue, then: Windows filesystems are case-preserving, so
if `README' became `readme', someone deliberately renamed it, in which
case it's okay for git to react as above.

> could be an intentional rename, but we don't know for sure.

It _must_ have been an intentional rename.  While years ago there used
to be old DOS programs that could cause such a rename as a side effect
of modifying a file, that time is long gone.  There's no longer a need
to cater to such programs, as even DOS programs can support
case-preserving APIs on Windows.

> To be honest though, there are so many places which do the readdir+stat
> that I don't think it'd be worth factoring it out

Something for Windows users to decide, I guess.  It's not hard to
refactor this, it just needs a motivated volunteer.

> especially since it
> *works* on windows. It's just slow, and only slow compared to various
> unices.

I think only the Linux filesystem is as fast as you say.  But I may be
wrong (did someone compare with *BSD, say?).

> I *think* (correct me if I'm wrong) that git is still faster
> than a whole bunch of other scm's on windows, but to one who's used to
> its performance on Linux that waiting several seconds to scan 10k files
> just feels wrong.

Unless that 10K is a typo and you really meant 100K, I don't think 10K
files should take several seconds to scan on Windows.  I just tried
"find -print" on a directory with 32K files in 4K subdirectories, and
it took 8 sec elapsed with a hot cache.  So 10K files should take at
most 2 seconds, even without optimizing file traversal code.  Doing
the same with native Windows system calls ("dir /s") brings that down
to 4 seconds for 32K files.

On the other hand, what packages have 100K files?  If there's only one
-- the Linux kernel -- then I think this kind of performance is for
all practical purposes unimportant on Windows, because while it is
reasonable to assume that someone would like to use git on Windows,
assuming that someone will develop the Linux kernel on Windows is --
how should I put it -- _really_ far-fetched ;-)

As for speed of file ops ``just feeling wrong'': it's not limited to
git in any way.  You will see the same with "tar -x", with "find" and
even with "cp -r", when you compare Linux filesystems, especially on a
fast 64-bit machine, with comparable Windows operations.  A Windows
user who occasionally works on GNU/Linux already knows that, so seeing
the same in git will not come as a surprise.  Again, I wonder how this
compares with other free OSes, like FreeBSD (unless they use the same
filesystem), and with proprietary Unices, like AIX and Solaris.

Re: Switching from CVS to GIT

[Johannes Sixt]

Shawn O. Pearce schrieb:
> Johannes Sixt <j.sixt@viscovery.net> wrote:
>>> Johannes Sixt <j.sixt@viscovery.net> wrote:
>>>> Unfortunately, "Fetch" does not yet work[*] from within git-gui, so you 
>>>> have to fall back to git-fetch on the command line.
>> It's the scripted fetch that does not work. The symptom is that the output 
             ^^^^^^^^^^^^^^
>> of at least one of the commands (upload-pack, I think, because what I see 
>> is wire protocol) goes to a newly spawned console instead of wherever it 
>> was redirected to.
>>
>> I didn't bother reporting since builtin-fetch is on the way (which will 
>> hopefully make this a moot point) and our team here is comfortable with 
>> calling git fetch on the command line.
> 
> Hmm.  The way the builtin-fetch works this shouldn't happen, but
> I'd appreciate it if you could test and report back before that
> topic merges into master.

This happens with git 1.5.3 plus the git-gui that comes with that.

FWIW, I'm in the process of merging master of git.git into git/mingw.git, 
and then the builtin-fetch series (because on top of that there is my 
fork/exec removal series, which I'd like to adjust for Windows). And *then* 
I'll be able to report back to you.

-- Hannes

Re: Switching from CVS to GIT

[Johannes Sixt]

Eli Zaretskii schrieb:
> If that's the issue, then it's not a good idea to call `stat' and
> `readdir' on Windows at all.  `stat' is a single system call on Posix
> systems, while on Windows it usually needs to go out of its way
> calling half a dozen system services to gather the `struct stat' info.

Thanks to Marius Storm-Olsen we already have a stat replacement that's twice 
as fast as msvcrt's stat. I calls only one API function 
(GetFileAttributesEx, but of course I don't know what's going on under its 
hood), because we need only a small part of struct stat filled in correctly.

-- Hannes

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 01:56:46 -0400 (EDT)
> From: Daniel Barkalow <barkalow@iabervon.org>
> cc: raa.lkml@gmail.com, Johannes.Schindelin@gmx.de, ae@op5.se, 
>     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
> 
> Ah, that's helpful. We don't actually care too much about the particular 
> info in stat; we just want to know quickly if the file has changed, so we 
> can hash only the ones that have been touched and get the actual content 
> changes.

As I wrote in my other message, using native APIs improves performance
by at least a factor of two.

> The tricky thing is that, while the optimization 
> process is running, other programs may be reading the database, so (1) the 
> files that are no longer needed, because better-optimized versions are in 
> place, may be open in another task

Is this because another user might be accessing the database, or are
there other popular use cases that cause this?  If the former, then
this is not terribly important on Windows, since the situation when
more than one user is logged and actively works is quite rare,
basically limited to some scheduled task (the equivalent of a cron
job) running for some user while another one is logged in
interactively.

This might be different on machines that use Cygwin, though.

> Now, it's entirely possible that a completely different database 
> implementation would be better on Windows, but our current one does a lot 
> of creating files under different names, moving them to names where 
> they'll be seen (since this is atomic under POSIX, and partial files are 
> never seen by other tasks). Also, once we have new files in place, we 
> unlink the files that they replace, so that new tasks will use the new 
> ones and tasks that already have old ones open can still get the data out 
> of them. Also, the files generally get mmaped, 

Perhaps mmap introduces complications (I simply don't know), but in
general, as I show elsewhere in this thread, you can do similar things
on Windows, if you use native APIs (as opposed to emulations of Posix,
like `open'), although you may need to rename the old file to get it
out of the way of the new one with the same name, because otherwise
the old file will still be seen, even if deleted, as long as it's open
in some process.

Re: Switching from CVS to GIT

[Daniel Barkalow]

On Tue, 16 Oct 2007, Eli Zaretskii wrote:

> > Date: Tue, 16 Oct 2007 07:14:56 +0200
> > From: Andreas Ericsson <ae@op5.se>
> > CC: Daniel Barkalow <barkalow@iabervon.org>,  raa.lkml@gmail.com, 
> >  Johannes.Schindelin@gmx.de,  tsuna@lrde.epita.fr,  git@vger.kernel.org, 
> >  make-w32@gnu.org
> > 
> > > Sorry I'm asking potentially stupid questions out of ignorance: why
> > > would you want readdir to return `README' when you have `readme'?
> > > 
> > 
> > Because it might have been checked in as README, and since git is case
> > sensitive that is what it'll think should be there when it reads the
> > directories. If it's not, users get to see
> > 
> > 	removed: README
> > 	untracked: readme
> 
> This is a non-issue, then: Windows filesystems are case-preserving, so
> if `README' became `readme', someone deliberately renamed it, in which
> case it's okay for git to react as above.
> 
> > could be an intentional rename, but we don't know for sure.
> 
> It _must_ have been an intentional rename.  While years ago there used
> to be old DOS programs that could cause such a rename as a side effect
> of modifying a file, that time is long gone.  There's no longer a need
> to cater to such programs, as even DOS programs can support
> case-preserving APIs on Windows.

I'm partially worried about cases where checking out a "README" fails to 
replace the name of an existing "readme", or something of that sort.

> > To be honest though, there are so many places which do the readdir+stat
> > that I don't think it'd be worth factoring it out
> 
> Something for Windows users to decide, I guess.  It's not hard to
> refactor this, it just needs a motivated volunteer.
> 
> > especially since it
> > *works* on windows. It's just slow, and only slow compared to various
> > unices.
> 
> I think only the Linux filesystem is as fast as you say.  But I may be
> wrong (did someone compare with *BSD, say?).

I think you're right (nothing else can compete with Linux for doing half a 
million trivial syscalls), but other unixes aren't terrible, either. 
IIRC, on OS X, we had problems when we were doing 4 times as many syscalls 
as necessary, but was fine with that fixed.

> > I *think* (correct me if I'm wrong) that git is still faster
> > than a whole bunch of other scm's on windows, but to one who's used to
> > its performance on Linux that waiting several seconds to scan 10k files
> > just feels wrong.
> 
> Unless that 10K is a typo and you really meant 100K, I don't think 10K
> files should take several seconds to scan on Windows.  I just tried
> "find -print" on a directory with 32K files in 4K subdirectories, and
> it took 8 sec elapsed with a hot cache.  So 10K files should take at
> most 2 seconds, even without optimizing file traversal code.  Doing
> the same with native Windows system calls ("dir /s") brings that down
> to 4 seconds for 32K files.
> 
> On the other hand, what packages have 100K files?  If there's only one
> -- the Linux kernel -- then I think this kind of performance is for
> all practical purposes unimportant on Windows, because while it is
> reasonable to assume that someone would like to use git on Windows,
> assuming that someone will develop the Linux kernel on Windows is --
> how should I put it -- _really_ far-fetched ;-)

Actually, there are a number of projects much bigger than the Linux 
kernel; I think KDE was considering using git, and wanted Windows support, 
and KDE is insanely huge, mostly as a result of having one big repository 
for everything.

> As for speed of file ops ``just feeling wrong'': it's not limited to
> git in any way.  You will see the same with "tar -x", with "find" and
> even with "cp -r", when you compare Linux filesystems, especially on a
> fast 64-bit machine, with comparable Windows operations.  A Windows
> user who occasionally works on GNU/Linux already knows that, so seeing
> the same in git will not come as a surprise.  Again, I wonder how this
> compares with other free OSes, like FreeBSD (unless they use the same
> filesystem), and with proprietary Unices, like AIX and Solaris.

For most things, Unix filesystems are fast enough that the bulk of the 
time is spent elsewhere. "git status" without any changes and a hot cache 
is unusual in being both a common operation and entirely trivial syscalls 
if the filesystem makes it efficient.

The problem we've had is that Linux users who occasionally work on Windows 
say git seems impossibly slow on Windows.

	-Daniel
*This .sig left intentionally blank*

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 16, 2007, at 7:14 AM, Andreas Ericsson wrote:

> Eli Zaretskii wrote:
>>> Date: Mon, 15 Oct 2007 20:45:02 -0400 (EDT)
>>> From: Daniel Barkalow <barkalow@iabervon.org>
>>> cc: Alex Riesen <raa.lkml@gmail.com>, Johannes.Schindelin@gmx.de,  
>>> ae@op5.se,     tsuna@lrde.epita.fr, git@vger.kernel.org, make- 
>>> w32@gnu.org
>>>
>>> I believe the hassle is that readdir doesn't necessarily report a  
>>> README in a directory which is supposed to have a README, when it  
>>> has a readme instead.
>> Sorry I'm asking potentially stupid questions out of ignorance: why
>> would you want readdir to return `README' when you have `readme'?
>
> Because it might have been checked in as README, and since git is case
> sensitive that is what it'll think should be there when it reads the
> directories. If it's not, users get to see
>
> 	removed: README
> 	untracked: readme
>
> and there's really no easy way out of this one, since users on a case-
> sensitive filesystem might be involved in this project too, so it
> could be an intentional rename, but we don't know for sure. Just
> clobbering the in-git file is wrong, but overwriting a file on disk
> is wrong too. git tries hard to not ever lose any data for the user.

Maybe we need a configuration similar to core.autocrlf (which controls
newline conversion) to control filename comparison and normalization?

Most obviously for the case (in-)sensitivity on Windows, but I also
remember the unicode normalization happening on Mac's HFS filesystem
that caused trouble in the past.

	Steffen

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 08:42:53 +0200
> From: Johannes Sixt <j.sixt@viscovery.net>
> Cc: Daniel Barkalow <barkalow@iabervon.org>, raa.lkml@gmail.com,
> 	Johannes.Schindelin@gmx.de, ae@op5.se,
> 	Benoit SIGOURE <tsuna@lrde.epita.fr>,
> 	"git@vger.kernel.org >> Git Mailing List" <git@vger.kernel.org>,
> 	Make Windows <make-w32@gnu.org>
> 
> Thanks to Marius Storm-Olsen we already have a stat replacement that's twice 
> as fast as msvcrt's stat. I calls only one API function 
> (GetFileAttributesEx, but of course I don't know what's going on under its 
> hood), because we need only a small part of struct stat filled in correctly.

Yes, I've seen that.  What I'm saying is that you can combine
`readdir' with `stat' in one API call (FindFirstFile/FindNextFile),
which will both read the directory and return you the attributes you
get from `stat'.  Think about `readdir' that brings you mode bits and
modification time together with the name, as some modern systems do.

Re: Switching from CVS to GIT

[Nguyen Thai Ngoc Duy]

On 10/16/07, Eli Zaretskii <eliz@gnu.org> wrote:
> > Date: Tue, 16 Oct 2007 09:24:50 +0700
> > From: "Nguyen Thai Ngoc Duy" <pclouds@gmail.com>
> > Cc: "Git Mailing List" <git@vger.kernel.org>, "Alex Riesen" <raa.lkml@gmail.com>,
> >       "Eli Zaretskii" <eliz@gnu.org>, "Andreas Ericsson" <ae@op5.se>,
> >       tsuna@lrde.epita.fr,
> >       "Johannes Schindelin" <Johannes.Schindelin@gmx.de>
> >
> > I did try hard to find the legendary
> > zsh for Windows before giving up and porting busybox's ash instead.
>
> Where can one find this port of busybox's ash?
>

http://repo.or.cz/w/git/pclouds.git?a=shortlog;h=gitbox

In directory box/shell.
-- 
Duy

Re: Switching from CVS to GIT

[Peter Karlsson]

> - it is the only OS in the world with multi-root (/a/b/c and /a/b/c
>   can be not the same, depending on what current "drive" is) and
>   multi-cwd, which hasn't had formed itself into a problem yet, but
>   surely will

It's not the only OS with drive letters (although I don't see Git
coming to my Symbian OS phone any time soon), but there is only one
root. The problem is that it isn't addressable in the file system, and
that the concept of what is the root is different depending on what you
ask (either it's above the drive letters, or "My Computer").

You can create a search path rooted in "My Computer" if you want (using
shell APIs), but you probably can't get a readable text representation
of it.

> - it has only one argument (limited in size) passed to started
>   programs, which means that there is no possible way to safely pass
>   file and text arguments on command line (more than one, that is)

Well, there are many other ways of passing arguments than on the
command line, but they are probably difficult to access from console
applications (things like DDE or whatever the current implementation is
called).

-- 
\\// Peter - http://www.softwolves.pp.se/

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 17:09:08 +0700
> From: "Nguyen Thai Ngoc Duy" <pclouds@gmail.com>
> Cc: prohaska@zib.de, git@vger.kernel.org, raa.lkml@gmail.com, ae@op5.se, 
> 	tsuna@lrde.epita.fr, Johannes.Schindelin@gmx.de
> 
> > Where can one find this port of busybox's ash?
> >
> 
> http://repo.or.cz/w/git/pclouds.git?a=shortlog;h=gitbox
> 
> In directory box/shell.

Thanks!

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

[by explicit request culling make-w32 from the Cc list]

On Tue, 16 Oct 2007, Eli Zaretskii wrote:

> > Date: Tue, 16 Oct 2007 07:14:56 +0200
> > From: Andreas Ericsson <ae@op5.se>
> > CC: Daniel Barkalow <barkalow@iabervon.org>,  raa.lkml@gmail.com, 
> >  Johannes.Schindelin@gmx.de,  tsuna@lrde.epita.fr,  git@vger.kernel.org, 
> >  make-w32@gnu.org
> > 
> > > Sorry I'm asking potentially stupid questions out of ignorance: why
> > > would you want readdir to return `README' when you have `readme'?
> > > 
> > 
> > Because it might have been checked in as README, and since git is case
> > sensitive that is what it'll think should be there when it reads the
> > directories. If it's not, users get to see
> > 
> > 	removed: README
> > 	untracked: readme
> 
> This is a non-issue, then: Windows filesystems are case-preserving, so 
> if `README' became `readme', someone deliberately renamed it, in which 
> case it's okay for git to react as above.

No, it is not.  On FAT filesystems, for example, I experienced Windows 
happily naming a file "head" which was created under then name "HEAD".

This is the single reason why I cannot have non-bare repositories on a USB 
stick.

> > could be an intentional rename, but we don't know for sure.
> 
> It _must_ have been an intentional rename.

No.  It can also be the output of a program which deletes the file first, 
and then (since the filesystem is so "conveniently" case insensitive) 
creates it again, with a lowercase filename.

And don't you tell me that there are no such programs.  I have to use 
them, and they are closed source.

Sigh.

> > To be honest though, there are so many places which do the 
> > readdir+stat that I don't think it'd be worth factoring it out
> 
> Something for Windows users to decide, I guess.  It's not hard to 
> refactor this, it just needs a motivated volunteer.

You?

> > I *think* (correct me if I'm wrong) that git is still faster
> > than a whole bunch of other scm's on windows, but to one who's used to
> > its performance on Linux that waiting several seconds to scan 10k files
> > just feels wrong.
> 
> Unless that 10K is a typo and you really meant 100K, I don't think 10K
> files should take several seconds to scan on Windows.  I just tried
> "find -print" on a directory with 32K files in 4K subdirectories, and
> it took 8 sec elapsed with a hot cache.  So 10K files should take at
> most 2 seconds, even without optimizing file traversal code.  Doing
> the same with native Windows system calls ("dir /s") brings that down
> to 4 seconds for 32K files.

On Linux, I would have hit Control-C already.  Such an operation typically 
takes less than 0.1 seconds.

> On the other hand, what packages have 100K files?

Mozilla, KDE, OpenOffice.org, X.org, ....

Ciao,
Dscho

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

[culled make-w32 list by explicit request]

On Tue, 16 Oct 2007, Steffen Prohaska wrote:

> On Oct 16, 2007, at 7:14 AM, Andreas Ericsson wrote:
> 
> > Eli Zaretskii wrote:
> > > > Date: Mon, 15 Oct 2007 20:45:02 -0400 (EDT)
> > > > From: Daniel Barkalow <barkalow@iabervon.org>
> > > > cc: Alex Riesen <raa.lkml@gmail.com>, Johannes.Schindelin@gmx.de,
> > > > ae@op5.se,     tsuna@lrde.epita.fr, git@vger.kernel.org,
> > > > make-w32@gnu.org
> > > > 
> > > > I believe the hassle is that readdir doesn't necessarily report a 
> > > > README in a directory which is supposed to have a README, when it 
> > > > has a readme instead.
> > > Sorry I'm asking potentially stupid questions out of ignorance: why 
> > > would you want readdir to return `README' when you have `readme'?
> > 
> > Because it might have been checked in as README, and since git is case 
> > sensitive that is what it'll think should be there when it reads the 
> > directories. If it's not, users get to see
> > 
> > 	removed: README
> > 	untracked: readme
> > 
> > and there's really no easy way out of this one, since users on a case- 
> > sensitive filesystem might be involved in this project too, so it 
> > could be an intentional rename, but we don't know for sure. Just 
> > clobbering the in-git file is wrong, but overwriting a file on disk is 
> > wrong too. git tries hard to not ever lose any data for the user.
> 
> Maybe we need a configuration similar to core.autocrlf (which controls 
> newline conversion) to control filename comparison and normalization?
> 
> Most obviously for the case (in-)sensitivity on Windows, but I also 
> remember the unicode normalization happening on Mac's HFS filesystem 
> that caused trouble in the past.

Robin Rosenberg has some preliminary code for that.  The idea is to wrap 
all filesystem operations in cache.h, and do a filename normalisation 
first.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Peter Karlsson]

> No, it is not.  On FAT filesystems, for example, I experienced Windows 
> happily naming a file "head" which was created under then name "HEAD".

If you create a file name with only capital letters, I believe Explorer
and the file browser will display the name with an initial capital, and
the rest lowercase, or in all lowercase. IIRC, this is because such a
file is saved with only an MS-DOS name and no LFN entry, and those have
special rules to avoid them being displayed in all-uppercase.

I believe it is possible to create a LFN entry for such a file, but I
can't remember right now how to do it.

-- 
\\// Peter - http://www.softwolves.pp.se/

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

[culled make-w32, as per explicit request]

On Tue, 16 Oct 2007, Eli Zaretskii wrote:

> > Date: Tue, 16 Oct 2007 01:56:46 -0400 (EDT)
> > From: Daniel Barkalow <barkalow@iabervon.org>
> > cc: raa.lkml@gmail.com, Johannes.Schindelin@gmx.de, ae@op5.se, 
> >     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
> > 
> > Ah, that's helpful. We don't actually care too much about the 
> > particular info in stat; we just want to know quickly if the file has 
> > changed, so we can hash only the ones that have been touched and get 
> > the actual content changes.
> 
> As I wrote in my other message, using native APIs improves performance 
> by at least a factor of two.

Somehow this does not appeal to my "portability is good" side.  You know, 
if we had to do such trickeries for every platform we support, we'd soon 
be as big as Subversion *cough*.

For me, this is the most annoying part about programming Win32.  They went 
out of their way to make it incompatible with everything else, and as a 
consequence it is a PITA to maintain crossplatform programs.

> > The tricky thing is that, while the optimization process is running, 
> > other programs may be reading the database, so (1) the files that are 
> > no longer needed, because better-optimized versions are in place, may 
> > be open in another task
> 
> Is this because another user might be accessing the database, or are 
> there other popular use cases that cause this?  If the former, then this 
> is not terribly important on Windows, since the situation when more than 
> one user is logged and actively works is quite rare, basically limited 
> to some scheduled task (the equivalent of a cron job) running for some 
> user while another one is logged in interactively.

Quite to the contrary.  Explorer often accesses files it should not lock.  
On the machine I test msysGit on, this is the most common reason for a 
test case to fail: it cannot delete the temporary directory, which 
_should_ be unused.  Indeed, a second after that, it _is_ unused.

Ciao,
Dscho

Re: Switching from CVS to GIT

[David Kastrup]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi,
>
> [culled make-w32, as per explicit request]
>
> On Tue, 16 Oct 2007, Eli Zaretskii wrote:
>
>> > Date: Tue, 16 Oct 2007 01:56:46 -0400 (EDT)
>> > From: Daniel Barkalow <barkalow@iabervon.org>
>> > cc: raa.lkml@gmail.com, Johannes.Schindelin@gmx.de, ae@op5.se, 
>> >     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
>> > 
>> > Ah, that's helpful. We don't actually care too much about the 
>> > particular info in stat; we just want to know quickly if the file has 
>> > changed, so we can hash only the ones that have been touched and get 
>> > the actual content changes.
>> 
>> As I wrote in my other message, using native APIs improves
>> performance by at least a factor of two.
>
> Somehow this does not appeal to my "portability is good" side.  You
> know, if we had to do such trickeries for every platform we support,
> we'd soon be as big as Subversion *cough*.

With a reasonable way of factoring out things, the per-platform
overhead should be tolerable.

> For me, this is the most annoying part about programming Win32.
> They went out of their way to make it incompatible with everything
> else, and as a consequence it is a PITA to maintain crossplatform
> programs.

Well, they certainly score high on the "almost, but not quite,
entirely unlike tea" metric.  Enough compatibility to make it into
projects with cross-platform specifications, and enough
incompatibility to make it inconvenient to support anything but
Windows once they made it into the door.

-- 
David Kastrup

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 13:29:41 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Andreas Ericsson <ae@op5.se>, barkalow@iabervon.org, raa.lkml@gmail.com, 
>     tsuna@lrde.epita.fr, git@vger.kernel.org
> 
> > > 	removed: README
> > > 	untracked: readme
> > 
> > This is a non-issue, then: Windows filesystems are case-preserving, so 
> > if `README' became `readme', someone deliberately renamed it, in which 
> > case it's okay for git to react as above.
> 
> No, it is not.  On FAT filesystems, for example, I experienced Windows 
> happily naming a file "head" which was created under then name "HEAD".

What program did that, and how did you see that the file was named
"head" instead of "HEAD"?  (The latter question is because Explorer,
for example, does not show the file names exactly like they are
written in the directory, it capitalises them.  But this is
application-level code; in the directory the file names are written
like you gave them in the argument to whatever "create file" API you
used.

> No.  It can also be the output of a program which deletes the file first, 
> and then (since the filesystem is so "conveniently" case insensitive) 
> creates it again, with a lowercase filename.
> 
> And don't you tell me that there are no such programs.  I have to use 
> them, and they are closed source.

Can you name them?

> > Something for Windows users to decide, I guess.  It's not hard to 
> > refactor this, it just needs a motivated volunteer.
> 
> You?

Maybe some day.

> > Unless that 10K is a typo and you really meant 100K, I don't think 10K
> > files should take several seconds to scan on Windows.  I just tried
> > "find -print" on a directory with 32K files in 4K subdirectories, and
> > it took 8 sec elapsed with a hot cache.  So 10K files should take at
> > most 2 seconds, even without optimizing file traversal code.  Doing
> > the same with native Windows system calls ("dir /s") brings that down
> > to 4 seconds for 32K files.
> 
> On Linux, I would have hit Control-C already.  Such an operation typically 
> takes less than 0.1 seconds.

We were not comparing Linux with Windows, we were talking about
Windows user experience.  On Windows 4 seconds is not too long.

Re: Switching from CVS to GIT

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Tue, 16 Oct 2007 13:29:41 +0100 (BST)
>> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
>> 
>> On Linux, I would have hit Control-C already.  Such an operation
>> typically takes less than 0.1 seconds.
>
> We were not comparing Linux with Windows, we were talking about
> Windows user experience.  On Windows 4 seconds is not too long.

If it is accompanied by some animation of papers winging across the
screen or a progress bar or similar.  Otherwise people might think
that Windows crashed and reboot.

Anyway, the problem is that 4 seconds for 32K files means 40 seconds
(at least) for 320K files.  At some point of time, things become
really unpleasant.

-- 
David Kastrup

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 13:38:59 +0100 (CET)
> From: Peter Karlsson <peter@softwolves.pp.se>
> cc: Eli Zaretskii <eliz@gnu.org>, Andreas Ericsson <ae@op5.se>,
>    barkalow@iabervon.org, raa.lkml@gmail.com, tsuna@lrde.epita.fr,
>    git@vger.kernel.org
> 
> If you create a file name with only capital letters, I believe Explorer
> and the file browser will display the name with an initial capital, and
> the rest lowercase, or in all lowercase.

That's true, but the names are only displayed like that, what's on
disk is not changed in any way.

> IIRC, this is because such a
> file is saved with only an MS-DOS name and no LFN entry, and those have
> special rules to avoid them being displayed in all-uppercase.

I don't think this true anymore in modern versions of Windows, but I
might be mistaken.  In any case, the reason for the Explorer behavior
is immaterial for us, what matters is that file names on disk preserve
the lettercase of the program that created them, at least AFAIK.

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Tue, 16 Oct 2007, Eli Zaretskii wrote:

> > Date: Tue, 16 Oct 2007 13:29:41 +0100 (BST)
> > From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > cc: Andreas Ericsson <ae@op5.se>, barkalow@iabervon.org, raa.lkml@gmail.com, 
> >     tsuna@lrde.epita.fr, git@vger.kernel.org
> > 
> > > > 	removed: README
> > > > 	untracked: readme
> > > 
> > > This is a non-issue, then: Windows filesystems are case-preserving, so 
> > > if `README' became `readme', someone deliberately renamed it, in which 
> > > case it's okay for git to react as above.
> > 
> > No, it is not.  On FAT filesystems, for example, I experienced Windows 
> > happily naming a file "head" which was created under then name "HEAD".
> 
> What program did that, and how did you see that the file was named
> "head" instead of "HEAD"?

Git and ... Git.

> > > Something for Windows users to decide, I guess.  It's not hard to 
> > > refactor this, it just needs a motivated volunteer.
> > 
> > You?
> 
> Maybe some day.

Cool.

> > > Unless that 10K is a typo and you really meant 100K, I don't think 
> > > 10K files should take several seconds to scan on Windows.  I just 
> > > tried "find -print" on a directory with 32K files in 4K 
> > > subdirectories, and it took 8 sec elapsed with a hot cache.  So 10K 
> > > files should take at most 2 seconds, even without optimizing file 
> > > traversal code.  Doing the same with native Windows system calls 
> > > ("dir /s") brings that down to 4 seconds for 32K files.
> > 
> > On Linux, I would have hit Control-C already.  Such an operation 
> > typically takes less than 0.1 seconds.
> 
> We were not comparing Linux with Windows, we were talking about Windows 
> user experience.  On Windows 4 seconds is not too long.

Well, I was talking about user experience.  In this case of a user who 
happens to be on Windows, but knows Linux' speed.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 16, 2007, at 2:33 PM, Johannes Schindelin wrote:

>> Maybe we need a configuration similar to core.autocrlf (which  
>> controls
>> newline conversion) to control filename comparison and normalization?
>>
>> Most obviously for the case (in-)sensitivity on Windows, but I also
>> remember the unicode normalization happening on Mac's HFS filesystem
>> that caused trouble in the past.
>
> Robin Rosenberg has some preliminary code for that.  The idea is to  
> wrap
> all filesystem operations in cache.h, and do a filename normalisation
> first.

At that point we could add a safety check. Paths that differ only by
case, or whitespace, or ... (add general and project specific rules  
here)
should be denied. This would guarantee that tree objects can always be
checked out. Even if the filesystem capabilities are limited.

Robin, what do you think?

	Steffen

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 13:39:12 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Daniel Barkalow <barkalow@iabervon.org>, raa.lkml@gmail.com, ae@op5.se, 
>     tsuna@lrde.epita.fr, git@vger.kernel.org
> 
> > As I wrote in my other message, using native APIs improves performance 
> > by at least a factor of two.
> 
> Somehow this does not appeal to my "portability is good" side.  You know, 
> if we had to do such trickeries for every platform we support, we'd soon 
> be as big as Subversion *cough*.

You have to decide whether you care about performance enough to do
that or not.  If you do, then introducing file I/O abstractions at
higher level than the normal ``use-library-functions'' method is not
such a hard problem, and doesn't make the binary larger because each
platform gets only its own backend.  In practice, I have found that in
most cases a few well-designed and strategically placed macros is all
you need.

> For me, this is the most annoying part about programming Win32.  They went 
> out of their way to make it incompatible with everything else, and as a 
> consequence it is a PITA to maintain crossplatform programs.

Portability is a two-way street.  A program that wasn't designed to be
portable will by definition be hard to port.  To me, what's annoying
is a program that was designed around a single-OS model of APIs.

Cross-platform programs are not that hard if you design them to be
like that from the ground up.  I'm working for a firm that does that
for a living: we develop software that compiles and runs on Windows
and Linux from the same source.

> Explorer often accesses files it should not lock.  
> On the machine I test msysGit on, this is the most common reason for a 
> test case to fail: it cannot delete the temporary directory, which 
> _should_ be unused.  Indeed, a second after that, it _is_ unused.

One more reason not to launch Explorer, if you ask me ;-)  But maybe
you have valid reasons to do that.  All I can say is that I never saw
such problems, but then I don't usually run programs that rewrite
files in a frenzy.

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Tue, 16 Oct 2007, Steffen Prohaska wrote:

> On Oct 16, 2007, at 2:33 PM, Johannes Schindelin wrote:
> 
> > > Maybe we need a configuration similar to core.autocrlf (which controls
> > > newline conversion) to control filename comparison and normalization?
> > > 
> > > Most obviously for the case (in-)sensitivity on Windows, but I also
> > > remember the unicode normalization happening on Mac's HFS filesystem
> > > that caused trouble in the past.
> > 
> > Robin Rosenberg has some preliminary code for that.  The idea is to wrap
> > all filesystem operations in cache.h, and do a filename normalisation
> > first.
> 
> At that point we could add a safety check. Paths that differ only by
> case, or whitespace, or ... (add general and project specific rules here)
> should be denied. This would guarantee that tree objects can always be
> checked out. Even if the filesystem capabilities are limited.

This would be an independent change.  The method I talked about only ever 
looks at one filename, never what is already there.

What you want would probably be all too easy with a pre-commit hook.  No 
need to clutter the git-core with code that is usually not needed (you'd 
only ever activate it on Linux when other developers use Windows or 
MacOSX).

Ciao,
Dscho

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Tue, 16 Oct 2007, Eli Zaretskii wrote:

> > Date: Tue, 16 Oct 2007 13:39:12 +0100 (BST)
> > From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > cc: Daniel Barkalow <barkalow@iabervon.org>, raa.lkml@gmail.com, ae@op5.se, 
> >     tsuna@lrde.epita.fr, git@vger.kernel.org
> > 
> > > As I wrote in my other message, using native APIs improves 
> > > performance by at least a factor of two.
> > 
> > Somehow this does not appeal to my "portability is good" side.  You 
> > know, if we had to do such trickeries for every platform we support, 
> > we'd soon be as big as Subversion *cough*.
> 
> You have to decide whether you care about performance enough to do
> that or not.

Yes, I know that we'll have to use more special casing of Windows for 
performance reasons.  I was only lamenting that it would not need to be 
that way.  Just ignore me.

> > For me, this is the most annoying part about programming Win32.  They 
> > went out of their way to make it incompatible with everything else, 
> > and as a consequence it is a PITA to maintain crossplatform programs.
> 
> Portability is a two-way street.  A program that wasn't designed to be 
> portable will by definition be hard to port.  To me, what's annoying is 
> a program that was designed around a single-OS model of APIs.

You're obviously not talking about git here.

> > Explorer often accesses files it should not lock.  On the machine I 
> > test msysGit on, this is the most common reason for a test case to 
> > fail: it cannot delete the temporary directory, which _should_ be 
> > unused.  Indeed, a second after that, it _is_ unused.
> 
> One more reason not to launch Explorer, if you ask me ;-)  But maybe you 
> have valid reasons to do that.  All I can say is that I never saw such 
> problems, but then I don't usually run programs that rewrite files in a 
> frenzy.

Funny.  Last time I checked the toolbar went away, as well as the desktop, 
when I killed explorer.exe.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 16, 2007, at 3:21 PM, Johannes Schindelin wrote:

> Hi,
>
> On Tue, 16 Oct 2007, Steffen Prohaska wrote:
>
>> On Oct 16, 2007, at 2:33 PM, Johannes Schindelin wrote:
>>
>>>> Maybe we need a configuration similar to core.autocrlf (which  
>>>> controls
>>>> newline conversion) to control filename comparison and  
>>>> normalization?
>>>>
>>>> Most obviously for the case (in-)sensitivity on Windows, but I also
>>>> remember the unicode normalization happening on Mac's HFS  
>>>> filesystem
>>>> that caused trouble in the past.
>>>
>>> Robin Rosenberg has some preliminary code for that.  The idea is  
>>> to wrap
>>> all filesystem operations in cache.h, and do a filename  
>>> normalisation
>>> first.
>>
>> At that point we could add a safety check. Paths that differ only by
>> case, or whitespace, or ... (add general and project specific  
>> rules here)
>> should be denied. This would guarantee that tree objects can  
>> always be
>> checked out. Even if the filesystem capabilities are limited.
>
> This would be an independent change.  The method I talked about  
> only ever
> looks at one filename, never what is already there.

Oh, hmm ... obviously, ... if I think about it ;)


> What you want would probably be all too easy with a pre-commit  
> hook.  No
> need to clutter the git-core with code that is usually not needed  
> (you'd
> only ever activate it on Linux when other developers use Windows or
> MacOSX).

Personally, I'd be very happy if git enforced the minimal consent  
between
(supported) filesystems and provided a system to guarantee that I can  
only
create tree objects that can be checked out on all (supported)  
filesystems.

I'd _always_ switch on such a mechanism. I think the idea of relying on
filenames that only differ by whitespace or case is insane  
independent of
the capabilities of the filesystem used. Humans hardly see such  
differences.
There may be other characters that should be avoided purely for  
technical reasons. If git checked this, too, I'd be happy.

An update hook is only very loosely coupled to git. I'd prefer a tighter
integration. 'git add <something>' should immediately report the  
problem.
But, maybe I'll try a commit hook first.

	Steffen

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Tue, 16 Oct 2007, Steffen Prohaska wrote:

> On Oct 16, 2007, at 3:21 PM, Johannes Schindelin wrote:
> 
> > What you want would probably be all too easy with a pre-commit hook.  
> > No need to clutter the git-core with code that is usually not needed 
> > (you'd only ever activate it on Linux when other developers use 
> > Windows or MacOSX).
> 
> Personally, I'd be very happy if git enforced the minimal consent 
> between (supported) filesystems and provided a system to guarantee that 
> I can only create tree objects that can be checked out on all 
> (supported) filesystems.

This will not happen.  In the Linux kernel, there were exactly such cases, 
where the filenames differed only in case.

Also, some projects I checked out (notably Perl) assume that Makefile is 
different from makefile.

So I think this will always be something Windows users would wish to 
impose onto others, while Linux users would always refuse.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Steffen Prohaska]

On Oct 16, 2007, at 4:14 PM, Johannes Schindelin wrote:

> On Tue, 16 Oct 2007, Steffen Prohaska wrote:
>
>> On Oct 16, 2007, at 3:21 PM, Johannes Schindelin wrote:
>>
>>> What you want would probably be all too easy with a pre-commit hook.
>>> No need to clutter the git-core with code that is usually not needed
>>> (you'd only ever activate it on Linux when other developers use
>>> Windows or MacOSX).
>>
>> Personally, I'd be very happy if git enforced the minimal consent
>> between (supported) filesystems and provided a system to guarantee  
>> that
>> I can only create tree objects that can be checked out on all
>> (supported) filesystems.
>
> This will not happen.  In the Linux kernel, there were exactly such  
> cases,
> where the filenames differed only in case.
>
> Also, some projects I checked out (notably Perl) assume that  
> Makefile is
> different from makefile.

weird Linux and Perl world, indeed.


> So I think this will always be something Windows users

and Mac users, who also need to deal with a case-preserving,
but case-insensitive filesystem.

> would wish to
> impose onto others, while Linux users would always refuse.

maybe Linux kernel developers. When I work on Linux, I'd be happy
if git saved me from creating directories containing Readme and
readme at the same time.

	Steffen

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 14:24:34 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: barkalow@iabervon.org, raa.lkml@gmail.com, ae@op5.se, tsuna@lrde.epita.fr, 
>     git@vger.kernel.org
> 
> Funny.  Last time I checked the toolbar went away, as well as the desktop, 
> when I killed explorer.exe.

That's a ``feature'': Explorer is the parent of all the desktop
display.  Kinda like the login shell on Unix: if you kill it, there
goes your whole session.  Except that on Windows, the OS pays
attention and restarts Explorer right away to get you back in
business.  (In first versions of Windows, there was no restarting of
Explorer, so if you killed it, you needed to reboot :-()

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 15:14:19 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: Git Mailing List <git@vger.kernel.org>, 
>     Robin Rosenberg <robin.rosenberg.lists@dewire.com>, 
>     Eli Zaretskii <eliz@gnu.org>, Daniel Barkalow <barkalow@iabervon.org>, 
>     Alex Riesen <raa.lkml@gmail.com>, tsuna@lrde.epita.fr, 
>     Andreas Ericsson <ae@op5.se>
> 
> So I think this will always be something Windows users would wish to 
> impose onto others, while Linux users would always refuse.

Here is one Windows user that will never try to impose that ;-)

However, it's possible that an option could be supported to do that
when the user particularly wants that in her database.  Just a
thought...

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Tue, 16 Oct 2007, Johannes Sixt wrote:

> Shawn O. Pearce schrieb:
> > Johannes Sixt <j.sixt@viscovery.net> wrote:
> > > Unfortunately, "Fetch" does not yet work[*] from within git-gui, so you
> > > have to fall back to git-fetch on the command line.
> > > 
> > > [*] Note the distinction between "not available" and "does not work".
> > 
> > What's broken?  Is this that Git protocol dump showing up in
> > git-gui's console window thing?
> > 
> > Are you using the C based fetch that is in git.git's next branch,
> > or the shell script based one that is in master?  Which Tcl/Tk
> > version are you using to run git-gui?
> 
> It's the scripted fetch that does not work. The symptom is that the output of
> at least one of the commands (upload-pack, I think, because what I see is
> wire protocol) goes to a newly spawned console instead of wherever it was
> redirected to.
> 
> I didn't bother reporting since builtin-fetch is on the way (which will
> hopefully make this a moot point) and our team here is comfortable with
> calling git fetch on the command line.

Note that Issue 57 on msysgit.googlecode.com talks exactly about the same 
issue.

Ciao,
Dscho

Re: Switching from CVS to GIT

[Johannes Schindelin]

Hi,

On Tue, 16 Oct 2007, Eli Zaretskii wrote:

> > Date: Tue, 16 Oct 2007 14:24:34 +0100 (BST)
> > From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > cc: barkalow@iabervon.org, raa.lkml@gmail.com, ae@op5.se, tsuna@lrde.epita.fr, 
> >     git@vger.kernel.org
> > 
> > Funny.  Last time I checked the toolbar went away, as well as the desktop, 
> > when I killed explorer.exe.
> 
> That's a ``feature'': Explorer is the parent of all the desktop
> display.  Kinda like the login shell on Unix: if you kill it, there
> goes your whole session.  Except that on Windows, the OS pays
> attention and restarts Explorer right away to get you back in
> business.  (In first versions of Windows, there was no restarting of
> Explorer, so if you killed it, you needed to reboot :-()

I kinda knew that.  But what's now with your recommendation to never run 
Explorer?

Ciao,
Dscho

Re: Switching from CVS to GIT

[Eli Zaretskii]

> Date: Tue, 16 Oct 2007 16:18:10 +0100 (BST)
> From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> cc: barkalow@iabervon.org, raa.lkml@gmail.com, ae@op5.se, tsuna@lrde.epita.fr, 
>     git@vger.kernel.org
> 
> > That's a ``feature'': Explorer is the parent of all the desktop
> > display.  Kinda like the login shell on Unix: if you kill it, there
> > goes your whole session.  Except that on Windows, the OS pays
> > attention and restarts Explorer right away to get you back in
> > business.  (In first versions of Windows, there was no restarting of
> > Explorer, so if you killed it, you needed to reboot :-()
> 
> I kinda knew that.  But what's now with your recommendation to never run 
> Explorer?

I meant not to open "My Computer" and use the GUI for browsing the
directories.  If you meant that the touching of files is done even if
you don't open the GUI, then just ignore my advice: Explorer cannot be
killed.  I'm surprised that it touches files and directories,
though...

RE: Switching from CVS to GIT

[Dave Korn]

[-- Attachment #1: Type: text/plain, Size: 803 bytes --]

On 16 October 2007 07:25, Eli Zaretskii wrote:

> On the other hand, what packages have 100K files?  If there's only one
> -- the Linux kernel -- then I think this kind of performance is for
> all practical purposes unimportant on Windows, because while it is
> reasonable to assume that someone would like to use git on Windows,
> assuming that someone will develop the Linux kernel on Windows is --
> how should I put it -- _really_ far-fetched ;-)

  Hi there!  Did someone call?

  Cross-development in general isn't what I'd call "far-fetched", and there's
no law of cross-development that says the host has to be the same platform as
the target.  :-)[*]

    cheers,
      DaveK

[*] - this smiley sponsored by the Department of the Bleedin' Obvious.
-- 
Can't think of a witty .sigline today....

[-- Attachment #2: developed-on-windows.diff --]
[-- Type: application/octet-stream, Size: 3660 bytes --]

Index: firmware_class.c
===================================================================
RCS file: /sources/repository/external_source/linux/linux-2.6.12.2/drivers/base/firmware_class.c,v
retrieving revision 1.1
retrieving revision 1.2
diff -p -u -r1.1 -r1.2
--- firmware_class.c	17 Jan 2006 16:49:35 -0000	1.1
+++ firmware_class.c	15 Feb 2006 14:01:29 -0000	1.2
@@ -31,6 +31,7 @@ enum {
 };
 
 static int loading_timeout = 10;	/* In seconds */
+static char grow_faster = 1;      /* Boolean */
 
 /* fw_lock could be moved to 'struct firmware_priv' but since it is just
  * guarding for corner cases a global lock should be OK */
@@ -79,6 +80,28 @@ firmware_timeout_store(struct class *cla
 
 static CLASS_ATTR(timeout, 0644, firmware_timeout_show, firmware_timeout_store);
 
+static ssize_t
+firmware_grow_faster_show(struct class *class, char *buf)
+{
+	return sprintf(buf, "%d\n", grow_faster);
+}
+
+/**
+ * firmware_grow_faster_store:
+ * Description:
+ *	Sets or clears a flag that causes the reallocate routine to
+ *	grow the firmware buffer size more or less quickly.
+ *  
+ **/
+static ssize_t
+firmware_grow_faster_store(struct class *class, const char *buf, size_t count)
+{
+	grow_faster = simple_strtol(buf, NULL, 10) != 0;
+	return count;
+}
+
+static CLASS_ATTR(grow_faster, 0644, firmware_grow_faster_show, firmware_grow_faster_store);
+
 static void  fw_class_dev_release(struct class_device *class_dev);
 int firmware_class_hotplug(struct class_device *dev, char **envp,
 			   int num_envp, char *buffer, int buffer_size);
@@ -198,18 +221,27 @@ static int
 fw_realloc_buffer(struct firmware_priv *fw_priv, int min_size)
 {
 	u8 *new_data;
+  int new_size;
 
 	if (min_size <= fw_priv->alloc_size)
 		return 0;
 
-	new_data = vmalloc(fw_priv->alloc_size + PAGE_SIZE);
+#define ONE_MEG (1024 * 1024)
+
+  new_size = grow_faster 
+    ? ((fw_priv->alloc_size >= ONE_MEG)
+      ? (fw_priv->alloc_size + ONE_MEG)
+      : ((fw_priv->alloc_size >= PAGE_SIZE) ? (fw_priv->alloc_size * 2) : PAGE_SIZE))
+    : (fw_priv->alloc_size + PAGE_SIZE);
+  new_data = vmalloc (new_size);
 	if (!new_data) {
-		printk(KERN_ERR "%s: unable to alloc buffer\n", __FUNCTION__);
+		printk(KERN_ERR "%s: unable to alloc buffer old size %d new size %d\n",
+      __FUNCTION__, fw_priv->alloc_size, new_size);
 		/* Make sure that we don't keep incomplete data */
 		fw_load_abort(fw_priv);
 		return -ENOMEM;
 	}
-	fw_priv->alloc_size += PAGE_SIZE;
+  fw_priv->alloc_size = new_size;
 	if (fw_priv->fw->data) {
 		memcpy(new_data, fw_priv->fw->data, fw_priv->fw->size);
 		vfree(fw_priv->fw->data);
@@ -249,6 +281,13 @@ firmware_data_write(struct kobject *kobj
 		goto out;
 
 	memcpy(fw->data + offset, buffer, count);
+  /*  A successful write should cause us to reset the timeout
+  delay, as very large firmware files might take a while to
+  send through the sysfs file.  We have the fw_lock taken at
+  the moment but the timeout function doesn't lock as it only
+  has to set a single volatile bit, so we're ok to mod it. */
+  if (timer_pending (&fw_priv->timeout))
+    mod_timer (&fw_priv->timeout, jiffies + loading_timeout * HZ);
 
 	fw->size = max_t(size_t, offset + count, fw->size);
 	retval = count;
@@ -568,6 +607,12 @@ firmware_class_init(void)
 		       __FUNCTION__);
 		class_unregister(&firmware_class);
 	}
+	error = class_create_file(&firmware_class, &class_attr_grow_faster);
+	if (error) {
+		printk(KERN_ERR "%s: class_create_file failed\n",
+		       __FUNCTION__);
+		class_unregister(&firmware_class);
+	}
 	return error;
 
 }

[-- Attachment #3: Type: text/plain, Size: 134 bytes --]

_______________________________________________
Make-w32 mailing list
Make-w32@gnu.org
http://lists.gnu.org/mailman/listinfo/make-w32

Re: Switching from CVS to GIT

[David Brown]

On Tue, Oct 16, 2007 at 02:25:21AM -0400, Eli Zaretskii wrote:

>On the other hand, what packages have 100K files?  If there's only one
>-- the Linux kernel -- then I think this kind of performance is for
>all practical purposes unimportant on Windows, because while it is
>reasonable to assume that someone would like to use git on Windows,
>assuming that someone will develop the Linux kernel on Windows is --
>how should I put it -- _really_ far-fetched ;-)

Oh, I wish others could think this clearly.  Quoting a serious line off of
a task list at an unnamed company:

   - Make Linux kernel compile under windows.

I don't think it will move past just being a wish list item, but there seem
to be people that think it should be done.

Admittedly, they don't want developers doing it on windows, but want to
integrate kernel building into a windows-heavy build and release process.

David

Re: Switching from CVS to GIT

[Nicolas Pitre]

On Tue, 16 Oct 2007, David Brown wrote:

> On Tue, Oct 16, 2007 at 02:25:21AM -0400, Eli Zaretskii wrote:
> 
> > On the other hand, what packages have 100K files?  If there's only one
> > -- the Linux kernel -- then I think this kind of performance is for
> > all practical purposes unimportant on Windows, because while it is
> > reasonable to assume that someone would like to use git on Windows,
> > assuming that someone will develop the Linux kernel on Windows is --
> > how should I put it -- _really_ far-fetched ;-)
> 
> Oh, I wish others could think this clearly.  Quoting a serious line off of
> a task list at an unnamed company:
> 
>   - Make Linux kernel compile under windows.
> 
> I don't think it will move past just being a wish list item, but there seem
> to be people that think it should be done.

Linux is compilable on Windows, and has been for a long time already.
With Cygwin it is pretty trivial to do.  I prefer native Linux though.


Nicolas

RE: Switching from CVS to GIT

[Dave Korn]

On 16 October 2007 16:56, David Brown wrote:

> On Tue, Oct 16, 2007 at 02:25:21AM -0400, Eli Zaretskii wrote:
> 
>> On the other hand, what packages have 100K files?  If there's only one
>> -- the Linux kernel -- then I think this kind of performance is for
>> all practical purposes unimportant on Windows, because while it is
>> reasonable to assume that someone would like to use git on Windows,
>> assuming that someone will develop the Linux kernel on Windows is --
>> how should I put it -- _really_ far-fetched ;-)
> 
> Oh, I wish others could think this clearly.  Quoting a serious line off of
> a task list at an unnamed company:
> 
>    - Make Linux kernel compile under windows.
> 
> I don't think it will move past just being a wish list item, but there seem
> to be people that think it should be done.
> 
> Admittedly, they don't want developers doing it on windows, but want to
> integrate kernel building into a windows-heavy build and release process.

  Do that kind of thing here all the time, hence my previous post.  Apart from
the netfilter stuff with the filenames-that-match-in-all-but-case, no real
problems, took me a couple of hours one afternoon.

  Cygwin is a good match for linux dev work.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: Switching from CVS to GIT

[Andreas Ericsson]

Eli Zaretskii wrote:
>> From: Andreas Ericsson <ae@op5.se>
> 
>> I *think* (correct me if I'm wrong) that git is still faster
>> than a whole bunch of other scm's on windows, but to one who's used to
>> its performance on Linux that waiting several seconds to scan 10k files
>> just feels wrong.
> 
> Unless that 10K is a typo and you really meant 100K, I don't think 10K
> files should take several seconds to scan on Windows.  I just tried
> "find -print" on a directory with 32K files in 4K subdirectories, and
> it took 8 sec elapsed with a hot cache.  So 10K files should take at
> most 2 seconds, even without optimizing file traversal code.  Doing
> the same with native Windows system calls ("dir /s") brings that down
> to 4 seconds for 32K files.
> 

It was a typo. Thanks for correcting me.

> On the other hand, what packages have 100K files?  If there's only one
> -- the Linux kernel -- then I think this kind of performance is for
> all practical purposes unimportant on Windows

But it's most definitely not. The *huge* projects that have looked at
git have sometimes turned it down simply because they're either cross-
platform (Mozilla) or they have translators that use windows exclusively
(KDE and Mozilla, just to mention two).

Both Mozilla and KDE repos are *much* larger than the Linux repo.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Switching from CVS to GIT

[Daniel Barkalow]

On Tue, 16 Oct 2007, Johannes Schindelin wrote:

> Hi,
> 
> [culled make-w32, as per explicit request]
> 
> On Tue, 16 Oct 2007, Eli Zaretskii wrote:
> 
> > > Date: Tue, 16 Oct 2007 01:56:46 -0400 (EDT)
> > > From: Daniel Barkalow <barkalow@iabervon.org>
> > > cc: raa.lkml@gmail.com, Johannes.Schindelin@gmx.de, ae@op5.se, 
> > >     tsuna@lrde.epita.fr, git@vger.kernel.org, make-w32@gnu.org
> > > 
> > > Ah, that's helpful. We don't actually care too much about the 
> > > particular info in stat; we just want to know quickly if the file has 
> > > changed, so we can hash only the ones that have been touched and get 
> > > the actual content changes.
> > 
> > As I wrote in my other message, using native APIs improves performance 
> > by at least a factor of two.
> 
> Somehow this does not appeal to my "portability is good" side.  You know, 
> if we had to do such trickeries for every platform we support, we'd soon 
> be as big as Subversion *cough*.

I think that it would be a worthwhile project, from the point of view of 
making the code easier to follow and making the internal APIs clearer, to 
organize git's source to abstract the object database to read_sha1_file(), 
has_sha1_file(), hash_sha1_file(), and write_sha1_file() as the arbiters 
of what is in the local database (with other functions public as support 
for over-the-wire protocols, which may, by not-really-coincidence, by used 
for local storage as well); then Windows could have an entirely different 
storage mechanism that doesn't rely on filesystem metadata speed.

It would also be worthwhile to untangle the index's stat cache aspects and 
its tree-object-related aspects, so that there can be a platform- and 
repository-specific concept of how to handle the working area, and then 
Windows could do different stuff for the default case of setting up a 
directory on the local filesystem.

	-Daniel
*This .sig left intentionally blank*

Re: Switching from CVS to GIT

[Christopher Faylor]

On Tue, Oct 16, 2007 at 05:23:11PM +0100, Dave Korn wrote:
>On 16 October 2007 16:56, David Brown wrote:
>
>> On Tue, Oct 16, 2007 at 02:25:21AM -0400, Eli Zaretskii wrote:
>> 
>>> On the other hand, what packages have 100K files?  If there's only one
>>> -- the Linux kernel -- then I think this kind of performance is for
>>> all practical purposes unimportant on Windows, because while it is
>>> reasonable to assume that someone would like to use git on Windows,
>>> assuming that someone will develop the Linux kernel on Windows is --
>>> how should I put it -- _really_ far-fetched ;-)
>> 
>> Oh, I wish others could think this clearly.  Quoting a serious line off of
>> a task list at an unnamed company:
>> 
>>    - Make Linux kernel compile under windows.
>> 
>> I don't think it will move past just being a wish list item, but there seem
>> to be people that think it should be done.
>> 
>> Admittedly, they don't want developers doing it on windows, but want to
>> integrate kernel building into a windows-heavy build and release process.
>
>  Do that kind of thing here all the time, hence my previous post.  Apart from
>the netfilter stuff with the filenames-that-match-in-all-but-case, no real
>problems, took me a couple of hours one afternoon.

Ditto.

Coincidentially enough this is the reason I wrote managed mode for cygwin's
mount.

But, we're pretty far off-topic aren't we?

cgf

Re: Switching from CVS to GIT

[Robin Rosenberg]

tisdag 16 oktober 2007 skrev Steffen Prohaska:
> 
> On Oct 16, 2007, at 2:33 PM, Johannes Schindelin wrote:
> 
> >> Maybe we need a configuration similar to core.autocrlf (which  
> >> controls
> >> newline conversion) to control filename comparison and normalization?
> >>
> >> Most obviously for the case (in-)sensitivity on Windows, but I also
> >> remember the unicode normalization happening on Mac's HFS filesystem
> >> that caused trouble in the past.
> >
> > Robin Rosenberg has some preliminary code for that.  The idea is to  
> > wrap
> > all filesystem operations in cache.h, and do a filename normalisation
> > first.
> 
> At that point we could add a safety check. Paths that differ only by
> case, or whitespace, or ... (add general and project specific rules  
> here)
> should be denied. This would guarantee that tree objects can always be
> checked out. Even if the filesystem capabilities are limited.
> 
> Robin, what do you think?

My code only normalizes filenames to UTF-8 inside git, which isn't the same 
thing. I think that can be extended to handling MacOSX normalized UTF-8 and
Windows UTF-16 so, when you check out a thing from git there will be no 
surprises. Case insensitivity is another dimension. I have no idea as to the
performance of the code, it's more like a proof-that-it-can-be-done.

The code cannot "fail", it always does something reasonable, like not 
converting when that is not possible. Something else has to be done for 
validation.

The UTF-16 that windows use is not a current issue because git  only does 
local code page. Jgit, but it isn't very smart either because git doesn't say 
anything about filename encoding, while Windows/MacOSX/CIFS and other 
filesystems does.

The fact that git uses eigth bit file names may also be a reason performance 
is slower on Windows, because the eight-bit Win32API transforms all strings 
and filenames to the native UTF-16 encoding on *every* system call, in and 
out; that's a lot of work when you do it thousands of times. If git itself 
did the transform it might be made smarter and more suited to git's purposes, 
and most importantly faster. I have no idea about the performance hit. One
has to measure something.

I notice a number of SCM's out there, including one with a \$\d{4} pricetag 
gets you into trouble if you rename a file from Foo to FOO on Windows.

-- robin