git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Linus Torvalds <torvalds@osdl.org>
To: Alex Riesen <raa.lkml@gmail.com>
Cc: Radoslaw Szkodzinski <astralstorm@gorzow.mm.pl>,
	Keith Packard <keithp@keithp.com>,
	Junio C Hamano <junkio@cox.net>,
	cworth@cworth.org, Martin Langhoff <martin.langhoff@gmail.com>,
	Git Mailing List <git@vger.kernel.org>
Subject: Re: [Census] So who uses git?
Date: Tue, 31 Jan 2006 13:25:08 -0800 (PST)	[thread overview]
Message-ID: <Pine.LNX.4.64.0601311314030.7301@g5.osdl.org> (raw)
In-Reply-To: <20060130225107.GA3857@limbo.home>



On Mon, 30 Jan 2006, Alex Riesen wrote:
> 
> I use git in cygwin for a project with more then 17k files (almost 6M lines).
> It's real slow on ntfs (on 3.2Mhz PIV!)

One thing that git does rely on is a fast "lstat()" system call. The index 
file means that we almost never need to read the contents of a file to 
compare, but git _does_ check that files haven't been modified, and doing 
an "lstat()" on every single file it knows about is the way to do that.

Now, I suspect that you simply can't do basic filename lookups much faster 
than Linux does them. The Linux VFS layer name caching reigns supreme: the 
dentries are just incredibly powerful, and the reason Linux kicks ass on 
many benchmarks.

And yes, git was designed for it. git is _really_ fast on Linux, but any 
operating system that is so stupid that it has to call down to the 
low-level filesystem for filename lookup (which is most of them, and from 
what I have heard, the NT VFS layer is worse than most) will take a lot 
longer.

This is sadly not something I think you can possibly avoid. Git is 
literally being as fast as is humanly possible without doing explicit 
locking. You _can_ avoid the "lstat()" calls if you are willing to always 
explicitly mark files that you have changed (so that the SCM can stat just 
_those_ files and ignore all the others), but I personally much prefer 
being able to use any random tools on the files without having to prepare 
them some way.

So we could speed it up on cygwin (and yes, it would speed git up a lot 
even on Linux, but since the cached lstat() case is so fast anyway, I 
doubt a lot of Linux users care - the biggest win would be on a cold-cache 
tree).  But it would require that you explicitly _mark_ the files you edit 
some way.

Btw, BK wanted that, and it wasn't _too_ painful. You had to do

	bk edit

to mark a file as being ready to be dirtied, and as a helper command you 
would use

	bk editor

which would first do the "bk edit" thing and then start up your favourite 
editor (the usual ${EDITOR:${VISUAL:vi}} rules applied) on it, and it 
worked fine. We _could_ do the same in git.

I'd just prefer not to.

For small projects (or big projects with fairly few files), it really 
shouldn't matter. Your 17k files example is hopefully fairly rare..

> But its more intuitive and more powerful than any alternatives here (Perforce,
> SVN and CVS come to mind).

Good to know.

		Linus

  reply	other threads:[~2006-01-31 21:25 UTC|newest]

Thread overview: 110+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2006-01-26  2:10 LCA06 Cogito/GIT workshop - (Re: git-whatchanged: exit out early on errors) Martin Langhoff
2006-01-28  4:47 ` Linus Torvalds
2006-01-28  5:33   ` Martin Langhoff
2006-01-28  5:53     ` Linus Torvalds
2006-01-28  6:32       ` Junio C Hamano
2006-01-29 10:12       ` Fredrik Kuivinen
2006-01-29 20:15         ` Junio C Hamano
2006-01-28 11:00     ` Keith Packard
2006-01-28 21:08       ` [Census] So who uses git? Junio C Hamano
2006-01-29  2:14         ` Morten Welinder
2006-01-29  3:53           ` Junio C Hamano
2006-01-29 14:19             ` Morten Welinder
2006-01-29 20:15               ` Junio C Hamano
2006-01-29 10:09         ` Keith Packard
2006-01-29 11:18           ` Radoslaw Szkodzinski
2006-01-29 18:12             ` Greg KH
2006-01-31 18:33               ` Radoslaw Szkodzinski
2006-01-31 19:50                 ` Radoslaw Szkodzinski
2006-01-31 20:43                   ` Junio C Hamano
2006-01-31 21:02                     ` Radoslaw Szkodzinski
2006-01-30 22:51             ` Alex Riesen
2006-01-31 21:25               ` Linus Torvalds [this message]
2006-01-31 21:52                 ` J. Bruce Fields
2006-01-31 22:01                 ` Alex Riesen
     [not found]                   ` <20060201013901.GA16832@mail.com>
2006-02-01  2:04                     ` Linus Torvalds
2006-02-01  2:09                       ` Linus Torvalds
2006-02-09  5:15                         ` [PATCH] "Assume unchanged" git Junio C Hamano
2006-02-09  5:49                           ` [PATCH] "Assume unchanged" git: do not set CE_VALID with --refresh Junio C Hamano
2006-02-09  5:50                           ` [PATCH] ls-files: debugging aid for CE_VALID changes Junio C Hamano
2006-02-01  2:31                       ` [Census] So who uses git? Junio C Hamano
2006-02-01  3:43                         ` Linus Torvalds
2006-02-01  7:03                           ` Junio C Hamano
     [not found]                         ` <20060201045337.GC25753@mail.com>
2006-02-01  5:04                           ` Linus Torvalds
2006-02-01  5:42                           ` Junio C Hamano
2006-02-01 16:15                       ` Jason Riedy
2006-02-01 19:20                       ` Julian Phillips
2006-02-01 19:29                         ` Linus Torvalds
2006-02-06 21:15                       ` Chuck Lever
2006-02-01  2:52                     ` Martin Langhoff
2006-02-01  3:48                       ` Linus Torvalds
2006-02-01 19:30                         ` H. Peter Anvin
2006-02-01 14:55                       ` Alex Riesen
2006-02-01 16:25                         ` Linus Torvalds
2006-02-02  9:12                           ` Alex Riesen
2006-01-29 18:37         ` Dave Jones
2006-01-29 20:17           ` Daniel Barkalow
2006-01-29 20:29             ` Martin Langhoff
2006-01-30 15:23             ` Mike McCormack
2006-01-30 18:58         ` Carl Baldwin
2006-01-31 10:27           ` Johannes Schindelin
2006-01-31 15:24             ` Carl Baldwin
2006-01-31 15:31               ` Johannes Schindelin
2006-01-31 17:30             ` Linus Torvalds
2006-01-31 18:12               ` J. Bruce Fields
2006-01-31 19:33                 ` Junio C Hamano
2006-01-31 19:44                   ` Jon Loeliger
2006-01-31 19:52                     ` Junio C Hamano
     [not found]                     ` <7vd5i8w2nc.fsf@assigned-by-dhcp.cox.net>
2006-01-31 20:56                       ` J. Bruce Fields
2006-01-31 20:06                   ` J. Bruce Fields
2006-01-31 19:01               ` Keith Packard
2006-01-31 19:21                 ` Linus Torvalds
2006-01-31 22:55                   ` Joel Becker
2006-02-01 14:43                     ` Johannes Schindelin
2006-01-31 20:56                 ` Sam Ravnborg
2006-01-31 22:21                   ` Junio C Hamano
2006-02-01 19:34               ` H. Peter Anvin
2006-01-31 23:16             ` Daniel Barkalow
2006-01-31 23:36               ` Petr Baudis
2006-01-31 23:47               ` Junio C Hamano
2006-02-01  0:38                 ` Linus Torvalds
2006-02-01  0:52                   ` Junio C Hamano
2006-02-01  2:19                   ` Daniel Barkalow
2006-02-01  6:42                   ` Junio C Hamano
2006-02-01  7:22                     ` Carl Worth
2006-02-01  8:26                       ` Junio C Hamano
2006-02-01  9:59                         ` Randal L. Schwartz
2006-02-01 20:48                           ` Junio C Hamano
2006-02-01 17:11                     ` Linus Torvalds
2006-02-01 17:18                     ` Nicolas Pitre
2006-02-01 20:27                       ` Junio C Hamano
2006-02-01 21:09                         ` Linus Torvalds
2006-02-01 21:34                           ` Nicolas Pitre
2006-02-01 21:59                           ` Junio C Hamano
2006-02-01 22:25                             ` Nicolas Pitre
2006-02-01 22:50                               ` Junio C Hamano
2006-02-02 14:59                                 ` Andreas Ericsson
2006-02-01 22:35                             ` Linus Torvalds
2006-02-01 23:33                               ` Two ideas for improving git's user interface Carl Worth
2006-02-02  0:38                                 ` Junio C Hamano
2006-02-02  1:16                                   ` Carl Worth
2006-02-02  2:25                                     ` Junio C Hamano
2006-02-03 23:57                                       ` Carl Worth
2006-02-02  1:23                                 ` Linus Torvalds
2006-02-02  1:44                                   ` Linus Torvalds
2006-02-04  8:03                                     ` Alan Chandler
2006-02-04  8:25                                       ` Junio C Hamano
2006-02-04  9:30                                         ` Alan Chandler
2006-02-04  0:20                                   ` Carl Worth
2006-02-04  2:08                                     ` Linus Torvalds
2006-02-06 23:42                                       ` Carl Worth
2006-02-02 12:31                                 ` Florian Weimer
2006-02-02 16:30                                 ` Carl Baldwin
2006-02-01 22:57                             ` [Census] So who uses git? Daniel Barkalow
2006-02-01 22:00                         ` Joel Becker
2006-02-01 19:32           ` H. Peter Anvin
  -- strict thread matches above, loose matches on Subject: below --
2006-02-01  7:08 linux
2006-02-01  8:51 ` Junio C Hamano
2006-02-01 16:04 ` Linus Torvalds
2006-02-01 16:10 ` Alex Riesen
2006-02-01 21:27   ` linux

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Pine.LNX.4.64.0601311314030.7301@g5.osdl.org \
    --to=torvalds@osdl.org \
    --cc=astralstorm@gorzow.mm.pl \
    --cc=cworth@cworth.org \
    --cc=git@vger.kernel.org \
    --cc=junkio@cox.net \
    --cc=keithp@keithp.com \
    --cc=martin.langhoff@gmail.com \
    --cc=raa.lkml@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).