* Re: [PATCH] gitweb: Use File::Find::find in git_get_projects_list
From: Jakub Narebski @ 2006-09-14 17:02 UTC (permalink / raw)
To: git
In-Reply-To: <7vmz924cxy.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
>> Sorry, this comment was leftover from before, when no_chdir was not
>> used. Then $_ was the last part of directory,...
>
> Ah, thanks. I missed that difference. Did you choose to use
> no_chdir for performance reasons or coding convenience (somehow
> I had an impression that no_chdir would be slower)?
First benchmarks showed that no_chdir was some faster. I have rechecked,
and they are the same within the margin of error, perhaps without
no_chdir is slightly faster. 447 +/- 11 ms vs. 450 +/- 10 ms according
to ApacheBench (ab -n 10).
So it is really the matter of convenience. The default (without no_chdir) is
I think more convenient.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: cvs import
From: Michael Haggerty @ 2006-09-14 17:01 UTC (permalink / raw)
To: Jon Smirl; +Cc: monotone-devel, dev, git, Jakub Narebski
In-Reply-To: <9e4733910609140927y30ecaa42wae0ff0597b8c3842@mail.gmail.com>
Jon Smirl wrote:
> On 9/14/06, Jakub Narebski <jnareb@gmail.com> wrote:
>> Shawn Pearce wrote:
>>
>> > Originally I wanted Jon Smirl to modify the cvs2svn (...)
>>
>> By the way, will cvs2git (modified cvs2svn) and git-fast-import publicly
>> available?
>
> It has some unresolved problems so I wasn't spreading it around everywhere.
>
> It is based on cvs2svn from August. There has been too much change to
> the current cvs2svn to merge it anymore. [...]
>
> If the repo is missing branch tags cvs2svn may turn a single missing
> branch into hundreds of branches. The Mozilla repo has about 1000
> extra branches because of this.
[To explain to our studio audience:] Currently, if there is an actual
branch in CVS but no symbol associated with it, cvs2svn generates branch
labels like "unlabeled-1.2.3", where "1.2.3" is the branch revision
number in CVS for the particular file. The problem is that the branch
revision numbers for files in the same logical branch are usually
different. That is why many extra branches are generated.
Such unnamed branches cannot reasonably be accessed via CVS anyway, and
somebody probably made the conscious decision to delete the branch from
CVS (though without doing it correctly). Therefore such revisions are
probably garbage. It would be easy to add an option to discard such
revisions, and we should probably do so. (In fact, they can already be
excluded with "--exclude=unlabeled-.*".) The only caveat is that it is
possible for other, named branches to sprout from an unnamed branch. In
this case either the second branch would have to be excluded too, or the
unlabeled branch would have to be included.
Alternatively, there was a suggestion to add heuristics to guess which
files' "unlabeled" branches actually belong in the same original branch.
This would be a lot of work, and the result would never be very
accurate (for one thing, there is no evidence of the branch whatsoever
in files that had no commits on the branch).
Other ideas are welcome.
Michael
^ permalink raw reply
* Re: [PATCH 2/2] Handle invalid argc gently
From: Junio C Hamano @ 2006-09-14 16:53 UTC (permalink / raw)
To: Dmitry V. Levin; +Cc: git
In-Reply-To: <20060914131626.GE6709@basalt.office.altlinux.org>
"Dmitry V. Levin" <ldv@altlinux.org> writes:
>> What is the valid reason to do execlp("git", NULL, NULL)?
>
> Personally I do not plan to execute git this way on regular basis, indeed. :)
>
> But argc == 0 is allowed, so why should git crash?
Oh, no I was not arguing for making git crash. I was just
trying to learn if there is a valid reason to choose to, or
common misconfiguration that causes it to, run with ac == 0,
since I did not think of any.
It is good to be extra careful and defensive -- I just wanted to
know if there is more to it.
^ permalink raw reply
* Re: [PATCH] gitweb: Use File::Find::find in git_get_projects_list
From: Junio C Hamano @ 2006-09-14 16:42 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200609141201.41711.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> Sorry, this comment was leftover from before, when no_chdir was not
> used. Then $_ was the last part of directory,...
Ah, thanks. I missed that difference. Did you choose to use
no_chdir for performance reasons or coding convenience (somehow
I had an impression that no_chdir would be slower)?
> Sorry for the confusion. By index I meant alternate way of specyfying
> projects,...
Again, thanks for clarification. I forgot about the handcrafted
project index file gitweb uses.
> First of all, why duplicated repositories are considered
> error? ... it is not that something bad would happen if there
> are duplicated repositories.
Fair enough, and I agree that that is a sane reasoning. Thanks.
^ permalink raw reply
* Re: open(2) vs fopen(3)
From: Junio C Hamano @ 2006-09-14 16:37 UTC (permalink / raw)
To: moreau francis; +Cc: git, Linus Torvalds
In-Reply-To: <Pine.LNX.4.64.0609140835080.4388@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> On Thu, 14 Sep 2006, moreau francis wrote:
>>
>> I'm reading git source code and falling on this stupid question:
>> Why sometime open(2) is used and other time fopen(3) is
>> prefered. I'm sorry for this dump question but I have no clue.
>
> fopen() tends to result in easier usage, especially if the file in
> question is a line-based ASCII file, and you can just use "fgets()" to
> read it. So fopen is the simple alternative for simple problems.
>
> Using a direct open() means that you have to use the low-level IO
> functions (I'm ignoring the use of "fdopen()"), but if done right, it has
> a number of advantages:
>...
> - error conditions are a lot more obvious and repeatable with the
> low-level things, at least so I find personally. Error handling with
> stdio routines is _possible_, but probably because almost nobody ever
> does it, it's not something that people are conditioned to do, so it
> ends up beign "strange".
Another issue related with this is that stdio implementations
tend to have unintuitive interaction with signals, one fine
example of it being the problem we fixed with commit fb7a653,
where on Solaris fgets(3) did not restart the underlying read(2)
upon SIGALRM.
Technically it was a bug on our part not Solaris, but that was
something unexpected to see.
^ permalink raw reply
* Re: cvs import
From: Jon Smirl @ 2006-09-14 16:27 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, monotone-devel, dev
In-Reply-To: <eebuih$u32$1@sea.gmane.org>
On 9/14/06, Jakub Narebski <jnareb@gmail.com> wrote:
> Shawn Pearce wrote:
>
> > Originally I wanted Jon Smirl to modify the cvs2svn (...)
>
> By the way, will cvs2git (modified cvs2svn) and git-fast-import publicly
> available?
It has some unresolved problems so I wasn't spreading it around everywhere.
It is based on cvs2svn from August. There has been too much change to
the current cvs2svn to merge it anymore. It is going to need
significant rewrite. But cvs2svn will all change again if it converts
to the dependency model. It is better to get a backend independent
interface build into cvs2svn.
It it not generating an accurate repo. cvs2svn is outputting tags
based on multiple revisions, git can't do that. I'm just tossing some
of the tag data that git can't handle. I base the tag on the fist
revision which is not correct.
If the repo is missing branch tags cvs2svn may turn a single missing
branch into hundreds of branches. The Mozilla repo has about 1000
extra branches because of this.
Sometime cvs2svn will partial copy from another rev to generate a new
rev. Git doesn't do this so I am tossing the copy requests. I need to
figure out how to hook into the data before cvs2svn tries to copy
things.
cvs2svn makes no attempt to detect merges so gitk will show 1,700
active branches when there are really only 10 currently active
branches in Mozilla.
That said 99.9% of Mozilla CVS is in the output git repo, but it isn't
quite right.
If you still want the code I'll send it to you.
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: cvs import
From: Shawn Pearce @ 2006-09-14 16:18 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, monotone-devel, dev
In-Reply-To: <eebuih$u32$1@sea.gmane.org>
Jakub Narebski <jnareb@gmail.com> wrote:
> Shawn Pearce wrote:
>
> > Originally I wanted Jon Smirl to modify the cvs2svn (...)
>
> By the way, will cvs2git (modified cvs2svn) and git-fast-import publicly
> available?
Yes. I want to submit git-fast-import to the main Git project and
ask Junio to bring it in.
However right now I feel like the code isn't up-to-snuff and won't
pass peer review on the Git mailing list. So I wanted to spend a
little bit of time cleaning it up before asking Junio to carry it
in the main distribution. My pack mmap window code is actually
part of that cleanup.
I think the goal of this thread is to try and merge the ideas
behind Jon's modified cvs2svn into the core cvs2svn, possibly
causing cvs2svn to be renamed to cvs2notcvs (or some such) and
having a slightly more modular output format so Git, Monotone and
Subversion can all benefit from the difficult-to-do-right changeset
generation logic.
--
Shawn.
^ permalink raw reply
* Re: cvs import
From: Jakub Narebski @ 2006-09-14 16:04 UTC (permalink / raw)
To: git; +Cc: monotone-devel, dev, monotone-devel, git
In-Reply-To: <20060914155003.GB9657@spearce.org>
Shawn Pearce wrote:
> Originally I wanted Jon Smirl to modify the cvs2svn (...)
By the way, will cvs2git (modified cvs2svn) and git-fast-import publicly
available?
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: Historical kernel repository size
From: Shawn Pearce @ 2006-09-14 15:52 UTC (permalink / raw)
To: Andy Whitcroft; +Cc: tglx, Petr Baudis, git
In-Reply-To: <45097775.4040005@shadowen.org>
Andy Whitcroft <apw@shadowen.org> wrote:
> tglx@linutronix.de wrote:
> > Petr,
> >
> >> just to test the packing improvements we had achieved over the last
> >> year, I have repacked the historical kernel repository and achieved a
> >> significant improvement:
> >> ....
> >> Since it's a nice place for people to check about how efficient we are
> >> with compressing the repository, perhaps it would be a good idea to
> >> repack the historical repository on kernel.org?
> >
> > I'll do once I'm back home.
>
> Is there any reason this isn't a live history. ie that we don't
> constantly pull linus' master branch into this history to make it a real
> complete history?
>
> Perhaps that isn't possible ... hmmm. I guess it might only work if
> linus' repo was actually a grafted version of this history?
Right - the only way to join the two is to graft them together.
Since grafts are a purely local matter anyone can pull both into
the same repository and insert the correct grafts to get a complete
history. You would just want to publish on the kernel.org website
the correct grafts file, so users don't have to figure it out on
their own.
Since I'm not a kernel developer I haven't even looked to see if
such a grafts file has been published. :-)
--
Shawn.
^ permalink raw reply
* Re: Historical kernel repository size
From: Petr Baudis @ 2006-09-14 15:52 UTC (permalink / raw)
To: Andy Whitcroft; +Cc: tglx, git
In-Reply-To: <45097775.4040005@shadowen.org>
Dear diary, on Thu, Sep 14, 2006 at 05:38:29PM CEST, I got a letter
where Andy Whitcroft <apw@shadowen.org> said that...
> Is there any reason this isn't a live history. ie that we don't
> constantly pull linus' master branch into this history to make it a real
> complete history?
Because at the early times of Git, things were evolving fast and it
would be unfeasible to have to drag this old history around in case of
format changes and stuff. Also, at that time history was still very big
and it would be impractical to require all the kernel developers to grab
all the bitkeeper history (it still kind of is).
> Perhaps that isn't possible ... hmmm. I guess it might only work if
> linus' repo was actually a grafted version of this history?
>
> /me watches his head explode.
http://lkml.org/lkml/2006/6/17/110
may be useful.
It wasn't accepted. Oh well, I may try to resubmit it again soon. :-)
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: cvs import
From: Shawn Pearce @ 2006-09-14 15:50 UTC (permalink / raw)
To: Michael Haggerty
Cc: Martin Langhoff, monotone-devel, Jon Smirl, dev, Git Mailing List
In-Reply-To: <4508EA78.5030001@alum.mit.edu>
Michael Haggerty <mhagger@alum.mit.edu> wrote:
> The only difference between our SCMs that might be difficult
> to paper over in a universal dumpfile is that SVN wants its changesets
> in chronological order, whereas I gather that others would prefer the
> data in dependency order branch by branch.
This really isn't an issue for Git.
Originally I wanted Jon Smirl to modify the cvs2svn code to emit
only one branch at a time as that would be much faster than jumping
around branches in chronological order. But it turned out to
be too much work to change cvs2svn. So git-fast-import (the Git
program that consumes the dump stream from Jon's modified cvs2svn)
maintains an LRU of the branches in memory and reloads inactive
branches as necessary when cvs2svn jumps around.
It turns out it didn't matter if the git-fast-import maintained 5
active branches in the LRU or 60. Apparently the Mozilla repo didn't
jump around more than 5 branches at a time - most of the time anyway.
Branches in git-fast-import seemed to cost us only 2 MB of memory
per active branch on the Mozilla repository. Holding 60 of them at
once (120 MB) is peanuts on most machines today. But really only 5
(10 MB) were needed for an efficient import.
I don't know how the Monotone guys feel about it but I think Git
is happy with the data in any order, just so long as the dependency
chains aren't fed out of order. Which I think nearly all changeset
based SCMs would have an issue with. So we should be just fine
with the current chronological order produced by cvs2svn.
--
Shawn.
^ permalink raw reply
* Re: open(2) vs fopen(3)
From: Linus Torvalds @ 2006-09-14 15:46 UTC (permalink / raw)
To: moreau francis; +Cc: git
In-Reply-To: <20060914091513.19826.qmail@web25812.mail.ukl.yahoo.com>
On Thu, 14 Sep 2006, moreau francis wrote:
>
> I'm reading git source code and falling on this stupid question:
> Why sometime open(2) is used and other time fopen(3) is
> prefered. I'm sorry for this dump question but I have no clue.
fopen() tends to result in easier usage, especially if the file in
question is a line-based ASCII file, and you can just use "fgets()" to
read it. So fopen is the simple alternative for simple problems.
Using a direct open() means that you have to use the low-level IO
functions (I'm ignoring the use of "fdopen()"), but if done right, it has
a number of advantages:
- with the proper use, it's potentially more efficient (but stdio is a
lot more efficient if you do lots of small writes without buffering)
- you can control the creation flags better (ie if you want to do an
exclusive open, you _have_ to use "open()" - there's no portable way to
say O_EXCL with "fopen()")
- error conditions are a lot more obvious and repeatable with the
low-level things, at least so I find personally. Error handling with
stdio routines is _possible_, but probably because almost nobody ever
does it, it's not something that people are conditioned to do, so it
ends up beign "strange".
(So this third one is more a psychological issue than a really
technical issue - at least for me. I'd not use stdio for things I
might expect to do fsync() on, for example. It's _possible_, but very
non-intuitive, because that's now how people generally use stdio).
So it boils down to the fact that people tend to do higher-level things
with stdio interfaces (fopen and friends), and lower-level things with the
raw system call ("unistd.h") interfaces.
In git, you'd expect to see code that actually works on the object
database or the refs using "open()" (both because it's low-level, and it
generally wants to use O_EXCL and friends), and then things that open the
".gitignore" file to use fopen() (because it's a line-based ASCII
interface, and it's not an "important" file in the sense that we don't
really care about some strange situation where it could give us an IO
error).
There might also be a difference in personality. I probably tend to use
the core unistd interfaces more than some other people would, and some
other people might end up using stdio for pretty much everything.
Linus
^ permalink raw reply
* Re: Historical kernel repository size
From: Andy Whitcroft @ 2006-09-14 15:38 UTC (permalink / raw)
To: tglx; +Cc: Petr Baudis, git
In-Reply-To: <46900.194.138.39.52.1158244729.squirrel@www.tglx.de>
tglx@linutronix.de wrote:
> Petr,
>
>> just to test the packing improvements we had achieved over the last
>> year, I have repacked the historical kernel repository and achieved a
>> significant improvement:
>> ....
>> Since it's a nice place for people to check about how efficient we are
>> with compressing the repository, perhaps it would be a good idea to
>> repack the historical repository on kernel.org?
>
> I'll do once I'm back home.
Is there any reason this isn't a live history. ie that we don't
constantly pull linus' master branch into this history to make it a real
complete history?
Perhaps that isn't possible ... hmmm. I guess it might only work if
linus' repo was actually a grafted version of this history?
/me watches his head explode.
-apw
^ permalink raw reply
* Re: Historical kernel repository size
From: Linus Torvalds @ 2006-09-14 15:31 UTC (permalink / raw)
To: Petr Baudis; +Cc: Thomas Gleixner, Git Mailing List
In-Reply-To: <20060914142249.GK23891@pasky.or.cz>
On Thu, 14 Sep 2006, Petr Baudis wrote:
>
> just to test the packing improvements we had achieved over the last
> year, I have repacked the historical kernel repository and achieved a
> significant improvement:
Umm.. Only apparently because the old pack was really really bad. It also
has the wrong name, probably because it's using the original naming that
had the SHA1 computed on the unsorted input. That was changed a long time
ago.
Yours isn't wonderful either.
> -rw-r--r-- 1 xpasky users 13600376 Sep 14 16:18 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.idx
> -rw-r--r-- 1 xpasky users 197168186 Sep 14 16:18 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.pack
> -rw-r--r-- 1 xpasky users 13600376 Sep 14 12:18 pack-cc3517351ecce3ef7ba010559992bdfc10b7acd4.idx
> -rw-r--r-- 1 xpasky users 262818936 Sep 14 12:29 pack-cc3517351ecce3ef7ba010559992bdfc10b7acd4.pack
Mine are:
-rw-r--r-- 1 torvalds torvalds 13600376 Apr 19 10:06 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.idx
-rw-r--r-- 1 torvalds torvalds 185374386 Apr 19 10:06 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.pack
and as you can see from the date, they aren't exactly very recent, but
they shave an additional 6% off the size.
I agree that the _oroginal_ history pack by Thomas seems to be bad, and
that's from Aug 9 2005, so it's likely with some really really old packing
rules.
For better packing, I think I used a larger depth, ie try something like
git repack -a -f --depth=50
to get more improvement. For a historical archive that you don't much use,
doign the deeper depth is definitely worth it.
Linus
^ permalink raw reply
* Re: Historical kernel repository size
From: tglx @ 2006-09-14 14:38 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20060914142249.GK23891@pasky.or.cz>
Petr,
> just to test the packing improvements we had achieved over the last
> year, I have repacked the historical kernel repository and achieved a
> significant improvement:
> ....
> Since it's a nice place for people to check about how efficient we are
> with compressing the repository, perhaps it would be a good idea to
> repack the historical repository on kernel.org?
I'll do once I'm back home.
tglx
^ permalink raw reply
* Historical kernel repository size
From: Petr Baudis @ 2006-09-14 14:22 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: git
Hi,
just to test the packing improvements we had achieved over the last
year, I have repacked the historical kernel repository and achieved a
significant improvement:
xpasky@machine[0:0]~/hi/history$ git-repack -a -f
Generating pack...
Done counting 566638 objects.
Deltifying 566638 objects.
100% (566638/566638) done
Writing 566638 objects.
100% (566638/566638) done
Total 566638, written 566638 (delta 456212), reused 98435 (delta 0)
Pack pack-4d27038611fe7755938efd4a2745d5d5d35de1c1 created.
xpasky@machine[0:0]~/hi/history$ l .git/objects/pack/
total 476264
-rw-r--r-- 1 xpasky users 13600376 Sep 14 16:18 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.idx
-rw-r--r-- 1 xpasky users 197168186 Sep 14 16:18 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.pack
-rw-r--r-- 1 xpasky users 13600376 Sep 14 12:18 pack-cc3517351ecce3ef7ba010559992bdfc10b7acd4.idx
-rw-r--r-- 1 xpasky users 262818936 Sep 14 12:29 pack-cc3517351ecce3ef7ba010559992bdfc10b7acd4.pack
Since it's a nice place for people to check about how efficient we are
with compressing the repository, perhaps it would be a good idea to
repack the historical repository on kernel.org?
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: [PATCH 2/2] Handle invalid argc gently
From: Dmitry V. Levin @ 2006-09-14 13:16 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vpsdz5542.fsf@assigned-by-dhcp.cox.net>
[-- Attachment #1: Type: text/plain, Size: 659 bytes --]
On Wed, Sep 13, 2006 at 11:34:05PM -0700, Junio C Hamano wrote:
> "Dmitry V. Levin" <ldv@altlinux.org> writes:
> > On Wed, Sep 13, 2006 at 06:12:02PM -0700, Junio C Hamano wrote:
> >> "Dmitry V. Levin" <ldv@altlinux.org> writes:
> >>
> >> > describe, git: Handle argc==0 case the same way as argc==1.
> >>
> >> When does one get (ac == 0)?
> >
> > When one executes something like
> > execlp("git", NULL, NULL);
>
> Fair enough.
>
> What is the valid reason to do execlp("git", NULL, NULL)?
Personally I do not plan to execute git this way on regular basis, indeed. :)
But argc == 0 is allowed, so why should git crash?
--
ldv
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: open(2) vs fopen(3)
From: Andy Whitcroft @ 2006-09-14 10:52 UTC (permalink / raw)
To: moreau francis; +Cc: git
In-Reply-To: <20060914091513.19826.qmail@web25812.mail.ukl.yahoo.com>
moreau francis wrote:
> Hi GIT folks,
>
> I'm reading git source code and falling on this stupid question:
> Why sometime open(2) is used and other time fopen(3) is
> prefered. I'm sorry for this dump question but I have no clue.
It looks very much from a quick random sampling, that open is used where
we are going to mmap the file for quick access. fopen otherwise.
-apw
^ permalink raw reply
* Re: [PATCH] gitweb: Use File::Find::find in git_get_projects_list
From: Jakub Narebski @ 2006-09-14 10:01 UTC (permalink / raw)
To: Junio C Hamano, git
In-Reply-To: <7vvenq4wgp.fsf@assigned-by-dhcp.cox.net>
Dnia czwartek 14. września 2006 11:40, napisałeś:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > ... But I have realized that for gitweb
> > for only one project one could have ".git" as a project name,
> > e.g. by putting $projectroot to be live git repository (working
> > directory of git repository).
>
> I do not understand this comment. I have /git/{git,linux-2.6}
> and run test gitweb with projectroot set to /git to see it can
> display them both. Both are with working tree, so there are
> /git/git/.git and /git/linux-2.6/.git directories.
>
> Do you mean to say "project name" is always taken from the last
> component of the directory name and the above setup would result
> in ambiguity? If that is the case that sounds awful, but I
> haven't noticed it.
Sorry, this comment was leftover from before, when no_chdir was not
used. Then $_ was the last part of directory, and 'return if (/^\./);'
would not take into consideration git/.git nor linux-2.6/.git as
valid project names ($projectroot / $projects_list is stripped from
project name).
Now the test should read "if (m!^.*/\.!)" if we want to skip
dot files and dot directories.
> > It was intended I think to avoid adding '.' and '..' as git
> > repositories, not stray directories. Well, perhaps index file
> > if it was used.
>
> Having and not having index are both valid, so there is no value
> in checking the index, even if we wanted to be more paranoid.
> Existence of HEAD, refs/heads, and/or objects/ would be a more
> meaningful alternative, but again only if we wanted to be more
> paranoid.
Sorry for the confusion. By index I meant alternate way of specyfying
projects, and up till now the only way to specify hierarchical
repository structure, namely $projects_list being file with encoded
directories and encoded owners (it was named index.aux or
index/index.aux by default).
> >> > + File::Find::find({
> >> > + no_chdir => 1, # do not change directory
> >> > + follow_fast => 1, # follow symbolic links
> >>
> >> What is the reason behind choosing follow_fast? By saying
> >> follow_anything, you choose to care about cases where there are
> >> symlinks under projectroot to point at various projects. If
> >> that is the case, don't you want to make sure you include the
> >> same project only once?
> >
> > First, it is faster. Second, for testing if it works I used copy
> > of a one "live" git repository I have (git.git repository), by
> > making second symlink to it.
>
> That was not what I wanted to ask; slower and correct is always
> preferred over fast and incorrect. I did not see anything that
> compensates the duplicates follow_fast might give you in the
> code, so I wondered there were some other trick you used to avoid
> it.
>
> In other words, "because I have such and such check to avoid
> duplicates, so I can safely use 'follow_fast', without using
> slower 'follow'" was the answer I was after.
First of all, why duplicated repositories are considered error?
It is perhaps error/mistake (besides testing of course) in doing the
layout of repositories (projectroot), but it is not that something bad
would happen if there are duplicated repositories.
Second, follow_fast is faster, especially that in gitweb we would
have to set "follow_skip => 2" to not spew errors to web server log when
there are duplicated repositories.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH] gitweb: Use File::Find::find in git_get_projects_list
From: Junio C Hamano @ 2006-09-14 9:40 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200609140959.04061.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> ... But I have realized that for gitweb
> for only one project one could have ".git" as a project name,
> e.g. by putting $projectroot to be live git repository (working
> directory of git repository).
I do not understand this comment. I have /git/{git,linux-2.6}
and run test gitweb with projectroot set to /git to see it can
display them both. Both are with working tree, so there are
/git/git/.git and /git/linux-2.6/.git directories.
Do you mean to say "project name" is always taken from the last
component of the directory name and the above setup would result
in ambiguity? If that is the case that sounds awful, but I
haven't noticed it.
> It was intended I think to avoid adding '.' and '..' as git
> repositories, not stray directories. Well, perhaps index file
> if it was used.
Having and not having index are both valid, so there is no value
in checking the index, even if we wanted to be more paranoid.
Existence of HEAD, refs/heads, and/or objects/ would be a more
meaningful alternative, but again only if we wanted to be more
paranoid.
>> > + File::Find::find({
>> > + no_chdir => 1, # do not change directory
>> > + follow_fast => 1, # follow symbolic links
>>
>> What is the reason behind choosing follow_fast? By saying
>> follow_anything, you choose to care about cases where there are
>> symlinks under projectroot to point at various projects. If
>> that is the case, don't you want to make sure you include the
>> same project only once?
>
> First, it is faster. Second, for testing if it works I used copy
> of a one "live" git repository I have (git.git repository), by making
> second symlink to it.
That was not what I wanted to ask; slower and correct is always
preferred over fast and incorrect. I did not see anything that
compensates the duplicates follow_fast might give you in the
code, so I wondered there were some other trick you used to avoid
it.
In other words, "because I have such and such check to avoid
duplicates, so I can safely use 'follow_fast', without using
slower 'follow'" was the answer I was after.
^ permalink raw reply
* Re: [PATCH] gitweb: Use File::Find::find in git_get_projects_list
From: Junio C Hamano @ 2006-09-14 9:23 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200609140959.04061.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> Not true. Link to directory is both -d $_ and -l $_, so
>
> return unless (-d $_ || (-l $_ && -d readlink($_)));
>
> is not needed.
I think you mis-read what I said. I first wondered why you did
not say "return unless -d _" and wrote (seemingly more
inefficient) "return unless -d $_". The comment is to clarify
why '$' is needed.
In other words, after this setup:
$ rm -fr d dl
$ mkdir d
$ ln -s d dl
you do not see an output from this:
$ perl -e 'lstat "dl"; print "is-dir\n" if -d _;'
but you do from this:
$ perl -e 'lstat "dl"; print "is-dir\n" if -d "dl";'
^ permalink raw reply
* open(2) vs fopen(3)
From: moreau francis @ 2006-09-14 9:15 UTC (permalink / raw)
To: git
Hi GIT folks,
I'm reading git source code and falling on this stupid question:
Why sometime open(2) is used and other time fopen(3) is
prefered. I'm sorry for this dump question but I have no clue.
thanks
Francis
^ permalink raw reply
* Re: [PATCH] gitweb: Use File::Find::find in git_get_projects_list
From: Jakub Narebski @ 2006-09-14 7:59 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <7v8xkm6gr6.fsf@assigned-by-dhcp.cox.net>
Dnia czwartek 14. września 2006 09:37, napisałeś:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > + sub wanted {
> > + # skip dot files (hidden files), check only directories
> > + #return if (/^\./);
>
> Leftover comment?
Leftover comment, from copying anonymous 'wanted' subroutine
from git_get_refs_list. But I have realized that for gitweb
for only one project one could have ".git" as a project name,
e.g. by putting $projectroot to be live git repository (working
directory of git repository).
> > + my $subdir = substr($File::Find::name, $pfxlen + 1);
> > + # we check related file in $projectroot
> > + if (-e "$projectroot/$subdir/HEAD") {
> > + push @list, { path => $subdir };
> > + $File::Find::prune = 1;
>
> We might want to do an extra cheap check to make what we found
> is sane, to prevent us getting confused by a random file whose
> name happens to be HEAD.
That is what we did before. Simplest check, also to avoid now to
claim top directory as git repository, and to know when to cut-off
(prune) finding.
It was intended I think to avoid adding '.' and '..' as git
repositories, not stray directories. Well, perhaps index file
if it was used.
> For example, it is a regular file whose contents is a single
> line and begins with "ref: refs/heads/" (16 bytes) or it is a
> symlink and readlink result begins with "refs/heads/" (11
> bytes).
We can do that, but I think it is unnecessary. Let's assume that
$projectroot contains _only_ git repositories, perhaps in subdirs
(directory hierarchy), and perhaps some stray files like not used
now index file.
> If you feel opening and reading the file is an added overhead,
> checking for $project/$subdir/{objects,refs}/ directories might
> be a good alternative.
Probably overkill.
> > + File::Find::find({
> > + no_chdir => 1, # do not change directory
> > + follow_fast => 1, # follow symbolic links
>
> What is the reason behind choosing follow_fast? By saying
> follow_anything, you choose to care about cases where there are
> symlinks under projectroot to point at various projects. If
> that is the case, don't you want to make sure you include the
> same project only once?
First, it is faster. Second, for testing if it works I used copy
of a one "live" git repository I have (git.git repository), by making
second symlink to it.
> > + #follow_skip => 2, # ignore duplicated files and directories
>
> Leftover comment?
Leftover from benchmarking what set of options is faster.
By the way, if we choose to use 'follow' instead of 'follow_fast' we
might want to uncomment it, to not spew errors in the log.
> About these two leftover comments, if you decided you did not
> want them, please do not leave them behind.
O.K.
> If on the other hand you wanted to hint others that you are not
> sure about your decision, it would probably be better to say
> that honestly in the comment, perhaps mark the message as RFC,
> and describe what the issues are, like so:
>
> sub wanted {
> # We might want to also ignore dot files, by
> # saying "return if /^\./;" here, but there is
> # no inherent reason for us to forbid a repository
> # name from starting with a dot.
True.
> # We check only if a directory looks like a git
> # repo and do not care about non directories.
> # Note that this cannot be done with "-d _"
> # because we are using follow_fast and the last
> # stat was done with lstat(); we want to catch a
> # symlink that points at a directory.
> return unless -d $_;
> ...
Not true. Link to directory is both -d $_ and -l $_, so
return unless (-d $_ || (-l $_ && -d readlink($_)));
is not needed.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH] gitweb: Use File::Find::find in git_get_projects_list
From: Junio C Hamano @ 2006-09-14 7:37 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200609140839.56181.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> + sub wanted {
> + # skip dot files (hidden files), check only directories
> + #return if (/^\./);
Leftover comment?
> + my $subdir = substr($File::Find::name, $pfxlen + 1);
> + # we check related file in $projectroot
> + if (-e "$projectroot/$subdir/HEAD") {
> + push @list, { path => $subdir };
> + $File::Find::prune = 1;
We might want to do an extra cheap check to make what we found
is sane, to prevent us getting confused by a random file whose
name happens to be HEAD.
For example, it is a regular file whose contents is a single
line and begins with "ref: refs/heads/" (16 bytes) or it is a
symlink and readlink result begins with "refs/heads/" (11
bytes).
If you feel opening and reading the file is an added overhead,
checking for $project/$subdir/{objects,refs}/ directories might
be a good alternative.
> + File::Find::find({
> + no_chdir => 1, # do not change directory
> + follow_fast => 1, # follow symbolic links
What is the reason behind choosing follow_fast? By saying
follow_anything, you choose to care about cases where there are
symlinks under projectroot to point at various projects. If
that is the case, don't you want to make sure you include the
same project only once?
> + #follow_skip => 2, # ignore duplicated files and directories
Leftover comment?
About these two leftover comments, if you decided you did not
want them, please do not leave them behind.
If on the other hand you wanted to hint others that you are not
sure about your decision, it would probably be better to say
that honestly in the comment, perhaps mark the message as RFC,
and describe what the issues are, like so:
sub wanted {
# We might want to also ignore dot files, by
# saying "return if /^\./;" here, but there is
# no inherent reason for us to forbid a repository
# name from starting with a dot.
# We check only if a directory looks like a git
# repo and do not care about non directories.
# Note that this cannot be done with "-d _"
# because we are using follow_fast and the last
# stat was done with lstat(); we want to catch a
# symlink that points at a directory.
return unless -d $_;
...
^ permalink raw reply
* Re: cvsimport fails with cvsps core dump
From: Rajkumar S @ 2006-09-14 7:22 UTC (permalink / raw)
To: git
In-Reply-To: <64de5c8b0609120914p4ea47860x60a294093d43bc24@mail.gmail.com>
On 9/12/06, Rajkumar S <rajkumars+git@gmail.com> wrote:
> While trying to cvsimport pfsense [1] source [2], cvsps coredumps and
> cvsimport aborts with the following error.
the function read_line does not check if the string is bigger that the
space alloted, and core dumps when such a string is encountered.
I have a small patch which prevents the crashing, by stopping the copy
when the buffer is full.
diff --git a/cvs_direct.c b/cvs_direct.c
index 920487d..3857ce2 100644
--- a/cvs_direct.c
+++ b/cvs_direct.c
@@ -679,8 +679,11 @@ static int read_line(CvsServerCtx * ctx,
}
p++;
len++;
+ if (len >= BUFSIZ){
+ *p = 0;
+ break;
+ }
}
-
return len;
}
raj
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox