* Re: People unaware of the importance of "git gc"?
From: Johan Herland @ 2007-09-05 8:41 UTC (permalink / raw)
To: Matthieu Moy; +Cc: git, Junio C Hamano, Linus Torvalds
In-Reply-To: <vpqtzq91p5z.fsf@bauges.imag.fr>
[-- Attachment #1: Type: text/plain, Size: 725 bytes --]
On Wednesday 05 September 2007, Matthieu Moy wrote:
> Johan Herland <johan@herland.net> writes:
>
> > When git-fetch and git-commit has done its job and is about to exit, it checks
> > the number of loose object, and if too high tells the user something
> > like "There are too many loose objects in the repo, do you want me to repack?
> > (y/N)". If the user answers "n" or simply <Enter>,
>
> I don't like commands to be interactive if they don't _need_ to be so.
> It kills scripting, it makes it hard for a front-end (git gui or so)
> to use the command, ...
Ok, so add an option or config variable to turn on/off this behaviour.
...Johan
--
Johan Herland, <johan@herland.net>
www.herland.net
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: David Kastrup @ 2007-09-05 8:47 UTC (permalink / raw)
To: git
In-Reply-To: <200709051042.04345.johan@herland.net>
Johan Herland <johan@herland.net> writes:
> On Wednesday 05 September 2007, Matthieu Moy wrote:
>> Johan Herland <johan@herland.net> writes:
>>
>> > When git-fetch and git-commit has done its job and is about to exit, it checks
>> > the number of loose object, and if too high tells the user something
>> > like "There are too many loose objects in the repo, do you want me to repack?
>> > (y/N)". If the user answers "n" or simply <Enter>,
>>
>> I don't like commands to be interactive if they don't _need_ to be so.
>> It kills scripting, it makes it hard for a front-end (git gui or so)
>> to use the command, ...
>
> Ok, so add an option or config variable to turn on/off this behaviour.
A bad idea which one can turn optionally off remains a bad idea for
everyone that has not been bitten enough by it already to actually
look up the problem and remedy.
Make this a warning.
--
David Kastrup
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Steven Grimm @ 2007-09-05 8:50 UTC (permalink / raw)
To: Linus Torvalds, Git Mailing List
In-Reply-To: <20070905074206.GA31750@artemis.corp>
Pierre Habouzit wrote:
> Well independently from the fact that one could suppose that users
> should use gc on their own, the big nasty problem with repacking is that
> it's really slow. And I just can't imagine git that I use to commit
> blazingly fast, will then be unavailable for a very long time (repacks
> on my projects -- that are not as big as the kernel but still -- usually
> take more than 10 to 20 seconds each).
>
What about kicking off a repack in the background at the ends of certain
commands? With an option to disable, of course. It could run at a low
priority and could even sleep a lot to avoid saturating the system's
disks -- since it'd be running asynchronously there should be no problem
if it takes longer to run.
Alternately, if it's possible to break the repack work up into chunks
that can be executed a bit at a time, you could do a small amount of
repacking very frequently (possibly still in the background) rather than
the whole thing at once. I suspect the nature of a repack, where you
presumably want everything loaded at once, would make that a challenge,
but it might not be impossible.
On the more general question...
IMO expecting end users to regularly perform what are essentially
database administration tasks (running git-gc is akin to rebuilding
indexes or packing tables on a DBMS) is naive. Heck, even database
administrators don't like to run database administration commands;
PostgreSQL added the "autovacuum" feature precisely because manual
periodic repacking (and the associated monitoring to figure out when to
do it) was too annoying for developers and DBAs. But you don't have to
look that far; anyone who has worked in IT can tell you horror stories
of users, including developers, whose computers have slowed to a crawl
because the users never bothered to defrag their hard disks. And that
affects *everything* the users do, not just version control operations!
It'll get worse as better UIs and tool integration become available and
git gains large numbers of users who are neither software developers nor
system administrators, and wouldn't know a packfile from a hole in the
ground. I'm talking web designers, graphic artists, mechanical
engineers, even managers and secretaries -- all of those people are in
git's ultimate target audience, even if it's not ready for them today.
None of them is going to be interested in doing random housekeeping
operations by hand, but they'll all appreciate a fast environment.
The fact that git sometimes stores your files individually in the .git
directory and sometimes bundles them together into big archives should
be an implementation detail that end-users don't have to worry about day
to day; git should do the right thing to remain fast under typical usage
scenarios, while leaving the plumbing exposed so people with atypical
usage can get their stuff done too.
-Steve
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Wincent Colaiuta @ 2007-09-05 8:51 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Linus Torvalds, Git Mailing List
In-Reply-To: <7vsl5tk1r8.fsf@gitster.siamese.dyndns.org>
El 5/9/2007, a las 9:30, Junio C Hamano escribió:
> Perhaps _exiting_ "git-commit" and "git-fetch" before doing
> anything, when the repository has more than 5000 loose objects
> with a LOUD bang that instructs an immediate repack would be
> good?
>
> I really do not like the idea of automatically running a repack
> after first interrupting the original command and then resuming.
> For one thing it would make a horribly difficult situation to
> debug if anything goes wrong. You cannot reproduce such a
> situation easily.
I would strongly oppose any *automatic* repacking and strongly
support any *advisory* recommandation to repack when the loose object
count exceeds a certain threshold. I don't think *exiting* a command
in such cases is a good idea; worse than automatic repacking this
would be *forced* manual repacking, which isn't very user-friendly.
Cheers,
Wincent
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Pierre Habouzit @ 2007-09-05 8:51 UTC (permalink / raw)
To: Matthieu Moy; +Cc: Johan Herland, git, Junio C Hamano, Linus Torvalds
In-Reply-To: <vpqtzq91p5z.fsf@bauges.imag.fr>
[-- Attachment #1: Type: text/plain, Size: 1374 bytes --]
On Wed, Sep 05, 2007 at 08:39:52AM +0000, Matthieu Moy wrote:
> Johan Herland <johan@herland.net> writes:
>
> > When git-fetch and git-commit has done its job and is about to exit, it checks
> > the number of loose object, and if too high tells the user something
> > like "There are too many loose objects in the repo, do you want me to repack?
> > (y/N)". If the user answers "n" or simply <Enter>,
>
> I don't like commands to be interactive if they don't _need_ to be so.
> It kills scripting, it makes it hard for a front-end (git gui or so)
> to use the command, ...
There is absolutely no problem here, as it can be avoided if the
output is not a tty. It's not _that_ hard to guess if you're currently
running in a script or in an interactive shell after all.
Really, git commit/fetch/... whatever suggesting to repack/gc when it
believes it begins to be critical to performance is not a bad idea.
Though the risk is that the warning could be printed very often, but
that can be avoided trivially by just writing to a state file in the
.git directory that the warning was printed not so long time ago, and
that git should STFU for some more commits/time.
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: [PATCH] Simplify strbuf uses in fast-import.c using the proper functions.
From: Pierre Habouzit @ 2007-09-05 8:57 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, René Scharfe
In-Reply-To: <7vodghk057.fsf@gitster.siamese.dyndns.org>
[-- Attachment #1: Type: text/plain, Size: 1975 bytes --]
On Wed, Sep 05, 2007 at 08:05:24AM +0000, Junio C Hamano wrote:
> Pierre Habouzit <madcoder@debian.org> writes:
>
> > oh boy, yes I fixed that in my local patch collection. I'm waiting for
> > a few hours (days ?) to see if there will be some more comments, I've
> > integrated every single one done here already (and some I had on IRC
> > too), and I'll repost a new clean series that I intend to be a real
> > proposal for inclusion.
>
> Ah, I actually did the single trivial fix-up (ALLOC_GROW) and
> have been looking at it, but I'll discard it. Thanks.
Yeah I integrated that as well already. I think I'll post the patches
at the end of the Day (French Timezone so in 5 or 6 hours at least) to
be sure nobody has any good remarks to do first. If you want I can share
my branch somewhere if you want to look at it, just ask.
> > And yes, this patch is a perfect example of the gain we have to share
> > a common buffer API. The code looks (at least to me) way nicer, and if
> > you look in the details, we perform as many memory allocations, copies,
> > and so on as in the previous version.
>
> Wait. What is your point in saying this? Is that a good thing
> to do "as many"? "API is cleaned-up and it is much easier to
> read but we do not do more than before" is certainly a *BIG*
> plus, so perhaps that is what you meant, but when I first read
> it I thought you were saying "we are not optimizing it at all"
> in a negative sense.
Sorry if I chose wrong words to say that, it is clearly a big plus ! I
wrote it having in mind that very often abstractions and high level APIs
often degrade performance. It does not, and that's a huge win. The point
is that "despite the somehow higher level API we don't loose an inch of
performance".
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: David Kastrup @ 2007-09-05 9:02 UTC (permalink / raw)
To: git
In-Reply-To: <20070905085158.GC31750@artemis.corp>
Pierre Habouzit <madcoder@debian.org> writes:
> On Wed, Sep 05, 2007 at 08:39:52AM +0000, Matthieu Moy wrote:
>> Johan Herland <johan@herland.net> writes:
>>
>> > When git-fetch and git-commit has done its job and is about to exit, it checks
>> > the number of loose object, and if too high tells the user something
>> > like "There are too many loose objects in the repo, do you want me to repack?
>> > (y/N)". If the user answers "n" or simply <Enter>,
>>
>> I don't like commands to be interactive if they don't _need_ to be so.
>> It kills scripting, it makes it hard for a front-end (git gui or so)
>> to use the command, ...
>
> There is absolutely no problem here, as it can be avoided if the
> output is not a tty.
Which output? stdout? stderr? Where is the question appearing?
What if the command has been started in the background? What if stdin
(not stdout) is from a pipe, maybe for taking a commit message? What
if stdin is from a pseudo-tty because the commit has been started with
an internal shell command inside of Emacs, and the command/message
will only get echoed once git-commit completes?
> It's not _that_ hard to guess if you're currently running in a
> script or in an interactive shell after all.
Oh, it is not hard to _guess_. Just throw a die. What is hard is to
_know_ 100% sure that one is doing the right thing and not breaking
any legitimate use.
--
David Kastrup
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Matthieu Moy @ 2007-09-05 9:04 UTC (permalink / raw)
To: Johan Herland; +Cc: git, Junio C Hamano, Linus Torvalds
In-Reply-To: <20070905085158.GC31750@artemis.corp>
Pierre Habouzit <madcoder@debian.org> writes:
> On Wed, Sep 05, 2007 at 08:39:52AM +0000, Matthieu Moy wrote:
>> Johan Herland <johan@herland.net> writes:
>>
>> > When git-fetch and git-commit has done its job and is about to exit, it checks
>> > the number of loose object, and if too high tells the user something
>> > like "There are too many loose objects in the repo, do you want me to repack?
>> > (y/N)". If the user answers "n" or simply <Enter>,
>>
>> I don't like commands to be interactive if they don't _need_ to be so.
>> It kills scripting, it makes it hard for a front-end (git gui or so)
>> to use the command, ...
>
> There is absolutely no problem here, as it can be avoided if the
> output is not a tty. It's not _that_ hard to guess if you're currently
> running in a script or in an interactive shell after all.
I do find it hard to guess _reliably_ if you're running interactively
or not. For example, I've been bitten recently by "git log" running
inside a pager while I was launching it non-interactively inside Emacs
(as part of DVC). I don't know whether this was git's or Emacs's
fault, and the fix was not too hard (GIT_PAGER=cat), but it took some
of my time to get it working.
Adding more interactive stuff means adding more opportunities for this
kind of problems. None will be a huge problem, but each problem will
take some time to be fixed (I'm pretty sure adding an interactive
prompt in git-commit will break DVC's commit functionality, and we'll
have to fix it).
> Really, git commit/fetch/... whatever suggesting to repack/gc when it
> believes it begins to be critical to performance is not a bad idea.
_Suggesting_ is a good idea, definitely. Something like
if (number_of_unpacked > 1000 && number_of_unpacked < 10000) {
printf ("more than 1000 unpacked objects. Think of running git-gc\n");
} else if (number_of_unpacked >= 10000) {
printf ("HEY, WHAT THE HELL ARE YOU DOING WITH >10000 UNPACKED OBJECTS???\n"
"I TOLD YOU TO REPACK\n");
}
would be fine with me. The proposal to run git-gc in the background,
with low priority seems to be a good idea too.
But please, don't put an interactive prompt where it's not needed.
--
Matthieu
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Steven Grimm @ 2007-09-05 9:07 UTC (permalink / raw)
To: David Kastrup; +Cc: Linus Torvalds, Git Mailing List
In-Reply-To: <86ps0xcwxo.fsf@lola.quinscape.zz>
David Kastrup wrote:
> You'll potentially get accumulating unfinished files from
> aborted/killed repack processes.
Which can get cleaned up when the next repack starts. This is no
different from unfinished files accumulating from aborted/killed manual
repacks.
> If communication fails, you'll get a
> new repack session for every command you start.
Git handles this already:
$ git-gc
fatal: unable to create '.git/packed-refs.lock': File exists
error: failed to run pack-refs
Presumably in that case you would simply not fire up a new repack.
> If a repository is used by multiple people...
>
Then the first one will kick off the repack, and subsequent ones won't.
> And so on. The multiuser aspect makes it a bad idea to do any
> janitorial tasks automatically. You don't really want every user to
> start a repack at the same time.
>
Quite true, but that's already impossible, so not a problem.
One other thing: The heuristics for this can be such that users who are
already regularly running git-gc by hand will see no change in behavior.
Their repos will never get to a bad enough state that the automatic
git-gc is invoked. Old-timers who run git-gc might, in theory, never
even notice a change like this.
-Steve
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Junio C Hamano @ 2007-09-05 9:07 UTC (permalink / raw)
To: Steven Grimm; +Cc: Linus Torvalds, Git Mailing List
In-Reply-To: <46DE6DBC.30704@midwinter.com>
Steven Grimm <koreth@midwinter.com> writes:
> The fact that git sometimes stores your files individually in the .git
> directory and sometimes bundles them together into big archives should
> be an implementation detail that end-users don't have to worry about
> day to day...
[alias]
begin = gc
leave = gc
That is, the user's manual says 'at the beginning of the day,
run "git begin" to start the day, and at the end of day, run
"git leave" to conclude your day', without saying why ;-)
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: David Kastrup @ 2007-09-05 9:13 UTC (permalink / raw)
To: git
In-Reply-To: <46DE71BC.5040008@midwinter.com>
Steven Grimm <koreth@midwinter.com> writes:
> David Kastrup wrote:
>> You'll potentially get accumulating unfinished files from
>> aborted/killed repack processes.
>
> Which can get cleaned up when the next repack starts. This is no
> different from unfinished files accumulating from aborted/killed
> manual repacks.
>
>> If communication fails, you'll get a
>> new repack session for every command you start.
>
> Git handles this already:
>
> $ git-gc
> fatal: unable to create '.git/packed-refs.lock': File exists
> error: failed to run pack-refs
>
> Presumably in that case you would simply not fire up a new repack.
>
>> If a repository is used by multiple people...
>>
>
> Then the first one will kick off the repack, and subsequent ones won't.
And the first one might get habitually killed by the user unwittingly
having started it (because he really only logs in for shorter amounts
of times than needed for git-gc to finish), wasting disk space and
time all the while.
--
David Kastrup
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Pierre Habouzit @ 2007-09-05 9:14 UTC (permalink / raw)
To: Steven Grimm; +Cc: Linus Torvalds, Git Mailing List
In-Reply-To: <46DE6DBC.30704@midwinter.com>
[-- Attachment #1: Type: text/plain, Size: 3080 bytes --]
On Wed, Sep 05, 2007 at 08:50:04AM +0000, Steven Grimm wrote:
> Pierre Habouzit wrote:
> > Well independently from the fact that one could suppose that users
> >should use gc on their own, the big nasty problem with repacking is that
> >it's really slow. And I just can't imagine git that I use to commit
> >blazingly fast, will then be unavailable for a very long time (repacks
> >on my projects -- that are not as big as the kernel but still -- usually
> >take more than 10 to 20 seconds each).
> >
>
> What about kicking off a repack in the background at the ends of certain
> commands? With an option to disable, of course. It could run at a low
> priority and could even sleep a lot to avoid saturating the system's
> disks -- since it'd be running asynchronously there should be no problem
> if it takes longer to run.
there is an issue with that: repack is memory and CPU intensive. Of
course renicing the process deals with the CPU issue, but not with the
memory one. I've often seen repacks eat more than 300 to 400Mo of memory
on not so big repositories: it seems (and experience tells me that, not
looking at the code) that if you have some big binary blobs (we have
.swf's and .fla's in our repository) it can consume quite a lot of RAM
to (presumably) compute efficient deltas.
Sadly there is no way to "renice" the ram usage of a process. Once a
repack is launched, it will make your system swap, and put the whole
computer on its knees.
> IMO expecting end users to regularly perform what are essentially
> database administration tasks (running git-gc is akin to rebuilding
> indexes or packing tables on a DBMS) is naive. Heck, even database
> administrators don't like to run database administration commands;
Well that's what crons are for. When you install a SGBD in a
reasonable enough distro, it comes with the optimizing scripts in crons,
launched at a reasonable period of the day (localtime). So the
comparison doesn't hold. And that's exactly the problem: it's quite hard
to ship git with an optimizing cron task, because we can't know where
the user will keep his repositories, and when he works, so you have
somehow to do it yourself.
Or you can deal with that with a "rule". At work, we have our devel
trees under $HOME/dev/, so the cron we use is just a (roughly):
find $HOME/dev/ -name .git -type d -maxdepth 4 | while read repo
do
GIT_DIR="$repo" git gc
done
As we work on NFS, with a new developper, we can just setup the cron
for him at a date where he's not supposed to be at work, and that's it.
I'm not sure there is a good solution at all.
Or we could also provide a: git-coffee-break command that would tell
git: do whatever you want with this computer in the next 10 minutes,
there won't be anyone watching, but I assume tea-lovers will feel
excluded.
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: David Kastrup @ 2007-09-05 9:13 UTC (permalink / raw)
To: git
In-Reply-To: <46DE6DBC.30704@midwinter.com>
Steven Grimm <koreth@midwinter.com> writes:
> Pierre Habouzit wrote:
>> Well independently from the fact that one could suppose that users
>> should use gc on their own, the big nasty problem with repacking is that
>> it's really slow. And I just can't imagine git that I use to commit
>> blazingly fast, will then be unavailable for a very long time (repacks
>> on my projects -- that are not as big as the kernel but still -- usually
>> take more than 10 to 20 seconds each).
>>
>
> What about kicking off a repack in the background at the ends of
> certain commands? With an option to disable, of course. It could run
> at a low priority and could even sleep a lot to avoid saturating the
> system's disks -- since it'd be running asynchronously there should
> be no problem if it takes longer to run.
You'll potentially get accumulating unfinished files from
aborted/killed repack processes. If communication fails, you'll get a
new repack session for every command you start. If a repository is
used by multiple people...
And so on. The multiuser aspect makes it a bad idea to do any
janitorial tasks automatically. You don't really want every user to
start a repack at the same time.
--
David Kastrup
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Martin Langhoff @ 2007-09-05 9:27 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Steven Grimm, Linus Torvalds, Git Mailing List
In-Reply-To: <7vbqchjx9f.fsf@gitster.siamese.dyndns.org>
On 9/5/07, Junio C Hamano <gitster@pobox.com> wrote:
> [alias]
> begin = gc
> leave = gc
>
> That is, the user's manual says 'at the beginning of the day,
> run "git begin" to start the day, and at the end of day, run
> "git leave" to conclude your day', without saying why ;-)
I actually like that one ;-)
Anyway - this is turning out to be a bit of a bikeshed-painting event.
You guys should google earlier discussions on this very same subject.
They have always ended in "automatic=bad", "warning=good", and
"careful or you might be called an idiot" before ;-)
cheers,
martin
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Matthieu Moy @ 2007-09-05 9:33 UTC (permalink / raw)
To: Martin Langhoff
Cc: Junio C Hamano, Steven Grimm, Linus Torvalds, Git Mailing List
In-Reply-To: <46a038f90709050227u777ed7b9w23dc3bab13c7b09b@mail.gmail.com>
"Martin Langhoff" <martin.langhoff@gmail.com> writes:
> On 9/5/07, Junio C Hamano <gitster@pobox.com> wrote:
>> [alias]
>> begin = gc
>> leave = gc
>>
>> That is, the user's manual says 'at the beginning of the day,
>> run "git begin" to start the day, and at the end of day, run
>> "git leave" to conclude your day', without saying why ;-)
>
> I actually like that one ;-)
There's indeed a real idea behind that. The issue is that the alias
shouldn't be just "gc", but "find-all-repositories-and-do-gc-there".
Currently, AFAIK, that can only be done with a (trivial) script
external to git. I suppose this can easily be added to the core git
porcelain. Perhaps a "git gc --recursive" would do.
It doesn't solve the problem, but makes it easier to solve it (git gc
--recursive in cron for example).
--
Matthieu
^ permalink raw reply
* [RFC/PATCH] git-svn: add support for --first-parent
From: Lars Hjemli @ 2007-09-05 9:35 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Eric Wong
When git-svn uses git-log to find embedded 'git-svn-id'-lines in commit
messages, it can get confused when local history contains merges with
other git-svn branches. But if --first-parent is supplied to git-log,
working_head_info() will only see 'branch-local' commits and thus the
first commit containing a 'git-svn-id' line should refer to the correct
subversion branch.
Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---
This passes the test-suite and I've used it to correctly dcommit against a
real-life repository after merging subversion-branches in git ('dcommit -n'
reported the wrong subversion branch while 'dcommit -n --first-parent' got
it right, so I did 'dcommit --first-parent' and then inspected the logs and
diffs in the subversion repo and everything looked swell).
But I'm not a perl person, my understanding of git-svn is limited and the
use of --first-parent may not be the right solution to the problem...
Documentation/git-svn.txt | 10 ++++++++++
git-svn.perl | 17 +++++++++++++----
2 files changed, 23 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index be2e34e..42d7b82 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -317,6 +317,16 @@ This is only used with the 'dcommit' command.
Print out the series of git arguments that would show
which diffs would be committed to SVN.
+--first-parent::
+
+This is only used with the 'dcommit', 'rebase', 'log', 'find-rev' and
+'show-ignore' commands.
+
+These commands tries to detect the upstream subversion branch by means of
+the embedded 'git-svn-id' line in commit messages. When --first-parent is
+specified, git-svn only follows the first parent of each commit, effectively
+ignoring commits brought into the current branch through merge-operations.
+
--
ADVANCED OPTIONS
diff --git a/git-svn.perl b/git-svn.perl
index d3c8cd0..d21eb7f 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -59,7 +59,7 @@ my ($_stdin, $_help, $_edit,
$_template, $_shared,
$_version, $_fetch_all, $_no_rebase,
$_merge, $_strategy, $_dry_run, $_local,
- $_prefix, $_no_checkout, $_verbose);
+ $_prefix, $_no_checkout, $_verbose, $_first_parent);
$Git::SVN::_follow_parent = 1;
my %remote_opts = ( 'username=s' => \$Git::SVN::Prompt::_username,
'config-dir=s' => \$Git::SVN::Ra::config_dir,
@@ -119,12 +119,15 @@ my %cmd = (
'dry-run|n' => \$_dry_run,
'fetch-all|all' => \$_fetch_all,
'no-rebase' => \$_no_rebase,
+ 'first-parent' => \$_first_parent,
%cmt_opts, %fc_opts } ],
'set-tree' => [ \&cmd_set_tree,
"Set an SVN repository to a git tree-ish",
{ 'stdin|' => \$_stdin, %cmt_opts, %fc_opts, } ],
'show-ignore' => [ \&cmd_show_ignore, "Show svn:ignore listings",
- { 'revision|r=i' => \$_revision } ],
+ { 'revision|r=i' => \$_revision,
+ 'first-parent' => \$_first_parent
+ } ],
'multi-fetch' => [ \&cmd_multi_fetch,
"Deprecated alias for $0 fetch --all",
{ 'revision|r=s' => \$_revision, %fc_opts } ],
@@ -145,15 +148,19 @@ my %cmd = (
'authors-file|A=s' => \$_authors,
'color' => \$Git::SVN::Log::color,
'pager=s' => \$Git::SVN::Log::pager,
+ 'first-parent' => \$_first_parent
} ],
'find-rev' => [ \&cmd_find_rev, "Translate between SVN revision numbers and tree-ish",
- { } ],
+ {
+ 'first-parent' => \$_first_parent
+ } ],
'rebase' => [ \&cmd_rebase, "Fetch and rebase your working directory",
{ 'merge|m|M' => \$_merge,
'verbose|v' => \$_verbose,
'strategy|s=s' => \$_strategy,
'local|l' => \$_local,
'fetch-all|all' => \$_fetch_all,
+ 'first-parent' => \$_first_parent,
%fc_opts } ],
'commit-diff' => [ \&cmd_commit_diff,
'Commit a diff between two trees',
@@ -811,7 +818,9 @@ sub cmt_metadata {
sub working_head_info {
my ($head, $refs) = @_;
- my ($fh, $ctx) = command_output_pipe('log', '--no-color', $head);
+ my @args = ('log', '--no-color');
+ push @args, '--first-parent' if $_first_parent;
+ my ($fh, $ctx) = command_output_pipe(@args, $head);
my $hash;
my %max;
while (<$fh>) {
--
1.5.3.1.g7e90d-dirty
^ permalink raw reply related
* Re: git-svn and a nested branches folder
From: Russ Brown @ 2007-09-05 9:56 UTC (permalink / raw)
To: Eric Wong; +Cc: git
In-Reply-To: <20070905001513.GA9362@soma>
Eric Wong wrote:
> Russ Brown <pickscrape@gmail.com> wrote:
>> Hi,
>
> Hi Russ,
>
>> I'm having some trouble with using git-svn to fetch a repository, and I
>> think it's because the repository doesn't store branches as a flat list
>> directly under the 'branches' directory.
>>
>> Basically, we have a structure like this:
>>
>> |
>> +-trunk
>> +-tags
>> +-branches
>> + category-a
>> + branch-a
>> + branch-b
>> + category-b
>> + branch-c
>> + branch-d
>>
>> etc. category-a and category-b are simple directories created using svn
>> mkdir. The branches are created using svn cp.
>>
>> It helps us to organise the branches better, but the rationale is
>> besides the point. The problem is that git-svn seems to want to treat
>> category-a and category-b as branches, which isn't right at all. As a
>> result, git-svn seems to skip most (if not all) revisions that occur in
>> these directories and creates a lot of entries in unhandled.log.
>
> This is a known problem with the way git-svn handles branches and tags.
> Nested branch (and tags) structures aren't supported with globbing and
> so you can't have more than one branches/tags specification in your
> config.
>
>> I've also encountered an index corruption in one of the
>> .git/svn/<branch>/index files which I think it probably related.
>
> Not sure about that. git-svn should detect and attempt to fix index
> corruptions (which happened for me since I interrupt git-svn quite
> often when working on it).
>
I've since trashes that repo and it's not happened again, so it must
have been cosmic particles or something. :)
>> I've had a quick look at the source myself, but perl isn't my strong
>> point. What I think it should do is something like recurse down the tree
>> from the directory given looking for folders that copy from trunk (or
>> some other branch that already exists). That would work perfectly well
>> for the default flat branch storage method as well as the one we use.
>
> The globbing functionality of branches in git-svn is some of the ugliest
> code I've ever written in my life. Somehow I got it working for the
> simple general cases but I've been afraid to touch it for a while...
>
Yeah, I know what you mean. :) A really nasty thing to get working and
once you do you don't want to touch it again. :)
>> The only other problem is in branch naming, which could clash if you
>> only use the outer-most directory name, so I'd suggest something that
>> involves concatenating the folders in the path relative to 'branches' to
>> keep them unique (if git can handle slashes in branch names then all the
>> better).
>
> As Peter suggested, disable globbing for branches and use explicit
> fetch refspecs for now...
>
I've actually knocked up a rough script which generates a list of
refspec lines for you given a repo URL, trunk reference and branches
directory. It uses svn log -v --xml and pipes it through a couple of
XSLT templates, and basically looks for all copies that copy from trunk
(recursively: so it includes branches of branches too). I can post it to
the list if you'd find it useful or interesting.
It's generating output that looks sensible to me, but the results aren't
quite what I'd expected. I'll paste a sample in here in case there's
anything obvious someone might spot that I've missed
# This line was generated by git-svn init, and I kept it
fetch = all/trunk:refs/remotes/trunk
# These lines generated by my tool, dirnames replaced for security reasons:
fetch = branches/folder/projecta:refs/remotes/svn/folder/projecta
fetch = branches/folder/projectb:refs/remotes/svn/folder/projectb
fetch = branches/folder/projectc:refs/remotes/svn/folder/projectc
fetch = branches/folder/projectd:refs/remotes/svn/folder/projectd
fetch = branches/folder/projecte:refs/remotes/svn/folder/projecte
fetch = branches/folder/projectf:refs/remotes/svn/folder/projectf
fetch = branches/folder/projectg:refs/remotes/svn/folder/projectg
git branch -a doesn't list any of those branches after fetch completes.
Looking back at the output from fetch, all revisions applied were to trunk.
Anything wrong with those fetch lines?
Thanks for your time.
--
Russ
^ permalink raw reply
* Re: git-svn and a nested branches folder
From: Eric Wong @ 2007-09-05 10:09 UTC (permalink / raw)
To: Russ Brown; +Cc: git
In-Reply-To: <46DE7D37.9040905@gmail.com>
Russ Brown <pickscrape@gmail.com> wrote:
> Eric Wong wrote:
> > Russ Brown <pickscrape@gmail.com> wrote:
> >> Basically, we have a structure like this:
> >>
> >> |
> >> +-trunk
> >> +-tags
> >> +-branches
> >> + category-a
> >> + branch-a
> >> + branch-b
> >> + category-b
> >> + branch-c
> >> + branch-d
> >> The only other problem is in branch naming, which could clash if you
> >> only use the outer-most directory name, so I'd suggest something that
> >> involves concatenating the folders in the path relative to 'branches' to
> >> keep them unique (if git can handle slashes in branch names then all the
> >> better).
> >
> > As Peter suggested, disable globbing for branches and use explicit
> > fetch refspecs for now...
> >
>
> I've actually knocked up a rough script which generates a list of
> refspec lines for you given a repo URL, trunk reference and branches
> directory. It uses svn log -v --xml and pipes it through a couple of
> XSLT templates, and basically looks for all copies that copy from trunk
> (recursively: so it includes branches of branches too). I can post it to
> the list if you'd find it useful or interesting.
>
> It's generating output that looks sensible to me, but the results aren't
> quite what I'd expected. I'll paste a sample in here in case there's
> anything obvious someone might spot that I've missed
>
> # This line was generated by git-svn init, and I kept it
> fetch = all/trunk:refs/remotes/trunk
>
> # These lines generated by my tool, dirnames replaced for security reasons:
>
> fetch = branches/folder/projecta:refs/remotes/svn/folder/projecta
> fetch = branches/folder/projectb:refs/remotes/svn/folder/projectb
> fetch = branches/folder/projectc:refs/remotes/svn/folder/projectc
> fetch = branches/folder/projectd:refs/remotes/svn/folder/projectd
> fetch = branches/folder/projecte:refs/remotes/svn/folder/projecte
> fetch = branches/folder/projectf:refs/remotes/svn/folder/projectf
> fetch = branches/folder/projectg:refs/remotes/svn/folder/projectg
>
> git branch -a doesn't list any of those branches after fetch completes.
> Looking back at the output from fetch, all revisions applied were to trunk.
>
> Anything wrong with those fetch lines?
>From your tree diagram, it seemed that trunk/ and branches/ were at
the same depth in your SVN repository. However, in your generated
fetch lines they all started with "branches/" in front, yet your
trunk fetch line had "all/" in front of trunk, so maybe prefixing
the generated ones with "all/" helps?
> Thanks for your time.
No problem.
--
Eric Wong
^ permalink raw reply
* Re: git-svn and a nested branches folder
From: Russ Brown @ 2007-09-05 10:15 UTC (permalink / raw)
To: Eric Wong; +Cc: git
In-Reply-To: <20070905100919.GA11074@soma>
Eric Wong wrote:
> Russ Brown <pickscrape@gmail.com> wrote:
>> Eric Wong wrote:
>>> Russ Brown <pickscrape@gmail.com> wrote:
>>>> Basically, we have a structure like this:
>>>>
>>>> |
>>>> +-trunk
>>>> +-tags
>>>> +-branches
>>>> + category-a
>>>> + branch-a
>>>> + branch-b
>>>> + category-b
>>>> + branch-c
>>>> + branch-d
>
>>>> The only other problem is in branch naming, which could clash if you
>>>> only use the outer-most directory name, so I'd suggest something that
>>>> involves concatenating the folders in the path relative to 'branches' to
>>>> keep them unique (if git can handle slashes in branch names then all the
>>>> better).
>>> As Peter suggested, disable globbing for branches and use explicit
>>> fetch refspecs for now...
>>>
>> I've actually knocked up a rough script which generates a list of
>> refspec lines for you given a repo URL, trunk reference and branches
>> directory. It uses svn log -v --xml and pipes it through a couple of
>> XSLT templates, and basically looks for all copies that copy from trunk
>> (recursively: so it includes branches of branches too). I can post it to
>> the list if you'd find it useful or interesting.
>>
>> It's generating output that looks sensible to me, but the results aren't
>> quite what I'd expected. I'll paste a sample in here in case there's
>> anything obvious someone might spot that I've missed
>>
>> # This line was generated by git-svn init, and I kept it
>> fetch = all/trunk:refs/remotes/trunk
>>
>> # These lines generated by my tool, dirnames replaced for security reasons:
>>
>> fetch = branches/folder/projecta:refs/remotes/svn/folder/projecta
>> fetch = branches/folder/projectb:refs/remotes/svn/folder/projectb
>> fetch = branches/folder/projectc:refs/remotes/svn/folder/projectc
>> fetch = branches/folder/projectd:refs/remotes/svn/folder/projectd
>> fetch = branches/folder/projecte:refs/remotes/svn/folder/projecte
>> fetch = branches/folder/projectf:refs/remotes/svn/folder/projectf
>> fetch = branches/folder/projectg:refs/remotes/svn/folder/projectg
>>
>> git branch -a doesn't list any of those branches after fetch completes.
>> Looking back at the output from fetch, all revisions applied were to trunk.
>>
>> Anything wrong with those fetch lines?
>
> From your tree diagram, it seemed that trunk/ and branches/ were at
> the same depth in your SVN repository. However, in your generated
> fetch lines they all started with "branches/" in front, yet your
> trunk fetch line had "all/" in front of trunk, so maybe prefixing
> the generated ones with "all/" helps?
>
Ah! I think I know what I've done. I generated the fetch list from the
svn repository itself, but ran svn-init against an svk mirror (using
--use-svm-props). Nicely spotted. :)
Thanks again!
>> Thanks for your time.
>
> No problem.
>
--
Russ
^ permalink raw reply
* Re: [PATCH] Add a new lstat and fstat implementation based on Win32 API
From: Johannes Schindelin @ 2007-09-05 10:15 UTC (permalink / raw)
To: Marius Storm-Olsen; +Cc: git, Johannes Sixt, Johannes Sixt, Git Mailing List
In-Reply-To: <46DE4B2E.70105@trolltech.com>
Hi,
On Wed, 5 Sep 2007, Marius Storm-Olsen wrote:
> Rutger Nijlunsing said the following on 04.09.2007 23:02:
> > On Tue, Sep 04, 2007 at 01:57:38PM +0100, Johannes Schindelin wrote:
> > > Now I am utterly confused. MSDN says
> > > FILETIME
> > > Contains a 64-bit value representing the number of 100-nanosecond
> > > intervals since January 1, 1601 (UTC).
> > > Hmm.
> >
> > [Warning: war stories ahead...]
> > If you really, really want to know more:
> > http://search.cpan.org/~shay/Win32-UTCFileTime-1.45/lib/Win32/UTCFileTime.pm
>
> Thanks, seems like it's the right decision then to ensure that we use UTC
> throughout Git on Windows
>
> Hannes & Dscho, looks like we should revert
> http://repo.or.cz/w/git/mingw/4msysgit.git?a=commitdiff;h=683775c00d9fb95bcbe4632f95b67a96b902fa59
> then, and rather apply Dscho's patch for a custom time() implementation.
> Dscho, was the custom implementation of time() enough to fix the issues
> for you?
Umm. I am helplessly overloaded with non-Git work, and just made sure
that it compiles ;-) Besides, I did not really understand what I was
doing; just copying your code to convert from FILETIME to time_t.
Sorry,
Dscho
^ permalink raw reply
* Re: [RFC/PATCH] git-svn: add support for --first-parent
From: Eric Wong @ 2007-09-05 10:19 UTC (permalink / raw)
To: Lars Hjemli; +Cc: git, Junio C Hamano
In-Reply-To: <1188984929315-git-send-email-hjemli@gmail.com>
Lars Hjemli <hjemli@gmail.com> wrote:
> When git-svn uses git-log to find embedded 'git-svn-id'-lines in commit
> messages, it can get confused when local history contains merges with
> other git-svn branches. But if --first-parent is supplied to git-log,
> working_head_info() will only see 'branch-local' commits and thus the
> first commit containing a 'git-svn-id' line should refer to the correct
> subversion branch.
>
> Signed-off-by: Lars Hjemli <hjemli@gmail.com>
> ---
>
> This passes the test-suite and I've used it to correctly dcommit against a
> real-life repository after merging subversion-branches in git ('dcommit -n'
> reported the wrong subversion branch while 'dcommit -n --first-parent' got
> it right, so I did 'dcommit --first-parent' and then inspected the logs and
> diffs in the subversion repo and everything looked swell).
>
> But I'm not a perl person, my understanding of git-svn is limited and the
> use of --first-parent may not be the right solution to the problem...
At first glance this seems reasonable. I didn't know about git-log
--first-parent until now, but it seems like a good 80% solution.
Ideally, we'd probably stop, say something and give the user the choice
of branches if multiple parents available.
Acked-by-for-next-or-pu: Eric Wong <normalperson@yhbt.net>
> Documentation/git-svn.txt | 10 ++++++++++
> git-svn.perl | 17 +++++++++++++----
> 2 files changed, 23 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
> index be2e34e..42d7b82 100644
> --- a/Documentation/git-svn.txt
> +++ b/Documentation/git-svn.txt
> @@ -317,6 +317,16 @@ This is only used with the 'dcommit' command.
> Print out the series of git arguments that would show
> which diffs would be committed to SVN.
>
> +--first-parent::
> +
> +This is only used with the 'dcommit', 'rebase', 'log', 'find-rev' and
> +'show-ignore' commands.
> +
> +These commands tries to detect the upstream subversion branch by means of
> +the embedded 'git-svn-id' line in commit messages. When --first-parent is
> +specified, git-svn only follows the first parent of each commit, effectively
> +ignoring commits brought into the current branch through merge-operations.
> +
> --
>
> ADVANCED OPTIONS
> diff --git a/git-svn.perl b/git-svn.perl
> index d3c8cd0..d21eb7f 100755
> --- a/git-svn.perl
> +++ b/git-svn.perl
> @@ -59,7 +59,7 @@ my ($_stdin, $_help, $_edit,
> $_template, $_shared,
> $_version, $_fetch_all, $_no_rebase,
> $_merge, $_strategy, $_dry_run, $_local,
> - $_prefix, $_no_checkout, $_verbose);
> + $_prefix, $_no_checkout, $_verbose, $_first_parent);
> $Git::SVN::_follow_parent = 1;
> my %remote_opts = ( 'username=s' => \$Git::SVN::Prompt::_username,
> 'config-dir=s' => \$Git::SVN::Ra::config_dir,
> @@ -119,12 +119,15 @@ my %cmd = (
> 'dry-run|n' => \$_dry_run,
> 'fetch-all|all' => \$_fetch_all,
> 'no-rebase' => \$_no_rebase,
> + 'first-parent' => \$_first_parent,
> %cmt_opts, %fc_opts } ],
> 'set-tree' => [ \&cmd_set_tree,
> "Set an SVN repository to a git tree-ish",
> { 'stdin|' => \$_stdin, %cmt_opts, %fc_opts, } ],
> 'show-ignore' => [ \&cmd_show_ignore, "Show svn:ignore listings",
> - { 'revision|r=i' => \$_revision } ],
> + { 'revision|r=i' => \$_revision,
> + 'first-parent' => \$_first_parent
> + } ],
> 'multi-fetch' => [ \&cmd_multi_fetch,
> "Deprecated alias for $0 fetch --all",
> { 'revision|r=s' => \$_revision, %fc_opts } ],
> @@ -145,15 +148,19 @@ my %cmd = (
> 'authors-file|A=s' => \$_authors,
> 'color' => \$Git::SVN::Log::color,
> 'pager=s' => \$Git::SVN::Log::pager,
> + 'first-parent' => \$_first_parent
> } ],
> 'find-rev' => [ \&cmd_find_rev, "Translate between SVN revision numbers and tree-ish",
> - { } ],
> + {
> + 'first-parent' => \$_first_parent
> + } ],
> 'rebase' => [ \&cmd_rebase, "Fetch and rebase your working directory",
> { 'merge|m|M' => \$_merge,
> 'verbose|v' => \$_verbose,
> 'strategy|s=s' => \$_strategy,
> 'local|l' => \$_local,
> 'fetch-all|all' => \$_fetch_all,
> + 'first-parent' => \$_first_parent,
> %fc_opts } ],
> 'commit-diff' => [ \&cmd_commit_diff,
> 'Commit a diff between two trees',
> @@ -811,7 +818,9 @@ sub cmt_metadata {
>
> sub working_head_info {
> my ($head, $refs) = @_;
> - my ($fh, $ctx) = command_output_pipe('log', '--no-color', $head);
> + my @args = ('log', '--no-color');
> + push @args, '--first-parent' if $_first_parent;
> + my ($fh, $ctx) = command_output_pipe(@args, $head);
> my $hash;
> my %max;
> while (<$fh>) {
--
Eric Wong
^ permalink raw reply
* Re: git-commit: if run with <file> arguments, include files removed through git rm
From: Gerrit Pape @ 2007-09-05 10:20 UTC (permalink / raw)
To: git
In-Reply-To: <7vabs2pgj6.fsf@gitster.siamese.dyndns.org>
On Tue, Sep 04, 2007 at 09:00:13AM -0700, Junio C Hamano wrote:
> Gerrit Pape <pape@smarden.org> writes:
> > This patch lets it additionally use git ls-tree to look for the files in
> > the HEAD tree, but I guess there's a smarter way to fix this.
>
> You raised a good issue, but this also needs other parts of the
Thanks for your detailed explanations.
Sebastian Harl actually noticed this unexpected behavior, and reported
through
http://bugs.debian.org/437817
Regards, Gerrit.
^ permalink raw reply
* Re: [PATCH] Function for updating refs.
From: Carlos Rica @ 2007-09-05 12:03 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Johannes Schindelin
In-Reply-To: <7vy7flk2z5.fsf@gitster.siamese.dyndns.org>
2007/9/5, Junio C Hamano <gitster@pobox.com>:
> Carlos Rica <jasampler@gmail.com> writes:
> > - snprintf(msg, sizeof(msg), "%s: %s", rla, action);
> > - lock = lock_any_ref_for_update(refname, oldval, 0);
> > - if (!lock)
> > - return 1;
> > - if (write_ref_sha1(lock, sha1, msg) < 0)
> > - return 1;
> > - return 0;
> > + if (snprintf(msg, sizeof(msg), "%s: %s", rla, action) >= sizeof(msg))
> > + error("reflog message too long: %.*s...", 50, msg);
>
> The original I did was sloppy and did not detect this situation;
> thanks for fixing it. You do not refuse the primary operation,
> which is to update the ref, so this should be a warning instead
> of an error, I think.
Yes, it is. Also, I tested what would happen when the lock fails. I tried to
lock an already locked ref, and it died printing the message
die("unable to create '%s.lock': %s", path, strerror(errno)); from
lockfile.c. I think this is interesting. There are other failing
reasons that could
make the update_ref function to print its error, but I haven't tested them.
> > diff --git a/send-pack.c b/send-pack.c
> > ...
> This removal makes "struct ref_lock *lock" (not shown in the
> context) unused. I will remove the declaration.
Thank you. Also in builtin-update-ref.c the main function could return
directly that value returned from the call to update_ref().
^ permalink raw reply
* Re: Git's database structure
From: Jon Smirl @ 2007-09-05 13:41 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Theodore Tso, Junio C Hamano, Git Mailing List
In-Reply-To: <46DE5861.4050201@op5.se>
On 9/5/07, Andreas Ericsson <ae@op5.se> wrote:
> Jon Smirl wrote:
> > On 9/4/07, Theodore Tso <tytso@mit.edu> wrote:
> >> On Tue, Sep 04, 2007 at 01:44:47PM -0400, Jon Smirl wrote:
> >>> The current data store design is not very flexible. Databases solved
> >>> the flexibility problem long ago. I'm just wondering if we should
> >>> steal some good ideas out of the database world and apply them to git.
> >>> Ten years from now we may have 100GB git databases and really wish we
> >>> had more flexible ways of querying them.
> >> Databases solved the flexibility problem, at the cost of performance.
> >> And if you use full normalized form in your database scheme, it costs
> >> you even more in performance, because of all of the joins that you
> >> need in order get the information you need to do, you know, useful
> >> work as opposed to database wanking.
> >>
> >> If you take a look at the really big databases with super high
> >> performance requirements, say like those used to managed airline
> >> tickets/reservation/fares, you will find that they are not normalized,
> >> and they are not relational; they can't afford to be. And if you take
> >> a look at some of git competition that use relational databases to
> >> store their SCM data, and take a look at how loooooong they they take
> >> to do even basic operations, I would say that the onus is on you to
> >> prove that normalization is actually a win in terms of real (not
> >> theoretical) advantages, and that it doesn't cause performance to go
> >> into the toilet.
> >>
> >> I think the fundamental disconnect here is that no one is buying your
> >> claim that just because the data design is "more flexible" that this
> >> is automatically a good thing in and of itself, and we should even for
> >> a moment, "put performance aside".
> >
> > It is very easy to get bogged down in performance arguments on
> > database design when the correct answer is that there are always lots
> > of different ways to achieve the same goal. I wanted to defer debating
> > performance until we closely looked at the relationships between the
> > data at an abstract level.
> >
>
> But you cannot. Git is performance-critical, for the same reason every
> other performance-critical application is: It's a tool to save human
> time. Linux development *could* be done using patchfiles by the bundle
> and masses of tarballs. It's just not the fastest way to do it, so enter
> git, and lots of problems just go away. It's not the only way of doing
> it, but it saves time. If you were to add 2 seconds to each commit,
> that's several months of developer time that is lost every day!
>
>
> > Since git hasn't stored all of the fields in the object table (the
> > path is encoded in the index) we are never going to be able to build
> > an alternative way of indexing the object table.
>
> We can still build alternative indexes. They just have to be separate
> from the DAG and the current indexing scheme. Junio has pointed out
> ways of doing this already.
>
> > Not being able to
> > build alternative indexes is likely to cause problems when the
> > database starts getting really big. Without an index every query that
> > can't use the path name index is reduced to doing full table scans.
> >
>
> I've said it before; The most common delimiter used today is paths. It's
> a behaviour git was designed to handle well, because it *is* the most
> common way of limiting and separating content. It's not some random
> fluke that has made git perform very well on actions that commonly
> performed in large scale software projects; Linus designed it that way
> from the start, and kudos to him for a job well done.
This is why I wanted to separate the abstract data structure design
discussion from the performance one. In the flat design indexes are
like caches and can be created and destroyed. There will definitely be
an index created on the the paths. This index will work like the
current tree nodes. The difference is that this index is a cache
unlike the current tree nodes which are an immutable part of the the
data base.
The path name field needs to be moved back into the blobs to support
alternative indexes. For example I want an index on the Signed-off-by
field. I use this index to give me the SHAs for the blobs
Signed-off-by a particular person. In the current design I have no way
of recovering the path name for these blobs other than a brute force
search following every path looking for the right SHA.
>
> > A few things that could benefit from alternative indexing, blame,
> > full-text search, automating the Maintainers file, etc.
> >
>
> Yes, but getting rid of the tree objects and storing pathnames in
> blob objects would penalize log-viewing, diffs and merges, which
> are far more common operations than full-text searches in a software
> project.
>
> > I'm just asking if we really want to make full table scans the only
> > possible way to implement these types of queries. If the answer is no,
> > then let's first explore how to fix things at an abstract level before
> > diving into the performance arguments.
> >
>
> Personally, I really don't care. But you should really have read Junio's
> mail a bit more carefully. He explained about 'notes' that can be attached
> to commits and contain arbitrary data. By all means, create your indexes
> there and use them for whatever you like, but leave the foundation on which
> git was built *alone*. The design hasn't changed since April 2006 (subtrees
> were introduced April 26, I think), because it's a *good* design.
>
> --
> Andreas Ericsson andreas.ericsson@op5.se
> OP5 AB www.op5.se
> Tel: +46 8-230225 Fax: +46 8-230231
>
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Johan De Messemaeker @ 2007-09-05 14:17 UTC (permalink / raw)
To: Matthieu Moy
Cc: Martin Langhoff, Junio C Hamano, Steven Grimm, Linus Torvalds,
Git Mailing List
In-Reply-To: <vpqzm01v4li.fsf@bauges.imag.fr>
On 05 Sep 2007, at 11:33, Matthieu Moy wrote:
>
> There's indeed a real idea behind that. The issue is that the alias
> shouldn't be just "gc", but "find-all-repositories-and-do-gc-there".
>
> Currently, AFAIK, that can only be done with a (trivial) script
> external to git. I suppose this can easily be added to the core git
> porcelain. Perhaps a "git gc --recursive" would do.
>
> It doesn't solve the problem, but makes it easier to solve it (git gc
> --recursive in cron for example).
I'm a git newb so I can be wrong here but ...
Why --recursive? Why not use the submodule-information ?
Johan
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox