* [PATCH (take 4)] gitweb: Use File::Find::find in git_get_projects_list
From: Jakub Narebski @ 2006-09-14 20:18 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <7vejue2omq.fsf@assigned-by-dhcp.cox.net>
Earlier code to get list of projects when $projects_list is a
directory (e.g. when it is equal to $projectroot) had a hardcoded flat
(one level) list of directories. Allow for projects to be in
subdirectories also for $projects_list being a directory by using
File::Find.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/gitweb.perl | 30 ++++++++++++++++++++----------
1 files changed, 20 insertions(+), 10 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index c3544dd..25383bc 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -715,16 +715,26 @@ sub git_get_projects_list {
if (-d $projects_list) {
# search in directory
my $dir = $projects_list;
- opendir my ($dh), $dir or return undef;
- while (my $dir = readdir($dh)) {
- if (-e "$projectroot/$dir/HEAD") {
- my $pr = {
- path => $dir,
- };
- push @list, $pr
- }
- }
- closedir($dh);
+ my $pfxlen = length("$dir");
+
+ File::Find::find({
+ follow_fast => 1, # follow symbolic links
+ dangling_symlinks => 0, # ignore dangling symlinks, silently
+ wanted => sub {
+ # skip current directory
+ return if (m!^[/.]$!);
+ # only directories can be git repositories
+ return unless (-d $_);
+
+ my $subdir = substr($File::Find::name, $pfxlen + 1);
+ # we check related file in $projectroot
+ if (-e "$projectroot/$subdir/HEAD") {
+ push @list, { path => $subdir };
+ $File::Find::prune = 1;
+ }
+ },
+ }, "$dir");
+
} elsif (-f $projects_list) {
# read from file(url-encoded):
# 'git%2Fgit.git Linus+Torvalds'
--
1.4.2
^ permalink raw reply related
* Re: Notes on supporting Git operations in/on partial Working Directories
From: A Large Angry SCM @ 2006-09-14 20:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vu03a2po8.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> A Large Angry SCM <gitzilla@gmail.com> writes:
...
>
> While this may be a good start, you need a lot more than this if
> you want to do (1) and (2):
>
> The tree object contained by a commit is by definition a full
> tree snapshot, so if you want to do a WD_Prefix, you somehow
> need a way to come up with the final tree that is a combination
> of what write-tree would write out from such a partial index
> (i.e. an index that describes only a subdirectory) and the rest
> of the tree from the current HEAD. I think you can more or less
> do this change to Porcelain using today's git core. The
> sequence to emulate it with the today's git would be:
I think you misunderstood, the index file would list all of the tree
entries of the the checked out commit, same as the current index, but
would flag the entries that are actually present in the working
directory. The WD_Prefix is to identify which index entries _can not_ be
part of the working directory, and where the working directory fits in
the full index. That way, all the information needed by the top level
write-tree is still in the index and the cache-tree extension.
^ permalink raw reply
* Re: [PATCH (take 4)] gitweb: Use File::Find::find in git_get_projects_list
From: Jakub Narebski @ 2006-09-14 20:24 UTC (permalink / raw)
To: git
In-Reply-To: <200609142218.59428.jnareb@gmail.com>
Jakub Narebski wrote:
> + # skip current directory
Perhaps it would be better to say "skip $projects_list (top) directory".
I don't think it is important enough to resend a patch.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: Historical kernel repository size
From: Nicolas Pitre @ 2006-09-14 21:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Petr Baudis, Thomas Gleixner, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0609140824580.4388@g5.osdl.org>
On Thu, 14 Sep 2006, Linus Torvalds wrote:
> For better packing, I think I used a larger depth, ie try something like
>
> git repack -a -f --depth=50
>
> to get more improvement. For a historical archive that you don't much use,
> doign the deeper depth is definitely worth it.
Using a larger window helps too. It of course has a direct impact on
the processing to perform a full repack, but it has no runtime costs
when the pack is used. So I'd suggest adding --window=50 to the above.
[ I made those suggestions in person to Thomas at OLS to which
he replied he'd do it when he'd get back home. ;-) ]
Nicolas
^ permalink raw reply
* [PATCH] gitweb: Do not parse refs by hand, use git-peek-remote instead
From: Jakub Narebski @ 2006-09-14 21:27 UTC (permalink / raw)
To: git; +Cc: Junio Hamano
This is response to Linus work on packed refs. Additionally it
makes gitweb work with symrefs, too.
Do not parse refs by hand, using File::Find and reading individual
heads to get hash of reference, but use git-peek-remote output
instead. Assume that the hash for deref (with ^{}) always follows hash
for ref, and that we hav derefs only for tags objects; this removes
call to git_get_type (and git-cat-file -t invocation) for tags, which
speeds "summary" and "tags" views generation, but might slow
generation of "heads" view a bit. As of now we do not save and use the
deref hash.
Remove git_get_hash_by_ref while at it, as git_get_refs_list was the
only place it was used.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
By the way, the previous patch
"gitweb: Use File::Find::find in git_get_projects_list"
was added during working on this feature.
gitweb/gitweb.perl | 39 +++++++++++++++++----------------------
1 files changed, 17 insertions(+), 22 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 25383bc..4e58a6b 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -676,19 +676,6 @@ sub git_get_hash_by_path {
## ......................................................................
## git utility functions, directly accessing git repository
-# assumes that PATH is not symref
-sub git_get_hash_by_ref {
- my $path = shift;
-
- open my $fd, "$projectroot/$path" or return undef;
- my $head = <$fd>;
- close $fd;
- chomp $head;
- if ($head =~ m/^[0-9a-fA-F]{40}$/) {
- return $head;
- }
-}
-
sub git_get_project_description {
my $path = shift;
@@ -1098,17 +1085,25 @@ sub git_get_refs_list {
my @reflist;
my @refs;
- my $pfxlen = length("$projectroot/$project/$ref_dir");
- File::Find::find(sub {
- return if (/^\./);
- if (-f $_) {
- push @refs, substr($File::Find::name, $pfxlen + 1);
+ open my $fd, "-|", $GIT, "peek-remote", "$projectroot/$project/"
+ or return;
+ while (my $line = <$fd>) {
+ chomp $line;
+ if ($line =~ m/^([0-9a-fA-F]{40})\t$ref_dir\/?([^\^]+)$/) {
+ push @refs, { hash => $1, name => $2 };
+ } elsif ($line =~ m/^[0-9a-fA-F]{40}\t$ref_dir\/?.*\^\{\}$/) {
+ # assume that "peeled" ref is always after ref,
+ # and that you "peel" (deref) tags only
+ $refs[$#refs]->{'type'} = "tag";
}
- }, "$projectroot/$project/$ref_dir");
+ }
+ close $fd;
+
+ foreach my $ref (@refs) {
+ my $ref_file = $ref->{'name'};
+ my $ref_id = $ref->{'hash'};
- foreach my $ref_file (@refs) {
- my $ref_id = git_get_hash_by_ref("$project/$ref_dir/$ref_file");
- my $type = git_get_type($ref_id) || next;
+ my $type = $ref->{'type'} || git_get_type($ref_id) || next;
my %ref_item = parse_ref($ref_file, $ref_id, $type);
push @reflist, \%ref_item;
--
1.4.2
^ permalink raw reply related
* Re: Historical kernel repository size
From: Thomas Gleixner @ 2006-09-14 21:32 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Linus Torvalds, Petr Baudis, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0609141714010.2627@xanadu.home>
On Thu, 2006-09-14 at 17:23 -0400, Nicolas Pitre wrote:
> On Thu, 14 Sep 2006, Linus Torvalds wrote:
>
> > For better packing, I think I used a larger depth, ie try something like
> >
> > git repack -a -f --depth=50
> >
> > to get more improvement. For a historical archive that you don't much use,
> > doign the deeper depth is definitely worth it.
>
> Using a larger window helps too. It of course has a direct impact on
> the processing to perform a full repack, but it has no runtime costs
> when the pack is used. So I'd suggest adding --window=50 to the above.
>
> [ I made those suggestions in person to Thomas at OLS to which
> he replied he'd do it when he'd get back home. ;-) ]
Thanks for the reminder. I actually logged into kernel.org already :)
tglx
^ permalink raw reply
* Re: Historical kernel repository size
From: Thomas Gleixner @ 2006-09-14 21:37 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Linus Torvalds, Petr Baudis, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0609141714010.2627@xanadu.home>
On Thu, 2006-09-14 at 17:23 -0400, Nicolas Pitre wrote:
> On Thu, 14 Sep 2006, Linus Torvalds wrote:
>
> > For better packing, I think I used a larger depth, ie try something like
> >
> > git repack -a -f --depth=50
> >
> when the pack is used. So I'd suggest adding --window=50 to the above.
Great advise !
git repack neither accepts --depth nor --window
tglx
^ permalink raw reply
* Re: Historical kernel repository size
From: Nicolas Pitre @ 2006-09-14 21:42 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, Petr Baudis, Git Mailing List
In-Reply-To: <1158269854.5724.240.camel@localhost.localdomain>
On Thu, 14 Sep 2006, Thomas Gleixner wrote:
> On Thu, 2006-09-14 at 17:23 -0400, Nicolas Pitre wrote:
> > On Thu, 14 Sep 2006, Linus Torvalds wrote:
> >
> > > For better packing, I think I used a larger depth, ie try something like
> > >
> > > git repack -a -f --depth=50
> > >
> > when the pack is used. So I'd suggest adding --window=50 to the above.
>
> Great advise !
>
> git repack neither accepts --depth nor --window
Is the GIT version on kernel.org _that_ old?
What a shame...
Nicolas
^ permalink raw reply
* Re: [PATCH (amend)] gitweb: Use File::Find::find in git_get_projects_list
From: Randal L. Schwartz @ 2006-09-14 21:50 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, Junio Hamano
In-Reply-To: <200609141939.39406.jnareb@gmail.com>
>>>>> "Jakub" == Jakub Narebski <jnareb@gmail.com> writes:
Jakub> + sub wanted {
Jakub> + # only directories can be git repositories
Jakub> + return unless (-d $_);
Jakub> +
Jakub> + my $subdir = substr($File::Find::name, $pfxlen + 1);
Jakub> + # we check related file in $projectroot
Jakub> + if (-e "$projectroot/$subdir/HEAD") {
Jakub> + push @list, { path => $subdir };
Jakub> + $File::Find::prune = 1;
Jakub> }
Jakub> }
Jakub> - closedir($dh);
Jakub> +
Jakub> + File::Find::find({
Jakub> + follow_fast => 1, # follow symbolic links
Jakub> + dangling_symlinks => 0, # ignore dangling symlinks, silently
Jakub> + wanted => \&wanted,
Jakub> + }, "$dir");
Don't define a sub inside a sub. That's the "won't stay shared"
problem.
Move that like this:
find( {
follow_fast => 1,
dangling_symlinks => 0,
wanted => sub {
... everything in &wanted above ...
}, $dir);
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
^ permalink raw reply
* Re: Historical kernel repository size
From: Thomas Gleixner @ 2006-09-14 21:54 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Linus Torvalds, Petr Baudis, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0609141742000.2627@xanadu.home>
On Thu, 2006-09-14 at 17:42 -0400, Nicolas Pitre wrote:
> > git repack neither accepts --depth nor --window
>
> Is the GIT version on kernel.org _that_ old?
>
> What a shame...
[tglx@hera history.git]$ git --version
git version 1.4.2.1
tglx
^ permalink raw reply
* Re: [Monotone-devel] cvs import
From: Petr Baudis @ 2006-09-14 21:57 UTC (permalink / raw)
To: Jon Smirl, Keith Packard, dev, monotone-devel, Git Mailing List
In-Reply-To: <20060914015324.GX29625@bcd.geek.com.au>
Dear diary, on Thu, Sep 14, 2006 at 03:53:24AM CEST, I got a letter
where Daniel Carosone <dan@geek.com.au> said that...
> On Wed, Sep 13, 2006 at 08:57:33PM -0400, Jon Smirl wrote:
> > Mozilla is 120,000 files. The complexity comes from 10 years worth of
> > history. A few of the files have around 1,700 revisions. There are
> > about 1,600 branches and 1,000 tags. The branch number is inflated
> > because cvs2svn is generating extra branches, the real number is
> > around 700. The CVS repo takes 4.2GB disk space. cvs2svn turns this
> > into 250,000 commits over about 1M unique revisions.
>
> Those numbers are pretty close to those in the NetBSD repository, and
> between them these probably represent just about the most extensive
> public CVS test data available.
Don't forget OpenOffice. It's just a shame that the OpenOffice CVS
tree is not available for cloning.
http://wiki.services.openoffice.org/wiki/SVNMigration
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: [Monotone-devel] cvs import
From: Shawn Pearce @ 2006-09-14 22:04 UTC (permalink / raw)
To: Petr Baudis
Cc: Jon Smirl, Keith Packard, dev, monotone-devel, Git Mailing List
In-Reply-To: <20060914215728.GL23891@pasky.or.cz>
Petr Baudis <pasky@suse.cz> wrote:
> Don't forget OpenOffice. It's just a shame that the OpenOffice CVS
> tree is not available for cloning.
>
> http://wiki.services.openoffice.org/wiki/SVNMigration
Hmm, the KDE repo is even larger than Mozilla: 19 GB in CVS and
499,367 revisions. Question is, are those distinct file revisions
or SVN revisions? And just what machine did they use that completed
that conversion in 38 hours?
--
Shawn.
^ permalink raw reply
* Re: Historical kernel repository size
From: Thomas Gleixner @ 2006-09-14 22:24 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Linus Torvalds, Petr Baudis, Git Mailing List
In-Reply-To: <1158270859.5724.244.camel@localhost.localdomain>
On Thu, 2006-09-14 at 23:54 +0200, Thomas Gleixner wrote:
> On Thu, 2006-09-14 at 17:42 -0400, Nicolas Pitre wrote:
> > > git repack neither accepts --depth nor --window
> >
> > Is the GIT version on kernel.org _that_ old?
> >
> > What a shame...
>
> [tglx@hera history.git]$ git --version
> git version 1.4.2.1
I know I'm stupid
"git-repack --window=50 --depth=50 -a -f" works
"git-repack -a -f --window=50 --depth=50" does not
Intuitive user interfaces are my favorite pitfalls.
-rw-rw-r-- 1 tglx ftpadmin 13600376 Sep 14 22:16 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.idx
-rw-rw-r-- 1 tglx ftpadmin 158679705 Sep 14 22:16 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.pack
tglx
^ permalink raw reply
* Re: [PATCH] gitweb: Do not parse refs by hand, use git-peek-remote instead
From: Junio C Hamano @ 2006-09-14 23:09 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200609142327.23059.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> Remove git_get_hash_by_ref while at it, as git_get_refs_list was the
> only place it was used.
That's a very good news.
> + open my $fd, "-|", $GIT, "peek-remote", "$projectroot/$project/"
> + or return;
> + while (my $line = <$fd>) {
> + chomp $line;
> + if ($line =~ m/^([0-9a-fA-F]{40})\t$ref_dir\/?([^\^]+)$/) {
> + push @refs, { hash => $1, name => $2 };
> + } elsif ($line =~ m/^[0-9a-fA-F]{40}\t$ref_dir\/?.*\^\{\}$/) {
> + # assume that "peeled" ref is always after ref,
> + # and that you "peel" (deref) tags only
> + $refs[$#refs]->{'type'} = "tag";
> }
The assumption is good; we never show a ref^{} before showing
ref itself. But you probably would want to safeguard yourself
from future changes by not depending on "immediately after".
At least you can check $refs[-1]{'name'} is the same as the
unpeeled ref you are looking at, like this:
if ($line =~ m/^([0-9a-fA-F]{40})\t$ref_dir\/?([^\^]+)$/) {
push @refs, { hash => $1, name => $2 };
} elsif ($line =~ m/^[0-9a-fA-F]{40}\t$ref_dir\/?(.*)\^\{\}$/ &&
$1 eq $refs[-1]{'name'}) {
# most likely a tag is followed by its peeled
# one, and when that happens we know the
# previous one was of type 'tag'.
$refs[$#refs]->{'type'} = "tag";
} ...
> + foreach my $ref (@refs) {
> + my $ref_file = $ref->{'name'};
> + my $ref_id = $ref->{'hash'};
> - my $type = git_get_type($ref_id) || next;
> + my $type = $ref->{'type'} || git_get_type($ref_id) || next;
And this is a good incremental change to reduce number of
external calls.
^ permalink raw reply
* Re: Historical kernel repository size
From: Junio C Hamano @ 2006-09-14 23:15 UTC (permalink / raw)
To: tglx; +Cc: git
In-Reply-To: <1158272651.5724.251.camel@localhost.localdomain>
Thomas Gleixner <tglx@linutronix.de> writes:
>> [tglx@hera history.git]$ git --version
>> git version 1.4.2.1
>
> I know I'm stupid
>
> "git-repack --window=50 --depth=50 -a -f" works
> "git-repack -a -f --window=50 --depth=50" does not
>
> Intuitive user interfaces are my favorite pitfalls.
Whaaaat?
I've run them under "sh -x" and both results in a pipe of:
git-rev-list --objects --all |
git-pack-objects --non-empty --no-reuse-delta --window=50 --depth=50 \
.git/.tmp-<somepid>-pack
Now you are making me really worried.
^ permalink raw reply
* Re: Historical kernel repository size
From: Nicolas Pitre @ 2006-09-15 1:19 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, Petr Baudis, Git Mailing List
In-Reply-To: <1158272651.5724.251.camel@localhost.localdomain>
On Fri, 15 Sep 2006, Thomas Gleixner wrote:
> On Thu, 2006-09-14 at 23:54 +0200, Thomas Gleixner wrote:
> > On Thu, 2006-09-14 at 17:42 -0400, Nicolas Pitre wrote:
> > > > git repack neither accepts --depth nor --window
> > >
> > > Is the GIT version on kernel.org _that_ old?
> > >
> > > What a shame...
> >
> > [tglx@hera history.git]$ git --version
> > git version 1.4.2.1
OK that's recent enough indeed.
> I know I'm stupid
>
> "git-repack --window=50 --depth=50 -a -f" works
> "git-repack -a -f --window=50 --depth=50" does not
>
> Intuitive user interfaces are my favorite pitfalls.
Erm... Both incantations work fine fine here.
> -rw-rw-r-- 1 tglx ftpadmin 13600376 Sep 14 22:16 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.idx
> -rw-rw-r-- 1 tglx ftpadmin 158679705 Sep 14 22:16 pack-4d27038611fe7755938efd4a2745d5d5d35de1c1.pack
And I get the same result as well.
Nicolas
^ permalink raw reply
* [PATCH (take 2)] gitweb: Do not parse refs by hand, use git-peek-remote instead
From: Jakub Narebski @ 2006-09-15 1:43 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7v8xkm2gfs.fsf@assigned-by-dhcp.cox.net>
This is in response to Linus work on packed refs. Additionally it
makes gitweb work with symrefs, too.
Do not parse refs by hand, using File::Find and reading individual
heads to get hash of reference, but use git-peek-remote output
instead. Assume that the hash for deref (with ^{}) always follows hash
for ref, and that we hav derefs only for tags objects; this removes
call to git_get_type (and git-cat-file -t invocation) for tags, which
speeds "summary" and "tags" views generation, but might slow
generation of "heads" view a bit. As of now we do not save and use the
deref hash.
Remove git_get_hash_by_ref while at it, as git_get_refs_list was the
only place it was used.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/gitweb.perl | 41 +++++++++++++++++++----------------------
1 files changed, 19 insertions(+), 22 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 25383bc..b4a890b 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -676,19 +676,6 @@ sub git_get_hash_by_path {
## ......................................................................
## git utility functions, directly accessing git repository
-# assumes that PATH is not symref
-sub git_get_hash_by_ref {
- my $path = shift;
-
- open my $fd, "$projectroot/$path" or return undef;
- my $head = <$fd>;
- close $fd;
- chomp $head;
- if ($head =~ m/^[0-9a-fA-F]{40}$/) {
- return $head;
- }
-}
-
sub git_get_project_description {
my $path = shift;
@@ -1098,17 +1085,27 @@ sub git_get_refs_list {
my @reflist;
my @refs;
- my $pfxlen = length("$projectroot/$project/$ref_dir");
- File::Find::find(sub {
- return if (/^\./);
- if (-f $_) {
- push @refs, substr($File::Find::name, $pfxlen + 1);
+ open my $fd, "-|", $GIT, "peek-remote", "$projectroot/$project/"
+ or return;
+ while (my $line = <$fd>) {
+ chomp $line;
+ if ($line =~ m/^([0-9a-fA-F]{40})\t$ref_dir\/?([^\^]+)$/) {
+ push @refs, { hash => $1, name => $2 };
+ } elsif ($line =~ m/^[0-9a-fA-F]{40}\t$ref_dir\/?(.*)\^\{\}$/ &&
+ $1 eq $refs[-1]{'name'}) {
+ # most likely a tag is followed by its peeled
+ # (deref) one, and when that happens we know the
+ # previous one was of type 'tag'.
+ $refs[-1]{'type'} = "tag";
}
- }, "$projectroot/$project/$ref_dir");
+ }
+ close $fd;
+
+ foreach my $ref (@refs) {
+ my $ref_file = $ref->{'name'};
+ my $ref_id = $ref->{'hash'};
- foreach my $ref_file (@refs) {
- my $ref_id = git_get_hash_by_ref("$project/$ref_dir/$ref_file");
- my $type = git_get_type($ref_id) || next;
+ my $type = $ref->{'type'} || git_get_type($ref_id) || next;
my %ref_item = parse_ref($ref_file, $ref_id, $type);
push @reflist, \%ref_item;
--
1.4.2
^ permalink raw reply related
* Re: Notes on supporting Git operations in/on partial Working Directories
From: Junio C Hamano @ 2006-09-15 2:43 UTC (permalink / raw)
To: gitzilla; +Cc: git
In-Reply-To: <4509B954.60101@gmail.com>
A Large Angry SCM <gitzilla@gmail.com> writes:
> Junio C Hamano wrote:
>> A Large Angry SCM <gitzilla@gmail.com> writes:
> ...
>>
>> While this may be a good start, you need a lot more than this if
>> you want to do (1) and (2):
>>
>> The tree object contained by a commit is by definition a full
>> tree snapshot, so if you want to do a WD_Prefix, you somehow
>> need a way to come up with the final tree that is a combination
>> of what write-tree would write out from such a partial index
>> (i.e. an index that describes only a subdirectory) and the rest
>> of the tree from the current HEAD. I think you can more or less
>> do this change to Porcelain using today's git core. The
>> sequence to emulate it with the today's git would be:
>
> I think you misunderstood, the index file would list all of the tree
> entries of the the checked out commit, same as the current index, but
> would flag the entries that are actually present in the working
> directory. The WD_Prefix is to identify which index entries _can not_
> be part of the working directory, and where the working directory fits
> in the full index. That way, all the information needed by the top
> level write-tree is still in the index and the cache-tree extension.
Ah indeed. That makes it more palatable ;-).
Having said that, I do not necessarily agree that highly modular
projects would want to put everything in one git repository and
track everything as a whole unit.
The primary audience of git, the kernel project, is reasonably
modular (although Andrew seems to be suffering from subsystem
maintainers touching overlapping areas these days and says it is
rather unusual), and is a non-trivial size, yet it has
everything under one umbrella. The model makes sense in that
project, since the core developers need to occasionally change
an internal API wholesale across the tree. The people at fringe
who work only on limited part of the system (e.g. one particular
filesystem implementation), on the other hand, may not care what
happens in the other parts (e.g. random device drivers that
should not interact directly with the filesystem implementation
in question) of the system most of the time, but they do have to
care if the layer closer to the core that their work depends on
changes (e.g. a VFS layer update changes the rule filesystems
must play under), so having to check out the full kernel tree
while they usually work only on one part of it often cumbersome
but sometimes absolutely necessary so it is tolerated.
Everybody is forced to work on the same codebase and merge the
whole tree as a unit, which might inconvenience the people
really at the fringe (e.g. driver writers), but being able to
make sure everything is in sync is a good thing to the core
developers, and that benefit outweighs the convenience of fringe
people (also the core people are who gets to pick the tool they
use ;-).
In the kernel case, out-of-tree driver people have a choice to
build out of tree against just kernel headers as modlues.
Nobody (including git) gives mechanical support to enforce that
this version of the out-of-tree driver must be used only with
such and such main tree, but build procedure and INSTALL
documents of such a driver usually take care of that integration
issues.
I suspect most highly modular projects are run that way, not
just from the version control point of view, but simply because
of people interaction issues. Nobody can be on top of all
possible interface details between many modular pieces of a
truly huge project, so there would be clean separation of parts
and narrow definition of how they mesh together (after all, that
is what being highly modular is all about). And in such a case,
subsystems can be (and I'd even claim they had better be)
version controlled more or less independently with each other,
with certain version dependencies, such as "libfoo subsystem is
used by all of our programs A, B, ..., Z, but recent libfoo 1.29
release added some feature to support enhancement in version 2.4
of program Z. So libfoo 1.29 or later is required if you are
building the latest tip of program Z, but everybody else can
stay at 1.28 if updating libfoo is not convenient, oh by the way
1.30 has a thinko that broke what is used heavily only by
program A, so if you are working on program A use libfoo 1.30 or
later, or stay at libfoo 1.28." And there would be tons of tiny
commits between these point releases.
My point is that while there will always be _some_ version
synchronization requirements between subcomponents of such a
huge highly modular project, it is a lot looser than tracking
each and every change in the entire tree as a whole, like git's
commit does. The model of throwing all subcomponents in a
single repository and trying to track everything as a whole may
not match the real requirement of such a project.
In other words, when somebody adds a line in a file in a tiny
corner of libfoo to fix a typo in the comment and makes a
commit, that should not have to necessarily mean the version
number of the project as a whole needs to be bumped up. It is
my understanding that people who house collection of related
projects in subversion gets this wrong, because subversion makes
it too eacy to propagate the revision number increment up to the
root level when you update something in a subtree. It may be a
cheap operation from the storage point of view (incrementing the
revision number stored in a few tree nodes near the root), but
it does not change the fact that it changes the revision of the
whole project and affects other parts of projects that is not
affected by the particular change at all (it is not Subversion's
fault, but more of a fault of people who put everything in one
repository). You could manage the versions that way, but you do
not have to. And if it gets in the way of things you would want
to do, maybe you shouldn't.
I think what truly huge but highly modular projects need is a
good support to lay-out check-outs from multiple subprojects,
each of which is managed in its own repository but has loose
(looser than the level of individual commits) version
dependency. That would need to solve three issues: (1) the
right versions from many repositories need to be checked out in
correct locations for a build, (2) after building and testing to
make sure they work together as a whole, these specific versions
from the subcomponent repositories need to be tagged to mark a
release, and (3) maybe a single large tarball that contains all
subprojects' checkout can be made easily.
So the issue may not be partial repository support, but support
for managing multiple projects.
^ permalink raw reply
* [PATCH 1/3] gitweb: Add git_project_index for generating index.aux
From: Jakub Narebski @ 2006-09-15 2:56 UTC (permalink / raw)
To: git
In-Reply-To: <200609150453.42231.jnareb@gmail.com>
Add git_project_index, which generates index.aux file that can be used
as a source of projects list, instead of generating projects list from
a directory. Using file as a source of projects list allows for some
projects to be not present in gitweb main (project_list) page, and/or
correct project owner info. And is probably faster.
Additionally it can be used to get the list of all available repositories
for scripts (in easily parseable form).
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/gitweb.perl | 25 +++++++++++++++++++++++++
1 files changed, 25 insertions(+), 0 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index b4a890b..7dbcb88 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -296,6 +296,7 @@ my %actions = (
# those below don't need $project
"opml" => \&git_opml,
"project_list" => \&git_project_list,
+ "project_index" => \&git_project_index,
);
if (defined $project) {
@@ -2210,6 +2211,30 @@ sub git_project_list {
git_footer_html();
}
+sub git_project_index {
+ my @projects = git_get_projects_list();
+
+ print $cgi->header(
+ -type => 'text/plain',
+ -charset => 'utf-8',
+ -content_disposition => qq(inline; filename="index.aux"));
+
+ foreach my $pr (@projects) {
+ if (!exists $pr->{'owner'}) {
+ $pr->{'owner'} = get_file_owner("$projectroot/$project");
+ }
+
+ my ($path, $owner) = ($pr->{'path'}, $pr->{'owner'});
+ # quote as in CGI::Util::encode, but keep the slash, and use '+' for ' '
+ $path =~ s/([^a-zA-Z0-9_.\-\/ ])/sprintf("%%%02X", ord($1))/eg;
+ $owner =~ s/([^a-zA-Z0-9_.\-\/ ])/sprintf("%%%02X", ord($1))/eg;
+ $path =~ s/ /\+/g;
+ $owner =~ s/ /\+/g;
+
+ print "$path $owner\n";
+ }
+}
+
sub git_summary {
my $descr = git_get_project_description($project) || "none";
my $head = git_get_head_hash($project);
--
1.4.2
^ permalink raw reply related
* [PATCH 0/3] Add git_project_index, improve href()
From: Jakub Narebski @ 2006-09-15 2:53 UTC (permalink / raw)
To: git
This series of patches introduces "project_index" view, which is text/plain
equivalent of projects list view, again. This time it is accompanied by
the patch which adds links to this view.
"project_index" view has at least two uses: first, to automatically generate
gitweb project index file (which we then edit, e.g. removing some projects
from being visible from gitweb and correcting ownership information).
Second, to get list of projects for scripts, in very easily parseable form.
Even if patches 1 and 3 got discarded, patch 2 is I think worth applying.
It finishes consolidation of URL generation.
Shortlog:
[PATCH 1/3] gitweb: Add git_project_index for generating index.aux
[PATCH 2/3] gitweb: Allow for href() to be used for projectless links
[PATCH 3/3] gitweb: Add link to "project_index" view to "project_list" page
Diffstat:
gitweb/gitweb.perl | 51 ++++++++++++++++++++++++++++++++++++++++++++-------
1 files changed, 44 insertions(+), 7 deletions(-)
P.S. Should shortlog be before, or after diffstat (if it matters)?
--
Jakub Narebski
ShadeHawk on #git
Poland
^ permalink raw reply
* [PATCH 3/3] gitweb: Add link to "project_index" view to "project_list" page
From: Jakub Narebski @ 2006-09-15 2:59 UTC (permalink / raw)
To: git
In-Reply-To: <200609150453.42231.jnareb@gmail.com>
Add link to "project_index" view as [TXT] beside link to "opml" view,
(which is marked by [OPML]) to "project_list" page.
While at it add alternate links for "opml" and "project_list" to HTML
header for "project_list" view.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
In the future we might want to change appereance of [TXT] link
to "project_index" view (e.g. green background instead of orange,
smaller width).
gitweb/gitweb.perl | 11 ++++++++++-
1 files changed, 10 insertions(+), 1 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index e900713..1f26365 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1255,6 +1255,13 @@ EOF
printf('<link rel="alternate" title="%s log" '.
'href="%s" type="application/rss+xml"/>'."\n",
esc_param($project), href(action=>"rss"));
+ } else {
+ printf('<link rel="alternate" title="%s" '.
+ 'href="%s" type="text/plain; charset=utf-8"/>'."\n",
+ esc_param($project), href(project=>undef, action=>"project_index"));
+ printf('<link rel="alternate" title="%s" '.
+ 'href="%s" type="text/x-opml"/>'."\n",
+ esc_param($project), href(project=>undef, action=>"opml"));
}
if (defined $favicon) {
print qq(<link rel="shortcut icon" href="$favicon" type="image/png"/>\n);
@@ -1309,7 +1316,9 @@ sub git_footer_html {
-class => "rss_logo"}, "RSS") . "\n";
} else {
print $cgi->a({-href => href(project=>undef, action=>"opml"),
- -class => "rss_logo"}, "OPML") . "\n";
+ -class => "rss_logo"}, "OPML") . " ";
+ print $cgi->a({-href => href(project=>undef, action=>"project_index"),
+ -class => "rss_logo"}, "TXT") . "\n";
}
print "</div>\n" .
"</body>\n" .
--
1.4.2
^ permalink raw reply related
* [PATCH 2/3] gitweb: Allow for href() to be used for projectless links
From: Jakub Narebski @ 2006-09-15 2:57 UTC (permalink / raw)
To: git
In-Reply-To: <200609150453.42231.jnareb@gmail.com>
Change adding project to params if $params{"project"} is false
to adding project to params if it not exist. It allows for href()
to be used for projectless links by using "project=>undef" as
argument, while still adding project to params by default
in the most common case.
Convert remaining links (except $home_link and anchor links)
to use href(); this required adding 'order => "o"' to @mapping
in href(). This finishes consolidation of URL generation.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/gitweb.perl | 17 ++++++++++-------
1 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7dbcb88..e900713 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -326,11 +326,12 @@ sub href(%) {
hash_base => "hb",
hash_parent_base => "hpb",
page => "pg",
+ order => "o",
searchtext => "s",
);
my %mapping = @mapping;
- $params{"project"} ||= $project;
+ $params{'project'} = $project unless exists $params{'project'};
my @result = ();
for (my $i = 0; $i < @mapping; $i += 2) {
@@ -1304,9 +1305,11 @@ sub git_footer_html {
if (defined $descr) {
print "<div class=\"page_footer_text\">" . esc_html($descr) . "</div>\n";
}
- print $cgi->a({-href => href(action=>"rss"), -class => "rss_logo"}, "RSS") . "\n";
+ print $cgi->a({-href => href(action=>"rss"),
+ -class => "rss_logo"}, "RSS") . "\n";
} else {
- print $cgi->a({-href => href(action=>"opml"), -class => "rss_logo"}, "OPML") . "\n";
+ print $cgi->a({-href => href(project=>undef, action=>"opml"),
+ -class => "rss_logo"}, "OPML") . "\n";
}
print "</div>\n" .
"</body>\n" .
@@ -2153,7 +2156,7 @@ sub git_project_list {
print "<th>Project</th>\n";
} else {
print "<th>" .
- $cgi->a({-href => "$my_uri?" . esc_param("o=project"),
+ $cgi->a({-href => href(project=>undef, order=>'project'),
-class => "header"}, "Project") .
"</th>\n";
}
@@ -2162,7 +2165,7 @@ sub git_project_list {
print "<th>Description</th>\n";
} else {
print "<th>" .
- $cgi->a({-href => "$my_uri?" . esc_param("o=descr"),
+ $cgi->a({-href => href(project=>undef, order=>'descr'),
-class => "header"}, "Description") .
"</th>\n";
}
@@ -2171,7 +2174,7 @@ sub git_project_list {
print "<th>Owner</th>\n";
} else {
print "<th>" .
- $cgi->a({-href => "$my_uri?" . esc_param("o=owner"),
+ $cgi->a({-href => href(project=>undef, order=>'owner'),
-class => "header"}, "Owner") .
"</th>\n";
}
@@ -2180,7 +2183,7 @@ sub git_project_list {
print "<th>Last Change</th>\n";
} else {
print "<th>" .
- $cgi->a({-href => "$my_uri?" . esc_param("o=age"),
+ $cgi->a({-href => href(project=>undef, order=>'age'),
-class => "header"}, "Last Change") .
"</th>\n";
}
--
1.4.2
^ permalink raw reply related
* stgit: cannot push a patch - Python trace dump
From: Auke Kok @ 2006-09-15 5:07 UTC (permalink / raw)
To: git
Hi all,
I'm preparing patches for upstream and am haunted by an apparent patch breaking
stg. The problem appears to be one or more of my patches breaking a push:
$ stg push
Traceback (most recent call last):
File "/usr/bin/stg", line 43, in ?
main()
File "/usr/lib/python2.4/site-packages/stgit/main.py", line 255, in main
command.func(parser, options, args)
File "/usr/lib/python2.4/site-packages/stgit/commands/push.py", line 101, in
func
push_patches(patches, options.merged)
File "/usr/lib/python2.4/site-packages/stgit/commands/common.py", line 160,
in push_patches
forwarded = crt_series.forward_patches(patches)
File "/usr/lib/python2.4/site-packages/stgit/stack.py", line 789, in
forward_patches
committer_email = committer_email)
File "/usr/lib/python2.4/site-packages/stgit/git.py", line 439, in commit
if message[-1:] != '\n':
TypeError: unsubscriptable object
I've been abusing stg by leaving the commit messages empty so I assume that
that's the cause here, or related to the problem.
This happens with today's stg as well as stg-0.10
After scraping all my patches out manually and adding meaningfull (non-empty)
commit messages I seem to be able to push and pop them all again.
Cheers,
Auke
^ permalink raw reply
* Re: [PATCH (take 2)] gitweb: Do not parse refs by hand, use git-peek-remote instead
From: Junio C Hamano @ 2006-09-15 6:15 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200609150343.28334.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> This is in response to Linus work on packed refs. Additionally it
> makes gitweb work with symrefs, too.
>
> Do not parse refs by hand, using File::Find and reading individual
> heads to get hash of reference, but use git-peek-remote output
> instead.
Looks nicer. Will apply.
Now, once we start doing this, it may make sense to rethink how
this function and git_get_references functions are used. I
think
git grep -n -e '^sub ' \
-e git_get_references \
-e git_get_refs_list gitweb/gitweb.perl
would be instructive how wasteful the current code is.
get_refs_list is called _TWICE_ in git_summary and worse yet
very late in the function, after calling git_get_references that
could already have done what the function does (by the way,
git_get_references already knows how to use peek-remote output
but for some reason it uses ls-remote -- I think you can safely
rewrite it to use peek-remote). So you end up doing peek-remote
three times to draw the summary page.
git_get_references are called from almost everywhere that shows
the list of commits, which is understandable because we would
want to see those markers in the list.
I very much suspect that you can use git_get_refs_list to return
a hash and a sorted list at the same time from the same input
and make git_summary to do with just a single call to it, and
get rid of git_get_references with a little bit of restructuring
of the caller.
Hmm?
^ permalink raw reply
* Re: [PATCH 2/3] gitweb: Allow for href() to be used for projectless links
From: Junio C Hamano @ 2006-09-15 6:17 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200609150457.16924.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> Change adding project to params if $params{"project"} is false
> to adding project to params if it not exist. It allows for href()
> to be used for projectless links by using "project=>undef" as
> argument, while still adding project to params by default
> in the most common case.
This did not parse very well, at least for me.
> @@ -1304,9 +1305,11 @@ sub git_footer_html {
> if (defined $descr) {
> print "<div class=\"page_footer_text\">" . esc_html($descr) . "</div>\n";
> }
> - print $cgi->a({-href => href(action=>"rss"), -class => "rss_logo"}, "RSS") . "\n";
> + print $cgi->a({-href => href(action=>"rss"),
> + -class => "rss_logo"}, "RSS") . "\n";
> } else {
> - print $cgi->a({-href => href(action=>"opml"), -class => "rss_logo"}, "OPML") . "\n";
> + print $cgi->a({-href => href(project=>undef, action=>"opml"),
> + -class => "rss_logo"}, "OPML") . "\n";
> }
> print "</div>\n" .
> "</body>\n" .
Argh. While I very much welcome folding of the long lines, I'd
rather see a separate patch to clean up other loooooooong lines
the current gitweb code has.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox