* Tracking empty directories
From: Jonathan Nieder @ 2011-02-01 17:28 UTC (permalink / raw)
To: Jakub Narebski
Cc: Dmitry S. Kravtsov, git, Nguyen Thai Ngoc Duy, Shawn Pearce
In-Reply-To: <201102011451.17456.jnareb@gmail.com>
Jakub Narebski wrote:
> Also one needs to remember that this would require adding extension
> to git index, because currently it tracks only files, and not
> directories. Explicitly tracking directories in the index could be
> useful for other purposes...
>
> The major difficulty of this is IMHO not the UI, but tracking all those
> tricky corner cases (like directory/file conflict, etc.).
I have ideas about how to resolve those tricky corner cases, but not
about what the UI should look like. How does one go about adding a
directory? Does it ever get implicitly removed?
Would this actually require an index extension, strictly speaking?
Certainly one ought to register an extension name or bump the version
number to avoid confusing gits that don't know about the feature.
But after that, couldn't we (e.g.) allow the directory name (ending
with '/') as index entry?
A related question is backward compatibility (both for alternative git
implementations and for scripts that did not know that "git ls-files"
might mention an empty directory) which somehow seems less
daunting. ;-)
Jonathan
^ permalink raw reply
* Re: Features from GitSurvey 2010
From: Nguyen Thai Ngoc Duy @ 2011-02-01 17:11 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Jakub Narebski, Jonathan Nieder, Dmitry S. Kravtsov, git
In-Reply-To: <AANLkTinPAL2rEUMe-tRGFxSQ0-gfAJvSO7WW+f+2Fd2u@mail.gmail.com>
On Tue, Feb 1, 2011 at 11:27 PM, Shawn Pearce <spearce@spearce.org> wrote:
> On Tue, Feb 1, 2011 at 05:51, Jakub Narebski <jnareb@gmail.com> wrote:
>>
>>> > resumable clone/fetch (and other remote operations)
>>>
>>> Jakub Narebski seems to be interested in this and Nicolas Pitre has
>>> given some good advice about it. You can get something usable today
>>> by putting up a git bundle for download over HTTP or rsync, so it is
>>> possible that this just involves some UI (porcelain) and documentation
>>> work to become standard practice.
>>
>> I wouldn't say that: it is Nicolas Pitre (IIRC) who was doing the work;
>> I was only interested party posting comments, but no code.
>>
>> Again, this feature is not very easy to implement, and would require
>> knowledge of git internals including "smart" git transport ("Pro Git"
>> book can help there).
>
> I think Nico and I have mostly solved this with the pack caching idea.
> If we cache the pack file, we can resume anywhere in about 97% of the
> transfer. The first 3% cannot be resumed easily, its back to the old
> "git cannot be resumed" issue. Fixing that last 3% is incredibly
I thought the cached pack contained anything and for initial clone, we
simply send the pack. What is this 3%? Commit list? Initial commit?
> difficult... but resuming within the remaining 97% is a pretty simple
> extension of the protocol. The hard part is the client side
> infrastructure to remember where we left off and restart.
Narrow/Subtree clone is still just an idea, but can pack cache support
be made to resumable initial narrow clone too?
--
Duy
^ permalink raw reply
* Re: Features from GitSurvey 2010
From: Nguyen Thai Ngoc Duy @ 2011-02-01 17:05 UTC (permalink / raw)
To: Shawn Pearce
Cc: Jakub Narebski, Jonathan Nieder, Dmitry S. Kravtsov, git,
Junio C Hamano
In-Reply-To: <AANLkTinPAL2rEUMe-tRGFxSQ0-gfAJvSO7WW+f+2Fd2u@mail.gmail.com>
On Tue, Feb 1, 2011 at 11:27 PM, Shawn Pearce <spearce@spearce.org> wrote:
>>> > subtree clone
>>>
>>> Nguyễn Thái Ngọc Duy and Elijah Newren have done some design and
>>> prototyping work.
>>
>> Git mailing list archives should contain proof of concept / RFC patches
>> for this feature. Quite interesting.
>
> I think Junio has already started thinking about this one.
I need to get nd/pathspec right and implement negative pathspecs
before returning to this feature. But there are still interesting
issues:
- narrow by directories or pathspecs (or unpack_trees() by
directories or pathspecs)
- widen a clone (negative pathspecs should help calculating necessary objects)
- should commit objects that does not update narrow area be fetched
(I recall it consumes a considerable amount. Security is also an
issue)
- push support for shallow clone (Elijah approach does not base on
shallow clone, so it's non-issue)
--
Duy
^ permalink raw reply
* [PATCH (version B) 1/2] gitweb: Prepare for splitting gitweb
From: Jakub Narebski @ 2011-02-01 16:50 UTC (permalink / raw)
To: git
Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
Jakub Narebski
In-Reply-To: <1296579016-13356-1-git-send-email-jnareb@gmail.com>
Prepare gitweb for being split into modules that would be installed
in gitweblibdir, by default alongside gitweb in 'lib/' subdirectory,
by adding
use lib $ENV{'GITWEBLIBDIR'} || "++GITWEBLIBDIR++";
to gitweb.perl (to main gitweb script). The first part allow to run
gitweb scripts using source version of gitweb (after small change to
t/gitweb-lib.sh). The second part is set to appropriate value during
build (generating gitweb.cgi). This allows to select where to install
gitweb modules via 'gitweblibdir' build time configuration variable,
e.g.
$ make gitwebdir=/var/www/cgi-bin gitweblibdir=/usr/lib/perl5
install-gitweb
This preparatory work allows to add new module to gitweb by simply
adding
GITWEB_MODULES += <module>
to gitweb/Makefile (assuming that the module is in 'gitweb/lib/'
directory).
While at it pass GITWEBLIBDIR in addition to GITWEB_TEST_INSTALLED to
allow testing installed version of gitweb and installed version of
modules (for future tests which would check individual (sub)modules).
At Pavan Kumar Sankara suggestion gitweb/Makefile uses
install [OPTION]... SOURCE... DIRECTORY
format (2nd format) with single SOURCE rather than
install [OPTION]... SOURCE DEST
format (1st format) because of security reasons (race conditions).
Modern GNU install has `-T' / `--no-target-directory' option, but we
cannot rely that the $(INSTALL) we are using supports this option.
The install-modules target in gitweb/Makefile uses shell 'for' loop,
instead of make's $(foreach) function, to avoid possible problem with
generating a command line that exceeded the maximum argument list
length.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/Makefile | 19 +++++++++++++++++--
gitweb/gitweb.perl | 4 ++++
t/gitweb-lib.sh | 2 ++
3 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/gitweb/Makefile b/gitweb/Makefile
index 0a6ac00..abe9db8 100644
--- a/gitweb/Makefile
+++ b/gitweb/Makefile
@@ -13,6 +13,7 @@ all::
prefix ?= $(HOME)
bindir ?= $(prefix)/bin
gitwebdir ?= /var/www/cgi-bin
+gitweblibdir ?= $(gitwebdir)/lib
RM ?= rm -f
INSTALL ?= install
@@ -57,6 +58,7 @@ PERL_PATH ?= /usr/bin/perl
bindir_SQ = $(subst ','\'',$(bindir))#'
gitwebdir_SQ = $(subst ','\'',$(gitwebdir))#'
gitwebstaticdir_SQ = $(subst ','\'',$(gitwebdir)/static)#'
+gitweblibdir_SQ = $(subst ','\'',$(gitweblibdir))#'
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))#'
PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))#'
DESTDIR_SQ = $(subst ','\'',$(DESTDIR))#'
@@ -115,6 +117,7 @@ GITWEB_FILES += static/git-logo.png static/git-favicon.png
GITWEB_REPLACE = \
-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \
-e 's|++GIT_BINDIR++|$(bindir)|g' \
+ -e 's|++GITWEBLIBDIR++|$(gitweblibdir)|g' \
-e 's|++GITWEB_CONFIG++|$(GITWEB_CONFIG)|g' \
-e 's|++GITWEB_CONFIG_SYSTEM++|$(GITWEB_CONFIG_SYSTEM)|g' \
-e 's|++GITWEB_HOME_LINK_STR++|$(GITWEB_HOME_LINK_STR)|g' \
@@ -153,20 +156,32 @@ test:
test-installed:
GITWEB_TEST_INSTALLED='$(DESTDIR_SQ)$(gitwebdir_SQ)' \
+ GITWEBLIBDIR='$(DESTDIR_SQ)$(gitweblibdir_SQ)' \
$(MAKE) -C ../t gitweb-test
### Installation rules
-install: all
+install: all install-modules
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebdir_SQ)'
$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)'
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
$(INSTALL) -m 644 $(GITWEB_FILES) '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
+install-modules:
+ install_dirs="$(sort $(dir $(GITWEB_MODULES)))" && \
+ for dir in $$install_dirs; do \
+ test -d '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir" || \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir"; \
+ done
+ gitweb_modules="$(GITWEB_MODULES)" && \
+ for mod in $$gitweb_modules; do \
+ $(INSTALL) -m 644 "lib/$$mod" '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$(dirname $$mod)"; \
+ done
+
### Cleaning rules
clean:
$(RM) gitweb.cgi static/gitweb.min.js static/gitweb.min.css GITWEB-BUILD-OPTIONS
-.PHONY: all clean install test test-installed .FORCE-GIT-VERSION-FILE FORCE
+.PHONY: all clean install install-modules test test-installed .FORCE-GIT-VERSION-FILE FORCE
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 1025c2f..bed1eec 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -10,6 +10,10 @@
use 5.008;
use strict;
use warnings;
+
+use lib $ENV{'GITWEBLIBDIR'} || "++GITWEBLIBDIR++";
+
+use File::Spec;
use CGI qw(:standard :escapeHTML -nosticky);
use CGI::Util qw(unescape);
use CGI::Carp qw(fatalsToBrowser set_message);
diff --git a/t/gitweb-lib.sh b/t/gitweb-lib.sh
index b9bb95f..8b98047 100644
--- a/t/gitweb-lib.sh
+++ b/t/gitweb-lib.sh
@@ -48,6 +48,8 @@ EOF
say "# Testing $SCRIPT_NAME"
else # normal case, use source version of gitweb
SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.perl"
+ GITWEBLIBDIR="$GIT_BUILD_DIR/gitweb/lib"
+ export GITWEBLIBDIR
fi
export SCRIPT_NAME
}
--
1.6.5.GIT
^ permalink raw reply related
* [PATCH (proof of concept) 2/2] gitweb: Create Gitweb::Util module
From: Jakub Narebski @ 2011-02-01 16:50 UTC (permalink / raw)
To: git
Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
Pavan Kumar Sunkara, Jakub Narebski
In-Reply-To: <1296579016-13356-1-git-send-email-jnareb@gmail.com>
From: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>
Create a Gitweb::Util module, which is meant to contain internal
utilities used by gitweb. Currently it includes all the
quoting/unquoting and escaping subroutines that are used by the
gitweb.
Update gitweb/Makefile to install Gitweb::Util module alongside gitweb
Signed-off-by: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This patch serves two purposes. First, it serves as test that earlier
"gitweb: Prepare for splitting gitweb" patch acually work correctly.
Second, it might be good starting point to splitting gitweb.
Refactoring well defined parts into separate modules (Perl packages)
could be a better, easier way than trying to come with good separation
(split) into modules upfront. Such leisure approach to splitting
gitweb has more chance to be accepted. Perhaps if such approach were
proposed on GSoC 2010, maybe "gitweb write" project wouldn't fail
midterm evaluations...
This module was taken out of unfinished GSoC 2010 project with
Pavan Kumar Sunkara as a student
git://repo.or.cz/git/gsoc2010-gitweb.git
The module was renamed from Gitweb::Escape to Gitweb::Util. Currently
the contents is the same, but it might change.
Code was updated to more modern codebase; since then esc_path_info and
esc_attr were added to gitweb - both of those are now in Gitweb::Util.
There were also required some changes and conflicts resolved due to
the fact that creating Gitweb::Util (formerly Gitweb::Escape) is no
longer in the middle of larger patch series. In particular lack of
Gitweb::Config means that $fallback_encoding needed to be added to
Gitweb::Util module.
While at it do not export quot_cec and quot_upr helper subroutines by
default, but mark them exportable nevrtheless.
gitweb/Makefile | 3 +
gitweb/gitweb.perl | 140 +-----------------------------------
gitweb/lib/Gitweb/Util.pm | 177 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 183 insertions(+), 137 deletions(-)
create mode 100644 gitweb/lib/Gitweb/Util.pm
diff --git a/gitweb/Makefile b/gitweb/Makefile
index abe9db8..9a4053b 100644
--- a/gitweb/Makefile
+++ b/gitweb/Makefile
@@ -114,6 +114,9 @@ endif
GITWEB_FILES += static/git-logo.png static/git-favicon.png
+# Modules: Gitweb::*
+GITWEB_MODULES += Gitweb/Util.pm
+
GITWEB_REPLACE = \
-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \
-e 's|++GIT_BINDIR++|$(bindir)|g' \
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index ea8ab56..e8d8589 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -23,11 +23,13 @@ use lib "++GITWEBLIBDIR++";
use CGI qw(:standard :escapeHTML -nosticky);
use CGI::Util qw(unescape);
use CGI::Carp qw(fatalsToBrowser set_message);
-use Encode;
use Fcntl ':mode';
use File::Find qw();
use File::Basename qw(basename);
use Time::HiRes qw(gettimeofday tv_interval);
+
+use Gitweb::Util;
+
binmode STDOUT, ':utf8';
our $t0 = [ gettimeofday() ];
@@ -1382,128 +1384,6 @@ sub validate_refname {
return $input;
}
-# decode sequences of octets in utf8 into Perl's internal form,
-# which is utf-8 with utf8 flag set if needed. gitweb writes out
-# in utf-8 thanks to "binmode STDOUT, ':utf8'" at beginning
-sub to_utf8 {
- my $str = shift;
- return undef unless defined $str;
- if (utf8::valid($str)) {
- utf8::decode($str);
- return $str;
- } else {
- return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
- }
-}
-
-# quote unsafe chars, but keep the slash, even when it's not
-# correct, but quoted slashes look too horrible in bookmarks
-sub esc_param {
- my $str = shift;
- return undef unless defined $str;
- $str =~ s/([^A-Za-z0-9\-_.~()\/:@ ]+)/CGI::escape($1)/eg;
- $str =~ s/ /\+/g;
- return $str;
-}
-
-# the quoting rules for path_info fragment are slightly different
-sub esc_path_info {
- my $str = shift;
- return undef unless defined $str;
-
- # path_info doesn't treat '+' as space (specially), but '?' must be escaped
- $str =~ s/([^A-Za-z0-9\-_.~();\/;:@&= +]+)/CGI::escape($1)/eg;
-
- return $str;
-}
-
-# quote unsafe chars in whole URL, so some characters cannot be quoted
-sub esc_url {
- my $str = shift;
- return undef unless defined $str;
- $str =~ s/([^A-Za-z0-9\-_.~();\/;?:@&= ]+)/CGI::escape($1)/eg;
- $str =~ s/ /\+/g;
- return $str;
-}
-
-# quote unsafe characters in HTML attributes
-sub esc_attr {
-
- # for XHTML conformance escaping '"' to '"' is not enough
- return esc_html(@_);
-}
-
-# replace invalid utf8 character with SUBSTITUTION sequence
-sub esc_html {
- my $str = shift;
- my %opts = @_;
-
- return undef unless defined $str;
-
- $str = to_utf8($str);
- $str = $cgi->escapeHTML($str);
- if ($opts{'-nbsp'}) {
- $str =~ s/ / /g;
- }
- $str =~ s|([[:cntrl:]])|(($1 ne "\t") ? quot_cec($1) : $1)|eg;
- return $str;
-}
-
-# quote control characters and escape filename to HTML
-sub esc_path {
- my $str = shift;
- my %opts = @_;
-
- return undef unless defined $str;
-
- $str = to_utf8($str);
- $str = $cgi->escapeHTML($str);
- if ($opts{'-nbsp'}) {
- $str =~ s/ / /g;
- }
- $str =~ s|([[:cntrl:]])|quot_cec($1)|eg;
- return $str;
-}
-
-# Make control characters "printable", using character escape codes (CEC)
-sub quot_cec {
- my $cntrl = shift;
- my %opts = @_;
- my %es = ( # character escape codes, aka escape sequences
- "\t" => '\t', # tab (HT)
- "\n" => '\n', # line feed (LF)
- "\r" => '\r', # carrige return (CR)
- "\f" => '\f', # form feed (FF)
- "\b" => '\b', # backspace (BS)
- "\a" => '\a', # alarm (bell) (BEL)
- "\e" => '\e', # escape (ESC)
- "\013" => '\v', # vertical tab (VT)
- "\000" => '\0', # nul character (NUL)
- );
- my $chr = ( (exists $es{$cntrl})
- ? $es{$cntrl}
- : sprintf('\%2x', ord($cntrl)) );
- if ($opts{-nohtml}) {
- return $chr;
- } else {
- return "<span class=\"cntrl\">$chr</span>";
- }
-}
-
-# Alternatively use unicode control pictures codepoints,
-# Unicode "printable representation" (PR)
-sub quot_upr {
- my $cntrl = shift;
- my %opts = @_;
-
- my $chr = sprintf('&#%04d;', 0x2400+ord($cntrl));
- if ($opts{-nohtml}) {
- return $chr;
- } else {
- return "<span class=\"cntrl\">$chr</span>";
- }
-}
-
# git may return quoted and escaped filenames
sub unquote {
my $str = shift;
@@ -1540,20 +1420,6 @@ sub unquote {
return $str;
}
-# escape tabs (convert tabs to spaces)
-sub untabify {
- my $line = shift;
-
- while ((my $pos = index($line, "\t")) != -1) {
- if (my $count = (8 - ($pos % 8))) {
- my $spaces = ' ' x $count;
- $line =~ s/\t/$spaces/;
- }
- }
-
- return $line;
-}
-
sub project_in_list {
my $project = shift;
my @list = git_get_projects_list();
diff --git a/gitweb/lib/Gitweb/Util.pm b/gitweb/lib/Gitweb/Util.pm
new file mode 100644
index 0000000..a213d3f
--- /dev/null
+++ b/gitweb/lib/Gitweb/Util.pm
@@ -0,0 +1,177 @@
+# Gitweb::Util -- Internal utilities used by gitweb (git web interface)
+#
+# This module is licensed under the GPLv2
+
+package Gitweb::Util;
+
+use strict;
+use warnings;
+use Exporter qw(import);
+
+our @EXPORT = qw(to_utf8
+ esc_param esc_path_info esc_url
+ esc_html esc_path esc_attr
+ untabify
+ $fallback_encoding);
+our @EXPORT_OK = qw(quot_cec quot_upr);
+
+use Encode;
+use CGI;
+
+# ......................................................................
+# Perl encoding (utf-8)
+
+# decode sequences of octets in utf8 into Perl's internal form,
+# which is utf-8 with utf8 flag set if needed. gitweb writes out
+# in utf-8 thanks to "binmode STDOUT, ':utf8'" at beginning of gitweb.perl
+our $fallback_encoding = 'latin1';
+sub to_utf8 {
+ my $str = shift;
+ return undef unless defined $str;
+ if (utf8::valid($str)) {
+ utf8::decode($str);
+ return $str;
+ } else {
+ return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
+ }
+}
+
+# ......................................................................
+# CGI encoding
+
+# quote unsafe chars, but keep the slash, even when it's not
+# correct, but quoted slashes look too horrible in bookmarks
+sub esc_param {
+ my $str = shift;
+ return undef unless defined $str;
+
+ $str =~ s/([^A-Za-z0-9\-_.~()\/:@ ]+)/CGI::escape($1)/eg;
+ $str =~ s/ /\+/g;
+
+ return $str;
+}
+
+# the quoting rules for path_info fragment are slightly different
+sub esc_path_info {
+ my $str = shift;
+ return undef unless defined $str;
+
+ # path_info doesn't treat '+' as space (specially), but '?' must be escaped
+ $str =~ s/([^A-Za-z0-9\-_.~();\/;:@&= +]+)/CGI::escape($1)/eg;
+
+ return $str;
+}
+
+# quote unsafe chars in whole URL, so some characters cannot be quoted
+sub esc_url {
+ my $str = shift;
+ return undef unless defined $str;
+
+ $str =~ s/([^A-Za-z0-9\-_.~();\/;?:@&= ]+)/CGI::escape($1)/eg;
+ $str =~ s/ /\+/g;
+
+ return $str;
+}
+
+# ......................................................................
+# (X)HTML escaping
+
+# replace invalid utf8 character with SUBSTITUTION sequence
+sub esc_html {
+ my $str = shift;
+ my %opts = @_;
+
+ return undef unless defined $str;
+
+ $str = to_utf8($str);
+ $str = CGI::escapeHTML($str);
+ if ($opts{'-nbsp'}) {
+ $str =~ s/ / /g;
+ }
+ $str =~ s|([[:cntrl:]])|(($1 ne "\t") ? quot_cec($1) : $1)|eg;
+ return $str;
+}
+
+# quote unsafe characters in HTML attributes
+sub esc_attr {
+
+ # for XHTML conformance escaping '"' to '"' is not enough
+ return esc_html(@_);
+}
+
+# quote control characters and escape filename to HTML
+sub esc_path {
+ my $str = shift;
+ my %opts = @_;
+
+ return undef unless defined $str;
+
+ $str = to_utf8($str);
+ $str = CGI::escapeHTML($str);
+ if ($opts{'-nbsp'}) {
+ $str =~ s/ / /g;
+ }
+ $str =~ s|([[:cntrl:]])|quot_cec($1)|eg;
+ return $str;
+}
+
+# ......................................................................
+# Other
+
+# escape tabs (convert tabs to spaces)
+sub untabify {
+ my $line = shift;
+
+ while ((my $pos = index($line, "\t")) != -1) {
+ if (my $count = (8 - ($pos % 8))) {
+ my $spaces = ' ' x $count;
+ $line =~ s/\t/$spaces/;
+ }
+ }
+
+ return $line;
+}
+
+# ----------------------------------------------------------------------
+# Showing "unprintable" characters (utility functions)
+
+# Make control characters "printable", using character escape codes (CEC)
+sub quot_cec {
+ my $cntrl = shift;
+ my %opts = @_;
+ my %es = ( # character escape codes, aka escape sequences
+ "\t" => '\t', # tab (HT)
+ "\n" => '\n', # line feed (LF)
+ "\r" => '\r', # carrige return (CR)
+ "\f" => '\f', # form feed (FF)
+ "\b" => '\b', # backspace (BS)
+ "\a" => '\a', # alarm (bell) (BEL)
+ "\e" => '\e', # escape (ESC)
+ "\013" => '\v', # vertical tab (VT)
+ "\000" => '\0', # nul character (NUL)
+ );
+ my $chr = ( (exists $es{$cntrl})
+ ? $es{$cntrl}
+ : sprintf('\%2x', ord($cntrl)) );
+ if ($opts{-nohtml}) {
+ return $chr;
+ } else {
+ return "<span class=\"cntrl\">$chr</span>";
+ }
+}
+
+# Alternatively use unicode control pictures codepoints,
+# Unicode "printable representation" (PR)
+sub quot_upr {
+ my $cntrl = shift;
+ my %opts = @_;
+
+ my $chr = sprintf('&#%04d;', 0x2400+ord($cntrl));
+ if ($opts{-nohtml}) {
+ return $chr;
+ } else {
+ return "<span class=\"cntrl\">$chr</span>";
+ }
+}
+
+1;
--
1.7.3
^ permalink raw reply related
* [PATCH (version C) 1/2] gitweb: Prepare for splitting gitweb
From: Jakub Narebski @ 2011-02-01 16:50 UTC (permalink / raw)
To: git
Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
Jakub Narebski
In-Reply-To: <1296579016-13356-1-git-send-email-jnareb@gmail.com>
Prepare gitweb for being split into modules that would be installed
in gitweblibdir, by default alongside gitweb in 'lib/' subdirectory.
Gitweb would search first in 'lib/' subdirectory from where it is
installed, via
use lib __DIR__.'/lib';
(This allow for tests to work with source version of gitweb without
changes.) Then it searches in $(gitweblibdir) directory (set during
build time), by default "$(gitwebdir)/lib", via
use lib "++GITWEBLIBDIR++";
"++GITWEBLIBDIR++" is set to appropriate value during build time
(generating gitweb.cgi). This allows to select where to install
gitweb modules via 'gitweblibdir' build time configuration variable,
for example
$ make gitwebdir=/var/www/cgi-bin gitweblibdir=/usr/lib/perl5
install-gitweb
This preparatory work allows to add new module to gitweb by simply
adding
GITWEB_MODULES += <module>
to gitweb/Makefile (assuming that the module is in 'gitweb/lib/'
directory).
While at it pass GITWEBLIBDIR in addition to GITWEB_TEST_INSTALLED to
allow testing installed version of gitweb and installed version of
modules (for future tests which would check individual (sub)modules).
Using __DIR__ from Dir::Self module (not in core, that's why currently
gitweb includes excerpt of code from Dir::Self defining __DIR__) was
chosen over using FindBin-based solution (in core since perl 5.00307,
while gitweb itself requires at least perl 5.8.0) because FindBin uses
BEGIN block, which is a problem under mod_perl and other persistent
Perl environments (thought there are workarounds).
At Pavan Kumar Sankara suggestion gitweb/Makefile uses
install [OPTION]... SOURCE... DIRECTORY
format (2nd format) with single SOURCE rather than
install [OPTION]... SOURCE DEST
format (1st format) because of security reasons (race conditions).
Modern GNU install has `-T' / `--no-target-directory' option, but we
cannot rely that the $(INSTALL) we are using supports this option.
The install-modules target in gitweb/Makefile uses shell 'for' loop,
instead of make's $(foreach) function, to avoid possible problem with
generating a command line that exceeded the maximum argument list
length.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/Makefile | 19 +++++++++++++++++--
gitweb/gitweb.perl | 10 ++++++++++
2 files changed, 27 insertions(+), 2 deletions(-)
diff --git a/gitweb/Makefile b/gitweb/Makefile
index 0a6ac00..abe9db8 100644
--- a/gitweb/Makefile
+++ b/gitweb/Makefile
@@ -13,6 +13,7 @@ all::
prefix ?= $(HOME)
bindir ?= $(prefix)/bin
gitwebdir ?= /var/www/cgi-bin
+gitweblibdir ?= $(gitwebdir)/lib
RM ?= rm -f
INSTALL ?= install
@@ -57,6 +58,7 @@ PERL_PATH ?= /usr/bin/perl
bindir_SQ = $(subst ','\'',$(bindir))#'
gitwebdir_SQ = $(subst ','\'',$(gitwebdir))#'
gitwebstaticdir_SQ = $(subst ','\'',$(gitwebdir)/static)#'
+gitweblibdir_SQ = $(subst ','\'',$(gitweblibdir))#'
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))#'
PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))#'
DESTDIR_SQ = $(subst ','\'',$(DESTDIR))#'
@@ -115,6 +117,7 @@ GITWEB_FILES += static/git-logo.png static/git-favicon.png
GITWEB_REPLACE = \
-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \
-e 's|++GIT_BINDIR++|$(bindir)|g' \
+ -e 's|++GITWEBLIBDIR++|$(gitweblibdir)|g' \
-e 's|++GITWEB_CONFIG++|$(GITWEB_CONFIG)|g' \
-e 's|++GITWEB_CONFIG_SYSTEM++|$(GITWEB_CONFIG_SYSTEM)|g' \
-e 's|++GITWEB_HOME_LINK_STR++|$(GITWEB_HOME_LINK_STR)|g' \
@@ -153,20 +156,32 @@ test:
test-installed:
GITWEB_TEST_INSTALLED='$(DESTDIR_SQ)$(gitwebdir_SQ)' \
+ GITWEBLIBDIR='$(DESTDIR_SQ)$(gitweblibdir_SQ)' \
$(MAKE) -C ../t gitweb-test
### Installation rules
-install: all
+install: all install-modules
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebdir_SQ)'
$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)'
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
$(INSTALL) -m 644 $(GITWEB_FILES) '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
+install-modules:
+ install_dirs="$(sort $(dir $(GITWEB_MODULES)))" && \
+ for dir in $$install_dirs; do \
+ test -d '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir" || \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir"; \
+ done
+ gitweb_modules="$(GITWEB_MODULES)" && \
+ for mod in $$gitweb_modules; do \
+ $(INSTALL) -m 644 "lib/$$mod" '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$(dirname $$mod)"; \
+ done
+
### Cleaning rules
clean:
$(RM) gitweb.cgi static/gitweb.min.js static/gitweb.min.css GITWEB-BUILD-OPTIONS
-.PHONY: all clean install test test-installed .FORCE-GIT-VERSION-FILE FORCE
+.PHONY: all clean install install-modules test test-installed .FORCE-GIT-VERSION-FILE FORCE
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 1025c2f..ea8ab56 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -10,6 +10,16 @@
use 5.008;
use strict;
use warnings;
+
+use File::Spec;
+
+# __DIR__ is excerpt from Dir::Self
+sub __DIR__ () {
+ File::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);
+}
+use lib __DIR__ . '/lib';
+use lib "++GITWEBLIBDIR++";
+
use CGI qw(:standard :escapeHTML -nosticky);
use CGI::Util qw(unescape);
use CGI::Carp qw(fatalsToBrowser set_message);
--
1.7.3
^ permalink raw reply related
* [PATCH (version A) 1/2] gitweb: Prepare for splitting gitweb
From: Jakub Narebski @ 2011-02-01 16:50 UTC (permalink / raw)
To: git
Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
Jakub Narebski
In-Reply-To: <1296579016-13356-1-git-send-email-jnareb@gmail.com>
Prepare gitweb for having been split into modules that are to be
installed alongside gitweb in 'lib/' subdirectory, by adding
use lib __DIR__.'/lib';
to gitweb.perl (to main gitweb script), and preparing for putting
modules (relative path) in $(GITWEB_MODULES) in gitweb/Makefile.
This preparatory work allows to add new module to gitweb by simply
adding
GITWEB_MODULES += <module>
to gitweb/Makefile (assuming that the module is in 'gitweb/lib/'
directory).
While at it pass GITWEBLIBDIR in addition to GITWEB_TEST_INSTALLED to
allow testing installed version of gitweb and installed version of
modules (for future tests which would check individual (sub)modules).
Using __DIR__ from Dir::Self module (not in core, that's why currently
gitweb includes excerpt of code from Dir::Self defining __DIR__) was
chosen over using FindBin-based solution (in core since perl 5.00307,
while gitweb itself requires at least perl 5.8.0) because FindBin uses
BEGIN block, which is a problem under mod_perl and other persistent
Perl environments (thought there are workarounds).
At Pavan Kumar Sankara suggestion gitweb/Makefile uses
install [OPTION]... SOURCE... DIRECTORY
format (2nd format) with single SOURCE rather than
install [OPTION]... SOURCE DEST
format (1st format) because of security reasons (race conditions).
Modern GNU install has `-T' / `--no-target-directory' option, but we
cannot rely that the $(INSTALL) we are using supports this option.
The install-modules target in gitweb/Makefile uses shell 'for' loop,
instead of make's $(foreach) function, to avoid possible problem with
generating a command line that exceeded the maximum argument list
length.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/Makefile | 17 +++++++++++++++--
gitweb/gitweb.perl | 8 ++++++++
2 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/gitweb/Makefile b/gitweb/Makefile
index 0a6ac00..e6029e1 100644
--- a/gitweb/Makefile
+++ b/gitweb/Makefile
@@ -57,6 +57,7 @@ PERL_PATH ?= /usr/bin/perl
bindir_SQ = $(subst ','\'',$(bindir))#'
gitwebdir_SQ = $(subst ','\'',$(gitwebdir))#'
gitwebstaticdir_SQ = $(subst ','\'',$(gitwebdir)/static)#'
+gitweblibdir_SQ = $(subst ','\'',$(gitwebdir)/lib)#'
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))#'
PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))#'
DESTDIR_SQ = $(subst ','\'',$(DESTDIR))#'
@@ -153,20 +154,32 @@ test:
test-installed:
GITWEB_TEST_INSTALLED='$(DESTDIR_SQ)$(gitwebdir_SQ)' \
+ GITWEBLIBDIR='$(DESTDIR_SQ)$(gitweblibdir_SQ)' \
$(MAKE) -C ../t gitweb-test
### Installation rules
-install: all
+install: all install-modules
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebdir_SQ)'
$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)'
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
$(INSTALL) -m 644 $(GITWEB_FILES) '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
+install-modules:
+ install_dirs="$(sort $(dir $(GITWEB_MODULES)))" && \
+ for dir in $$install_dirs; do \
+ test -d '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir" || \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir"; \
+ done
+ gitweb_modules="$(GITWEB_MODULES)" && \
+ for mod in $$gitweb_modules; do \
+ $(INSTALL) -m 644 "lib/$$mod" '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$(dirname $$mod)"; \
+ done
+
### Cleaning rules
clean:
$(RM) gitweb.cgi static/gitweb.min.js static/gitweb.min.css GITWEB-BUILD-OPTIONS
-.PHONY: all clean install test test-installed .FORCE-GIT-VERSION-FILE FORCE
+.PHONY: all clean install install-modules test test-installed .FORCE-GIT-VERSION-FILE FORCE
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 1025c2f..1ccea49 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -10,6 +10,14 @@
use 5.008;
use strict;
use warnings;
+
+use File::Spec;
+# __DIR__ is taken from Dir::Self __DIR__ fragment
+sub __DIR__ () {
+ File::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);
+}
+use lib __DIR__ . '/lib';
+
use CGI qw(:standard :escapeHTML -nosticky);
use CGI::Util qw(unescape);
use CGI::Carp qw(fatalsToBrowser set_message);
--
1.6.5.GIT
^ permalink raw reply related
* [PATCH 0/2] gitweb: Begin splitting gitweb
From: Jakub Narebski @ 2011-02-01 16:50 UTC (permalink / raw)
To: git
Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
Jakub Narebski
Gitweb is currently next to largest file (after gitk) in git sources,
more than 220K with more than 25,000 lines. Therefore adding any
large feature that would require large amount of code added, like
gitweb caching by J.H. and my rewrite of it, or "gitweb admin/write"
[failed] GSoC 2010 project by Pavan Kumar Sunkara require for new code
to be added as a separate module or module. Otherwise gitweb would
fast become unmaintainable.
Not in all cases it would require splitting gitweb upfront. At least
in the case of gitweb caching it doesn't. What must be done however
is preparing the infrastructure for modular gitweb sources; to
properly test such infrastructure we need at least one split gitweb
module.
This series is intended to bring such infrastructure to gitweb, to
prepare way for adding output caching to gitweb. Alternatively it can
be thought as beginning of splitting gitweb into smaller submodules,
for better maintability.
Table of contents:
~~~~~~~~~~~~~~~~~~
* [PATCH (version A) 1/2] gitweb: Prepare for splitting gitweb
sub __DIR__ () {
File::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);
}
use lib __DIR__ . '/lib';
Advantages:
- no changes to t/gitweb-lib.sh, ability to run source version
of gitweb without any changes
Disadvantages:
- supports only modules installed either alongside gitweb, or in one
of PERL5LIB directories; no support for installing modules not
alongside gitweb
- because we cannot rely on FindBin::again being available nor on
having Dir::Self installed, __DIR__ must be defined -- more code.
* [PATCH (version B) 1/2] gitweb: Prepare for splitting gitweb
use lib $ENV{'GITWEBLIBDIR'} || "++GITWEBLIBDIR++";
Advantages:
- supports relocating gitweb modules (to gitweblibdir)
- shortest code of all the cases
Disadvantages:
- required changes to t/gitweb-lib.sh to pick up gitweb modules
by source version of gitweb
* [PATCH (version C) 1/2] gitweb: Prepare for splitting gitweb
sub __DIR__ () {
File::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);
}
use lib __DIR__ . '/lib';
use lib "++GITWEBLIBDIR++";
Advantages:
- can run source version of gitweb (gitweb/gitweb.perl) as a script simply
- supports relocating gitweb modules (to gitweblibdir)
Disadvantages:
- most complicated code of all cases
* [PATCH (proof of concept) 2/2] gitweb: Create Gitweb::Util module
Something to actually test previous patch(es) with... and I guess
good start to splitting gitweb into smaller modules.
All versions pass "make -C gitweb test" and "make -C gitweb test-installed"
after "make -C gitweb install" / "make install-gitweb".
Shortlog:
~~~~~~~~~
Jakub Narebski (1):
gitweb: Prepare for splitting gitweb
Pavan Kumar Sunkara (1):
gitweb: Create Gitweb::Util module
Diffstat (for version C):
~~~~~~~~~~~~~~~~~~~~~~~~~
gitweb/Makefile | 22 +++++-
gitweb/gitweb.perl | 150 ++++-----------------------------------
gitweb/lib/Gitweb/Util.pm | 177 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 210 insertions(+), 139 deletions(-)
create mode 100644 gitweb/lib/Gitweb/Util.pm
--
1.7.3
^ permalink raw reply
* Re: Features from GitSurvey 2010
From: Shawn Pearce @ 2011-02-01 16:27 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Jonathan Nieder, Dmitry S. Kravtsov, git
In-Reply-To: <201102011451.17456.jnareb@gmail.com>
On Tue, Feb 1, 2011 at 05:51, Jakub Narebski <jnareb@gmail.com> wrote:
>
>> > resumable clone/fetch (and other remote operations)
>>
>> Jakub Narebski seems to be interested in this and Nicolas Pitre has
>> given some good advice about it. You can get something usable today
>> by putting up a git bundle for download over HTTP or rsync, so it is
>> possible that this just involves some UI (porcelain) and documentation
>> work to become standard practice.
>
> I wouldn't say that: it is Nicolas Pitre (IIRC) who was doing the work;
> I was only interested party posting comments, but no code.
>
> Again, this feature is not very easy to implement, and would require
> knowledge of git internals including "smart" git transport ("Pro Git"
> book can help there).
I think Nico and I have mostly solved this with the pack caching idea.
If we cache the pack file, we can resume anywhere in about 97% of the
transfer. The first 3% cannot be resumed easily, its back to the old
"git cannot be resumed" issue. Fixing that last 3% is incredibly
difficult... but resuming within the remaining 97% is a pretty simple
extension of the protocol. The hard part is the client side
infrastructure to remember where we left off and restart.
>> > GitTorrent Protocol, or git-mirror
>>
>> Sam Vilain and Jonas Fonseca did some good work on this, but it's
>> stalled.
>
> There was some recent discussion on this on git mailing llist, but
> without any code.
>
> One would need to know similar areas as for "resumable clone" feature.
> Plus some knowledge on P2P transport in GitTorrent case.
I think this is very similar to resumable clone. With the cached
pack, clients could use torrent to find it. But right now Nico and I
are sort of expecting a cached pack to live for about the release
cycle of a project... e.g. only a couple of months. I don't know if
that can be seeded fast enough on P2P networks to make it useful to
torrent the ~97% of the project that is the cached pack during an
initial clone request.
>> > subtree clone
>>
>> Nguyễn Thái Ngọc Duy and Elijah Newren have done some design and
>> prototyping work.
>
> Git mailing list archives should contain proof of concept / RFC patches
> for this feature. Quite interesting.
I think Junio has already started thinking about this one.
--
Shawn.
^ permalink raw reply
* Re: Features from GitSurvey 2010
From: Shawn Pearce @ 2011-02-01 16:33 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy
Cc: Jakub Narebski, Jonathan Nieder, Dmitry S. Kravtsov, git
In-Reply-To: <AANLkTinJVa++tttPDav1g1+w128fWsouM=+gf14eUOOK@mail.gmail.com>
On Tue, Feb 1, 2011 at 07:52, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> On Tue, Feb 1, 2011 at 8:51 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>> On Sun, 30 Jan 2011, Jonathan Nieder wrote:
>>> > support for tracking empty directories
>>>
>>> Tricky to get the UI right. I am interested in and would be glad to
>>> help with this one.
>>
>> Also one needs to remember that this would require adding extension
>> to git index, because currently it tracks only files, and not
>> directories. Explicitly tracking directories in the index could be
>> useful for other purposes...
>>
>> The major difficulty of this is IMHO not the UI, but tracking all those
>> tricky corner cases (like directory/file conflict, etc.).
>
> Sort order in index is quite special/strange and must be handled
> correctly when dirs and files are mixed.
Its not the order in the index that is confusing, its the order in the
tree objects. The index sort order is simple, since every path is a
full path string from the top of the repository... you use strcmp() to
order them into a natural order. This however skews where a
subdirectory should live relative to a sibling file, because the
"subdirectory" sorts as though its name ends with '/'.
> There are already special
> directories in index: the submodules. Current git code treats
> S_ISDIR() and S_ISGITLINK() the same in ce_to_dtype() and some more
> places. You need to decouple it somehow.
More confusingly, the GITLINK type is handled as though its *not* a
directory. Storing an empty directory probably means tracking it like
a real directory, but using the empty tree SHA-1 as its value.
Otherwise we probably have all sorts of stuff broken.
> I tried this (for another purpose) and pulled back. I recall Shawn had
> a tree-based index implementation, don't know if he still has it.
No, we threw out the tree-based index that was used inside of EGit
years ago. It turned out to be a horrible idea because it wasn't
compatible with the C tools, and it didn't have the inode stat cache
to tell us which files were clean or dirty quickly.
> Actually tree-based index with dictionary (something like trees in
> packv4) is a good feature itself. It could shrink index size down a
> lot. index is frequently read/written so small index helps (webkit's
> index is 16M, 4M after gzipped).
I think a lot of the reason the webkit index is 16M, gzip to 4M is
because of the duplicate path prefixes that appear on all files within
the same directory. If the index was still a single file, but was
organized into sections by tree (like the TREE extension within the
index itself) you could avoid having the full path within the index
file and save a lot of space when there are many files within
subdirectories. But this does complicate the C code because you would
need to copy each of those path segments together into a path buffer
in order to access the file in the working tree.
Its probably faster to copy those path segments on read into a big
path buffer, and break them apart on write, than to have a huge index
file. We already reformat the index during reading/writing to expand
some of the fields for in-memory only flags.
--
Shawn.
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Nguyen Thai Ngoc Duy @ 2011-02-01 16:02 UTC (permalink / raw)
To: Thomas Rast; +Cc: Nicolas Pitre, Jeff King, Junio C Hamano, git
In-Reply-To: <201102011342.06910.trast@student.ethz.ch>
On Tue, Feb 1, 2011 at 7:42 PM, Thomas Rast <trast@student.ethz.ch> wrote:
> Nicolas Pitre wrote:
>> What I see in the root of the Git source
>> tree is a huge clutter of source files, binary files, scripts, and
>> subdirectories all mixed together. If you know by hart where things are
>> because you've been hacking on them for the last 5 years then of course
>> you might not see the point. But since I didn't work much on Git
>> lately, things are not as obvious to me as they used to be. Looking
>> back at it now with some distance, this tree looks like a mess and it is
>> really annoying to work with.
>
> But judging by that assessment, shouldn't we strive to make it
> *easier* to find things?
>
> In particular a prospective git hacker would not care whether
> something is a source file or a script (you seem to imply the
> opposite). He would instead expect ...
A hacker is expected to RTFM first, IMO. Put up a document describing
how things are organized in git and we're good. git-grep will take
care from there.
--
Duy
^ permalink raw reply
* Re: [1.8.0] make two-argument fetch update remote branches
From: Nguyen Thai Ngoc Duy @ 2011-02-01 15:58 UTC (permalink / raw)
To: Jay Soffian; +Cc: Thomas Rast, Junio C Hamano, git
In-Reply-To: <AANLkTi=Y9PBs_jXyCiAL9YLA8Y_jzWwqxw63hKm7fVBO@mail.gmail.com>
On Tue, Feb 1, 2011 at 2:04 PM, Jay Soffian <jaysoffian@gmail.com> wrote:
> On Mon, Jan 31, 2011 at 4:44 PM, Thomas Rast <trast@student.ethz.ch> wrote:
>> Add a fetch.updateRemoteNamespace (or so) configuration variable that
>> defaults to false. When enabled, it turns on the auto-updating
>> behaviour.
>
> Would it make sense to group the pre-1.8 compatibility switches
> together in some way, if there will be several of them? Maybe
>
> [compat]
> fetchUpdateRemoteNamespace = false
> ...
It is. I was thinking of it as a group of "short"-lived configs to
help maintain backward compatibility for some time (not for ever)
until users are forced to migrate.
--
Duy
^ permalink raw reply
* Re: Features from GitSurvey 2010
From: Nguyen Thai Ngoc Duy @ 2011-02-01 15:52 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Jonathan Nieder, Dmitry S. Kravtsov, git, Shawn O. Pearce
In-Reply-To: <201102011451.17456.jnareb@gmail.com>
On Tue, Feb 1, 2011 at 8:51 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> On Sun, 30 Jan 2011, Jonathan Nieder wrote:
>> > support for tracking empty directories
>>
>> Tricky to get the UI right. I am interested in and would be glad to
>> help with this one.
>
> Also one needs to remember that this would require adding extension
> to git index, because currently it tracks only files, and not
> directories. Explicitly tracking directories in the index could be
> useful for other purposes...
>
> The major difficulty of this is IMHO not the UI, but tracking all those
> tricky corner cases (like directory/file conflict, etc.).
Sort order in index is quite special/strange and must be handled
correctly when dirs and files are mixed. There are already special
directories in index: the submodules. Current git code treats
S_ISDIR() and S_ISGITLINK() the same in ce_to_dtype() and some more
places. You need to decouple it somehow.
I tried this (for another purpose) and pulled back. I recall Shawn had
a tree-based index implementation, don't know if he still has it.
Could be a good point to start adding dirs to index.
Actually tree-based index with dictionary (something like trees in
packv4) is a good feature itself. It could shrink index size down a
lot. index is frequently read/written so small index helps (webkit's
index is 16M, 4M after gzipped).
--
Duy
^ permalink raw reply
* Re: [1.8.0] Remote tag namespace
From: Nguyen Thai Ngoc Duy @ 2011-02-01 15:35 UTC (permalink / raw)
To: Marc Branchaud; +Cc: Git Mailing List, Nicolas Pitre
In-Reply-To: <4D48219D.8060603@xiplink.com>
On Tue, Feb 1, 2011 at 10:07 PM, Marc Branchaud <marcnarc@xiplink.com> wrote:
>> Config branch.*.globalTags (perhaps takes a pattern?) may be defined
>> to create refs/tags/* in addition to refs/remote-tags/<remote>/* when
>> fetching tags.
>
> I may be getting into the weeds prematurely here, but why put the config item
> under branch.* ? Or did you mean remote.*.globalTags? Personally, I don't
> see a need for this. I'd rather have the rev-parse machinery search in
> remote tag namespaces if it can't find anything local.
Ahh.. yeah it's remote.*.globalTags. I don't know, some people might
find current behavior useful. So instead of dropping it entirely, I'd
limit it to certain remotes.
>
>> Migration plan:
>>
>> refs/remote-tags will be used to store new tags unconditionally, which
>> means there will be duplicates with the already-fetched tags in global
>> namespace. Perhaps we can check if they point to the same sha-1, then
>> choose not to annoy users with ambiguous tag messages?
>
> (Again with the weeds...) I don't think we could do that. I'd want to be
> able to have my own (local) tags that refer to the same commits as one or
> more remote tags, and I'd want to see them all.
For listing tags (I forgot this) I think we just follow how git-branch
does it: show only local tags unless -r (or some other option) is
given. What I meant in the above paragraph is "some-ref" can refer to
refs/tags/some-ref or refs/remotes/foo/tags/some-ref, but I was wrong
on this. The latter can only be referred by foo/some-ref or with
migration support in your proposal.
>
> Better for "git tag" to learn scoping options like "git branch": -a and -r.
> (Hmm, maybe git-tag's current -a could become -A...)
When tags are put in remote namespace (wherever it actually is),
git-tag must learn -r like git-branch. I think option name change for
-a is too late though. When "git-ng" rewrite project comes (that is
after libgit2 replaces git core), we may have everything consistent
again.
PS. I bet git-ng would start after Wine 2.0 is released.
--
Duy
^ permalink raw reply
* Re: [1.8.0] Tag namespaces
From: Nguyen Thai Ngoc Duy @ 2011-02-01 15:21 UTC (permalink / raw)
To: Marc Branchaud; +Cc: git
In-Reply-To: <4D481EA0.9090802@xiplink.com>
On Tue, Feb 1, 2011 at 9:54 PM, Marc Branchaud <marcnarc@xiplink.com> wrote:
> On 11-01-31 10:20 PM, Nguyen Thai Ngoc Duy wrote:
>> On Tue, Feb 1, 2011 at 12:05 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Now the 1.7.4 release is out, I'd like people to help thinking about the
>>> next cycle(s).
>>>
>>> As a discussion-starter, here are my random wishes. Even though this does
>>> not attempt to be exhaustive, keeping the number of goals manageably small
>>> may help us focus.
>>
>> Another random wish, which does not come with a proposal. How about
>> tag namespace (ie. tags from a remote stay in remote namespace)?
>
> I had just started writing up such a proposal yesterday. What I have so far
> is pretty preliminary:
Thanks. I wrote another proposal (should we have more or less standard
"Git Enhancement Proposal" process like Python's PEP or Scheme's SRFI
to keep track of these over time?) but it's good to see others look at
it too.
> Proposal:
>
> Change tag refspecs to distinguish between remote and local tags. An
> unadorned tag "foo" could point to different commits in different
> repositories. A remote could move/edit it's "foo" tag and have that update
> smoothly propagated to clones.
>
> I believe this was last brought up in November while discussing the refs base
> for notes:
>
> http://thread.gmane.org/gmane.comp.version-control.git/160503/focus=160655
One problem with the proposed ref layout is that it breaks current
layout (remotes/<remote>/head -> remotes/<remote>/heads/head). Some
sort of migration support is needed. On the other hand, new layout is
cleaner than my proposal (remote-tags/<remote>/tag).
> Risks:
>
> I think the main risk lies in breaking plain <tagname> refs, as they would
> become "origin/<tagname>" refs instead. But I think that can be mitagated
> against (see below).
> ...
> To help mitigate the risk of breaking plain "<tagname>" refs, "git rev-parse"
> can look for plain names (i.e. ones without a /) in the remote tags location.
Hmm I thought "a-ref" would check "refs/tags/a-ref" and
"refs/remotes/*/a-ref". But you are right. Maybe "tags.relative" can
take three values instead of boolean (names TBD):
- deprecated (current behavior)
- migrating (fetch tags to refs/remotes, but tag lookup will look in
refs/tags as well as refs/remotes/*/tags)
- migrated (fetch tags to refs/remotes, look up in order)
We may slowly turn default value step by step until it becomes "migrated".
--
Duy
^ permalink raw reply
* Re: [1.8.0] forbid full fetchspecs in git-pull
From: Thomas Rast @ 2011-02-01 15:14 UTC (permalink / raw)
To: Dmitry Potapov
Cc: Junio C Hamano, git, Sean Estabrooks, Björn Steinbrink
In-Reply-To: <AANLkTikxcd+gzeuJsQX1V5Wses8xWMnshdrOnYTvXgTq@mail.gmail.com>
Dmitry Potapov wrote:
> As to disallowing ':' in refspec completely, I am not so sure... Not
> that I think it is very useful, but also I don't see how it can hurt
> someone provided that the target branch cannot be the current branch.
IRC experience shows that people, while on some topic branch, run
git pull origin master:master
expecting it to "pull master into master" (or even worse with three
different branch names). So no, the current branch safeguard does
not prevent the fundamental mistake.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Re: [1.8.0] Remote tag namespace
From: Marc Branchaud @ 2011-02-01 15:07 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: Git Mailing List, Nicolas Pitre
In-Reply-To: <AANLkTi=yFwOAQMHhvLsB1_xmYOE9HHP2YB4H4TQzwwc8@mail.gmail.com>
On 11-02-01 05:44 AM, Nguyen Thai Ngoc Duy wrote:
> On Tue, Feb 1, 2011 at 11:16 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
>> On Tue, 1 Feb 2011, Nguyen Thai Ngoc Duy wrote:
>>> Another random wish, which does not come with a proposal. How about
>>> tag namespace (ie. tags from a remote stay in remote namespace)?
>>
>> Please make this into a proper proposal. this would be indeed a huge
>> improvement.
>
> OK I'm not familiar with tag code, but I can try.
OK, that teaches me to read through _all_ the unread messages before posting!
Needless to say, I support this proposal.
> Proposal:
>
> Reserve refs/remote-tags namespace to store tags from remotes. Its
> structure is the same as in refs/remotes. When pulling tags, put them
> in refs/remote-tags/<remote> instead of refs/tags.
> Tag dereference code will be taught about refs/remote-tags with
> similar deref order as in remote branches.
I suggested a different home for the tags, but I don't have any insight into
what makes the most sense. I'll defer to wiser folk on this.
> Config branch.*.globalTags (perhaps takes a pattern?) may be defined
> to create refs/tags/* in addition to refs/remote-tags/<remote>/* when
> fetching tags.
I may be getting into the weeds prematurely here, but why put the config item
under branch.* ? Or did you mean remote.*.globalTags? Personally, I don't
see a need for this. I'd rather have the rev-parse machinery search in
remote tag namespaces if it can't find anything local.
> Migration plan:
>
> refs/remote-tags will be used to store new tags unconditionally, which
> means there will be duplicates with the already-fetched tags in global
> namespace. Perhaps we can check if they point to the same sha-1, then
> choose not to annoy users with ambiguous tag messages?
(Again with the weeds...) I don't think we could do that. I'd want to be
able to have my own (local) tags that refer to the same commits as one or
more remote tags, and I'd want to see them all.
Better for "git tag" to learn scoping options like "git branch": -a and -r.
(Hmm, maybe git-tag's current -a could become -A...)
> I suggest to add config compatibility.remoteTagNamespace, default to
> false, which retains current behavior (i.e. also create tags in global
> namespace in addition to refs/remote-tags). After 1.8.0 (or a few more
> cycles) the default value becomes true. Users who wish to keep old
> behavior can put "false" in their ~/.gitconfig.
>
> After a few years, remove support for the config key. Unrecognized
> compatibility.* keys will abort program. Users are forced to new
> behavior. I don't know, we may want to start annoy users that have the
> config key set a few cycles before we drop support.
Sounds good. I'd vote for a faster transition, but that's just me. :)
M.
^ permalink raw reply
* Re: [1.8.0] Unify "pathspec" semantics
From: Nguyen Thai Ngoc Duy @ 2011-02-01 14:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7voc6x57el.fsf_-_@alter.siamese.dyndns.org>
On Tue, Feb 1, 2011 at 12:07 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Some projects may track a file whose name is asterisk (e.g. "foo/*") and
> output from "git log 'foo/*'" would look different. Before the change,
> only commits that touch that exact path would be shown, but after the
> change, any commit that touch a path underneath "foo/" directory will be
> shown. This is a backward incompatible change.
Can we support quoting wildcards? I can imagine a file name such as
'***DO NOT DO IT***'. People who wish to match exactly that file would
have hard time ahead without a way to tell git those stars are
literal.
A prefix/special leading symbol or cmdline option to indicate the
given pathspec is literal is fine too (e.g "!literal:***hey***" or
--literal "***hey***"). In fact I can extend that to support negative
pathspecs.
--
Duy
^ permalink raw reply
* [1.8.0] Tag namespaces
From: Marc Branchaud @ 2011-02-01 14:54 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: git
In-Reply-To: <AANLkTikeqsg+qJ0z4iQ6ZmKL=_HB8YX_z20L=dFFApmA@mail.gmail.com>
On 11-01-31 10:20 PM, Nguyen Thai Ngoc Duy wrote:
> On Tue, Feb 1, 2011 at 12:05 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Now the 1.7.4 release is out, I'd like people to help thinking about the
>> next cycle(s).
>>
>> As a discussion-starter, here are my random wishes. Even though this does
>> not attempt to be exhaustive, keeping the number of goals manageably small
>> may help us focus.
>
> Another random wish, which does not come with a proposal. How about
> tag namespace (ie. tags from a remote stay in remote namespace)?
I had just started writing up such a proposal yesterday. What I have so far
is pretty preliminary:
Proposal:
Change tag refspecs to distinguish between remote and local tags. An
unadorned tag "foo" could point to different commits in different
repositories. A remote could move/edit it's "foo" tag and have that update
smoothly propagated to clones.
I believe this was last brought up in November while discussing the refs base
for notes:
http://thread.gmane.org/gmane.comp.version-control.git/160503/focus=160655
Risks:
I think the main risk lies in breaking plain <tagname> refs, as they would
become "origin/<tagname>" refs instead. But I think that can be mitagated
against (see below).
The other risk folks might raise, though I disagree, is breaking the
immutability assumption for tags. I'm willing to debate this, though (see
the above-linked thread).
Another "risk" is that this change might be too much of an earthquake. It
may be something more suitable to a major release, like 2.0.
Migration plan:
Add a "tags.relative" (name TBD) configuration variable which defaults to
false. When tags.relative is true, "git fetch" puts any received tags under
(location TBD) refs/remotes/<remote>/tags/. In 1.8.0 we flip tags.realtive's
default value.
To help mitigate the risk of breaking plain "<tagname>" refs, "git rev-parse"
can look for plain names (i.e. ones without a /) in the remote tags location.
M.
^ permalink raw reply
* Re: does the clone I do have the commits within them .
From: Konstantin Khomoutov @ 2011-02-01 14:47 UTC (permalink / raw)
To: Vaishali; +Cc: git
In-Reply-To: <AANLkTim4EPoYujCcxMMTnOZ4kYR_zG_Nfra2=9am-bLa@mail.gmail.com>
On Tue, 1 Feb 2011 18:50:39 +0530
Vaishali <vaishali.dhakate@sukrutsystems.com> wrote:
> hi , I am have cloned the
> git://gitorious.org/~dbeck/hawkboard/dbeck-hawkboard-linux-omapl1.git
> found at
> http://gitorious.org/~dbeck/hawkboard/dbeck-hawkboard-linux-omapl1...
> Now I see there is a tab called Commit log ...The git that I have
> cloned , would it include the commits in totality or will I have to
> do somethng diferent to get the commits into the git that I cloned on
> my PC. git checkout -f master git pull ... Is this .? Thanks
If I got it right, the question is "does cloning a repository bring
all the data or just changelogs?". In this case the answer is "yes, it
brings all the data". That is, you can check out any commit in the
history record of any branch fetched by git-clone.
As usually, starting from a good book on Git it recommended, see
http://git-scm.com/documentation
^ permalink raw reply
* Re: [PATCH 1/3] vcs-svn: Introduce svnload, a dumpfile producer
From: Erik Faye-Lund @ 2011-02-01 14:46 UTC (permalink / raw)
To: Ramkumar Ramachandra
Cc: Git List, Jonathan Nieder, David Barr, Sverre Rabbelier,
Junio C Hamano
In-Reply-To: <1296570403-9082-2-git-send-email-artagnon@gmail.com>
A very superficial review, because I don't have much time, and don't
know the surrounding code well. Sorry about that.
On Tue, Feb 1, 2011 at 3:26 PM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:
> diff --git a/vcs-svn/dir_cache.c b/vcs-svn/dir_cache.c
> new file mode 100644
> index 0000000..9a608ce
> --- /dev/null
> +++ b/vcs-svn/dir_cache.c
> @@ -0,0 +1,40 @@
> +/*
> + * Licensed under a two-clause BSD-style license.
> + * See LICENSE for details.
> + */
> +
> +#include "git-compat-util.h"
> +#include "string-list.h"
> +#include "line_buffer.h"
> +#include "dump_export.h"
> +
> +static struct string_list dirents = STRING_LIST_INIT_DUP;
> +static struct string_list_item *dir = NULL;
> +
> +void dir_cache_add(const char *path, enum node_kind kind) {
Style: we put the opening bracket of functions on the next line.
> + dir = string_list_insert(&dirents, path);
> + dir->util = malloc(sizeof(enum node_kind));
> + *((enum node_kind *)(dir->util)) = kind;
Unchecked malloc; perhaps you should use xmalloc instead?
> +}
> +
> +void dir_cache_remove(const char *path) {
Same style-violation as above.
> + dir = string_list_lookup(&dirents, path);
> + if (dir)
> + *((enum node_kind *)(dir->util)) = NODE_KIND_UNKNOWN;
> +}
> +
> +enum node_kind dir_cache_lookup(const char *path) {
> + dir = string_list_lookup(&dirents, path);
> + if (dir)
> + return *((enum node_kind *)(dir->util));
> + else
> + return NODE_KIND_UNKNOWN;
> +}
> +
> +void dir_cache_init() {
> + return;
> +}
> +
> +void dir_cache_deinit() {
> + string_list_clear(&dirents, 1);
> +}
Three more.
> +static void populate_revprops(struct strbuf *props, size_t author_len,
> + const char *author, size_t log_len, const char *log,
> + size_t date_len, const char *date)
> +{
> + strbuf_reset(props);
> + strbuf_addf(props, "K 10\nsvn:author\nV %lu\n%s\n", author_len, author);
> + strbuf_addf(props, "K 7\nsvn:log\nV %lu\n%s\n", log_len, log);
> + if (date_len)
> + /* SVN doesn't like an empty svn:date value */
> + strbuf_addf(props, "K 8\nsvn:date\nV %lu\n%s\n", date_len, date);
Perhaps a scope around here will make this a bit easier to read. At
first glance it looked like a missing scope to me, due to the indented
comment...
> + if (!val) die("Malformed author line");
> + if (!(tz_off = strrchr(val, ' '))) goto error;
> + *tz_off++ = '\0';
> + if (!(t = strrchr(val, ' '))) goto error;
style: use
"if (x)
do_stuff();"
instead of
"if (x) do_stuff();"
> + *(t - 1) = '\0'; /* Ignore '>' from email */
> + t++;
> + tz_off_buf = atoi(tz_off);
> + if (tz_off_buf > 1200 || tz_off_buf < -1200) goto error;
same
> + tm_time = time_to_tm(strtoul(t, NULL, 10), tz_off_buf);
> + strftime(time_buf, SVN_DATE_LEN + 1, SVN_DATE_FORMAT, tm_time);
> + strbuf_add(date, time_buf, SVN_DATE_LEN);
> + if (!(t = strchr(val, '<'))) goto error;
same
> +int parse_filemodify_mode(char *val)
> +{
> + char *t;
> +
> + if (!(t = strchr(val, ' '))) goto error;
same
> +void svnload_read(void)
> +{
> + char *t, *val;
> + int len;
> +
> + while ((t = buffer_read_line(&input))) {
> + if ((val = strchr(t, ' ')))
> + *val++ = '\0';
> + len = (val ? val - t - 1 : strlen(t));
> +
> + switch (len) {
> + case 1:
> + if (!memcmp(t, "D", 1)) {
> + node_ctx.action = NODE_ACTION_DELETE;
> + } else if (!memcmp(t, "C", 1)) {
> + node_ctx.action = NODE_ACTION_ADD;
> + } else if (!memcmp(t, "R", 1)) {
> + node_ctx.action = NODE_ACTION_REPLACE;
> + } else if (!memcmp(t, "M", 1)) {
> + if (!val) goto error;
> + node_ctx.action = NODE_ACTION_CHANGE;
> + val += parse_filemodify_mode(val);
> + if (!(t = strchr(val, ' '))) goto error;
same
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Andreas Ericsson @ 2011-02-01 14:42 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Jeff King, Junio C Hamano, git
In-Reply-To: <alpine.LFD.2.00.1101311621150.8580@xanadu.home>
On 01/31/2011 10:28 PM, Nicolas Pitre wrote:
> On Mon, 31 Jan 2011, Jeff King wrote:
>
>> On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:
>>
>>> We do have subdirectories for documentation, tests, contributions, etc.
>>> But a sizeable part of the tree is just a big splat of source files
>>> dumped right in the root of the tree.
>>>
>>> So I'd suggest doing the following:
>>>
>>> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
>>> the builtin directory from the root directory to it.
>>
>> Wouldn't this just be the same giant splat of source files, but in a
>> different tree? I don't really see the advantage, and it seems like an
>> extra annoyance.
>
> Like I said to Junio, if you don't see the advantage, there's nothing I
> can do for you. To me this is simple good source code hygiene.
>
>> Besides being just one more directory to go up and down, it does make
>> history browsing more annoying. As much as I love git's "don't record
>> renames" philosophy, our handling of renames on the viewing side is
>> often annoying. I already get annoyed sometimes following stuff across
>> the s!builtin-!builtin/! change. This would be like that but more so.
>
> So... we do suck at something? So why not take this opportunity to
> shake yourself out of this easy comfort and improve Git as a result on
> both front? :-)
>
>> Or maybe it is a good thing for that reason, as we will eat our own
>> rename dogfood. :)
>
> Exactly! And maybe we'll make Git even more useful in the process.
>
>>> 5) Rename t/ to testsuite/ so this doesn't look like some garbage
>>> leftover.
>>
>> Ugh, more typing. :P
>
> Come on! You sound like an old fart now! ;-)
>
Personally, I kinda like the capital D in Documentation for tab
completion reasons. Keeping frequently used files and directories
with short unique prefixes makes perfect sense from a typing point
of view. Using longer mnemonic names makes perfect sense from a
regex search/replace point of view.
I'm kinda with Junio on the ./*.[ch] -> src/*.[ch] move though, but
perhaps that's just because I hate autoconf projects which generate
a ton of cruft in the root before one can even start building it.
It would probably help matters along if buildproducts ended up in
their own directory though. That way .gitignore won't have so many
extra commits when new source files are added, and 'make clean'
gets easier to maintain.
So to sum up what I'm for;
t/ -> Test/ (or Testsuite, but some more mnemonic name anyways
with a short unique prefix for tab completion). This would also
exercise our rename machinery quite a bit, altohugh not to the
point where people get annoyed if it gets sort-of-broken.
buildproducts to Build/ (capital B to avoid completing against
builtin*). This also has the benefit that %.o: %.c rules makes
tab completion work better when object files are already built,
and "git add git-foo.*" doesn't throw the "git-foo.o is ignored"
error and forces one to re-type it as "git-foo.[ch]"
The ppc stuff I don't really care about and it wouldn't be hard
to resurrect if we remove it and people complain. Everything
will also still work if we do, although with possibly a slight
decrease in performance.
Bulk of source-files stay as ./*.[ch]. I see absolutely no
benefit to moving them, but two potential drawbacks. One is that
it'll cause more typing for those of us who use console-based
editors and run 'make' manually (yes, we do exist). The afore-
mentioned merge+rebase hell is also a considerable drawback,
even though that's primarily an issue for maint releases.
Risks: Well... dunno about that really. Mucking up the build and
test systems, I suppose, but it's easy enough to test.
Cons: Rebasing and merging stuff to the Makefile and test-stuff
will suck a bit, but as has been pointed out that's not only a
bad thing.
Pros: Less typing all-round. Simpler "make clean" rules. Fewer
tacked-on .gitignore patches for new commands. We (well, Junio)
get to experience first-hand the problems of directory renames
across releases but on a smaller scale than moving *everything*
around in one go.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.
^ permalink raw reply
* [PATCH 3/3] vcs-svn: Refactor dump_export code into dispatch table
From: Ramkumar Ramachandra @ 2011-02-01 14:26 UTC (permalink / raw)
To: Git List; +Cc: Jonathan Nieder, David Barr, Sverre Rabbelier, Junio C Hamano
In-Reply-To: <1296570403-9082-1-git-send-email-artagnon@gmail.com>
Suggested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
---
vcs-svn/dump_export.c | 89 +++++++++++++++++++++++++------------------------
vcs-svn/dump_export.h | 6 ++-
2 files changed, 49 insertions(+), 46 deletions(-)
diff --git a/vcs-svn/dump_export.c b/vcs-svn/dump_export.c
index 2b23f77..a8331fd 100644
--- a/vcs-svn/dump_export.c
+++ b/vcs-svn/dump_export.c
@@ -11,6 +11,48 @@
static struct strbuf props;
+typedef void state_fn(void);
+
+static void Kfile (void) { printf("Node-kind: file\n"); }
+static void Kdir (void) { printf("Node-kind: dir\n"); }
+static void Achange (void) { printf("Node-action: change\n"); }
+static void Aadd (void) { printf("Node-action: add\n"); }
+static void Adelete (void) { printf("Node-action: delete\n"); }
+static void Areplace(void) { printf("Node-action: replace\n"); }
+static void Pexec (void) { strbuf_addf(&props, "K 14\nsvn:executable\nV 1\n*\n"); }
+static void Psym (void) { strbuf_addf(&props, "K 11\nsvn:special\nV 1\n*\n"); }
+static void Pend (void) { strbuf_add(&props, "PROPS-END\n", 10); }
+
+static void Nchange (void) { Kfile(); Achange(); }
+static void Nadd (void) { Kfile(); Aadd(); }
+static void Nreplace(void) { Kfile(); Areplace(); }
+static void Echange (void) { Pexec(); Pend(); Kfile(); Achange(); }
+static void Eadd (void) { Pexec(); Pend(); Kfile(); Aadd(); }
+static void Ereplace(void) { Pexec(); Pend(); Kfile(); Areplace(); }
+static void Schange (void) { Psym(); Pend(); Kfile(); Achange(); }
+static void Sadd (void) { Psym(); Pend(); Kfile(); Aadd(); }
+static void Sreplace(void) { Psym(); Pend(); Kfile(); Areplace(); }
+static void Dchange (void) { Kdir(); Achange(); }
+static void Dadd (void) { Kdir(); Aadd(); }
+static void Dreplace(void) { Kdir(); Areplace(); }
+
+static state_fn *const dispatch_table[NODE_KIND_COUNT][NODE_ACTION_COUNT] = {
+ /* NODE_KIND_UNKNOWN */
+ {abort, abort, abort, Adelete, abort},
+ /* NODE_KIND_NORMAL */
+ {abort, Nchange, Nadd, Adelete, Nreplace},
+ /* NODE_KIND_EXECUTABLE */
+ {abort, Echange, Eadd, Adelete, Ereplace},
+ /* NODE_KIND_SYMLINK */
+ {abort, Schange, Sadd, Adelete, Sreplace},
+ /* NODE_KIND_GITLINK */
+ {abort, abort, abort, abort, abort},
+ /* NODE_KIND_DIR */
+ {abort, Dchange, Dadd, Adelete, Dreplace},
+ /* NODE_KIND_SUBDIR */
+ {abort, abort, abort, abort, abort}
+};
+
void dump_export_begin_rev(int revision, const char *revprops,
int prop_len)
{
@@ -24,56 +66,15 @@ void dump_export_node(const char *path, enum node_kind kind,
enum node_action action, unsigned long text_len,
unsigned long copyfrom_rev, const char *copyfrom_path)
{
- int dump_props = 1; /* Boolean */
strbuf_reset(&props);
printf("Node-path: %s\n", path);
- switch (kind) {
- case NODE_KIND_NORMAL:
- printf("Node-kind: file\n");
- break;
- case NODE_KIND_EXECUTABLE:
- printf("Node-kind: file\n");
- strbuf_addf(&props, "K 14\nsvn:executable\nV 1\n*\n");
- break;
- case NODE_KIND_SYMLINK:
- printf("Node-kind: file\n");
- strbuf_addf(&props, "K 11\nsvn:special\nV 1\n*\n");
- break;
- case NODE_KIND_GITLINK:
- printf("Node-kind: file\n");
- break;
- case NODE_KIND_DIR:
- printf("Node-kind: dir\n");
- break;
- case NODE_KIND_SUBDIR:
- die("Unsupported: subdirectory");
- default:
- break;
- }
- strbuf_add(&props, "PROPS-END\n", 10);
+ dispatch_table[kind][action]();
- switch (action) {
- case NODE_ACTION_CHANGE:
- printf("Node-action: change\n");
- break;
- case NODE_ACTION_ADD:
- printf("Node-action: add\n");
- break;
- case NODE_ACTION_REPLACE:
- printf("Node-action: replace\n");
- break;
- case NODE_ACTION_DELETE:
- printf("Node-action: delete\n");
- dump_props = 0;
- break;
- default:
- break;
- }
if (copyfrom_rev != SVN_INVALID_REV) {
printf("Node-copyfrom-rev: %lu\n", copyfrom_rev);
printf("Node-copyfrom-path: %s\n", copyfrom_path);
}
- if (dump_props) {
+ if (props.len) {
printf("Prop-delta: false\n");
printf("Prop-content-length: %lu\n", props.len);
}
@@ -81,7 +82,7 @@ void dump_export_node(const char *path, enum node_kind kind,
printf("Text-delta: false\n");
printf("Text-content-length: %lu\n", text_len);
}
- if (text_len || dump_props) {
+ if (text_len || props.len) {
printf("Content-length: %lu\n\n", text_len + props.len);
printf("%s", props.buf);
}
diff --git a/vcs-svn/dump_export.h b/vcs-svn/dump_export.h
index e9f51a3..8265170 100644
--- a/vcs-svn/dump_export.h
+++ b/vcs-svn/dump_export.h
@@ -9,7 +9,8 @@ enum node_action {
NODE_ACTION_CHANGE,
NODE_ACTION_ADD,
NODE_ACTION_DELETE,
- NODE_ACTION_REPLACE
+ NODE_ACTION_REPLACE,
+ NODE_ACTION_COUNT
};
enum node_kind {
@@ -19,7 +20,8 @@ enum node_kind {
NODE_KIND_SYMLINK,
NODE_KIND_GITLINK,
NODE_KIND_DIR, /* SVN-specific */
- NODE_KIND_SUBDIR
+ NODE_KIND_SUBDIR,
+ NODE_KIND_COUNT
};
void dump_export_begin_rev(int revision, const char *revprops, int prop_len);
--
1.7.4.rc1.7.g2cf08.dirty
^ permalink raw reply related
* [PATCH 1/3] vcs-svn: Introduce svnload, a dumpfile producer
From: Ramkumar Ramachandra @ 2011-02-01 14:26 UTC (permalink / raw)
To: Git List; +Cc: Jonathan Nieder, David Barr, Sverre Rabbelier, Junio C Hamano
In-Reply-To: <1296570403-9082-1-git-send-email-artagnon@gmail.com>
Design-wise, svnload resembles svndump. Include a Makefile rule to
build it into vcs-svn/lib.a.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
---
Makefile | 4 +-
vcs-svn/dir_cache.c | 40 ++++++
vcs-svn/dir_cache.h | 12 ++
vcs-svn/dump_export.c | 149 +++++++++++++++++++++++
vcs-svn/dump_export.h | 33 +++++
vcs-svn/svnload.c | 322 +++++++++++++++++++++++++++++++++++++++++++++++++
vcs-svn/svnload.h | 10 ++
7 files changed, 568 insertions(+), 2 deletions(-)
create mode 100644 vcs-svn/dir_cache.c
create mode 100644 vcs-svn/dir_cache.h
create mode 100644 vcs-svn/dump_export.c
create mode 100644 vcs-svn/dump_export.h
create mode 100644 vcs-svn/svnload.c
create mode 100644 vcs-svn/svnload.h
diff --git a/Makefile b/Makefile
index 1345c38..d9c2442 100644
--- a/Makefile
+++ b/Makefile
@@ -1834,9 +1834,9 @@ ifndef NO_CURL
endif
XDIFF_OBJS = xdiff/xdiffi.o xdiff/xprepare.o xdiff/xutils.o xdiff/xemit.o \
xdiff/xmerge.o xdiff/xpatience.o
-VCSSVN_OBJS = vcs-svn/line_buffer.o \
+VCSSVN_OBJS = vcs-svn/line_buffer.o vcs-svn/svnload.o vcs-svn/dump_export.o \
vcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/sliding_window.o \
- vcs-svn/svndiff.o vcs-svn/svndump.o
+ vcs-svn/svndiff.o vcs-svn/svndump.o vcs-svn/dir_cache.o
VCSSVN_TEST_OBJS = test-obj-pool.o \
test-line-buffer.o test-treap.o test-svn-fe.o
OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)
diff --git a/vcs-svn/dir_cache.c b/vcs-svn/dir_cache.c
new file mode 100644
index 0000000..9a608ce
--- /dev/null
+++ b/vcs-svn/dir_cache.c
@@ -0,0 +1,40 @@
+/*
+ * Licensed under a two-clause BSD-style license.
+ * See LICENSE for details.
+ */
+
+#include "git-compat-util.h"
+#include "string-list.h"
+#include "line_buffer.h"
+#include "dump_export.h"
+
+static struct string_list dirents = STRING_LIST_INIT_DUP;
+static struct string_list_item *dir = NULL;
+
+void dir_cache_add(const char *path, enum node_kind kind) {
+ dir = string_list_insert(&dirents, path);
+ dir->util = malloc(sizeof(enum node_kind));
+ *((enum node_kind *)(dir->util)) = kind;
+}
+
+void dir_cache_remove(const char *path) {
+ dir = string_list_lookup(&dirents, path);
+ if (dir)
+ *((enum node_kind *)(dir->util)) = NODE_KIND_UNKNOWN;
+}
+
+enum node_kind dir_cache_lookup(const char *path) {
+ dir = string_list_lookup(&dirents, path);
+ if (dir)
+ return *((enum node_kind *)(dir->util));
+ else
+ return NODE_KIND_UNKNOWN;
+}
+
+void dir_cache_init() {
+ return;
+}
+
+void dir_cache_deinit() {
+ string_list_clear(&dirents, 1);
+}
diff --git a/vcs-svn/dir_cache.h b/vcs-svn/dir_cache.h
new file mode 100644
index 0000000..43c3797
--- /dev/null
+++ b/vcs-svn/dir_cache.h
@@ -0,0 +1,12 @@
+#ifndef DIR_CACHE_H_
+#define DIR_CACHE_H_
+
+#include "dump_export.h"
+
+void dir_cache_add(const char *path, enum node_kind kind);
+void dir_cache_remove(const char *path);
+enum node_kind dir_cache_lookup(const char *path);
+void dir_cache_init();
+void dir_cache_deinit();
+
+#endif
diff --git a/vcs-svn/dump_export.c b/vcs-svn/dump_export.c
new file mode 100644
index 0000000..2b23f77
--- /dev/null
+++ b/vcs-svn/dump_export.c
@@ -0,0 +1,149 @@
+/*
+ * Licensed under a two-clause BSD-style license.
+ * See LICENSE for details.
+ */
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+#include "line_buffer.h"
+#include "dump_export.h"
+#include "dir_cache.h"
+
+static struct strbuf props;
+
+void dump_export_begin_rev(int revision, const char *revprops,
+ int prop_len)
+{
+ printf("Revision-number: %d\n", revision);
+ printf("Prop-content-length: %d\n", prop_len);
+ printf("Content-length: %d\n\n", prop_len);
+ printf("%s\n", revprops);
+}
+
+void dump_export_node(const char *path, enum node_kind kind,
+ enum node_action action, unsigned long text_len,
+ unsigned long copyfrom_rev, const char *copyfrom_path)
+{
+ int dump_props = 1; /* Boolean */
+ strbuf_reset(&props);
+ printf("Node-path: %s\n", path);
+ switch (kind) {
+ case NODE_KIND_NORMAL:
+ printf("Node-kind: file\n");
+ break;
+ case NODE_KIND_EXECUTABLE:
+ printf("Node-kind: file\n");
+ strbuf_addf(&props, "K 14\nsvn:executable\nV 1\n*\n");
+ break;
+ case NODE_KIND_SYMLINK:
+ printf("Node-kind: file\n");
+ strbuf_addf(&props, "K 11\nsvn:special\nV 1\n*\n");
+ break;
+ case NODE_KIND_GITLINK:
+ printf("Node-kind: file\n");
+ break;
+ case NODE_KIND_DIR:
+ printf("Node-kind: dir\n");
+ break;
+ case NODE_KIND_SUBDIR:
+ die("Unsupported: subdirectory");
+ default:
+ break;
+ }
+ strbuf_add(&props, "PROPS-END\n", 10);
+
+ switch (action) {
+ case NODE_ACTION_CHANGE:
+ printf("Node-action: change\n");
+ break;
+ case NODE_ACTION_ADD:
+ printf("Node-action: add\n");
+ break;
+ case NODE_ACTION_REPLACE:
+ printf("Node-action: replace\n");
+ break;
+ case NODE_ACTION_DELETE:
+ printf("Node-action: delete\n");
+ dump_props = 0;
+ break;
+ default:
+ break;
+ }
+ if (copyfrom_rev != SVN_INVALID_REV) {
+ printf("Node-copyfrom-rev: %lu\n", copyfrom_rev);
+ printf("Node-copyfrom-path: %s\n", copyfrom_path);
+ }
+ if (dump_props) {
+ printf("Prop-delta: false\n");
+ printf("Prop-content-length: %lu\n", props.len);
+ }
+ if (text_len) {
+ printf("Text-delta: false\n");
+ printf("Text-content-length: %lu\n", text_len);
+ }
+ if (text_len || dump_props) {
+ printf("Content-length: %lu\n\n", text_len + props.len);
+ printf("%s", props.buf);
+ }
+ if (!text_len)
+ printf("\n");
+}
+
+void dump_export_node_r(const char *path, enum node_kind kind,
+ enum node_action action, unsigned long text_len,
+ unsigned long copyfrom_rev, const char *copyfrom_path)
+{
+ char *start, *t;
+ start = (char *) path;
+
+ while ((t = strchr(start, '/'))) {
+ *t = '\0';
+ if (dir_cache_lookup(path) == NODE_KIND_UNKNOWN) {
+ dir_cache_add(path, NODE_KIND_NORMAL);
+ dump_export_node(path, NODE_KIND_DIR,
+ NODE_ACTION_ADD, 0,
+ SVN_INVALID_REV, NULL);
+ }
+ *t = '/'; /* Change it back */
+ start = t + 1;
+ }
+ switch (dir_cache_lookup(path)) {
+ case NODE_KIND_UNKNOWN:
+ action = NODE_ACTION_ADD;
+ break;
+ case NODE_KIND_SYMLINK:
+ dir_cache_remove(path);
+ dump_export_node(path, NODE_KIND_UNKNOWN,
+ NODE_ACTION_DELETE,
+ 0, SVN_INVALID_REV, NULL);
+ action = NODE_ACTION_ADD;
+ break;
+ case NODE_KIND_DIR:
+ die("File was previously a directory?");
+ break;
+ case NODE_KIND_SUBDIR:
+ die("Subdirectories unsupported");
+ break;
+ default:
+ action = NODE_ACTION_CHANGE;
+ break;
+ }
+ dir_cache_add(path, kind);
+ dump_export_node(path, kind, action, text_len, copyfrom_rev, copyfrom_path);
+}
+
+void dump_export_text(struct line_buffer *data, off_t len)
+{
+ buffer_copy_bytes(data, len);
+}
+
+void dump_export_init()
+{
+ strbuf_init(&props, MAX_GITSVN_LINE_LEN);
+ printf("SVN-fs-dump-format-version: 3\n\n");
+}
+
+void dump_export_deinit()
+{
+ strbuf_release(&props);
+}
diff --git a/vcs-svn/dump_export.h b/vcs-svn/dump_export.h
new file mode 100644
index 0000000..e9f51a3
--- /dev/null
+++ b/vcs-svn/dump_export.h
@@ -0,0 +1,33 @@
+#ifndef DUMP_EXPORT_H_
+#define DUMP_EXPORT_H_
+
+#define MAX_GITSVN_LINE_LEN 4096
+#define SVN_INVALID_REV 0
+
+enum node_action {
+ NODE_ACTION_UNKNOWN,
+ NODE_ACTION_CHANGE,
+ NODE_ACTION_ADD,
+ NODE_ACTION_DELETE,
+ NODE_ACTION_REPLACE
+};
+
+enum node_kind {
+ NODE_KIND_UNKNOWN, /* Missing node */
+ NODE_KIND_NORMAL,
+ NODE_KIND_EXECUTABLE,
+ NODE_KIND_SYMLINK,
+ NODE_KIND_GITLINK,
+ NODE_KIND_DIR, /* SVN-specific */
+ NODE_KIND_SUBDIR
+};
+
+void dump_export_begin_rev(int revision, const char *revprops, int prop_len);
+void dump_export_text(struct line_buffer *data, off_t len);
+void dump_export_node_r(const char *path, enum node_kind kind,
+ enum node_action action, unsigned long text_len,
+ unsigned long copyfrom_rev, const char *copyfrom_path);
+void dump_export_init();
+void dump_export_deinit();
+
+#endif
diff --git a/vcs-svn/svnload.c b/vcs-svn/svnload.c
new file mode 100644
index 0000000..40fc1db
--- /dev/null
+++ b/vcs-svn/svnload.c
@@ -0,0 +1,322 @@
+/*
+ * Produce a dumpfile v3 from a fast-import stream.
+ * Load the dump into the SVN repository with:
+ * svnrdump load <URL> <dumpfile
+ *
+ * Licensed under a two-clause BSD-style license.
+ * See LICENSE for details.
+ */
+
+#include "cache.h"
+#include "git-compat-util.h"
+#include "line_buffer.h"
+#include "dump_export.h"
+#include "dir_cache.h"
+
+#define SVN_DATE_FORMAT "%Y-%m-%dT%H:%M:%S.000000Z"
+#define SVN_DATE_LEN 27
+#define LENGTH_UNKNOWN (~0)
+
+static struct line_buffer input = LINE_BUFFER_INIT;
+
+static struct {
+ unsigned long prop_len, text_len, copyfrom_rev;
+ int text_delta, prop_delta; /* false=0, true=1, unknown=-1 */
+ enum node_action action;
+ enum node_kind kind;
+ struct strbuf copyfrom_path, path;
+} node_ctx;
+
+static struct {
+ int rev, text_len;
+ struct strbuf props, log;
+ struct strbuf svn_author, author, committer;
+ struct strbuf author_date, committer_date;
+ struct strbuf author_email, committer_email;
+} rev_ctx;
+
+static enum {
+ UNKNOWN_CTX,
+ COMMIT_CTX,
+ BLOB_CTX
+} active_ctx;
+
+static void populate_revprops(struct strbuf *props, size_t author_len,
+ const char *author, size_t log_len, const char *log,
+ size_t date_len, const char *date)
+{
+ strbuf_reset(props);
+ strbuf_addf(props, "K 10\nsvn:author\nV %lu\n%s\n", author_len, author);
+ strbuf_addf(props, "K 7\nsvn:log\nV %lu\n%s\n", log_len, log);
+ if (date_len)
+ /* SVN doesn't like an empty svn:date value */
+ strbuf_addf(props, "K 8\nsvn:date\nV %lu\n%s\n", date_len, date);
+ strbuf_add(props, "PROPS-END\n", 10);
+}
+
+static void parse_author_line(char *val, struct strbuf *name,
+ struct strbuf *email, struct strbuf *date)
+{
+ char *t, *tz_off;
+ char time_buf[SVN_DATE_LEN + 1];
+ int tz_off_buf;
+ const struct tm *tm_time;
+
+ /* Author Name <author@email.com> 1170199019 +0530 */
+ strbuf_reset(name);
+ strbuf_reset(email);
+ strbuf_reset(date);
+ if (!val) die("Malformed author line");
+ if (!(tz_off = strrchr(val, ' '))) goto error;
+ *tz_off++ = '\0';
+ if (!(t = strrchr(val, ' '))) goto error;
+ *(t - 1) = '\0'; /* Ignore '>' from email */
+ t++;
+ tz_off_buf = atoi(tz_off);
+ if (tz_off_buf > 1200 || tz_off_buf < -1200) goto error;
+ tm_time = time_to_tm(strtoul(t, NULL, 10), tz_off_buf);
+ strftime(time_buf, SVN_DATE_LEN + 1, SVN_DATE_FORMAT, tm_time);
+ strbuf_add(date, time_buf, SVN_DATE_LEN);
+ if (!(t = strchr(val, '<'))) goto error;
+ *(t - 1) = '\0'; /* Ignore ' <' from email */
+ t++;
+ strbuf_add(email, t, strlen(t));
+ strbuf_add(name, val, strlen(val));
+ return;
+error:
+ die("Malformed author line: %s", val);
+}
+
+void build_svn_author(struct strbuf *svn_author)
+{
+ char *t, *email;
+
+ strbuf_reset(svn_author);
+ email = rev_ctx.author_email.buf;
+ if (!(t = strchr(email, '@')))
+ goto error;
+ strbuf_add(svn_author, email, t - email);
+ return;
+error:
+ die("Malformed email: %s", email);
+}
+
+int parse_filemodify_mode(char *val)
+{
+ char *t;
+
+ if (!(t = strchr(val, ' '))) goto error;
+ switch (t - val) {
+ case 6:
+ if (!memcmp(val, "100644", 6))
+ node_ctx.kind = NODE_KIND_NORMAL;
+ else if (!memcmp(val, "100755", 6))
+ node_ctx.kind = NODE_KIND_EXECUTABLE;
+ else if (!memcmp(val, "120000", 6))
+ node_ctx.kind = NODE_KIND_SYMLINK;
+ else if (!memcmp(val, "160000", 6))
+ node_ctx.kind = NODE_KIND_GITLINK;
+ else if (!memcmp(val, "040000", 6))
+ node_ctx.kind = NODE_KIND_SUBDIR;
+ else
+ goto error;
+ break;
+ case 3:
+ if (!memcmp(val, "755", 3))
+ node_ctx.kind = NODE_KIND_EXECUTABLE;
+ else if (!memcmp(val, "644", 3))
+ node_ctx.kind = NODE_KIND_NORMAL;
+ else
+ goto error;
+ break;
+ default:
+ goto error;
+ }
+ return t - val + 1;
+error:
+ die("Unrecognized mode: %s", val);
+}
+
+void svnload_read(void)
+{
+ char *t, *val;
+ int len;
+
+ while ((t = buffer_read_line(&input))) {
+ if ((val = strchr(t, ' ')))
+ *val++ = '\0';
+ len = (val ? val - t - 1 : strlen(t));
+
+ switch (len) {
+ case 1:
+ if (!memcmp(t, "D", 1)) {
+ node_ctx.action = NODE_ACTION_DELETE;
+ } else if (!memcmp(t, "C", 1)) {
+ node_ctx.action = NODE_ACTION_ADD;
+ } else if (!memcmp(t, "R", 1)) {
+ node_ctx.action = NODE_ACTION_REPLACE;
+ } else if (!memcmp(t, "M", 1)) {
+ if (!val) goto error;
+ node_ctx.action = NODE_ACTION_CHANGE;
+ val += parse_filemodify_mode(val);
+ if (!(t = strchr(val, ' '))) goto error;
+ *t++ = '\0';
+ strbuf_reset(&node_ctx.path);
+ strbuf_add(&node_ctx.path, t, strlen(t));
+ if (!strncmp(val, "inline", 6))
+ active_ctx = BLOB_CTX;
+ else if (*val == ':')
+ die("Unsupported dataref: marks");
+ else {
+ error:
+ die("Malformed filemodify line: %s", t);
+ }
+ }
+ break;
+ case 2:
+ if (!memcmp(t, "ls", 2))
+ die("ls not supported");
+ case 3:
+ if (!memcmp(t, "tag", 3))
+ continue;
+ break;
+ case 4:
+ if (!memcmp(t, "blob", 4))
+ continue;
+ else if (!memcmp(t, "mark", 4))
+ continue;
+ else if (!memcmp(t, "from", 4))
+ continue;
+ else if (!memcmp(t, "data", 4)) {
+ switch (active_ctx) {
+ case COMMIT_CTX:
+ strbuf_reset(&rev_ctx.log);
+ buffer_read_binary(&input,
+ &rev_ctx.log,
+ strtoul(val, NULL, 10));
+ populate_revprops(&rev_ctx.props,
+ rev_ctx.svn_author.len,
+ rev_ctx.svn_author.buf,
+ rev_ctx.log.len,
+ rev_ctx.log.buf,
+ rev_ctx.author_date.len,
+ rev_ctx.author_date.buf);
+ dump_export_begin_rev(rev_ctx.rev,
+ rev_ctx.props.buf,
+ rev_ctx.props.len);
+ break;
+ case BLOB_CTX:
+ node_ctx.text_len = strtoul(val, NULL, 10);
+ dump_export_node_r(node_ctx.path.buf, node_ctx.kind,
+ node_ctx.action, node_ctx.text_len,
+ SVN_INVALID_REV, NULL);
+ buffer_copy_bytes(&input, node_ctx.text_len);
+ break;
+ default:
+ break;
+ }
+ }
+ break;
+ case 5:
+ if (!memcmp(t, "reset", 5))
+ continue;
+ if (!memcmp(t, "merge", 5))
+ continue;
+ break;
+ case 6:
+ if (!memcmp(t, "author", 6)) {
+ parse_author_line(val, &rev_ctx.author,
+ &rev_ctx.author_email,
+ &rev_ctx.author_date);
+ build_svn_author(&rev_ctx.svn_author);
+ } else if (!memcmp(t, "commit", 6)) {
+ rev_ctx.rev++;
+ active_ctx = COMMIT_CTX;
+ }
+ break;
+ case 8:
+ if (!memcmp(t, "cat-blob", 8))
+ die("cat-blob unsupported");
+ break;
+ case 9:
+ if (!memcmp(t, "deleteall", 9))
+ continue;
+ else if (!memcmp(t, "committer", 9))
+ parse_author_line(val, &rev_ctx.committer,
+ &rev_ctx.committer_email,
+ &rev_ctx.committer_date);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static void reset_rev_ctx(int revision)
+{
+ rev_ctx.rev = revision;
+ strbuf_reset(&rev_ctx.props);
+ strbuf_reset(&rev_ctx.log);
+ strbuf_reset(&rev_ctx.svn_author);
+ strbuf_reset(&rev_ctx.author);
+ strbuf_reset(&rev_ctx.committer);
+ strbuf_reset(&rev_ctx.author_date);
+ strbuf_reset(&rev_ctx.committer_date);
+ strbuf_reset(&rev_ctx.author_email);
+ strbuf_reset(&rev_ctx.committer_email);
+}
+
+static void reset_node_ctx(void)
+{
+ node_ctx.prop_len = LENGTH_UNKNOWN;
+ node_ctx.text_len = LENGTH_UNKNOWN;
+ node_ctx.copyfrom_rev = SVN_INVALID_REV;
+ node_ctx.text_delta = -1;
+ node_ctx.prop_delta = -1;
+ strbuf_reset(&node_ctx.copyfrom_path);
+ strbuf_reset(&node_ctx.path);
+}
+
+int svnload_init(const char *filename)
+{
+ if (buffer_init(&input, filename))
+ return error("cannot open %s: %s", filename, strerror(errno));
+ active_ctx = UNKNOWN_CTX;
+ strbuf_init(&rev_ctx.props, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.log, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.author, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.committer, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.svn_author, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.author_date, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.committer_date, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.author_email, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&rev_ctx.committer_email, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&node_ctx.path, MAX_GITSVN_LINE_LEN);
+ strbuf_init(&node_ctx.copyfrom_path, MAX_GITSVN_LINE_LEN);
+ dump_export_init();
+ dir_cache_init();
+ return 0;
+}
+
+void svnload_deinit(void)
+{
+ reset_rev_ctx(0);
+ reset_node_ctx();
+ strbuf_release(&rev_ctx.props);
+ strbuf_release(&rev_ctx.log);
+ strbuf_release(&rev_ctx.author);
+ strbuf_release(&rev_ctx.committer);
+ strbuf_release(&rev_ctx.svn_author);
+ strbuf_release(&rev_ctx.author_date);
+ strbuf_release(&rev_ctx.committer_date);
+ strbuf_release(&rev_ctx.author_email);
+ strbuf_release(&rev_ctx.committer_email);
+ strbuf_release(&node_ctx.path);
+ strbuf_release(&node_ctx.copyfrom_path);
+ dump_export_deinit();
+ dir_cache_deinit();
+ if (buffer_deinit(&input))
+ fprintf(stderr, "Input error\n");
+ if (ferror(stdout))
+ fprintf(stderr, "Output error\n");
+}
diff --git a/vcs-svn/svnload.h b/vcs-svn/svnload.h
new file mode 100644
index 0000000..0c8fe8b
--- /dev/null
+++ b/vcs-svn/svnload.h
@@ -0,0 +1,10 @@
+#ifndef SVNLOAD_H_
+#define SVNLOAD_H_
+
+#define SVN_INVALID_REV -1
+
+int svnload_init(const char *filename);
+void svnload_deinit(void);
+void svnload_read(void);
+
+#endif
--
1.7.4.rc1.7.g2cf08.dirty
^ permalink raw reply related
* [PATCH 2/3] t9010-svn-fi: Add tests for svn-fi
From: Ramkumar Ramachandra @ 2011-02-01 14:26 UTC (permalink / raw)
To: Git List; +Cc: Jonathan Nieder, David Barr, Sverre Rabbelier, Junio C Hamano
In-Reply-To: <1296570403-9082-1-git-send-email-artagnon@gmail.com>
Create a test-svn-fi in toplevel directory, add rules to build it, and
add some basic tests.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
---
.gitignore | 1 +
Makefile | 5 +-
t/t9010-svn-fi.sh | 303 +++++++++++++++++++++++++++++++++++++++++++++++++++++
test-svn-fi.c | 20 ++++
4 files changed, 328 insertions(+), 1 deletions(-)
create mode 100644 t/t9010-svn-fi.sh
create mode 100644 test-svn-fi.c
diff --git a/.gitignore b/.gitignore
index b48d1ee..c8c8a17 100644
--- a/.gitignore
+++ b/.gitignore
@@ -177,6 +177,7 @@
/test-sigchain
/test-subprocess
/test-svn-fe
+/test-svn-fi
/common-cmds.h
*.tar.gz
*.dsc
diff --git a/Makefile b/Makefile
index d9c2442..cb21b78 100644
--- a/Makefile
+++ b/Makefile
@@ -431,6 +431,7 @@ TEST_PROGRAMS_NEED_X += test-sha1
TEST_PROGRAMS_NEED_X += test-sigchain
TEST_PROGRAMS_NEED_X += test-subprocess
TEST_PROGRAMS_NEED_X += test-svn-fe
+TEST_PROGRAMS_NEED_X += test-svn-fi
TEST_PROGRAMS_NEED_X += test-index-version
TEST_PROGRAMS = $(patsubst %,%$X,$(TEST_PROGRAMS_NEED_X))
@@ -1838,7 +1839,7 @@ VCSSVN_OBJS = vcs-svn/line_buffer.o vcs-svn/svnload.o vcs-svn/dump_export.o \
vcs-svn/repo_tree.o vcs-svn/fast_export.o vcs-svn/sliding_window.o \
vcs-svn/svndiff.o vcs-svn/svndump.o vcs-svn/dir_cache.o
VCSSVN_TEST_OBJS = test-obj-pool.o \
- test-line-buffer.o test-treap.o test-svn-fe.o
+ test-line-buffer.o test-treap.o test-svn-fe.o test-svn-fi.o
OBJECTS := $(GIT_OBJS) $(XDIFF_OBJS) $(VCSSVN_OBJS)
dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)
@@ -2129,6 +2130,8 @@ test-parse-options$X: parse-options.o
test-svn-fe$X: vcs-svn/lib.a
+test-svn-fi$X: vcs-svn/lib.a
+
.PRECIOUS: $(TEST_OBJS)
test-%$X: test-%.o $(GITLIBS)
diff --git a/t/t9010-svn-fi.sh b/t/t9010-svn-fi.sh
new file mode 100644
index 0000000..676c7fc
--- /dev/null
+++ b/t/t9010-svn-fi.sh
@@ -0,0 +1,303 @@
+#!/bin/sh
+
+test_description='check svn dumpfile exporter'
+
+. ./test-lib.sh
+
+if ! svnadmin -h >/dev/null 2>&1
+then
+ skip_all='skipping svn-fi tests, svn not available'
+ test_done
+fi
+
+svnrepo="testsvn"
+
+reinit_svn () {
+ rm -rf "$svnrepo" &&
+ rm -f stream &&
+ svnadmin create "$svnrepo" &&
+ printf "#!/bin/sh" > "$svnrepo"/hooks/pre-revprop-change &&
+ chmod +x "$svnrepo"/hooks/pre-revprop-change &&
+ mkfifo stream
+}
+
+svn_look () {
+ subcommand=$1 &&
+ shift &&
+ svnlook "$subcommand" "$svnrepo" "$@"
+}
+
+try_load () {
+ input=$1 &&
+ maybe_fail=${2:+test_$2} &&
+
+ {
+ $maybe_fail test-svn-fi "$input" >stream &
+ } &&
+ svnadmin load "$svnrepo" <stream &&
+ wait $!
+}
+
+test_expect_success 'normal empty files' '
+ reinit_svn &&
+ cat >expect.tree <<-\EOF &&
+ /
+ foo
+ bar
+ EOF
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 0
+ M 100644 inline foo
+ data 0
+ M 644 inline bar
+ data 0
+
+ EOF
+ try_load input &&
+ svn_look tree >actual.tree &&
+ test_cmp expect.tree actual.tree
+'
+
+# TODO: How to test date? Need to convert from local timestamp
+test_expect_success 'svn:author and svn:log' '
+ reinit_svn &&
+ echo "nothing" >expect.log &&
+ echo "nobody" >expect.author &&
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 7
+ nothing
+ M 100644 inline foo
+ data 0
+
+ EOF
+ try_load input &&
+ svn_look log >actual.log &&
+ svn_look author >actual.author &&
+ test_cmp expect.log actual.log &&
+ test_cmp expect.author actual.author
+'
+
+test_expect_success 'missing author line' '
+ reinit_svn &&
+ cat >expect.tree <<-\EOF &&
+ /
+ foo
+ EOF
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 0
+ M 100644 inline foo
+ data 0
+
+ EOF
+ try_load input &&
+ svn_look tree >actual.tree &&
+ test_cmp expect.tree actual.tree
+'
+
+test_expect_success 'blob marks unsupported' '
+ reinit_svn &&
+ cat >input <<-\EOF &&
+ blob
+ mark :1
+ data 0
+
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :2
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 0
+ M 100644 :1 foo
+
+ EOF
+ try_load input must_fail
+'
+
+test_expect_success 'malformed fast-import stream: filemodify' '
+ reinit_svn &&
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 0
+ M 100644 inline
+
+ EOF
+ try_load input must_fail
+'
+
+test_expect_success 'malformed fast-import stream: author' '
+ reinit_svn &&
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author 2d3%*s&f#k|
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 0
+ M 100644 inline foo
+ data 0
+
+ EOF
+ try_load input must_fail
+'
+
+test_expect_success 'malformed fast-import stream: author 2' '
+ reinit_svn &&
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 0
+ M 100644 inline foo
+ data 0
+
+ EOF
+ try_load input must_fail
+'
+
+test_expect_success 'malformed fast-import stream: data length' '
+ reinit_svn &&
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 0
+ M 100644 inline foo
+ data 14238
+
+ EOF
+ test_must_fail try_load input
+'
+
+test_expect_success 'recursive directory creation' '
+ reinit_svn &&
+ cat >expect.tree <<-\EOF &&
+ /
+ alpha/
+ beta/
+ gamma
+ EOF
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 7
+ nothing
+ M 100644 inline alpha/beta/gamma
+ data 12
+ some content
+
+ EOF
+ try_load input &&
+ svn_look tree >actual.tree &&
+ test_cmp expect.tree actual.tree
+'
+
+test_expect_success 'svn:special and svn:executable' '
+ reinit_svn &&
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 7
+ nothing
+ M 100755 inline foo
+ data 0
+ M 755 inline moo
+ data 0
+ M 120000 inline bar
+ data 0
+
+ EOF
+ try_load input &&
+ svn_look propget svn:executable foo &&
+ svn_look propget svn:executable moo &&
+ svn_look propget svn:special bar
+'
+
+test_expect_success 'replace symlink with normal file' '
+ reinit_svn &&
+ cat >expect.tree <<-\EOF &&
+ /
+ alpha/
+ beta/
+ gamma
+ EOF
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 7
+ nothing
+ M 120000 inline alpha/beta/gamma
+ data 0
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 7
+ nothing
+ M 100644 inline alpha/beta/gamma
+ data 0
+
+ EOF
+ try_load input &&
+ svn_look tree -r1 >actual.tree1 &&
+ svn_look tree -r2 >actual.tree2 &&
+ test_cmp expect.tree actual.tree1 &&
+ test_cmp expect.tree actual.tree2
+'
+
+test_expect_success 'path includes symlink' '
+ reinit_svn &&
+ cat >input <<-\EOF &&
+ reset refs/heads/master
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 7
+ nothing
+ M 120000 inline alpha/beta/gamma
+ data 0
+ commit refs/heads/master
+ mark :1
+ author nobody <nobody@localhost> 1170199019 +0100
+ committer nobody <nobody@localhost> 1170199019 +0100
+ data 7
+ nothing
+ M 100644 inline alpha/beta/gamma/bar
+ data 0
+
+ EOF
+ test_must_fail try_load input
+'
+
+test_done
diff --git a/test-svn-fi.c b/test-svn-fi.c
new file mode 100644
index 0000000..b0605fe
--- /dev/null
+++ b/test-svn-fi.c
@@ -0,0 +1,20 @@
+/*
+ * test-svn-fe: Code to exercise the svn import lib
+ */
+
+#include "git-compat-util.h"
+#include "vcs-svn/svnload.h"
+
+int main(int argc, char *argv[])
+{
+ static const char test_svnfe_usage[] =
+ "test-svn-fe (<dumpfile>";
+ if (argc == 2) {
+ if (svnload_init(argv[1]))
+ return 1;
+ svnload_read();
+ svnload_deinit();
+ return 0;
+ }
+ usage(test_svnfe_usage);
+}
--
1.7.4.rc1.7.g2cf08.dirty
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox