Git development
 help / color / mirror / Atom feed
* [RFC PATCH 00/10] gitweb: Simple file based output caching
From: Jakub Narebski @ 2010-01-23  0:27 UTC (permalink / raw)
  To: git
  Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
	Jakub Narebski
In-Reply-To: <1263432185-21334-10-git-send-email-warthog9@eaglescrag.net>

This 10 patches long patch series is intended as proof of concept
for splitting large 'gitweb: File based caching layer (from git.kernel.org)'
mega-patch by John 'Warthog9' Hawley aka J.H., by starting small and
adding features piece by piece.

This patch is meant as replacement for last two patches:
* [PATCH 8/9] gitweb: Convert output to using indirect file handle
  Message-ID: <1263432185-21334-9-git-send-email-warthog9@eaglescrag.net>
* [PATCH 9/9] gitweb: File based caching layer (from git.kernel.org)
  Message-ID: <1263432185-21334-10-git-send-email-warthog9@eaglescrag.net>

in the long patch series by J.H.
* [PATCH 0/9] Gitweb caching v5
  http://thread.gmane.org/gmane.comp.version-control.git/136913

Note that this patch series is part of 'gitweb/cache-kernel' branch of
http://repo.or.cz/w/git/jnareb-git.git repository (gitweb link), built
on top of modified patches from 'Gitweb caching v2' series (from
'gitweb-ml-v2' branch of http://git.kernel.org/?p=git/warthog9/gitweb.git
repository).  Therefore they might not apply as straight replacements
on top of early parts of 'gitweb-ml-v5' branch.

This is work in progress (showing how I see introducing output caching
to gitweb), it lacks proper documentation (POD for gitweb/cache.pm,
new configuration variables in gitweb/README, perhaps "Gitweb caching"
section in gitweb/README and gitweb/cache.pm mentioned in gitweb/INSTALL),
and commits/patches marked '(WIP)' lacks proper commit message.

Just food for thought...

Table of contents:
~~~~~~~~~~~~~~~~~~
 [RFC PATCH 01/10] gitweb: Print to explicit filehandle (preparing
                   for caching)
 [RFC PATCH 02/10] gitweb: href(..., -path_info => 0|1)
 [RFC PATCH 03/10] gitweb/cache.pm - Very simple file based caching
 [RFC PATCH 04/10] gitweb/cache.pm - Stat-based cache expiration
 [RFC PATCH 05/10] gitweb: Use Cache::Cache compatibile (get, set)
                   output caching (WIP)
 [RFC PATCH 06/10] gitweb/cache.pm - Adaptive cache expiration time (WIP)
 [RFC PATCH 07/10] gitweb: Use CHI compatibile (compute method) caching (WIP)
 [RFC PATCH 08/10] gitweb/cache.pm - Use locking to avoid 'stampeding herd'
                   problem (WIP)
 [RFC PATCH 09/10] gitweb/cache.pm - Serve stale data when waiting for
                   filling cache (WIP)
 [RFC PATCH 10/10] gitweb: Show appropriate "Generating..." page when
                   regenerating cache (WIP)


Diffstat:
~~~~~~~~~

 gitweb/cache.pm                        |  566 ++++++++++
 gitweb/gitweb.perl                     | 1923 +++++++++++++++++---------------
 t/gitweb-lib.sh                        |    2 +
 t/t9500-gitweb-standalone-no-errors.sh |   13 +
 t/t9503-gitweb-caching.sh              |   32 +
 t/t9503/test_cache_interface.pl        |  195 ++++
 t/test-lib.sh                          |    3 +
 7 files changed, 1836 insertions(+), 898 deletions(-)
 create mode 100644 gitweb/cache.pm
 create mode 100755 t/t9503-gitweb-caching.sh
 create mode 100755 t/t9503/test_cache_interface.pl

^ permalink raw reply

* [RFC PATCH 03/10] gitweb/cache.pm - Very simple file based caching
From: Jakub Narebski @ 2010-01-23  0:27 UTC (permalink / raw)
  To: git
  Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
	Jakub Narebski
In-Reply-To: <cover.1264198194.git.jnareb@gmail.com>

This is first step towards implementing file based output (response)
caching layer that is used on such large sites as kernel.org.

This patch introduces GitwebCaching::SimpleFileCache package, which
follows Cache::Cache / CHI interface, although do not implement it
fully.  The intent of following established convention is to be able
in the future to replace our simple file based cache e.g. by one using
memcached.

Like in original patch by John 'Warthog9' Hawley (J.H.) (the one this
commit intends to be incremental step to), the data is stored in the
case as-is, without adding metadata (like expiration date), and
without serialization (which means only scalar data).

To be implemented (from original patch by J.H.):
* cache expiration (based on file stats, current time and global
  expiration time); currently elements in cache do not expire
* actually using this cache in gitweb, except error pages
* adaptive cache expiration, based on average system load
* optional locking interface, where only one process can update cache
  (using flock)
* server-side progress indicator when waiting for filling cache,
  which in turn requires separating situations (like snapshots and
  other non-HTML responses) where we should not show 'please wait'
  message

Possible extensions (beyond what was in original patch):
* (optionally) show information about cache utilization
* AJAX (JavaScript-based) progress indicator
* JavaScript code to update relative dates in cached output
* make cache size-aware (try to not exceed specified maximum size)
* utilize X-Sendfile header (or equivalent) to show cached data
  (optional, as it makes sense only if web server supports sendfile
  feature and have it enabled)
* variable expiration feature from CHI, allowing items to expire a bit
  earlier than the stated expiration time to prevent cache miss
  stampedes (although locking, if available, should take care of
  this).

The code of GitwebCaching::SimpleFileCache package in gitweb/cache.pm
was heavily based on file-based cache in Cache::Cache package, i.e.
on Cache::FileCache, Cache::FileBackend and Cache::BaseCache
(including implementing atomic write, something that original patch
lacks).

This patch does not yet enable output caching in gitweb (it doesn't
have all required features yet); on the other hand it includes tests,
currently testing only cache Perl API.

Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Large parts of this code are based _heavily_ on Cache::FileCache
implementation (including Cache::FileBackend and Cache::BaseCache)
from Cache::Cache distribution (which is dual licensed using
(Perl) Artistic License and GNU General Public License, like Perl
itself).  In the final version of code it should probably be
cleaned up.

Also although it implements 'compute' interface from CHI (Unified
cache interface), actual CHI code was not used even as reference.
(Nevertheless gitweb/cache.pm is meant to use minimal dependencies,
like gitweb itself, so it won't use Moose / Mouse for OO).

 gitweb/cache.pm                 |  317 +++++++++++++++++++++++++++++++++++++++
 t/t9503-gitweb-caching.sh       |   32 ++++
 t/t9503/test_cache_interface.pl |   77 ++++++++++
 t/test-lib.sh                   |    3 +
 4 files changed, 429 insertions(+), 0 deletions(-)
 create mode 100644 gitweb/cache.pm
 create mode 100755 t/t9503-gitweb-caching.sh
 create mode 100755 t/t9503/test_cache_interface.pl

diff --git a/gitweb/cache.pm b/gitweb/cache.pm
new file mode 100644
index 0000000..ea544b0
--- /dev/null
+++ b/gitweb/cache.pm
@@ -0,0 +1,317 @@
+# gitweb - simple web interface to track changes in git repositories
+#
+# (C) 2006, John 'Warthog9' Hawley <warthog19@eaglescrag.net>
+#
+# This program is licensed under the GPLv2
+
+#
+# Gitweb caching engine
+#
+
+{
+# Minimalistic cache that stores data in the filesystem, without serialization
+# and currently without any kind of cache expiration (all keys last forever till
+# they got explicitely removed)
+#
+# It follows Cache::Cache and CHI interface (but does not implement it fully)
+
+package GitwebCache::SimpleFileCache;
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path);  # requires version >= 2.0
+use File::Spec;
+use File::Temp;
+use Digest::MD5 qw(md5_hex);
+
+# by default, the cache nests all entries on the filesystem two
+# directories deep
+
+our $DEFAULT_CACHE_DEPTH = 2;
+
+# by default, the root of the cache is located in 'cache'.
+
+our $DEFAULT_CACHE_ROOT = "cache";
+
+# ......................................................................
+# constructor
+
+# The options are set by passing in a reference to a hash containing
+# any of the following keys:
+#  * 'namespace'
+#    The namespace associated with this cache.  This allows easy separation of
+#    multiple, distinct caches without worrying about key collision.  Defaults
+#    to '' (which does not allow for simple implementation of clear() method).
+#  * 'cache_root'
+#    The location in the filesystem that will hold the root of the cache.
+#    Defaults to 'cache', relative to gitweb.cgi directory.
+#  * 'cache_depth'
+#    The number of subdirectories deep to cache object item.  This should be
+#    large enough that no cache directory has more than a few hundred objects.
+#    Defaults to 2 unless explicitly set.
+sub new {
+	my ($proto, $p_options_hash_ref) = @_;
+
+	my $class = ref($proto) || $proto;
+	my $self  = {};
+	$self = bless($self, $class);
+
+	my ($root, $depth, $ns);
+	if (defined $p_options_hash_ref) {
+		$root  = $p_options_hash_ref->{'cache_root'};
+		$depth = $p_options_hash_ref->{'cache_depth'};
+		$ns    = $p_options_hash_ref->{'namespace'};
+	}
+	$root  = $DEFAULT_CACHE_ROOT  unless defined($root);
+	$depth = $DEFAULT_CACHE_DEPTH unless defined($depth);
+	$ns    = '' unless defined($ns);
+
+	$self->set_root($root);
+	$self->set_depth($depth);
+	$self->set_namespace($ns);
+
+	return $self;
+}
+
+# ......................................................................
+# accessors
+
+sub get_depth {
+	my ($self) = @_;
+
+	return $self->{'_Depth'};
+}
+
+sub set_depth {
+	my ($self, $depth) = @_;
+
+	$self->{'_Depth'} = $depth;
+}
+
+sub get_root {
+	my ($self) = @_;
+
+	return $self->{'_Root'};
+}
+
+
+sub set_root {
+	my ($self, $root) = @_;
+
+	$self->{'_Root'} = $root;
+}
+
+sub get_namespace {
+	my ($self) = @_;
+
+	return $self->{'_Namespace'};
+}
+
+
+sub set_namespace {
+	my ($self, $namespace) = @_;
+
+	$self->{'_Namespace'} = $namespace;
+}
+
+# ----------------------------------------------------------------------
+# (private) utility functions and methods
+
+# Take an human readable key, and create a unique (hashed) key from it
+sub _Build_Hashed_Key {
+	my ($p_key) = @_;
+
+	return md5_hex($p_key);
+}
+
+# Take an human readable key, and return file path
+sub _path_to_key {
+	my ($self, $p_namespace, $p_key) = @_;
+
+	return $self->_path_to_hashed_key($p_namespace,
+	                                    _Build_Hashed_Key($p_key));
+}
+
+# Take hashed key, and return file path
+sub _path_to_hashed_key {
+	my ($self, $p_namespace, $p_hashed_key) = @_;
+
+	return File::Spec->catfile($self->get_root(), $p_namespace,
+	                           _Split_Word($p_hashed_key, $self->get_depth()));
+}
+
+# Split word into N components, where each component but last is two-letter word
+# e.g. _Split_Word("06b90e786e304a18fdfbd7c7bcc41a6b", 2) == qw(06 b90e786e304a18fdfbd7c7bcc41a6b);
+#      _Split_Word("06b90e786e304a18fdfbd7c7bcc41a6b", 3) == qw(06 b9 0e786e304a18fdfbd7c7bcc41a6b);
+sub _Split_Word {
+	my ($p_word, $p_depth) = @_;
+
+	$p_depth--; # now it is number of leading 2-letter components
+	return unpack("(a2)$p_depth a*", $p_word);
+}
+
+sub _Read_File {
+	my ($p_path) = @_;
+
+	-e $p_path
+		or return undef;
+
+	open(my $fh, '<', $p_path)
+		or return undef;
+
+	local $/ = undef;
+	my $data = <$fh>;
+
+	close($fh);
+
+	return $data;
+}
+
+# write a file atomically, assuming that path leading to file exists
+sub _Write_File {
+	my ($p_path, $p_data) = @_;
+
+	my ($volume, $directory, $filename) = File::Spec->splitpath($p_path);
+	if (defined $directory and defined $volume) {
+		$directory = File::Spec->catpath($volume, $directory, '');
+	}
+
+	my $temp = File::Temp->new(DIR => $directory,
+	                          TEMPLATE => "${filename}_XXXXX",
+	                          SUFFIX => '.tmp');
+	binmode($temp);
+	print {$temp} $p_data;
+	close($temp);
+
+	rename($temp, $p_path);
+}
+
+# ensures that directory leading to path exists, or dies
+sub _Make_Path {
+	my ($p_path, $p_dir) = @_;
+
+	my ($volume, $directory, $filename) = File::Spec->splitpath($p_path);
+	if (defined $directory and defined $volume) {
+		$directory = File::Spec->catpath($volume, $directory, "");
+	}
+
+	return
+		unless (defined $directory and not -d $directory);
+
+	my $numdirs = make_path($directory,
+	                        { mode => 0777, error => \my $mkdirerr });
+	if (@$mkdirerr) {
+		my $mkdirerrmsg = "";
+		for my $diag (@$mkdirerr) {
+			my ($file, $message) = %$diag;
+			if ($file eq '' ){
+				$mkdirerrmsg .= "general error: $message\n";
+			} else {
+				$mkdirerrmsg .= "problem unlinking $file: $message\n";
+			}
+		}
+		#die_error(500, "Could not create cache directory | $mkdirerrmsg");
+	}
+}
+
+sub _Remove_File {
+	my ($p_path) = @_;
+
+	if (-f $p_path) {
+		unlink($p_path);
+	}
+}
+
+# _read_data and _write_data methods do deserialization/serialization
+# in original implementation in Cache::Cache distribution
+
+sub _read_data {
+	my ($self, $p_path) = @_;
+
+	return _Read_File($p_path);
+}
+
+sub _write_data {
+	my ($self, $p_path, $p_data) = @_;
+
+	_Make_Path($p_path);
+	_Write_File($p_path, $p_data);
+}
+
+# ----------------------------------------------------------------------
+# worker methods (explicit namespace)
+
+sub restore {
+	my ($self, $p_namespace, $p_key) = @_;
+
+	return $self->_read_data($self->_path_to_key($p_namespace, $p_key));
+}
+
+sub store {
+	my ($self, $p_namespace, $p_key, $p_data) = @_;
+
+	$self->_write_data($self->_path_to_key($p_namespace, $p_key),
+	                   $p_data);
+}
+
+sub delete_key {
+	my ($self, $p_namespace, $p_key) = @_;
+
+	_Remove_File($self->_path_to_key($p_namespace, $p_key));
+}
+
+sub get_size {
+	my ($self, $p_namespace, $p_key) = @_;
+
+	my $path = $self->_path_to_key($p_namespace, $p_key);
+	if (-e $path) {
+		return -s $path;
+	}
+	return 0;
+}
+
+# ......................................................................
+# interface methods
+
+# Removing and expiring
+
+sub remove {
+	my ($self, $p_key) = @_;
+
+	$self->delete_key($self->get_namespace(), $p_key);
+}
+
+# Getting and setting
+
+sub set {
+	my ($self, $p_key, $p_data) = @_;
+
+	$self->store($self->get_namespace(), $p_key, $p_data);
+}
+
+sub get {
+	my ($self, $p_key) = @_;
+
+	my $data = $self->restore($self->get_namespace(), $p_key)
+		or return undef;
+
+	return $data;
+}
+
+sub compute {
+	my ($self, $p_key, $p_coderef) = @_;
+
+	my $data = $self->get($p_key);
+	if (!defined $data) {
+		$data = $p_coderef->($self, $p_key);
+		$self->set($p_key, $data);
+	}
+
+	return $data;
+}
+
+1;
+} # end of package GitwebCache::SimpleFileCache;
+
+1;
diff --git a/t/t9503-gitweb-caching.sh b/t/t9503-gitweb-caching.sh
new file mode 100755
index 0000000..768080c
--- /dev/null
+++ b/t/t9503-gitweb-caching.sh
@@ -0,0 +1,32 @@
+#!/bin/sh
+#
+# Copyright (c) 2010 Jakub Narebski
+#
+
+test_description='caching interface to be used in gitweb'
+#test_description='caching interface used in gitweb, gitweb caching
+#
+#This test checks cache (interface) used in gitweb caching, caching
+#infrastructure and gitweb response (output) caching (the last by
+#running gitweb as CGI script from commandline).'
+
+# for now we are running only cache interface tests
+. ./test-lib.sh
+
+# this test is present in gitweb-lib.sh
+if ! test_have_prereq PERL; then
+	say 'perl not available, skipping test'
+	test_done
+fi
+
+"$PERL_PATH" -MTest::More -e 0 >/dev/null 2>&1 || {
+	say 'perl module Test::More unavailable, skipping test'
+	test_done
+}
+
+# ----------------------------------------------------------------------
+
+test_external 'GitwebCache::* Perl API (in gitweb/cache.pm)' \
+	"$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_interface.pl
+
+test_done
diff --git a/t/t9503/test_cache_interface.pl b/t/t9503/test_cache_interface.pl
new file mode 100755
index 0000000..0b6628b
--- /dev/null
+++ b/t/t9503/test_cache_interface.pl
@@ -0,0 +1,77 @@
+#!/usr/bin/perl
+use lib (split(/:/, $ENV{GITPERLLIB}));
+
+use warnings;
+use strict;
+
+use Test::More;
+
+# test source version; there is no installation target for gitweb
+my $cache_pm = "$ENV{TEST_DIRECTORY}/../gitweb/cache.pm";
+
+unless (-f "$cache_pm") {
+	plan skip_all => "gitweb/cache.pm not found";
+}
+
+# it is currently not a proper Perl module, so we use 'do FILE'
+#ok(eval { do "$cache_pm"; 1 or die $!; }, "loading gitweb/cache.pm");
+my $return = do "$cache_pm";
+ok(!$@,              "parse gitweb/cache.pm");
+ok(defined $return,  "do    gitweb/cache.pm");
+ok($return,          "run   gitweb/cache.pm");
+# instead of: BEGIN { use_ok('GitwebCache::SimpleFileCache') }
+
+# Test creating a cache
+#
+my $cache = new_ok('GitwebCache::SimpleFileCache',
+	[ { 'cache_root' => 'cache', 'cache_depth' => 2 } ]);
+
+# Test that default values are defined
+#
+ok(defined $GitwebCache::SimpleFileCache::DEFAULT_CACHE_ROOT,
+	'$DEFAULT_CACHE_ROOT defined');
+ok(defined $GitwebCache::SimpleFileCache::DEFAULT_CACHE_DEPTH,
+	'$DEFAULT_CACHE_DEPTH defined');
+
+# Test accessors and default values for cache
+#
+SKIP: {
+	skip 'default values not defined', 3
+		unless ($GitwebCache::SimpleFileCache::DEFAULT_CACHE_ROOT &&
+		        $GitwebCache::SimpleFileCache::DEFAULT_CACHE_DEPTH);
+
+	is($cache->get_namespace(), '', "default namespace is ''");
+	is($cache->get_root(), $GitwebCache::SimpleFileCache::DEFAULT_CACHE_ROOT,
+		"default cache root is '$GitwebCache::SimpleFileCache::DEFAULT_CACHE_ROOT'");
+	cmp_ok($cache->get_depth(), '==', $GitwebCache::SimpleFileCache::DEFAULT_CACHE_DEPTH,
+		"default cache depth is $GitwebCache::SimpleFileCache::DEFAULT_CACHE_DEPTH");
+}
+
+# Test the getting, setting, and removal of a cached value
+# (Cache::Cache interface)
+#
+my $key = 'Test Key';
+my $value = 'Test Value';
+can_ok($cache, qw(get set remove));
+#ok(!defined($cache->get($key)),        'get before set')
+#	or diag("get returned '", $cache->get($key), "' for $key");
+$cache->set($key, $value);
+is($cache->get($key), $value,          'get after set, returns cached value');
+$cache->remove($key);
+ok(!defined($cache->get($key)),        'get after remove, is undefined');
+
+# Test the getting and setting of a cached value
+# (CHI interface)
+#
+my $call_count = 0;
+sub get_value {
+	$call_count++;
+	return $value;
+}
+can_ok($cache, qw(compute));
+is($cache->compute($key, \&get_value), $value, 'compute 1st time (set)');
+is($cache->compute($key, \&get_value), $value, 'compute 2nd time (get)');
+is($cache->compute($key, \&get_value), $value, 'compute 3rd time (get)');
+cmp_ok($call_count, '==', 1, 'get_value() is called once');
+
+done_testing();
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 142f36f..9282d9e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -429,6 +429,9 @@ test_external () {
 		# Announce the script to reduce confusion about the
 		# test output that follows.
 		say_color "" " run $test_count: $descr ($*)"
+		# Export TEST_DIRECTORY and TRASH_DIRECTORY
+		# to be able to use them in script
+		export TEST_DIRECTORY TRASH_DIRECTORY
 		# Run command; redirect its stderr to &4 as in
 		# test_run_, but keep its stdout on our stdout even in
 		# non-verbose mode.
-- 
1.6.6.1

^ permalink raw reply related

* [RFC PATCH 05/10] gitweb: Use Cache::Cache compatibile (get, set) output caching (WIP)
From: Jakub Narebski @ 2010-01-23  0:27 UTC (permalink / raw)
  To: git
  Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley,
	Jakub Narebski
In-Reply-To: <cover.1264198194.git.jnareb@gmail.com>

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This patch, as you can see, lack proper commit message: it is work in
progress.

As you can see cache_fetch() subroutine is much, much simpler that the
one in original patch by J.H.:
  [PATCH 9/9] gitweb: File based caching layer (from git.kernel.org)
  Message-ID: <1263432185-21334-10-git-send-email-warthog9@eaglescrag.net>
  http://permalink.gmane.org/gmane.comp.version-control.git/136917

As you can see I have introduced $cache_pm variable, to be able to
test caching in t/t9500-gitweb-standalone-no-errors.sh, but also to be
able to install cache.pm in some other place than along gitweb.cgi.
There would be no such problems if we used 'require GitwebCache' or
somesuch, in place of 'do "cache.pm"' like in original patch by J.H.
But at leats for now I have decided to follow J.H. in this issue.

Instead of using binary (sic!) valued $cache_enable as in J.H. patch,
I use set of two variables: $cache (to be able to select what caching
engine to use, and what features should be enabled), and
$caching_enabled to actually enable/disable cache.

When caching is enabled gitweb do not output timing info (time to
generate page), as it would contain incorrect information if the page
was retrieved from cache; in this place we could put cachetime info
from the original patch by J.H., which means writing

  Cache Last Updated: ". gmtime( time )

I have forgot about adding this feature...


ATTENTION !!!: I have run both tests (t9500 to check for errors in
gitweb.perl, and t9503 to test the API), but I haven't actually tested
that *gitweb itself* behaves correctly.

 gitweb/cache.pm                        |   36 ++++++++++++++++++++++++++++
 gitweb/gitweb.perl                     |   40 +++++++++++++++++++++++++++++--
 t/gitweb-lib.sh                        |    2 +
 t/t9500-gitweb-standalone-no-errors.sh |   13 ++++++++++
 4 files changed, 88 insertions(+), 3 deletions(-)

diff --git a/gitweb/cache.pm b/gitweb/cache.pm
index 12a7a78..3a33158 100644
--- a/gitweb/cache.pm
+++ b/gitweb/cache.pm
@@ -359,4 +359,41 @@ sub compute {
 1;
 } # end of package GitwebCache::SimpleFileCache;
 
+# human readable key identifying gitweb output
+sub gitweb_output_key {
+	return href(-replay => 1, -full => 1, -path_info => 0);
+}
+
+sub cache_fetch {
+	my ($cache, $action) = @_;
+
+	my $key = gitweb_output_key();
+	my $data = $cache->get($key);
+
+	if (defined $data) {
+		# print cached data
+		binmode STDOUT, ':raw';
+		print STDOUT $data;
+
+	} else {
+		# calculate data and regenerate data
+		open my $data_fh, '>', \$data
+			or die "Can't open memory file: $!";
+		# matches "binmode STDOUT, ':uft8'" at beginning
+		binmode $data_fh, ':utf8';
+
+		$out = $data_fh || \*STDOUT;
+		$actions{$action}->();
+
+		if (defined $data) {
+			$cache->set($key, $data);
+			binmode STDOUT, ':raw';
+			local $/ = undef;
+			print STDOUT $data;
+		}
+
+		close $data_fh;
+	}
+}
+
 1;
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index cd5073c..0394dc8 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -234,6 +234,22 @@ our $gitlinkurl_base = ("++GITWEB_BASE_URL++" =~ m!^(git://.*)$!) ? $1 : '';
 # Leave it undefined (or set to 'undef') to turn off load checking.
 our $maxload = 300;
 
+# This enables/disables the caching layer in gitweb.  Currently supported
+# is only output (response) caching, similar to the one used on git.kernel.org.
+our $caching_enabled = 0;
+# Set to _initialized_ instance of cache interface implementing (at least)
+# get($key) and set($key, $data) methods (Cache::Cache and CHI interfaces).
+# If unset, GitwebCache::SimpleFileCache would be used, which is 'dumb'
+# (but fast) file based caching layer, currently without any support for
+# cache size limiting.  It is therefore recommended that the cache directory
+# be periodically completely deleted; this operation is safe to perform.
+# Suggested mechanism:
+# mv $cachedir $cachedir.flush && mkdir $cachedir && rm -rf $cachedir.flush
+our $cache;
+# Locations of 'cache.pm' file; if it is relative path, it is relative to
+# the directory gitweb is run from
+our $cache_pm = 'cache.pm';
+
 # You define site-wide feature defaults here; override them with
 # $GITWEB_CONFIG as necessary.
 our %feature = (
@@ -998,7 +1014,21 @@ if ($action !~ m/^(?:opml|project_list|project_index)$/ &&
     !$project) {
 	die_error(400, "Project needed");
 }
-$actions{$action}->();
+
+if ($caching_enabled) {
+	do $cache_pm;
+	die $@ if $@;
+
+	$cache ||= GitwebCache::SimpleFileCache->new({
+		'cache_root'  => '/tmp/cache',
+		'cache_depth' => 2,
+		'expires_in'  => 20, # in seconds
+	});
+	cache_fetch($cache, $action);
+} else {
+	$actions{$action}->();
+}
+
 exit;
 
 ## ======================================================================
@@ -3207,7 +3237,9 @@ sub git_header_html {
 	# 'application/xhtml+xml', otherwise send it as plain old 'text/html'.
 	# we have to do this because MSIE sometimes globs '*/*', pretending to
 	# support xhtml+xml but choking when it gets what it asked for.
-	if (defined $cgi->http('HTTP_ACCEPT') &&
+	# Disable content-type negotiation when caching (use mimetype good for all).
+	if (!$caching_enabled &&
+	    defined $cgi->http('HTTP_ACCEPT') &&
 	    $cgi->http('HTTP_ACCEPT') =~ m/(,|;|\s|^)application\/xhtml\+xml(,|;|\s|$)/ &&
 	    $cgi->Accept('application/xhtml+xml') != 0) {
 		$content_type = 'application/xhtml+xml';
@@ -3380,7 +3412,9 @@ sub git_footer_html {
 	}
 	print {$out} "</div>\n"; # class="page_footer"
 
-	if (defined $t0 && gitweb_check_feature('timed')) {
+	# timing info doesn't make much sense with output (response) caching
+	if (!$caching_enabled &&
+	    defined $t0 && gitweb_check_feature('timed')) {
 		print {$out} "<div id=\"generating_info\">\n";
 		print {$out} 'This page took '.
 		             '<span id="generating_time" class="time_span">'.
diff --git a/t/gitweb-lib.sh b/t/gitweb-lib.sh
index d9ffc90..d041083 100755
--- a/t/gitweb-lib.sh
+++ b/t/gitweb-lib.sh
@@ -27,6 +27,8 @@ our \$export_ok = '';
 our \$strict_export = '';
 our \$git_versions_must_match = 0;
 
+our \$cache_pm = '$TEST_DIRECTORY/../gitweb/cache.pm';
+
 EOF
 
 	cat >.git/description <<EOF
diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 2fc7fdb..0f93962 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -639,4 +639,17 @@ test_expect_success \
 	 gitweb_run "p=.git;a=summary"'
 test_debug 'cat gitweb.log'
 
+# ----------------------------------------------------------------------
+# caching
+
+cat >>gitweb_config.perl <<\EOF
+
+$caching_enabled = 1;
+EOF
+test_expect_success \
+	'caching enabled' \
+	'gitweb_run "p=.git;a=summary"'
+test_debug 'cat gitweb.log'
+
+
 test_done
-- 
1.6.6.1

^ permalink raw reply related

* Re: [RFC PATCH 01/10] gitweb: Print to explicit filehandle (preparing for caching)
From: Jakub Narebski @ 2010-01-23  0:48 UTC (permalink / raw)
  To: git; +Cc: John 'Warthog9' Hawley, John 'Warthog9' Hawley
In-Reply-To: <0dd15cb3f18e2a26fc834fd3b071e6d3ecc00557.1264198194.git.jnareb@gmail.com>

As you can (or rather can't ;-)) see this patch didn't made it into list,
because with 119,993 characters in format-patch patch it probably exceeds
a little bit exceeding 100,000 characters message size limit on VGER.

The problem is that it doesn't make sense to send partial patch... well,
perhaps reindent and breaking of exceedingly long lines should be split
into separate patch...

For now you can view the patch via gitweb
  http://repo.or.cz/w/git/jnareb-git.git/commitdiff/0dd15cb3f18e2a26fc834fd3b071e6d3ecc00557
and apply it from
  http://repo.or.cz/w/git/jnareb-git.git/patch/0dd15cb3f18e2a26fc834fd3b071e6d3ecc00557

The comment for this message (for this patch) can be seen below.

-- >8 --
On Sat, 23 Jan 2010, Jakub Narebski wrote:

> This means replacing
> 
>   print <something>;
> by
>   print {$out} <something>;
> 
> and
> 
>   binmode STDOUT, <layer>;
> by
>   binmode $out, <layer>;
> 
> where $out is global variable set to \*STDOUT at the beginning of
> gitweb, but after reading gitweb config.  This way it would be simple
> to e.g. tie output filehandle or use PerlIO layers to simultaneously
> write to standard output and to some specified file (like "tee"
> utility does), or redirect output to a scalar, or a file.
> 
> die_error (re)sets $out to \*STDOUT; we would (probably) want to treat
> errors in a special way, and do not cache them.
> 
> 
> The only other differences are reindent of continued lines (if
> needed), and sometimes word-wrapping lines which this change made too
> long.
> 
> Signed-off-by: Jakub Narebski <jnareb@gmail.com>
> ---
> This patch is meant as (straight) replacement for the following patch
> by J.H. (John 'Warthog9' Hawley):
> * [PATCH 8/9] gitweb: Convert output to using indirect file handle
>   Message-ID: <1263432185-21334-9-git-send-email-warthog9@eaglescrag.net>
>   http://permalink.gmane.org/gmane.comp.version-control.git/136915
> 
> Actually this patch precedes (was written before) the patch by J.H.
> 
> This patch was written _before_ comment from Junio that it would be
> better to simply use
>   print $out <something>;
> and do not try to be too clever.
> 
> 
> Differences from patch by J.H.:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> * Only one output handle, instead of having one output handle for text,
>   (':utf8'), and another output handler for binary files (:raw).  
> 
>   I do assume that I can write to handler with appropriate layer:
>   :utf8/:raw, and then I can simply read from cache file in :raw
>   binmode, as the data is already converted correctly.
> 
> * Shorter name for output handle: $out instead of $output_handler
> 
> * Set output handler to \*STDOUT (default value in declaration of this
>   variable, and also in die_error() subroutine), instead of *STDOUT.
>   This way $out is indirect filehandle, instead of using direct
>   filehandle which is _global_ to current package (see perlopentut(1)).
> 
> * Reindent continuation lines, i.e.
> 
>     print <line1> .
>           <line2>;
> 
>   got replaced (reindented) with
> 
>     print {$out} <line1> .
>                  <line2>;
> 
>   In some places lines were broken into two, when after indent the
>   line got too long.
> 
> * Slightly different replacement for printf

-- 
Jakub Narebski
Poland

^ permalink raw reply

* What's cooking in git.git (Jan 2010, #07; Fri, 22)
From: Junio C Hamano @ 2010-01-23  3:28 UTC (permalink / raw)
  To: git

Here are the topics that have been cooking.  Commits prefixed with '-' are
only in 'pu' while commits prefixed with '+' are in 'next'.  The ones
marked with '.' do not appear in any of the integration branches, but I am
still holding onto them.

--------------------------------------------------
[Graduated to "master"]

* jc/conflict-marker-size (2010-01-16) 8 commits
  (merged to 'next' on 2010-01-18 at f1f6023)
 + rerere: honor conflict-marker-size attribute
 + rerere: prepare for customizable conflict marker length
 + conflict-marker-size: new attribute
 + rerere: use ll_merge() instead of using xdl_merge()
 + merge-tree: use ll_merge() not xdl_merge()
 + xdl_merge(): allow passing down marker_size in xmparam_t
 + xdl_merge(): introduce xmparam_t for merge specific parameters
 + git_attr(): fix function signature

* ag/maint-apply-too-large-p (2010-01-17) 1 commit
  (merged to 'next' on 2010-01-18 at 8bd106a)
 + builtin-apply.c: Skip filenames without enough components

* ag/patch-header-verify (2010-01-18) 1 commit
  (merged to 'next' on 2010-01-18 at 2cd0ddc)
 + builtin-apply.c: fix the --- and +++ header filename consistency check

* bw/cvsimport (2010-01-19) 3 commits
  (merged to 'next' on 2010-01-19 at 63f4c8d)
 + cvsimport: standarize system() calls to external git tools
 + cvsimport: standarize open() calls to external git tools
 + cvsimport: modernize callouts to git subcommands

* jc/checkout-merge-base (2010-01-19) 1 commit
  (merged to 'next' on 2010-01-19 at 3665110)
 + Fix "checkout A..." synonym for "checkout A...HEAD" on Windows

* jc/maint-refresh-index-is-optional-for-status (2010-01-19) 1 commit
 + status: don't require the repository to be writable

* nd/status-partial-refresh (2010-01-17) 2 commits
  (merged to 'next' on 2010-01-19 at 64f0c0b)
 + rm: only refresh entries that we may touch
  (merged to 'next' on 2010-01-16 at f77bc8f)
 + status: only touch path we may need to check

* ap/merge-backend-opts (2008-07-18) 7 commits
  (merged to 'next' on 2010-01-18 at cb1f6b7)
 + Document that merge strategies can now take their own options
 + Extend merge-subtree tests to test -Xsubtree=dir.
 + Make "subtree" part more orthogonal to the rest of merge-recursive.
 + pull: Fix parsing of -X<option>
 + Teach git-pull to pass -X<option> to git-merge
 + git merge -X<option>
 + git-merge-file --ours, --theirs

* jc/maint-limit-note-output (2010-01-21) 2 commits
  (merged to 'next' on 2010-01-21 at bcb80b9)
 + Fix "log --oneline" not to show notes
  (merged to 'next' on 2010-01-20 at 526bfcc)
 + Fix "log" family not to be too agressive about showing notes

* nd/ls-files-sparse-fix (2010-01-20) 1 commit
  (merged to 'next' on 2010-01-20 at 0f61dbc)
 + Fix memory corruption when .gitignore does not end by \n

* il/branch-set-upstream (2010-01-18) 2 commits
  (merged to 'next' on 2010-01-18 at b9b0993)
 + branch: warn and refuse to set a branch as a tracking branch of itself.
 + Add branch --set-upstream

* il/remote-updates (2010-01-18) 1 commit
  (merged to 'next' on 2010-01-18 at 5c3e805)
 + Add git remote set-url

* il/rev-glob (2010-01-22) 3 commits
  (merged to 'next' on 2010-01-21 at 453a21c)
 + Documentation: improve description of --glob=pattern and friends
  (merged to 'next' on 2010-01-20 at 928ba0a)
 + rev-parse --branches/--tags/--remotes=pattern
 + rev-parse --glob

This is a re-rolled "--namespace=" one.

* jl/submodule-diff (2010-01-18) 4 commits
  (merged to 'next' on 2010-01-20 at 95cb513)
 + Performance optimization for detection of modified submodules
  (merged to 'next' on 2010-01-17 at 525075b)
 + git status: Show uncommitted submodule changes too when enabled
  (merged to 'next' on 2010-01-16 at 0a99e3c)
 + Teach diff that modified submodule directory is dirty
 + Show submodules as modified when they contain a dirty work tree

* js/refer-upstream (2010-01-19) 3 commits
  (merged to 'next' on 2010-01-20 at 5a5547a)
 + Teach @{upstream} syntax to strbuf_branchanme()
 + t1506: more test for @{upstream} syntax
 + Introduce <branch>@{upstream} notation

Updated to teach the new syntax to commands like "checkout" and "merge"
that want to behave better when they know what were given was a branch
name, not a random SHA-1.

* jc/branch-d (2009-12-29) 1 commit
  (merged to 'next' on 2010-01-10 at 61a14b7)
 + branch -d: base the "already-merged" safety on the branch it merges with

--------------------------------------------------
[Will merge to 'master' after a bit more cooking in 'next']

* jc/fix-tree-walk (2009-09-14) 7 commits
  (merged to 'next' on 2010-01-13 at 1c01b87)
 + read-tree --debug-unpack
 + unpack-trees.c: look ahead in the index
 + unpack-trees.c: prepare for looking ahead in the index
 + Aggressive three-way merge: fix D/F case
 + traverse_trees(): handle D/F conflict case sanely
 + more D/F conflict tests
 + tests: move convenience regexp to match object names to test-lib.sh

Resurrected from "Ejected" category.  This is fix for a tricky codepath
and testing and improving before it hits 'master' is greatly appreciated.
(I have been using this in my private build for some time).

--------------------------------------------------
[Cooking]

* jh/notes (2010-01-17) 20 commits
 . builtin-gc: Teach the new --notes option to garbage-collect notes
 . Notes API: gc_notes(): Prune notes that belong to non-existing objects
 . t3305: Verify that removing notes triggers automatic fanout consolidation
 . builtin-notes: Teach -d option for deleting existing notes
 . Teach builtin-notes to remove empty notes
 . Teach notes code to properly preserve non-notes in the notes tree
 . t3305: Verify that adding many notes with git-notes triggers increased fanout
 . t3301: Verify successful annotation of non-commits
 . Builtin-ify git-notes
 . Refactor notes concatenation into a flexible interface for combining notes
 . Notes API: Allow multiple concurrent notes trees with new struct notes_tree
 . Notes API: write_notes_tree(): Store the notes tree in the database
 . Notes API: for_each_note(): Traverse the entire notes tree with a callback
 . Notes API: get_note(): Return the note annotating the given object
 . Notes API: remove_note(): Remove note objects from the notes tree structure
 . Notes API: add_note(): Add note objects to the internal notes tree structure
 . Notes API: init_notes(): Initialize the notes tree from the given notes ref
 . Add tests for checking correct handling of $GIT_NOTES_REF and core.notesRef
 . Notes API: get_commit_notes() -> format_note() + remove the commit restriction
 . Minor non-functional fixes to notes.c

Tentatively ejected, as its tests conflict with tests in a higher priority
fix.

* jh/gitweb-cached (2010-01-13) 9 commits
 - gitweb: File based caching layer (from git.kernel.org)
 - gitweb: Convert output to using indirect file handle
 - gitweb: cleanup error message produced by undefined $site_header
 - gitweb: add a get function to compliment print_sort_th
 - gitweb: add a get function to compliment print_local_time
 - gitweb: Makefile improvements
 - gitweb: Add option to force version match
 - gitweb: change die_error to take "extra" argument for extended die information
 - gitweb: Load checking

Replaced with a re-roll.  Update to t9500 is probably needed.

* jc/grep-author-all-match-implicit (2010-01-17) 1 commit
 - "log --author=me --grep=it" should find intersection, not union

^ permalink raw reply

* Re: [PATCH v2 1/5] reset: add option "--keep" to "git reset"
From: Christian Couder @ 2010-01-23  4:18 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Linus Torvalds, Johannes Schindelin, Stephan Beyer,
	Daniel Barkalow, Jakub Narebski, Paolo Bonzini, Johannes Sixt,
	Stephen Boyd, Andreas Schwab, Daniel Convissor
In-Reply-To: <7vockqbq44.fsf@alter.siamese.dyndns.org>

On mardi 19 janvier 2010, Junio C Hamano wrote:
> Christian Couder <chriscool@tuxfamily.org> writes:
> > The purpose of this new option is to discard some of the
> > last commits but to keep current changes in the work tree.
> >
> > The use case is when you work on something and commit
> > that work. And then you work on something else that touches
> > other files, but you don't commit it yet. Then you realize
> > that what you commited when you worked on the first thing
> > is not good or belongs to another branch.
> >
> > So you want to get rid of the previous commits (at least in
> > the current branch) but you want to make sure that you keep
> > the changes you have in the work tree.
>
> You did this:
>
> 	git checkout master
> 	work; git commit ; work; git commit ; work; git commit
> 	edit ; git add ; ... (but not commit)
>
> and noticed the three commits should not be on master but on a new
> branch.
>
> I think we currently teach users to do something like this:
>
> 	git stash
>         git branch topic
>         git reset --hard HEAD~3
>         git stash pop
>
> Instead you want to do this:
>
> 	git branch topic
>         git reset --keep HEAD~3
>
> Surely you halved the number of command involved, but is this really an
> improvement?  At least I can visualize (and more importantly, explain to
> new users) how the "stash, flip and unstash" works, why it is safe, and
> how to recover when "pop" stops in conflicts, but I have no confidence in
> explaining what "reset --keep" does and how to recover when it refuses to
> work to new users.

Of course new users should be told about "git stash" before being told 
about "git reset --keep" and "git reset --merge". These 2 options are 
mostly for advanced users who want shortcuts and are ready to learn a few 
more options to speed up some of their common tasks.

Commit 9e8eceab ("Add 'merge' mode to 'git reset'", 2008-12-01), ended with:

-------------------
    Yes, yes, with a dirty tree you could always do

        git stash
        git reset --hard
        git stash apply

    instead, but isn't "git reset --merge" a nice way to handle one 
particular simple case?
-------------------

and I claim that the same is true for "--keep". It is just a nice way to 
handle one particular simple case.

> Another way to accomplish the same thing might be:
>
> 	git branch -m topic
>         git checkout -b master HEAD~3
>
> and with the same number of commands, conceptually it may be easier to
> understand than "reset --keep".  What you committed so far belongs to
> another branch "topic", so you name the current history that way, and
> then you switch branches with "checkout" that keeps your local
> modifications. It also opens the possibility of retrying with "-m" after
> checkout refuses to acti, to take the same mix-up risk CVS/SVN users
> have, if you are very confident that your local change conflicts only
> minimally with the change made on the topic and you can resolve them.

Sorry but I don't find that easier to understand. On the contrary I find 
awkward to have to rename the current branch first.

My first reaction when I realize that my current work belongs to another 
branch is just to create another branch with a good name, and then I try to 
find a way to make the new branch clean. It would be strange and perhaps a 
little bit unsafe, at least for me, to have to rename the current branch 
and then recreate it with some good content.

> Of course, when you are not interested in keeping the topmost commits at
> all, you either
>
> 	git stash ; git reset --hard HEAD~3 ; git stash pop
>
> or
>
> 	git reset --keep HEAD~3
>
> but even in this case, I think "stash, flip and unstash" is easier to
> explain, especially when teaching what to do if things go wrong.
>
> I dunno.  Is this really an useful addition that helps real-world
> workflows and is worth a five patch series, or is this just "because we
> can" patch?

I know that I will use the new option. At least I will use it much more 
than --merge that I never used at $dayjob, probably because I don't merge a 
lot of stuff.

Best regards,
Christian.

^ permalink raw reply

* [PATCH] t7800-difftool.sh: Test mergetool.prompt fallback
From: David Aguilar @ 2010-01-23  5:58 UTC (permalink / raw)
  To: gitster; +Cc: git, sschuberth

4cacc621 made difftool fall back to mergetool.prompt
when difftool.prompt is unconfigured.  This adds a test.

Signed-off-by: David Aguilar <davvid@gmail.com>
---
 t/t7800-difftool.sh |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index fad5472..a16bfa0 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -27,6 +27,7 @@ remove_config_vars()
 	git config --unset difftool.prompt
 	git config --unset merge.tool
 	git config --unset mergetool.test-tool.cmd
+	git config --unset mergetool.prompt
 	return 0
 }
 
@@ -159,6 +160,17 @@ test_expect_success 'difftool.prompt config variable is false' '
 	restore_test_defaults
 '
 
+# Test that we don't have to pass --no-prompt when mergetool.prompt is false
+test_expect_success 'difftool merge.promptconfig = false' '
+	git config --unset difftool.prompt
+	git config mergetool.prompt false &&
+
+	diff=$(git difftool branch) &&
+	test "$diff" = "branch" &&
+
+	restore_test_defaults
+'
+
 # Test that the -y flag can override difftool.prompt = true
 test_expect_success 'difftool.prompt can overridden with -y' '
 	git config difftool.prompt true &&
-- 
1.6.6.1.436.gaba7d.dirty

^ permalink raw reply related

* [PATCH v2] t7800-difftool.sh: Test mergetool.prompt fallback
From: David Aguilar @ 2010-01-23  6:03 UTC (permalink / raw)
  To: gitster; +Cc: sschuberth, git

4cacc621 made difftool fall back to mergetool.prompt
when difftool.prompt is unconfigured.  This adds a test.

Signed-off-by: David Aguilar <davvid@gmail.com>
---

Oops, the last one had a typo in the test string.

 t/t7800-difftool.sh |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index fad5472..19c72f5 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -27,6 +27,7 @@ remove_config_vars()
 	git config --unset difftool.prompt
 	git config --unset merge.tool
 	git config --unset mergetool.test-tool.cmd
+	git config --unset mergetool.prompt
 	return 0
 }
 
@@ -159,6 +160,17 @@ test_expect_success 'difftool.prompt config variable is false' '
 	restore_test_defaults
 '
 
+# Test that we don't have to pass --no-prompt when mergetool.prompt is false
+test_expect_success 'difftool merge.prompt = false' '
+	git config --unset difftool.prompt
+	git config mergetool.prompt false &&
+
+	diff=$(git difftool branch) &&
+	test "$diff" = "branch" &&
+
+	restore_test_defaults
+'
+
 # Test that the -y flag can override difftool.prompt = true
 test_expect_success 'difftool.prompt can overridden with -y' '
 	git config difftool.prompt true &&
-- 
1.6.6.1.436.gaba7d.dirty

^ permalink raw reply related

* Re: Remove diff machinery dependency from read-cache
From: Brian Campbell @ 2010-01-23  6:31 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <alpine.LFD.2.00.1001211119130.13231@localhost.localdomain>

On Jan 21, 2010, at 2:37 PM, Linus Torvalds wrote:

> ---
> builtin-add.c |   76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
> read-cache.c  |   78 ---------------------------------------------------------
> 2 files changed, 76 insertions(+), 78 deletions(-)
> 
> diff --git a/builtin-add.c b/builtin-add.c
> index cb6e590..2705f8d 100644

I've recently tried building "master" from git://git.kernel.org/pub/scm/git/git.git, and got the following error:

    $ make
...snip...	
        LINK git-fast-import
    Undefined symbols:
      "_git_mailmap_file", referenced from:
          _git_default_config in libgit.a(config.o)
    ld: symbol(s) not found
    collect2: ld returned 1 exit status
    make: *** [git-fast-import] Error 1

When I bisect, I find this commit to blame:

    $ git bisect start master v1.6.6.1
    Bisecting: 197 revisions left to test after this (roughly 8 steps)
    [f8eb50f60b5c8efda3529fcf89517080c686ce0b] Merge branch 'jh/commit-status'
    $ git bisect run make -j2
    running make -j2
    GIT_VERSION = 1.6.6.240.gf8eb5
...snip...
    fb7d3f32b283a3847e6f151a06794abd14ffd81b is the first bad commit
    commit fb7d3f32b283a3847e6f151a06794abd14ffd81b
    Author: Linus Torvalds <torvalds@linux-foundation.org>
    Date:   Thu Jan 21 11:37:38 2010 -0800

I also verified that it fails from with "make clean; make", so a dirty tree or -j2 aren't to blame. I'm not terribly familiar with the code base, so I'm a bit puzzled about why this commit would have cause the error that it does; I don't see any obvious connection between add_files_to_cache() and git_mailmap_file. 

Can anyone explain why I'd be seeing such an error? Is this a problem on my end?

    $ uname -a
    Darwin erlang.local 10.2.0 Darwin Kernel Version 10.2.0: Tue Nov  3 10:37:10 PST 2009; root:xnu-1486.2.11~1/RELEASE_I386 i386
    $ gcc --version
    i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5646) (dot 1)
    Copyright (C) 2007 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

^ permalink raw reply

* [PATCH] Documentation: rev-list: fix synopsys for --tags and and --remotes
From: Christian Couder @ 2010-01-23  7:26 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Ilari Liusvaara, Johannes Sixt


Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/git-rev-list.txt |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-rev-list.txt b/Documentation/git-rev-list.txt
index ae17c8a..173f3fc 100644
--- a/Documentation/git-rev-list.txt
+++ b/Documentation/git-rev-list.txt
@@ -22,8 +22,8 @@ SYNOPSIS
 	     [ \--not ]
 	     [ \--all ]
 	     [ \--branches[=pattern] ]
-	     [ \--tags=[pattern] ]
-	     [ \--remotes=[pattern] ]
+	     [ \--tags[=pattern] ]
+	     [ \--remotes[=pattern] ]
 	     [ \--glob=glob-pattern ]
 	     [ \--stdin ]
 	     [ \--quiet ]
-- 
1.6.6.1.557.g77031

^ permalink raw reply related

* [PATCH] rebase -p: Preserve fast-forwardable merges
From: Alex Scarborough @ 2010-01-23  7:29 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Johannes Schindelin, Michael Haggerty,
	Alex Scarborough

Previously, rebase -p would not preserve a merge commit if the merge
could be resolved as a fast-forward.  rebase -p now passes --no-ff to
git merge when recreating a merge commit, which ensures that merge
commits created with git merge --no-ff are preserved.

Signed-off-by: Alex Scarborough <alex@gameclay.com>
---
First patch, so here's hoping neither I nor my mail client messed up
too much.

This patch will not apply cleanly to branches which do not have
mh/rebase-fixup merged in, as that series removed the wrap of the
line I changed.

 git-rebase--interactive.sh                         |    2 +-
 t/t3416-rebase-preserve-fast-forwardable-merges.sh |   41 ++++++++++++++++++++
 2 files changed, 42 insertions(+), 1 deletions(-)
 create mode 100755 t/t3416-rebase-preserve-fast-forwardable-merges.sh

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index c2f6089..a7a2acc 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -343,7 +343,7 @@ pick_one_preserving_merges () {
 			# No point in merging the first parent, that's HEAD
 			new_parents=${new_parents# $first_parent}
 			if ! do_with_author output \
-				git merge $STRATEGY -m "$msg" $new_parents
+				git merge --no-ff $STRATEGY -m "$msg" $new_parents
 			then
 				printf "%s\n" "$msg" > "$GIT_DIR"/MERGE_MSG
 				die_with_patch $sha1 "Error redoing merge $sha1"
diff --git a/t/t3416-rebase-preserve-fast-forwardable-merges.sh
b/t/t3416-rebase-preserve-fast-forwardable-merges.sh
new file mode 100755
index 0000000..d46bf91
--- /dev/null
+++ b/t/t3416-rebase-preserve-fast-forwardable-merges.sh
@@ -0,0 +1,41 @@
+#!/bin/sh
+
+test_description='git rebase preserve fastforwardable merges
+
+This test runs git rebase with -p and checks that merges created by
+git merge --no-ff are properly carried along.
+'
+. ./test-lib.sh
+
+# set up three branches like this:
+#
+# A1 - C1 - - - E1
+#   \   \       /
+#    \   -- D1 --
+#     \
+#      -- B1
+
+test_expect_success 'setup' '
+	test_commit A1 &&
+	test_commit B1 &&
+	git reset --hard A1 &&
+	test_commit C1 &&
+	test_commit D1 &&
+	git reset --hard C1 &&
+	test_tick &&
+	git merge --no-ff -m "E1" "D1" &&
+	git tag "E1"
+'
+
+# Should result in:
+#
+# A1 - B1 - C2 - - - E2
+#            \       /
+#             -- D2 --
+#
+test_expect_success 'rebase C1 onto B1' '
+	git rebase -p B1 &&
+	git rev-parse HEAD^2
+'
+
+test_done
-- 
1.6.6

^ permalink raw reply related

* [PATCH 1/2] git-svn: allow UUID to be manually remapped via rewriteUUID
From: Jay Soffian @ 2010-01-23  8:30 UTC (permalink / raw)
  To: git; +Cc: Jay Soffian, Junio C Hamano, Eric Wong
In-Reply-To: <1264235401-44051-1-git-send-email-jaysoffian@gmail.com>

In certain situations it may be necessary to manually remap an svn
repostitory UUID. For example:

                  o--- [git-svn clone]
                 /
[origin svn repo]
                 \
                  o--- [svnsync clone]

Imagine that only "git-svn clone" and "svnsync clone" are made available
to external users. Furthur, "git-svn clone" contains only trunk, and for
reasons unknown, "svnsync clone" is missing the revision properties that
normally provide the origin svn repo's UUID.

A git user who has cloned the "git-svn clone" repo now wishes to use
git-svn to pull in the missing branches from the "synsync clone" repo.
In order for git-svn to get the history correct for those branches,
it needs to know the origin svn repo's UUID. Hence rewriteUUID.

Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
---
 Documentation/git-svn.txt       |   15 ++++++--
 git-svn.perl                    |   33 ++++++++++++++---
 t/t9153-git-svn-rewrite-uuid.sh |   25 +++++++++++++
 t/t9153/svn.dump                |   75 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 140 insertions(+), 8 deletions(-)
 create mode 100755 t/t9153-git-svn-rewrite-uuid.sh
 create mode 100644 t/t9153/svn.dump

diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 4cdca0d..98fe439 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -62,6 +62,8 @@ COMMANDS
 	Set the 'useSvnsyncProps' option in the [svn-remote] config.
 --rewrite-root=<URL>;;
 	Set the 'rewriteRoot' option in the [svn-remote] config.
+--rewrite-uuid=<UUID>;;
+	Set the 'rewriteUUID' option in the [svn-remote] config.
 --username=<USER>;;
 	For transports that SVN handles authentication for (http,
 	https, and plain svn), specify the username.  For other
@@ -616,6 +618,12 @@ svn-remote.<name>.rewriteRoot::
 	the repository with a public http:// or svn:// URL in the
 	metadata so users of it will see the public URL.
 
+svn-remote.<name>.rewriteUUID::
+	Similar to the useSvmProps option; this is for users who need
+	to remap the UUID manually. This may be useful in situations
+	where the original UUID is not available via either useSvmProps
+	or useSvnsyncProps.
+
 svn.brokenSymlinkWorkaround::
 	This disables potentially expensive checks to workaround
 	broken symlinks checked into SVN by broken clients.  Set this
@@ -625,13 +633,14 @@ svn.brokenSymlinkWorkaround::
 	revision fetched.  If unset, 'git svn' assumes this option to
 	be "true".
 
-Since the noMetadata, rewriteRoot, useSvnsyncProps and useSvmProps
+Since the noMetadata, rewriteRoot, rewriteUUID, useSvnsyncProps and useSvmProps
 options all affect the metadata generated and used by 'git svn'; they
 *must* be set in the configuration file before any history is imported
 and these settings should never be changed once they are set.
 
-Additionally, only one of these four options can be used per-svn-remote
-section because they affect the 'git-svn-id:' metadata line.
+Additionally, only one of these options can be used per svn-remote
+section because they affect the 'git-svn-id:' metadata line, except
+for rewriteRoot and rewriteUUID which can be used together.
 
 
 BASIC EXAMPLES
diff --git a/git-svn.perl b/git-svn.perl
index 650c9e5..1fab210 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -115,6 +115,7 @@ my %init_opts = ( 'template=s' => \$_template, 'shared:s' => \$_shared,
 		  'use-svm-props' => sub { $icv{useSvmProps} = 1 },
 		  'use-svnsync-props' => sub { $icv{useSvnsyncProps} = 1 },
 		  'rewrite-root=s' => sub { $icv{rewriteRoot} = $_[1] },
+		  'rewrite-uuid=s' => sub { $icv{rewriteUUID} = $_[1] },
                   %remote_opts );
 my %cmt_opts = ( 'edit|e' => \$_edit,
 		'rmdir' => \$SVN::Git::Editor::_rmdir,
@@ -2189,6 +2190,10 @@ sub svnsync {
 		die "Can't have both 'useSvnsyncProps' and 'rewriteRoot' ",
 		    "options set!\n";
 	}
+	if ($self->rewrite_uuid) {
+		die "Can't have both 'useSvnsyncProps' and 'rewriteUUID' ",
+		    "options set!\n";
+	}
 
 	my $svnsync;
 	# see if we have it in our config, first:
@@ -2470,6 +2475,20 @@ sub rewrite_root {
 	$self->{-rewrite_root} = $rwr;
 }
 
+sub rewrite_uuid {
+	my ($self) = @_;
+	return $self->{-rewrite_uuid} if exists $self->{-rewrite_uuid};
+	my $k = "svn-remote.$self->{repo_id}.rewriteUUID";
+	my $rwid = eval { command_oneline(qw/config --get/, $k) };
+	if ($rwid) {
+		$rwid =~ s#/+$##;
+		if ($rwid !~ m#^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$#) {
+			die "$rwid is not a valid UUID (key: $k)\n";
+		}
+	}
+	$self->{-rewrite_uuid} = $rwid;
+}
+
 sub metadata_url {
 	my ($self) = @_;
 	($self->rewrite_root || $self->{url}) .
@@ -3253,6 +3272,10 @@ sub make_log_entry {
 			die "Can't have both 'useSvmProps' and 'rewriteRoot' ",
 			    "options set!\n";
 		}
+		if ($self->rewrite_uuid) {
+			die "Can't have both 'useSvmProps' and 'rewriteUUID' ",
+			    "options set!\n";
+		}
 		my ($uuid, $r) = $headrev =~ m{^([a-f\d\-]{30,}):(\d+)$}i;
 		# we don't want "SVM: initializing mirror for junk" ...
 		return undef if $r == 0;
@@ -3283,10 +3306,10 @@ sub make_log_entry {
 	} else {
 		my $url = $self->metadata_url;
 		remove_username($url);
-		$log_entry{metadata} = "$url\@$rev " .
-		                       $self->ra->get_uuid;
-		$email ||= "$author\@" . $self->ra->get_uuid;
-		$commit_email ||= "$author\@" . $self->ra->get_uuid;
+		my $uuid = $self->rewrite_uuid || $self->ra->get_uuid;
+		$log_entry{metadata} = "$url\@$rev " . $uuid;
+		$email ||= "$author\@" . $uuid;
+		$commit_email ||= "$author\@" . $uuid;
 	}
 	$log_entry{name} = $name;
 	$log_entry{email} = $email;
@@ -3368,7 +3391,7 @@ sub rebuild {
 				'--');
 	my $metadata_url = $self->metadata_url;
 	remove_username($metadata_url);
-	my $svn_uuid = $self->ra_uuid;
+	my $svn_uuid = $self->rewrite_uuid || $self->ra_uuid;
 	my $c;
 	while (<$log>) {
 		if ( m{^commit ($::sha1)$} ) {
diff --git a/t/t9153-git-svn-rewrite-uuid.sh b/t/t9153-git-svn-rewrite-uuid.sh
new file mode 100755
index 0000000..88a2cfa
--- /dev/null
+++ b/t/t9153-git-svn-rewrite-uuid.sh
@@ -0,0 +1,25 @@
+#!/bin/sh
+#
+# Copyright (c) 2010 Jay Soffian
+#
+
+test_description='git svn --rewrite-uuid test'
+
+. ./lib-git-svn.sh
+
+uuid=6cc8ada4-5932-4b4a-8242-3534ed8a3232
+
+test_expect_success 'load svn repo' "
+	svnadmin load -q '$rawsvnrepo' < '$TEST_DIRECTORY/t9153/svn.dump' &&
+	git svn init --minimize-url --rewrite-uuid='$uuid' '$svnrepo' &&
+	git svn fetch
+	"
+
+test_expect_success 'verify uuid' "
+	git cat-file commit refs/remotes/git-svn~0 | \
+	   grep '^${git_svn_id}: .*@2 $uuid$' &&
+	git cat-file commit refs/remotes/git-svn~1 | \
+	   grep '^${git_svn_id}: .*@1 $uuid$'
+	"
+
+test_done
diff --git a/t/t9153/svn.dump b/t/t9153/svn.dump
new file mode 100644
index 0000000..0ddfe70
--- /dev/null
+++ b/t/t9153/svn.dump
@@ -0,0 +1,75 @@
+SVN-fs-dump-format-version: 2
+
+UUID: b4885626-c94f-4a6c-b179-00c030fc68e8
+
+Revision-number: 0
+Prop-content-length: 56
+Content-length: 56
+
+K 8
+svn:date
+V 27
+2010-01-23T06:41:03.908576Z
+PROPS-END
+
+Revision-number: 1
+Prop-content-length: 109
+Content-length: 109
+
+K 7
+svn:log
+V 11
+initial foo
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T06:41:48.353776Z
+PROPS-END
+
+Node-path: foo
+Node-kind: file
+Node-action: add
+Prop-content-length: 10
+Text-content-length: 4
+Text-content-md5: d3b07384d113edec49eaa6238ad5ff00
+Text-content-sha1: f1d2d2f924e986ac86fdf7b36c94bcdf32beec15
+Content-length: 14
+
+PROPS-END
+foo
+
+
+Revision-number: 2
+Prop-content-length: 110
+Content-length: 110
+
+K 7
+svn:log
+V 12
+now with bar
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T06:42:14.214640Z
+PROPS-END
+
+Node-path: foo
+Node-kind: file
+Node-action: change
+Text-content-length: 8
+Text-content-md5: f47c75614087a8dd938ba4acff252494
+Text-content-sha1: 4e48e2c9a3d2ca8a708cb0cc545700544efb5021
+Content-length: 8
+
+foo
+bar
+
+
-- 
1.6.6.1.515.g288caf

^ permalink raw reply related

* [PATCH 0/2] two git-svn changes to help work with chromium.org
From: Jay Soffian @ 2010-01-23  8:29 UTC (permalink / raw)
  To: git; +Cc: Jay Soffian, Junio C Hamano, Eric Wong

Google nicely mirrors the subversion chrome repository with git-svn. However,
the git-svn mirror has only trunk. I wanted to add in a couple other branches
to my clone of the git-svn mirror, but to do so I needed the ability to lie
about the UUID since the publically accessible subversion repository has a
different UUID than that which google is cloning from using git-svn.

In addition, google has a ton of branches in the subversion repository, and
I only wanted a few. Hence this two-patch series. For posterity, here's what
my .git/config looks like:

[remote "origin"]
	url = git://git.chromium.org/chromium.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[svn-remote "svn"]
	url = http://src.chromium.org/svn
	fetch = trunk/src:refs/remotes/svn/trunk
	branches = branches/{249,302}/src:refs/remotes/svn/*
	rewriteRoot = svn://svn.chromium.org/chrome
	rewriteUUID = 0039d316-1c4b-4281-b951-d872f2087c98

Jay Soffian (2):
  git-svn: allow UUID to be manually remapped via rewriteUUID
  git-svn: allow subset of branches/tags to be specified in glob spec

 Documentation/git-svn.txt       |   31 +++++-
 git-svn.perl                    |   93 ++++++++++++-----
 t/t9153-git-svn-rewrite-uuid.sh |   25 +++++
 t/t9153/svn.dump                |   75 +++++++++++++
 t/t9154-git-svn-fancy-glob.sh   |   42 ++++++++
 t/t9154/svn.dump                |  222 +++++++++++++++++++++++++++++++++++++++
 6 files changed, 460 insertions(+), 28 deletions(-)
 create mode 100755 t/t9153-git-svn-rewrite-uuid.sh
 create mode 100644 t/t9153/svn.dump
 create mode 100755 t/t9154-git-svn-fancy-glob.sh
 create mode 100644 t/t9154/svn.dump

^ permalink raw reply

* [PATCH 2/2] git-svn: allow subset of branches/tags to be specified in glob spec
From: Jay Soffian @ 2010-01-23  8:30 UTC (permalink / raw)
  To: git; +Cc: Jay Soffian, Junio C Hamano, Eric Wong
In-Reply-To: <1264235401-44051-1-git-send-email-jaysoffian@gmail.com>

For very large projects it is useful to be able to clone a subset of the
upstream SVN repo's branches. Allow for this by letting the left-side of
the branches and tags glob specs contain a brace-delineated comma-separated
list of names. e.g.:

	branches = branches/{red,green}/src:refs/remotes/branches/*

Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
---
The only part of this I don't like is asking the user to manually edit
.git/svn/.metadata, but I couldn't think of a clean way to invalidate
the cached maxRev property. Further, if you know the starting point
of your branches, it can speed things up quite a bit to just set maxRev
to the appropriate value, esp for repos with a long history.

 Documentation/git-svn.txt     |   16 +++
 git-svn.perl                  |   60 ++++++++----
 t/t9154-git-svn-fancy-glob.sh |   42 ++++++++
 t/t9154/svn.dump              |  222 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 320 insertions(+), 20 deletions(-)
 create mode 100755 t/t9154-git-svn-fancy-glob.sh
 create mode 100644 t/t9154/svn.dump

diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 98fe439..190a62a 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -825,6 +825,22 @@ independent path component (surrounded by '/' or EOL).   This
 type of configuration is not automatically created by 'init' and
 should be manually entered with a text-editor or using 'git config'.
 
+It is also possible to fetch a subset of branches or tags by using a
+comma-separated list of names within braces. For example:
+
+------------------------------------------------------------------------
+[svn-remote "huge-project"]
+	url = http://server.org/svn
+	fetch = trunk/src:refs/remotes/trunk
+	branches = branches/{red,green}/src:refs/remotes/branches/*
+	tags = tags/{1.0,2.0}/src:refs/remotes/tags/*
+------------------------------------------------------------------------
+
+Note that git-svn keeps track of the highest revision in which a branch
+or tag has appeared. If the subset of branches or tags is changed after
+fetching, then .git/svn/.metadata must be manually edited to remove (or
+reset) branches-maxRev and/or tags-maxRev as appropriate.
+
 SEE ALSO
 --------
 linkgit:git-rebase[1]
diff --git a/git-svn.perl b/git-svn.perl
index 1fab210..80bdef7 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -1807,8 +1807,8 @@ sub read_all_remotes {
 			my $rs = {
 			    t => $t,
 			    remote => $remote,
-			    path => Git::SVN::GlobSpec->new($local_ref),
-			    ref => Git::SVN::GlobSpec->new($remote_ref) };
+			    path => Git::SVN::GlobSpec->new($local_ref, 1),
+			    ref => Git::SVN::GlobSpec->new($remote_ref, 0) };
 			if (length($rs->{ref}->{right}) != 0) {
 				die "The '*' glob character must be the last ",
 				    "character of '$remote_ref'\n";
@@ -5180,6 +5180,7 @@ sub match_globs {
 			next if (length $g->{path}->{right} &&
 				 ($self->check_path($p, $r) !=
 				  $SVN::Node::dir));
+			next unless $p =~ /$g->{path}->{regex}/;
 			$exists->{$p} = Git::SVN->init($self->{url}, $p, undef,
 					 $g->{ref}->full_path($de), 1);
 		}
@@ -5953,29 +5954,48 @@ use strict;
 use warnings;
 
 sub new {
-	my ($class, $glob) = @_;
+	my ($class, $glob, $pattern_ok) = @_;
 	my $re = $glob;
 	$re =~ s!/+$!!g; # no need for trailing slashes
-	$re =~ m!^([^*]*)(\*(?:/\*)*)(.*)$!;
-	my $temp = $re;
-	my ($left, $right) = ($1, $3);
-	$re = $2;
-	my $depth = $re =~ tr/*/*/;
-	if ($depth != $temp =~ tr/*/*/) {
-		die "Only one set of wildcard directories " .
-			"(e.g. '*' or '*/*/*') is supported: '$glob'\n";
+	my (@left, @right, @patterns);
+	my $state = "left";
+	my $die_msg = "Only one set of wildcard directories " .
+				"(e.g. '*' or '*/*/*') is supported: '$glob'\n";
+	for my $part (split(m|/|, $glob)) {
+		if ($part =~ /\*/ && $part ne "*") {
+			die "Invalid pattern in '$glob': $part\n";
+		} elsif ($pattern_ok && $part =~ /[{}]/ &&
+			 $part !~ /^\{[^{}]+\}/) {
+			die "Invalid pattern in '$glob': $part\n";
+		}
+		if ($part eq "*") {
+			die $die_msg if $state eq "right";
+			$state = "pattern";
+			push(@patterns, "[^/]*");
+		} elsif ($pattern_ok && $part =~ /^\{(.*)\}$/) {
+			die $die_msg if $state eq "right";
+			$state = "pattern";
+			my $p = quotemeta($1);
+			$p =~ s/\\,/|/g;
+			push(@patterns, "(?:$p)");
+		} else {
+			if ($state eq "left") {
+				push(@left, $part);
+			} else {
+				push(@right, $part);
+				$state = "right";
+			}
+		}
 	}
+	my $depth = @patterns;
 	if ($depth == 0) {
-		die "One '*' is needed for glob: '$glob'\n";
-	}
-	$re =~ s!\*!\[^/\]*!g;
-	$re = quotemeta($left) . "($re)" . quotemeta($right);
-	if (length $left && !($left =~ s!/+$!!g)) {
-		die "Missing trailing '/' on left side of: '$glob' ($left)\n";
-	}
-	if (length $right && !($right =~ s!^/+!!g)) {
-		die "Missing leading '/' on right side of: '$glob' ($right)\n";
+		die "One '*' is needed in glob: '$glob'\n";
 	}
+	my $left = join('/', @left);
+	my $right = join('/', @right);
+	$re = join('/', @patterns);
+	$re = join('\/',
+		   grep(length, quotemeta($left), "($re)", quotemeta($right)));
 	my $left_re = qr/^\/\Q$left\E(\/|$)/;
 	bless { left => $left, right => $right, left_regex => $left_re,
 	        regex => qr/$re/, glob => $glob, depth => $depth }, $class;
diff --git a/t/t9154-git-svn-fancy-glob.sh b/t/t9154-git-svn-fancy-glob.sh
new file mode 100755
index 0000000..a6a56a6
--- /dev/null
+++ b/t/t9154-git-svn-fancy-glob.sh
@@ -0,0 +1,42 @@
+#!/bin/sh
+#
+# Copyright (c) 2010 Jay Soffian
+#
+
+test_description='git svn fancy glob test'
+
+. ./lib-git-svn.sh
+
+test_expect_success 'load svn repo' "
+	svnadmin load -q '$rawsvnrepo' < '$TEST_DIRECTORY/t9154/svn.dump' &&
+	git svn init --minimize-url -T trunk '$svnrepo' &&
+	git svn fetch
+	"
+
+test_expect_success 'add red branch' "
+	git config svn-remote.svn.branches 'branches/{red}:refs/remotes/*' &&
+	git svn fetch &&
+	git rev-parse refs/remotes/red &&
+	test_must_fail git rev-parse refs/remotes/green &&
+	test_must_fail git rev-parse refs/remotes/blue
+	"
+
+test_expect_success 'add green branch' "
+	GIT_CONFIG=.git/svn/.metadata git config --unset svn-remote.svn.branches-maxRev &&
+	git config svn-remote.svn.branches 'branches/{red,green}:refs/remotes/*' &&
+	git svn fetch &&
+	git rev-parse refs/remotes/red &&
+	git rev-parse refs/remotes/green &&
+	test_must_fail git rev-parse refs/remotes/blue
+	"
+
+test_expect_success 'add all branches' "
+	GIT_CONFIG=.git/svn/.metadata git config --unset svn-remote.svn.branches-maxRev &&
+	git config svn-remote.svn.branches 'branches/*:refs/remotes/*' &&
+	git svn fetch &&
+	git rev-parse refs/remotes/red &&
+	git rev-parse refs/remotes/green &&
+	git rev-parse refs/remotes/blue
+	"
+
+test_done
diff --git a/t/t9154/svn.dump b/t/t9154/svn.dump
new file mode 100644
index 0000000..3dfabb6
--- /dev/null
+++ b/t/t9154/svn.dump
@@ -0,0 +1,222 @@
+SVN-fs-dump-format-version: 2
+
+UUID: a18093a0-5f0b-44e0-8d88-8911ac7078db
+
+Revision-number: 0
+Prop-content-length: 56
+Content-length: 56
+
+K 8
+svn:date
+V 27
+2010-01-23T07:40:25.660053Z
+PROPS-END
+
+Revision-number: 1
+Prop-content-length: 104
+Content-length: 104
+
+K 7
+svn:log
+V 7
+initial
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T07:41:33.636365Z
+PROPS-END
+
+Node-path: trunk
+Node-kind: dir
+Node-action: add
+Prop-content-length: 10
+Content-length: 10
+
+PROPS-END
+
+
+Node-path: trunk/foo
+Node-kind: file
+Node-action: add
+Prop-content-length: 10
+Text-content-length: 4
+Text-content-md5: d3b07384d113edec49eaa6238ad5ff00
+Text-content-sha1: f1d2d2f924e986ac86fdf7b36c94bcdf32beec15
+Content-length: 14
+
+PROPS-END
+foo
+
+
+Revision-number: 2
+Prop-content-length: 110
+Content-length: 110
+
+K 7
+svn:log
+V 12
+add branches
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T07:42:37.290694Z
+PROPS-END
+
+Node-path: branches
+Node-kind: dir
+Node-action: add
+Prop-content-length: 10
+Content-length: 10
+
+PROPS-END
+
+
+Node-path: branches/blue
+Node-kind: dir
+Node-action: add
+Node-copyfrom-rev: 1
+Node-copyfrom-path: trunk
+
+
+Node-path: branches/green
+Node-kind: dir
+Node-action: add
+Node-copyfrom-rev: 1
+Node-copyfrom-path: trunk
+
+
+Node-path: branches/red
+Node-kind: dir
+Node-action: add
+Node-copyfrom-rev: 1
+Node-copyfrom-path: trunk
+
+
+Revision-number: 3
+Prop-content-length: 108
+Content-length: 108
+
+K 7
+svn:log
+V 10
+red change
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T07:43:02.208918Z
+PROPS-END
+
+Node-path: branches/red/foo
+Node-kind: file
+Node-action: change
+Text-content-length: 8
+Text-content-md5: 64c3c8cf7d0233ab7627623a68888bd1
+Text-content-sha1: 95a0492027876adfd3891ec71ee37b79ee44d640
+Content-length: 8
+
+foo
+red
+
+
+Revision-number: 4
+Prop-content-length: 110
+Content-length: 110
+
+K 7
+svn:log
+V 12
+green change
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T07:43:15.746586Z
+PROPS-END
+
+Node-path: branches/green/foo
+Node-kind: file
+Node-action: change
+Text-content-length: 10
+Text-content-md5: 0209b6450891abc033d5eaaa9d3a8023
+Text-content-sha1: 87fc3bef9faeec48c0cd61dfc9851db377fdccf7
+Content-length: 10
+
+foo
+green
+
+
+Revision-number: 5
+Prop-content-length: 109
+Content-length: 109
+
+K 7
+svn:log
+V 11
+blue change
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T07:43:29.364811Z
+PROPS-END
+
+Node-path: branches/blue/foo
+Node-kind: file
+Node-action: change
+Text-content-length: 9
+Text-content-md5: 9fbe4c13d0bae86386ae5209b2e6b275
+Text-content-sha1: cc4575083459a16f9aaef796c4a2456d64691ba0
+Content-length: 9
+
+foo
+blue
+
+
+Revision-number: 6
+Prop-content-length: 110
+Content-length: 110
+
+K 7
+svn:log
+V 12
+trunk change
+K 10
+svn:author
+V 3
+jay
+K 8
+svn:date
+V 27
+2010-01-23T07:44:01.313130Z
+PROPS-END
+
+Node-path: trunk/foo
+Node-kind: file
+Node-action: change
+Text-content-length: 10
+Text-content-md5: 1c4db977d7a57c3bae582aab87948516
+Text-content-sha1: 469c08df449e702cf2a1fe746244a9ef3f837fad
+Content-length: 10
+
+foo
+trunk
+
+
-- 
1.6.6.1.515.g288caf

^ permalink raw reply related

* [PATCH v2 0/7] clarify 'git merge' documentation
From: Jonathan Nieder @ 2010-01-23  9:25 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano

For the previous round [1], I wrote something like this to clarify why
merging with dirty trees is safe.  I hope it also makes git-merge.1
more useful for other purposes.  I have not changed the warning in
question, which belongs to a different topic.

Patches 1-3 move some material towards the end of the 'merge' manual,
to make the manpage easier to read straight through.

Patch 4 adds a brief introduction for the reader who might not yet be
familiar with branching and merging.  I am especially interested in
feedback on this one.  My writing tends not to be as clear as I would
like.  Already some advice from Junio helped a great deal.

Patches 5-7 organize the material on how the merge works into short,
digestible pieces.  Patch 5 explains how a merge can abort without
doing anything, patch 6 the fast-forward merge, and patch 7 the true
merge, all from the point of view of what matters for the person
invoking ‘git merge’.  Patch 7 is basically rewritten using Thomas’s
advice.

The patches are against doc-style/for-next of Thomas’s repo at
git://repo.or.cz/git/trast .

The advice from last round was very helpful.  Apologies for taking so
long to respond to it.

Enjoy,
Jonathan Nieder (7):
  Documentation: merge: move configuration section to end
  Documentation: suggest `reset --merge` in How Merge Works section
  Documentation: merge: move merge strategy list to end
  Documentation: merge: add an overview
  Documentation: emphasize when git merge terminates early
  Documentation: merge: add a section about fast-forward
  Documentation: simplify How Merge Works

 Documentation/git-merge.txt |  164 ++++++++++++++++++++++++-------------------
 1 files changed, 92 insertions(+), 72 deletions(-)

[1] http://thread.gmane.org/gmane.comp.version-control.git/136356/focus=136617

^ permalink raw reply

* [PATCH 1/7] Documentation: merge: move configuration section to end
From: Jonathan Nieder @ 2010-01-23  9:26 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano
In-Reply-To: <20100123092551.GA7571@progeny.tock>

Configuration and environment variables belong to the back matter
of a manual page.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Petr Baudis <pasky@suse.cz>
---
 Documentation/git-merge.txt |   18 +++++++++---------
 1 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt
index c88bebe..6aa2bf3 100644
--- a/Documentation/git-merge.txt
+++ b/Documentation/git-merge.txt
@@ -48,15 +48,6 @@ include::merge-strategies.txt[]
 If you tried a merge which resulted in complex conflicts and
 want to start over, you can recover with 'git reset'.
 
-CONFIGURATION
--------------
-include::merge-config.txt[]
-
-branch.<name>.mergeoptions::
-	Sets default options for merging into branch <name>. The syntax and
-	supported options are the same as those of 'git merge', but option
-	values containing whitespace characters are currently not supported.
-
 HOW MERGE WORKS
 ---------------
 
@@ -249,6 +240,15 @@ changes into a merge commit.  Small fixups like bumping
 release/version name would be acceptable.
 
 
+CONFIGURATION
+-------------
+include::merge-config.txt[]
+
+branch.<name>.mergeoptions::
+	Sets default options for merging into branch <name>. The syntax and
+	supported options are the same as those of 'git merge', but option
+	values containing whitespace characters are currently not supported.
+
 SEE ALSO
 --------
 linkgit:git-fmt-merge-msg[1], linkgit:git-pull[1],
-- 
1.6.6

^ permalink raw reply related

* [PATCH 2/7] Documentation: suggest `reset --merge` in How Merge Works section
From: Jonathan Nieder @ 2010-01-23  9:31 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano
In-Reply-To: <20100123092551.GA7571@progeny.tock>

The 'merge' manual suggests 'reset' to cancel a merge at the end
of the Merge Strategies list.  It is more logical to explain this
right before explaining how merge conflicts work, so the daunted
reader can have a way out when he or she needs it most.

While at it, make the advice more dependable and self-contained
by providing the --merge option.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
This should make the reference to 'git reset' easier to find on
second reading.  Something like this ought to happen if we are
going to move the Merge Strategies list down towards the end of
the man page.

 Documentation/git-merge.txt |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt
index 6aa2bf3..1fecedb 100644
--- a/Documentation/git-merge.txt
+++ b/Documentation/git-merge.txt
@@ -45,9 +45,6 @@ include::merge-options.txt[]
 include::merge-strategies.txt[]
 
 
-If you tried a merge which resulted in complex conflicts and
-want to start over, you can recover with 'git reset'.
-
 HOW MERGE WORKS
 ---------------
 
@@ -115,6 +112,9 @@ When there are conflicts, the following happens:
    same and the index entries for them stay as they were,
    i.e. matching `HEAD`.
 
+If you tried a merge which resulted in complex conflicts and
+want to start over, you can recover with `git reset --merge`.
+
 HOW CONFLICTS ARE PRESENTED
 ---------------------------
 
-- 
1.6.6

^ permalink raw reply related

* [PATCH 3/7] Documentation: merge: move merge strategy list to end
From: Jonathan Nieder @ 2010-01-23  9:33 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano
In-Reply-To: <20100123092551.GA7571@progeny.tock>

So the section layout changes as follows:

 NAME
 SYNOPSIS
 DESCRIPTION
 OPTIONS
-MERGE STRATEGIES
 HOW MERGE WORKS
 HOW CONFLICTS ARE PRESENTED
 HOW TO RESOLVE CONFLICTS
 EXAMPLES
+MERGE STRATEGIES
 CONFIGURATION
 SEE ALSO
 AUTHOR
 DOCUMENTATION
 GIT
 NOTES

The first-time user will care more about conflicts than about
strategies other than 'recursive'.

One of the examples uses -s ours, but I do not think this hinders
readability.

Suggested-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Thomas Rast wrote [1]:

> While I agree with the general intent of deferring the strategies
> further back, wouldn't it be better go all the way and instead put
> them before (or even after, but one of them uses -s ours) "EXAMPLES"?
> The average user will care more about conflicts than about strategies
> other than 'recursive'.

Good idea.  Thanks!

[1] http://thread.gmane.org/gmane.comp.version-control.git/136356/focus=136679

 Documentation/git-merge.txt |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt
index 1fecedb..83bf3e7 100644
--- a/Documentation/git-merge.txt
+++ b/Documentation/git-merge.txt
@@ -42,8 +42,6 @@ include::merge-options.txt[]
 	You need at least one <commit>.  Specifying more than one
 	<commit> obviously means you are trying an Octopus.
 
-include::merge-strategies.txt[]
-
 
 HOW MERGE WORKS
 ---------------
@@ -240,6 +238,8 @@ changes into a merge commit.  Small fixups like bumping
 release/version name would be acceptable.
 
 
+include::merge-strategies.txt[]
+
 CONFIGURATION
 -------------
 include::merge-config.txt[]
-- 
1.6.6

^ permalink raw reply related

* [PATCH 4/7] Documentation: merge: add an overview
From: Jonathan Nieder @ 2010-01-23  9:42 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano
In-Reply-To: <20100123092551.GA7571@progeny.tock>

The reader unfamiliar with the concepts of branching and merging
would have been completely lost.  Try to help him with a diagram.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
The advice in [1] was very helpful.  I fear I’ve made a mess of
the language even so.  At least it does seem much clearer now.

Thoughts?

[1] http://thread.gmane.org/gmane.comp.version-control.git/136356/focus=136629

 Documentation/git-merge.txt |   28 ++++++++++++++++++++++++++--
 1 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt
index 83bf3e7..a7a40c6 100644
--- a/Documentation/git-merge.txt
+++ b/Documentation/git-merge.txt
@@ -15,8 +15,32 @@ SYNOPSIS
 
 DESCRIPTION
 -----------
-Merges the history specified by <commit> into HEAD, optionally using a
-specific merge strategy.
+Incorporates changes from the named commits (since the time their
+histories diverged from the current branch) into the current
+branch.  This command is used by 'git pull' to incorporate changes
+from another repository and can be used by hand to merge changes
+from one branch into another.
+
+Assume the following history exists and the current branch is
+"`master`":
+
+------------
+          A---B---C topic
+         /
+    D---E---F---G master
+------------
+
+Then "`git merge topic`" will replay the changes made on the
+`topic` branch since it diverged from `master` (i.e., `E`) until
+its current commit (`C`) on top of `master`, and record the result
+in a new commit along with the names of the two parent commits and
+a log message from the user describing the changes.
+
+------------
+          A---B---C topic
+         /         \
+    D---E---F---G---H master
+------------
 
 The second syntax (<msg> `HEAD` <commit>...) is supported for
 historical reasons.  Do not use it from the command line or in
-- 
1.6.6

^ permalink raw reply related

* [PATCH 5/7] Documentation: emphasize when git merge terminates early
From: Jonathan Nieder @ 2010-01-23  9:44 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano
In-Reply-To: <20100123092551.GA7571@progeny.tock>

A merge-based operation in git can fail in two ways: one that
stops before touching anything, or one that goes ahead and
results in conflicts.

As the 'git merge' manual explains:

| A merge is always between the current `HEAD` and one or more
| commits (usually, branch head or tag), and the index file must
| match the tree of `HEAD` commit (i.e. the contents of the last commit)
| when it starts out.

Unfortunately, the placement of this sentence makes it easy to
skip over, and its formulation leaves the important point, that
any other attempted merge will be gracefully aborted, unspoken.

So give this point its own section and expand upon it.

Probably this could be simplified somewhat: after all, a change
registered in the index is just a special kind of local
uncommited change, so the second added paragraph is only a
special case of the first.  It seemed more helpful to be explicit
here.

Inspired by <http://gitster.livejournal.com/25801.html>.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Too technical?

 Documentation/git-merge.txt |   31 +++++++++++++++++++++----------
 1 files changed, 21 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt
index a7a40c6..3663d58 100644
--- a/Documentation/git-merge.txt
+++ b/Documentation/git-merge.txt
@@ -67,21 +67,32 @@ include::merge-options.txt[]
 	<commit> obviously means you are trying an Octopus.
 
 
+PRE-MERGE CHECKS
+----------------
+
+Before applying outside changes, you should get your own work in
+good shape and committed locally, so it will not be clobbered if
+there are conflicts.  See also linkgit:git-stash[1].
+'git pull' and 'git merge' will stop without doing anything when
+local uncommitted changes overlap with files that 'git pull'/'git
+merge' may need to update.
+
+To avoid recording unrelated changes in the merge commit,
+'git pull' and 'git merge' will also abort if there are any changes
+registered in the index relative to the `HEAD` commit.  (One
+exception is when the changed index entries are in the state that
+would result from the merge already.)
+
+If all named commits are already ancestors of `HEAD`, 'git merge'
+will exit early with the message "Already up-to-date."
+
 HOW MERGE WORKS
 ---------------
 
 A merge is always between the current `HEAD` and one or more
-commits (usually, branch head or tag), and the index file must
-match the tree of `HEAD` commit (i.e. the contents of the last commit)
-when it starts out.  In other words, `git diff --cached HEAD` must
-report no changes.  (One exception is when the changed index
-entries are already in the same state that would result from
-the merge anyway.)
-
-Three kinds of merge can happen:
+commits (usually a branch head or tag).
 
-* The merged commit is already contained in `HEAD`. This is the
-  simplest case, called "Already up-to-date."
+Two kinds of merge can happen:
 
 * `HEAD` is already contained in the merged commit. This is the
   most common case especially when invoked from 'git pull':
-- 
1.6.6

^ permalink raw reply related

* [PATCH 6/7] Documentation: merge: add a section about fast-forward
From: Jonathan Nieder @ 2010-01-23  9:45 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano
In-Reply-To: <20100123092551.GA7571@progeny.tock>

Novices sometimes find the behavior of 'git merge' in the
fast-forward case surprising.  Describe it thoroughly.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Sometimes people ask on IRC.

 Documentation/git-merge.txt |   31 ++++++++++++++++++-------------
 1 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt
index 3663d58..0b86f2b 100644
--- a/Documentation/git-merge.txt
+++ b/Documentation/git-merge.txt
@@ -86,25 +86,30 @@ would result from the merge already.)
 If all named commits are already ancestors of `HEAD`, 'git merge'
 will exit early with the message "Already up-to-date."
 
+FAST-FORWARD MERGE
+------------------
+
+Often the current branch head is an ancestor of the named commit.
+This is the most common case especially when invoked from 'git
+pull': you are tracking an upstream repository, you have committed
+no local changes, and now you want to update to a newer upstream
+revision.  In this case, a new commit is not needed to store the
+combined history; instead, the `HEAD` (along with the index) is
+updated to point at the named commit, without creating an extra
+merge commit.
+
+This behavior can be suppressed with the `--no-ff` option.
+
 HOW MERGE WORKS
 ---------------
 
 A merge is always between the current `HEAD` and one or more
 commits (usually a branch head or tag).
 
-Two kinds of merge can happen:
-
-* `HEAD` is already contained in the merged commit. This is the
-  most common case especially when invoked from 'git pull':
-  you are tracking an upstream repository, have committed no local
-  changes and now you want to update to a newer upstream revision.
-  Your `HEAD` (and the index) is updated to point at the merged
-  commit, without creating an extra merge commit.  This is
-  called "Fast-forward".
-
-* Both the merged commit and `HEAD` are independent and must be
-  tied together by a merge commit that has both of them as its parents.
-  The rest of this section describes this "True merge" case.
+Except in a fast-forward merge (see above), the branches to be
+merged must be tied together by a merge commit that has both of them
+as its parents.
+The rest of this section describes this "True merge" case.
 
 The chosen merge strategy merges the two commits into a single
 new source tree.
-- 
1.6.6

^ permalink raw reply related

* [PATCH 7/7] Documentation: simplify How Merge Works
From: Jonathan Nieder @ 2010-01-23  9:48 UTC (permalink / raw)
  To: git; +Cc: Thomas Rast, Petr Baudis, Junio C Hamano
In-Reply-To: <20100123092551.GA7571@progeny.tock>

The user most likely does not care about the exact order of
operations because he cannot see it happening anyway.  Instead,
try to explain what it means to merge two commits into a single
tree.

While at it:

 - Change the heading to TRUE MERGE.  The entire manual page is
   about how merges work.

 - Document MERGE_HEAD.  It is a useful feature, since it makes
   the parents of the intended merge commit easier to refer to.

 - Do not assume commits named on the 'git merge' command line come
   from another repository.  For simplicity, the discussion of
   conflicts still does assume that there is only one and it is a
   branch head.

 - Do not start list items with `code`.  Otherwise, a toolchain bug
   produces a line break in the generated nroff, resulting in odd
   extra space.

Suggested-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
This is the end of series.  Thanks for reading.

Thomas Rast wrote: [1]

> So how about
[...]
> and then snip everything up to

Oh!  Much better, thanks.

[1] http://thread.gmane.org/gmane.comp.version-control.git/136356/focus=136629

 Documentation/git-merge.txt |   52 +++++++++++++-----------------------------
 1 files changed, 16 insertions(+), 36 deletions(-)

diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt
index 0b86f2b..7aa3f3f 100644
--- a/Documentation/git-merge.txt
+++ b/Documentation/git-merge.txt
@@ -100,52 +100,32 @@ merge commit.
 
 This behavior can be suppressed with the `--no-ff` option.
 
-HOW MERGE WORKS
----------------
-
-A merge is always between the current `HEAD` and one or more
-commits (usually a branch head or tag).
+TRUE MERGE
+----------
 
 Except in a fast-forward merge (see above), the branches to be
 merged must be tied together by a merge commit that has both of them
 as its parents.
-The rest of this section describes this "True merge" case.
-
-The chosen merge strategy merges the two commits into a single
-new source tree.
-When things merge cleanly, this is what happens:
-
-1. The results are updated both in the index file and in your
-   working tree;
-2. Index file is written out as a tree;
-3. The tree gets committed; and
-4. The `HEAD` pointer gets advanced.
 
-Because of 2., we require that the original state of the index
-file matches exactly the current `HEAD` commit; otherwise we
-will write out your local changes already registered in your
-index file along with the merge result, which is not good.
-Because 1. involves only those paths differing between your
-branch and the branch you are merging
-(which is typically a fraction of the whole tree), you can
-have local modifications in your working tree as long as they do
-not overlap with what the merge updates.
+A merged version reconciling the changes from all branches to be
+merged is committed, and your `HEAD`, index, and working tree are
+updated to it.  It is possible to have modifications in the working
+tree as long as they do not overlap; the update will preserve them.
 
-When there are conflicts, the following happens:
+When it is not obvious how to reconcile the changes, the following
+happens:
 
-1. `HEAD` stays the same.
-
-2. Cleanly merged paths are updated both in the index file and
+1. The `HEAD` pointer stays the same.
+2. The `MERGE_HEAD` ref is set to point to the other branch head.
+3. Paths that merged cleanly are updated both in the index file and
    in your working tree.
-
-3. For conflicting paths, the index file records up to three
-   versions; stage1 stores the version from the common ancestor,
-   stage2 from `HEAD`, and stage3 from the other branch (you
+4. For conflicting paths, the index file records up to three
+   versions: stage 1 stores the version from the common ancestor,
+   stage 2 from `HEAD`, and stage 3 from `MERGE_HEAD` (you
    can inspect the stages with `git ls-files -u`).  The working
    tree files contain the result of the "merge" program; i.e. 3-way
-   merge results with familiar conflict markers `<<< === >>>`.
-
-4. No other changes are done.  In particular, the local
+   merge results with familiar conflict markers `<<<` `===` `>>>`.
+5. No other changes are made.  In particular, the local
    modifications you had before you started merge will stay the
    same and the index entries for them stay as they were,
    i.e. matching `HEAD`.
-- 
1.6.6

^ permalink raw reply related

* Саня *** Ti-3 *** Бурим добавил Вас в друзья на сайте ВКонтакте.ру
From: ВКонтакте.ру @ 2010-01-23 10:03 UTC (permalink / raw)
  To: Здравствуйте

Здравствуйте,

Саня *** Ti-3 *** Бурим добавил Вас в друзья на сайте ВКонтакте.ру

Вы можете зайти на сайт и просмотреть страницы Ваших друзей, используя
Ваш e-mail и автоматически созданный пароль: 1qhVbtWv

ВКонтакте.ру - сайт, который ежедневно позволяет десяткам миллионов людей находить старых друзей и оставаться с ними на связи, делиться фотографиями
и событиями из жизни.

Чтобы войти на сайт, пройдите по ссылке:
http://vk.com/login.php?regemail=git@vger.kernel.org#1qhVbtWv

Внимание: Ваша регистрация не будет активирована, если Вы проигнорируете
это приглашение.

Желаем удачи!
С уважением,
Администрация ВКонтакте.ру

^ permalink raw reply

* [PATCHv4 1/5] git-gui: handle non-standard worktree locations
From: Giuseppe Bilotta @ 2010-01-23 10:03 UTC (permalink / raw)
  To: git; +Cc: Shawn O. Pearce, Markus Heidelberg, Giuseppe Bilotta
In-Reply-To: <1264241018-6616-1-git-send-email-giuseppe.bilotta@gmail.com>

Don't rely on the git worktree being the updir of the gitdir, since it
might not be. Instead, define (and use) a new _gitworktree global
variable, setting it to $GIT_WORK_TREE if present, falling back to
core.worktree if defined, and finally to whatever we guess the correct
worktree is. Getting core.worktree requires the config from the alleged
git dir _gitdir to be loaded early.

Supporting non-standard worktree locations also breaks the git-gui
assumption (made when calling gitk) that the worktree was the dirname of
$_gitdir and that, by consequence, the git dir could be set to the tail
of $_gitdir once we changed to the worktree root directory. Therefore,
we need to export a GIT_DIR environment variable set to the full,
normalized path of $_gitdir instead. We also skip changing to the worktree
directory if it's empty (i.e. if we're working on a bare repository).

Signed-off-by: Giuseppe Bilotta <giuseppe.bilotta@gmail.com>
---
 git-gui/git-gui.sh |   37 ++++++++++++++++++++++++++++---------
 1 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/git-gui/git-gui.sh b/git-gui/git-gui.sh
index 718277a..7e633f4 100755
--- a/git-gui/git-gui.sh
+++ b/git-gui/git-gui.sh
@@ -121,6 +121,7 @@ unset oguimsg
 
 set _appname {Git Gui}
 set _gitdir {}
+set _gitworktree {}
 set _gitexec {}
 set _githtmldir {}
 set _reponame {}
@@ -1090,13 +1091,25 @@ if {![file isdirectory $_gitdir]} {
 	error_popup [strcat [mc "Git directory not found:"] "\n\n$_gitdir"]
 	exit 1
 }
+# _gitdir exists, so try loading the config
+load_config 0
+apply_config
+# try to set work tree from environment, falling back to core.worktree
+if {[catch { set _gitworktree $env(GIT_WORK_TREE) }]} {
+	set _gitworktree [get_config core.worktree]
+}
 if {$_prefix ne {}} {
-	regsub -all {[^/]+/} $_prefix ../ cdup
+	if {$_gitworktree eq {}} {
+		regsub -all {[^/]+/} $_prefix ../ cdup
+	} else {
+		set cdup $_gitworktree
+	}
 	if {[catch {cd $cdup} err]} {
 		catch {wm withdraw .}
 		error_popup [strcat [mc "Cannot move to top of working directory:"] "\n\n$err"]
 		exit 1
 	}
+	set _gitworktree [pwd]
 	unset cdup
 } elseif {![is_enabled bare]} {
 	if {[lindex [file split $_gitdir] end] ne {.git}} {
@@ -1104,11 +1117,15 @@ if {$_prefix ne {}} {
 		error_popup [strcat [mc "Cannot use funny .git directory:"] "\n\n$_gitdir"]
 		exit 1
 	}
-	if {[catch {cd [file dirname $_gitdir]} err]} {
+	if {$_gitworktree eq {}} {
+		set _gitworktree [file dirname $_gitdir]
+	}
+	if {[catch {cd $_gitworktree} err]} {
 		catch {wm withdraw .}
-		error_popup [strcat [mc "No working directory"] " [file dirname $_gitdir]:\n\n$err"]
+		error_popup [strcat [mc "No working directory"] " $_gitworktree:\n\n$err"]
 		exit 1
 	}
+	set _gitworktree [pwd]
 }
 set _reponame [file split [file normalize $_gitdir]]
 if {[lindex $_reponame end] eq {.git}} {
@@ -1921,6 +1938,7 @@ proc incr_font_size {font {amt 1}} {
 set starting_gitk_msg [mc "Starting gitk... please wait..."]
 
 proc do_gitk {revs} {
+	global _gitworktree
 	# -- Always start gitk through whatever we were loaded with.  This
 	#    lets us bypass using shell process on Windows systems.
 	#
@@ -1938,8 +1956,10 @@ proc do_gitk {revs} {
 		}
 
 		set pwd [pwd]
-		cd [file dirname [gitdir]]
-		set env(GIT_DIR) [file tail [gitdir]]
+		if { $_gitworktree ne {} } {
+			cd $_gitworktree
+		}
+		set env(GIT_DIR) [file normalize [gitdir]]
 
 		eval exec $cmd $revs &
 
@@ -1958,6 +1978,7 @@ proc do_gitk {revs} {
 }
 
 proc do_explore {} {
+	global _gitworktree
 	set explorer {}
 	if {[is_Cygwin] || [is_Windows]} {
 		set explorer "explorer.exe"
@@ -1967,7 +1988,7 @@ proc do_explore {} {
 		# freedesktop.org-conforming system is our best shot
 		set explorer "xdg-open"
 	}
-	eval exec $explorer [list [file nativename [file dirname [gitdir]]]] &
+	eval exec $explorer $_gitworktree &
 }
 
 set is_quitting 0
@@ -2331,8 +2352,6 @@ proc show_less_context {} {
 ##
 ## ui construction
 
-load_config 0
-apply_config
 set ui_comm {}
 
 # -- Menu Bar
@@ -3370,7 +3389,7 @@ unset i
 set file_lists($ui_index) [list]
 set file_lists($ui_workdir) [list]
 
-wm title . "[appname] ([reponame]) [file normalize [file dirname [gitdir]]]"
+wm title . "[appname] ([reponame]) [file normalize $_gitworktree]"
 focus -force $ui_comm
 
 # -- Warn the user about environmental problems.  Cygwin's Tcl
-- 
1.6.6.rc1.295.g3a4e71

^ permalink raw reply related

* [PATCHv4 0/5] git-gui: more robust handling of fancy repos
From: Giuseppe Bilotta @ 2010-01-23 10:03 UTC (permalink / raw)
  To: git; +Cc: Shawn O. Pearce, Markus Heidelberg, Giuseppe Bilotta

A re-issue of the patchest to make git-gui more robust towards
non-standard repository setup, with two additional fixes.

The first three patches are unchanged from the previous iteration (still
waiting for review, actually).

The 4th sets GIT_DIR and GIT_WORK_TREE in git-gui to ensure that all
external tool invocation work in the same setup.

The last one also updates the shortcut library to make use of
_gitworktree instead of the old assumption that the worktree is the
updir of the git dir. I can't really test this patch though because
I don't have Windows.


Giuseppe Bilotta (5):
  git-gui: handle non-standard worktree locations
  git-gui: handle bare repos correctly
  git-gui: work from the .git dir
  git-gui: set GIT_DIR and GIT_WORK_TREE after setup
  git-gui: update shortcut tools to use _gitworktree

 git-gui/git-gui.sh       |  101 ++++++++++++++++++++++++++++++++++------------
 git-gui/lib/shortcut.tcl |    7 ++-
 2 files changed, 79 insertions(+), 29 deletions(-)

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox