[PATCH 0/3] gitweb: Improve reading of repo config

[PATCH 0/3] gitweb: Improve reading of repo config

[Jakub Narebski]

This series of patches introduces single-call reading of
gitweb-related part of repository configuration in gitweb. Instead of
calling git-config once for each config variable we check, read all
variables at once for first request of repo config variable.

First patch adds test to check if our changes do not break gitweb too
badly, second one introduces mentioned above "eager" repo config
reading, and third makes use of the fact that adding new configuration
variables to check is not that costly now.

Table of contents:
==================
 [PATCH 1/3] gitweb: Add tests for overriding gitweb config with repo config
 [PATCH 2/3] gitweb: Read repo config using 'git config -z -l'
 [PATCH 3/3] gitweb: Use config file for repository description and URLs

Diffstat:
=========
 gitweb/gitweb.perl |   82 +++++++++++++++++++++----------------------------
 1 files changed, 33 insertions(+), 47 deletions(-)

-- 
Jakub Narebski, Poland
git version 1.5.3.5

[PATCH 1/3] gitweb: Add tests for overriding gitweb config with repo config

[Jakub Narebski]

Make blame view and snapshot support overridable by repository
config. Test tree view with both features disabled, and with both
features enabled.

Test with features enabled also tests multiple formats snapshot
support (in tree view).

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
If I remember correctly it was when creating this test that there were
probles with CGI::Carp::set_programname() in t9500 test. And it took
me some time to discover where the bug was.

 t/t9500-gitweb-standalone-no-errors.sh |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 1bf0988..35fff3d 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -557,4 +557,27 @@ test_expect_success \
 	'gitweb_run "p=.git;a=tree;opt=--no-merges"'
 test_debug 'cat gitweb.log'
 
+# ----------------------------------------------------------------------
+# gitweb config and repo config
+
+cat >>gitweb_config.perl <<EOF
+
+\$feature{'blame'}{'override'} = 1;
+\$feature{'snapshot'}{'override'} = 1;
+EOF
+
+test_expect_success \
+	'config override: tree view, features disabled in repo config' \
+	'git config gitweb.blame no &&
+	 git config gitweb.snapshot none &&
+	 gitweb_run "p=.git;a=tree"'
+test_debug 'cat gitweb.log'
+
+test_expect_success \
+	'config override: tree view, features enabled in repo config' \
+	'git config gitweb.blame yes &&
+	 git config gitweb.snapshot "zip,tgz, tbz2" &&
+	 gitweb_run "p=.git;a=tree"'
+test_debug 'cat gitweb.log'
+
 test_done
-- 
1.5.3.5

[PATCH 2/3] gitweb: Read repo config using 'git config -z -l'

[Jakub Narebski]

Change git_get_project_config to run git-config only once per
repository, without changing its signature (its calling convention).
This means for example that it returns 'true' or 'false' when called
with second argument '--bool', and not true or false value.

Instead of calling 'git config [<type>] --get gitweb.<key>' once for
each config variable, call 'git config -z -l' only once, parsing and
saving its output to %config variable.  This makes possible to add new
per repository configuration without paying cost of forking once per
variable checked.  We can now allow repository description and
repository URLs to be stored in config file without badly affecting
gitweb performance.

For now only configuration variables for 'gitweb' section are stored.

Multiple values for single configuration variable are stored as
anonymous array reference; configuration variable with no value is
stored as undef.

Converting configuration variable values to boolean or integer value
are done in Perl.  Results differ from git-config in the fact that no
conversion error is ever raised.  For boolean values no value, 'true'
(any case) and 'false' (any case) are considered true, numbers are
true if not zero; all other values (even invalid for bool) are
considered false.  For integer values value suffix of 'k', 'm', or 'g'
following decimal number will cause the value to be multiplied by
1024, 1048576, or 1073741824; other values are returned as-is, only
whitespace stripped.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This makes use of commit 2275d502114c71045af991697048191fed88aac4
  "config: Add --null/-z option for null-delimted output"
by Frank Lichtenheld. The --null/-z option to git-config was created
among others with gitweb in mind, if I remember correctly.

In the future we would most probably want to move this to Git.pm, and
use Git.pm in gitweb.

 gitweb/gitweb.perl |  115 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 108 insertions(+), 7 deletions(-)

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 827f977..f54455b 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1432,20 +1432,121 @@ sub git_get_type {
 	return $type;
 }
 
+# repository configuration
+our $config_file = '';
+our %config;
+
+# store multiple values for single key as anonymous array reference
+# single values stored directly in the hash, not as [ <value> ]
+sub hash_set_multi {
+	my ($hash, $key, $value) = @_;
+
+	if (!exists $hash->{$key}) {
+		$hash->{$key} = $value;
+	} elsif (!ref $hash->{$key}) {
+		$hash->{$key} = [ $hash->{$key}, $value ];
+	} else {
+		push @{$hash->{$key}}, $value;
+	}
+}
+
+# return hash of git project configuration
+# optionally limited to some section, e.g. 'gitweb'
+sub git_parse_project_config {
+	my $section_regexp = shift;
+	my %config;
+
+	local $/ = "\0";
+
+	open my $fh, "-|", git_cmd(), "config", '-z', '-l',
+		or return;
+
+	while (my $keyval = <$fh>) {
+		chomp $keyval;
+		my ($key, $value) = split(/\n/, $keyval, 2);
+
+		hash_set_multi(\%config, $key, $value)
+			if (!defined $section_regexp || $key =~ /^(?:$section_regexp)\./o);
+	}
+	close $fh;
+
+	return %config;
+}
+
+# convert config value to boolean, 'true' or 'false'
+# no value, number > 0, 'true' and 'yes' values are true
+# rest of values are treated as false (never as error)
+sub config_to_bool {
+	my $val = shift;
+
+	# strip leading and trailing whitespace
+	$val =~ s/^\s+//;
+	$val =~ s/\s+$//;
+
+	return (!defined $val ||               # section.key
+	        ($val =~ /^\d+$/ && $val) ||   # section.key = 1
+	        ($val =~ /^(?:true|yes)$/i));  # section.key = true
+}
+
+# convert config value to simple decimal number
+# an optional value suffix of 'k', 'm', or 'g' will cause the value
+# to be multiplied by 1024, 1048576, or 1073741824
+sub config_to_int {
+	my $val = shift;
+
+	# strip leading and trailing whitespace
+	$val =~ s/^\s+//;
+	$val =~ s/\s+$//;
+
+	if (my ($num, $unit) = ($val =~ /^([0-9]*)([kmg])$/i)) {
+		$unit = lc($unit);
+		# unknown unit is treated as 1
+		return $num * ($unit eq 'g' ? 1073741824 :
+		               $unit eq 'm' ?    1048576 :
+		               $unit eq 'k' ?       1024 : 1);
+	}
+	return $val;
+}
+
+# convert config value to array reference, if needed
+sub config_to_multi {
+	my $val = shift;
+
+	return ref($val) ? $val : [ $val ];
+}
+
 sub git_get_project_config {
 	my ($key, $type) = @_;
 
+	# key sanity check
 	return unless ($key);
 	$key =~ s/^gitweb\.//;
 	return if ($key =~ m/\W/);
 
-	my @x = (git_cmd(), 'config');
-	if (defined $type) { push @x, $type; }
-	push @x, "--get";
-	push @x, "gitweb.$key";
-	my $val = qx(@x);
-	chomp $val;
-	return ($val);
+	# type sanity check
+	if (defined $type) {
+		$type =~ s/^--//;
+		$type = undef
+			unless ($type eq 'bool' || $type eq 'int');
+	}
+
+	# get config
+	if (!defined $config_file ||
+	    $config_file ne "$git_dir/config") {
+		%config = git_parse_project_config('gitweb');
+		$config_file = "$git_dir/config";
+	}
+
+	# ensure given type
+	if (!defined $type) {
+		return $config{"gitweb.$key"};
+	} elsif ($type eq 'bool') {
+		# backward compatibility: 'git config --bool' returns true/false
+		return config_to_bool($config{"gitweb.$key"}) ? 'true' : 'false';
+	} elsif ($type eq 'int') {
+		return config_to_int($config{"gitweb.$key"});
+	}
+	return $config{"gitweb.$key"};
 }
 
 # get hash of given path at given ref
-- 
1.5.3.5

[PATCH 3/3] gitweb: Use config file for repository description and URLs

[Jakub Narebski]

Allow to use configuration variable gitweb.description for repository
description if there is no $GIT_DIR/description file, and multivalued
configuration variable gitweb.url for URLs of a project (to clone or
fetch from) if there is no $GIT_DIR/cloneurl file.

While repository description is shown in the projects list page, so it
is better to use file and not config variable for performance, it is I
think better to use gitweb.url for URLs (as it is shown only on
project summary page).

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
 gitweb/gitweb.perl                     |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index f54455b..7cac629 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1606,7 +1606,9 @@ sub git_get_path_by_hash {
 sub git_get_project_description {
 	my $path = shift;
 
-	open my $fd, "$projectroot/$path/description" or return undef;
+	$git_dir = "$projectroot/$path";
+	open my $fd, "$projectroot/$path/description"
+		or return git_get_project_config('description');
 	my $descr = <$fd>;
 	close $fd;
 	if (defined $descr) {
@@ -1618,7 +1620,11 @@ sub git_get_project_description {
 sub git_get_project_url_list {
 	my $path = shift;
 
-	open my $fd, "$projectroot/$path/cloneurl" or return;
+	$git_dir = "$projectroot/$path";
+	open my $fd, "$projectroot/$path/cloneurl"
+		or return wantarray ?
+		@{ config_to_multi(git_get_project_config('url')) } :
+		   config_to_multi(git_get_project_config('url'));
 	my @git_project_url_list = map { chomp; $_ } <$fd>;
 	close $fd;
 
-- 
1.5.3.5