[PATCH/RFC] gitweb: selectable configurations that change with each request

[PATCH/RFC] gitweb: selectable configurations that change with each request

[Jakub Narebski]

Allow selecting whether configuration file should be (re)parsed on each
request (the default, for backward compatibility with configurations that
change per session, see commit 7f425db (gitweb: allow configurations that
change with each request, 2010-07-30)), or whether should it be parsed only
once (for performance speedup for persistent environments, though currently
only FastCGI is able to make use of it, when flexibility is not important).
You can also have configuration file parsed only once, but have parts of
configuration (re)evaluated once per each request.

This is done by introducing $per_request_config variable: if set to code
reference, this code would be run once per request, while config file would
be parsed only once.  For example gitolite's contrib/gitweb/gitweb.conf
fragment mentioned in 7f425db could be rewritten as

  our $per_request_config = sub {
  	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
  };

to make use of this feature.

If $per_request_config is not a code reference, it is taken to be boolean
variable, to choose between running config file for each request
(flexibility), and running config file only once (performance in
persistent environments).

The default value for $per_request_config is 1 (true), which means that
old configuration that require to change per session (like gitolite's)
will keep working.


Because there is a call to evaluate_gitweb_config() in the run() subroutine
(the call in run_config() is not invoked at first request to avoid double
running it), therefore evaluate_git_version() could be moved back there, and
is now evaluated only once.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Commit 7f425db914 says: 

    Probably in the end there should be a way to specify in the
    configuration whether a particular installation wants the speedup or
    the flexibility.  But for now it is easier to just undo the relevant
    change.

This is the way... well, at least a proposal.

 gitweb/README      |    7 +++++++
 gitweb/gitweb.perl |   23 +++++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)

diff --git a/gitweb/README b/gitweb/README
index bf3664f..6646fda 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -246,6 +246,13 @@ not include variables usually directly set during build):
    http://www.andre-simon.de due to assumptions about parameters and output).
    Useful if highlight is not installed on your webserver's PATH.
    [Default: highlight]
+ * $per_request_config
+   If set to code reference, it would be run once per each request.  You can
+   set parts of configuration that change per session, e.g. by setting it to
+     sub { $ENV{GL_USER} = $cgi->remote_user || "gitweb"; }
+   Otherwise it is treated as boolean value: if true gitweb would process
+   config file once per request, if false it would process config file only
+   once.  The default is true.
 
 Projects list file format
 ~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7dbaf0c..c3cfefb 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -599,6 +599,14 @@ sub filter_snapshot_fmts {
 		!$known_snapshot_formats{$_}{'disabled'}} @fmts;
 }
 
+# If it is set to code reference, it is code that it is to be run once per
+# request, allowing updating configurations that change with each request,
+# while running other code in config file only once.
+#
+# Otherwise, if it is false then gitweb would process config file only once;
+# if it is true then gitweb config would be run for each request.
+our $per_request_config = 1;
+
 our ($GITWEB_CONFIG, $GITWEB_CONFIG_SYSTEM);
 sub evaluate_gitweb_config {
 	our $GITWEB_CONFIG = $ENV{'GITWEB_CONFIG'} || "++GITWEB_CONFIG++";
@@ -1068,12 +1076,18 @@ sub reset_timer {
 	our $number_of_git_cmds = 0;
 }
 
+our $first_request = 1;
 sub run_request {
 	reset_timer();
 
 	evaluate_uri();
-	evaluate_gitweb_config();
-	evaluate_git_version();
+	if ($per_request_config) {
+		if (ref($per_request_config) eq 'CODE') {
+			$per_request_config->();
+		} elsif (!$first_request) {
+			evaluate_gitweb_config();
+		}
+	}
 	check_loadavg();
 
 	# $projectroot and $projects_list might be set in gitweb config file
@@ -1126,6 +1140,10 @@ sub evaluate_argv {
 
 sub run {
 	evaluate_argv();
+	evaluate_gitweb_config();
+	evaluate_git_version();
+
+	$first_request = 1;
 
 	$pre_listen_hook->()
 		if $pre_listen_hook;
@@ -1139,6 +1157,7 @@ sub run {
 
 		$post_dispatch_hook->()
 			if $post_dispatch_hook;
+		$first_request = 0;
 
 		last REQUEST if ($is_last_request->());
 	}

Re: [PATCH/RFC] gitweb: selectable configurations that change with each request

[Jonathan Nieder]

Jakub Narebski wrote:

> This is done by introducing $per_request_config variable: if set to code
> reference, this code would be run once per request, while config file would
> be parsed only once.  For example gitolite's contrib/gitweb/gitweb.conf
> fragment mentioned in 7f425db could be rewritten as
> 
>   our $per_request_config = sub {
>   	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
>   };
> 
> to make use of this feature.
> 
> If $per_request_config is not a code reference, it is taken to be boolean
> variable, to choose between running config file for each request
> (flexibility), and running config file only once (performance in
> persistent environments).
> 
> The default value for $per_request_config is 1 (true)

Flexible, fast, backward-compatible.  Nice.

Re: [PATCH/RFC] gitweb: selectable configurations that change with each request

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> The default value for $per_request_config is 1 (true), which means that
> old configuration that require to change per session (like gitolite's)
> will keep working.
>
> Because there is a call to evaluate_gitweb_config() in the run() subroutine
> (the call in run_config() is not invoked at first request to avoid double
> running it), therefore evaluate_git_version() could be moved back there, and
> is now evaluated only once.
>
> Signed-off-by: Jakub Narebski <jnareb@gmail.com>

I like the "we can keep the old behaviour by default, but you can go
fancier if you wanted to" attitude of this patch.

I however am not sure if this particular implementation is regression
free.  People who wanted the per-request configuration used to be able to
access what evaluate_uri() did while running evaluate_gitweb_config(),
which in turn means that their custom configuration could take things like
my-url and path-info into account, but now for the first invocation they
cannot.  It probably is just the matter of moving some code around from
their old custom gitweb-config to a per_request_config sub they need to
write and point $per_request_config variable at, but that makes "setting 1
will keep the old behaviour" a false promise, no?

> @@ -1068,12 +1076,18 @@ sub reset_timer {
>  	our $number_of_git_cmds = 0;
>  }
>  
> +our $first_request = 1;
>  sub run_request {
>  	reset_timer();
>  
>  	evaluate_uri();
> -	evaluate_gitweb_config();
> -	evaluate_git_version();
> +	if ($per_request_config) {
> +		if (ref($per_request_config) eq 'CODE') {
> +			$per_request_config->();
> +		} elsif (!$first_request) {
> +			evaluate_gitweb_config();
> +		}
> +	}
>  	check_loadavg();
>  
>  	# $projectroot and $projects_list might be set in gitweb config file
> @@ -1126,6 +1140,10 @@ sub evaluate_argv {
>  
>  sub run {
>  	evaluate_argv();
> +	evaluate_gitweb_config();
> +	evaluate_git_version();
> +
> +	$first_request = 1;
>  
>  	$pre_listen_hook->()
>  		if $pre_listen_hook;
> @@ -1139,6 +1157,7 @@ sub run {
>  
>  		$post_dispatch_hook->()
>  			if $post_dispatch_hook;
> +		$first_request = 0;
>  
>  		last REQUEST if ($is_last_request->());
>  	}

Re: [PATCH/RFC] gitweb: selectable configurations that change with each request

[Jakub Narebski]

On Wed, 24 Nov 2010, Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> > The default value for $per_request_config is 1 (true), which means that
> > old configuration that require to change per session (like gitolite's)
> > will keep working.
> >
> > Because there is a call to evaluate_gitweb_config() in the run() subroutine
> > (the call in run_config() is not invoked at first request to avoid double
> > running it), therefore evaluate_git_version() could be moved back there, and
> > is now evaluated only once.
> >
> > Signed-off-by: Jakub Narebski <jnareb@gmail.com>
> 
> I like the "we can keep the old behaviour by default, but you can go
> fancier if you wanted to" attitude of this patch.

Thanks.

> I however am not sure if this particular implementation is regression
> free.  People who wanted the per-request configuration used to be able to
> access what evaluate_uri() did while running evaluate_gitweb_config(),
> which in turn means that their custom configuration could take things like
> my-url and path-info into account, but now for the first invocation they
> cannot.  It probably is just the matter of moving some code around from
> their old custom gitweb-config to a per_request_config sub they need to
> write and point $per_request_config variable at, but that makes "setting 1
> will keep the old behaviour" a false promise, no?

Thanks for catching this; I forgot to take into account the fact that
gitweb config changing with request would need request-related variables.

So instead of code flow looking like this

  evaluate_gitweb_config();
  $first_request = 1;

  while (...) {

  	if ($per_request_config) {
		if (ref($per_request_config) eq 'CODE') {
			$per_request_config->();
		} elsif (!$first_request) {
			evaluate_gitweb_config();
		}
	}

	$first_request = 0;

  }


we would move to

  $first_request = 1;
  while (...) {

  	if ($first_request) {
		evaluate_gitweb_config();
	} elsif ($per_request_config) {
		if (ref($per_request_config) eq 'CODE') {
			$per_request_config->();
		} else {
  			evaluate_gitweb_config();
  		}
	}
	$first_request = 0;

  }


I'll send new version soon.


P.S. if we could assume that nobody ever edits gitweb.cgi directly
to set $per_request_config to non-default value, we could probably
get rid of $first_request variable...
 
> > @@ -1068,12 +1076,18 @@ sub reset_timer {
> >  	our $number_of_git_cmds = 0;
> >  }
> >  
> > +our $first_request = 1;
> >  sub run_request {
> >  	reset_timer();
> >  
> >  	evaluate_uri();
> > -	evaluate_gitweb_config();
> > -	evaluate_git_version();
> > +	if ($per_request_config) {
> > +		if (ref($per_request_config) eq 'CODE') {
> > +			$per_request_config->();
> > +		} elsif (!$first_request) {
> > +			evaluate_gitweb_config();
> > +		}
> > +	}
> >  	check_loadavg();
> >  
> >  	# $projectroot and $projects_list might be set in gitweb config file
> > @@ -1126,6 +1140,10 @@ sub evaluate_argv {
> >  
> >  sub run {
> >  	evaluate_argv();
> > +	evaluate_gitweb_config();
> > +	evaluate_git_version();
> > +
> > +	$first_request = 1;
> >  
> >  	$pre_listen_hook->()
> >  		if $pre_listen_hook;
> > @@ -1139,6 +1157,7 @@ sub run {
> >  
> >  		$post_dispatch_hook->()
> >  			if $post_dispatch_hook;
> > +		$first_request = 0;
> >  
> >  		last REQUEST if ($is_last_request->());
> >  	}
> 

-- 
Jakub Narebski
Poland

[PATCHv2] gitweb: selectable configurations that change with each request

[Jakub Narebski]

Allow selecting whether configuration file should be (re)parsed on each
request (the default, for backward compatibility with configurations that
change per session, see commit 7f425db (gitweb: allow configurations that
change with each request, 2010-07-30)), or whether should it be parsed only
once (for performance speedup for persistent environments, though currently
only FastCGI is able to make use of it, when flexibility is not important).

You can also have configuration file parsed only once, but have parts of
configuration (re)evaluated once per each request.

This is done by introducing $per_request_config variable: if set to code
reference, this code would be run once per request, while config file would
be parsed only once.  For example gitolite's contrib/gitweb/gitweb.conf
fragment mentioned in 7f425db could be rewritten as

  our $per_request_config = sub {
  	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
  };

to make use of this feature.

If $per_request_config is not a code reference, it is taken to be boolean
variable, to choose between running config file for each request
(flexibility), and running config file only once (performance in
persistent environments).

The default value for $per_request_config is 1 (true), which means that
old configuration that require to change per session (like gitolite's)
will keep working.


While at it, make it so evaluate_git_version() is run only once.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
On Wed, 24 Nov 2010, Jakub Narebski wrote:

> I'll send new version soon.
> 

And here it is, rebased on top of current 'master'.

 gitweb/README      |    7 +++++++
 gitweb/gitweb.perl |   23 +++++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)

diff --git a/gitweb/README b/gitweb/README
index bf3664f..6646fda 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -246,6 +246,13 @@ not include variables usually directly set during build):
    http://www.andre-simon.de due to assumptions about parameters and output).
    Useful if highlight is not installed on your webserver's PATH.
    [Default: highlight]
+ * $per_request_config
+   If set to code reference, it would be run once per each request.  You can
+   set parts of configuration that change per session, e.g. by setting it to
+     sub { $ENV{GL_USER} = $cgi->remote_user || "gitweb"; }
+   Otherwise it is treated as boolean value: if true gitweb would process
+   config file once per request, if false it would process config file only
+   once.  The default is true.
 
 Projects list file format
 ~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7dbaf0c..ac38a92 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -599,6 +599,14 @@ sub filter_snapshot_fmts {
 		!$known_snapshot_formats{$_}{'disabled'}} @fmts;
 }
 
+# If it is set to code reference, it is code that it is to be run once per
+# request, allowing updating configurations that change with each request,
+# while running other code in config file only once.
+#
+# Otherwise, if it is false then gitweb would process config file only once;
+# if it is true then gitweb config would be run for each request.
+our $per_request_config = 1;
+
 our ($GITWEB_CONFIG, $GITWEB_CONFIG_SYSTEM);
 sub evaluate_gitweb_config {
 	our $GITWEB_CONFIG = $ENV{'GITWEB_CONFIG'} || "++GITWEB_CONFIG++";
@@ -1068,12 +1076,21 @@ sub reset_timer {
 	our $number_of_git_cmds = 0;
 }
 
+our $first_request = 1;
 sub run_request {
 	reset_timer();
 
 	evaluate_uri();
-	evaluate_gitweb_config();
-	evaluate_git_version();
+	if ($first_request) {
+		evaluate_gitweb_config();
+		evaluate_git_version();
+	} elsif ($per_request_config) {
+		if (ref($per_request_config) eq 'CODE') {
+			$per_request_config->();
+		} else {
+			evaluate_gitweb_config();
+		}
+	}
 	check_loadavg();
 
 	# $projectroot and $projects_list might be set in gitweb config file
@@ -1127,6 +1144,7 @@ sub evaluate_argv {
 sub run {
 	evaluate_argv();
 
+	$first_request = 1;
 	$pre_listen_hook->()
 		if $pre_listen_hook;
 
@@ -1139,6 +1157,7 @@ sub run {
 
 		$post_dispatch_hook->()
 			if $post_dispatch_hook;
+		$first_request = 0;
 
 		last REQUEST if ($is_last_request->());
 	}
-- 
1.7.3

Re: [PATCHv2] gitweb: selectable configurations that change with each request

[Jonathan Nieder]

Jakub Narebski wrote:

> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
[...]
> @@ -1068,12 +1076,21 @@ sub reset_timer {
>  	our $number_of_git_cmds = 0;
>  }
>  
> +our $first_request = 1;
>  sub run_request {
>  	reset_timer();
>  
>  	evaluate_uri();
> -	evaluate_gitweb_config();
> -	evaluate_git_version();
> +	if ($first_request) {
> +		evaluate_gitweb_config();
> +		evaluate_git_version();
> +	} elsif ($per_request_config) {
> +		if (ref($per_request_config) eq 'CODE') {
> +			$per_request_config->();
> +		} else {
> +			evaluate_gitweb_config();
> +		}
> +	}

Should per_request_config() be run for the first request, too?  Maybe:

	if ($first_request) {
		evaluate_gitweb_config();
		evaluate_git_version();
	}
	if ($per_request_config) {
		if (ref($per_request_config) eq 'CODE') {
			$per_request_config->();
		} elsif (!$first_request) {
			evaluate_gitweb_config();
		}
	}

[PATCHv3] gitweb: selectable configurations that change with each request

[Jakub Narebski]

Allow selecting whether configuration file should be (re)parsed on each
request (the default, for backward compatibility with configurations that
change per session, see commit 7f425db (gitweb: allow configurations that
change with each request, 2010-07-30)), or whether should it be parsed only
once (for performance speedup for persistent environments, though currently
only FastCGI is able to make use of it, when flexibility is not important).

You can also have configuration file parsed only once, but have parts of
configuration (re)evaluated once per each request.

This is done by introducing $per_request_config variable: if set to code
reference, this code would be run once per request, while config file would
be parsed only once.  For example gitolite's contrib/gitweb/gitweb.conf
fragment mentioned in 7f425db could be rewritten as

  our $per_request_config = sub {
  	$ENV{GL_USER} = ($cgi && $cgi->remote_user) || "gitweb";
  };

to make use of this feature.

If $per_request_config is not a code reference, it is taken to be boolean
variable, to choose between running config file for each request
(flexibility), and running config file only once (performance in
persistent environments).

The default value for $per_request_config is 1 (true), which means that
old configuration that require to change per session (like gitolite's)
will keep working.


While at it, make it so evaluate_git_version() is run only once.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
On Thu, 25 Nov 2010, Jonathan Nieder wrote:
> Should per_request_config() be run for the first request, too?  Maybe:
> 
>         if ($first_request) {
>                 evaluate_gitweb_config();
>                 evaluate_git_version();
>         }
>         if ($per_request_config) {
>                 if (ref($per_request_config) eq 'CODE') {
>                         $per_request_config->();
>                 } elsif (!$first_request) {
>                         evaluate_gitweb_config();
>                 }
>         }

You are right, thanks for pointing that out.  I'm sorry for my mistake.

 gitweb/README      |    7 +++++++
 gitweb/gitweb.perl |   24 ++++++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/gitweb/README b/gitweb/README
index bf3664f..6646fda 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -246,6 +246,13 @@ not include variables usually directly set during build):
    http://www.andre-simon.de due to assumptions about parameters and output).
    Useful if highlight is not installed on your webserver's PATH.
    [Default: highlight]
+ * $per_request_config
+   If set to code reference, it would be run once per each request.  You can
+   set parts of configuration that change per session, e.g. by setting it to
+     sub { $ENV{GL_USER} = $cgi->remote_user || "gitweb"; }
+   Otherwise it is treated as boolean value: if true gitweb would process
+   config file once per request, if false it would process config file only
+   once.  The default is true.
 
 Projects list file format
 ~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7dbaf0c..1d94718 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -599,6 +599,14 @@ sub filter_snapshot_fmts {
 		!$known_snapshot_formats{$_}{'disabled'}} @fmts;
 }
 
+# If it is set to code reference, it is code that it is to be run once per
+# request, allowing updating configurations that change with each request,
+# while running other code in config file only once.
+#
+# Otherwise, if it is false then gitweb would process config file only once;
+# if it is true then gitweb config would be run for each request.
+our $per_request_config = 1;
+
 our ($GITWEB_CONFIG, $GITWEB_CONFIG_SYSTEM);
 sub evaluate_gitweb_config {
 	our $GITWEB_CONFIG = $ENV{'GITWEB_CONFIG'} || "++GITWEB_CONFIG++";
@@ -1068,12 +1076,22 @@ sub reset_timer {
 	our $number_of_git_cmds = 0;
 }
 
+our $first_request = 1;
 sub run_request {
 	reset_timer();
 
 	evaluate_uri();
-	evaluate_gitweb_config();
-	evaluate_git_version();
+	if ($first_request) {
+		evaluate_gitweb_config();
+		evaluate_git_version();
+	}
+	if ($per_request_config) {
+		if (ref($per_request_config) eq 'CODE') {
+			$per_request_config->();
+		} elsif (!$first_request) {
+			evaluate_gitweb_config();
+		}
+	}
 	check_loadavg();
 
 	# $projectroot and $projects_list might be set in gitweb config file
@@ -1127,6 +1145,7 @@ sub evaluate_argv {
 sub run {
 	evaluate_argv();
 
+	$first_request = 1;
 	$pre_listen_hook->()
 		if $pre_listen_hook;
 
@@ -1139,6 +1158,7 @@ sub run {
 
 		$post_dispatch_hook->()
 			if $post_dispatch_hook;
+		$first_request = 0;
 
 		last REQUEST if ($is_last_request->());
 	}
-- 
1.7.3