FastCGI support in gitweb

FastCGI support in gitweb

[Juan Jose Comellas]

I've seen that the current version of Gitweb (1.5.4.2) does not
support FastCGI, and looking in Google I found that about a year ago a
patch was sent to the list adding this functionality to it. I couldn't
find any additional emails indicating why they weren't accepted. Is
there any plan to add FastCGI support to Gitweb?

Re: FastCGI support in gitweb

[Jakub Narebski]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

"Juan Jose Comellas" <juanjo@comellas.org> writes:

> I've seen that the current version of Gitweb (1.5.4.2) does not
> support FastCGI, and looking in Google I found that about a year ago a
> patch was sent to the list adding this functionality to it. I couldn't
> find any additional emails indicating why they weren't accepted. Is
> there any plan to add FastCGI support to Gitweb?

First, as far as I understand it, the patch made gitweb had to be used
with FastCGI server.  It has no comments in code, and if I remember
correctly the commit message was also fairly nonexistent.  It mixes
FastCGI changes with site-wide changes.  It wrapped the whole file in
"until last request" loop instead of wrapping dispatch only.

Also I am not sure if gitweb doesn't rely on variables being correctly
set which they are not in FastCGI mode.

But most important part: it was never resend (IIRC it was send when
gitweb development was in separate repository)


I have attached the FastCGI patch and CC-ed the author.

[-- Attachment #2: gitweb-FastCGI.patch --]
[-- Type: text/plain, Size: 4373 bytes --]

From: root <root@vgasm.watts.utsl.gen.nz>
Date: Wed, 22 Mar 2006 00:53:07 +0000 (+1200)
Subject: add support for FastCGI
X-Git-Url: http://utsl.gen.nz/gitweb/?p=gitweb;a=commitdiff;h=56d7d436644ab296155a697552ea1345f2701620

add support for FastCGI
---

--- a/gitweb.cgi
+++ b/gitweb.cgi
@@ -5,10 +5,22 @@
 # (C) 2005, Kay Sievers <kay.sievers@vrfy.org>
 # (C) 2005, Christian Gierke <ch@gierke.de>
 #
+# FastCGI adaptations:
+#   (c) 2006, Sam Vilain <sam.vilain@catalyst.net.nz>
+#
 # This program is licensed under the GPLv2
 
+# 
+
 use strict;
 use warnings;
+
+my $last_request = sub { 1 };
+my $pre_dispatch_hook = sub { };
+my $post_dispatch_hook = sub { };
+my $CGI = "CGI";
+my $pre_listen_hook = sub { };
+
 use CGI qw(:standard :escapeHTML -nosticky);
 use CGI::Util qw(unescape);
 use CGI::Carp qw(fatalsToBrowser);
@@ -16,7 +28,34 @@ use Encode;
 use Fcntl ':mode';
 binmode STDOUT, ':utf8';
 
-my $cgi = new CGI;
+    if (@ARGV) {
+	require Getopt::Long;
+	Getopt::Long::GetOptions
+	    ( "fastcgi|f" => sub {
+		  require CGI::Fast;
+		  $CGI = "CGI::Fast";
+		  my $c;
+		  # let each child service 100 requests
+		  $last_request = sub { ++$c > 100 }
+	      },
+              "nproc|n=i"   => sub {
+		  my ($arg, $val) = @_;
+		  require FCGI::ProcManager;
+                  my $pm = FCGI::ProcManager->new({
+			n_processes => $val,
+                  });
+	          $pre_listen_hook    = sub { $pm->pm_manage        };
+	          $pre_dispatch_hook  = sub { $pm->pm_pre_dispatch  };
+	          $post_dispatch_hook = sub { $pm->pm_post_dispatch };
+              },
+            );
+    }
+
+$pre_listen_hook->();
+do {
+my $cgi = $CGI->new or last;
+$pre_dispatch_hook->();
+
 my $version =		"264";
 my $my_url =		$cgi->url();
 my $my_uri =		$cgi->url(-absolute => 1);
@@ -24,7 +63,7 @@ my $rss_link =		"";
 
 # absolute fs-path which will be prepended to the project path
 #my $projectroot =	"/pub/scm";
-my $projectroot =	"/home/kay/public_html/pub/scm";
+my $projectroot =	"/var/lib/git";
 
 # location of the git-core binaries
 my $gitbin =		"/usr/bin";
@@ -40,7 +79,7 @@ my $home_text =		"indextext.html";
 
 # source of projects list
 #my $projects_list =	$projectroot;
-my $projects_list =	"index/index.aux";
+my $projects_list =	"/var/lib/git";
 
 # input validation and dispatch
 my $action = $cgi->param('a');
@@ -51,10 +90,10 @@ if (defined $action) {
 	}
 	if ($action eq "git-logo.png") {
 		git_logo();
-		exit;
+	goto out;
 	} elsif ($action eq "opml") {
 		git_opml();
-		exit;
+	goto out;
 	}
 }
 
@@ -85,7 +124,7 @@ if (defined $project) {
 	$ENV{'GIT_DIR'} = "$projectroot/$project";
 } else {
 	git_project_list();
-	exit;
+	goto out;
 }
 
 my $file_name = $cgi->param('f');
@@ -154,61 +193,62 @@ sub validate_input {
 
 if (!defined $action || $action eq "summary") {
 	git_summary();
-	exit;
+	goto out;
 } elsif ($action eq "heads") {
 	git_heads();
-	exit;
+	goto out;
 } elsif ($action eq "tags") {
 	git_tags();
-	exit;
+	goto out;
 } elsif ($action eq "blob") {
 	git_blob();
-	exit;
+	goto out;
 } elsif ($action eq "blob_plain") {
 	git_blob_plain();
-	exit;
+	goto out;
 } elsif ($action eq "tree") {
 	git_tree();
-	exit;
+	goto out;
 } elsif ($action eq "rss") {
 	git_rss();
-	exit;
+	goto out;
 } elsif ($action eq "commit") {
 	git_commit();
-	exit;
+	goto out;
 } elsif ($action eq "log") {
 	git_log();
-	exit;
+	goto out;
 } elsif ($action eq "blobdiff") {
 	git_blobdiff();
-	exit;
+	goto out;
 } elsif ($action eq "blobdiff_plain") {
 	git_blobdiff_plain();
-	exit;
+	goto out;
 } elsif ($action eq "commitdiff") {
 	git_commitdiff();
-	exit;
+	goto out;
 } elsif ($action eq "commitdiff_plain") {
 	git_commitdiff_plain();
-	exit;
+	goto out;
 } elsif ($action eq "history") {
 	git_history();
-	exit;
+	goto out;
 } elsif ($action eq "search") {
 	git_search();
-	exit;
+	goto out;
 } elsif ($action eq "shortlog") {
 	git_shortlog();
-	exit;
+	goto out;
 } elsif ($action eq "tag") {
 	git_tag();
-	exit;
+	goto out;
 } else {
 	undef $action;
 	die_error(undef, "Unknown action.");
-	exit;
 }
 
+$post_dispatch_hook->();
+
 # quote unsafe chars, but keep the slash, even when it's not
 # correct, but quoted slashes look too horrible in bookmarks
 sub esc_param {
@@ -2405,3 +2445,7 @@ sub git_shortlog {
 	print "</table\n>";
 	git_footer_html();
 }
+
+out:
+    exit;
+} until ( $last_request->() );

[-- Attachment #3: Type: text/plain, Size: 45 bytes --]


-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: FastCGI support in gitweb

[Sam Vilain]

Jakub Narebski wrote:
> First, as far as I understand it, the patch made gitweb had to be used
> with FastCGI server.  It has no comments in code, and if I remember
> correctly the commit message was also fairly nonexistent.  It mixes
> FastCGI changes with site-wide changes.  It wrapped the whole file in
> "until last request" loop instead of wrapping dispatch only.
> 
> Also I am not sure if gitweb doesn't rely on variables being correctly
> set which they are not in FastCGI mode.
> 
> But most important part: it was never resend (IIRC it was send when
> gitweb development was in separate repository)
> 
> 
> I have attached the FastCGI patch and CC-ed the author.

Thanks.  I didn't submit this because I couldn't fix the bugs in it.
Glad you found it.  I had to make many changes in a similar vein with a
current gitweb version; did it work for you in its current form?

You can get it to run externally using FCGI_SOCKET=:3000 (eg, to listen
on FastCGI TCP port 3000)

Sam.

Re: FastCGI support in gitweb

[Jakub Narebski]

On Mon, 3 Mar 2008, Sam Vilain wrote:
> Jakub Narebski wrote:
>> First, as far as I understand it, the patch made gitweb had to be used
>> with FastCGI server.  It has no comments in code, and if I remember
>> correctly the commit message was also fairly nonexistent.  It mixes
>> FastCGI changes with site-wide changes.  It wrapped the whole file in
>> "until last request" loop instead of wrapping dispatch only.
>> 
>> Also I am not sure if gitweb doesn't rely on variables being correctly
>> set which they are not in FastCGI mode.
>> 
>> But most important part: it was never resend (IIRC it was send when
>> gitweb development was in separate repository)
>> 
>> 
>> I have attached the FastCGI patch and CC-ed the author.
> 
> Thanks.  I didn't submit this because I couldn't fix the bugs in it.
> Glad you found it.  I had to make many changes in a similar vein with a
> current gitweb version; did it work for you in its current form?

I couldn't test it with FastCGI server, as I use Apache2, and all
FastCGI modules for Apache are third part modules. Besides, it doesn't
make much sense to use generic FastCGI for Perl, when there is mod_perl
module.

For me for the FastCGI change to be made into mainline it would have
to have the following properties:
 * it should be able to run as both CGI module, and under mod_perl
   module (in legacy mode with ModPerl::Registry) without trouble
 * is should be able to run even if CGI::Fast or FCGI Perl modules
   are not installed in the system
 * the loop over requests should try to be minimal, and not encompass
   whole file
The last condition would probably require to separate option parsing
and validation into separate subroutine.

> You can get it to run externally using FCGI_SOCKET=:3000 (eg, to listen
> on FastCGI TCP port 3000)

Could you elaborate on this?


P.S. It would be good to have examples for web servers other than
Apache2 how to configure them to run gitweb: perhaps lighthttpd, Cheetah,
maybe IIS...

-- 
Jakub Narebski
Poland

Re: FastCGI support in gitweb

[John Goerzen]

On 2008-03-04, Jakub Narebski <jnareb@gmail.com> wrote:
>> Thanks.  I didn't submit this because I couldn't fix the bugs in it.
>> Glad you found it.  I had to make many changes in a similar vein with a
>> current gitweb version; did it work for you in its current form?
>
> I couldn't test it with FastCGI server, as I use Apache2, and all
> FastCGI modules for Apache are third part modules. Besides, it doesn't

On Debian, this was as easy as apt-get install libapache2-mod-fcgid

> make much sense to use generic FastCGI for Perl, when there is mod_perl
> module.

Sure it can.  I currently am running both Python and Ruby code under
FastCGI.  Adding mod_perl for just one program increases my
webserver's memory footprint, potentially dramatically, and increases
my complexity as well.  I'd much rather run FastCGI than mod_perl.

Re: FastCGI support in gitweb

[Jakub Narebski]

John Goerzen <jgoerzen@complete.org> writes:
> On 2008-03-04, Jakub Narebski <jnareb@gmail.com> wrote:
>>
>>> Thanks.  I didn't submit this because I couldn't fix the bugs in it.
>>> Glad you found it.  I had to make many changes in a similar vein with a
>>> current gitweb version; did it work for you in its current form?
>>
>> I couldn't test it with FastCGI server, as I use Apache2, and all
>> FastCGI modules for Apache are third part modules.
> 
> On Debian, this was as easy as apt-get install libapache2-mod-fcgid

And for Fedora Core?

>From what I have found there exists three 3rd-party (i.e. not
developed by Apache) modules for FastCGI support: mod_fastcgi,
mod_fcgi and mod_proxy_fcgi. Which one to choose?
 
>> Besides, it doesn't
>> make much sense to use generic FastCGI for Perl, when there is mod_perl
>> module.
> 
> Sure it can.  I currently am running both Python and Ruby code under
> FastCGI.  Adding mod_perl for just one program increases my
> webserver's memory footprint, potentially dramatically, and increases
> my complexity as well.  I'd much rather run FastCGI than mod_perl.

Well, if you are running FastCGI for other scripts, it makes sense
then.

Although... doesn't there exists modules for Python (mod_python,
mod_wsgi, mod_snake) and for Ruby (mod_ruby)?
-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: FastCGI support in gitweb

[John Goerzen]

On 2008-03-04, Jakub Narebski <jnareb@gmail.com> wrote:
>> On Debian, this was as easy as apt-get install libapache2-mod-fcgid
>
> And for Fedora Core?
>
> From what I have found there exists three 3rd-party (i.e. not
> developed by Apache) modules for FastCGI support: mod_fastcgi,
> mod_fcgi and mod_proxy_fcgi. Which one to choose?

I like mod_fcgid, but that's just me.

>> Sure it can.  I currently am running both Python and Ruby code under
>> FastCGI.  Adding mod_perl for just one program increases my
>> webserver's memory footprint, potentially dramatically, and increases
>> my complexity as well.  I'd much rather run FastCGI than mod_perl.
>
> Well, if you are running FastCGI for other scripts, it makes sense
> then.
>
> Although... doesn't there exists modules for Python (mod_python,
> mod_wsgi, mod_snake) and for Ruby (mod_ruby)?

Sure.  But think of the horrendous memory footprint if I have mod_php,
mod_python, mod_perl, and mod_ruby all loaded into Apache at once!  (I
do have mod_php in my installation)  Remember that if it is configured
that way, *each* Apache process/thread carries the module for PHP,
Python, Perl, AND Ruby, even if it uses none of them.  I've been
there, done that, and it's not pretty.

FastCGI is much, much lighter on resource requirements if you are
deploying apps written in various languages on a single server.
Sometimes even if you aren't.

-- John

Re: FastCGI support in gitweb

[Jakub Narebski]

The following message is a courtesy copy of an article
that has been posted to gmane.comp.version-control.git as well.

[Cc: Sam Vilain, Juan Jose Comellas]

John Goerzen <jgoerzen@complete.org> writes:

> FastCGI is much, much lighter on resource requirements if you are
> deploying apps written in various languages on a single server.
> Sometimes even if you aren't.

If you want to add FastCGI support to gitweb, besides fulfilling
constraints I have mentioned earlier (it must run as CGI and mod_perl,
must run without CGI::Fast / FCGI), I think it would be best to
separate input validation and dispatch into separate subroutine.

Perhaps even going as far as getting only required input in the
"action" routines, for example
  our ($hash, $hash_base) = gitweb_params('hash', 'hash_base');
at the beginning of git_commit() subroutine, etc.

-- 
Jakub Narebski
Poland
ShadeHawk on #git