Re: gitweb, FastCGI and PSGI/Plack

Re: gitweb, FastCGI and PSGI/Plack

[Jakub Narebski]

On Sun, 9 May 2010, Tatsuhiko Miyagawa wrote:

> I'm not subscribed to the git mailing list but:
> 
> http://www.spinics.net/lists/git/msg131014.html

You don't need to be subscribed to post to git mailing list.  I personally
read it with a news reader using GMane NNTP interface

  nntp://news.gmane.org/gmane.comp.version-control.git

but sending messages via email, to git@ver.kernel.org.  Usually people
reply both to original poster and to git mailing list, but just in case
you can mention that you are not subscribed, and to not forget to send
you a CC copy.


All CC re-added.

> 
> > Actually both CGI and CGI::Fast are in Perl core distribution since
> > perl 5.004 (Perl 5.4.0).  I assume that CGI::Fast simply degrades to CGI
> > if FCGI module is not present.
> 
> No, it just dies if FCGI is not installed

Ah, so that is why CGI distribution has FCGI as requirement!

  http://deps.cpantesters.org/?module=CGI;perl=latest

> (so CGI::Fast in core doesn't make any sense).

Yes, it doesn't.

Nevertheless gitweb would try to require CGI::Fast only when it is run
with '--fastcgi' / '--fcgi' / '-f' option.  It should probably check if
it can be require'd without errors, like it is done for FCGI::ProcManager
in
  [RFC/PATCHv2 2/2] gitweb: Add support for FastCGI, using CGI::Fast
  Message-Id: <201005080959.01800.jnareb@gmail.com>
  http://article.gmane.org/gmane.comp.version-control.git/146647

> 
> > JN> Yes, it can.  CGI::Compile is used for example by CGI::Emulate::PSGI,
> > JN> and you can run PSGI app on standalone Perl web server (pure Perl
> > JN> HTTP::Server::PSGI, or HTTP::Server::Simple::PSGI which in turn uses
> > JN> HTTP::Server::Simple, or Starman, or Twiggy, or Perlbal)
> 
> And FastCGI.
> 
> I don't understand why you implemented FastCGI interface *in addition
> to* PSGI/Plack interface.

I didn't implement support for PSGI/Plack, at least not in patch send in
"[PATCH 0/2] gitweb: Add support for running gitweb as FastCGI script"
series.

Adding support for FastCGI to CGI application is as simple as using
CGI::Fast object in place of CGI, and wrapping running main subroutine
in a loop that processes requests, like described in CGI::Fast manpage.
Well, for gitweb it required a few more small changes, anyway...

Moving to PSGI, or adding possibility to run gitweb as PSGI script (like
the series adds *ability* to run gitweb as FastCGI script) by modifying
gitweb would not be that easy, even with help of CGI::PSGI.


CGI::Compile was referring to an alternate approach, where instead of
modifying gitweb to be able to run it as FastCGI script (you can run it
as CGI script and as ModPerl::Registry script from mod_perl) there was
added gitweb.fcgi wrapper:

JN> The alternate solution would be to add gitweb.fcgi wrapper, like e.g.:
JN> in the following patch by Eric Wong
JN> 
JN>  "[PATCH 1/2] gitweb: add a simple wrapper for FCGI support"
JN>  http://thread.gmane.org/gmane.comp.version-control.git/35920/focus=35921
JN>
JN> which was part of the "[0/2 PATCH] FastCGI and nginx support for gitweb"
JN> series.  (Note that the patch does 'do $gitweb_cgi;' without checking for
JN> errors, see the bottom of `perldoc -f do` documentation on how it should
JN> be done).

One can of course use this approach wrapping gitweb to be run on PSGI,
using CGI::Emulate::PSGI (via Plack::App::WrapCGI), which in turn uses
CGI::Compile.  The gitweb.fcgi wrapper could use CGI::Emulate::FCGI...
if it existed.

NB I use the following gitweb.psgi wrapper to run gitweb from "plackup"
for tests (after running "make gitweb", of course, and with appropriate
gitweb_config.perl, unversioned, in gitweb/ alongside gitweb.perl, the
gitweb.psgi wrapper, and generated gitweb.cgi):

-- 8< --
#!/usr/bin/env plackup

# gitweb - simple web interface to track changes in git repositories
#          PSGI wrapper (see http://plackperl.org)

use strict;
use warnings;

use Plack::Builder;
use Plack::App::WrapCGI;
use CGI::Emulate::PSGI 0.07; # minimum version required to work

use File::Spec;
# __DIR__ is taken from Dir::Self __DIR__ fragment
sub __DIR__ () {
	File::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);
}

builder {
	enable 'Static',
		path => sub { m!\.(js|css|png)$! && s!^/gitweb/!! }, root => __DIR__."/";
	Plack::App::WrapCGI->new(script => __DIR__."/gitweb.cgi")->to_app;
}

__END__
-- >8 --

> 
> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,
> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,
> pure perl alternative.

It is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not
exist, so that gitweb.psgi wrapper would not require indirectly 
CGI::Compile.

> 
> Come over to #plack (irc://irc.perl.org/#plack) for more discussion.

Email not good?

-- 
Jakub Narebski
ShadeHawk on #git, jnareb on #plack
Poland

Re: gitweb, FastCGI and PSGI/Plack

[Tatsuhiko Miyagawa]

On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:

> Moving to PSGI, or adding possibility to run gitweb as PSGI script (like
> the series adds *ability* to run gitweb as FastCGI script) by modifying
> gitweb would not be that easy, even with help of CGI::PSGI.

You don't need it - since you can just use Plack::App::WrapCGI.
>
> CGI::Compile was referring to an alternate approach, where instead of
> modifying gitweb to be able to run it as FastCGI script (you can run it
> as CGI script and as ModPerl::Registry script from mod_perl) there was
> added gitweb.fcgi wrapper:

and the .fcgi wrapper can just use Plack::Loader, or the plackup
executable with FCGI environment variable set, to DWIM.

> One can of course use this approach wrapping gitweb to be run on PSGI,
> using CGI::Emulate::PSGI (via Plack::App::WrapCGI), which in turn uses
> CGI::Compile.  The gitweb.fcgi wrapper could use CGI::Emulate::FCGI...
> if it existed.

Again, you still don't understand - once your CGI script is turned
into PSGI, plackup can take over the web server interface, including
the FastCGI interface. If you need .fcgi wrapper to be spawned from
web servers, just put plackup command line call (or Plack::Runner or
::Loader) in the .fcgi script.

> NB I use the following gitweb.psgi wrapper to run gitweb from "plackup"
> for tests (after running "make gitweb", of course, and with appropriate
> gitweb_config.perl, unversioned, in gitweb/ alongside gitweb.perl, the
> gitweb.psgi wrapper, and generated gitweb.cgi):

>> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,
>> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,
>> pure perl alternative.
>
> It is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not
> exist, so that gitweb.psgi wrapper would not require indirectly
> CGI::Compile.

it's not a pity - it's the simplification by wrapping CGI environment
into PSGI, which is a pure perl web server interface that then can be
turned into ANY web server handlers including standalone, CGI,
FastCGI, mod_perl and SCGI.




-- 
Tatsuhiko Miyagawa

Re: gitweb, FastCGI and PSGI/Plack

[Jakub Narebski]

On Sun, May 9, 2010 at 17:59:11 -0700, Tatsuhiko Miyagawa wrote:
> On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> 
> > Moving to PSGI, or adding possibility to run gitweb as PSGI script (like
> > the series adds *ability* to run gitweb as FastCGI script) by modifying
> > gitweb would not be that easy, even with help of CGI::PSGI.
> 
> You don't need it - since you can just use Plack::App::WrapCGI.

It is simplest method, but I don't think it is best one from the
performance and especially latency point of view.  

As I understand it, CGI::Compile catches all STDOUT output of compiled
CGI script, and passes it to PSGI layer (PSGI interface).  That doesn't
allow, I think, for serving request as it is generated.  This might be
important for actions (pages) such as 'snapshot', 'blob_plain' and
'patches', which simply dump output of external comand to STDOUT.  If
gitweb was written as PSGI app, or with PSGI-compliant web framework, 
it would return IO::Handle-like object.

Also I think that currently you can currently see in web browser page as
it is being generated by gitweb (e.g. 'projects_list' page).  This is
impossible (I guess) with CGI::Compile.


There is also a problem that there is no way, I guess, to automatically
reload / refresh app if underlying CGI script changes (a la "plackup -r").

> > CGI::Compile was referring to an alternate approach, where instead of
> > modifying gitweb to be able to run it as FastCGI script (you can run it
> > as CGI script and as ModPerl::Registry script from mod_perl) there was
> > added gitweb.fcgi wrapper:
> 
> and the .fcgi wrapper can just use Plack::Loader, or the plackup
> executable with FCGI environment variable set, to DWIM.

Right, having gitweb.psgi PSGI wrapper, which can be run not only via
FCGI, but also on many other web servers, is certainly superior to
having gitweb.fcgi FastCGI wrapper, which allows to run gitweb as
FastCGI script (on FCGI server).
 
> > One can of course use this approach wrapping gitweb to be run on PSGI,
> > using CGI::Emulate::PSGI (via Plack::App::WrapCGI), which in turn uses
> > CGI::Compile.  The gitweb.fcgi wrapper could use CGI::Emulate::FCGI...
> > if it existed.
> 
> Again, you still don't understand - once your CGI script is turned
> into PSGI, plackup can take over the web server interface, including
> the FastCGI interface. If you need .fcgi wrapper to be spawned from
> web servers, just put plackup command line call (or Plack::Runner or
> ::Loader) in the .fcgi script.

I didn't made myself clear here.

What I want, like I wrote in neighbour subthread, is for the FastCGI app
that looks like this:

   use CGI::Fast;

   do_some_initialization();

   while ($q = new CGI::Fast) {
      process_request();  # it uses 'my $cgi = CGI->new()' inside
   }

to make PSGI application that runs do_some_initialization() only once,
and returns converted-to-PSGI process_request() as PSGI subroutine (as
PSGI $app).

-- 
Jakub Narebski
jnareb on #plack
Poland

Re: gitweb, FastCGI and PSGI/Plack

[Tatsuhiko Miyagawa]

On Mon, May 10, 2010 at 9:26 AM, Jakub Narebski <jnareb@gmail.com> wrote:

> As I understand it, CGI::Compile catches all STDOUT output of compiled
> CGI script, and passes it to PSGI layer (PSGI interface).  That doesn't
> allow, I think, for serving request as it is generated.  This might be
> important for actions (pages) such as 'snapshot', 'blob_plain' and
> 'patches', which simply dump output of external comand to STDOUT.

Yes, good point.

PSGI allows "streaming response interface", which gives you a writer
object to call ->write method, to implement the non-buffering output
from your application. (Of course, returning IO::Handle is another way
to do so, but that's "pull" style streaming, not "push" which is much
easier when you convert CGI applications).

It would still require you to rewrite some of the gitweb.cgi HTML
generation code, such as;

  # original
  my $cgi = CGI->new;
  print $cgi->header(...);
  print "<html>";
  print "<body>...";

into:

  # CGI::PSGI
  my $app = sub {
      my $env = shift;
      return sub {
          my $r = shift;
          my $cgi = CGI::PSGI->new($env);
          my $writer = $r->([ $cgi->header(...) ]);
          $writer->write("<html>");
          $writer->write("<body>...");
      }
  };

With some work to do the initialization and turn ->write call into a
callback, it should be possible to make one code do both the normal
CGI and PSGI.

> Also I think that currently you can currently see in web browser page as
> it is being generated by gitweb (e.g. 'projects_list' page).  This is
> impossible (I guess) with CGI::Compile.

Correct, and possible with CGI::PSGI.

> There is also a problem that there is no way, I guess, to automatically
> reload / refresh app if underlying CGI script changes (a la "plackup -r").

Yes there is. "plackup -R" can specify the path containing your cgi
script to refresh whenever they're updated.

There's also "plackup -L Shotgun" to fork your application to get a
fresh copy in every request, handy for the development.


-- 
Tatsuhiko Miyagawa

Re: gitweb, FastCGI and PSGI/Plack

[Tatsuhiko Miyagawa]

On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:

>> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,
>> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,
>> pure perl alternative.
>
> It is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not
> exist, so that gitweb.psgi wrapper would not require indirectly
> CGI::Compile.

I *think* we discussed about this on IRC, but you still seems to be
misunderstanding:

You have a CGI script and you want to turn it into a PSGI application,
hence we have CGI::Emulate::PSGI and CGI::Compile.

You usually do not have a FCGI "application". You're writing a .fcgi
"wrapper" to make your CGI script runnable from a web server (like
you're doing with gitweb.fcgi).

Writing an FCGI emulation layer for PSGI would allow you to run the
FCGI wrapper from PSGI compatible web server - which does not make
sense AT ALLl. It's even one more indirection.


--
Tatsuhiko Miyagawa

Re: gitweb, FastCGI and PSGI/Plack

[Jakub Narebski]

On Mon, May 10, 2010, Tatsuhiko Miyagawa wrote:
> On Sun, May 9, 2010 at 4:05 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>>> 
>>> `plackup -s FCGI` makes your PSGI app a fastcgi handler using FCGI.pm,
>>> or `plackup -s Net::FastCGI` does the same but using Net::FastCGI,
>>> pure perl alternative.
>>
>> It is a pity that Plack::App::WrapFCGI / FCGI::Emulate::PSGI does not
>> exist, so that gitweb.psgi wrapper would not require indirectly
>> CGI::Compile.
> 
> I *think* we discussed about this on IRC, but you still seems to be
> misunderstanding:

I'm sorry for the confusion; I didn't make myself clear (and there also
was some misunderstanding of technologies involved on my side).

> 
> You have a CGI script and you want to turn it into a PSGI application,
> hence we have CGI::Emulate::PSGI and CGI::Compile.
> 
> You usually do not have a FCGI "application". You're writing a .fcgi
> "wrapper" to make your CGI script runnable from a web server (like
> you're doing with gitweb.fcgi).

After thinking about it a bit, I realized that I don't want to have
Plack::App::WrapFCGI wrapper (which if there are no FastCGI-only Perl
web apps, e.g. using FCGI directly and which do not have support for
running as ordinary CGI would be totally unnecessary), but I want to
avoid price of using CGI::Compile.

>From what I understand Plack::App::WrapCGI does something like that
(example taken from CGI::Compile manpage):

   use CGI::Emulate::PSGI;
   use CGI::Compile;

   my $cgi_script = "/path/to/foo.cgi";
   my $sub = CGI::Compile->compile($cgi_script);
   my $app = CGI::Emulate::PSGI->handler($sub);

   # $app is a PSGI application


A typical application that uses CGI::Fast to provide support for running
as FastCGI script looks like this, according to CGI::Fast manpage 
(the example is slightly modified):

   use CGI::Fast;

   do_some_initialization();

   while ($q = new CGI::Fast) {
      process_request();  # it uses 'my $cgi = CGI->new()' inside
   }

I'd like to use the fact that per-request part is separated from
initialization part in wrapper for PSGI.

  use CGI::Emulate::PSGI;
  use CGI::Fast::Loader; # or something like that

  my $fcgi_script = "/path/to/foo.fcgi";
  my $fcgi = CGI::Fast::Loader->load($fcgi_script);
  $fcgi->import(qw(do_some_initialization process_request));

  do_some_initialization();
  my $app = CGI::Emulate::PSGI->handler(\&process_request);

   # $app is a PSGI application


That is of course heavy handwaving, and I am not sure if it is something
that can be generalized for scripts that use FCGI module (and 
FCGI::Request) directly.

> 
> Writing an FCGI emulation layer for PSGI would allow you to run the
> FCGI wrapper from PSGI compatible web server - which does not make
> sense AT ALL!. It's even one more indirection.

Well, it would allow to run FastCGI application in *any* PSGI compatibile
web server, including standalone (HTTP::Server::PSGI), mod_perl and
Test::Plack 'server'.

Also even if wrapping FCGI application as PSGI application to run it
on FCGI server doesn't make much sense on first glance, it neverheless
allows to use many, many Plack::Middleware::*.

Mind you, I don't think that there are many FastCGI-only Perl web apps...
if there are any.


I hope that clarify what I want (and no, I guess FCGI::Emulate::PSGI
ain't it).
-- 
Jakub Narebski
Poland

Re: gitweb, FastCGI and PSGI/Plack

[Tatsuhiko Miyagawa]

On Mon, May 10, 2010 at 3:32 AM, Jakub Narebski <jnareb@gmail.com> wrote:
>> You have a CGI script and you want to turn it into a PSGI application,
>> hence we have CGI::Emulate::PSGI and CGI::Compile.
>>
>> You usually do not have a FCGI "application". You're writing a .fcgi
>> "wrapper" to make your CGI script runnable from a web server (like
>> you're doing with gitweb.fcgi).
>
> After thinking about it a bit, I realized that I don't want to have
> Plack::App::WrapFCGI wrapper (which if there are no FastCGI-only Perl
> web apps, e.g. using FCGI directly and which do not have support for
> running as ordinary CGI would be totally unnecessary), but I want to
> avoid price of using CGI::Compile.

Yes, that makes sense - if implementing FastCGI is just switching CGI
to CGI::Fast and a while loop, implementing PSGI interface just for
that is far more complicated. I just argued about it because i saw on
a separate thread that PSGI implementation is also on its way.



-- 
Tatsuhiko Miyagawa

Re: gitweb, FastCGI and PSGI/Plack

[Jakub Narebski]

On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:

> Yes, that makes sense - if implementing FastCGI is just switching CGI
> to CGI::Fast and a while loop, implementing PSGI interface just for
> that is far more complicated. I just argued about it because I saw on
> a separate thread that PSGI implementation is also on its way.

What!?  No, currently there are no plans to add PSGI support to gitweb,
nor moving gitweb from CGI (and mod_perl's ModPerl::Registry, and
FastCGI now) to PSGI.

This separate thread is about adding support for 'plackup' to
git-instaweb (and perhaps also adding gitweb.psgi wrapper).

>From git-instaweb manpage:

  NAME
       git-instaweb - Instantly browse your working repository in gitweb

  SYNOPSIS
       git instaweb [--local] [--httpd=<httpd>] [--port=<port>]
                    [--browser=<browser>]
       git instaweb [--start] [--stop] [--restart]

  DESCRIPTION
       A simple script to set up gitweb and a web server for browsing the local
       repository.

  OPTIONS
       -d, --httpd
              The HTTP daemon command-line that  will  be  executed.  Command-line
              options  may  be  specified here, and the configuration file will be
              added at the end of the command-line. Currently  apache2,  lighttpd,
              mongoose and webrick are supported. (Default: lighttpd)

I wanted to add support for '--httpd=plackup', see the following thread
on git mailing list:

  "Adding support for "plackup" and similar web server tools to git-instaweb"
  Message-ID: <201005020317.42112.jnareb@gmail.com>
  http://thread.gmane.org/gmane.comp.version-control.git/146124

*Not* moving gitweb to PSGI/Plack.

-- 
Jakub Narebski
Poland

Re: gitweb, FastCGI and PSGI/Plack

[Tatsuhiko Miyagawa]

On Mon, May 10, 2010 at 11:40 AM, Jakub Narebski <jnareb@gmail.com> wrote:
> On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:

>> Yes, that makes sense - if implementing FastCGI is just switching CGI
>> to CGI::Fast and a while loop, implementing PSGI interface just for
>> that is far more complicated. I just argued about it because I saw on
>> a separate thread that PSGI implementation is also on its way.
>
> What!?  No, currently there are no plans to add PSGI support to gitweb,
> nor moving gitweb from CGI (and mod_perl's ModPerl::Registry, and
> FastCGI now) to PSGI.
>
> This separate thread is about adding support for 'plackup' to
> git-instaweb (and perhaps also adding gitweb.psgi wrapper).

Yes, that's what i meant.

If you get plackup support by converting your CGI application into a
PSGI app, then adding support to FastCGI is just one command line
option away.


-- 
Tatsuhiko Miyagawa

Re: gitweb, FastCGI and PSGI/Plack

[Jakub Narebski]

On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:
> On Mon, May 10, 2010 at 11:40 AM, Jakub Narebski <jnareb@gmail.com> wrote:
>> On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:
>>> 
>>> Yes, that makes sense - if implementing FastCGI is just switching CGI
>>> to CGI::Fast and a while loop, implementing PSGI interface just for
>>> that is far more complicated. I just argued about it because I saw on
>>> a separate thread that PSGI implementation is also on its way.
>>
>> What!?  No, currently there are no plans to add PSGI support to gitweb,
>> nor moving gitweb from CGI (and mod_perl's ModPerl::Registry, and
>> FastCGI now) to PSGI.
>>
>> This separate thread is about adding support for 'plackup' to
>> git-instaweb (and perhaps also adding gitweb.psgi wrapper).
> 
> Yes, that's what I meant.
> 
> If you get plackup support by converting your CGI application into a
> PSGI app, then adding support to FastCGI is just one command line
> option away.

Well, the support for 'plackup' in git-instaweb is / would be done not
by converting gitweb from CGI to PSGI app, but by using gitweb.psgi
wrapper.  This wrapper uses Plack::App::WrapCGI (which in turn uses
CGI::Emulate::PSGI, which in turn use CGI::Compile) to wrap gitweb.cgi,
and Plack::Middleware::Static to serve static files (gitweb.css,
gitweb.js, git-favicon.png, git-logo.png).

To be more exact in current, as yet unpublished version, git-instaweb
generates gitweb.psgi on-the-fly.  I am considering adding gitweb.psgi
wrapper to git repository (to git sources); when building git-instaweb
it would get embedded in $(gitexecdir)/git-instaweb, just like
gitweb.cgi, gitweb.css (or gitweb.min.css) and gitweb.js (or
gitweb.min.js) are.  Running git-instaweb for first time would create
_configured_ gitweb.cgi in $GIT_DIR/gitweb.  For "plackup" git-instaweb
would put configured gitweb.psgi (or app.psgi) there; for other web
servers git-instaweb puts httpd.conf there.

It is true that with gitweb.psgi wrapper running gitweb as FastCGI
script is just one command line option away.  But it does require Plack
to be installed... well, on the other hand the modified gitweb needs
FCGI module to be ran as FastCGI script, which is not a CORE Perl module
either.


I also wonder how running via wrapper script affect performance, as
compared to modified gitweb running as FastCGI script, using CGI::Fast
and FCGI.


P.S. A bit of history: original patch by Sam Vilain adding FastCGI
support by using CGI::Fast is from 2006.  Eric Wong patch adding
gitweb.fcgi wrapper (with 'no warnings; do $file' instead of modern
CGI::Compile->compile($file)) is from 2007.

I have originally replaced 'exit' in die_error() subroutine by non-local
jump to the end of request processing in ultimately a bit failed attempt
to use die_error() in CGI::Carp::set_message(), because when using
'exit' the error didn't get logged (unfortunately you can set error
message, but you can set HTTP headers for error message this way).

I have then noticed that this change would also allow to add FastCGI
support to gitweb in a very simple way.  Therefore I have ported Sam
Vilain patch to modern gitweb codebase.  Well, it did require some
restructuring of gitweb code (some refactoring), so it wasn't that
simple... nevertheless I think that this refactoring is mainly a good
change anyway.


P.P.S. One of constraints to gitweb development is that it should run
with minimal set of non-core modules.  Some people even complain that
gitweb (or was it about git in general?) requires at least Perl 5.8.6 or
about (because of Encode module and Unicode support).

If one wants Modern Perl git web interface, there is always Gitalist...

-- 
Jakub Narebski, Poland
ShadeHawk on #git at FreeNode
jnareb on #plack at irc.perl.org

Re: gitweb, FastCGI and PSGI/Plack

[Tatsuhiko Miyagawa]

On Mon, May 10, 2010 at 2:10 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>
> Well, the support for 'plackup' in git-instaweb is / would be done not
> by converting gitweb from CGI to PSGI app, but by using gitweb.psgi
> wrapper.

wrapping gitweb.cgi with gitweb.psgi is a legitimate way to say
"converting CGI to PSGI app".

> It is true that with gitweb.psgi wrapper running gitweb as FastCGI
> script is just one command line option away.  But it does require Plack
> to be installed... well, on the other hand the modified gitweb needs
> FCGI module to be ran as FastCGI script, which is not a CORE Perl module
> either.

*nods*

> I also wonder how running via wrapper script affect performance, as
> compared to modified gitweb running as FastCGI script, using CGI::Fast
> and FCGI.

Based on my experience CGI::Emulate::PSGI doesn't have much of an
overhead because it's just swapping STDIN and STDOUT handles and
doesn't require stuff like tie or overload which tends to be slower.

> P.P.S. One of constraints to gitweb development is that it should run
> with minimal set of non-core modules.  Some people even complain that
> gitweb (or was it about git in general?) requires at least Perl 5.8.6 or
> about (because of Encode module and Unicode support).

I know, but git-instaweb is a different story, since you rely on the
fact that the system has one of web servers like apache, lighttpd or
mongrel.

And that's exactly why I've been suggesting to you use WrapCGI
(CGI::Emulate::PSGI  + CGI::Compile) instead of converting gitweb.cgi
to natively support PSGI. We're on the same page and i don't
understand why you keep disagreeing with me :)


-- 
Tatsuhiko Miyagawa

Re: gitweb, FastCGI and PSGI/Plack

[Jakub Narebski]

On Mon, 10 May 2010, Tatsuhiko Miyagawa wrote:
> On Mon, May 10, 2010 at 2:10 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> >
> > Well, the support for 'plackup' in git-instaweb is / would be done not
> > by converting gitweb from CGI to PSGI app, but by using gitweb.psgi
> > wrapper.
> 
> Wrapping gitweb.cgi with gitweb.psgi is a legitimate way to say
> "converting CGI to PSGI app".

I have misunderstood you then, I'm sorry.
 
> > I also wonder how running via wrapper script affect performance, as
> > compared to modified gitweb running as FastCGI script, using CGI::Fast
> > and FCGI.
> 
> Based on my experience CGI::Emulate::PSGI doesn't have much of an
> overhead because it's just swapping STDIN and STDOUT handles and
> doesn't require stuff like tie or overload which tends to be slower.

Nice to know.

Unfortunately I don't have mod_fcgid / mod_fastcgi installed, so I can't
do a benchmark comparing PSGI wrapper + FCGI handler with modified
gitweb running as FastCGI script (using ab, ApacheBench).  Do you know
any pure-Perl FastCGI server with minimal dependencies, and pure-Perl
HTTP server benchmarking tool (like ab and httperf)?
 
> > P.P.S. One of constraints to gitweb development is that it should run
> > with minimal set of non-core modules.  Some people even complain that
> > gitweb (or was it about git in general?) requires at least Perl 5.8.6 or
> > about (because of Encode module and Unicode support).
> 
> I know, but git-instaweb is a different story, since you rely on the
> fact that the system has one of web servers like apache, lighttpd or
> mongrel.
> 
> And that's exactly why I've been suggesting to you use WrapCGI
> (CGI::Emulate::PSGI + CGI::Compile) instead of converting gitweb.cgi
> to natively support PSGI. We're on the same page and I don't
> understand why you keep disagreeing with me :)

I might have misunderstand you as arguing against modifying gitweb to
add FastCGI support via CGI::Fast...

P.S. discussion != disagreeing ;-)
-- 
Jakub Narębski
Poland

Re: gitweb, FastCGI and PSGI/Plack

[Peter Vereshagin]

I know St. Peter won't call your name, Jakub!
2010/05/11 11:29:07 +0200 Jakub Narebski <jnareb@gmail.com> => To Tatsuhiko Miyagawa :

JN> gitweb running as FastCGI script (using ab, ApacheBench).  Do you know
JN> any pure-Perl FastCGI server with minimal dependencies, and pure-Perl

I'd like to know about the pure-perl FastCGI protocol implementation either, at
least to make tests for FCGI::Spawn.

73! Peter pgp: A0E26627 (4A42 6841 2871 5EA7 52AB  12F8 0CE1 4AAC A0E2 6627)
-- 
http://vereshagin.org

Re: gitweb, FastCGI and PSGI/Plack

[Tatsuhiko Miyagawa]

2010/5/11 Peter Vereshagin <peter@vereshagin.org>:
> I know St. Peter won't call your name, Jakub!
> 2010/05/11 11:29:07 +0200 Jakub Narebski <jnareb@gmail.com> => To Tatsuhiko Miyagawa :
>
> JN> gitweb running as FastCGI script (using ab, ApacheBench).  Do you know
> JN> any pure-Perl FastCGI server with minimal dependencies, and pure-Perl
>
> I'd like to know about the pure-perl FastCGI protocol implementation either, at
> least to make tests for FCGI::Spawn.

There are two pure perl FCGI protocol implementations (caller side)
available: Net::FastCGI and FCGI::Client.

Plack::App::FCGIDispatcher is a PSGI application to connect to a
FastCGI external daemon over TCP or UNIX sockets, so you can use pure
perl PSGI web servers such as Starman to talk to them. But since it's
all pure perl and experimental and not suited for the benchmarking,
but it could be handy for testing (we actually use FCGI::Client to
test our own FastCGI server side implementation in Plack).

When I seriously want to benchmark FastCGI i always use nginx or
lighttpd, which is easy to install and doesn't need external modules
such as the case with Apache.

-- 
Tatsuhiko Miyagawa