Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

[Jeff Garzik <jeff@garzik.org>]

Jakub Narebski wrote:
> Sending Last-Modified: should be easy; sending ETag needs some consensus
> on the contents: mainly about validation. Responding to If-Modified-Since:
> and If-None-Match: should cut at least _some_ of the page generating time.

Definitely.


> As I said, I'm not talking (at least now) about saving generated HTML
> output. This I think is better solved in caching engine like Squid can
> be. Although even here some git specific can be of help: we can invalidate
> cache on push, and we know that some results doesn't ever change (well,
> with exception of changing output of gitweb).

It depends on how creatively you think ;-)

Consider generating static HTML files on each push, via a hook, for many 
of the toplevel files.  The static HTML would then link to the CGI for 
further dynamic querying of the git database.



> What can be _easily_ done:
>  * Use post 1.4.4 gitweb, which uses git-for-each-ref to generate summary
>    page; this leads to around 3 times faster summary page.

This re-opens the question mentioned earlier, is Kay (or anyone?) still 
actively maintaining gitweb on k.org?


>  * Perhaps using projects list file (which can be now generated by gitweb)
>    instead of scanning directories and stat()-ing for owner would help
>    with time to generate projects lis page

This could be statically generated by a robot.  I think everybody would 
shrink in horror if a human needed to maintain such a file.


> What can be quite easy incorporated into gitweb:
>  * For immutable pages set Expires: or Cache-Control: max-age (or both)
>    to infinity

nice!


>  * Generate Last-Modified: for those views where it can be calculated,
>    and respond with 304 Not Modified as soon as it can.

agreed


> What can be easily done using caching engine:
>  * Select top 10 of common queries, and cache them, invalidating cache on push
>    (depending on query: for example invalidate project list on push to any
>    project, invalidate RSS/Atom feed and summary pages only on push to specific
>    project) - can be done with git hooks.

Or simply generate regular filesystem files into the webspace, as 
triggered by a hook.  Let the standard filesystem mirroring/caching work 
its magic.

	Jeff