Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

[Jakub Narebski <jnareb@gmail.com>]

Linus Torvalds wrote:
> On Sun, 10 Dec 2006, Jakub Narebski wrote:
>>>> If-Modified-Since:, If-Match:, If-None-Match: do you?
>> 
>> And in CGI standard there is a way to access additional HTTP headers
>> info from CGI script: the envirionmental variables are HTTP_HEADER,
>> for example if browser sent If-Modified-Since: header it's value
>> can be found in HTTP_IF_MODIFIED_SINCE environmental variable.
> 
> Guys, you're missing something fairly fundamnetal. 
> 
> It helps almost _nothing_ to support client-side caching with all these 
> fancy "If-Modified-Since:" etc crap.
> 
> That's not the _problem_.
> 
> It's usually not one client asking for the gitweb pages: the load comes 
> from just lots of people independently asking for it. So client-side 
> caching may help a tiny tiny bit, but it's not actually fixing the 
> fundamental problem at all.

Well, the idea (perhaps stupid idea: I don't know how caching engines
/ reverse proxy works) was that there would be caching engine / reverse
proxy in the front (Squid for example) would cache results and serve it
to rampaging hordes. But this caching engine has to ask gitweb if the
cache is valid using "If-Modified-Since:" and "If-None-Match:" headers.
If gitweb returns 304 Not Modified then it serves contents from cache.

> So forget about "If-Modified-Since:" etc. It may help in benchmarks when 
> you try it yourself, and use "refresh" on the client side. But the basic 
> problem is all about lots of clients that do NOT have things cached, 
> because all teh client caches are all filled up with pr0n, not with gitweb 
> data from yesterday.

What about the other idea, the one with raising expires to infinity for
immutable pages like "commit" view for commit given by SHA-1? Even if
the clients won't cache it, the proxies and caches between gitweb and
client might cache it...

Talking about most accessed gitweb pages, the project list page changes
on every push, the project summary page and project main RSS feed
(now in both RSS and Atom formats) changes on every push to given project.
With a help of hooks they can be static pages, generated by push...
...with the exception that projects list and summary pages have _relative_
dates.

-- 
Jakub Narebski