Re: Smart fetch via HTTP?

["Shawn O. Pearce" <spearce@spearce.org>]

"Shawn O. Pearce" <spearce@spearce.org> wrote:
> david@lang.hm wrote:
> > On Wed, 16 May 2007, Shawn O. Pearce wrote:
> > >
> > >The other thing is kernel.org should really try to encourage the
> > >folks with repositories there to try and share against one master
> > >repository, so the poor OS has a better chance at holding the bulk
> > >of linux-2.6.git in buffer cache.
> > 
> > do you mean more precisely share against one object store or do you really 
> > mean repository?
> 
> Sorry, I did mean "object store".  ;-)

And even there, I don't mean symlink objects to a shared database,
I mean use the objects/info/alternates file to point to the shared,
read-only location.

Its not perfect.  The hotter parts of the object database is almost
always the recent stuff, as that's what people are actively trying
to fetch, or are using as a base when they are trying to fetch from
someone else.  The hotter parts are also probably too new to be
in the shared store offered by kernel.org admins, which means you
cannot get good IO buffering.  Back to the current set of problems.

A single shared object directory that everyone can write new files
into, but cannot modify or delete from, would help that problem quite
a bit.  But it opens up huge problems about pruning, as there is no
way to perform garbage collection on that database without scanning
every ref on the system, and that's just not simply possible on a
busy system like kernel.org.

-- 
Shawn.