Configurable callbacks for missing objects (we Re: upload-pack: support subtree packing)

[Avery Pennarun <apenwarr@gmail.com>]

On Tue, Jul 27, 2010 at 07:46:05AM -0700, Shawn O. Pearce wrote:

> But I disagree with the client rewriting the commits in order to
> work with them locally.  Doing so means you can't take a commit
> from your team's issue tracker and look it up.  And any commit
> you create can't be pushed back to the server without rewriting.
> Its messy for the end-user to work with.

Yeah, that doesn't sound ideal.  And I wrote git-subtree, which does exactly
that, so I should know :)

> I would prefer doing something more like what we do with shallow
> on the client side.  Record in a magic file the path(s) that we
> did actually obtain.  During fsck, rev-list, or read-tree the
> client skips over any paths that don't match that file's listing.
> Then we can keep the same commit SHA-1s, but we won't complain that
> there are objects missing.

Disclaimer: I've never looked at any of the fetch code.

But I've been thinking that a really elegant way to solve the problem could
be to have a user-configurable "get the missing objects" callback.  If any
part of git that *needs* an object can't find it, it calls this callback to
go try to retrieve it (either just that one object, or it can request to
download the object recursively, ie. everything it points to).

Then shallow clones could just auto-fill themselves if you really need a
prior version, for example.

It's also conceivable that we could limit this just to blobs: downloading
the complete set of commit objects, and probably even the complete set of
tree objects, is probably not that expensive.  And that would allow someone
to do virtually all operations (other than three-way merges to resolve blob
conflicts) without having the entire repo.

I say it could be user-configurable because that's where you could plugin a
gittorrent, or a tool that just tries fetching from a series of repositories in
turn, etc.

Have fun,

Avery