Lazy clone ideas

Lazy clone ideas

[Jakub Narebski]

I've started new thread for lazy clone ideas,
splitting from "Figured out how to get Mozilla into git"

Rogan Dawes wrote:
> Here's an idea. How about separating trees and commits from the actual 
> blobs (e.g. in separate packs)? My reasoning is that the commits and 
> trees should only be a small portion of the overall repository size, and 
> should not be that expensive to transfer. (Of course, this is only a 
> guess, and needs some numbers to back it up.)
> 
> So, a shallow clone would receive all of the tree objects, and all of 
> the commit objects, and could then request a pack containing the blobs 
> represented by the current HEAD.

That would be _lazy_ clone (with on-demand pack downloading from "master"
full history repository), rather than shallow clone.

I had an idea for having all the commit objects (without all the tree
objects) below the soft-grafts line (beyond the line we cut-off full
history and start being lazy).
 
> In this way, the user has a history that will show all of the commit 
> messages, and would be able to see _which_ files have changed over time 
> e.g. gitk would still work - except for the actual file level diff, "git 
> log" should also still work, etc
> 
> This would also enable other optimisations.
> 
> For example, documentation people would only need to get the objects 
> under the doc/ tree, and would not need to actually check out the 
> source. Git could detect any actual changes by checking whether it has 
> the previous blob in its local repository, and whether the file exists 
> locally. Creating a patch would obviously require that the person checks 
> out the previous version, but one could theoretically commit a new blob 
> to a repo without having the previous one (not saying that this would be 
> a good idea, of course)

Something akin to CVS's modules, or rather to how CVS modules can be abused?
Something called, I think, partial checkout?

This is a separate idea and I think worth implementing even for full
repository.

> This would probably require Eric Biederman's "direct access to blob" 
> patches, I guess, in order to be feasible.

And it would need place to store URI from where to doenload objects
on-demand: perhaps 'remote alternatives'?

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Lazy clone ideas

[Elrond]

Jakub Narebski <jnareb <at> gmail.com> writes:
> 
> I've started new thread for lazy clone ideas,
> splitting from "Figured out how to get Mozilla into git"
[...]

I like the lazy clone idea, I think, I said that earlier.


> > This would probably require Eric Biederman's "direct access to blob" 
> > patches, I guess, in order to be feasible.

Are those patches allowing the git: protocol to request a list of objects
directly? (Like my "remote git-cat-file" request?)

What's the status of the patch?


> And it would need place to store URI from where to doenload objects
> on-demand: perhaps 'remote alternatives'?

Yep, that would be the next step.
Having direct access to blobs would be needed first though.


    Elrond