Re: git-clone --how-much-disk-space-will-this-cost-me? [--depth n]

["Shawn O. Pearce" <spearce@spearce.org>]

Nicolas Pitre <nico@cam.org> wrote:
> On Tue, 16 Dec 2008, Jean-Luc Herren wrote:
> > jidanni@jidanni.org wrote:
> > > JH> So maybe what you really want is an ETA display during the cloning
> > > JH> process?  Sounds like a good idea to me.
> 
> And then you'll end up being the unlucky bastard to be the first to 
> clones the new latest revision of a repository, and ETA won't be 
> available, and you'll complain about the fact that sometimes it is there 
> and sometimes it is not.
> 
> The fact is, fundamentally, we don't know how many bytes to push when 
> generating a pack to answer the clone request.  Sometimes we _could_ but 
> not always.  It is therefore better to be consistent and let people know 
> that there is simply no ETA.

Hmm.

What if on an initial clone (no "have" lines received) we sum up
the sizes of the *.pack and all of the loose objects and sent
that as an initial size estimate.  Its going to be the upper bound
of the final pack that we send.  At worst it over-estimates on the
size and download finishes faster.

I'm willing to bet that most of the "big" repositories out there
don't have a lot of garbage in them.  Linus' kernel repository
doesn't rewind, so he has 0 garbage.  Anyone cloning from him would
get a reasonable estimate.  Likewise with a Gentoo/KDE/WebKit/gcc
sort of giant tree most of that is in a huge historical pack.
That one pack file alone is completely reachable and dominates the
transfer size.

On smaller trees where people may have a lot of rebase garbage or
everything is loose the estimate will be quite a bit above what we
transfer, but how much so that it matters?

Yea, a single stray binary of some *.mpg or *.iso accidentally
added and then removed (and now unreachable) will vastly inflate
the numbers.  In which case the repository owner will be encouraged
to prune when people won't clone his estimated 8 GiB download,
which is actually only 1 MiB.

-- 
Shawn.