Re: [Patch] Prevent cloning over http from spewing

[Tay Ray Chuan <rctay89@gmail.com>]

Hi,

On Thu, Jun 4, 2009 at 3:15 AM, Shawn O. Pearce <spearce@spearce.org> wrote:
> No, you are right Peff, you can't give a "percent complete" because
> you don't know how much you need to fetch.
>
> What we could do is try to organize the fetch queue by object type,
> get all commits, then all trees, then blobs.  The blobs are the
> bulk of the data, and by the time we hit them, we should be able
> to give some estimate on progress because we have all of the ones
> we need to fetch in our fetch queue.  But its only a "object count"
> sort of thing, not a byte count.
>
> When fetching a .idx or a .pack though, we should be able to show
> progress... assuming the server sent us a Content-Length header.
> If not, in the case of a pack, we could still show receive speed
> like index-pack does.

I have a branch 'http-progress-indicators' at my repo
git://github.com/rctay/git.git. It contains a patch on walker.c that
updates the object total as the fetch goes along. The progress
indicator says "Processing targets:..." for lack of a better name; I'm
all for suggestions.

The branch also patches that display progress meters for the
downloading of .idx and .pack files. I also added a progress indicator
for verifying pack files in pack-check.c, because I noticed some
significant time was spent doing that without informing the user about
what was going on, but I'm not really sure if everyone else would
accept it.

So far, I've used git built using that branch to fetch the git repo,
and I also attempted the linux 2.6 kernel repo. Counting the objects
fetched was accurate in both cases (ie. matching counts of objects
fetched and objects to-be-fetched).

I plan to clean up the patches and send it in when the http
refactoring patches are finalized.

> Jeff King <peff@peff.net> wrote:
>>   1. summarize what we have fetched (N packs, N loose objects)
>>   2. show what we are currently fetching (object or pack)
>>   3. show the number of bytes retrieved for the current item
>>   4. if the server provides content-length, show the percentage
>>      completed for this object
>>   5. show the current throughput

Points 1. and 5. can probably be combined, because showing info by
type (packs and objects) isn't very easy to do. http-walker.c first
tries to fetch the raw pack; if it can't be found at the url, or at
alternate locations, it then tries to fetching packs. In other words,
it's hard to know in advance if the object is found in unpacked or
packed form.

-- 
Cheers,
Ray Chuan