Re: No progress from push when using bitmaps

[Jeff King <peff@peff.net>]

On Thu, Mar 13, 2014 at 03:01:09PM -0700, Shawn Pearce wrote:

> > It would definitely be good to have throughput measurements while
> > writing out the pack. However, I'm not sure we have anything useful to
> > count. We know the total number of objects we're reusing, but we're not
> > actually parsing the data; we're just blitting it out as a stream. I
> > think the progress code will need some refactoring to handle a
> > throughput-only case.
> 
> Yes. I think JGit suffers from this same bug, and again we never
> noticed it because usually only the servers are bitmapped, not the
> clients.
> 
> pack-objects writes a throughput meter when its writing objects.
> Really just the bytes out/second would be enough to let the user know
> the client is working. Unfortunately I think that is still tied to the
> overall progress system having some other counter?

Yes, I'm looking at it right now. The throughput meter is actually
connected to the sha1fd output. So really we just need to call
display_progress periodically as we loop through the data. It's a
one-liner fix.

_But_ it still looks ugly, because, as you mention, it's tied to the
progress meter, which is counting up to N objects. So we basically sit
there at "0", pumping data, and then after the pack is done, we can say
we sent N. :)

There are a few ways around this:

  1. Add a new phase "Writing packs" which counts from 0 to 1. Even
     though it's more accurate, moving from 0 to 1 really isn't that
     useful (the throughput is, but the 0/1 just looks like noise).

  2. Add a new phase "Writing reused objects" that counts from 0 bytes
     up to N bytes. This looks stupid, though, because we are repeating
     the current byte count both here and in the throughput.

  3. Use the regular "Writing objects" progress, but fake the object
     count. We know we are writing M bytes with N objects. Bump the
     counter by 1 for every M/N bytes we write.

The first two require some non-trivial surgery to the progress code. I
am leaning towards the third. Not just because it's easy, but because I
think it actually shows the most intuitive display. Yes, it's fudging
the object numbers, but those are largely meaningless anyway (in fact,
it makes them _better_ because now they're even, instead of getting 95%
done and then hitting some blob that is as big as the rest of the repo
combined).

-Peff