Re: [Qemu-devel] [PATCH alt 4/7] block/qcow2: Implement status CB for amend

[Max Reitz <mreitz@redhat.com>]

On 01.08.2014 22:38, Eric Blake wrote:
> On 08/01/2014 02:18 PM, Max Reitz wrote:
>
>>>> +        if (status_cb) {
>>>> +            status_cb(bs, *visited_l1_entries << (s->l2_bits +
>>>> s->cluster_bits),
>>>> +                      l1_entries << (s->l2_bits + s->cluster_bits));
>>> Shifting is a multiplication so it keep proportionality intact.
>>> So do we really need these shifts ?
>> As of patch 1, the variables are defined as "offset" and "working_size"
>> which I meant to be byte ranges. We could indeed leave the unit for
>> BlockDriverAmendStatusCB's parameters undefined (and leave it up to the
>> block driver, only specifying that offset / working_size has to be the
>> progress ratio), but then we could just as well just give a floating
>> point percentage to that function.
> As it is, block jobs are already documented as merely exposing
> completion percentage, and NOT tied to the size of the underlying block
> device.  Your own pending patch is proof of this:
>
> https://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00960.html
>
> When doing drive-mirror, we WANT to have the total size grow according
> to how many dirty blocks are encountered through each pass, and the
> current offset grow in rough proportion to how fast we are converging to
> a mirrored state (or even having the percentage go backwards, when we
> are diverging from too much I/O pressure, as a sign that some throttling
> is needed).  Artificially trying to constrain that progress reporting to
> the size in bytes of the block device does not help matters.

Well, this would just as well work with a single floating point percentage.

>> Bytes as a unit seemed safe to me, however, since all of qemu's code
>> assumes byte ranges to always fit in int64_t; and the reason I preferred
>> them over a percentage is that block jobs use bytes as well.
>>
>> A real reason not to use bytes would be that some driver is unable to
>> give a "byte" representation of its workload during the amend progress;
>> however, this seems unlikely to me (every large workload which should be
>> part of the progress report should be somehow representable as bytes).
> I don't think we can promise anything more than two relative numbers,
> with no bearing on what they are measuring under the hood, so scaling
> both numbers just to produce progress output buys nothing.  There may
> eventually be a device where we can't report progress in any more
> granularity than 0 (still working) or 1 (done).

In that case it should never call the callback.

Note how qcow2 can do a whole number of things including resizing the 
image during an amend operation; but only the zero cluster expansion is 
costly enough to justify a progress output. Consequently, it's only that 
function which invokes the callback. If the image is not being 
downgraded from compat=1.1 to compat=0.10, the callback is never 
invoked; there even is a small comment about this in qemu-img.c (/* In 
case the driver does not call amend_status_cb() */).

I can however see that a driver invokes a couple of time-consuming 
operations and there is no progress report for each of those; so uses 
the number of operations as the workload size and the parameter 
currently named "offset" as the index of the operation executed next. 
Frankly, I'd blame the driver then, since the operations should give a 
real progress report (which I still think *has* to be convertable to 
bytes somehow).

Anyway, you're probably both right and we should just drop the unit 
enforcement and allow the driver to use any unit it desires; but in this 
case I still think we could just give a floating point percentage 
instead of a numerator and a denominator.

Max